MySQL server running out of memory, could be a bug?
Posted on 2010-11-12
I need to resolve this problem ASAP. Recently I started receiving these error messages (as recorded in /var/log/mysqld.log):
101112 17:48:04 /usr/sbin/mysqld: Out of memory (Needed 41 bytes)
101112 17:58:04 /usr/sbin/mysqld: Out of memory (Needed 8160 bytes)
The problem is, I am quite sure there is enough memory.
max_connections = 100
#log-bin = /var/log/mysql/mysql-bin.log
#binlog-do-db = icloaker_maintenance
log-slow-queries = /var/log/mysql-slow.log
long_query_time = 1
key_buffer_size = 128M
table_cache = 2048
sort_buffer_size = 1M
read_buffer_size = 1M
max_allowed_packet = 16M
tmp_table_size = 300M
myisam_sort_buffer_size = 64M
join_buffer_size = 2M
thread_cache_size = 16
interactive_timeout = 25
wait_timeout = 1800
connect_timeout = 10
max_connect_errors = 10
query_cache_limit = 2M
query_cache_size = 32M
query_cache_type = 1
innodb_buffer_pool_size = 128M
#innodb_log_file_size = 64M
innodb_additional_mem_pool_size = 20M
innodb_flush_log_at_trx_commit = 2
Using an application called mysqltuner, I found that mysql is allowed to use roughly 823MB now, while the standard useage is just 460M and there do not seem to be spikes. There are very few concurrent connections according to it too.
The server has enough memory, it is the only VPS on a dedicated server with 2GB RAM.
Memory stats on the hardware node, as reported by "top -M":
Mem: 1975.121M total, 1794.250M used, 180.871M free, 73.145M buffers
Swap: 4094.617M total, 152.000k used, 4094.469M free, 1117.539M cached
Memory stats on the VPS, as reported by "top -M":
Mem: 1975.121M total, 809.543M used, 1165.578M free, 0.000k buffers
Swap: 0.000k total, 0.000k used, 0.000k free, 0.000k cached
As you can see, the VPS is allowed to use 2GB RAM as it is the only running VPS on the dedicated server, it can get it easily without fighting for resources, although it could take a small part from the HW node swap if necessary too.
This server has been running OK for a long time although recently I optimized my.cnf to resolve a similar problem but yesterday it got back.
Both HW node and VPS server are running on CentOS 5 64-bit.
Any ideas are more than welcome!