?
Solved

Mysql Slave Replication not running on some slaves. - urgent!

Posted on 2008-02-02
11
Medium Priority
?
851 Views
Last Modified: 2010-04-21
Mysql replication with one master and several slaves but replication has suddently stopped working on a few slaves.
Version 4.0.22-standard
The table status on the slaves not working is show this error:
Error 'Can't find file: './%db%/%table_name%.frm' (errno: 24)  

I added %db% and %table% name as placeholders because db and table names not relevant.  This db and table does exist on master and is replicating to other machines.

Slave status also shows:
     Master_Log_File: db-master-bin.420
  Read_Master_Log_Pos: 878231783
       Relay_Log_File: slave2-relay-bin.1118
        Relay_Log_Pos: 21051484
Relay_Master_Log_File: db-master-bin.420
     Slave_IO_Running: Yes
    Slave_SQL_Running: No

How can I restart Slaves correctly?  Why is slave Error saying  it can't find this .fry file???

Please help urgently - will award maximum points.

Many thanks!


0
Comment
Question by:duendon
  • 6
  • 5
11 Comments
 
LVL 10

Expert Comment

by:adrpo
ID: 20805185
Hi,

I think you don't have enough file handles available:
http://www.browardphp.com/mysql_manual_en/manual_Problems.html#Not_enough_file_handles

Error code  24:  Too many open files

That is error 24 telling you.
adrpo@...:~> ulimit -n
1024

Try increasing this number:
Login as root and: type:
root@...:~> ulimit -n 2048
See what is happening then.
You might want to set ulimit -n 2048 in your init scripts on your machine.

Cheers,
za-k/
0
 
LVL 10

Expert Comment

by:adrpo
ID: 20805199

More here on how you can prevent this:
http://dev.mysql.com/doc/refman/5.0/en/not-enough-file-handles.html

Cheers,
za-k/
0
 

Author Comment

by:duendon
ID: 20805248
Thanks for your quick reply.

I will look into that, thanks - but do you know why that table is now not on the slave? why would it suddenly dissappea causing that table not found error?

0
Receive 1:1 tech help

Solve your biggest tech problems alongside global tech experts with 1:1 help.

 

Author Comment

by:duendon
ID: 20805263
Ulimit - I have checked that and it returns unlimited - and there is space on the disk.

Also it is affecting several slave machines.
0
 
LVL 10

Expert Comment

by:adrpo
ID: 20805344

Hi,

Even if ulimit -n say unlimited there is a limit in MySQL configuration.
How big is your open_files_limit system variable?
Have a look into your my.cnf and see to what number is:
open_files_limit set to.
You can also check this from mysql client, see below....

More here:
http://dev.mysql.com/doc/refman/4.1/en/option-files.html
http://dev.mysql.com/doc/refman/4.1/en/server-options.html#option_mysqld_open-files-limit

Cheers,
za-k/
mysql> show variables like 'open_files_limit';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| open_files_limit | 622   |
+------------------+-------+
1 row in set (0.00 sec)

Open in new window

0
 
LVL 10

Expert Comment

by:adrpo
ID: 20805357

There is a bug in MySQL version less than: 5.1.6
Instead of saying "Too many open files", it says:
Can't find file: './cluster_replication/binlog_index.frm' (errno: 24)
http://bugs.mysql.com/bug.php?id=15020

Which seems exactly like your error.

Cheers,
za-k/
0
 

Author Comment

by:duendon
ID: 20805398
It turned out that all I needed do do was START SLAVE and all was ok - thought I needed to solve a big enormous problem before starting sdlave again on each machine but not.

Thanks for your help.
0
 
LVL 10

Expert Comment

by:adrpo
ID: 20805409

For nothing! I'm glad you fixed it!

Cheers,
za-k/
0
 

Author Comment

by:duendon
ID: 20805414
So , was the original cause of my error the open files limit?

It affected 6 machines all at the same time.

I want to award you all the points for helping me but I don't want to accept this as a solution if is not solution - eg for other users with my problem.

Please confirm and advise.
Many thanks.
0
 
LVL 10

Accepted Solution

by:
adrpo earned 2000 total points
ID: 20805518

Well, I don't know. You don't have to give me the points.
You can request to close this question and get your points refunded.

The error 24 is "Too many open files.", but if you didn't
play since you got the error with your ulimit -n or
with open_files_limit in MySQL settings, then what
I tell you didn't help.

If you only needed to start the slaves again and it worked,
then I don't know what your problem was.
 
Cheers,
za-k/
0
 

Author Closing Comment

by:duendon
ID: 31427490
Thanks for your help working me through the issue.  Your comments will be useful to future users with similar problems I'm sure.
0

Featured Post

The 14th Annual Expert Award Winners

The results are in! Meet the top members of our 2017 Expert Awards. Congratulations to all who qualified!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

By, Vadim Tkachenko. In this article we’ll look at ClickHouse on its one year anniversary.
In this article, I’ll talk about multi-threaded slave statistics printed in MySQL error log file.
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses

594 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question