Link to home
Start Free TrialLog in
Avatar of andieje
andieje

asked on

linux system crash involving perl mysql

Hello

I was running a perl program on a linux virtual machine (virtual box) and it has caused the whole VM to crash. I was able to close the machine but I would like to try and find out what just happened. A perl program that writes to mysql was running.

I know nothing about linux admin.  the /var/log/mysql/error.log is below.

I was running this perl program because a particular set of input data made another person's machine crash while running the same perl code. The input data does not appear any different than other data which has been through the program many times before. I do not have a copy of the other persons error log.

Mysql is running fine and i can look at the data that was already written to mysql before the crash.

thanks
110311 20:58:09 [Note] Plugin 'FEDERATED' is disabled.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
110311 20:58:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
110311 20:58:23  InnoDB: Started; log sequence number 36 2238638524
110311 20:58:24 [Note] Event Scheduler: Loaded 0 events
110311 20:58:24 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.49-1ubuntu8.1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Ubuntu)
110311 20:59:24 [ERROR] /usr/sbin/mysqld: Table './annotationDB/harry' is marked as crashed and should be repaired
110311 20:59:24 [Warning] Checking table:   './annotationDB/harry'
110311 20:59:24 [Warning] Recovering table: './annotationDB/harry'
error.log (END)

Open in new window

Avatar of tearman
tearman
Flag of United States of America image

Good place to start is to run strace while you're executing the perl program to see what the kernel is doing at that point in time.  Wish we had dtrace on Linux, but not available.

Otherwise, check /var/logs, especially the dmesg log for anything interesting.
Avatar of droyden
droyden

What was the perl script attepting to do?
Avatar of andieje

ASKER

i dont know the point at which it crashed and now the perl code runs through perfectly but i would like to find out why 2 machines crashed

what might be interesting in dmesg as it all looks the same to me?

the perl program was probabily trying to update a record into the db when it crashed but i'm not 100%. the perl program had created some tables, added some records before it crashed and was due to update the records with some data from an external program. the external program had successfully completed before the perl crashed
ASKER CERTIFIED SOLUTION
Avatar of paulqna
paulqna
Flag of Netherlands image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Check the logs.

If the script was doing inserts and a disk filled up - the system shouldn't crash - the query would just be stopped until the problem was fixed. However I guess a full disk might cause a problem.

Apart from that mysql really does network I/O and disk I/O - with file locking.

I can see 2 causes of a crash:
Some sort of kernel bug - to do with file handling or locking.
or
A hardware problem - bad memory or disk that could be a coincidence or was triggered by the query.

Software/kernel errors should be logged - hardware errors - maybe not.