Solved

linux system crash involving perl mysql

Posted on 2011-03-11
5
446 Views
Last Modified: 2013-12-16
Hello

I was running a perl program on a linux virtual machine (virtual box) and it has caused the whole VM to crash. I was able to close the machine but I would like to try and find out what just happened. A perl program that writes to mysql was running.

I know nothing about linux admin.  the /var/log/mysql/error.log is below.

I was running this perl program because a particular set of input data made another person's machine crash while running the same perl code. The input data does not appear any different than other data which has been through the program many times before. I do not have a copy of the other persons error log.

Mysql is running fine and i can look at the data that was already written to mysql before the crash.

thanks
110311 20:58:09 [Note] Plugin 'FEDERATED' is disabled.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
110311 20:58:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
110311 20:58:23  InnoDB: Started; log sequence number 36 2238638524
110311 20:58:24 [Note] Event Scheduler: Loaded 0 events
110311 20:58:24 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.49-1ubuntu8.1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Ubuntu)
110311 20:59:24 [ERROR] /usr/sbin/mysqld: Table './annotationDB/harry' is marked as crashed and should be repaired
110311 20:59:24 [Warning] Checking table:   './annotationDB/harry'
110311 20:59:24 [Warning] Recovering table: './annotationDB/harry'
error.log (END)

Open in new window

0
Comment
Question by:andieje
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 3

Expert Comment

by:tearman
ID: 35112789
Good place to start is to run strace while you're executing the perl program to see what the kernel is doing at that point in time.  Wish we had dtrace on Linux, but not available.

Otherwise, check /var/logs, especially the dmesg log for anything interesting.
0
 
LVL 7

Expert Comment

by:droyden
ID: 35113082
What was the perl script attepting to do?
0
 

Author Comment

by:andieje
ID: 35113308
i dont know the point at which it crashed and now the perl code runs through perfectly but i would like to find out why 2 machines crashed

what might be interesting in dmesg as it all looks the same to me?

the perl program was probabily trying to update a record into the db when it crashed but i'm not 100%. the perl program had created some tables, added some records before it crashed and was due to update the records with some data from an external program. the external program had successfully completed before the perl crashed
0
 
LVL 5

Accepted Solution

by:
paulqna earned 500 total points
ID: 35133229
Messages around the time of the crash in these logs could be interesting:

/var/log/syslog*
/var/log/messages*

So that would be anything before a line like:

Mar 12 21:10:49 hostname syslogd version: restart

Open in new window

0
 
LVL 1

Expert Comment

by:jmreidy
ID: 35195845
Check the logs.

If the script was doing inserts and a disk filled up - the system shouldn't crash - the query would just be stopped until the problem was fixed. However I guess a full disk might cause a problem.

Apart from that mysql really does network I/O and disk I/O - with file locking.

I can see 2 causes of a crash:
Some sort of kernel bug - to do with file handling or locking.
or
A hardware problem - bad memory or disk that could be a coincidence or was triggered by the query.

Software/kernel errors should be logged - hardware errors - maybe not.
0

Featured Post

NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question