Solved

Tracking which Unix process pid just crashed

Posted on 2010-09-05
3
706 Views
Last Modified: 2012-05-10

I got a monitoring alert that says a Unix process just crashed but the alert did not
specify the pid or process name that just crashed.

Is there any way to find out?

What about the directory /usr/ucb/... : does it hold any clue

I recall in Linux  /var/run   there are *.pid files that hold the pid of processes.
If a process was abrupted terminated or manually "killed", does the pid file
stays behind?  I thought of going thru one by one the .pid files to see the pid
& check which ones are no longer found in "ps -ef"

/var/log/messages did not give any clue

any good Shell script / command to check this easily would be most welcome
as well
0
Comment
Question by:sunhux
3 Comments
 

Author Comment

by:sunhux
Comment Utility


What does the date stamp of those /var/run/*.pid files mean?
0
 
LVL 6

Accepted Solution

by:
apresence earned 470 total points
Comment Utility
Not all applications write the pid files.  But, you are right, the pid files are usually deleted when a process exits normally.  The date stamp (mtime) on the /var/run/*.pid files is the last time the process was started.

If you want to check those pid files to see if any of their processes are missing, the attached code will do it for ya.

Sample output (I create a theoretical testproc.pid file with a pid that doesn't exist for testing):
root@beta:~/exex/test9 $ echo 999 >/var/run/testproc.pid
root@beta:~/exex/test9 $ ./show_missing_pids.sh
PID 3146 (/var/run/atd.pid): RUNNING
PID 2334 (/var/run/auditd.pid): RUNNING
PID 3015 (/var/run/crond.pid): RUNNING
...
PID 999 (/var/run/testproc.pid): NOT RUNNING
...
root@beta:~/exex/test9 $
#!/bin/sh

for i in `ls /var/run/*.pid`; do

  actual_pid=`perl -ne 'print "$1\n" if /^(\d+)/' < $i`

  if [ -n "$actual_pid" ]; then

    ps -p $actual_pid >/dev/null 2>&1

    if [ $? -eq 0 ]; then

      echo "PID $actual_pid ($i): RUNNING"

    else

      echo "PID $actual_pid ($i): NOT RUNNING"

    fi

  fi

done

Open in new window

0
 
LVL 61

Assisted Solution

by:gheist
gheist earned 30 total points
Comment Utility
do you have any log or boot message entry confirming a process crash?
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Suggested Solutions

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now