Error log shows on Linux server shows: Caught SIGTERM, shutting down. Any ideas what that means?

hi guys,

So I posted this week about how our EC2 Linux instance with Amazon suddenly became inaccessible on port 80 (SSH worked fine) and that things only worked once we restarted the httpd service. This happened around 4 to 5 times over the duration of two days.  

Thanks to EE people, I got help and  looked in the right places. Having trawled through some error logs in the /etc/httpd/logs I opened a log called 'error_log' with the date/timestamp of that time.

One thing I can see is that at exactly the time this issue occurred, this line was in the log:

Caught SIGTERM, shutting down


Does this mean the Apache service was shutdown for some reason? If so, then is there a way of finding out what caused it?

Thanks
Yashy
LVL 1
YashyAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

arnoldCommented:
A SIGTERM is a notice to terminate.

As to who or what issued the signal is a more difficult thing to track down.
This can be a result of a monitoring service that detected that the process exceeded a set threshold. If this was a consequence of a monitoring system, the app should have been restarted shortly there after.  It is possible that the detection process has a 5 or longer gap between, terminate and start.
You know when the signal was issued. In the environment you are it is more difficult to say.  
Checking whether something else was going on I.e. Updates being applied, admin on system performing a task...

Adding logging events to the service scripts to revord when anyone starts/stops/restarts services ..... Will have some answers going forward.

Apache/openssl patch and the admin issued a stop instead of restart in error.
0
YashyAuthor Commented:
Thank you Arnold for the info. I appreciate it.

That last line 'Apache/openssl patch and the admin issued a stop instead of restart in error', was it an example of what you think it is? Or that it is something which we should add the logging script?
0
arnoldCommented:
Mere guesstimate of what might have happened.  A SIGTERM is a "controlled" event.
i.e. if your firm uses secure site https: the recent OPENSSL issue would necessitate the update of httpd/openssl which was done. To get the new running, the service needs to be restarted.


when you issue service httpd stop SIGTERM is what is sent.  This gives the application the time it needs to go through the process of exiting: i.e a SIGTERM received while there are 100 requests pending, all 100 requests will be responded to while no new ones will be accepted, at the conclusion of answering all 100, the service will stop the child processes if any, release memory and exit.

Not sure how much control/auditing capacity you have within the EC2 setup, often on ones own managed server, looking at last to see who was logged on at the time. Looking at the cron jobs to see if any of these correspond to the event.
etc. a logrorationg script usually issues a HUP when logs are rotated, but an admin may have thought they were adjusting/altering a different script when they resumed their editing and added a or replaced a USR2 with a TERM signal.

The possibilities are endless of what might have happened.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
gheistCommented:
SIGTERM is normal way to stop apache (apachectl stop)
You must see system accounting logs for who was logged in at that moment and killed the process. Apache does not do that itself.
normally one goes with SIGUSR2 (apachectl graceful-stop)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.