Solved

Apache Issues

Posted on 2016-10-18
9
47 Views
Last Modified: 2016-10-24
Suppose you had a LAMP stack, and the Apache portion of this LAMP stack would work fine and serve a bunch of pages for a few days, but then it would randomly stop serving pages, and then when you tried to do a graceful Apache restart of this A portion of the LAMP stack, it wouldn't work, so you'd have to completely reboot the entire server to get it to work. Any idea on what is happening? We are using a apache2ctl restart command but it does not work so we have to reboot the server.

This is a SuSE Linux box.
0
Comment
Question by:CityInfoSys
  • 4
  • 2
  • 2
  • +1
9 Comments
 
LVL 7

Expert Comment

by:Jason
ID: 41848656
I have very little experience with suse in particular but is their anything in the log files that will help point you in the right direction and give you an idea of what is going wrong?
0
 
LVL 1

Author Comment

by:CityInfoSys
ID: 41848664
The log file makes no indication of what is causing the issue it is totally random.
0
 
LVL 23

Expert Comment

by:Dr. Klahn
ID: 41849314
Use apache2ctl stop and apache2ctl start instead.  This will break existing sessions and invalidate cookies, but it should always work.  Restart just tickles the running server.  

Check the memory usage on the system when the problem occurs and see if the system is page thrashing or using excessive pagefile.
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41849843
you'd need to debug when the problem occurs

post the output of "top" and "ps -auxww | grep httpd", try apachetop as well

try to kill the apache processes in other (more violent) ways such as "pkill httpd" or "pkill -9 httpd"

there are quite a few reasons why an apache process may become unresponsive... do you serve files over NFS ? do you have php processes that may last forever and clutter server threads ? is the machine properly responsive when apache dies ? which threading mechanism do you use ? mpm_prefork ?
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:CityInfoSys
ID: 41854515
On the latest crash Apache2ctl stop and start were successful in stopping and starting the processes, but I still couldn't access the website. Only a full off/on brought it back. Here are the outputs you requested from the most resent crashes.

Please scroll down it posted the photos differently.

ps -auxww | grep httpd
 1.pngps -auxww | grep httpd
2.pngTop
3.png
0
 
LVL 23

Expert Comment

by:Dr. Klahn
ID: 41854630
I do have to say I'm curious about how an Apache system is using up 5 GB of the available 8 GB of memory (according to top).  The processes shown on the top display account for around 2% of the 8 GB.
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41854949
given the output of ps, it does not seem apache is the one using up 5Gb of ram ( unless i misread something )... a complete "ps -faux" or "pstree" might shed some light regarding what is actually using up the 5GB if necessary

... and there is apparently no resource shortage either ( cpu ok, ram ok ), no long wait for IOs ( waiting for a slow remote filesystem would produce a higher load average )...

my guess would be one of :
- either a script runs with no timeout and clutters all the available threads... looks unlikely and should be visible in the logs and/or apachetop or the likes so
- or apache is serving files from a mounted remote filesystem ( most likely NFS ) and the mount point broke
- if something like heartbeat is in use, network issues could arise as well and be solved by a reboot but you'd most likely have noticed a long time ago

when you reach a page when the problem is occuring
- does apache log the query ?
- does it actually answer with a blank page ? an error ? no answer at all ?

if there is no log and no answer, you might want to check there is no weird network issue : try the page locally with wget, or maybe with "apachectl status", or run "tcpdump -i any port 80" and check that your queries reach the server

if the queries do reach the server and unless one of the above information rings a bell, i'd probably go on running an strace on the apache processes while the problem is occuring and possibly sending a few queries manually while the strace is running so we know what apache is doing. if the httpd processes are locked, their status in top is an indication, and the current system call should be output by strace
0
 
LVL 26

Accepted Solution

by:
skullnobrains earned 500 total points
ID: 41854951
On the latest crash Apache2ctl stop and start were successful

just to make sure, did you actually check that the apache processes were dead after running "apache2ctl stop" ?

if the process were actually restarted, that would indicate a system problem such as a lost mount point, if no, that would direct us towards some lock in apache's internals or a script cluttering the available threads

---

you may also want to check for established connexion during downtimes
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 41857121
did you find what was wrong ?

if yes i'm interested, and others might be

if no you can accept your own solution for 0 points cost
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Over the last year I have answered a couple of basic URL rewriting questions several times so I thought I might as well have a stab at: explaining the basics, providing a few useful links and consolidating some of the most common queries into a sing…
It is possible to boost certain documents at query time in Solr. Query time boosting can be a powerful resource for finding the most relevant and "best" content. Of course the more information you index, the more fields you will be able to use for y…
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now