Solved

Updating php causes apache to "hang" on restart

Posted on 2010-11-12
7
775 Views
Last Modified: 2012-05-10
I compiled a newer version of PHP, now apache will not reload or stop, because the child processes and main apache process just "hang".  I have to use 'kill -9' to remove the process.  So far that is the only defect that we've noticed, but this is not something we want to put into production.

No php code is executed when testing this, just a restart or reload at anytime, will have the problem.  I see this in the log file:

    Nov 8 14:54:30 qa31 apache2[19028]: child process 19037 still did not exit, sending a SIGKILL

The child processes are eventually killed.  Googling around, I see the error msg comes from server/mpm_common.c.  Not much help, it just means the parent process has noticed the child process didn't die.

I used strace on the processes to see the last OS call where they stopped. Setting up a gdb env. isn't really possible, because our servers are still on the old etch Debian base, so the gdb packages are not available anymore.

The child and parent processes all stop with:

    futex(0xb6d6e520, FUTEX_WAIT, 2, NULL

Googling with: apache2 prefork php futex

shows other people have the "futex" symptom, but the fixes suggested haven't been relevant or don't work.

A third-party interface requires at least php-5.2.6.  I'll probably try earlier versions to see when the problem starts, then I can do a diff across the source files, and maybe I'll find the relevant change.

Any ideas on the cause, fix, or what to try next, to diagnose how to fix this would be great.

Best regards,

BruceRafn

------------------------------------
Versions:

I've compiled several versons of php now:
php-5.2.0  - works
php-5.2.6  - processes don't die
php-5.2.12 - processes don't die
php-5.2.14 - processes don't die
php-5.3.3  - processes don't die

OS: Debian 4.0 etch
Kernel: 2.6.18-5-686 #1 SMP

Package Versions:
ii  apache2        2.2.3-4+etch11 Next generation, scalable, extendable web
ii  apache2.2-common     2.2.3-4+etch11  Next gen., scalable, extendable web
ii  apache2-mpm-prefork  2.2.3-4+etch11  Traditional model for Apache HTTPD 2.1
ii  apache2-utils        2.2.3-4+etch11  utility programs for webservers
ii  libapache2-mod-security 2 2.1.1-0    Tighten web app. security for Apache

ii  libapr1        1.2.7-9        The Apache Portable Runtime Library
ii  libaprutil1    1.2.7+dfsg-2+e The Apache Portable Runtime Utility Library
ii  libmysqlclient15off  5.0.32-7etch12   mysql database client library
ii  mysql-client         5.0.32-7etch12   mysql database client (meta package
ii  mysql-client-5.0     5.0.32-7etch12   mysql database client binaries
ii  mysql-common         5.0.32-7etch12   mysql database common files

PHP extentions (custom compiled with the new php versions):
ii  php-ssh2   0.11.0-1  Bindings, for the libssh2 library, for PHP
ii  php-zip    1.10.2-1  A zip management extension
ii  memcache   2.2.6-1   memcached module for PHP


More config details are in the attached file.
config-files.txt
0
Comment
Question by:BruceRafn
  • 5
  • 2
7 Comments
 

Author Comment

by:BruceRafn
ID: 34125767
Update on other versions:
php-5.2.1  - works
php-5.2.2  - works
php-5.2.3  - works
php-5.2.4  - processes don't die

I'll next look at the PHP release notes, and diff between the source files to see if there are changes the might have caused this problem.  Low probability that I'll notice anything though.
0
 
LVL 61

Expert Comment

by:gheist
ID: 34126895
upgrade apache to 2.2.17 to plug millions of security holes. nobody tests builds with obsolete insecure software.
0
 
LVL 61

Accepted Solution

by:
gheist earned 500 total points
ID: 34126900
you need at least 2.2.5 to get it to kill correct processes.
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 

Author Comment

by:BruceRafn
ID: 34138427
Thanks gheist for your suggestions.  I will be trying them out, then update this question.

It *is correct* to build and test with obsolete software, if that is what is in production, because it exposes exactly this kind of problem.  The main error is not keeping production updated, and that will be a top priority now.  (I see the latest Debian apache is on version 2.2.16)
0
 

Assisted Solution

by:BruceRafn
BruceRafn earned 0 total points
ID: 34193393
Upgrading the server to Debian lenny will no doubt fix this problem.

But this is the solution that works for our particular set of packages:
http://www.mail-archive.com/php-bugs@lists.php.net/msg108495.html
( If broken link, see: http://www.webcitation.org/5uRSziHvk )

In summary, when compiling php, use only one of these options, not both:
--with-mysql=
--with-mysqli=

We don't use mysqli.  php-5.3.3 now works apache2-2.2.3-4+etch11
We'll still be upgrading our OS and packages soon.
0
 

Author Comment

by:BruceRafn
ID: 34200898
More information: The problem will solve itself when you rename /lib/tls, so that those so files are not used.  (mysql and mysqli extentions can both be compiled into php).

Solution found at the bottom of this page:
http://bugs.php.net/bug.php?id=42625
( Use this, if link broken: http://www.webcitation.org/5uSq0irej )
0
 

Author Closing Comment

by:BruceRafn
ID: 34221356
The solution that works for our particular set of packages: compile php, using only one of these options, not both:
--with-mysql=
--with-mysqli=
0

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will explain how to establish a SSH connection to Ubuntu through the firewall and using a different port other then 22. I have set up a Ubuntu virtual machine in Virtualbox and I am running a Windows 7 workstation. From the Ubuntu vi…
If you are a web developer, you would be aware of the <iframe> tag in HTML. The <iframe> stands for inline frame and is used to embed another document within the current HTML document. The embedded document could be even another website.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to count occurrences of each item in an array.

929 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now