Solved

Updating php causes apache to "hang" on restart

Posted on 2010-11-12
7
774 Views
Last Modified: 2012-05-10
I compiled a newer version of PHP, now apache will not reload or stop, because the child processes and main apache process just "hang".  I have to use 'kill -9' to remove the process.  So far that is the only defect that we've noticed, but this is not something we want to put into production.

No php code is executed when testing this, just a restart or reload at anytime, will have the problem.  I see this in the log file:

    Nov 8 14:54:30 qa31 apache2[19028]: child process 19037 still did not exit, sending a SIGKILL

The child processes are eventually killed.  Googling around, I see the error msg comes from server/mpm_common.c.  Not much help, it just means the parent process has noticed the child process didn't die.

I used strace on the processes to see the last OS call where they stopped. Setting up a gdb env. isn't really possible, because our servers are still on the old etch Debian base, so the gdb packages are not available anymore.

The child and parent processes all stop with:

    futex(0xb6d6e520, FUTEX_WAIT, 2, NULL

Googling with: apache2 prefork php futex

shows other people have the "futex" symptom, but the fixes suggested haven't been relevant or don't work.

A third-party interface requires at least php-5.2.6.  I'll probably try earlier versions to see when the problem starts, then I can do a diff across the source files, and maybe I'll find the relevant change.

Any ideas on the cause, fix, or what to try next, to diagnose how to fix this would be great.

Best regards,

BruceRafn

------------------------------------
Versions:

I've compiled several versons of php now:
php-5.2.0  - works
php-5.2.6  - processes don't die
php-5.2.12 - processes don't die
php-5.2.14 - processes don't die
php-5.3.3  - processes don't die

OS: Debian 4.0 etch
Kernel: 2.6.18-5-686 #1 SMP

Package Versions:
ii  apache2        2.2.3-4+etch11 Next generation, scalable, extendable web
ii  apache2.2-common     2.2.3-4+etch11  Next gen., scalable, extendable web
ii  apache2-mpm-prefork  2.2.3-4+etch11  Traditional model for Apache HTTPD 2.1
ii  apache2-utils        2.2.3-4+etch11  utility programs for webservers
ii  libapache2-mod-security 2 2.1.1-0    Tighten web app. security for Apache

ii  libapr1        1.2.7-9        The Apache Portable Runtime Library
ii  libaprutil1    1.2.7+dfsg-2+e The Apache Portable Runtime Utility Library
ii  libmysqlclient15off  5.0.32-7etch12   mysql database client library
ii  mysql-client         5.0.32-7etch12   mysql database client (meta package
ii  mysql-client-5.0     5.0.32-7etch12   mysql database client binaries
ii  mysql-common         5.0.32-7etch12   mysql database common files

PHP extentions (custom compiled with the new php versions):
ii  php-ssh2   0.11.0-1  Bindings, for the libssh2 library, for PHP
ii  php-zip    1.10.2-1  A zip management extension
ii  memcache   2.2.6-1   memcached module for PHP


More config details are in the attached file.
config-files.txt
0
Comment
Question by:BruceRafn
  • 5
  • 2
7 Comments
 

Author Comment

by:BruceRafn
ID: 34125767
Update on other versions:
php-5.2.1  - works
php-5.2.2  - works
php-5.2.3  - works
php-5.2.4  - processes don't die

I'll next look at the PHP release notes, and diff between the source files to see if there are changes the might have caused this problem.  Low probability that I'll notice anything though.
0
 
LVL 61

Expert Comment

by:gheist
ID: 34126895
upgrade apache to 2.2.17 to plug millions of security holes. nobody tests builds with obsolete insecure software.
0
 
LVL 61

Accepted Solution

by:
gheist earned 500 total points
ID: 34126900
you need at least 2.2.5 to get it to kill correct processes.
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:BruceRafn
ID: 34138427
Thanks gheist for your suggestions.  I will be trying them out, then update this question.

It *is correct* to build and test with obsolete software, if that is what is in production, because it exposes exactly this kind of problem.  The main error is not keeping production updated, and that will be a top priority now.  (I see the latest Debian apache is on version 2.2.16)
0
 

Assisted Solution

by:BruceRafn
BruceRafn earned 0 total points
ID: 34193393
Upgrading the server to Debian lenny will no doubt fix this problem.

But this is the solution that works for our particular set of packages:
http://www.mail-archive.com/php-bugs@lists.php.net/msg108495.html
( If broken link, see: http://www.webcitation.org/5uRSziHvk )

In summary, when compiling php, use only one of these options, not both:
--with-mysql=
--with-mysqli=

We don't use mysqli.  php-5.3.3 now works apache2-2.2.3-4+etch11
We'll still be upgrading our OS and packages soon.
0
 

Author Comment

by:BruceRafn
ID: 34200898
More information: The problem will solve itself when you rename /lib/tls, so that those so files are not used.  (mysql and mysqli extentions can both be compiled into php).

Solution found at the bottom of this page:
http://bugs.php.net/bug.php?id=42625
( Use this, if link broken: http://www.webcitation.org/5uSq0irej )
0
 

Author Closing Comment

by:BruceRafn
ID: 34221356
The solution that works for our particular set of packages: compile php, using only one of these options, not both:
--with-mysql=
--with-mysqli=
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

In my business, I use the LTS (Long Term Support) versions of Linux. My workstations do real work, and so I rarely have the patience to deal with silly problems caused by an upgraded kernel that had experimental software on it to begin with from a r…
Since pre-biblical times, humans have sought ways to keep secrets, and share the secrets selectively.  This article explores the ways PHP can be used to hide and encrypt information.
The viewer will learn how to count occurrences of each item in an array.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now