Solved

Apache bloated with unecessary underlying processes, consuming excessive RAM

Posted on 2011-03-03
14
921 Views
Last Modified: 2012-05-11
Per below, why is libldap-2.3.so.0.2.31 showing up in my Apache processes when there's no LDAP module being called out by either httpd.conf (or any other *.conf files) or the php.ini files?

Now Apache does load PHP and in my PHP complied settings I show '--with-ldap=shared' (see below). However I believe when the module is "shared" it means I have to explicitly load the module meaning that something like "extension=/usr/lib64/php/modules/ldap.so" would need to be in my php.ini file (but there no such entry).

[root@www /]# ps 31673
  PID TTY      STAT   TIME COMMAND
31673 ?        S      0:00 /usr/sbin/httpd

[root@www /]# pmap -d 31673 | grep ldap
00002b362d5b0000     224 r-x-- 0000000000000000 0fd:00000 libldap-2.3.so.0.2.31
00002b362d5e8000    2048 ----- 0000000000038000 0fd:00000 libldap-2.3.so.0.2.31
00002b362d7e8000       8 rw--- 0000000000038000 0fd:00000 libldap-2.3.so.0.2.31
[root@www /]#

Open in new window


[root@www /]# php -i | more
phpinfo()
PHP Version => 5.1.6

System => Linux www.t1shopper.com 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64
Build Date => Nov 29 2010 16:41:38
Configure Command =>  './configure' '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '
--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--cache-file
=../config.cache' '--with-libdir=lib64' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d' '--disable-debug' '--with-pic' '--disable-rpath' '--without-pear' '--with-bz2' '--with-curl' '--with-exec-dir=/usr/bin' '--wit
h-freetype-dir=/usr' '--with-png-dir=/usr' '--enable-gd-native-ttf' '--without-gdbm' '--with-gettext' '--with-gmp' '--with-iconv' '--with-jpeg-dir=/usr' '--with-openssl' '--with-png' '--with-pspell' '--with-expat-dir=/usr' '--with-pcre-re
gex=/usr' '--with-zlib' '--with-layout=GNU' '--enable-exif' '--enable-ftp' '--enable-magic-quotes' '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' '--enable-track-vars' '--enable-trans-sid' '--enable-yp' '--ena
ble-wddx' '--with-kerberos' '--enable-ucd-snmp-hack' '--with-unixODBC=shared,/usr' '--enable-memory-limit' '--enable-shmop' '--enable-calendar' '--enable-dbx' '--enable-dio' '--with-mime-magic=/usr/share/file/magic.mime' '--without-sqlite
' '--with-libxml-dir=/usr' '--with-xml' '--with-system-tzdata' '--enable-force-cgi-redirect' '--enable-pcntl' '--with-imap=shared' '--with-imap-ssl' '--enable-mbstring=shared' '--enable-mbstr-enc-trans' '--enable-mbregex' '--with-ncurses=
shared' '--with-gd=shared' '--enable-bcmath=shared' '--enable-dba=shared' '--with-db4=/usr' '--with-xmlrpc=shared' '--with-ldap=shared' '--with-ldap-sasl' '--with-mysql=shared,/usr' '--with-mysqli=shared,/usr/lib64/mysql/mysql_config' '--
enable-dom=shared' '--with-dom-xslt=/usr' '--with-dom-exslt=/usr' '--with-pgsql=shared' '--with-snmp=shared,/usr' '--enable-soap=shared' '--with-xsl=shared,/usr' '--enable-xmlreader=shared' '--enable-xmlwriter=shared' '--enable-fastcgi' '
--enable-pdo=shared' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-mysql=shared,/usr/lib64/mysql/mysql_config' '--with-pdo-pgsql=shared,/usr' '--with-pdo-sqlite=shared,/usr' '--enable-dbase=shared'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc/php.ini
Scan this dir for additional .ini files => /etc/php.d
additional .ini files parsed => /etc/php.d/bcmath.ini,
/etc/php.d/dom.ini,
/etc/php.d/mysql.ini,
/etc/php.d/soap.ini
...

Open in new window

0
Comment
Question by:Geoff Millikan
  • 6
  • 5
  • 3
14 Comments
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Do you encounter any problems with standard apache included in your distribution?

Maybe PHP loads LDAP extension?

I would not call 2kB import a Bload....
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 150 total points
Comment Utility
These are just shared libraries, not apache modules. Presumably apache specifies them in the build. It's not really bloat - if no process in the system actually uses them then they will never have been paged in, although their linkage sections will use a little per-process memory. If some process does use them, then text (i.e. code) pages will have been brought into RAM, but only one copy system-wide.
Really, it's nothing to worry about.
0
 

Author Comment

by:Geoff Millikan
Comment Utility
I hear you saying that it's nothing to worry about but we run 100+ Apache threads.  Each thread has the 2kB bloat.  And of course this isn't the only shared library that Apache is linking that isn't in use at all.  

I was loading some default PHP modules (like dbase.so) into PHP and until using pmap, I had no idea that because PHP is loading up dbase.so, Apache loads it up too!  By commenting out these extra PHP modules, we were able to take the average Apache thread writeable/private memory size down from 7000K to 5736K.  When running as many threads as we are, this really adds up.

So can you convince me again that I shouldn't try to remove the LDAP?  I'd love to free up some more precious RAM. :-)
0
 

Author Comment

by:Geoff Millikan
Comment Utility
Maybe PHP loads LDAP extension?
 PHP was complied with the '--with-ldap=shared' flag.  Does this complies setting mean it's always going to load the /usr/lib64/php/modules/ldap.so module?  If yes, then of course Apache is going to pull it on too.  My phpinfo() isn't showing that LDAP is supported (other than the compile setting)
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
I could only recommend using worker MPM instead of prefork to have one apache process per 200 connections instead of 1:1
0
 

Author Comment

by:Geoff Millikan
Comment Utility
I've read worker MPM isn't very thread safe so we've stayed away from it but lots of people seem to recommend it so I'm surprised.  Also, the RPM binaries from CentOS and RHEL don't offer the worker MPM and we really like getting the security patches via yum instead of have to recompiling if there's a security update.
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
http://serverfault.com/questions/97969/how-to-install-mpm-worker-on-centos-5-3

Please complement your "i've read" with a repeatable bug report, preferably to Apache team.
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 34

Expert Comment

by:Duncan Roe
Comment Utility
Can you compile PHP with --without-LDAP?
0
 

Author Comment

by:Geoff Millikan
Comment Utility
gheist: I see your point but there seems to be no bug report, just offical documentation that seems to advise against using the Worker MPM with PHP.  Per link below, "If you feel you have to use a threaded MPM, look at a FastCGI configuration where PHP is running in its own memory space."

http://www.php.net/manual/en/faq.installation.php#faq.installation.apache2

duncan_roe: Yes.  But are you sure that would fix our issue?  (If it did this though, it woudl be a pian because we really like getting the security patches via yum instead of having to recompiling if there's a security update.  

It would be awesome though if someone knew if when PHP is complied with the '--with-ldap=shared' flag it means the /usr/lib64/php/modules/ldap.so module is always going to be loaded?
0
 
LVL 61

Accepted Solution

by:
gheist earned 350 total points
Comment Utility
--with-ldap compiles in ldap support.

read here - if you can get worker running enjoy otherwose it works like it works.
http://brian.moonspot.net/2008/02/13/apache-worker-and-php/
0
 
LVL 34

Assisted Solution

by:Duncan Roe
Duncan Roe earned 150 total points
Comment Utility
are you sure that would fix our issue? Not completely, you're the one who can try it, not I. In any case I suspect --without-LDAP refers to /usr/lib64/libldap-2.3.so.0.2.31  and not /usr/lib64/php/modules/ldap.so. The latter library did not show up in your pmap output, but the former did. Here's what I have (please excuse the 32-bit installation:)
19:26:21$ cd /usr/lib
19:58:54$ ls -Fl *ldap*
lrwxrwxrwx 1 root root     20 Jul 25  2010 libldap-2.4.so.2 -> libldap-2.4.so.2.5.4
-rw-r--r-- 1 root root 256280 Feb 13  2010 libldap-2.4.so.2.5.4
-rw-r--r-- 1 root root    868 Feb 13  2010 libldap.la
lrwxrwxrwx 1 root root     20 Jul 25  2010 libldap.so -> libldap-2.4.so.2.5.4
lrwxrwxrwx 1 root root     22 Jul 25  2010 libldap_r-2.4.so.2 -> libldap_r-2.4.so.2.5.4
-rw-r--r-- 1 root root 273508 Feb 13  2010 libldap_r-2.4.so.2.5.4
-rw-r--r-- 1 root root    889 Feb 13  2010 libldap_r.la
lrwxrwxrwx 1 root root     22 Jul 25  2010 libldap_r.so -> libldap_r-2.4.so.2.5.4

Open in new window

I imagine the _r variants have functions with extra arguments to be fully re-entrant (thread safe). The  thing I find interesting is that the .so files do not have the execute bit set. Is that the case on your system also?
A consequence of loading libldap.so is that some other libraries get dragged in as well:
20:06:06$ cat libldap.la
# libldap.la - a libtool library file
# Generated by ltmain.sh - GNU libtool 1.5.22-OpenLDAP (1.1220.2.365 2005/12/18 22:14:06)
#
# Please DO NOT delete this file!
# It is necessary for linking the library.

# The name that we can dlopen(3).
dlname='libldap-2.4.so.2'

# Names of this library.
library_names='libldap-2.4.so.2.5.4 libldap-2.4.so.2 libldap.so'

# The name of the static archive.
old_library=''

# Libraries that this one depends upon.
dependency_libs=' /usr/lib/liblber.la /usr/lib/libsasl2.la -ldl -lresolv -lssl -lcrypto'

# Version information for libldap.
current=7
age=5
revision=4

# Is this an already installed library?
installed=yes

# Should we warn about portability when linking against -modules?
shouldnotlink=no

# Files to dlopen/dlpreopen
dlopen=''
dlpreopen=''

# Directory that this library needs to be installed in:
libdir='/usr/lib'

Open in new window

libldap.so will pull in liblber.so, libsasl2.so, libdl.so, libresolv.so, libssl.so and libcrypto.so. It's possible that many of these libraries would have been loaded anyway, but some might not have been, so that omitting libldap.so might save you more than you expected.
If it did this though, it would be a pain because we really like getting the security patches via yum instead of having to recompiling if there's a security update.: that's the price you pay for nonstandard fine-tuning. But rather than go to all that trouble, why not buy more RAM?
0
 

Author Comment

by:Geoff Millikan
Comment Utility
Sorry guys, this is taking a bit more time to research, hold please..
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
php 5.1 is NO LONGER MAINTAINED...
0
 

Author Comment

by:Geoff Millikan
Comment Utility
duncan_roe: Holy cow, thanks for all the detail! I thought I was good at Apache but I can see I'm not as cool as I thought I was.  But to your questions/feedback:

..the .so files do not have the execute bit set. Is that the case on your system also?

Mine are set to execute, 0755.  Note that I'm probably using the 64 bit versions.
[root@www /]# cd /usr/lib
[root@www lib]# ls -Fl *ldap*
lrwxrwxrwx 1 root root     21 Jan  3 18:30 libldap-2.3.so.0 -> libldap-2.3.so.0.2.31*
-rwxr-xr-x 1 root root 238576 Nov 29 08:50 libldap-2.3.so.0.2.31*
lrwxrwxrwx 1 root root     23 Jan  3 18:30 libldap_r-2.3.so.0 -> libldap_r-2.3.so.0.2.31*
-rwxr-xr-x 1 root root 255796 Nov 29 08:50 libldap_r-2.3.so.0.2.31*
lrwxrwxrwx 1 root root     26 Aug 22  2010 libnss_ldap.so -> ../../lib/libnss_ldap.so.2*
[root@www lib]# cd /usr/lib64
[root@www lib64]# ls -Fl *ldap*
lrwxrwxrwx 1 root root     21 Jan  3 18:30 libldap-2.3.so.0 -> libldap-2.3.so.0.2.31*
-rwxr-xr-x 1 root root 241392 Nov 29 08:50 libldap-2.3.so.0.2.31*
-rwxr-xr-x 1 root root 220136 Dec 11  2007 libldap60.so*
lrwxrwxrwx 1 root root     23 Jan  3 18:30 libldap_r-2.3.so.0 -> libldap_r-2.3.so.0.2.31*
-rwxr-xr-x 1 root root 257520 Nov 29 08:50 libldap_r-2.3.so.0.2.31*
lrwxrwxrwx 1 root root     28 Aug 22  2010 libnss_ldap.so -> ../../lib64/libnss_ldap.so.2*
-rwxr-xr-x 1 root root  20560 Dec 11  2007 libprldap60.so*
-rwxr-xr-x 1 root root  49024 Dec 11  2007 libssldap60.so*

Open in new window


...omitting libldap.so might save you more than you expected.

That would be nice!

...that's the price you pay for nonstandard fine-tuning.

True.  And rats.  But that doesn't mean we have to like it.  ;-)

But rather than go to all that trouble, why not buy more RAM?

Great question. Because this is an "older" server (like 3 years old), getting sticks of 4GB registered server RAM runs about $1000 USD per stick.  So it's cheaper to get a new box, and that's not something we want to do now.  Rather than having to watch the security releases and recompile every time, we're probably just stick with it as it is.

gheist: php 5.1 is NO LONGER MAINTAINED   Did you bring this up because you're concerned for our security? Thank you!  Correct me if I'm wrong here but since PHP 5.1 is the version shipping with RHEL5 (and CentOS 5) it's receiving back ported security patches via RedHat per link here.

Have we answered the question in the OP?  Yes. gheist said, --with-ldap compiles in ldap support.  I tested this by looking at other "--with-service" complied in modules and almost all of them (like XML and gmp) that are in the PHP compile settings are also showing up in the Apache process.

Really appreciate all the help here.  Many thanks.
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

It is possible to boost certain documents at query time in Solr. Query time boosting can be a powerful resource for finding the most relevant and "best" content. Of course the more information you index, the more fields you will be able to use for y…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
The viewer will learn how to dynamically set the form action using jQuery.

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now