Hi Expert,

We are a bit new to the Apache server performance tuning and would like to seek some advice. Our Apache web server is configured to be a reverse proxy to send the API call to a remote site and return the result back to the enquiry host. The traffic loading is expected to be very high Max can reach around 40,000 requests per minute which cause error shown below:

[Thu Jan 05 18:42:23.845077 2023] [proxy:error] [pid 111370] (70007)The timeout specified has expired: AH00957: HTTPS: attempt to connect to 10.161.200.67:443 (eid.immd.hksarg) failed

[Thu Jan 05 18:42:23.845124 2023] [proxy:error] [pid 111370] AH00959: ap_proxy_connect_backend disabling worker for (eid.immd.hksarg) for 60s

[Thu Jan 05 18:42:23.845147 2023] [proxy_http:error] [pid 111370] [client 10.88.32.191:35629] AH01114: HTTP: failed to make connection to backend: eid.immd.hksarg

[Thu Jan 05 18:42:25.097383 2023] [proxy:error] [pid 111370] AH00940: HTTPS: disabled connection for (eid.immd.hksarg)

[Thu Jan 05 18:42:26.196036 2023] [proxy:error] [pid 111370] AH00940: HTTPS: disabled connection for (eid.immd.hksarg)

[Thu Jan 05 18:42:27.363899 2023] [proxy:error] [pid 111370] AH00940: HTTPS: disabled connection for (eid.immd.hksarg)

[Thu Jan 05 18:42:28.814837 2023] [proxy:error] [pid 111370] AH00940: HTTPS: disabled connection for (eid.immd.hksarg)

[Thu Jan 05 18:42:30.070462 2023] [proxy:error] [pid 111370] AH00940: HTTPS: disabled connection for (eid.immd.hksarg)

[Thu Jan 05 18:42:30.988064 2023] [proxy:error] [pid 111493] (70007)The timeout specified has expired: AH00957: HTTPS: attempt to connect to 10.161.200.67:443 (eid.immd.hksarg) failed

Since then, we have implemented the following fine tuning on the testing Apache Server:

1) To cater for too many files opening scenario, we've set the ulimit (soft and hard):

modifying the /etc/security/limits.conf  

# With the following value:

root soft nofile 50000

root hard nofile 50000

root soft nproc  131072

root hard nproc  131072

2) We've set the MPM (Multiple processing module) mode from mpm_prefork_module to mpm_event_module. Also, we have configured the event module paremeters as shown below:

#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

#LoadModule mpm_worker_module modules/mod_mpm_worker.so

LoadModule mpm_event_module modules/mod_mpm_event.so

# event MPM

# StartServers: initial number of server processes to start

# MinSpareThreads: minimum number of worker threads which are kept spare

# MaxSpareThreads: maximum number of worker threads which are kept spare

# ThreadsPerChild: constant number of worker threads in each server process

# MaxRequestWorkers: maximum number of worker threads

# MaxConnectionsPerChild: maximum number of connections a server process serves

#                         before terminating

<IfModule mpm_event_module>

    StartServers             8

    MinSpareThreads        200

    MaxSpareThreads        750

    ThreadsPerChild         64

    MaxRequestWorkers      1024

    MaxConnectionsPerChild   0

</IfModule>

We've restart apache "systemctl restart httpd" after making the above changes.

Our questions are:

(1) How can we check if the ulimit value and the MPM module attribute value (i.e.  Min & Max SpareThreads, MaxRequestWorkers etc.) is indeed in place ?

(2) We were wondering what addition configuration is needed in order to fine tune our Apache server configuration to handle such high volume of request (i.e. 40,000+ request/min) as we are going to do a load test to prove the setting very soon ?

Thanks a lot

Patrick