sctccomm
asked on
CGI takes almost 2 minutes to load using thttpd
Hello,
I am testing out thttpd v2.27, and I have a few CGI files that I have been using with Appweb for a long time now.
I noticed immediately that it takes a significant amount of time to finish loading a simple CGI page using thttpd, while the same page completes loading almost immediately.
Is there any configurations that I should look at try to speed up the loading time?
I am testing out thttpd v2.27, and I have a few CGI files that I have been using with Appweb for a long time now.
I noticed immediately that it takes a significant amount of time to finish loading a simple CGI page using thttpd, while the same page completes loading almost immediately.
Is there any configurations that I should look at try to speed up the loading time?
Dave Baldwin
Appweb is a currently supported product. The last release for thttpd was in 1998. A lot of things have changed since then.
http://opensource.dyc.edu/sthttpd - is a currently maintained version... (at least until 2014)
now are you sure the problem is with the webserver or are the .cgi scripts slow to respond.
Or might the server be memory starved? "Disk" near full might also be a source for slowness.
now are you sure the problem is with the webserver or are the .cgi scripts slow to respond.
Or might the server be memory starved? "Disk" near full might also be a source for slowness.
ASKER
I don't believe the CGI is slow because using Appweb it is very fast.
I also monitored the CPU and memory usage, and the CPU seems to be less than 1% for thttpd and CGI pages. The memory usage currently is at 30%.
I can try using sthttpd.
I also monitored the CPU and memory usage, and the CPU seems to be less than 1% for thttpd and CGI pages. The memory usage currently is at 30%.
I can try using sthttpd.
How about latencies introduced because of DNS (reverse) lookups failing.
ASKER
I just tested using sthttpd, but I get the same slow behavior.
I wonder if it could be how I designed my CGI pages, which uses a lot of different shared libraries that I've written. Maybe thttpd does not handle that very well.
I do not believe DNS is involved with the problem since I'm testing locally and using the IP address of the server.
I wonder if it could be how I designed my CGI pages, which uses a lot of different shared libraries that I've written. Maybe thttpd does not handle that very well.
I do not believe DNS is involved with the problem since I'm testing locally and using the IP address of the server.
wrt. DNS you can enter the server's local ip addresses in the /etc/hosts if needed.
CGI runs in a separate processes per link (also keep in mind that a browser may open more than one link) so there may be more processes active. opening lot's of files may cause delays. but only the first time after that they should be in cache (unless the system is memory starved). btw. 30% free isn't a quantitative measurement (say 1GB system, 30% used = 670MB free, if each process needs 1GB and 4 are started it fill fast become a swapping system, which will hurt).
Just to be sure: no throttling entries have been defined which could extremely limit throughput?
If you need measurements on your system look for collectd & CGP as graph producer. (CGP can send to a neighbour system)
http://collectd.org & https://github.com/pommi/CGP
I did try another tiny webserver namen mongoose ( https://github.com/cesanta/mongoose )
which seems more responsive then thhtpd.
CGI runs in a separate processes per link (also keep in mind that a browser may open more than one link) so there may be more processes active. opening lot's of files may cause delays. but only the first time after that they should be in cache (unless the system is memory starved). btw. 30% free isn't a quantitative measurement (say 1GB system, 30% used = 670MB free, if each process needs 1GB and 4 are started it fill fast become a swapping system, which will hurt).
Just to be sure: no throttling entries have been defined which could extremely limit throughput?
If you need measurements on your system look for collectd & CGP as graph producer. (CGP can send to a neighbour system)
http://collectd.org & https://github.com/pommi/CGP
I did try another tiny webserver namen mongoose ( https://github.com/cesanta/mongoose )
which seems more responsive then thhtpd.
First dump taffic to make sure no reverse DNS queries are made.
If that is not the case thttpd uses system event library which may by unlucky chance not interact with kernel well. Can you strace to sched some light on if it works well?
If that is not the case thttpd uses system event library which may by unlucky chance not interact with kernel well. Can you strace to sched some light on if it works well?
ASKER
We have discovered that thttpd does not support other features that we require, so further investigation regarding this server has been stopped.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Yes, but still you did not reply regarding DNS.
ASKER
Can you explain once again why DNS would be an issue? If our system does not have DNS, would this matter?
We currently only use IP addresses, so no domain names are ever used to access the web server.
We currently only use IP addresses, so no domain names are ever used to access the web server.
Yes it may very well matter, as lower libraries might try to translate an address to a hostname (just for logging purposes).
So if lookups fail consistently you may well introduce 30s timeouts to EVERY request.
(That may also happen further on when your systems connect to others that may try to resolve the addresses they are contacted from even before accepting a call)
So if lookups fail consistently you may well introduce 30s timeouts to EVERY request.
(That may also happen further on when your systems connect to others that may try to resolve the addresses they are contacted from even before accepting a call)
ASKER
But we do not see any slow loading when using a normal (non-CGI-generated) HTML page.
So if a simple HTML page loads quickly, while CGI pages do not, then would this eliminate the DNS factor?
So if a simple HTML page loads quickly, while CGI pages do not, then would this eliminate the DNS factor?
It should be easy to check from REMOTE_HOST CGI/1.1 variable inside script.
thttpd does not have any options to stop this behaviour. Maybe some dstribution package patches both gethostbya() occurences out of it.
thttpd does not have any options to stop this behaviour. Maybe some dstribution package patches both gethostbya() occurences out of it.
And another quicktest would be to enter the addresses where you come from in the /etc/hosts file on the embedded server.
(I understand that this will not work at large, but it will for a test. from a known system)
If that eliminates slowness then the slowness can be gethostbyaddress() related.
(I understand that this will not work at large, but it will for a test. from a known system)
If that eliminates slowness then the slowness can be gethostbyaddress() related.
ASKER
Tried adding the client IP address to /etc/hosts, but there is no difference in behavior.
Since there are other features that we require that are not supported on thttpd, we would like to stop any further troubleshooting with this application.
Since there are other features that we require that are not supported on thttpd, we would like to stop any further troubleshooting with this application.
You could also remove resolv.conf with same effect.
ASKER
Tried removing /etc/resolv.conf, but still no change.
You do know that you can run CGI scripts just from the command line do you?
Just provide all HTTP Header data as environment variables and remaining data as input on STDIN of the script.
(In the case of POST it needs to be MIME formatted.
Just provide all HTTP Header data as environment variables and remaining data as input on STDIN of the script.
(In the case of POST it needs to be MIME formatted.
ASKER
Project has been stopped.