Python CGI Runs Fine In SSH, But Not In Browser?
Hello Experts!
Any of you Python experts ever experienced a similar issue to the one I am facing right now? I have set up a simple "Hello World" python cgi script which runs just fine in SSH, but gives a 500 Internal Server Error when viewed through a browser.
Here is the script:
Any of you Python experts ever experienced a similar issue to the one I am facing right now? I have set up a simple "Hello World" python cgi script which runs just fine in SSH, but gives a 500 Internal Server Error when viewed through a browser.
Here is the script:
#!/usr/bin/python
import cgi
print("Content-Type: text/html\n\n")
print("Hello World!")
Hmm,
It's quite trick to know without further info.
So I'll ask you some questions a little blindly.
Q1.) What web server is being used?
- Do you have any other cgi script written in an other language, that is working? like for example:
Q2.) Do you have access to the webserver's logs? (This would definitely simplify the debugging)
Q3.) I assume you execute the script on the same machine on which the server is run. Am I right?
Q4.) How do you call the script via ssh? (let's assume you called your cgi-script mycgiscript.py)
with
python ./mycgiscript.py
or just with
./mycgiscript.py
Q5.) What's the output of following command?
which python
Q6.) what's the output of following command?
python -m cgi > /dev/null && echo $?
It's quite trick to know without further info.
So I'll ask you some questions a little blindly.
Q1.) What web server is being used?
- Do you have any other cgi script written in an other language, that is working? like for example:
#!/bin/sh
cat << eot
Content-Type: text/html
Hello world
eot
Q2.) Do you have access to the webserver's logs? (This would definitely simplify the debugging)
Q3.) I assume you execute the script on the same machine on which the server is run. Am I right?
Q4.) How do you call the script via ssh? (let's assume you called your cgi-script mycgiscript.py)
with
python ./mycgiscript.py
or just with
./mycgiscript.py
Q5.) What's the output of following command?
which python
Q6.) what's the output of following command?
python -m cgi > /dev/null && echo $?
Wow gelonida, thank you so much for your response. It sounds like you really know what you are doing.
Ok, I've set up the following answers to correspond with each of your questions:
A1.) I am running a Linux system using Apache version 2.2.24 and CentOS release 6.5. I took your script and saved it as a.sh under the cgi-bin subdirectory and made it executable. I ran it using ./a.sh and it ran fine. I tried it on the web browser using the format (url is just an example) "http://mysite.com/cgi-bin/a.sh" and I got the same error as I did when I tried to run "http://mysite.com/cgi-bin/a.py".
A2.) I do have access to the webserver logs. Currently the error log is showing only the following error: [Fri May 09 15:49:08 2014] [client xx.xx.xx.xx] File does not exist: /var/www/test/500.shtml (ip and directory have been changed), which indicates to me that the script is throwing a 500 error and is simply trying to access the page to display that error.
A3.) Yes, you are right.
A4.) I call it with ./mycgiscript.py
A5.) /usr/bin/python
A6.) 0
Ok, I've set up the following answers to correspond with each of your questions:
A1.) I am running a Linux system using Apache version 2.2.24 and CentOS release 6.5. I took your script and saved it as a.sh under the cgi-bin subdirectory and made it executable. I ran it using ./a.sh and it ran fine. I tried it on the web browser using the format (url is just an example) "http://mysite.com/cgi-bin/a.sh" and I got the same error as I did when I tried to run "http://mysite.com/cgi-bin/a.py".
A2.) I do have access to the webserver logs. Currently the error log is showing only the following error: [Fri May 09 15:49:08 2014] [client xx.xx.xx.xx] File does not exist: /var/www/test/500.shtml (ip and directory have been changed), which indicates to me that the script is throwing a 500 error and is simply trying to access the page to display that error.
A3.) Yes, you are right.
A4.) I call it with ./mycgiscript.py
A5.) /usr/bin/python
A6.) 0
A1 indicates the problem is related to the server and not to python.
A3 to A6 look all OK, no surprises there.
A2 is not reall helpful :-(
So it could be access right problems or a problem with the apache config.
perhaps the server cannot read / execute the script or they cgi scripts are not located in the right directory
Can you check the file permissions of our cgi scripts and of the directory containing it and of all its parent directories?
Has the user running the web server enough permissions to read and execute the cgi scripts?
Could you check the apache server configuration?
You can post it. Just be sure to anonmize anything you don't want to share or to just show the extracts containing the setup of the cgi directory?
You might also try to rename our scripts to
a_sh.cgi and to a_py.cgi
I'm not reall convinced about this attempt, but some server configs insist on a .cgi suffix
A3 to A6 look all OK, no surprises there.
A2 is not reall helpful :-(
So it could be access right problems or a problem with the apache config.
perhaps the server cannot read / execute the script or they cgi scripts are not located in the right directory
Can you check the file permissions of our cgi scripts and of the directory containing it and of all its parent directories?
Has the user running the web server enough permissions to read and execute the cgi scripts?
Could you check the apache server configuration?
You can post it. Just be sure to anonmize anything you don't want to share or to just show the extracts containing the setup of the cgi directory?
You might also try to rename our scripts to
a_sh.cgi and to a_py.cgi
I'm not reall convinced about this attempt, but some server configs insist on a .cgi suffix
Hi gelonida,
Yeah, A2 has been really frustrating for me too. When I go ahead and install the missing 500.shtml file as I have on my other sites, all I get is a blank page and no error messages at all. This is why I got rid of the 500.shtml in the hope that I could at least capture some errors.
I tried the file rename, and per your suspicions it made no difference.
Permissions don't appear to be a problem. All the files and the cgi-bin directory are set to 755.
Your question regarding the users, "Has the user running the web server enough permissions to read and execute the cgi scripts?" might be a key to this problem, however.
Let me ask you (because I seriously don't know): Does the fact that the entire parent directory and all its subfolders are under one user affect the ability of the site to be seen? I don't think so, because the user in question sets up html files and they display just fine in the browser. All my sites are under a different user (apache), and they all display fine in the browser, however all of my sites are either html or php. I have no python or cgi sites. Am I wrong?
The only thing I see related to the /cgi-bin/ directory in the httpd.conf file is the following:
Do you see an issue here?
Yeah, A2 has been really frustrating for me too. When I go ahead and install the missing 500.shtml file as I have on my other sites, all I get is a blank page and no error messages at all. This is why I got rid of the 500.shtml in the hope that I could at least capture some errors.
I tried the file rename, and per your suspicions it made no difference.
Permissions don't appear to be a problem. All the files and the cgi-bin directory are set to 755.
Your question regarding the users, "Has the user running the web server enough permissions to read and execute the cgi scripts?" might be a key to this problem, however.
Let me ask you (because I seriously don't know): Does the fact that the entire parent directory and all its subfolders are under one user affect the ability of the site to be seen? I don't think so, because the user in question sets up html files and they display just fine in the browser. All my sites are under a different user (apache), and they all display fine in the browser, however all of my sites are either html or php. I have no python or cgi sites. Am I wrong?
The only thing I see related to the /cgi-bin/ directory in the httpd.conf file is the following:
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin/"
</IfModule>
<Directory "/usr/local/apache/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
Do you see an issue here?
* My bad, the search wasn't done looking. There is also a valid ScriptAlias /cgi-bin/ for the directory where the scripts in question are located.
Not sure I understood correctly. Do you have two script aliases for /cgi-bin/ ?
This should not be the case and will not work.
Aliases are not like a search path. Only one of the aliases will be taken into account. ( I don't know whether the first or the last entry)
So first you should remove the ambiguity from your config file and make sure, that aliases / script aliases are unique.
You can do this be removing one of the script alias or by renaming one of them.
Example:
rename yours to
ScriptAlias /my-cgi-bin/ "/path/to/my/cgi-scripts/"
and then change the url to http://mysite.com/my-cgi-bin/a.py"
Of course you can also (after having uniqified your aliases) copy your cgi-script to /usr/local/apache/cgi-bin/
Concerning file ownerships.
if a user tries to access any file it has to start accessing from the root directory.
So the parent directories have to be at least executable (and normally also readably) for the user apache (proably meaning read/execute permissions for others)
This should not be the case and will not work.
Aliases are not like a search path. Only one of the aliases will be taken into account. ( I don't know whether the first or the last entry)
So first you should remove the ambiguity from your config file and make sure, that aliases / script aliases are unique.
You can do this be removing one of the script alias or by renaming one of them.
Example:
rename yours to
ScriptAlias /my-cgi-bin/ "/path/to/my/cgi-scripts/"
and then change the url to http://mysite.com/my-cgi-bin/a.py"
Of course you can also (after having uniqified your aliases) copy your cgi-script to /usr/local/apache/cgi-bin/
Concerning file ownerships.
if a user tries to access any file it has to start accessing from the root directory.
So the parent directories have to be at least executable (and normally also readably) for the user apache (proably meaning read/execute permissions for others)
Hi gelonida!
Sorry for the delay in getting back to you. There was no way I could respond today.
My httpd.conf file is super lengthy, which is why I am hesitant to post it here. I did revisit the /cgi-bin/ script aliases and it appears each one is enveloped within a <VirtualHost></VirtualHost
Here is the VirtualHost that affects the directory in question (changed for security):
Now, one thing I can note is the ownership of /home/domain/public_html/s
Sorry for the delay in getting back to you. There was no way I could respond today.
My httpd.conf file is super lengthy, which is why I am hesitant to post it here. I did revisit the /cgi-bin/ script aliases and it appears each one is enveloped within a <VirtualHost></VirtualHost
Here is the VirtualHost that affects the directory in question (changed for security):
<VirtualHost xx.xx.xx.xx:80>
ServerName somename.domain.com
ServerAlias www.somename.com www.somename.domain.com somename.com
DocumentRoot /home/domain/public_html/somename
ServerAdmin webmaster@somename.domain.com
UseCanonicalName Off
CustomLog /usr/local/apache/domlogs/somename.domain.com combined
CustomLog /usr/local/apache/domlogs/somename.domain.com-bytes_log "%{%s}t %I .\n%{%s}t %O ."
## User domain # Needed for Cpanel::ApacheConf
UserDir enabled domain
<IfModule mod_suphp.c>
suPHP_UserGroup domain domain
</IfModule>
<IfModule !mod_disable_suexec.c>
<IfModule !mod_ruid2.c>
SuexecUserGroup domain domain
</IfModule>
</IfModule>
<IfModule mod_ruid2.c>
RUidGid domain domain
</IfModule>
ScriptAlias /cgi-bin/ /home/domain/public_html/somename/cgi-bin/
</VirtualHost>
Now, one thing I can note is the ownership of /home/domain/public_html/s
You;re right.
one script alias per virtual host is of course OK. only muultiple ones per virtual host would cause problems.
I assume, that you have the proof, that this virtrualhost section is really used, because you can access the non-cgi contents without problems, right?
To be sure. that there's no ownership problem I propose to post the output of following command
and also
one script alias per virtual host is of course OK. only muultiple ones per virtual host would cause problems.
I assume, that you have the proof, that this virtrualhost section is really used, because you can access the non-cgi contents without problems, right?
To be sure. that there's no ownership problem I propose to post the output of following command
ls -lsd / /home/ /home/domain/ /home/domain/public_html/ /home/domain/public_html/somename/ /home/domain/public_html/somename/cgi-bin/
and also
ls -ld /home/domain/public_html/somename/cgi-bin/a.*
Thanks gelonida, I really appreciate the help.
Yes, we can access html files through the browser with no problem, and these are files saved with someotheruser as the owner.
Here is what you requested:
I assume, that you have the proof, that this virtrualhost section is really used, because you can access the non-cgi contents without problems, right?
Yes, we can access html files through the browser with no problem, and these are files saved with someotheruser as the owner.
Here is what you requested:
# ls -lsd / /home/ /home/domain/ /home/domain/public_html/ /home/domain/public_html/somename/ /home/domain/public_html/somename/cgi-bin/
4 dr-xr-xr-x. 31 root root 4096 May 9 17:21 /
4 drwx--x--x. 14 root root 4096 May 7 01:32 /home/
4 drwx--x--x 21 domain nobody 4096 May 13 01:02 /home/domain/
4 drwxr-xr-x 14 domain domain 4096 May 12 18:29 /home/domain/public_html/
4 drwxr-xr-x 17 someotheruser someotheruser 4096 May 8 23:02 /home/domain/public_html/somename/
4 drwxr-xr-x 2 someotheruser someotheruser 4096 May 9 16:50 /home/domain/public_html/somename/cgi-bin/
# ls -ld /home/domain/public_html/somename/cgi-bin/*.*
-rwxr-xr-x 1 root root 62 May 9 15:47 /home/domain/public_html/somename/cgi-bin/a_sh.cgi
-rwxr-xr-x 1 someotheruser someotheruser 34819 May 8 22:25 /home/domain/public_html/somename/cgi-bin/cgi.py
-rw-r--r-- 1 root root 34967 May 8 22:30 /home/domain/public_html/somename/cgi-bin/cgi.pyc
-rwxr-xr-x 1 someotheruser someotheruser 123 May 8 22:30 /home/domain/public_html/somename/cgi-bin/a.py
#
Hmmm, I'm slowly running out of ideas.
The top level permissions are quite restricted (user apache can not read the contents of /home or of /home/domain, but the x permission is set so it can enter the directory
this should normally cause no problem.
But you can try to make access a little more permissive (perhaps apache, python or bash don't like it.)
Try to call next command as root (for testing):
Then restart your apache server and try to access /cgi-bin/a.cgi
What I am a little confused about is, that the cgi-bin directory is a sub directory of the Documentroot.
In my opinion this should not cause problems, but I never setup my web projects like this.
I had the cgi directory in a director parallel to he document root.
Perhaps you can try to change
the script alias to
You should also add following in your virtualhost section
The top level permissions are quite restricted (user apache can not read the contents of /home or of /home/domain, but the x permission is set so it can enter the directory
this should normally cause no problem.
But you can try to make access a little more permissive (perhaps apache, python or bash don't like it.)
Try to call next command as root (for testing):
chmod a+rx /home /home/domain
Then restart your apache server and try to access /cgi-bin/a.cgi
What I am a little confused about is, that the cgi-bin directory is a sub directory of the Documentroot.
In my opinion this should not cause problems, but I never setup my web projects like this.
I had the cgi directory in a director parallel to he document root.
Perhaps you can try to change
the script alias to
ScriptAlias /cgi-bin/ /home/domain/public_html/somename_cgi-bin/
mv /home/domain/public_html/somename/cgi-bin/ /home/domain/public_html/somename_cgi-bin/
You should also add following in your virtualhost section
<Directory "/home/domain/public_html/somename_cgi-bin/">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
Hi gelonida,
Thanks for getting back to me so quick.
Before I do that, I just want to make sure I can get the permissions back to where they were in case the change has no effect.
On the listing, there is something I do not understand. Did you see how the /home/ directory permissions were listed:
Did you notice the dot (.) after the permissions? That doesn't show up on all the directories. What does that mean?
Thanks for getting back to me so quick.
Before I do that, I just want to make sure I can get the permissions back to where they were in case the change has no effect.
On the listing, there is something I do not understand. Did you see how the /home/ directory permissions were listed:
4 drwx--x--x. 14 root root 4096 May 7 01:32 /home/
Did you notice the dot (.) after the permissions? That doesn't show up on all the directories. What does that mean?
chmod go-r /home /home/domain
chmod a+rx /home /home/domain
chmod go+r /home /home/domain
No idea about the trailing dot, but the chmod commands, that I suggested explicitely just change single permissions and leave the rest intact
Hi gelonida,
Thanks again for all your help.
I tried making the change. The permissions on /home and /home/domain were changed successfully to become executable, but that made no difference displaying the python or cgi files in the browser.
Your last statement
Do you think it's time I ask for more help? Maybe get some server specialists in here?
Thanks again for all your help.
I tried making the change. The permissions on /home and /home/domain were changed successfully to become executable, but that made no difference displaying the python or cgi files in the browser.
Your last statement
chmod a+rx /home /home/domain
Do you think it's time I ask for more help? Maybe get some server specialists in here?
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
Whoa, I think you may have stumbled onto something!
I copied the files from /home/domain/public_html/s
The directory /usr/local/apache/cgi-bin/
Do you think I should just open another question?
I copied the files from /home/domain/public_html/s
The directory /usr/local/apache/cgi-bin/
<Directory "/">
Options ExecCGI IncludesNOEXEC Indexes SymLinksIfOwnerMatch
AllowOverride All
</Directory>
<Directory "/usr/local/apache/htdocs">
Options Includes Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>
<Files ~ "^error_log$">
Order allow,deny
Deny from all
Satisfy All
</Files>
<FilesMatch "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</FilesMatch>
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog "logs/access_log" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin/"
</IfModule>
<Directory "/usr/local/apache/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
<IfModule mod_log_config.c>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog logs/access_log common
</IfModule>
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule>
Do you think I should just open another question?
Thanks gelonida for sticking in there in spite of the fact you were running out of ideas. Your question about being able to run cgi on a different virtualserver sparked me to do some invesigation and found out this was a CPanel issue. I needed to create a new account through WHM which would create a separate public_html folder for just that user. Once that folder was set up, and ownership and permissions of the files and directories were normalized to the new user, everything worked like a charm.
Thanks again, and if I have any more python questions, I will probably post to this question to see if I can get your attention.
Best Regards,
Jason
Thanks again, and if I have any more python questions, I will probably post to this question to see if I can get your attention.
Best Regards,
Jason
Join our community and discover your potential
Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.
*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.