Solved

Send session cookie from client to server (python cgi) to fix apache CSRF errors

Posted on 2013-11-04
17
1,367 Views
Last Modified: 2013-11-15
My python cgi script takes form values submitted by a user and prints them to html. I need to protect this from CSRF attacks but my server must request that clients generate and send their session ID information / cookie when the form is submitted (I think) and I'm stumped. The script:

form_list = ['Local_Time','Your_Name', 'Fav_Color', 'Other_Color', 'Submit1', 'Submit2', 'Reset']

colors_dict = {'Local_Time':{'Local Time':''},\
              'Your_Name':{'Your Name':''},\
              'Fav_Color':{'Favorite Color':''},\
              'Other_Color':{'Another Color':''} }

def write_html(colors_dict):

    for item in form_list:
       for fn in colors_dict:
           if fn == item:
              ...print html form data

def read_form(**kw):   

    fields = kw.get('fields', {})
    
    for field in form_list:
        if isinstance(form.getvalue(field), list):
            fields[field] = form.getfirst(field)
        else:
            fields[field] = form.getvalue(field)

        if form.getvalue(field) == None:
            fields[field] = ''
            
    #store the values 
    for field in fields:
        value = fields[field]
        
        for fn in colors_dict:
            if fn == field:
                colors_dict[fn] = value   
    
    #print dictionary values to html form 
    write_html(colors_dict)

if __name__ == '__main__':

    print "Content-Type: text/html; charset=utf-8 \n"

    if 'Submit1' in form:
       read_form(fields=fields)  
       ...use template substitution to load an html page

    if 'Submit2' in form:
       read_form(fields=fields)
       ... use template substitution to load another html page

    else:
       read_form(fields=fields)
       ...load the start page, which contains:
          <form action="my_script.cgi" method="post">

Open in new window


Modsecurity logs CSRF errors in my apache error log and to fix this, I think the client must send a session ID token or cookie to the server to guard my server from sending replies to the wrong/malicious clients. I found this:

URL = 'https://path/to/my_script.cgi'
client = requests.session(config={'verbose': sys.stderr})
my_cookie = client.get(URL)
r = client.post(URL, headers=dict(Referer=URL))

Open in new window


When I run this in a terminal window, the output is:
2013-11-04T20:19:12.551841   GET   https://path/to/my_script.cgi
2013-11-04T20:19:12.890499   POST  https://path/to/my_script.cgi

But my apache error log shows errors:
[Mon Nov 04 16:34:40 2013] [client 127.0.0.1] 2013-11-04T15:34:40.736838   GET   https://path/to/my_script.cgi
[Mon Nov 04 16:34:41 2013] [client 127.0.0.1] 2013-11-04T15:34:41.016961   POST  https://path/to/my_script.cgi
and the script hangs in a browser window.

I can generate a cookie on the server with:

string_cookie = os.environ.get('HTTP_COOKIE') 
   
now_utc = datetime.utcnow().replace(tzinfo=pytz.utc) 
str_utc = str(now_utc)
localtime = datetime.now().strftime("%Y-%m-%d %H:%M:%S")

if not string_cookie:
   sid = sha.new(repr(localtime)).hexdigest()
   
else:
   cookie.load(string_cookie)
   sid = string_cookie

Open in new window


I can import the sid to my_script.cgi, write the sid(s) to file and run string comparisons on submit transactions but this does not solve the CSRF problem because the errors persist.  

A hidden form name/value may be the solution but if it is, the client must generate the token/cookie to populate the form value, and I don't understand how that would work.

TIA
0
Comment
Question by:sara_bellum
  • 10
  • 7
17 Comments
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39623617
That is confusing.  The way cookies work is pretty simple.  When a page request is made by a browser, it sends with that request all the cookies that it has for that domain.  When the response is made by the program on the server, it can send the cookies to the browser.  Those cookies can then be read on the Next request from the browser.  So a new cookie can not be read until the next page request from the browser.

Also, cookies can Not be set or sent from a form.  They have to be set and sent as cookies, not as query strings or POST data.  While there are a couple of programs like 'curl' that can set and send cookies, cookies normally come from a web browser because web browsers store the cookies on the client.  A command line request will not have a 'cookie store' to fetch and send cookies from so it is difficult, maybe impossible to use cookies with a command line request.
0
 

Author Comment

by:sara_bellum
ID: 39625340
Ok this is progress, thanks.  But if this automatic process is handled correctly, then I should not be seeing CSRF errors in my apache logs. Printing a cookie string was my first step in the debugging process - if apache is reading cookies from clients requesting this form, these cookies are not set the way they should be.  Let me know how to fix this.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39625465
The cookies do have to be set first.  Cookies (that have been set) are included as part of the HTTP request.  You can see them, if they are there, by using a network monitor like Fiddler in your browser or Wireshark on your server.

This article seems to explain CSRF http://www.linuxforu.com/2010/11/securing-apache-part-3-xsrf-csrf/ but I'm not understanding all of it.  You did mention command line.  To create a 'legitimate' cookie, you have to load a page from a domain so that domain can set it's cookie.  

If you are trying to POST from a command line program without loading a page and getting a cookie, that doesn't have a domain associated with it so I can see where it would cause a CSRF error.
0
 

Author Comment

by:sara_bellum
ID: 39625636
Thanks Dave, this was helpful.  But like much of the documentation I've found on the subject, the focus is on the CSRF threat itself and not the fix. The recommended fix here is for the server to use unpredictable tokens to validate requests.  I know (I think) how to generate tokens, if the code above applies. The problem is that I don't know where to put them.  

Cookie: PHPSESSIONID=7757ADD8766d455NFJJ23875JBJKBFR from=35367021&to48412334&amount=5000&date=05072010&token=40E03EF45T443W20K4IC567HY4334DD44&timestamp=1184001456

was the example cited in the post.  I'd love to try that, but don't know how.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39625681
What do your "CSRF errors" say in the log?  And why do you have PHPSESSIONID cookie?

This page shows how to set cookies: http://docs.python.org/3.3/library/http.cookies.html?highlight=cookies#http.cookies
0
 

Author Comment

by:sara_bellum
ID: 39626332
log error:

[error] [client 192.168.1.6] ModSecurity: Warning. Match of "eq 1" against "&ARGS:CSRF_TOKEN" required. [file "/etc/modsecurity/activated_rules/modsecurity_crs_43_csrf_protection.conf"] [line "31"] [id "981143"] [msg "CSRF Attack Detected - Missing CSRF Token."] [hostname "hn.mydomain.com"] [uri "/path/to/my_script.cgi"] [unique_id "Unm@yMCoAQQAAA5xH-sAAAAF"]

Open in new window


I have no cookies that I can find - I copied the PHP session ID cookie from the website you recommended. I'll tag my questions more carefully next time - I'm using Ubuntu 12.04, whose default version of python is 2.7.3. I checked http://docs.python.org/2/library/cookielib.html,  copied the first example (the most common usage of cookielib) to a script  and ran wireshark on execution.  Nothing helpful was returned - the apache error (posted above) remains.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39626540
The error message above is not about cookies but about not seeing a CSRF token.  I don't know whether that 'unique_id is supposed to be the value or not.  You would have to read the docs for that function to see how to pass the CSRF token.
0
 

Author Comment

by:sara_bellum
ID: 39628762
The error message doesn't mention the cookie but unless I'm missing something you can't pass a CSRF token without one: the token is set inside the cookie and the cookie is the mechanism for passing the token from the client to the server.  

I quoted the PHP session ID earlier from the page you recommended because the method for setting the token should not differ too widely between programming languages. If we know how the cookies are generated, we can edit them to send tokens.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39628936
I think that last value in the error message is most likely the token that Apache is looking for.  There must be a method for getting and sending that value.  The docs for that function have to tell you how to do it.

I found this very long page about ModSecurity: https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#wiki-Introduction  The probablity is that you have to read the form page to get the cookie with the token, save it, and send it back with your POST info.  I would expect the token value to change with every request.
0
 

Author Comment

by:sara_bellum
ID: 39635105
The first link you posted was consistent with the CSRF errors in my apache logs, urging developers to use "unpredictable tokens, a piece of data that the server can use to validate the request which an attacker can’t guess." Nothing in the ModSecurity link you posted helps me figure out how to do that. I've installed ModSecurity and it's working as advertised, warning me of a security vulnerability.  The Internet is full of information on how hackers can break your site, but all of these pages assume that developers will know how to fix the problem once they see it.

There are python frameworks like Django and Flask that have built-in solutions to this problem, but a built-in solution implies that there's one that can be scripted for python cgi. Given how old the CSRF problem is, I'm having trouble understanding why there are no examples of CSRF safeguards using the python cgi module.

I doubt that sending the token via a form field will work: the cookie works in the same way as a token unless I'm missing something. In my server's cgi form, the 40-character cookie changes on every Submit transaction. (I've seen the same CSRF error when index.html pages are accessed but without a Submit button, it's harder to ascertain and compare cookie values.)  In short, if you put a dynamic server-generated value into a client html form, apache will generate an (sql) injection error and the client browser will  return a "Permission denied" page.
0
 
LVL 82

Accepted Solution

by:
Dave Baldwin earned 500 total points
ID: 39635189
From reading the docs, I'm pretty sure that the token is in a cookie.  That means you have to make a request for the form page.  In the response will be the cookie with the token.  You have to return that cookie with the token (along with all other cookies for that domain) when you make your post to the 'action' page.
0
 

Author Comment

by:sara_bellum
ID: 39636306
I tried submitting a random session id number via the form again - your comment as well as a post at http://www.wellho.net/mouth/3698_How-to-stop-forms-on-other-sites-submitting-to-your-scripts.html indicate this to be a workable solution. I had tried and failed earlier with an sql injection error but this time I was able to submit a random 40-character session id as a form value that changes on each submit action. Perhaps my earlier problem was due to a syntax error.  

That's encouraging but it's not the solution because the modsecurity error persists.  I followed advice at http://blog.spiderlabs.com/2011/01/detecting-malice-with-modsecurity-csrf-attacks.html and started a debug log, which tells me that my server could not set variable session.ip_hash and session.ua_hash as these collections do not exist. These hash collections are called up by  modsecurity_crs_16_session_hijacking.conf, an active ruleset.  Will keep on digging.
0
 

Author Comment

by:sara_bellum
ID: 39640358
More recent versions of modsecurity are built for later versions of apache than what's on my server (apache 2.2.22)
I removed modsecurity and installed an older version.  No errors are surfacing now no matter what I do, which makes me nervous.  But I'll go ahead and correct my demo server config to see if the modsecurity behavior is the same as on localhost.
0
 

Author Comment

by:sara_bellum
ID: 39646868
The solution:
- Enable the apache usertrack module
- Configure apache to use it (edit the domain file in sites-available)
- Restart apache
Then in your script you can print the cookie, which in my case contains the cookie name, the source IP and a 16-digit tracking number which does not change from one submit action to the next, given a) same source IP and b) a short transaction time; I'll have to research the default time available for the cookie to remain unchanged.  

This is not as secure as using the 40-character tracking number that apache randomly assigns to each transaction by default, but it's close enough given what I am trying to do (verify that the same user is submitting each cgi form transaction), and my version of Apache (2.2.22). I'll review this tomorrow and award points. Even though the solution was mine, I appreciate the help.
0
 

Author Comment

by:sara_bellum
ID: 39652333
Ok I take that back. Apache doesn't enable cookie tracking by default so I can't rely on the remote host having it enabled, tracking by IP is not very useful with dynamically assigned connections, and for other reasons.  

So the only way I can figure out how to do this with the programs I'm working with is to save the first session ID, assign it as a hidden form value, and on the final Submit action, read that value from the form to make sure that it corresponds to the first session ID assigned.  Then I can be reasonably sure that same user is sending data to my server in consecutive Submit transactions.
0
 

Author Closing Comment

by:sara_bellum
ID: 39652352
There was a two-fold problem: the compatibility of my version of modsecurity with my version of Apache, and finding a way to verify via Apache tokens that the same user was sending the data across consecutive form submit actions.  Thanks for your patience.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
ID: 39652418
You're welcome, glad to help and glad you got it figured out.
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Article by: Swadhin
Introduction of Lists in Python: There are six built-in types of sequences. Lists and tuples are the most common one. In this article we will see how to use Lists in python and how we can utilize it while doing our own program. In general we can al…
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now