hunting for what is changing a cookie on the *browser* (not on the server)

I'm writing a bot that mimics my browser, Chrome.

In Chrome, when I interact with the website I'm interested in, the website passes back a cookie "session" using the "Set-Cookie" header in the HTTP response. *However*, on the next HTTP request from the browser, the request headers have an entry "Cookie" with an *altered* value.

Please read my assumptions/guesses about what is happening, and comment if they are valid assumptions or not:

(1) the browser has changed the cookie's value (explanation: the server sent valueA in its http response, but now the browser is using valueB in its next http request)

If assumption (1) is true...
(2) What changed the cookie's value on the browser must be javascript, because only a script can change a cookie's value  on the browser, and the only script in this page is javascipt.

if assumption (2) is true...
(3) Chromes "Inspect Element > Network" tool shows me that 5 javascript files were part of the page request (out of a total of 28 files), so it must be one of these 5 that are changing the "session" cookie.

if assumption (3) is true...
(4) I should be able to find the code that changes the cookie value by searching for ".cookie=" (or similar with whitespace). (Explanation: http://www.w3schools.com/jsref/dom_obj_document.asp shows that the syntax for changing a cookie is document.cookie = "session=abc94g290" for example. Document may be assigned to a variable, so not safe to search for "document.cookie=")


Ok, following this logic, only two instances of ".cookie=" turn up.

one is in analytics.js
the other is in jquery-ui.min.js

from your knowledge of javascript libraries, is any of these likely to be the code changing my "session" cookie?

if not (as I suspect), then what is changing it??
zorba111Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

RobOwner (Aidellio)Commented:
The altered value is changed by javascript but identifying where it is would be near impossible.  If you're able to post a link to the site in question, we may have a better idea.  The reason the value is altered could be for a myriad of reasons, one of which is just to see that you're active on their site and that the session will be extended.

A session is basically a match up of a session id stored in a cookie on the client with the same session id on the server.  I don't believe this is a session id being altered but rather some kind of encrypted information.  If it was, you'd be logged out each time you requested a page.
Dave BaldwinFixer of ProblemsCommented:
Google's analytics.js would not be changing it, it would be tracking the page.  'jquery-ui.min.js' is a library and the routine that concerns cookies could be called by a different name.  I would do a search for just 'cookie' or even 'cooki' to see what turns up.
Dave BaldwinFixer of ProblemsCommented:
Thinking about it (a little), if I was trying to block your web bot, I might even load the JavaScript that changes the cookie thru an AJAX call so it would not be seen in the initial page source.  The Web Developer add-on for Firefox will let you view the 'generated' source and that might get more info than you have so far.  I don't know if the version for Chrome will let you do that or not.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

zorba111Author Commented:
Hi Dave, how do I look for these AJAX calls? cheers
**UPDATE: from my 5mins research I should be looking for creation of XMLHttpRequest objects. Is this the only way AJAX can be facilitated?
**UPDATE2: from another 15mins research, yes XMLHttpRequest objects are central to AJAX. And I can see how a file can be loaded into a JS variable (calls to open and send methods of the XMLHttpRequest ojbect). Next to research is how to execute this if its actually a file containing JS.... (instead of just text or XML)...
**UPDATE3: more research tells me that the JS can be loaded via XMLHttpRequest, presumably set to the innerHTML attribute of some DOM element, e.g. <div> or <p>. Then use JS's eval function. So I'm now looking for a chain of events like this happening anywhere in the loaded (or target document's) JS.
zorba111Author Commented:
It's just occurred to me that there are more HTTP responses per page download than just the response that contains the main page. There are 20-50 other sub-documents being pulled in, typically (e.g. out of 28 in the page I'm looking at now, 1 is HTML - the main page, 5 are JavaScript, 4 are CSS, 14 are images, and 4 are other). It could be that each/some of these are also changing the "session" cookie in their header. And if they are, the last  to arrive will be the one that perhaps sets the cookie to the value that the next request will use.... Sound plausible?

Or do only requests for HTML (or ASP or PHP) pages involve the server sending cookies...?

UPDATE: Using the Chrome tools (Inspect Element etc.), I was able to see all the headers (request and response) for each of the 28 files downloaded as part of this "page" download. As well as the main page, there are 3 other docs where the response sets cookies.

I can follow the progression through, and see where the cookie is being changed by the server, being passed back intact in the next response and being changed again, being passed back intact etc. etc.

So... its not a case of the browser changing the cookie after all. I was missing all the sub-page HTTP request/response cycles that can potentially change the cookie too :-)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Dave BaldwinFixer of ProblemsCommented:
Good research.  Yes, the response headers for any request can change any cookie that has been set by that site.  Sounds like they have gotten pretty serious about blocking web bots.
Ray PaseurCommented:
... its not a case of the browser changing the cookie after all.
That makers sense to me.  The server sets the cookies and the browser returns the cookies.  If the browser changes the value of the cookie, the server might not understand the new value.  Cookies are the Netscape 2.0 solution to the fact that HTTP is a stateless client/server protocol.  Response headers can add, change or remove any cookie at any time.  There are some things we do in PHP to try to reduce the risk that we don't know who we are dealing with.  This is one of those session-related tools.  It helps reduce the risk of session hijacking.
http://php.net/manual/en/function.session-regenerate-id.php
zorba111Author Commented:
I had made a false assumption in my initial understanding of the problem.

The other guys made some very valid points and suggestions that helped my understanding (or to confirm my understanding) and they would have helped had it not been for my initial bad assumption.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Web Browsers

From novice to tech pro — start learning today.