Community Pick: Many members of our community have endorsed this article.
Editor's Choice: This article has been selected by our editors as an exceptional contribution.

Understanding Client/Server Protocols and Web Applications

HyperText Transfer Protocol or "HTTP" is the underpinning of internet communication.  As a teacher of web development I have heard many questions, mostly from my younger students who have come to the WWW in the days after 2005 (and therefore after the advent of jQuery), that seem to evince a misunderstanding of the way the HTTP protocol works.  Since HTTP is the basic building block of web sites and web applications, a clear understanding of the protocol is required to understand how web sites really work.  And perhaps more importantly, it is necessary to understand the protocol so you can build a web site that works like another site you've seen and appreciated.  This article is intended to lift the cloud of confusion that has appeared in the years since the arrival of jQuery, CSS3, animation in web sites, and the phenomenon called Web 2.0.

The confusion has arisen in large measure because of the design movement of web interactivity away from the static page loads and into design paradigms that more closely mimic native applications.  If the hyperlink was the design element that launched the internet revolution, the event handler may be thought of as the design element that has most refined the client experience.  In all cases, at the foundation is the HTTP protocol.

HTTP is a Stateless Client/Server Protocol
Client/Server protocols are two-way communication mechanisms that allow humans to get information from web sites.  These same protocols also allow computers to talk to one another, share data, and react in real time to changes in their environment.  The starting point for any communication, and a basic building block of the protocol is a request.  If you want to delve deeply into this concept, see Roy Fielding's dissertation that coined the phrase REST.  The entire WWW is built on RESTful communications.

By definition, the client machine makes the request.  And the server machine makes the response.  Without the request, nothing happens on the internet.  The only thing that the server can do is make a response after it has received a request.  Otherwise the server sits idle, in stateless ignorance of the rest of the world-wide web.

When you grasp this concept, you've got a good understanding of one of the most essential building blocks of modern computing.

      Clients make Requests
      Servers make Responses

The only way that any internet communication can be initiated is with a request.  Servers cannot initiate communications with a client.  This is true, in part, because of security considerations.  Consider what would happen if a server could start some kind of application running on your home computer without your knowledge or permission.  Your family room could suddenly become a source of email spam or financial fraud.  The attacking server could turn your computer into a "zombie" that spread viruses to other computers.  Obviously this is unacceptable.  So servers by their very nature are limited to "servicing" the requests that have been received from their clients.  It follows that there is a predictable timeline: the client request happens first and the server response happens next.

The Nature of the Stateless Protocol
The client request to the server is often described as "atomic, complete and stateless."  The server response is described as "complete" and usually instantaneous.  Let's deconstruct these terms to show what the words really mean.

The "atomic" nature of a client request means that the request exists as a standalone data element in the world of the internet.  Many clients may make exactly the same request to a server.  But for the request to be the "same" each such request must send exactly the same information.  In chemistry an atom is made up from protons, neutrons and electrons, and cannot be subdivided; it is a single element.  In the HTTP protocol, the request is made up from a URL, headers (including HTTP cookies) and client-supplied data.  And like an element, a request cannot be subdivided.

The "complete" nature of a client request means that the request contains all of the external information that the server can use to respond to the request. The server cannot go back to the client to get any more information.  The request is a one-way communication.  Once sent, the client has nothing more to say to the server.  (Note that the server may use other non-request information to respond to the client, but this is not the same as getting additional information from the client.)

The "stateless" nature of a client request is subtle and a bit more complicated.  It means, in general terms, that the client and server know nothing of each other except what is presented in the request and returned in the response.  In the current environment of web interactivity, this truth may be obscured.  We all know that we can "log in" to a web site, and that the web site "remembers" our status.  Despite the appearance of memory, the fact of statelessness remains.  The appearance of stateful behavior is a stratagem, accomplished through the use of HTTP cookies and server-side data bases.

HTTP Headers Must Come First, Period
It is a law of HTTP that headers must come first and be complete before any browser output can be sent.  This rule includes even invisible white space characters.  The server is responsible for implementing this rule, and failure to comply has consequences that include the risk of lost data.  

In writing PHP scripts the programmer must be aware of this "headers-first" rule and deliberately avoid sending browser output at an inappropriate time.  Since HTTP cookies are part of the HTTP headers, it follows that no browser output can precede the use of setcookie().  What may not be completely obvious is that the PHP session_start() function implicitly calls setcookie().  When coupled with the fact that invisible white space can trigger browser output, this may become a point of confusion.  Consider the code snippet to see why.  This script will fail to start the session because the invisible white space that is the end-of-line character is in the browser output stream after line 1.
<?php error_reporting(E_ALL); ?>
                      <?php session_start();
                      // ETC

Open in new window

There are two PHP strategies for dealing with this.  The first and most obvious is to organize your script so that the program logic creates and finishes all the headers before the browser output is started.  The second is to use output buffering.  If the PHP script calls ob_start() before any browser output, PHP will buffer the output, holding it back from the output stream and thus allowing the script to intersperse the creation of headers and browser output.  So if we go back to our example above and add ob_start() to the script on line 1 our application will work correctly.  

If your PHP script generates a Warning message that says something to the effect of "cannot modify headers" the script has run afoul of the "headers-first" rule.  One of the solutions here, either refactor or buffer, will solve the problem.

Learn more here about HTTP headers.

Using HTTP Cookies and Server-Side Data Bases to Appear Stateful
In the normal course of creating a response, a server can send an HTTP cookie to the browser and the browser can return the cookie to the server on each subsequent request. This concept (and with it, the end of privacy on the internet) came about in 1994.  The typical HTTP cookie is a very small data file.  Usually it contains information that provides a key to a server-side repository of client information. Upon receiving a request that contains a cookie the server can look up the information in its repository, tailor its response to the client, and thus appear to have an understanding of the state of the client.  Used correctly, cookies allow us to become "logged in" to a web site, to have a "shopping cart" or "wish list" or to have a server "remember" our preferences.  Cookies enable the server to recognize its clients on second and subsequent visits.

Any client machine that sends an identical cookie will cause the server to locate the same repository of client information. (It is the server's responsibility to ensure that the cookies it receives are authentic; without such verification security breaches and forgeries are possible.)  And a client machine that does not send the cookie(s) will not be treated in the stateful way -- without the cookie, the server cannot locate the repository of client information and the server will simply not recognize the request as one in a series from a known client.

Knowledgeable computer scientists may recognize edge cases where these statements are not strictly true, but for the overwhelming majority of internet activity today (March 2013) the principles hold their ground, and artful server-side programming is required to bend the principles even a little bit.

With or without cookies, each HTTP request is a standalone, stateless event.  With cookies, the client and server can work together to create the appearance of server-side knowledge of the state of the client.

Developer Madness and Shared Cookie Jars
There is a phenomenon of modern browsers that is never a problem for clients, but that drives application developers insane, until they learn the nature of how browsers store cookies.  All instances of the same browser share the same "cookie jar" file.  This means that if you have a browser running in different tabs or even in different windows every page load will send the same set of cookies with the HTTP request.  

Consider this scenario.  The developer opens a browser window and visits his test site under the identity of User1.  He logs in and adds some products to his shopping cart.  Then, to test alternate functionality, the developer opens another browser window and logs in as User2.  Satisfied that things look right, he closes window number two and goes back to window number one, and to his horror, everything is wrong.  He's not user User1 any more.  The site seems to think he is User2 now, but he has User1 items in his shopping cart.  The site appears to be very badly broken and the developer begins tearing his hair out.  Actually, it's working correctly.  Here is the timeline of what just happened,

1. Client GET request to test web site receives a Session cookie from the server
2. Client POST request to log in as User1 sends the Session cookie back to the site
3. Server authenticates the log in and creates the server-side session data for User1
4. Client POST request to add product to cart receives a Cart cookie from the server
5. At this point, Client has two cookies and opens window number two
6. Client GET request sends both Session and Cart cookies
7. Client POST request to log in as User2 sends both Session and Cart cookies
8. Server authenticates the log in, and changes the server-side session data to User2
9. The Session cookie now points to server data for User2, but the Cart cookie is intact
10. Client closes window number two.
11. Using window number one, client makes a request to the server sending the existing cookies.  The Session cookie points to the server-side data for User2.  The Cart cookie points to server-side data for User1.  The developer is stupefied.

The single cookie jar never causes confusion in the "real world" because clients do not bounce back and forth between different identities.  They visit web sites but don't do the kinds of fiddly things that developers do when testing.  And developers, if they are knowledgeable of browser and cookie behavior, will know to use different machines (or at least different browsers) when testing different client behaviors!

Should the server have deleted the Cart cookie at the time of User2 log in?  Maybe.  But a better strategy might be a single and idempotent cookie to identify the user.  The server can keep track of lots of information associated with a single cookie.  And the risk of confusion is minimized.

The Timeline of Request and Response
The client request precedes the server response and is completed before the server can begin to formulate its response.  The server response is completed before the client's browser output begins.  And while there are edge cases that may appear to bend this timeline slightly, the true nature of the HTTP protocol remains unchanged.  

When a client requests a web page, the server creates the web page using program logic and data that is stored on the server (example: PHP and MySQL data bases). This information is packaged into the client-side information that takes the form of HTML, CSS, JavaScript and other content.  Once packaged, the server sends the response and disconnects from the network.  It follows that the server, having sent the response and disconnected, can no longer modify any part of the response.  This is why server-side PHP scripts cannot react to client actions until another request is received.  This is also why it is impossible for the server to know how long a client has been "on the page."

Using JQuery/AJAX Requests to Build Good Client User-Experience
Even though your server cannot talk to the client more than once per request, you can build the appearance of client-server interactivity through AJAX requests.  AJAX enables the client machine to send the HTTP request and receive the server response without reloading the entire web page.  Typically only a portion of the data in the browser viewport will be replaced.  This technology relies on JavaScript to affect the request and receive the response.  A Javascript framework, usually jQuery, is employed to ease the client-side programming. There is an article here at EE demonstrating the "hello world" exercise with jQuery and AJAX.

JavaScript can react to a variety of environmental changes on the client machine.  These changes are associated with "event handlers" that get triggered by actions such as page loads, mouse movement, keystrokes, etc.  One example of an event handler is the onKeyUp handler that captures the moment at which a previously pressed key is released.  Using this event handler, JavaScript can grab the contents of a form input control and initiate an AJAX request to the server. The server can take the information from the request, perform a data base lookup and respond to the client machine.  This design allows an auto-complete or suggestion to appear in the client viewport.  

JavaScript can set timers, and event handlers can respond to the expiration of timers by making AJAX requests to the server and replacing parts of the viewport.  Your JavaScript could, for example, initiate a request to the server every second.  When something interesting is found on the server, a response can be made to indicate the interesting thing to the client.  This activity can produce the illusion that the "server is calling the client" with things like the announcement of new email messages, or a popup notice about a special discount.

Even though "Web 2.0" applications may appear to contain server-initiated communication, that is not the case.  Client/server protocols and principles still apply, even if the human client cannot see them in action.  And while a complete page load need not be part of the server's response, the response is only made after receiving a request from the client machine.

Please give us your feedback!
If you found this article helpful, please click the "thumb's up" button below. Doing so lets the E-E community know what is valuable for E-E members and helps provide direction for future articles.  If you have questions or comments, please add them.  Thanks!

Comments (1)



A typically outstanding article. I've been fumbling through telling people this stuff for a long time, so it's nice to have a clear resource to which to send them.



Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.