HTTP Keep-Alive Sample

I've heared, it's possible - to open connection to some server, and get more than one URL without reopening socket for every HTTP transaction, something like
s = connect(... //open connection olny once
ssprintf(cBuf,... "...GET file1.html \r\n Connection: Keep-Alive...
read(s ... //get first page
ssprintf(cBuf,... "...GET file2.html \r\n Connection: Keep-Alive
read(s ... //get second page on the same socket

close(s); //close it after all
Does anybody have small working sample (flat C) ?
Thanks in advance.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I've never seen the "keep-alive" stuff (the http subtleties matter there), so will need collaboration from some other expert on that point (EE stuff is already supporting some sort of point sharing among experts, when more than one of them helps providing an answer).

If that side of the problem is solvable, i could give you a significant help with C code and socket connection handling.

I'll stay in touch...
This is an HTTP 1.1 capability.  Here are a couple of quote from the spec at Negotiation
   An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to
   maintain a persistent connection unless a Connection header including
   the connection-token "close" was sent in the request. If the server
   chooses to close the connection immediately after sending the
   response, it SHOULD send a Connection header including the
   connection-token close.
   An HTTP/1.1 client MAY expect a connection to remain open, but would
   decide to keep it open based on whether the response from a server
   contains a Connection header with the connection-token close. In case
   the client does not want to maintain a connection for more than that
   request, it SHOULD send a Connection header including the
   connection-token close.
   If either the client or the server sends the close token in the
   Connection header, that request becomes the last one for the   connection.
   Clients and servers SHOULD NOT assume that a persistent connection is
   maintained for HTTP versions less than 1.1 unless it is explicitly
   signaled. See section 19.7.1 for more information on backwards
   compatibility with HTTP/1.0 clients.
   In order to remain persistent, all messages on the connection must
   have a self-defined message length (i.e., one not defined by closure
   of the connection), as described in section 4.4.

19.7.1 Compatibility with HTTP/1.0 Persistent Connections
   Some clients and servers may wish to be compatible with some previous
   implementations of persistent connections in HTTP/1.0 clients and
   servers. Persistent connections in HTTP/1.0 must be explicitly
   negotiated as they are not the default behavior. HTTP/1.0
   experimental implementations of persistent connections are faulty,
   and the new facilities in HTTP/1.1 are designed to rectify these
   problems. The problem was that some existing 1.0 clients may be
   sending Keep-Alive to a proxy server that doesn't understand
   Connection, which would then erroneously forward it to the next
   inbound server, which would establish the Keep-Alive connection and
   result in a hung HTTP/1.0 proxy waiting for the close on the
   response. The result is that HTTP/1.0 clients must be prevented from
   using Keep-Alive when talking to proxies.
   However, talking to proxies is the most important use of persistent
   connections, so that prohibition is clearly unacceptable. Therefore,
   we need some other mechanism for indicating a persistent connection
   is desired, which is safe to use even when talking to an old proxy
   that ignores Connection. Persistent connections are the default for
   HTTP/1.1 messages; we introduce a new keyword (Connection: close) for
   declaring non-persistence.
   The following describes the original HTTP/1.0 form of persistent
   When it connects to an origin server, an HTTP client MAY send the
   Keep-Alive connection-token in addition to the Persist connection-   token:
          Connection: Keep-Alive
   An HTTP/1.0 server would then respond with the Keep-Alive connection
   token and the client may proceed with an HTTP/1.0 (or Keep-Alive)
   persistent connection.
   An HTTP/1.1 server may also establish persistent connections with
   HTTP/1.0 clients upon receipt of a Keep-Alive connection token.
   However, a persistent connection with an HTTP/1.0 client cannot make
   use of the chunked transfer-coding, and therefore MUST use a
   Content-Length for marking the ending boundary of each message.
   A client MUST NOT send the Keep-Alive connection token to a proxy
   server as HTTP/1.0 proxy servers do not obey the rules of HTTP/1.1
   for parsing the Connection header field. The Keep-Alive Header
   When the Keep-Alive connection-token has been transmitted with a
   request or a response, a Keep-Alive header field MAY also be
   included. The Keep-Alive header field takes the following form:
          Keep-Alive-header = "Keep-Alive" ":" 0# keepalive-param
          keepalive-param = param-name "=" value
   The Keep-Alive header itself is optional, and is used only if a
   parameter is being sent. HTTP/1.1 does not define any parameters.
   If the Keep-Alive header is sent, the corresponding connection token
   MUST be transmitted. The Keep-Alive header MUST be ignored if
   received without the connection token.

So the bottom line is that the code you show above would work if the server supports this capability.  A HTTP 1.0 server MAY support it but was not required to.  An HTTP 1.1 server MUST support it but doesn't have to HONOR the request.  So you should be prepared to handle any eventuality in your request code.
Become a Certified Penetration Testing Engineer

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

trofAuthor Commented:
Thanks, if I'm not mistaken, it's RFC#... Sorry, but I already read this(or similar) text, and trying to follow this instruction, but my code does not work on _any_ server, even on server, answered HTTP/1.1 in header. That's a reason, why I need _small_ _working_ _example_ . Thanks again.
is it correct if i assume you're going to care yourself of the http protocol itself, and just give you some C code to connect to a web server through sockets?
trofAuthor Commented:
ok, now I have code mmm...something like this
int GetURL(char *cURL, char*Buffer)l
It opens socket, sends "HTTP/ ...GET..." to server, receives Page of given URL and closes socket. If I need next page from the same server, connection must be reopened again, and of cousre, it's time consuming..
Examle I need: Let's say, three functions
#define SOCKET int
1. SOCKET OpenHTTPConnection(char *cServer);
opens socket to talk to given server and returns it.. Probably, says to server some magic words..
2 int GetHTTPPage(SOCKET s, char *cURL,  char *cBuf)
sends "HTTP ..." to server on socket s, receives page, connection is still opened, so, I can use this function as many times as I wish, without reopening socket.
3 void CloseHTTPConnection(SOCKET s)
 close(s); //that's all

I have tried it with an http 1.1 server and it works.  However, the server does close the connection if the 2nd request comes too later after the 1st one (about 8 seconds on my server).  And of course, it has to be an http 1.1 server.

One thing to note, http 1.1 requires that there must be a header:  In your code I did not see it, maybe this is where you fail.

HOST: XXXX.XXX.XX (the server)

Another reason why you fail is that you go through a (or some) proxy server, however, one or some of the proxy along the way does not support http 1.1.  Besides, if you does go through the proxy server, then you should use the following:

Proxy-Connection: Keep-Alive

Instead of the normal:

Connection: Keep-Alive

To supply you the code is of no problem, but since you can already get response from the server, I think you don't really need it.  All you need to is to add the HOST header and
Proxy-Connection: Keep-Alive header if proxy is used.  And you should only test with http 1.1 servers.

Finally, to ask the server close the socket, use Connection: Close

I have some typo in my answer, so to make it more clear:

If you directly access the http 1.1 server:

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost:\r\nConnection: Keep-Alive\r\n\r\n");

(there should be no space before the header and you must use HTTP/1.1 as the version)

If you go through a proxy server (which supports 1.1)

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost:\r\nProxy-Connection: Keep-Alive\r\n\r\n");
trofAuthor Commented:
Yes, I know that stuff about "HOST: ..", etc. - I have read RFC about HTTP/1.1 . By the way - "Connection: Keep-Alive" does work on HTTP/1.0 as well, you just have to check returned http header... The problem is - if, for examle I open commection to, it allows "alive" connection with big pleasure,and I cam read a page by
write(s,...) // request for page
while(res != 0)
  res = read(s, ...)
 but when I read next page,
read(s, ...)
returns 0 immediatelly.
Probably, I should say some magic words to socket to reset it or..whatever.
So, I still looking for _working_ _small_ sample of C-code..

I suggest you try it on a local server first.

I'd like to know whether you connect directly with the http server or through a proxy server?  Even the server reply that the connection is alive, actually it may already be closed.  That is the case I observe, I guess due to the proxy server in the way.

HTTP 1.0 does not define a Connection header, there may be some 1.0 client and server using it, but that is the standard.

I will soon give you a sample code that works on my server (on the lan).
I know your problem now.

Your code

while(res != 0)
        res = read(s, ...)

is not correct.  That is for http 1.0, because it will return until the other side close the socket.  So instead of wait until the result is 0, you should parse the response and get content-length, using this value you can know which response is for which request.

Below is my sample code.

#include <winsock.h>

void main()
      WSADATA wsadata;
      SOCKET s;
      SOCKADDR_IN hSockAddr;
      char buf1[300], buf2[40960];
      int r;

      WSAStartup(MAKEWORD(1, 1), &wsadata);
      s = socket(PF_INET, SOCK_STREAM, 0);

    hSockAddr.sin_family      = AF_INET;
    hSockAddr.sin_addr.s_addr = 0;
    hSockAddr.sin_port        = 0;

      bind(s,(struct sockaddr *)&hSockAddr,sizeof(hSockAddr));


      connect(s,(struct sockaddr *) &hSockAddr, sizeof(SOCKADDR_IN));

      wsprintf(buf1, "GET /feedback/ HTTP/1.1\r\nHost: buxley\r\nConnection: Keep-Alive\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      wsprintf(buf1, "GET /search/ HTTP/1.1\r\nHost: buxley\r\nConnection: Close\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      do      {
            r = recv(s, buf2, 40960, 0);
      } while (r);



In my code I send out two request at the same time, and then the following read will get both html.  I omit all error checking here.
This is from the document on recv()

If the function succeeds, recv returns the number of bytes received. If the connection has been closed, it returns zero

So you are actually waiting for the server to time out.
trofAuthor Commented:
does not work on Received HTTP header contains "Connection: keep-alive", but try to get the second file, recv=0, and connection is dead... :( What I'm doing wrong ? I did not change functional part of your sample..
You should give me more detail about how it fails.  "try to get the second file, recv=0", in my code, all the files goes into one buffer.  There is no special recv() for the 2nd file!  In other word, it is the upper layer (your app) to tell which is the first file and which is the 2nd from the data received by recv().  If you repear my code:

do {
     r = recv(s, buf2, 40960, 0);
} while (r);
do {
     r = recv(s, buf2, 40960, 0);
} while (r);

trying to get one file with a loop, then it is wrong.

Maybe you should paste all your code, so I can tell you where is wrong.

Basically, did you send out the two request before trying to recv()?  If you do so (like what I do in my code), the two response will all go to buf2.  You have to parse the buffer to know which part is for which.

If you call recv() after sending out ONE request, then you can not wait for the return code to become 0, instead, you should parse the response (or use a time out) to determine that it is terminated and then send out the 2nd request.

You should connect to microsoft WITHOUT a proxy server, since the proxy server may close the socket.

Finally, are you sure the microsoft server is using 1.1?  Check its response, is it using HTTP 1.1 or 1.0?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Scripting Languages

From novice to tech pro — start learning today.