Solved

HTTP Keep-Alive Sample

Posted on 1998-03-09
14
567 Views
Last Modified: 2013-12-25
I've heared, it's possible - to open connection to some server, and get more than one URL without reopening socket for every HTTP transaction, something like
s = connect(... //open connection olny once
ssprintf(cBuf,... "...GET file1.html \r\n Connection: Keep-Alive...
write(s,cBuf
read(s ... //get first page
ssprintf(cBuf,... "...GET file2.html \r\n Connection: Keep-Alive
write(s,cBuf..
read(s ... //get second page on the same socket

....
close(s); //close it after all
Does anybody have small working sample (flat C) ?
Thanks in advance.
0
Comment
Question by:trof
  • 6
  • 4
  • 2
  • +1
14 Comments
 
LVL 5

Expert Comment

by:julio011597
ID: 1832131
I've never seen the "keep-alive" stuff (the http subtleties matter there), so will need collaboration from some other expert on that point (EE stuff is already supporting some sort of point sharing among experts, when more than one of them helps providing an answer).

If that side of the problem is solvable, i could give you a significant help with C code and socket connection handling.

I'll stay in touch...
0
 
LVL 32

Expert Comment

by:jhance
ID: 1832132
This is an HTTP 1.1 capability.  Here are a couple of quote from the spec at http://www.w3.org/Protocols/rfc2068/rfc2068:

8.1.2.1 Negotiation
   An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to
   maintain a persistent connection unless a Connection header including
   the connection-token "close" was sent in the request. If the server
   chooses to close the connection immediately after sending the
   response, it SHOULD send a Connection header including the
   connection-token close.
   An HTTP/1.1 client MAY expect a connection to remain open, but would
   decide to keep it open based on whether the response from a server
   contains a Connection header with the connection-token close. In case
   the client does not want to maintain a connection for more than that
   request, it SHOULD send a Connection header including the
   connection-token close.
   If either the client or the server sends the close token in the
   Connection header, that request becomes the last one for the   connection.
   Clients and servers SHOULD NOT assume that a persistent connection is
   maintained for HTTP versions less than 1.1 unless it is explicitly
   signaled. See section 19.7.1 for more information on backwards
   compatibility with HTTP/1.0 clients.
   In order to remain persistent, all messages on the connection must
   have a self-defined message length (i.e., one not defined by closure
   of the connection), as described in section 4.4.



19.7.1 Compatibility with HTTP/1.0 Persistent Connections
   Some clients and servers may wish to be compatible with some previous
   implementations of persistent connections in HTTP/1.0 clients and
   servers. Persistent connections in HTTP/1.0 must be explicitly
   negotiated as they are not the default behavior. HTTP/1.0
   experimental implementations of persistent connections are faulty,
   and the new facilities in HTTP/1.1 are designed to rectify these
   problems. The problem was that some existing 1.0 clients may be
   sending Keep-Alive to a proxy server that doesn't understand
   Connection, which would then erroneously forward it to the next
   inbound server, which would establish the Keep-Alive connection and
   result in a hung HTTP/1.0 proxy waiting for the close on the
   response. The result is that HTTP/1.0 clients must be prevented from
   using Keep-Alive when talking to proxies.
   However, talking to proxies is the most important use of persistent
   connections, so that prohibition is clearly unacceptable. Therefore,
   we need some other mechanism for indicating a persistent connection
   is desired, which is safe to use even when talking to an old proxy
   that ignores Connection. Persistent connections are the default for
   HTTP/1.1 messages; we introduce a new keyword (Connection: close) for
   declaring non-persistence.
   The following describes the original HTTP/1.0 form of persistent
   connections.
   When it connects to an origin server, an HTTP client MAY send the
   Keep-Alive connection-token in addition to the Persist connection-   token:
          Connection: Keep-Alive
   An HTTP/1.0 server would then respond with the Keep-Alive connection
   token and the client may proceed with an HTTP/1.0 (or Keep-Alive)
   persistent connection.
   An HTTP/1.1 server may also establish persistent connections with
   HTTP/1.0 clients upon receipt of a Keep-Alive connection token.
   However, a persistent connection with an HTTP/1.0 client cannot make
   use of the chunked transfer-coding, and therefore MUST use a
   Content-Length for marking the ending boundary of each message.
   A client MUST NOT send the Keep-Alive connection token to a proxy
   server as HTTP/1.0 proxy servers do not obey the rules of HTTP/1.1
   for parsing the Connection header field.




19.7.1.1 The Keep-Alive Header
   When the Keep-Alive connection-token has been transmitted with a
   request or a response, a Keep-Alive header field MAY also be
   included. The Keep-Alive header field takes the following form:
          Keep-Alive-header = "Keep-Alive" ":" 0# keepalive-param
          keepalive-param = param-name "=" value
   The Keep-Alive header itself is optional, and is used only if a
   parameter is being sent. HTTP/1.1 does not define any parameters.
   If the Keep-Alive header is sent, the corresponding connection token
   MUST be transmitted. The Keep-Alive header MUST be ignored if
   received without the connection token.


0
 
LVL 32

Expert Comment

by:jhance
ID: 1832133
So the bottom line is that the code you show above would work if the server supports this capability.  A HTTP 1.0 server MAY support it but was not required to.  An HTTP 1.1 server MUST support it but doesn't have to HONOR the request.  So you should be prepared to handle any eventuality in your request code.
0
 
LVL 1

Author Comment

by:trof
ID: 1832134
Thanks, if I'm not mistaken, it's RFC#... Sorry, but I already read this(or similar) text, and trying to follow this instruction, but my code does not work on _any_ server, even on server, answered HTTP/1.1 in header. That's a reason, why I need _small_ _working_ _example_ . Thanks again.
0
 
LVL 5

Expert Comment

by:julio011597
ID: 1832135
trof:
is it correct if i assume you're going to care yourself of the http protocol itself, and just give you some C code to connect to a web server through sockets?
0
 
LVL 1

Author Comment

by:trof
ID: 1832136
ok, now I have code mmm...something like this
int GetURL(char *cURL, char*Buffer)l
It opens socket, sends "HTTP/ ...GET..." to server, receives Page of given URL and closes socket. If I need next page from the same server, connection must be reopened again, and of cousre, it's time consuming..
Examle I need: Let's say, three functions
#define SOCKET int
1. SOCKET OpenHTTPConnection(char *cServer);
opens socket to talk to given server and returns it.. Probably, says to server some magic words..
2 int GetHTTPPage(SOCKET s, char *cURL,  char *cBuf)
sends "HTTP ..." to server on socket s, receives page, connection is still opened, so, I can use this function as many times as I wish, without reopening socket.
3 void CloseHTTPConnection(SOCKET s)
{
 close(s); //that's all
}




0
 
LVL 7

Expert Comment

by:faster
ID: 1832137
I have tried it with an http 1.1 server and it works.  However, the server does close the connection if the 2nd request comes too later after the 1st one (about 8 seconds on my server).  And of course, it has to be an http 1.1 server.

One thing to note, http 1.1 requires that there must be a header:  In your code I did not see it, maybe this is where you fail.

HOST: XXXX.XXX.XX (the server)

Another reason why you fail is that you go through a (or some) proxy server, however, one or some of the proxy along the way does not support http 1.1.  Besides, if you does go through the proxy server, then you should use the following:

Proxy-Connection: Keep-Alive

Instead of the normal:

Connection: Keep-Alive


To supply you the code is of no problem, but since you can already get response from the server, I think you don't really need it.  All you need to is to add the HOST header and
Proxy-Connection: Keep-Alive header if proxy is used.  And you should only test with http 1.1 servers.

Finally, to ask the server close the socket, use Connection: Close

0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 7

Expert Comment

by:faster
ID: 1832138
I have some typo in my answer, so to make it more clear:

If you directly access the http 1.1 server:

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost: www.microsoft.com\r\nConnection: Keep-Alive\r\n\r\n");

(there should be no space before the header and you must use HTTP/1.1 as the version)

If you go through a proxy server (which supports 1.1)

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost: www.microsoft.com\r\nProxy-Connection: Keep-Alive\r\n\r\n");
 
0
 
LVL 1

Author Comment

by:trof
ID: 1832139
Yes, I know that stuff about "HOST: ..", etc. - I have read RFC about HTTP/1.1 . By the way - "Connection: Keep-Alive" does work on HTTP/1.0 as well, you just have to check returned http header... The problem is - if, for examle I open commection to www.microsoft.com, it allows "alive" connection with big pleasure,and I cam read a page by
write(s,...) // request for page
while(res != 0)
{
  res = read(s, ...)
}
 but when I read next page,
read(s, ...)
returns 0 immediatelly.
Probably, I should say some magic words to socket to reset it or..whatever.
So, I still looking for _working_ _small_ sample of C-code..

0
 
LVL 7

Expert Comment

by:faster
ID: 1832140
I suggest you try it on a local server first.

I'd like to know whether you connect directly with the http server or through a proxy server?  Even the server reply that the connection is alive, actually it may already be closed.  That is the case I observe, I guess due to the proxy server in the way.

HTTP 1.0 does not define a Connection header, there may be some 1.0 client and server using it, but that is the standard.

I will soon give you a sample code that works on my server (on the lan).
0
 
LVL 7

Expert Comment

by:faster
ID: 1832141
I know your problem now.

Your code

while(res != 0)
      {
        res = read(s, ...)
      }

is not correct.  That is for http 1.0, because it will return until the other side close the socket.  So instead of wait until the result is 0, you should parse the response and get content-length, using this value you can know which response is for which request.

Below is my sample code.


#include <winsock.h>

void main()
{
      WSADATA wsadata;
      SOCKET s;
      SOCKADDR_IN hSockAddr;
      char buf1[300], buf2[40960];
      int r;

      WSAStartup(MAKEWORD(1, 1), &wsadata);
      s = socket(PF_INET, SOCK_STREAM, 0);

    hSockAddr.sin_family      = AF_INET;
    hSockAddr.sin_addr.s_addr = 0;
    hSockAddr.sin_port        = 0;

      bind(s,(struct sockaddr *)&hSockAddr,sizeof(hSockAddr));

    hSockAddr.sin_port=htons(80);
      hSockAddr.sin_addr.s_addr=inet_addr("10.171.30.40");

      connect(s,(struct sockaddr *) &hSockAddr, sizeof(SOCKADDR_IN));

      wsprintf(buf1, "GET /feedback/ HTTP/1.1\r\nHost: buxley\r\nConnection: Keep-Alive\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      wsprintf(buf1, "GET /search/ HTTP/1.1\r\nHost: buxley\r\nConnection: Close\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      do      {
            r = recv(s, buf2, 40960, 0);
      } while (r);

      closesocket(s);

      WSACleanup();
}

In my code I send out two request at the same time, and then the following read will get both html.  I omit all error checking here.
0
 
LVL 7

Expert Comment

by:faster
ID: 1832142
This is from the document on recv()

If the function succeeds, recv returns the number of bytes received. If the connection has been closed, it returns zero


So you are actually waiting for the server to time out.
0
 
LVL 1

Author Comment

by:trof
ID: 1832143
does not work on www.microsoft.com. Received HTTP header contains "Connection: keep-alive", but try to get the second file, recv=0, and connection is dead... :( What I'm doing wrong ? I did not change functional part of your sample..
0
 
LVL 7

Accepted Solution

by:
faster earned 400 total points
ID: 1832144
You should give me more detail about how it fails.  "try to get the second file, recv=0", in my code, all the files goes into one buffer.  There is no special recv() for the 2nd file!  In other word, it is the upper layer (your app) to tell which is the first file and which is the 2nd from the data received by recv().  If you repear my code:

do {
     r = recv(s, buf2, 40960, 0);
} while (r);
do {
     r = recv(s, buf2, 40960, 0);
} while (r);

trying to get one file with a loop, then it is wrong.

Maybe you should paste all your code, so I can tell you where is wrong.

Basically, did you send out the two request before trying to recv()?  If you do so (like what I do in my code), the two response will all go to buf2.  You have to parse the buffer to know which part is for which.

If you call recv() after sending out ONE request, then you can not wait for the return code to become 0, instead, you should parse the response (or use a time out) to determine that it is terminated and then send out the 2nd request.

You should connect to microsoft WITHOUT a proxy server, since the proxy server may close the socket.

Finally, are you sure the microsoft server is using 1.1?  Check its response, is it using HTTP 1.1 or 1.0?
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

If you get a (Blue Screen of Death), your system writes a small file called a minidump. Your first step is to make certain your computer is setup to record memory dumps. Right click My Computer, choose properties. Click on the advanced tab, an…
Recently I have been answering a lot of questions like this in IT forums that I frequent. The question posed is usually something along the lines of "We have software X installed and need to uninstall it for reason Y" or some other variant of the sa…
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
The viewer will learn how to count occurrences of each item in an array.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now