Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

HTTP Keep-Alive Sample

Posted on 1998-03-09
14
Medium Priority
?
588 Views
Last Modified: 2013-12-25
I've heared, it's possible - to open connection to some server, and get more than one URL without reopening socket for every HTTP transaction, something like
s = connect(... //open connection olny once
ssprintf(cBuf,... "...GET file1.html \r\n Connection: Keep-Alive...
write(s,cBuf
read(s ... //get first page
ssprintf(cBuf,... "...GET file2.html \r\n Connection: Keep-Alive
write(s,cBuf..
read(s ... //get second page on the same socket

....
close(s); //close it after all
Does anybody have small working sample (flat C) ?
Thanks in advance.
0
Comment
Question by:trof
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 2
  • +1
14 Comments
 
LVL 5

Expert Comment

by:julio011597
ID: 1832131
I've never seen the "keep-alive" stuff (the http subtleties matter there), so will need collaboration from some other expert on that point (EE stuff is already supporting some sort of point sharing among experts, when more than one of them helps providing an answer).

If that side of the problem is solvable, i could give you a significant help with C code and socket connection handling.

I'll stay in touch...
0
 
LVL 32

Expert Comment

by:jhance
ID: 1832132
This is an HTTP 1.1 capability.  Here are a couple of quote from the spec at http://www.w3.org/Protocols/rfc2068/rfc2068:

8.1.2.1 Negotiation
   An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to
   maintain a persistent connection unless a Connection header including
   the connection-token "close" was sent in the request. If the server
   chooses to close the connection immediately after sending the
   response, it SHOULD send a Connection header including the
   connection-token close.
   An HTTP/1.1 client MAY expect a connection to remain open, but would
   decide to keep it open based on whether the response from a server
   contains a Connection header with the connection-token close. In case
   the client does not want to maintain a connection for more than that
   request, it SHOULD send a Connection header including the
   connection-token close.
   If either the client or the server sends the close token in the
   Connection header, that request becomes the last one for the   connection.
   Clients and servers SHOULD NOT assume that a persistent connection is
   maintained for HTTP versions less than 1.1 unless it is explicitly
   signaled. See section 19.7.1 for more information on backwards
   compatibility with HTTP/1.0 clients.
   In order to remain persistent, all messages on the connection must
   have a self-defined message length (i.e., one not defined by closure
   of the connection), as described in section 4.4.



19.7.1 Compatibility with HTTP/1.0 Persistent Connections
   Some clients and servers may wish to be compatible with some previous
   implementations of persistent connections in HTTP/1.0 clients and
   servers. Persistent connections in HTTP/1.0 must be explicitly
   negotiated as they are not the default behavior. HTTP/1.0
   experimental implementations of persistent connections are faulty,
   and the new facilities in HTTP/1.1 are designed to rectify these
   problems. The problem was that some existing 1.0 clients may be
   sending Keep-Alive to a proxy server that doesn't understand
   Connection, which would then erroneously forward it to the next
   inbound server, which would establish the Keep-Alive connection and
   result in a hung HTTP/1.0 proxy waiting for the close on the
   response. The result is that HTTP/1.0 clients must be prevented from
   using Keep-Alive when talking to proxies.
   However, talking to proxies is the most important use of persistent
   connections, so that prohibition is clearly unacceptable. Therefore,
   we need some other mechanism for indicating a persistent connection
   is desired, which is safe to use even when talking to an old proxy
   that ignores Connection. Persistent connections are the default for
   HTTP/1.1 messages; we introduce a new keyword (Connection: close) for
   declaring non-persistence.
   The following describes the original HTTP/1.0 form of persistent
   connections.
   When it connects to an origin server, an HTTP client MAY send the
   Keep-Alive connection-token in addition to the Persist connection-   token:
          Connection: Keep-Alive
   An HTTP/1.0 server would then respond with the Keep-Alive connection
   token and the client may proceed with an HTTP/1.0 (or Keep-Alive)
   persistent connection.
   An HTTP/1.1 server may also establish persistent connections with
   HTTP/1.0 clients upon receipt of a Keep-Alive connection token.
   However, a persistent connection with an HTTP/1.0 client cannot make
   use of the chunked transfer-coding, and therefore MUST use a
   Content-Length for marking the ending boundary of each message.
   A client MUST NOT send the Keep-Alive connection token to a proxy
   server as HTTP/1.0 proxy servers do not obey the rules of HTTP/1.1
   for parsing the Connection header field.




19.7.1.1 The Keep-Alive Header
   When the Keep-Alive connection-token has been transmitted with a
   request or a response, a Keep-Alive header field MAY also be
   included. The Keep-Alive header field takes the following form:
          Keep-Alive-header = "Keep-Alive" ":" 0# keepalive-param
          keepalive-param = param-name "=" value
   The Keep-Alive header itself is optional, and is used only if a
   parameter is being sent. HTTP/1.1 does not define any parameters.
   If the Keep-Alive header is sent, the corresponding connection token
   MUST be transmitted. The Keep-Alive header MUST be ignored if
   received without the connection token.


0
 
LVL 32

Expert Comment

by:jhance
ID: 1832133
So the bottom line is that the code you show above would work if the server supports this capability.  A HTTP 1.0 server MAY support it but was not required to.  An HTTP 1.1 server MUST support it but doesn't have to HONOR the request.  So you should be prepared to handle any eventuality in your request code.
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 1

Author Comment

by:trof
ID: 1832134
Thanks, if I'm not mistaken, it's RFC#... Sorry, but I already read this(or similar) text, and trying to follow this instruction, but my code does not work on _any_ server, even on server, answered HTTP/1.1 in header. That's a reason, why I need _small_ _working_ _example_ . Thanks again.
0
 
LVL 5

Expert Comment

by:julio011597
ID: 1832135
trof:
is it correct if i assume you're going to care yourself of the http protocol itself, and just give you some C code to connect to a web server through sockets?
0
 
LVL 1

Author Comment

by:trof
ID: 1832136
ok, now I have code mmm...something like this
int GetURL(char *cURL, char*Buffer)l
It opens socket, sends "HTTP/ ...GET..." to server, receives Page of given URL and closes socket. If I need next page from the same server, connection must be reopened again, and of cousre, it's time consuming..
Examle I need: Let's say, three functions
#define SOCKET int
1. SOCKET OpenHTTPConnection(char *cServer);
opens socket to talk to given server and returns it.. Probably, says to server some magic words..
2 int GetHTTPPage(SOCKET s, char *cURL,  char *cBuf)
sends "HTTP ..." to server on socket s, receives page, connection is still opened, so, I can use this function as many times as I wish, without reopening socket.
3 void CloseHTTPConnection(SOCKET s)
{
 close(s); //that's all
}




0
 
LVL 7

Expert Comment

by:faster
ID: 1832137
I have tried it with an http 1.1 server and it works.  However, the server does close the connection if the 2nd request comes too later after the 1st one (about 8 seconds on my server).  And of course, it has to be an http 1.1 server.

One thing to note, http 1.1 requires that there must be a header:  In your code I did not see it, maybe this is where you fail.

HOST: XXXX.XXX.XX (the server)

Another reason why you fail is that you go through a (or some) proxy server, however, one or some of the proxy along the way does not support http 1.1.  Besides, if you does go through the proxy server, then you should use the following:

Proxy-Connection: Keep-Alive

Instead of the normal:

Connection: Keep-Alive


To supply you the code is of no problem, but since you can already get response from the server, I think you don't really need it.  All you need to is to add the HOST header and
Proxy-Connection: Keep-Alive header if proxy is used.  And you should only test with http 1.1 servers.

Finally, to ask the server close the socket, use Connection: Close

0
 
LVL 7

Expert Comment

by:faster
ID: 1832138
I have some typo in my answer, so to make it more clear:

If you directly access the http 1.1 server:

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost: www.microsoft.com\r\nConnection: Keep-Alive\r\n\r\n");

(there should be no space before the header and you must use HTTP/1.1 as the version)

If you go through a proxy server (which supports 1.1)

ssprintf(cBuf, "GET file1.html HTTP/1.1\r\nHost: www.microsoft.com\r\nProxy-Connection: Keep-Alive\r\n\r\n");
 
0
 
LVL 1

Author Comment

by:trof
ID: 1832139
Yes, I know that stuff about "HOST: ..", etc. - I have read RFC about HTTP/1.1 . By the way - "Connection: Keep-Alive" does work on HTTP/1.0 as well, you just have to check returned http header... The problem is - if, for examle I open commection to www.microsoft.com, it allows "alive" connection with big pleasure,and I cam read a page by
write(s,...) // request for page
while(res != 0)
{
  res = read(s, ...)
}
 but when I read next page,
read(s, ...)
returns 0 immediatelly.
Probably, I should say some magic words to socket to reset it or..whatever.
So, I still looking for _working_ _small_ sample of C-code..

0
 
LVL 7

Expert Comment

by:faster
ID: 1832140
I suggest you try it on a local server first.

I'd like to know whether you connect directly with the http server or through a proxy server?  Even the server reply that the connection is alive, actually it may already be closed.  That is the case I observe, I guess due to the proxy server in the way.

HTTP 1.0 does not define a Connection header, there may be some 1.0 client and server using it, but that is the standard.

I will soon give you a sample code that works on my server (on the lan).
0
 
LVL 7

Expert Comment

by:faster
ID: 1832141
I know your problem now.

Your code

while(res != 0)
      {
        res = read(s, ...)
      }

is not correct.  That is for http 1.0, because it will return until the other side close the socket.  So instead of wait until the result is 0, you should parse the response and get content-length, using this value you can know which response is for which request.

Below is my sample code.


#include <winsock.h>

void main()
{
      WSADATA wsadata;
      SOCKET s;
      SOCKADDR_IN hSockAddr;
      char buf1[300], buf2[40960];
      int r;

      WSAStartup(MAKEWORD(1, 1), &wsadata);
      s = socket(PF_INET, SOCK_STREAM, 0);

    hSockAddr.sin_family      = AF_INET;
    hSockAddr.sin_addr.s_addr = 0;
    hSockAddr.sin_port        = 0;

      bind(s,(struct sockaddr *)&hSockAddr,sizeof(hSockAddr));

    hSockAddr.sin_port=htons(80);
      hSockAddr.sin_addr.s_addr=inet_addr("10.171.30.40");

      connect(s,(struct sockaddr *) &hSockAddr, sizeof(SOCKADDR_IN));

      wsprintf(buf1, "GET /feedback/ HTTP/1.1\r\nHost: buxley\r\nConnection: Keep-Alive\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      wsprintf(buf1, "GET /search/ HTTP/1.1\r\nHost: buxley\r\nConnection: Close\r\n\r\n");
      send(s, buf1, strlen(buf1), 0);
      do      {
            r = recv(s, buf2, 40960, 0);
      } while (r);

      closesocket(s);

      WSACleanup();
}

In my code I send out two request at the same time, and then the following read will get both html.  I omit all error checking here.
0
 
LVL 7

Expert Comment

by:faster
ID: 1832142
This is from the document on recv()

If the function succeeds, recv returns the number of bytes received. If the connection has been closed, it returns zero


So you are actually waiting for the server to time out.
0
 
LVL 1

Author Comment

by:trof
ID: 1832143
does not work on www.microsoft.com. Received HTTP header contains "Connection: keep-alive", but try to get the second file, recv=0, and connection is dead... :( What I'm doing wrong ? I did not change functional part of your sample..
0
 
LVL 7

Accepted Solution

by:
faster earned 1200 total points
ID: 1832144
You should give me more detail about how it fails.  "try to get the second file, recv=0", in my code, all the files goes into one buffer.  There is no special recv() for the 2nd file!  In other word, it is the upper layer (your app) to tell which is the first file and which is the 2nd from the data received by recv().  If you repear my code:

do {
     r = recv(s, buf2, 40960, 0);
} while (r);
do {
     r = recv(s, buf2, 40960, 0);
} while (r);

trying to get one file with a loop, then it is wrong.

Maybe you should paste all your code, so I can tell you where is wrong.

Basically, did you send out the two request before trying to recv()?  If you do so (like what I do in my code), the two response will all go to buf2.  You have to parse the buffer to know which part is for which.

If you call recv() after sending out ONE request, then you can not wait for the return code to become 0, instead, you should parse the response (or use a time out) to determine that it is terminated and then send out the 2nd request.

You should connect to microsoft WITHOUT a proxy server, since the proxy server may close the socket.

Finally, are you sure the microsoft server is using 1.1?  Check its response, is it using HTTP 1.1 or 1.0?
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It is a general practice to get rid of old user profiles on a computer  in a LAN environment. As I have been working with a company in a LAN environment where users move from one place to some other place at times. This will make many user profil…
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…
Suggested Courses

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question