Solved

recv socket call

Posted on 2004-10-28
8,596 Views
Last Modified: 2008-02-20
I am programming with sockets in Linux. I have a blocking socket and I want the following: When I call recv(), I want to know how many bytes of data there is in total, then I want to construct a buffer to hold that amount, and call recv() again with that buffer to get all the data that is waiting in one call. How do I do that?

Thanks
David
0
Question by:davidm82
    29 Comments
     
    LVL 45

    Expert Comment

    by:Kdo

    The protocol doesn't provide a way to tell you the number of bytes prior to executing the recv().

    Kent
    0
     

    Author Comment

    by:davidm82
    So what do i do if I do not know how many bytes are expected in the message?
    0
     
    LVL 45

    Expert Comment

    by:Kdo

    You'll have to make repeated calls to recv();

    Memory is cheap on most of these machines.  Assign a buffer that's "plenty large" and you'll never hit the looping controls.


    Kent
    0
     

    Author Comment

    by:davidm82
    So you are saying that if I set the buffer to be of length 1000, then if the return value is <1000, i read the whole thing, and if it is >1000, I should keep reading until it is less than 1000?
    If so, what do I do if it is exactly 1000?
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    I have to disagree partly with Kdo.
    it is possible to find the number of bytes  present in the
    socket receive buffer.  you will need to call recv() with
    MSG_PEEK flag.

    ssize_t
         recv(int s, void *buf, size_t len, int flags);

    if you specify flags = MSG_PEEK, the message in the
    socket receive buffer isnt discarded and can be read
    in the next read via recv. recv() returns the number of bytes
    present in the receive buffer. This flag is usually useful
    with UDP sockets.

    Take a look at the manpage of recv() for further information.
    as an example, consicder an open socket sockfd. to find out how
    many bytes are present to be read, use something like this:

              int num = 0;
               int flags = MSG_PEEK;
              num = recv(sockfd, NULL, 0, flags);


    num will contain the size you aRE asking for

    0
     

    Author Comment

    by:davidm82
    MSG_PEEK seems like it does the same thing a regular recv() call will do except that the next time you call recv() the buffer will be the same as the last time.
    The return value when using MSG_PEEK does not seem to be the size of the entire buffer, just the size of the segment you got, exactly the same thing as when not using MSG_PEEK.
    I had thought of using MSG_TRUNC, but that does nt seem to do what it is advertised to do in the man pages.

    David
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    MSG_PEEK flag is usually useful with UDP sockets.
    if you call recv() with MSG_PEEK set over an UDP
    socket, and u get say a value of 256, the next call to recv()
    over the same socket will return 256 bytes, even if more data
    arrived on socket.

    with TCP sockets, the situation is different. if you call
    recv() first time with MSG_PEEK, and get a return of 256,
    and then u read via another call to recv(), the number of bytes returned
    can be more than 256.

    when u are not using MSG_PEEK, the data is discarded, and you will
    not be able to re read the socket for that data. so u see the difference between
    using MSG_PEEK and using regular recv() ?  

    You cannot use MSG_TRUNC with recv().
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    In case you are looking for the TOTAL size of the
    socket receive buffer you can get that by calling getsockopt().

    int size;
    int len = sizeof(size);
    getsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &size, &len);

    size will contain the total size of the socket reveive buffer.
     to allocate  space for storage;

    char *buffer = malloc(sze);

    hope this helps
    0
     
    LVL 45

    Expert Comment

    by:sunnycoder
    There is an option

    MSG_WAITALL
    This flag requests that the operation block until the full request is satisfied.  However, the call may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned.

    declare a buf for say 1000 bytes and specify MSG_WAITALL in the flags field ... This will allow you to wait for entire message unless it is less than 1000 bytes in which case you can call recv again ...

    char buf[1000];

    do
    {
            bytes=0;
            bytes = recv (sockfd, buffer, 1000, MSG_WAITALL );
    }while ( bytes == 1000 );

    I think this should serve your purpose.

    cheers
    sunnycoder
    0
     

    Author Comment

    by:davidm82
    I tried the getsockopt() function above, and kept the getting the same really large number (about 80,000) returned. So it seems to be returning the size of the maximum buffer, rather than the size of what is in the current buffer.
    0
     
    LVL 45

    Expert Comment

    by:Kdo
    Hi David,

    The protocol doesn't "preinform" the receiver of the amount of data.  But you can program around this limitation.  The code below is an extension of what Sunnycoder provided.  It will repeatedly expand a buffer and issue recv() calls until the transfer is complete.  The effect is that the application can execute the code below and have the entire datablock returned as a single entity.

    Kent



    #define TRANSFER_SIZE 1024   /*  We're going to grab data in 1K chunks.  32K  might be better.  */

    char *Buffer;
    char *BufferSize;
    int    Bytes;

      Buffer = NULL;
      BufferSize = 0;
      do
      {
        BufferSize += TRANSFER_SIZE;
        Buffer = (char *) realloc (Buffer, BufferSize);
        Bytes = recv (sockfd, buffer + BufferSize - TRANSFER_SIZE, 1000, MSG_WAITALL);
      } while (Bytes = TRANSFER_SIZE);

      Bytes = BufferSize - TRANSFER_SIZE + Bytes;  /*  Compute the total transfer length  */
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    >>I have a blocking socket and I want the following: When I call recv(), I want to know how many bytes of data there is in total, then I want to construct a buffer to hold that amount.

    there are 2 interpretations possible to your "I want to know how many bytes of data there is in total"
    1) you want to know how many bytes are there in the socket receive buffer to be read: this can be accomplished by
            MSG_PEEK
    2) what is the maximum size of the socket receive buffer: the getsockopt() solves it.
    0
     

    Author Comment

    by:davidm82
    The correct interpretation is number 1.
    How would I get the size using MSG_PEEK?
    Also, just out of curiosity, what happens if someone sends a message larger than the maximum size given by getsockopt()?
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    >>  The correct interpretation is number 1.
    How would I get the size using MSG_PEEK?

    int sizeofcurrentdata = recv(sockfd, NULL, 0, MSG_PEEK); // this call to recv() will return the number of bytes present
                                                                                                      //  the socket receive buffer that will be returned in the next
                                                                                                      // call  of recv(). isnt this what you want? However, please
                                                                                                       // read my second post to your question to see the
                                                                                                       //  difference in behaviour of recv() for UDP and TCP
                                                                                                       //  sockets.

    >> Also, just out of curiosity, what happens if someone sends a message larger than the maximum size given by getsockopt()?

    There are two situations to consider:
    1) TCP:   before connection establishment, the protocol stacks of both the communicating systems determine the MSS
                    value for transactions. MSS = maximum segment size.  Data sizes greater than MSS are fragmented and tcp
                    takes care of delivering / receiving the segments in the proper order.

    2)  UDP: here there is no reliability. However, if your application writes a very big chunk of data to the UDP socket,
                   there is much higher possibility of fragmentation. In the cse when the socket receive buffer of a UDP
                   socket is full, other data coming to it will be simply discarded. (remember UDP is unreliable).

    hope this helps
    0
     

    Author Comment

    by:davidm82
    Kent and sunnycoder,

    I tried the MSG_WAITALL option and the function just blokcs when i use it. Does that option work on Linux?
    0
     
    LVL 45

    Expert Comment

    by:Kdo
    Hi Vandy,

    I didn't realize that MSG_PEEK behaved this way.

    This implies that you can actually block twice on the same data.  the recv(...MSG_PEEK) has to block until it knows the entire count, and the second call to recv() will block again as the data is copied to the application.

    Clearly not an effective way to move large amounts of data.


    Kent
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    Ok kdo, yea u are right about that,
    we will have to use some trhing like

    recv(sock, NULL, 0, MSG_PEEK | MSG_DONTWAIT) for a a blocking socket.
    0
     

    Author Comment

    by:davidm82
    yeah, i tried the MSG_PEEK code above, even for a nonblocking socket, and it always returned 0 as the length.
    0
     
    LVL 5

    Accepted Solution

    by:
    how ever there is another way, using ioctl().

    int num_bytes;

    ioctl(socket, FIONREAD, &num_bytes);

    when ioctl() returns, you will have the total number of readable bytes in the socket.
    I am not sure if that will work under Linux, give it a try
    0
     
    LVL 45

    Expert Comment

    by:Kdo
    Hi David,

    This thread started with you saying that you had a blocking socket.  Didn't realize that the blocking was an issue.

    Kent
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    ioctl(socket, FIONREAD, &num_bytes);                <<-- num_bytes = total number of readable bytes in the socket
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    hi  davidm82,
          as i suggested, you can use ioctl() method, i just checked it on linux,
    seems it should work fine there too.
    0
     

    Author Comment

    by:davidm82
    ok, i tried ioctl and it seems to work.
    does this seem correct for a blocking socket:
    i first call recv() with MSG_PEEK to block until i get something. then i call ioctl to see how big the message is. then i create a buffer of that length and call recv() again with that buffer to get everything.
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    Hi david,

       Here is what you need to do.

       int sockfd = socket(...);   // open your socket
        char *data;
       int num = 0;

       // use select() system call to see if the sockfd is readable
      //  please see the manpage for it. select will block till the
      // socket is readable

       // now once select returns with the result as socket as readable do this

      ioctl(sockfd, FIONREAD, &num_bytes);

       // num_bytes will contain the no of bytes to be red.
       data = malloc(num_bytes);

    hope this helps
    van_dy
    0
     

    Author Comment

    by:davidm82
    after the malloc call, then do i call recv with that buffer?
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    yes, after allocating the buffer , to receive the data
    you will have to call, recv() as follows:

     int sockfd = socket(...);   // open your socket
        char *data;
       int num = 0;
        int len = 0;

       // use select() system call to see if the sockfd is readable
      //  please see the manpage for it. select will block till the
      // socket is readable

       // now once select returns with the result as socket as readable do this

      ioctl(sockfd, FIONREAD, &num_bytes);

       // num_bytes will contain the no of bytes to be red.
       data = malloc(num_bytes);

        recv(sockfd, data, num_bytes, 0);


    this will put the data in ur buffer
    0
     

    Author Comment

    by:davidm82
    I am too lazy to look up the select system call. Right now, instead of using select, I just use redv(), which will also block until there is more data. I use MSG_PEEK in that recv() call so that it does not take anything off the buffer. I think this produces the same result as using the select() call would. Is that right?
    0
     
    LVL 5

    Expert Comment

    by:van_dy
    yes, you aare right. though i would still
    reccommend to look into select() since that
    is the standard way to block
    0
     

    Author Comment

    by:davidm82
    OK, thanks for your help.
    0

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    How to improve team productivity

    Quip adds documents, spreadsheets, and tasklists to your Slack experience
    - Elevate ideas to Quip docs
    - Share Quip docs in Slack
    - Get notified of changes to your docs
    - Available on iOS/Android/Desktop/Web
    - Online/Offline

    Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
    This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
    The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
    The goal of this video is to provide viewers with basic examples to understand how to use strings and some functions related to them in the C programming language.

    884 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now