Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

recv socket call

Posted on 2004-10-28
29
Medium Priority
?
8,629 Views
Last Modified: 2008-02-20
I am programming with sockets in Linux. I have a blocking socket and I want the following: When I call recv(), I want to know how many bytes of data there is in total, then I want to construct a buffer to hold that amount, and call recv() again with that buffer to get all the data that is waiting in one call. How do I do that?

Thanks
David
0
Comment
Question by:davidm82
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 11
  • 5
  • +1
29 Comments
 
LVL 46

Expert Comment

by:Kent Olsen
ID: 12438759

The protocol doesn't provide a way to tell you the number of bytes prior to executing the recv().

Kent
0
 

Author Comment

by:davidm82
ID: 12438800
So what do i do if I do not know how many bytes are expected in the message?
0
 
LVL 46

Expert Comment

by:Kent Olsen
ID: 12438835

You'll have to make repeated calls to recv();

Memory is cheap on most of these machines.  Assign a buffer that's "plenty large" and you'll never hit the looping controls.


Kent
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 

Author Comment

by:davidm82
ID: 12438947
So you are saying that if I set the buffer to be of length 1000, then if the return value is <1000, i read the whole thing, and if it is >1000, I should keep reading until it is less than 1000?
If so, what do I do if it is exactly 1000?
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12439085
I have to disagree partly with Kdo.
it is possible to find the number of bytes  present in the
socket receive buffer.  you will need to call recv() with
MSG_PEEK flag.

ssize_t
     recv(int s, void *buf, size_t len, int flags);

if you specify flags = MSG_PEEK, the message in the
socket receive buffer isnt discarded and can be read
in the next read via recv. recv() returns the number of bytes
present in the receive buffer. This flag is usually useful
with UDP sockets.

Take a look at the manpage of recv() for further information.
as an example, consicder an open socket sockfd. to find out how
many bytes are present to be read, use something like this:

          int num = 0;
           int flags = MSG_PEEK;
          num = recv(sockfd, NULL, 0, flags);


num will contain the size you aRE asking for

0
 

Author Comment

by:davidm82
ID: 12439132
MSG_PEEK seems like it does the same thing a regular recv() call will do except that the next time you call recv() the buffer will be the same as the last time.
The return value when using MSG_PEEK does not seem to be the size of the entire buffer, just the size of the segment you got, exactly the same thing as when not using MSG_PEEK.
I had thought of using MSG_TRUNC, but that does nt seem to do what it is advertised to do in the man pages.

David
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12439199
MSG_PEEK flag is usually useful with UDP sockets.
if you call recv() with MSG_PEEK set over an UDP
socket, and u get say a value of 256, the next call to recv()
over the same socket will return 256 bytes, even if more data
arrived on socket.

with TCP sockets, the situation is different. if you call
recv() first time with MSG_PEEK, and get a return of 256,
and then u read via another call to recv(), the number of bytes returned
can be more than 256.

when u are not using MSG_PEEK, the data is discarded, and you will
not be able to re read the socket for that data. so u see the difference between
using MSG_PEEK and using regular recv() ?  

You cannot use MSG_TRUNC with recv().
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12439892
In case you are looking for the TOTAL size of the
socket receive buffer you can get that by calling getsockopt().

int size;
int len = sizeof(size);
getsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &size, &len);

size will contain the total size of the socket reveive buffer.
 to allocate  space for storage;

char *buffer = malloc(sze);

hope this helps
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 12441340
There is an option

MSG_WAITALL
This flag requests that the operation block until the full request is satisfied.  However, the call may still return less data than requested if a signal is caught, an error or disconnect occurs, or the next data to be received is of a different type than that returned.

declare a buf for say 1000 bytes and specify MSG_WAITALL in the flags field ... This will allow you to wait for entire message unless it is less than 1000 bytes in which case you can call recv again ...

char buf[1000];

do
{
        bytes=0;
        bytes = recv (sockfd, buffer, 1000, MSG_WAITALL );
}while ( bytes == 1000 );

I think this should serve your purpose.

cheers
sunnycoder
0
 

Author Comment

by:davidm82
ID: 12447515
I tried the getsockopt() function above, and kept the getting the same really large number (about 80,000) returned. So it seems to be returning the size of the maximum buffer, rather than the size of what is in the current buffer.
0
 
LVL 46

Expert Comment

by:Kent Olsen
ID: 12447612
Hi David,

The protocol doesn't "preinform" the receiver of the amount of data.  But you can program around this limitation.  The code below is an extension of what Sunnycoder provided.  It will repeatedly expand a buffer and issue recv() calls until the transfer is complete.  The effect is that the application can execute the code below and have the entire datablock returned as a single entity.

Kent



#define TRANSFER_SIZE 1024   /*  We're going to grab data in 1K chunks.  32K  might be better.  */

char *Buffer;
char *BufferSize;
int    Bytes;

  Buffer = NULL;
  BufferSize = 0;
  do
  {
    BufferSize += TRANSFER_SIZE;
    Buffer = (char *) realloc (Buffer, BufferSize);
    Bytes = recv (sockfd, buffer + BufferSize - TRANSFER_SIZE, 1000, MSG_WAITALL);
  } while (Bytes = TRANSFER_SIZE);

  Bytes = BufferSize - TRANSFER_SIZE + Bytes;  /*  Compute the total transfer length  */
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12447664
>>I have a blocking socket and I want the following: When I call recv(), I want to know how many bytes of data there is in total, then I want to construct a buffer to hold that amount.

there are 2 interpretations possible to your "I want to know how many bytes of data there is in total"
1) you want to know how many bytes are there in the socket receive buffer to be read: this can be accomplished by
        MSG_PEEK
2) what is the maximum size of the socket receive buffer: the getsockopt() solves it.
0
 

Author Comment

by:davidm82
ID: 12447684
The correct interpretation is number 1.
How would I get the size using MSG_PEEK?
Also, just out of curiosity, what happens if someone sends a message larger than the maximum size given by getsockopt()?
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12447929
>>  The correct interpretation is number 1.
How would I get the size using MSG_PEEK?

int sizeofcurrentdata = recv(sockfd, NULL, 0, MSG_PEEK); // this call to recv() will return the number of bytes present
                                                                                                  //  the socket receive buffer that will be returned in the next
                                                                                                  // call  of recv(). isnt this what you want? However, please
                                                                                                   // read my second post to your question to see the
                                                                                                   //  difference in behaviour of recv() for UDP and TCP
                                                                                                   //  sockets.

>> Also, just out of curiosity, what happens if someone sends a message larger than the maximum size given by getsockopt()?

There are two situations to consider:
1) TCP:   before connection establishment, the protocol stacks of both the communicating systems determine the MSS
                value for transactions. MSS = maximum segment size.  Data sizes greater than MSS are fragmented and tcp
                takes care of delivering / receiving the segments in the proper order.

2)  UDP: here there is no reliability. However, if your application writes a very big chunk of data to the UDP socket,
               there is much higher possibility of fragmentation. In the cse when the socket receive buffer of a UDP
               socket is full, other data coming to it will be simply discarded. (remember UDP is unreliable).

hope this helps
0
 

Author Comment

by:davidm82
ID: 12447976
Kent and sunnycoder,

I tried the MSG_WAITALL option and the function just blokcs when i use it. Does that option work on Linux?
0
 
LVL 46

Expert Comment

by:Kent Olsen
ID: 12447988
Hi Vandy,

I didn't realize that MSG_PEEK behaved this way.

This implies that you can actually block twice on the same data.  the recv(...MSG_PEEK) has to block until it knows the entire count, and the second call to recv() will block again as the data is copied to the application.

Clearly not an effective way to move large amounts of data.


Kent
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12448054
Ok kdo, yea u are right about that,
we will have to use some trhing like

recv(sock, NULL, 0, MSG_PEEK | MSG_DONTWAIT) for a a blocking socket.
0
 

Author Comment

by:davidm82
ID: 12448068
yeah, i tried the MSG_PEEK code above, even for a nonblocking socket, and it always returned 0 as the length.
0
 
LVL 5

Accepted Solution

by:
van_dy earned 1000 total points
ID: 12448077
how ever there is another way, using ioctl().

int num_bytes;

ioctl(socket, FIONREAD, &num_bytes);

when ioctl() returns, you will have the total number of readable bytes in the socket.
I am not sure if that will work under Linux, give it a try
0
 
LVL 46

Expert Comment

by:Kent Olsen
ID: 12448080
Hi David,

This thread started with you saying that you had a blocking socket.  Didn't realize that the blocking was an issue.

Kent
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12448102
ioctl(socket, FIONREAD, &num_bytes);                <<-- num_bytes = total number of readable bytes in the socket
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12448140
hi  davidm82,
      as i suggested, you can use ioctl() method, i just checked it on linux,
seems it should work fine there too.
0
 

Author Comment

by:davidm82
ID: 12448418
ok, i tried ioctl and it seems to work.
does this seem correct for a blocking socket:
i first call recv() with MSG_PEEK to block until i get something. then i call ioctl to see how big the message is. then i create a buffer of that length and call recv() again with that buffer to get everything.
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12448511
Hi david,

   Here is what you need to do.

   int sockfd = socket(...);   // open your socket
    char *data;
   int num = 0;

   // use select() system call to see if the sockfd is readable
  //  please see the manpage for it. select will block till the
  // socket is readable

   // now once select returns with the result as socket as readable do this

  ioctl(sockfd, FIONREAD, &num_bytes);

   // num_bytes will contain the no of bytes to be red.
   data = malloc(num_bytes);

hope this helps
van_dy
0
 

Author Comment

by:davidm82
ID: 12448835
after the malloc call, then do i call recv with that buffer?
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12448925
yes, after allocating the buffer , to receive the data
you will have to call, recv() as follows:

 int sockfd = socket(...);   // open your socket
    char *data;
   int num = 0;
    int len = 0;

   // use select() system call to see if the sockfd is readable
  //  please see the manpage for it. select will block till the
  // socket is readable

   // now once select returns with the result as socket as readable do this

  ioctl(sockfd, FIONREAD, &num_bytes);

   // num_bytes will contain the no of bytes to be red.
   data = malloc(num_bytes);

    recv(sockfd, data, num_bytes, 0);


this will put the data in ur buffer
0
 

Author Comment

by:davidm82
ID: 12449010
I am too lazy to look up the select system call. Right now, instead of using select, I just use redv(), which will also block until there is more data. I use MSG_PEEK in that recv() call so that it does not take anything off the buffer. I think this produces the same result as using the select() call would. Is that right?
0
 
LVL 5

Expert Comment

by:van_dy
ID: 12449716
yes, you aare right. though i would still
reccommend to look into select() since that
is the standard way to block
0
 

Author Comment

by:davidm82
ID: 12449730
OK, thanks for your help.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Preface I don't like visual development tools that are supposed to write a program for me. Even if it is Xcode and I can use Interface Builder. Yes, it is a perfect tool and has helped me a lot, mainly, in the beginning, when my programs were small…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
The goal of this video is to provide viewers with basic examples to understand recursion in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use nested-loops in the C programming language.
Suggested Courses

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question