VC++ 5.0 Socket Send Problem

I have a CGI [client] written in VC++ 5.0 that is used to access a file and transmit it to a remote machine [server]. For small files (less than 5k) the CGI works fine, but when I send files larger than 5k (very rough figure -- larger than 4096 though ;-) the message simply disappears.  There are no socket errors of any kind.  The message quietly finds its way to the bit bucket.  setsockopt is used to set the send buffer size to 200K.

I'm looking into obtaining a socket monitor utility but since this is at work I have to go through some red tape before I can implement it.  If anyone has any ideas or any experience with a problem similar to this, I'm all ears!  Source code is not entirely available, but I can share fragments for investigation purposes.
msbinderAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ToronadoCommented:
Hi msbinder,

Do you use streaming sockets or datagram socket type? With the datagram socket type it is possible to lose data. This is likely to happen when your message is larger than the buffersize of the receiver. In most cases you don't know the receivers buffersize.

Hope this will help a bit.

Regards,
 Toronado.

0
msbinderAuthor Commented:
I'm using streaming sockets in a library one of our developers picked up ages ago called Socket++.  I'm also checking into the winsock32.lib to ensure it is the most recent.

One other detail I forgot to mention is that I can receive large messages just fine.
0
msbinderAuthor Commented:
I just installed the most recent version of winsock32 and there's no change to the problem.  I'm still waiting for my company to authorize the installation of a socket monitor.

I upped the value of this question to 200 points since the "easy" answers didn't help ;-)  I plan on sending an e-mail to MS about this problem in case the winsock32 has a bug.

In addition to my question, does anyone know of a program that successfully sends large messages by way of sockets?  IOW I'm looking for proof that there is indeed a problem with my implementation.
0
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

ToronadoCommented:
Hi msbinder,

Look here http://codeguru.developer.com/network/FileTransferUsingSockets.shtml for a simple client server program using sockets. It can send large files. So, try it out.

Regards,
 Toronado
0
msbinderAuthor Commented:
Well, with a Network Monitor on the system, it shows the message going out.  Therefore this is not a C++ problem.  This question can be considered "answered", but not solved.
0
waseemanisCommented:
I am using a similar code for sending file..actually an FTP client. I don't really know if that's what makes the difference , but I have been able to transfer several MBs of data also. That ways there doesn't seem to be a problem with the socket routines.

Waseem.
0
MarthiCommented:
When you transfer (send/receive) large chunks of data, the buffer allocated by the stack for your socket may not be large enough to hold the entire chunk.

This is why send() returns to you the bytes actually sent and recv() returns the bytes actually received.

The safe way to handle this problem is to do send() and recv() in a loop, until you are sure that the entire chunk of data has gone out or has been received.

Here's how I do it:
int CSocketConnection::processSend(const char *buffer, int bufferSize)
{
      // It may not always be possible to send the entire buffer in one-go
      // so, we send in blocks
      int bytesSent = 0;
      
      while(bytesSent < bufferSize)
      {
            int n = send(_socketId, (char *)&buffer[bytesSent], (bufferSize - bytesSent), 0);
            if (n <= 0)
            {
                  cout << "Error from send() " << WSAGetLastError() << endl << flush;
                  break;
            }
            bytesSent += n;
      }

      return bytesSent;
}

int CSocketConnection::processReceive(char *buffer, int bufferSize)

{
      // It may not always be possible to receive the entire buffer in one-go
      // so, we receive in blocks
      int bytesRecd = 0;
      
      while(bytesRecd < bufferSize)
      {
            int n = recv(_socketId, (char *)&buffer[bytesRecd], (bufferSize - bytesRecd), 0);
            if (n <= 0)
            {
                  cout << "Error from recv() " << WSAGetLastError() << endl << flush;
                  break;
            }
            bytesRecd += n;
      }

      return bytesRecd;
}

I call that code like this

----------RECEIVING END-----------
                        // wait for client to send the size of datachunk
                        if ((rc = serverConnection->processReceive((char *)&receiveBuffer.dataSize, sizeof(unsigned short))) <= 0)
                        {
                              // connection was lost
                              break;
                        }
                        // get the whole datachunk now
                        if ((rc = serverConnection->processReceive((char *)receiveBuffer.data, receiveBuffer.dataSize)) <= 0)
                        {
                              // connection was lost
                              break;
                        }
----------SENDING END-----------------
// send the size of data first
                        if ((rc = serverConnection->processSend((char *)&sendBuffer.dataSize, sizeof(unsigned short))) <= 0)
                        {
                              // connection was lost
                              break;
                        }

// Now, send the datachunk
                        if ((rc = serverConnection->processSend((char *)&sendBuffer, sendBuffer.dataSize)) <= 0)
                        {
                              // connection was lost
                              break;
                        }
0
msbinderAuthor Commented:
As I mentioned, the Network Monitor proved the message is being "sent". Besides, the buffers are resized, and the send/recv is in a loop. The Network Monitor also showed that during a 'bad' send, 2 additional sockets were opened.  Our internal LAN may account for these, but I'm trying to set-up my application(s) in another way to get rid of them, or to find out what is causing them.

Thanks for the response, though, and I'll be sure to post any further results I get.  For now this is not a C++ problem, but a LAN problem.
0
msbinderAuthor Commented:
Update:

The extra sockets were definitely not related to this problem.  From the tracing, it can be seen that the message is received by the Client which sends back an acknowledgement.  The acknowledgement is never received by the server, and so the client appears to be dumping the data.

Since this is a CGI, the socket can't be blocked.  Also, the socket library is used by other applications at my company which are located on UNIX machines.  I'm using the library for portability and reusability reasons so it does put some restrictions on the amount of changes I can implement.  Right now I believe the UNIX set up for listen(), select(), send(), recv() et. al. are probably not entirely portable to Winsock.  I 'believe' because I have no proof one way or the other ... yet ;-)

If someone can definitively point out these differences and post settings that work, then the points are yours.
0
MarthiCommented:
If as you said, the code uses listen(), select(), send() and recv()... you should have no problems porting to Winsock.

What you should be careful about is whether you use ioctl() and setsockopt(). If you do those, look carefully at the Winsock APIs, ioctlsocket() and setsockopt() before porting.

Make sure that select() does not does not poll pipes and sockets. The select() in Winsock supports polling of sockets alone. Socket descriptors in Winsock are not the same as file descriptors, rather they are structures.

If you have no problems in the two points I mentioned above, you should be able to port smoothly (almost without change) to Winsock.

You may want to have a copy of one of these books before starting the port
Windows Network Programming - Pat Bonner
            OR
Windows Network Programming - David Shute
0
msbinderAuthor Commented:
The problem is resolved.  It turns out that the Server was getting into an asynchronous race condition while doing a recv().  The code is very similar to what Marthi posted above.  The problem is that the loop with the recv(), for messages that require more than one 1460 block while using a non-buffering scheme with an INET socket, can read the buffer faster than it can be filled. IOW the recv() occasionally returns a 0 (zero) for bytes read.  The code in our library and what Marthi posted above for that matter will see this situation as an error when in fact it isn't.

My solution was to create a user configurable counter on the recv() loop. The loop increments the counter if no data is present, resets the counter if data is read, and only returns an error if the max counter value is exceeded.

Note: putting a select() on the recv() loop doesn't help since the select() will see a message (thus always returning true), but is not aware of the buffering scheme.

0
MarthiCommented:
msbinder,

It is true that the code I posted returns when a "zero read" occurs. Usually, a "zero read" means the connection was closed by the client. If not, recv should simply block until it gets a positive number of bytes from the client. If recv() returned a -ve number, then an error occured (may be a bad socket or some other).

The above assumption is justified if you look at how Stevens does send() and recv() in his famous "UNIX Network programming".

HOWEVER, the one thing I missed in my post was that recv() is a blocking call. A blocking socket call can be interrupted and it will set last error to WSAEINTR. If you encountered a "zero-length read" and WSAGetLastError() was WSAEINTR, you should go back into the recv() call without returning (as I did in the code I posted).

I suggest that you consider the following code

int CSocketConnection::processReceive(char *buffer, int bufferSize)

{
int bytesRecd = 0;

while(bytesRecd < bufferSize)
{
  int n = recv(_socketId, (char *)&buffer[bytesRecd], (bufferSize - bytesRecd), 0);
  if (n < 0)
  {
    cout << "Error from recv() " << WSAGetLastError() << endl << flush;
break;
  }
  else
  {
    if ( (n == 0) && (WSAGetLastError() == WSAEINTR) )
    {
      n = 0;
    }
  }
  bytesRecd += n;
}

return bytesRecd;
}

This condition is predominant when doing recv()s because the call is likely to block longer. However, it would be a good idea to have similar code on the send() too.

Cheers,
Kailash

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
msbinderAuthor Commented:
While your answer isn't the exact answer (we use non-blocking sockets) it's close enough and it's time to close this question.  Thanks for you help!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.