Solved

receiving-server

Posted on 2011-03-24
41
357 Views
Last Modified: 2012-05-11
on Server.c from line 115 to 126, how can know it finishes reading all trunk of the file? do I check the return of receiveMessage() to determine it? tks.

>>the message 2 gives filesize.

I thought about reading enough until reach filesize. however, i got stuck last night because my receivMessage return the the data string of file content only . hihi. So thought It couldn't return the filesize.

 So from the message2, it return filsize. So I need a loop to read until it reach to the filesize. Am i right? let me work on it now. tks.
0
Comment
Question by:BeginToLearn
  • 16
  • 11
  • 8
  • +1
41 Comments
 
LVL 45

Expert Comment

by:Kdo
ID: 35207143
Hi Begin,

I'm more concerned with the code around line 53.

The program reads an integer from the socket (size_to_read) which is the file length.  You then try to to read that many bytes from the network into buffer pch.  There is no testing to ensure that (size_to_read) is smaller than the buffer.


Kent
0
 

Author Comment

by:BeginToLearn
ID: 35207235
Let me review code around line 53. tks.
0
 

Author Comment

by:BeginToLearn
ID: 35207273
just updated. is the code on right track now? I just want to focus on the logic first before i run and debug. tks.
server.c
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35207341
Before submitting, you need to test. Otherwise, we are simply giving you pointers without you doing the real work. Be careful about using sizeof, which is the number of bytes that an object takes up.

>> unsigned int size_to_read
>> if(n < sizeof (size_to_read))

sizeof (size_to_read) is probably 4 since size_to_read is an unsigned int.

Does it make sense that your general receiveMessage() function tests for n < 4?
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35207556
You probably should include your latest client code, since this is what I have from previous question:
 sendMessage():
   
     char pch[BUFFER_LEN]= {'\0'};
    ...
     n =send (sock, pch, a.size(), 0);

    if(n != sizeof(pch))
    {	
        error("Error in sending data buffer");
    }

Open in new window

Then sizeof(pch) is BUFFER_LEN. For some messages, n < BUFFER_LEN, and you exit. So, assuming that you ran this code, I must have the wrong client code.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35207571
>> I just want to focus on the logic first before i run and debug.
I think you should focus on the logic, and when satsified, you should run and debug. If you then are stuck, you can ask a question.
0
 
LVL 45

Expert Comment

by:Kdo
ID: 35207655
Hi Begin,

At some point you'll want to change the way that you compute how long the length field will be.

  sizeof (int)

As long as you control the source code and both ends are compiled on equal systems, this works fine.  But if you ever "release" the code so that anyone can compile the client to work with your server you'll run the risk of the client being run on a 16 or (more likely) 64-bit system.  That will create a mismatch that just won't work.

But for now, sizeof(int) will work just fine!


Kent
0
 

Author Comment

by:BeginToLearn
ID: 35207670
In the client, at send data inside sendMessage(), we need to check the size of data sending with size of the send() return in order to make sure it sends enough size of data. so i think the code should like this:
       n =send (sock, pch, a.size(), 0);
      if(n != a.size())
      {      
            error("Error in sending data buffer");
      }

Am i right?
0
 
LVL 45

Assisted Solution

by:Kdo
Kdo earned 100 total points
ID: 35207711

That's probably a good idea.  :)

And it looks right.
0
 

Author Comment

by:BeginToLearn
ID: 35207775
hihi. So similar in server side,

      n = read( sock, pch, sizeof(pch));
      if(n < sizeof (pch))
      {      
            error("Error in receiving data buffer");
      }
Am i right this time?
0
 
LVL 45

Expert Comment

by:Kdo
ID: 35207851

I like it.  :)
0
 

Author Comment

by:BeginToLearn
ID: 35207883
tks a lot. I can start debugging now. However, i need to go to work now. I gonna be back 9 hrs later. sorry .SO exhausted . every i had code in my dream.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35207917
>> char  pch[BUFFER_LEN]
then  sizeof(pch) is BUFFER_LEN

>> n = read( sock, pch, sizeof(pch));
For shorter messages the number of bytes read will be less than BUFFER_LEN, but then

>>  if(n < sizeof (pch))
becomes
       if(n < BUFFER_LEN )
              error()

and you get an error.

If you take 1 minute to run your program in debugger, you will find the answers to your questions.
0
 
LVL 32

Expert Comment

by:sarabande
ID: 35208303
begintolearn,

you have to read message1, message2, message3.

any message contains of size_to_read (integer), message_id (integer), textdata.

only the textdata has variable size which you know because it was passed in size_to_read.

so you have 3 kinds of reading

   n = read(sock, &size_to_read, sizeof(int));

   n = read(sock, &message_id, sizeof(int));

   n = read(sock, pch, size_to_read);

and you not necessarily have to expect more or less bytes for any of the reads (cause you've written both client and server yourself).

but the tcpip message is not guaranteed to be received in total. you could get only parts of those. that means the n can be less than the expected length and you would get the remainder with a second or even third read.

i posted a code snippet of how to handle that in one of your previous q. you should use that at least for the message 3 cause a 2k buffer is not granted to get received in total.

Sara
0
 
LVL 32

Expert Comment

by:sarabande
ID: 35208363
when reading message 3 you should add all received size_to_read and check that sum against the filesize you got with message 2. that will tell you when you can stop reading message 3. same the number of files got by message 1 will tell you when to stop to read new files.

you should know that if you do more reads than for that what was sent you finally will be blocked (hang).

Sara
0
 

Author Comment

by:BeginToLearn
ID: 35208398
I dont have computer aces now.pls tell me any hiden error so I xan work on them once  I get home.tks
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35208412
When you get home, I think I'll be around to see if you have questions on your code.
0
 
LVL 32

Expert Comment

by:sarabande
ID: 35208454
please post your code (both client and server) again here in the thread cause new participients may not  know where your latest sources can be received from.

Sara
0
 

Author Comment

by:BeginToLearn
ID: 35208995
I will post them soon once I get access to my pc.sorry
0
 

Author Comment

by:BeginToLearn
ID: 35210790
I just review all comments during breaktime,i can see the main issue is message3 data can vary.let me think a way to fix it.tks
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 45

Expert Comment

by:Kdo
ID: 35211340
Hi Begin,

On the server side, all you really need to do is keep track of a few things (status, etc.) and the entire code writes very small.

Check out the sample below.  It should be a good skeleton and is very easily extendable.


Kent
enum
{
  EXPECT_BLOCKTYPE,
  EXPECT_BLOCKLENGTH,
  EXPECT_DATA
};

#define BUFFERLENGTH 2048

  int Function;
  int ExpectedLength;
  int FullLength;
  int DataLength;
  int ReadLength;
  int BlockType;
  char *Buffer;

  Function = EXPECT_BLOCKTYPE;
  Buffer = (char*) malloc (BUFFERLENGTH);
  ExpectedLength = sizeof (int);
  while (1)
  {
    ReadLength = (sock, &Buffer, ExectedLength);
    if (ReadLength != ExpectedLength)
    {
      perror ("Read failure.");
      exit (1);
    }
    switch (Function)
    {
      case EXPECT_BLOCKTYPE:
        BlockType = *(int*)Buffer;
        fprintf (stderr, " blocktype: %d\n", BlockType);
        ++Function;
        break;
        
      case EXPECT_BLOCKLENGTH:
        FullLength = *(int*)Buffer;
        fprintf (stderr, " blocklength: %d\n", BlockLength);
        ++Function;
        break;
        
      case EXPECT_DATA:
        // Do something with the data
        fprintf (stderr, " readlength: %d\n", ReadLength);
        if (ReadLength == 0 && ExpectedLength == 0)
        {
          Function = EXPECT_BLOCKTYPE;
          ExpectedLength = sizeof (int);
          break;
        }
        ExpectedLength = FullLength > BLOCKLENGTH ? BLOCKLENGTH : FullLength;
        ExpectedLength -= FullLength;
        break;
      default:
        perror ("Block type unknown.");
        exit (1);
    }
  }

Open in new window

0
 

Author Comment

by:BeginToLearn
ID: 35212410
Just got home. While driving , i think i should use the length of buffer and multiply to size of each element in order to get size of data in message3. Am i right? Anyway let me marinate pork then come back in immediately. I apply multitasking now :)
0
 

Author Comment

by:BeginToLearn
ID: 35212556
here is my code up-to-date. It can transfer 5/6 files and those new files written to server have wrong size. on server terminal i got this error message:
  received wrong size_to_read: Success
Now i can start debugging process now :)
client.c
server.c
0
 

Author Comment

by:BeginToLearn
ID: 35212821
I have a few question need to clarify:
   + in client.c  around line 126,
              while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0))
 how can result is always equal to 1
   + when we deal with sizeof( buffer), should we minus 1 because of the NULL character. We need NULL character just for other functions to know where to terminate. However, the content of buffer, which need to write to file or using to increase nread, should exclude the last NULL character.

  Please advise.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35212888
CLIENT:
Line 126: >>  how can result is always equal to 1?
I believe I tried to help you on this here http://rdsrc.us/7RsjEk and and in here http://rdsrc.us/DkpnHB

>> when we deal with sizeof( buffer), should we minus 1 because of the NULL character
I found one case of sizeof( buffer) in the client:

Line 138: >> memset(buffer, '\0',sizeof(buffer)*sizeof(buffer[0])) ;
When trying to completely zero out buffer, all you need to do is:
                     memset(buffer, '\0', sizeof(buffer) ) ;
If your buffer ever became an array of long, then your original memset would overflow your buffer when you write 0's.
Right now, your buffer is an array of char, so sizeof(buffer[0]) is 1, so equivalent (for now).

>> should we minus 1 because of the NULL character
You have a buffer, so you may as well use the entire buffer. If you are sending a string, then why not transfer the null byte? Just remember that when you send a binary file, there are likely to be null bytes in the middle of a block, and likely that there will be no null byte at the end - that file data is not a c-style or c++ string. In previous question, I posted that you were convert your file read buffer to a string and then converting it back to a char array. Very inefficient, and if null bytes are in the middle, I wonder what the effect will be when converting to a string.

Don't forget that at the lower layer socket functionality, you are dealing with older libraries that are written in C style code (i.e., use char* and not C++ string). So you wrapper around this code should be also lower level to handle the binary files (and now your char buffer is not a c-style string - it is just an array of 8-bit integers). This lower level function will have to include the size of the file. In C++ you can define a function with the same name but having a string interface instead of a char* interface. Then in this string type function, you just call the lower level function making the adjustments to the lower level api.

I'll be back tomorrow (my timezone is 3 hours different than yours). I believe others start arriving shortly. So, to get their attention, create a new question.
0
 
LVL 32

Expert Comment

by:sarabande
ID: 35213135
the

    memset(buffer, '\0',sizeof(buffer)*sizeof(buffer[0]))

was my fault.

the sizeof already returns number of bytes. so the multiplication is wrong even if it doesn't matter for char arrays.

i mixed things up with the case where you need to compute the number of elements of an array. there you can do:

   int num_elements = sizeof(array)/sizeof(array[0]);

what would work for arrays of any type.

Sara
0
 
LVL 32

Expert Comment

by:sarabande
ID: 35213225
regarding the point whether the text buffer already should contain a terminating zero char or not, i only would like to say that buffers sent via sockets shouldn't have a maximum size 2k + 1 but 2k. an 'odd' size like 2k+1 may not generate problems in nowadays computing but it surely increases the probability that a buffer was sent in two parts instead of one. you should see that it doesn't make any problems to add a zero character at buffer[number_of_bytes_read] by the receiver granted that the buffer has that extra byte left.

the

   while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0))

is wrong. it needs to be

  while ((result= fread (buffer, 1, BUFFER_SEND, fp)) > 0)

only so the result will compared with 0. otherwise you have implicit parantheses like

    result= (fread (buffer, 1, BUFFER_SEND, fp) > 0);

and the result is always 1 (boolean) as long as the fread doesn't fail.

Sara





   

Sara

 
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35215640
>> when we deal with sizeof( buffer), should we minus 1 because of the NULL character
When you said CLIENT, I thought you meant only the client. I took a look at your server and found in receiveOneFile:
#define BUFFER_LEN (BUFFER_SEND + 1)
  char buffer[BUFFER_LEN]= {'\0'}; ...
  string t= receiveMessage(sock);
  strcpy(buffer, t.c_str());
  int result= fwrite (buffer,1,sizeof(buffer),fp);

Open in new window

You know how many bytes you received in Message 3, but you then ignore this number of bytes received when writing to the file. This mismatch will usually cause the transferred file to have a larger size than the original. Once you fix this problem and get a good transfer for text files (that have no null bytes in them), see if the transfer for .exe files works OK. (It used to work OK - in my version - but that was before you converted char* to string and back to char*.)
=============

re: result being -1
When I wrote 4 days ago in http://rdsrc.us/OyC6hV
Put a break on this client line, step through it, and see if you can identify and fix the problem:
>> while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0))
I assumed you did this, and since you didn't get back to me at that time, I thought you fixed it. Looks like I didn't realize you hadn't fixed it. One time when I fixed the parens issue in my version, I still got -1. Anyway, as reported in http://rdsrc.us/DkpnHB , I found it odd that by using the const size_t type instead of #define, then I got the right answer, even though it wasn't necessary in previous versions to remove the #define. That was strange - not sure why that happened. Anyway, even though it may be not necessary, in C++ we stopped using #define's at work for integral types, and also used matching data types.

=============

>> how can know it finishes reading all trunk of the file
  If you do not ignore how many bytes you receive, then you can compare the total number of bytes received in each Message 3 to the total number of bytes of the file (that you know from Message 2).

I double-checked the previous posts. In http:#35208363 sarabande addressed your question. At that point, you were expected to close the question if you understood it, and ask new questions on other issues that crop up (like -1).
0
 

Author Comment

by:BeginToLearn
ID: 35215732
   
      I understand the error in while ((result= fread (buffer, 1, BUFFER_SEND, fp) > 0)) now. It like: read,compare, and assign to result. However, I want is: read, assign, and compare. So Sara pointed out I miss the parentheses and it altered my purpose.
     
 >>>> In previous question, I posted that you were convert your file read buffer to a string and then converting it back to a char array. Very inefficient, and if null bytes are in the middle, I wonder what the effect will be when converting to a string.
    Oh sorry I forgot to mention about this comment. The reason i work around like this way is because of sendMessage(). The third argument in sendMessage() is string a. So i need to convert to match type. About the wrapper i don't know. Let me search about it. If you have a good example for this case, please post the link.
         
     
0
 
LVL 45

Expert Comment

by:Kdo
ID: 35215754
Hi Begin,

You need to switch from sending a string to sending a buffer.  Otherwise you'll be limited to sending ONLY text files.


Kent
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35215757
>> The third argument in sendMessage() is string a.
Couldn't this sendMessage() call another function which is char * oriented? And when you actually have char * as in Message 3, then just use the lower level fuction.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35215822
But, as Infinity08 said in a previous question, if your OP question has been answered, we should move onto a new question. I think we have been too lax in going with the one question per question EE policy; thereby making the PAQ searches less useful.
0
 

Author Comment

by:BeginToLearn
ID: 35215853
>>>You need to switch from sending a string to sending a buffer.  Otherwise you'll be limited to sending ONLY text files.

   true.Lucky me is it transfer "client" file first. Otherwise, I can't catch the problem. Let me work on send buffer based on this idea:
>>   Couldn't this sendMessage() call another function which is char * oriented? And when you actually have char * as in Message 3, then just use the lower level fuction.

0
 
LVL 45

Expert Comment

by:Kdo
ID: 35215895
Hi Begin,

You can, maybe.  If the String uses the Z-terminator common to C, you'll have lost the length before you get to a character based object.


Kent
0
 

Author Comment

by:BeginToLearn
ID: 35215930
let me close this question first and asking new question about that new issue. . this question is  too long now. tks.
0
 
LVL 32

Expert Comment

by:phoffric
ID: 35216263
fyi - I just tested your program (with minor fixes) and at least on Cygwin, the .txt files transferred are true w.r.t. size and content.

But the .exe files although true to size are significantly different in content.

Now, given that you still are working to getting a directory structure transferred, and you have your deadlines, you need to prioritize what you consider most important to work on.
0
 

Author Comment

by:BeginToLearn
ID: 35216311
Oh i need to make it work with all type of files first. I try to finish all of them today and submit. tks a lot.
0
 
LVL 45

Expert Comment

by:Kdo
ID: 35216398
Hi Begin,

By far, the easiest way to debug this is to run both the client and server on the same box (your development system).

Then set the server to write to a relative path.  C:\servertest\....

Then it's easy to start/stop the server and application, verify file length and content, etc. all from your desktop.



Been there, done that....
Kent
0
 
LVL 32

Accepted Solution

by:
sarabande earned 200 total points
ID: 35217438
std::string can very well serve as a buffer. it has a constructor which takes char * and length and has no problems with binary characters.

so you could do

   send( sock, &a[0], a.length(), 0);


but using a fixed sized char buffer of 2k length is a valid concept as well and has the advantage that you could avoid any copies and use the same buffer for read and write. but that only can work when the send function makes a copy of the buffer before returning to the user. as far as i know that is the case for blocking sockets but cause i mostly worked with non-blocking sockets i always used a separate buffer for write and checked with select whether a former send request was done. for the performance it has little importance whether there is a copy of a buffer between reading from file and writing to socket cause the i/o needs so much more time than a memcpy.

there is another reason why the sendMessage should be changed to pass a char buffer rather than a string even if the files to transfer are text files only: when the size of a textfile was determined by fseek and ftell, the size returned by ftell actually is a file position and not the total number of bytes to read from begin. the issue is that windows stores a pair "\r\n"for end-of-line in file but when reading it in text mode it returns only "\n". so the number of bytes effectively read is less than the size calculeted by ftell. same happens when determining the filesize with fstat or stat function. the only senseful way out (you could read the file twice but that rarely is an alternative) is to read (and write) the file in binary mode which has the advantage that by this also binary files can be transfered without difference.

for your client is little to change. simply open the file using "rb" openmode. then change send sendMessage to take a const char * and a length instead of std::string. you can try to use the same buffer directly for send (as shown in sample code of Kent, if i remember rightly). you also could use the current std::string for message 1 and message 2 by simply passing t.c_str() and t.length().

for the server it will be easier. i would suggest to do it analogously to client and call the receiveMessage with a char buffer and size argument which contains the expected size for the next message. you would read from socket in a while loop cause it is not granted that data can be received with one read. you would only return on error or when the expected size could be read.

note, in a later phase you could do a select with timeout before read and thus avoid getting blocked if for any reason the client would not send the number of data expected. but in a first attempt i would rely on the sizes received from previous messages and i am sure it can work.

Sara
0
 
LVL 32

Assisted Solution

by:phoffric
phoffric earned 200 total points
ID: 35217494
ftell:
"For binary streams, the value returned corresponds to the number of bytes from the beginning of the file. For text streams, the value is not guaranteed to be the exact number of bytes from the beginning of the file..."
    http://www.cplusplus.com/reference/clibrary/cstdio/ftell/

So, to get the file length, do as in the example from this link (namely, use "rb").
/* ftell example : getting size of a file */
#include <stdio.h>

int main ()
{
  FILE * pFile;
  long size;

  pFile = fopen ("myfile.txt","rb");
  if (pFile==NULL) perror ("Error opening file");
  else
  {
    fseek (pFile, 0, SEEK_END);
    size=ftell (pFile);
    fclose (pFile);
    printf ("Size of myfile.txt: %ld bytes.\n",size);
  }
  return 0;
}

Open in new window

0
 

Author Comment

by:BeginToLearn
ID: 35217504
>>the issue is that windows stores a pair "\r\n"for end-of-line in file but when reading it in text mode it >>returns only "\n". so the number of bytes effectively read is less than the size calculeted by ftell. same >>happens when determining the filesize with fstat or stat function. the only senseful way out (you could >>read the file twice but that rarely is an alternative) is to read (and write) the file in binary mode which >>has the advantage that by this also binary files can be transfered without difference.

>>for your client is little to change. simply open the file using "rb" openmode. then change send >>sendMessage to take a const char * and a length instead of std::string

I see . let me update my code and close this question. tks a lot.
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now