handling file upload

Hi
I have problem with handling binary file upload
this is the form I use to upload a file

   <form action="http://localhost/cgi-bin/a.exe" method="post" enctype="multipart/form-data">
     <input type="file" name="F1" size="20" /><br />
     <input type="submit" name="submit" value="Upload" />
   </form>

and here is my C++ cgi to handle upload

   #include <iostream>
   #include <fstream>
   using namespace std;

   int main() {
         char c;
      
         ofstream fout("echo.txt", ios::binary);
         while(cin.get(c))
            fout.put(c);
         fout.close();
      
         cout << "Content-Type:text/html;charset=iso-8859-1\r\n\n";
         cout << "<b>HELLO</b>";
         return 0;
   }

I wrote this app just to see what will be sent to server from the user-agent (browser)
it works well when uploading a text file and outputs somthing like this

   -----------------------------3092210723600
   Content-Disposition: form-data; name="F1"; filename="file.txt"
   Content-Type: text/plain

   TEXT CONTENT GOES HERE

   -----------------------------3092210723600
   Content-Disposition: form-data; name="submit"

   Upload
   -----------------------------3092210723600--

as you see, everything goes well when uploading text/plain content
but when I try to upload binary content like exe or gif files, the CGI wont be able to get the file completely
I mean for example if the content of binary file is 40 KB, only the first 500 bytes (this amount is different depending on the file's content) will be read from stdin
the output would be like

   -----------------------------1469913527203
   Content-Disposition: form-data; name="F1"; filename="image.gif"
   Content-Type: image/gif

   INCOMPLETE BINARY DATA

as you see it does not even close the boundary with -----------------------------1469913527203--
please help me to solve this problem
LVL 14
hoomanvAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ahoffmannCommented:
I guess that you HTTP body containing the POST data shown in the question are seen on the browser, then the missing data is a browser problem not one on the server
hoomanvAuthor Commented:
im using apache2 webserver
the same problem occurs for both IE and firefox
if it was a browser problem so i whould have problem with all sites when I upload file to them
I also wrote a PHP script to handle upload but the same problem occured for binary data

does apache needs any configurations for this matter ?
please let me know what is the state of running my code on your own webservers ?
ahoffmannCommented:
> does apache needs any configurations for this matter ?
no

what is the first Content-Type in your HTTP header?
Best you check with the LiveHTTPheader extension for firefox.
Become a CompTIA Certified Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

hoomanvAuthor Commented:
where can i find LiveHTTPheader ?
currenlty im using Tamper Data Extension and I've found that the firefox sends the data completely through the POST
so the problem is within my CGI. what may be wrong with it ?
hoomanvAuthor Commented:
is it possible that cin.get(c) encounters EOF while there is still somthing to read from stdin ?
ahoffmannCommented:
> where can i find LiveHTTPheader ?
http://livehttpheaders.mozdev.org/
http://www.ieinspector.com/httpanalyzer/       

> I've found that the firefox sends the data completely through the POST ..
that's why I'm asking where you have prooven this. I doubt that the request contains all data.

> is it possible that cin.get(c)  ..
when it get's an EOF, then there'se no more to read on STDIN, either 'cause the browser does not send it Or 'cause the web server did not give you more.

Check in your web servers access_log if the size of the request is reliable.
hoomanvAuthor Commented:
> I doubt that the request contains all data
I checked the request content with LiveHTTPheader and it contained full data

> Check in your web servers access_log if the size of the request is reliable.
heres the request log in access.log
127.0.0.1 - - [19/Apr/2006:16:59:38 +0330] "POST /cgi-bin/a.exe HTTP/1.1" 200 12

so ?
ahoffmannCommented:
ok, thought that the access_log contains the size of the request ...
I guess that you have a limitation for posted data, either in httpd.conf (LimitRequestBody) or in php.ini (upload_max_filesize, not sure if there're more such things).
hoomanvAuthor Commented:
> I guess that you have a limitation for posted data either in httpd.conf or in php.ini
http.conf does not have such option
in php.ini , upload_max_filesize = 2M, but I have trouble uploading 40 KB files too

I dont think there is a limitation because I can send plain/text no matter how big it is
Artysystem administratorCommented:
cin.get(c) is for line input, not for char input
will get you one text _line_ from stdin (it should be text CRLF terminated, not binary data with more then 512 bytes in a 'line')

You may read char-by-char with cin.getc(&c)
ahoffmannCommented:
dooh, should look at the c++ library functions more often ... good eyes Nopius.
hoomanvAuthor Commented:
experts !!! ;-)

I found that this piece of code would be in trouble if I dont explicitly open the stream with ios::binary

   char ch;
   ifstream fin("binaryfile"); // wrong
   // ifstream fin("binaryfile", ios::binary);
   while( fin.get(ch) )
     cout << ch;
   fin.close();      

now I think you know what cause the problem !! cin has not been opened with ios::binary
any suggest ? how to read binary from cin ?
ahoffmannCommented:
> .. cin has not been opened
then you cannot read anything!
use getc() as Nopius suggested.
hoomanvAuthor Commented:
> use getc() as Nopius suggested.

here is the C counterpart of my code

#include <stdio.h>

int main()
{
      int c;
      
      FILE *fout = fopen("echo.txt", "wb");
      while((c = getc(stdin)) != EOF) {
            putc(c, fout);
      }
      fclose(fout);
      
      printf("Content-Type:text/html;charset=iso-8859-1\r\n\n");
      printf("<b>Hello</b>");
      
      return 0;
}

try running it in commandline like: a.exe < "binaryfile"
you can see it fails too
Artysystem administratorCommented:
hoomanv: your first code was C++, now you are writing in C. Please specify what language we are talking about.
Artysystem administratorCommented:
in C:
#include <fcntl.h>
#include <unistd.h>
int
main()
{
  int c;
  int fd;
  fd=open("echo.txt", O_CREAT | O_RDWR);
  while (read(0, &c, 1)) write(fd, &c, 1);
  return 0;
};
hoomanvAuthor Commented:
> your first code was C++, now you are writing in C. Please specify what language we are talking about.
do you think it does matter ?

Nopius, your code failed too.
Artysystem administratorCommented:
I've just compiled it and tested: a.exe < "binaryfile"
 What exactly doesn't work?
hoomanvAuthor Commented:
> What exactly doesn't work?
(Im using minGW gcc compiler)
when I pass it a 2-3 MB binary file it just prints out 160 bytes.
it outputs different amount for different files
but for text/plain content it works correctly
Artysystem administratorCommented:
Please copy-paste your latest code version here
hoomanvAuthor Commented:
the latest code is exactly what you gave me

#include <fcntl.h>
#include <unistd.h>
int
main()
{
  int c;
  int fd;
  fd=open("echo.txt", O_CREAT | O_RDWR);
  while (read(0, &c, 1)) write(fd, &c, 1);
  return 0;
};
Artysystem administratorCommented:
Yes, I see problem on Windows MinGW and no problems on *BSD/*NIX...
hoomanvAuthor Commented:
would be nice if I tell you when I compiled my first C++ code with borland compiler the output size increased to 140 KB out of 3.5 MB but still not complete.
hoomanvAuthor Commented:
this problem impacts PHP too. I cant handle [binary] file upload neither with PHP nor with CGI
??????????? any suggestion ??????????
Artysystem administratorCommented:
This hack is specially for Windows MinGW :)

#include <fcntl.h>
#include <unistd.h>
int
main()
{
  char c;
  int fd;
  int fd2;

  fd=open("echo.txt", O_CREAT | O_BINARY | O_RDWR );
  setmode(0, O_BINARY);
   
  while (read(0, &c, 1)) write(fd, &c, 1);
  close(fd);
  return 0;
};
Artysystem administratorCommented:
still somethong is wrong, file size is OK now, but 3 bytes are different :-(
hoomanvAuthor Commented:
yeah that worked

mine was exactly the same (no 3 extra byte)

but do you know how to set binary mode for cin in c++ ?
Artysystem administratorCommented:
read/write is the lowest possible level.
Problem will probably persist in C++ and in stream functions.
My guess: try another compiler...
Artysystem administratorCommented:
I did it :)
Read 2 bytes, not 1.

#include <fcntl.h>
#include <unistd.h>
int
main()
{
  int c, cnt;
  int fd;

  fd=open("echo.txt", O_CREAT | O_BINARY | O_RDWR );
  setmode(0, O_BINARY);
 
  while (cnt=read(0, &c, 2)) write(fd, &c, cnt);
  close(fd);
  return 0;
};

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Artysystem administratorCommented:
This works better:
while (cnt=read(0, &c, sizeof(int))) write(fd, &c, cnt);
hoomanvAuthor Commented:
ok thank you
Artysystem administratorCommented:
Now if you prefer C++, it doesn't matter,  just use

setmode(0, O_BINARY);

Then do input and output (by pieces of >= 2 chars)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Scripting Languages

From novice to tech pro — start learning today.