Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 187
  • Last Modified:

http upload and base64

Hi

I am writing a prgoram to send a file to a server using http connection. becos of codinf format, I have used Base64 to convert the file to Base 64 format before I send and then the server will decode it back.

But it workd fine for size up to 50k.  When it is more than that it does not work.  Does anyone know why. Or anyone can recommend me a method can can be use to send up to 100M of file to the server??

client side :

arg[0]="intest.jar";

File tmp = File.createTempFile(file,".tmp");
tmp.deleteOnExit();

                java.io.FileInputStream fis = new java.io.FileInputStream( file );
                byte[] rbytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rbytes.length + 1 ];
                    System.arraycopy( rbytes,0, temp,0,rbytes.length );
                    temp[rbytes.length] = (byte)b;
                    rbytes = temp;
                }  
                fis.close();

                java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
                Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
                b64os.write( rbytes );
                b64os.close();


Server side:

       InputStream fis = req.getInputStream();

                byte[] rebytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rebytes.length + 1 ];
                    System.arraycopy( rebytes,0, temp,0,rebytes.length );
                    temp[rebytes.length] = (byte)b;
                    rebytes = temp;
                }  
                fis.close();
               
                java.io.FileOutputStream fos = new java.io.FileOutputStream("c:\\infile\\newtemp.jar");
                Base64.OutputStream b64os = new Base64.OutputStream( fos, false);
                b64os.write( rebytes );
                b64os.close();



I am using the source code form source forge.

0
TLTEO
Asked:
TLTEO
1 Solution
 
objectsCommented:
Can you define exactly how it does not work?
0
 
Venci75Commented:
try not to read the whole data in the memory:

....
java.io.FileInputStream fis = new java.io.FileInputStream( file );
byte[] temp = new byte[2048]; // 2k buffer
java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
int b = -1;
while( (b = fis.read(temp)) >= 0 ){
  b64os.write(temp, 0, b);
}  
fis.close();
b64os.close();

....
0
 
mjzalewskiCommented:
The code might simply be timing out on whatever connection you use to write the Base64 encoded stream.

Each iteration of the loop makes a copy of the byte array, but allocates one addional byte. Then it copies the n -1 bytes from the old array to the new array.

So by the time you get to a file that is 50K, you will have copied approximately 2.5GB, and allocated 50K byte[] objects. You also initialize those 50K byte[] objects with zeros, another 2.5GB bytes which must be moved through the processor. And I think allocating 50K byte[] objects is actually where all the time would be spent.

Such an algorithm is unlikely to time out when the file is as small as 50K. But I think it is likely to take several seconds to maybe a minute on a Intel Pentium 1 GHz setup. And when the file to be transferred is 100M, you will need to copy about 10^16 bytes, which will take more than a million times as long.

In short, I recommend an approach like suggested by Venci75. Allocate a single buffer in the range of 1K to maybe 10K bytes. Use a loop something like what he suggested in the previous message.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now