Solved

http upload and base64

Posted on 2002-06-16
3
181 Views
Last Modified: 2010-08-05
Hi

I am writing a prgoram to send a file to a server using http connection. becos of codinf format, I have used Base64 to convert the file to Base 64 format before I send and then the server will decode it back.

But it workd fine for size up to 50k.  When it is more than that it does not work.  Does anyone know why. Or anyone can recommend me a method can can be use to send up to 100M of file to the server??

client side :

arg[0]="intest.jar";

File tmp = File.createTempFile(file,".tmp");
tmp.deleteOnExit();

                java.io.FileInputStream fis = new java.io.FileInputStream( file );
                byte[] rbytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rbytes.length + 1 ];
                    System.arraycopy( rbytes,0, temp,0,rbytes.length );
                    temp[rbytes.length] = (byte)b;
                    rbytes = temp;
                }  
                fis.close();

                java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
                Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
                b64os.write( rbytes );
                b64os.close();


Server side:

       InputStream fis = req.getInputStream();

                byte[] rebytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rebytes.length + 1 ];
                    System.arraycopy( rebytes,0, temp,0,rebytes.length );
                    temp[rebytes.length] = (byte)b;
                    rebytes = temp;
                }  
                fis.close();
               
                java.io.FileOutputStream fos = new java.io.FileOutputStream("c:\\infile\\newtemp.jar");
                Base64.OutputStream b64os = new Base64.OutputStream( fos, false);
                b64os.write( rebytes );
                b64os.close();



I am using the source code form source forge.

0
Comment
Question by:TLTEO
3 Comments
 
LVL 92

Expert Comment

by:objects
ID: 7082824
Can you define exactly how it does not work?
0
 
LVL 9

Accepted Solution

by:
Venci75 earned 100 total points
ID: 7083088
try not to read the whole data in the memory:

....
java.io.FileInputStream fis = new java.io.FileInputStream( file );
byte[] temp = new byte[2048]; // 2k buffer
java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
int b = -1;
while( (b = fis.read(temp)) >= 0 ){
  b64os.write(temp, 0, b);
}  
fis.close();
b64os.close();

....
0
 
LVL 3

Expert Comment

by:mjzalewski
ID: 7091275
The code might simply be timing out on whatever connection you use to write the Base64 encoded stream.

Each iteration of the loop makes a copy of the byte array, but allocates one addional byte. Then it copies the n -1 bytes from the old array to the new array.

So by the time you get to a file that is 50K, you will have copied approximately 2.5GB, and allocated 50K byte[] objects. You also initialize those 50K byte[] objects with zeros, another 2.5GB bytes which must be moved through the processor. And I think allocating 50K byte[] objects is actually where all the time would be spent.

Such an algorithm is unlikely to time out when the file is as small as 50K. But I think it is likely to take several seconds to maybe a minute on a Intel Pentium 1 GHz setup. And when the file to be transferred is 100M, you will need to copy about 10^16 bytes, which will take more than a million times as long.

In short, I recommend an approach like suggested by Venci75. Allocate a single buffer in the range of 1K to maybe 10K bytes. Use a loop something like what he suggested in the previous message.
0

Featured Post

ScreenConnect 6.0 Free Trial

Want empowering updates? You're in the right place! Discover new features in ScreenConnect 6.0, based on partner feedback, to keep you business operating smoothly and optimally (the way it should be). Explore all of the extras and enhancements for yourself!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
servlet doXXX methods 3 49
Should localization be done inside spring controller 5 26
map related example 6 49
Facing this issue for maven proxy setting 2 12
For customizing the look of your lightweight component and making it look opaque like it was made of plastic.  This tip assumes your component to be of rectangular shape and completely opaque.   (CODE)
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Viewers learn how to read error messages and identify possible mistakes that could cause hours of frustration. Coding is as much about debugging your code as it is about writing it. Define Error Message: Line Numbers: Type of Error: Break Down…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question