Solved

http upload and base64

Posted on 2002-06-16
3
183 Views
Last Modified: 2010-08-05
Hi

I am writing a prgoram to send a file to a server using http connection. becos of codinf format, I have used Base64 to convert the file to Base 64 format before I send and then the server will decode it back.

But it workd fine for size up to 50k.  When it is more than that it does not work.  Does anyone know why. Or anyone can recommend me a method can can be use to send up to 100M of file to the server??

client side :

arg[0]="intest.jar";

File tmp = File.createTempFile(file,".tmp");
tmp.deleteOnExit();

                java.io.FileInputStream fis = new java.io.FileInputStream( file );
                byte[] rbytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rbytes.length + 1 ];
                    System.arraycopy( rbytes,0, temp,0,rbytes.length );
                    temp[rbytes.length] = (byte)b;
                    rbytes = temp;
                }  
                fis.close();

                java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
                Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
                b64os.write( rbytes );
                b64os.close();


Server side:

       InputStream fis = req.getInputStream();

                byte[] rebytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rebytes.length + 1 ];
                    System.arraycopy( rebytes,0, temp,0,rebytes.length );
                    temp[rebytes.length] = (byte)b;
                    rebytes = temp;
                }  
                fis.close();
               
                java.io.FileOutputStream fos = new java.io.FileOutputStream("c:\\infile\\newtemp.jar");
                Base64.OutputStream b64os = new Base64.OutputStream( fos, false);
                b64os.write( rebytes );
                b64os.close();



I am using the source code form source forge.

0
Comment
Question by:TLTEO
3 Comments
 
LVL 92

Expert Comment

by:objects
ID: 7082824
Can you define exactly how it does not work?
0
 
LVL 9

Accepted Solution

by:
Venci75 earned 100 total points
ID: 7083088
try not to read the whole data in the memory:

....
java.io.FileInputStream fis = new java.io.FileInputStream( file );
byte[] temp = new byte[2048]; // 2k buffer
java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
int b = -1;
while( (b = fis.read(temp)) >= 0 ){
  b64os.write(temp, 0, b);
}  
fis.close();
b64os.close();

....
0
 
LVL 3

Expert Comment

by:mjzalewski
ID: 7091275
The code might simply be timing out on whatever connection you use to write the Base64 encoded stream.

Each iteration of the loop makes a copy of the byte array, but allocates one addional byte. Then it copies the n -1 bytes from the old array to the new array.

So by the time you get to a file that is 50K, you will have copied approximately 2.5GB, and allocated 50K byte[] objects. You also initialize those 50K byte[] objects with zeros, another 2.5GB bytes which must be moved through the processor. And I think allocating 50K byte[] objects is actually where all the time would be spent.

Such an algorithm is unlikely to time out when the file is as small as 50K. But I think it is likely to take several seconds to maybe a minute on a Intel Pentium 1 GHz setup. And when the file to be transferred is 100M, you will need to copy about 10^16 bytes, which will take more than a million times as long.

In short, I recommend an approach like suggested by Venci75. Allocate a single buffer in the range of 1K to maybe 10K bytes. Use a loop something like what he suggested in the previous message.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to get all the API from website? 11 123
Weekend adv creator 3 64
how to see all occupied ports on windows 10 laptop 15 82
Coding for the first time 9 62
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
Viewers will learn about the different types of variables in Java and how to declare them. Decide the type of variable desired: Put the keyword corresponding to the type of variable in front of the variable name: Use the equal sign to assign a v…

713 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question