Solved

http upload and base64

Posted on 2002-06-16
3
185 Views
Last Modified: 2010-08-05
Hi

I am writing a prgoram to send a file to a server using http connection. becos of codinf format, I have used Base64 to convert the file to Base 64 format before I send and then the server will decode it back.

But it workd fine for size up to 50k.  When it is more than that it does not work.  Does anyone know why. Or anyone can recommend me a method can can be use to send up to 100M of file to the server??

client side :

arg[0]="intest.jar";

File tmp = File.createTempFile(file,".tmp");
tmp.deleteOnExit();

                java.io.FileInputStream fis = new java.io.FileInputStream( file );
                byte[] rbytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rbytes.length + 1 ];
                    System.arraycopy( rbytes,0, temp,0,rbytes.length );
                    temp[rbytes.length] = (byte)b;
                    rbytes = temp;
                }  
                fis.close();

                java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
                Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
                b64os.write( rbytes );
                b64os.close();


Server side:

       InputStream fis = req.getInputStream();

                byte[] rebytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rebytes.length + 1 ];
                    System.arraycopy( rebytes,0, temp,0,rebytes.length );
                    temp[rebytes.length] = (byte)b;
                    rebytes = temp;
                }  
                fis.close();
               
                java.io.FileOutputStream fos = new java.io.FileOutputStream("c:\\infile\\newtemp.jar");
                Base64.OutputStream b64os = new Base64.OutputStream( fos, false);
                b64os.write( rebytes );
                b64os.close();



I am using the source code form source forge.

0
Comment
Question by:TLTEO
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 92

Expert Comment

by:objects
ID: 7082824
Can you define exactly how it does not work?
0
 
LVL 9

Accepted Solution

by:
Venci75 earned 100 total points
ID: 7083088
try not to read the whole data in the memory:

....
java.io.FileInputStream fis = new java.io.FileInputStream( file );
byte[] temp = new byte[2048]; // 2k buffer
java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
int b = -1;
while( (b = fis.read(temp)) >= 0 ){
  b64os.write(temp, 0, b);
}  
fis.close();
b64os.close();

....
0
 
LVL 3

Expert Comment

by:mjzalewski
ID: 7091275
The code might simply be timing out on whatever connection you use to write the Base64 encoded stream.

Each iteration of the loop makes a copy of the byte array, but allocates one addional byte. Then it copies the n -1 bytes from the old array to the new array.

So by the time you get to a file that is 50K, you will have copied approximately 2.5GB, and allocated 50K byte[] objects. You also initialize those 50K byte[] objects with zeros, another 2.5GB bytes which must be moved through the processor. And I think allocating 50K byte[] objects is actually where all the time would be spent.

Such an algorithm is unlikely to time out when the file is as small as 50K. But I think it is likely to take several seconds to maybe a minute on a Intel Pentium 1 GHz setup. And when the file to be transferred is 100M, you will need to copy about 10^16 bytes, which will take more than a million times as long.

In short, I recommend an approach like suggested by Venci75. Allocate a single buffer in the range of 1K to maybe 10K bytes. Use a loop something like what he suggested in the previous message.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
By the end of 1980s, object oriented programming using languages like C++, Simula69 and ObjectPascal gained momentum. It looked like programmers finally found the perfect language. C++ successfully combined the object oriented principles of Simula w…
Viewers will learn about the regular for loop in Java and how to use it. Definition: Break the for loop down into 3 parts: Syntax when using for loops: Example using a for loop:
This video teaches viewers about errors in exception handling.

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question