Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

http upload and base64

Posted on 2002-06-16
3
182 Views
Last Modified: 2010-08-05
Hi

I am writing a prgoram to send a file to a server using http connection. becos of codinf format, I have used Base64 to convert the file to Base 64 format before I send and then the server will decode it back.

But it workd fine for size up to 50k.  When it is more than that it does not work.  Does anyone know why. Or anyone can recommend me a method can can be use to send up to 100M of file to the server??

client side :

arg[0]="intest.jar";

File tmp = File.createTempFile(file,".tmp");
tmp.deleteOnExit();

                java.io.FileInputStream fis = new java.io.FileInputStream( file );
                byte[] rbytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rbytes.length + 1 ];
                    System.arraycopy( rbytes,0, temp,0,rbytes.length );
                    temp[rbytes.length] = (byte)b;
                    rbytes = temp;
                }  
                fis.close();

                java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
                Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
                b64os.write( rbytes );
                b64os.close();


Server side:

       InputStream fis = req.getInputStream();

                byte[] rebytes = new byte[0];
                int b = -1;
                while( (b = fis.read()) >= 0 ){
                    byte[] temp = new byte[ rebytes.length + 1 ];
                    System.arraycopy( rebytes,0, temp,0,rebytes.length );
                    temp[rebytes.length] = (byte)b;
                    rebytes = temp;
                }  
                fis.close();
               
                java.io.FileOutputStream fos = new java.io.FileOutputStream("c:\\infile\\newtemp.jar");
                Base64.OutputStream b64os = new Base64.OutputStream( fos, false);
                b64os.write( rebytes );
                b64os.close();



I am using the source code form source forge.

0
Comment
Question by:TLTEO
3 Comments
 
LVL 92

Expert Comment

by:objects
ID: 7082824
Can you define exactly how it does not work?
0
 
LVL 9

Accepted Solution

by:
Venci75 earned 100 total points
ID: 7083088
try not to read the whole data in the memory:

....
java.io.FileInputStream fis = new java.io.FileInputStream( file );
byte[] temp = new byte[2048]; // 2k buffer
java.io.FileOutputStream fos = new java.io.FileOutputStream(tmp);
Base64.OutputStream b64os = new Base64.OutputStream( fos, true );
int b = -1;
while( (b = fis.read(temp)) >= 0 ){
  b64os.write(temp, 0, b);
}  
fis.close();
b64os.close();

....
0
 
LVL 3

Expert Comment

by:mjzalewski
ID: 7091275
The code might simply be timing out on whatever connection you use to write the Base64 encoded stream.

Each iteration of the loop makes a copy of the byte array, but allocates one addional byte. Then it copies the n -1 bytes from the old array to the new array.

So by the time you get to a file that is 50K, you will have copied approximately 2.5GB, and allocated 50K byte[] objects. You also initialize those 50K byte[] objects with zeros, another 2.5GB bytes which must be moved through the processor. And I think allocating 50K byte[] objects is actually where all the time would be spent.

Such an algorithm is unlikely to time out when the file is as small as 50K. But I think it is likely to take several seconds to maybe a minute on a Intel Pentium 1 GHz setup. And when the file to be transferred is 100M, you will need to copy about 10^16 bytes, which will take more than a million times as long.

In short, I recommend an approach like suggested by Venci75. Allocate a single buffer in the range of 1K to maybe 10K bytes. Use a loop something like what he suggested in the previous message.
0

Featured Post

Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Apps blocked by Java 9 88
ejb example issues 3 27
Facing this issue for maven proxy setting 2 20
ejb entity bean example issue 2 17
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Java contains several comparison operators (e.g., <, <=, >, >=, ==, !=) that allow you to compare primitive values. However, these operators cannot be used to compare the contents of objects. Interface Comparable is used to allow objects of a cl…
Viewers learn about the third conditional statement “else if” and use it in an example program. Then additional information about conditional statements is provided, covering the topic thoroughly. Viewers learn about the third conditional statement …
This video teaches viewers about errors in exception handling.

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question