Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

FTP efficiency...

Posted on 1999-07-23
13
Medium Priority
?
441 Views
Last Modified: 2008-02-01
Hi,
  I am using the sun.net and sun.net.FtpClient
packages.

I have the ftp working but I am getting a pathetic
download rate of 22 KB/sec or so on our LAN.  An
FTP Program downloaded the same files at the speed
of 150-500 KB/sec

Does anyone know how to increase the transfer rate?

Maybe use another method?

Thanks,
CJ
===============================================
Accompanying Code...

      byte buffer[] = new byte[1000000];        
      FtpClient fc = new FtpClient(hostname);
      fc.login(username, password);
      fc.ascii();  
      fc.cd(filepath);
      InputStream theFile = fc.get(filename);
      BufferedInputStream bis = new       BufferedInputStream(theFile,1000000);
      int i;
      FileOutputStream fos = new       FileOutputStream(destination);
      BufferedOutputStream bos =new BufferedOutputStream(fos);
      while ((i = bis.read(buffer)) != -1) {
        bos.write(buffer,0,i);
      }
      fc.closeServer();
      fos.close();
    } catch (IOException e) { System.err.println(e); }
0
Comment
Question by:cheekycj
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 2
  • +2
13 Comments
 
LVL 2

Expert Comment

by:jasbro
ID: 1246965
Unfortunately, if you are using java, you will probably never approach the speed of the ftp program you mentioned.  You can increase the speed somewhat by using the JIT compiler (which is on by default in jdk1.2), and by streamlining your code, although looking at your code, it looks fairly clean.  You may want to look at other output streams for the file write, and in fact you probably don't need the buffered stream for output, you will probably never receive data over the network faster than you can write it to a file...  THe buffered stream adds overhead.
0
 
LVL 16

Expert Comment

by:heyhey_
ID: 1246966
you read the stream byte after byte

while ((i = bis.read(buffer)) != -1) {
 bos.write(buffer,0,i);
}


try something like      

byte[] chunk = new byte[10000];
     
int len = is.read(chunk);
while (len >0)
{
  bos.write(chunk, 0, len);
  len = is.read(chunk);
}

0
 
LVL 19

Author Comment

by:cheekycj
ID: 1246967
Isn't my code doing the same thing?

> byte buffer[] = new byte[1000000];          
> int i;
> while ((i = bis.read(buffer)) != -1) {
>        bos.write(buffer,0,i);
> }

CJ
0
The top UI technologies you need to be aware of

An important part of the job as a front-end developer is to stay up to date and in contact with new tools, trends and workflows. That’s why you cannot miss this upcoming webinar to explore the latest trends in UI technologies!

 
LVL 16

Expert Comment

by:heyhey_
ID: 1246968
it may be quite more time-consuming if your stream is unbuffered ... or if (for example) both read() and read(byte[]) call native methods - that my code will make only one native method call (which is slooow) and your code will have 10000.
0
 
LVL 19

Author Comment

by:cheekycj
ID: 1246969
heyhey
  I tried your way and didn't make a difference.  Any other ideas?
I refuse to believe that java reduces the ftp speed by 10x-20x.  It doesn't
seem logical.

jasbro
  I am rejecting your answer just to field more suggestions.  If I don't get
anything that improves the performance you will get the points.

CJ
0
 
LVL 2

Expert Comment

by:jasbro
ID: 1246970
I have seen some really bad performance from java, especially when doing input and output streams.  Much worse than you would think.  Java keeps all of your data in the most efficient storage places for Java, not for the rest of your computer.  When you decide to go outside of the little Java world to do work (reading from a port, writing to a file, etc)  Java has to map it's own data structures onto corresponding structures for the underlying system, then do the requested action.  (This is what heyhey was talking about with the Native Methods comment)  You are doing this type of thing with each byte, so Java is going through native code to grab a byte, pulling it into the interpreter, pushing it out the other side and writing it to disk.  This is where all of the time expense is.  
  You could access these FTP and net packages through Java Native Interface, then all of the reading and writing would be done in compiled code (not interpreted) and should go much faster.  If you would like more information on doing that, I can provide some links, and some advice on JNI.
0
 
LVL 19

Author Comment

by:cheekycj
ID: 1246971
Please do!

Any advice/links would be appreciated.

CJ
0
 
LVL 2

Expert Comment

by:jasbro
ID: 1246972
I'm going to warn you, JNI is ugly, and still won't give you the efficiency you would get from an entirely native application, but you would be able to use C or C++ code to actually write the data, speeding it up.  The read will still be interpreted, and fairly slow.  http://java.sun.com/docs/books/tutorial/native1.1/index.html is a good place to get started, then http://www.javasoft.com/products/jdk/1.1/docs/guide/jni/spec/jniTOC.doc.html for the specification.

What are your considerations for using Java to do this FTP thing?  If you are doing it just to get the GUI stuff from Java, you could actually write the working parts of the ftp application in C or C++, then just build a Java GUI that tells the C application when to do its work.  (In this way, you could get extremely close to the time results from the other ftp applications.)
0
 
LVL 19

Author Comment

by:cheekycj
ID: 1246973
JNI does look ugly... I have a thought- and please comment on this...

What if I launch the ftp via unix command?  Is there a way to launch unix
commands from Java - the application will be running on a Sun Workstation.

The only question about this is would I be able to run it as multiple threads.

I am performing anywhere from min. of 6 upto 60 ftp's and would like to do
them concurrently via threads (hence me using java for ftp) but if I could
do the ftp via unix but launch it using threads in java- that would suffice.

Any comments/thoughts?

CJ
0
 
LVL 2

Expert Comment

by:jasbro
ID: 1246974
I am pretty sure there is a way to issue commands to the operating system, but I don't know what it is.  
0
 

Accepted Solution

by:
tny earned 400 total points
ID: 1246975
It so slow because  you use fc.ascii(). In this case your
programm MUST read byte by byte! Because after read it perform some transformation.
Try binary mode it much more fast.
And the BufferedInputStream (bis) is useless
in this situation.
The Socket.getIn(Out)putStream() return the reference to
inner object which implements In(Out)putStream and calls native code for reading. This code have buffered I/O already.
0
 
LVL 19

Author Comment

by:cheekycj
ID: 1246976
tny:  It worked!  Wow, a simple transfer type setting and bam, such a huge
difference in transfer rate.

Question:  Why is it not such a difference here but not such in FTP programs?

thanks alot,
CJ

0
 
LVL 5

Expert Comment

by:mbormann
ID: 2141266
good
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Ready to improve network connectivity? Watch this webinar to learn how SD-WANs and a one-click instant connect tool can boost provisions, deployment, and management of your cloud connection.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
This was posted to the Netbeans forum a Feb, 2010 and I also sent it to Verisign. Who didn't help much in my struggles to get my application signed. ------------------------- Start The idea here is to target your cell phones with the correct…
Viewers learn about the “for” loop and how it works in Java. By comparing it to the while loop learned before, viewers can make the transition easily. You will learn about the formatting of the for loop as we write a program that prints even numbers…
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.
Suggested Courses

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question