FTP efficiency...

  I am using the sun.net and sun.net.FtpClient

I have the ftp working but I am getting a pathetic
download rate of 22 KB/sec or so on our LAN.  An
FTP Program downloaded the same files at the speed
of 150-500 KB/sec

Does anyone know how to increase the transfer rate?

Maybe use another method?

Accompanying Code...

      byte buffer[] = new byte[1000000];        
      FtpClient fc = new FtpClient(hostname);
      fc.login(username, password);
      InputStream theFile = fc.get(filename);
      BufferedInputStream bis = new       BufferedInputStream(theFile,1000000);
      int i;
      FileOutputStream fos = new       FileOutputStream(destination);
      BufferedOutputStream bos =new BufferedOutputStream(fos);
      while ((i = bis.read(buffer)) != -1) {
    } catch (IOException e) { System.err.println(e); }
LVL 19
Who is Participating?
tnyConnect With a Mentor Commented:
It so slow because  you use fc.ascii(). In this case your
programm MUST read byte by byte! Because after read it perform some transformation.
Try binary mode it much more fast.
And the BufferedInputStream (bis) is useless
in this situation.
The Socket.getIn(Out)putStream() return the reference to
inner object which implements In(Out)putStream and calls native code for reading. This code have buffered I/O already.
Unfortunately, if you are using java, you will probably never approach the speed of the ftp program you mentioned.  You can increase the speed somewhat by using the JIT compiler (which is on by default in jdk1.2), and by streamlining your code, although looking at your code, it looks fairly clean.  You may want to look at other output streams for the file write, and in fact you probably don't need the buffered stream for output, you will probably never receive data over the network faster than you can write it to a file...  THe buffered stream adds overhead.
you read the stream byte after byte

while ((i = bis.read(buffer)) != -1) {

try something like      

byte[] chunk = new byte[10000];
int len = is.read(chunk);
while (len >0)
  bos.write(chunk, 0, len);
  len = is.read(chunk);

7 new features that'll make your work life better

It’s our mission to create a product that solves the huge challenges you face at work every day. In case you missed it, here are 7 delightful things we've added recently to monday to make it even more awesome.

cheekycjAuthor Commented:
Isn't my code doing the same thing?

> byte buffer[] = new byte[1000000];          
> int i;
> while ((i = bis.read(buffer)) != -1) {
>        bos.write(buffer,0,i);
> }

it may be quite more time-consuming if your stream is unbuffered ... or if (for example) both read() and read(byte[]) call native methods - that my code will make only one native method call (which is slooow) and your code will have 10000.
cheekycjAuthor Commented:
  I tried your way and didn't make a difference.  Any other ideas?
I refuse to believe that java reduces the ftp speed by 10x-20x.  It doesn't
seem logical.

  I am rejecting your answer just to field more suggestions.  If I don't get
anything that improves the performance you will get the points.

I have seen some really bad performance from java, especially when doing input and output streams.  Much worse than you would think.  Java keeps all of your data in the most efficient storage places for Java, not for the rest of your computer.  When you decide to go outside of the little Java world to do work (reading from a port, writing to a file, etc)  Java has to map it's own data structures onto corresponding structures for the underlying system, then do the requested action.  (This is what heyhey was talking about with the Native Methods comment)  You are doing this type of thing with each byte, so Java is going through native code to grab a byte, pulling it into the interpreter, pushing it out the other side and writing it to disk.  This is where all of the time expense is.  
  You could access these FTP and net packages through Java Native Interface, then all of the reading and writing would be done in compiled code (not interpreted) and should go much faster.  If you would like more information on doing that, I can provide some links, and some advice on JNI.
cheekycjAuthor Commented:
Please do!

Any advice/links would be appreciated.

I'm going to warn you, JNI is ugly, and still won't give you the efficiency you would get from an entirely native application, but you would be able to use C or C++ code to actually write the data, speeding it up.  The read will still be interpreted, and fairly slow.  http://java.sun.com/docs/books/tutorial/native1.1/index.html is a good place to get started, then http://www.javasoft.com/products/jdk/1.1/docs/guide/jni/spec/jniTOC.doc.html for the specification.

What are your considerations for using Java to do this FTP thing?  If you are doing it just to get the GUI stuff from Java, you could actually write the working parts of the ftp application in C or C++, then just build a Java GUI that tells the C application when to do its work.  (In this way, you could get extremely close to the time results from the other ftp applications.)
cheekycjAuthor Commented:
JNI does look ugly... I have a thought- and please comment on this...

What if I launch the ftp via unix command?  Is there a way to launch unix
commands from Java - the application will be running on a Sun Workstation.

The only question about this is would I be able to run it as multiple threads.

I am performing anywhere from min. of 6 upto 60 ftp's and would like to do
them concurrently via threads (hence me using java for ftp) but if I could
do the ftp via unix but launch it using threads in java- that would suffice.

Any comments/thoughts?

I am pretty sure there is a way to issue commands to the operating system, but I don't know what it is.  
cheekycjAuthor Commented:
tny:  It worked!  Wow, a simple transfer type setting and bam, such a huge
difference in transfer rate.

Question:  Why is it not such a difference here but not such in FTP programs?

thanks alot,

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.