asked on

FTP efficiency...

Hi,
I am using the sun.net and sun.net.FtpClient
packages.

I have the ftp working but I am getting a pathetic
download rate of 22 KB/sec or so on our LAN. An
FTP Program downloaded the same files at the speed
of 150-500 KB/sec

Does anyone know how to increase the transfer rate?

Maybe use another method?

Thanks,
CJ
===============================================
Accompanying Code...

byte buffer[] = new byte[1000000];
FtpClient fc = new FtpClient(hostname);
fc.login(username, password);
fc.ascii();
fc.cd(filepath);
InputStream theFile = fc.get(filename);
BufferedInputStream bis = new BufferedInputStream(theFile,1000000);
int i;
FileOutputStream fos = new FileOutputStream(destination);
BufferedOutputStream bos =new BufferedOutputStream(fos);
while ((i = bis.read(buffer)) != -1) {
bos.write(buffer,0,i);
}
fc.closeServer();
fos.close();
} catch (IOException e) { System.err.println(e); }

jasbro

Unfortunately, if you are using java, you will probably never approach the speed of the ftp program you mentioned. You can increase the speed somewhat by using the JIT compiler (which is on by default in jdk1.2), and by streamlining your code, although looking at your code, it looks fairly clean. You may want to look at other output streams for the file write, and in fact you probably don't need the buffered stream for output, you will probably never receive data over the network faster than you can write it to a file... THe buffered stream adds overhead.

heyhey_

you read the stream byte after byte

while ((i = bis.read(buffer)) != -1) {
bos.write(buffer,0,i);
}

try something like

byte[] chunk = new byte[10000];

int len = is.read(chunk);
while (len >0)
{
bos.write(chunk, 0, len);
len = is.read(chunk);
}

cheekycj

ASKER

Isn't my code doing the same thing?

> byte buffer[] = new byte[1000000];
> int i;
> while ((i = bis.read(buffer)) != -1) {
> bos.write(buffer,0,i);
> }

CJ

heyhey_

it may be quite more time-consuming if your stream is unbuffered ... or if (for example) both read() and read(byte[]) call native methods - that my code will make only one native method call (which is slooow) and your code will have 10000.

cheekycj

ASKER

heyhey
I tried your way and didn't make a difference. Any other ideas?
I refuse to believe that java reduces the ftp speed by 10x-20x. It doesn't
seem logical.

jasbro
I am rejecting your answer just to field more suggestions. If I don't get
anything that improves the performance you will get the points.

CJ

jasbro

I have seen some really bad performance from java, especially when doing input and output streams. Much worse than you would think. Java keeps all of your data in the most efficient storage places for Java, not for the rest of your computer. When you decide to go outside of the little Java world to do work (reading from a port, writing to a file, etc) Java has to map it's own data structures onto corresponding structures for the underlying system, then do the requested action. (This is what heyhey was talking about with the Native Methods comment) You are doing this type of thing with each byte, so Java is going through native code to grab a byte, pulling it into the interpreter, pushing it out the other side and writing it to disk. This is where all of the time expense is.
You could access these FTP and net packages through Java Native Interface, then all of the reading and writing would be done in compiled code (not interpreted) and should go much faster. If you would like more information on doing that, I can provide some links, and some advice on JNI.

cheekycj

ASKER

Please do!

Any advice/links would be appreciated.

CJ

jasbro

I'm going to warn you, JNI is ugly, and still won't give you the efficiency you would get from an entirely native application, but you would be able to use C or C++ code to actually write the data, speeding it up. The read will still be interpreted, and fairly slow. http://java.sun.com/docs/books/tutorial/native1.1/index.html is a good place to get started, then http://www.javasoft.com/products/jdk/1.1/docs/guide/jni/spec/jniTOC.doc.html for the specification.

What are your considerations for using Java to do this FTP thing? If you are doing it just to get the GUI stuff from Java, you could actually write the working parts of the ftp application in C or C++, then just build a Java GUI that tells the C application when to do its work. (In this way, you could get extremely close to the time results from the other ftp applications.)

cheekycj

ASKER

JNI does look ugly... I have a thought- and please comment on this...

What if I launch the ftp via unix command? Is there a way to launch unix
commands from Java - the application will be running on a Sun Workstation.

The only question about this is would I be able to run it as multiple threads.

I am performing anywhere from min. of 6 upto 60 ftp's and would like to do
them concurrently via threads (hence me using java for ftp) but if I could
do the ftp via unix but launch it using threads in java- that would suffice.

Any comments/thoughts?

CJ

jasbro

I am pretty sure there is a way to issue commands to the operating system, but I don't know what it is.

ASKER CERTIFIED SOLUTION

tny

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

cheekycj

ASKER

tny: It worked! Wow, a simple transfer type setting and bam, such a huge
difference in transfer rate.

Question: Why is it not such a difference here but not such in FTP programs?

thanks alot,
CJ

mbormann

good