How to stream data to an FTP server

Posted on 2009-12-18
Last Modified: 2013-12-22
Hello all,

I would like an advice.
I have a huge amount of data that must go to node C BUT through node A and B.
I have to be fast so I want the data not to be writen to nodes A and B.
Simply I am thinking of using A and B as the neccessary hoping nodes in order to reach node C.

Possible Solution 1:
Use NETTY HTTPFileServer change it in orer to immediately forwad all traffic to node A, B and from there use an FTP client (ftpj possibly or something similar) in order to write the data to node C. Node A and B would have a seperate Netty server installed in order to forward the data (without writing anything on the HDD). Something like tunneling.

Possible solution 2:
Make Java clients on node A and B (with Channels and streams) that would send all data to the destination node C (with the use of an ftp client again).

Possible solution 3:
Something off the self ? Maybe something native and not Java.
Is there any FTP sotware that basically "tunnels" or forwards all the data to the fial FTP server?
I have searched for FTP bridge but I have found irrelevant topics.

Is it possible to provide your solution to this problem ?
Is Netty really worths using instead of the Java new NIO (1.5 or 1.6)/The documentation is really not a lot. Have you found any good documentation except the user's guide that they have online?

As the FTP server I am thinking Filezilla server.
FreeNas or Filezlla do you think that could be faster ? (If I could ask such a question)

My aim is to transfer lot of files with Java to node C (ftp server)

Thank you in advance,

Question by:pouli
    LVL 27

    Expert Comment

    I'm rather thinking about this idea:
    LVL 27

    Expert Comment

    so this example could be adapted.

    as far as I know you can link tunnels. But it requires you to install software on server a and b.


    Author Comment

    Hello Tolomir,

    thank you for your answer.
    Currently I have an application before node A (let's call it source) that produces the files. it writes them to the H.D.D.

    I am thinking that now after the new change I should not write them to the H.D.D. but forwarding directly to the node A, etc...

    Having these in mind ( I mean the fact that I can use Java API in order to stream data) how could I use the proposed solution ?

    The wiki speak about the concept in general but I cannot see how it can fit in my probelm.

    Could you please elaborate a little bit more about your idea ?
    How you think that this will be possible ?

    Thank you again.

    Author Comment


    the posted link says:

    "This tutorial will walk you through the steps to running a SSH server on your Windows machine and using it to create a secure tunnel through the Internet to use VNC"

    Why vnc ?
    This process is going to be automated by programs. No user interaction will be possible.
    The source geernates data that automatically will be transfered to the target FTP server through node A and B.
    LVL 27

    Expert Comment

    vnc is just an example. It could be any protocol.
    LVL 27

    Expert Comment

    Ftp is port 21

    Author Comment

    This solution is going to be faster than the Java channels or use of the Netty project ?
    By default SSH ise secured right ?
    LVL 27

    Accepted Solution

    I found a tutorial to forward data from

    source to a (ssh server) delivering to ftp server B. But I didn't find a 2 ( from source to A over B to C) hosts forwarder. So my suggestion might not even work.

    this is the tutorial:

    Additionally the connection from A to B is unencrypted. (Just the connection between source and server A is encrypted)


    Advantage of the solution is, data is never stored on harddisk so this is of cause fast.


    So my solution would only work (encrypted) if you could directly access server C from the source. If it were not ftp but ssh I could think of a connected ssh tunnel, but since ftp needs a control port and a second (data) port I was even unsure if ssh tunnels could deal with it.

    this is a ssh chaining solution:


    Author Comment

    Hello to Tolomir,

    here is the solution that I was looking for.
    I was looking for packages like the Apache's Java NIO framework.
    I have seen many packages like this but Apache's was the "easiest to use".
    I mean that I managed to change the included examples to something that worked for me.

    I used an insecure channels for the hopping and the next step is to create a secure channel in order to finally copy the files to the destination.

    The program easily saturated my locl LAN network speed (100Mbs) for 2500 files of 38 MB.
    Basically I have transferred everything through one socket and at the end of each file I have appended a new byte array that I specify the location where the previous data should be written to.

    Tolomir I wish you merry xmas and happy new year


    Author Closing Comment

    Reason has been specified inside the forum.
    LVL 27

    Expert Comment

    Thank you very much. I wish you the same.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Top 6 Sources for Identifying Threat Actor TTPs

    Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

    Suggested Solutions

    Title # Comments Views Activity
    Fails to login to FTP Server 13 32
    endX challenge 2 32
    mapAB Challlenge 35 50
    groovy example issue 10 32
    Cloud file services can fill many different roles for your business. Often, the use of cloud file services begins with employees using consumer products, like Dropbox, to share files with customers and each other. While sync-and-share can be an effe…
    As a financial services provider, your business is impacted by two of the strictest federal regulations on record: the Sarbanes-Oxley Act and the Gramm-Leach-Bliley Act. Correctly implementing faxing into your organization to provide secure, real-ti…
    The viewer will learn how to implement Singleton Design Pattern in Java.
    This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    11 Experts available now in Live!

    Get 1:1 Help Now