Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Upload to multiple locations concurrently?

Posted on 2011-02-16
8
Medium Priority
?
715 Views
Last Modified: 2012-05-11
I need a way to have clients upload large (up to 30GB) files to a single address via FTP and have that upload actually delivered to three separate locations simultanously.

Currently, we wait the two or three hours for the upload to complete and then we resend to the other two locations - which takes an additional two or three hours to complete.

We're not insistent on FTP, but we have many clients so that end needs to be either a standard technology or a web interface.

It sounds like a DFS thing, but can DFS start replicating before a file has fully been received?
0
Comment
Question by:frozenJim
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 96

Assisted Solution

by:Lee W, MVP
Lee W, MVP earned 1000 total points
ID: 34909643
If you're using FTP, you could just run 3 different ftp's at the same time - it would take 3 times as long and no one would get it "first" (maybe a few minutes different).
0
 
LVL 16

Accepted Solution

by:
AlexPace earned 1000 total points
ID: 34910457
I agree with leew, it depends on the location of the throughput bottleneck.

Currently you send 15 GB per hour (2 hours to send 30 GB)  

If this is the max your computer, network, and internet connection can manage then running concurrent uploads won't help... you'll just have 3 jobs running concurrently for 6 hours instead of 3 sequential jobs each running 2 hours for a total of 6 hours.

On the other hand if your computer, network, and internet connection are capable of doing a lot more than 15 GB per hour then you absolutely want to have three different instances of your ftp client running so you can transfer to all three remote sites concurrently.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 35339889
This question has been classified as abandoned and is being closed as part of the Cleanup Program. See my comment at the end of the question for more details.
0
Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

 

Author Comment

by:frozenJim
ID: 37616818
I think that I should explain the problem more clearly:

A variety of uploaders - over whom I have no control - spread throughout the world, need to upload very large files to my FTP server.  They will send one file, one time using their own FTP client software.  I have no control over them.

I need these files to arrive in three locations throughout the world simultaneously: India, England, and Canada.

What I envision is an FTP server smart enough to "pass- through" the bits as they arrive at the main server.  But I am unaware of where I can find this technology.

If anyone has a solution, I would be grateful.
0
 
LVL 96

Expert Comment

by:Lee W, MVP
ID: 37617238
You're looking for a broadcast... FTP is a protocol -  a set of rules defining how something (in this case, file transfer) gets done.  There is no facility I've ever heard of to allow FTP to do this.

You might want to look at services that don't use FTP - DropBox or similar file sharing sites.
0
 

Author Comment

by:frozenJim
ID: 37654092
Yeah, thanks Leew, that is what I was thinking.

Due to security considerations, I cannot use a PUBLIC CLOUD service.  I need to own my own DropBox (or something similar).

Rumour has it that Watchguard may be building this device now... you receive the file in one location and AS IT ARRIVES it is replicated to other sites on a block basis.  THIS is what I need.

Anyone else know of this kind of solution?
0

Featured Post

Flexible connectivity for any environment

The KE6900 series can extend and deploy computers with high definition displays across multiple stations in a variety of applications that suit any environment. Expand computer use to stations across multiple rooms with dynamic access.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For anyone that has accidentally used newSID with Server 2008 R2 (like I did) and hasn't been able to get the server running again because you were unlucky (as I was) and had no backups - I was able to get things working by doing a Registry Hive rec…
This article is in regards to the Cisco QSFP-4SFP10G-CU1M cables, which are designed to uplink/downlink 40GB ports to 10GB SFP ports. I recently experienced this and found very little configuration documentation on how these are supposed to be confi…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
There's a multitude of different network monitoring solutions out there, and you're probably wondering what makes NetCrunch so special. It's completely agentless, but does let you create an agent, if you desire. It offers powerful scalability …

704 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question