Solved

Uploading large files via FTP fails with timeout

Posted on 2009-04-05
21
3,021 Views
Last Modified: 2013-12-09
I have tried three different FTP clients and two different FTP servers and in every case I am unable to upload a file of about 33 Meg. Uploading smaller files works perfectly.

My computer is behind a LinkSys BEFSR41 router. My computer's firewall is on. I am running a fully patched Windows XP Pro.

I have included a log from my FTP client below.

Thank you.
Sent a total of 33641637 bytes

ReceiveLine: timeout waiting 30 seconds for response from server

ReceiveLine returned -2

ReceiveLine failed (winsock error 0 )

STOR failed with error 0

Upload failed, retrying upload

Open in new window

0
Comment
Question by:dangould
  • 10
  • 8
  • 2
  • +1
21 Comments
 
LVL 3

Expert Comment

by:csaint00
Comment Utility
Hmm, you can try setting your timeout value higher on the server or deactivating the Application Layer Gateway service on your client per this article, http://winscp.net/forum/viewtopic.php?t=6260
0
 

Author Comment

by:dangould
Comment Utility
I do not have administrative access to the FTP servers I use so I cannot change their timeout values.

I stopped the Application Layer Gateway service on the client and ran a test upload. There was no change in the result... the upload continues to timeout then restart.
0
 
LVL 3

Expert Comment

by:csaint00
Comment Utility
hmm, does this happen with any file you transfer and is it always around the same amount of data uploaded before the connection is dropped. From your snippet it appears that you are getting most of the file uploaded "Sent a total of 33641637 bytes" before it crashes, is this correct? Also if you need a quick and dirty solution WinRAR can break the file into smaller archive files that you can send one at a time to the FTP server while we try to figure out the bigger problem. One more thing, do you routinely upload this much data so I can rule out the ISP throttling you?
0
 
LVL 11

Accepted Solution

by:
packetguy earned 250 total points
Comment Utility
I've seen this kind of file transfer failure happen when a port speed/duplex mismatch exists somewhere on the path between devices. What happens is that during the port flap (usually on the client device), the client closes down all active sessions, since the interface has gone down, albeit briefly. The outage can be so brief that users don't notice it on normal web traffic, which is mostly short bursts of TCP traffic. But long-running sessions, such as file transfers, die.

Check the interface stats and logs on the client device for dropped packets and CRC errors, as well as on the intervening switches. Ethernet autonegotiation is an unreliable process, and sometimes you have to force the speed/duplex on one side or the other of a link, or both. If you have an unmanaged switch, I'd recommend replacing it with a managed switch (Cisco Catalyst switches are very high quality and available on eBay cheap; I just bought a 3548 for $200, and there are many more at that price). A managed switch will give you much more visibility into your network, as well as improved performance.
0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
One or both sides may be blocking ICMP type 3, which is needed for Path MTU discovery (determining if TCP packets need to be fragmented or not).  Try lowering the MTU size on the client side and see if you can complete your upload.

Assuming the FTP site responds to ICMP type 0 (ECHO), you can use the following test method to determine the correct MTU size:

open a cmd prompt and type:

ping -f -l 1492 <host or IP of FTP>

If you get messages that say:

Packet needs to be fragmented but DF set.

Lower the value by 10 and try again (i.e. 1482, 1472, etc.).

Once you start seeing successful return replies, you've found the right size.  If ICMP type 0 is blocked, try setting your MTU size to 1400.

Here are instructions on how to modify the MTU size in the Registry:

http://support.microsoft.com/kb/826159

Unfortunately you're going to need to reboot to apply the changes.
0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
Sorry.  Got my ping syntax reversed:

ping <ftp host or ip> -f -l 1492
0
 

Author Comment

by:dangould
Comment Utility
csaint00: With a bit more experimentation I was able to determine that the timeout occurs at the very end of the transfer. In fact, if I abort the transfer while it's waiting for that final handshake, the file appears to be Ok on the server.

I can't use RAR because I do not have access to the FTP machine to unpack it.

I don't transfer many files of this size, but I don't think this is a throttling problem because the file does seem to get there.
0
 

Author Comment

by:dangould
Comment Utility
zencrafter: Following your instructions I arrived at an MPU of 1432 on the client machine. Our office router has its MPU set as well... to a value of 1462.

Are you recommending that I change *both* the client machine registry *and* the router to 1432?
0
 

Author Comment

by:dangould
Comment Utility
packetguy: To test your theory I unplugged the entire office network from the router and connected my test machine directly to it. In this configuration there was no hardware between the client machine and the router.

The 33 Meg test file transferred perfectly to completion with no timeouts.

Unfortunately, we have a mix of hubs and switches here... none of them managed. Replacing the whole set with managed switches is not an option. Any suggestions as to where I might go from here?
0
 

Author Comment

by:dangould
Comment Utility
packetguy: As a further test I connected my test machine to the office router while leaving the rest of the office network connected to another port on the same router.

The transfer failed as before.

How is it possible that other network hardware could interfere with the transfer when my machine is connected directly to the router?
0
Too many email signature updates to deal with?

Do you feel like you are taking up all of your time constantly visiting users’ desks to make changes to email signatures? Wish you could manage all signatures from one central location, easily design them and deploy them quickly to users? Well, there is an easy way!

 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
Just change your client machine to 1462, as that packet is making it out and back without being fragmented.
0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
bah...blurry eyes this morning.  Change the client to 1432, not 1462.
0
 

Author Comment

by:dangould
Comment Utility
zencrafter: I changed the registry values as given in the section "Change the MTU Settings for PPP Connections" on the Microsoft Knowledge Book page you linked above.

There was no change. The transfer runs to what seems to be completion, waits a minute or so, then restarts the transfer from scratch.

I have included the registry changes I made to the client for your review.
REGEDIT4
 

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Ndiswan\Parameters\Protocols\0]

"ProtocolType"=dword:00000800

"PPPProtocolType"=dword:00000021

"ProtocolMTU"=dword:00000598

Open in new window

0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
It appears as if the router itself is the source of the problem.  Are you blocking any specific ports on the router's firewall? What brand of router is it?
0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
Ignore that about the brand. I see it at the top of your post.
0
 
LVL 2

Expert Comment

by:zencrafter
Comment Utility
Does any of your FTP clients support Passive (PASV) mode?  Try that and see if it works for you.
0
 

Author Comment

by:dangould
Comment Utility
zencrafter: I too thought the problem might lie in the router... after all NAT routers do look at FTP packets. But in a test I described above, connecting the client machine directly to the router with all other hubs and switches disconnected corrects the problem.

I use PASV mode exclusively.
0
 
LVL 2

Assisted Solution

by:zencrafter
zencrafter earned 250 total points
Comment Utility
Then packetguy's suggestion is most likely the right one.  If you ditch the hubs for switches (hubs can't duplex), it should eliminate the problem...you don't necessarily need managed switches.  Since you seem to have access to all the network gear and can make at least temporary changes in the network topology, try excluding all the hubs from your network as a test and see if that fixes the issue.
0
 

Author Comment

by:dangould
Comment Utility
zencrafter: Thanks for your comments. It will be a few days before I can get full access to our physical network to try the test you suggest. I'll report my findings shortly.
0
 

Author Comment

by:dangould
Comment Utility
My apologies for the delay in following up here. Today was the first day I could get full access to the network hubs and switches.

With the client PC connected directly to the router I unplugged the hubs and switches one at a time then did a test upload of a 35 Meg file.

Eventually I traced the problem to an 8-port NetGear Fast Ethernet Switch FS108. Then, to my surprise, when I reconnected this switch the problem did NOT return.

Since then I have done several tests and can no longer reproduce the problem. It appears that temporarily disconnecting then reconnecting the NetGear switch corrected the problem.

I'm sure that this switch has been powered down and up at least once since this problem began.

Can anyone explain this?
0
 

Author Comment

by:dangould
Comment Utility
Although this problem hasn't been completely solved, you have given me a good start. Thanks to everyone here for their assistance.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Understanding FTPS File transfer is a common requirement in most Enterprises. While there are numerous ways to get a file from Point A to Point B over a network, perhaps the most common method still in use is FTP – File Transfer Protocol. FTP is …
With the withdrawal of support for Windows Server 2003 this summer, many clients face the issue of moving away from their 2003 installs. There are a few options out there that many people/companies are selling. But the clients I have, haven't wanted…
Viewers will learn how to properly install and use Secure Shell (SSH) to work on projects or homework remotely. Download Secure Shell: Follow basic installation instructions: Open Secure Shell and use "Quick Connect" to enter credentials includi…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now