I have searched FTP issues and see mostly FTP hangs closing connections or has issues with users sitting in front of DOS windows. Ours in unattended batch jobs.
We cannot use MPUT as we have too many directories and new directories and files are added all the time.
We have used a VB6 to now VB.Net 2008 program to dynamically search for all files matching file pattern within a starting point on source drive, and we build all the FTP commands to navigate thru both source and destination servers. Program has behaved very well for years.
After script built, we shell out to DOS and run FTP passing in script, and redirecting output to file. Once FTP ends, we read file looking for errors and take action as needed. We also allow some retries, but usuallly we cause job failure and notification is sent out.
Intermittent - say 5 times in 3 months and we run this process maybe 30 times a day for between 1 file to 3000 files for an FTP instance. If 22 business days in a month, that is 5 issues out of 1980 runs.
So far, issue always well after connection, and after 10 to 2500 files already transfered, then it just hangs. FTP output shows no errors, just hung on next file.
Task manager shows VB program there, but NO CPU cycles, shows FTP running and NO CPU cycles. At first I noticed, as FTP runs, memory gets bigger and bigger so thought it maxed out and hung, but it hangs when running 50 files as well
We do run multiple jobs concurrently, usually not more than 2, but we use same credentials, same source and target servers, but not same source or target directories.
We do NOT see FTP hangs in development environment, does NOT hang on 2ndary Production environment, just hangs in Primary Production environment - or so it appears.
Dev & 2ndary Production do NOT use the FTP "gateway", thery connect firectly to the FTP server. Only Primary Production uses FTP "gateway".
Factors that changed last 6 months this has been occuring:
We moved from older W2K servers to W2K3 servers
New servers in newer locations, target servers not as close - 7 hops... maybe more for some environments. Assumes different version of FTP on new boxes. All boxes, old and new, 32bit.
Recompile on VS2008 from VS2005
Entire corporation network "seems" (antidotal) slower.
We switched from an older FTP "gateway" that was over capacity to a newer FTP "gateway". From our end, the name of the gateway changed, the hardware is beefier and redundant, but process remains the same. Target directories are mapped as virtual directories to the gateway. This did not change - but the gateway is new, and the target server are new. Same virtual gateway names.
We are installing clumsy workarounds, but we would like to get at root cause.
Are there little known constraints on DOS and/or FTP? Something beyond a well known insider issue that Microsoft FTP is notoriously unreliable and not secure.
I do not believe my homegrown code is to blame - as we just shell out and run native FTP in a DOS window.