Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

How FTP pulls files

Posted on 2013-11-04
6
Medium Priority
?
496 Views
Last Modified: 2013-11-05
I am pulling files from a customer's ftp sever on a daily basis.  Up until recently, we had no problems but suddenly we are getting files with today's date and the file has some other day's data.  It turns out that we rename the file on our side and the customer is trying to say we are messing up the data when we rename it.

Can someone please explain how ftp works when there are multiple files with virtually the same file name with a different date.

For example if the customer had three files on the ftp server
M700111.1234   10/31/2013
M700111.4567    11/01/2013
M700111.8999     11/02/2013

Given the following script what would be pulled from the customer's ftp server.    

1.  open 999.999.999.999
2.  user MICKEYMOUSE
3.  PASSWORD
4.  prompt
5.  lcd c:\TEMP\
6.  bin
7.  mget *.*
8.  quit
0
Comment
Question by:sherbug1015
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
6 Comments
 
LVL 84

Expert Comment

by:Dave Baldwin
ID: 39622301
That should get all of the files in the current remote directory.
0
 
LVL 57

Expert Comment

by:Bill Prew
ID: 39622387
It sounds like you are assuming that there will only be one file named M700111.<anything> ?  If that's the case, then you are likely to run into problems given that there can now be multiple files with that same base name, but different extensions.

I suspect that you are renaming the file based on the system clock on the receiving computer, and so multiple files could get the same time stamp when they are pulled, which may be a problem.

I guess the root question is what do you want to happen when there are more than one of these files?  Pull the latest one? Pull all and rename with the modified date from the remote system?  Etc...

~bp
0
 

Author Comment

by:sherbug1015
ID: 39622535
billprew -  that is what I would have thought as well.  I just went back to the customers ftp site and they had dropped another file out there since this morning when I first picked up.

There is a file called

BCL.SDOA.1442238  -  11/04/13  2:20 p.m.
BCL.SDOA.1442078  -  11/03/13  2:20 p.m.

I picked up the 11/03 file this morning and when I ran my script, it only picked up the 11/4 file.  Same file name, different extension.

How does it know that the 11/03 file has already been picked up is my question.  Is there something I am missing here.  It must be using some kind of system time to determine what has been picked up.  Can you confirm anything like that.

thanks
0
Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

 
LVL 84

Expert Comment

by:Dave Baldwin
ID: 39622590
If you have not renamed the file, then you probably won't see the 'over-write warning' dialog in a script like you would in a desktop client.  For a test, rename both files on your computer and run your script again.  I suspect it will then re-download both of them.
0
 
LVL 57

Assisted Solution

by:Bill Prew
Bill Prew earned 1200 total points
ID: 39622597
The "mget *.*" should pull ALL files in that folder down to the client computer.  That being said, it could depend a bit on the FTP client software you are using on your computer.  If the file is still in the destination folder on the client PC, it may be "smart" enough to see that it is already there and validate it hasn't changed, and therefore not pull it down again.  But I don't think the Windows built in FTP command line client is smart enough to do that.

I expect thought, even if that is true, if you deleted any existing files from the destination folder on the client computer, it will then pull all files down again.  I very much doubt the server end is doing anything to determine what has already been sent down, that would not be typical at all.

~bp
0
 
LVL 16

Accepted Solution

by:
AlexPace earned 800 total points
ID: 39623042
The FTP protocol does not support wildcards.
MGET is a client command, not an FTP protocol command.
It works like this:
  1. Client requests a directory LIST
  2. Client parses list for files that match *.* or whatever pattern you used.
  3. Client downloads the matching files one at a time.

The sad thing about the FTP protocol is that the format of the directory list is not standard.
There is significant variation in the listing formats returned by different FTP servers.
This means that sometimes step #2 above fails because the list parsing routine failed.
It happens but it is rare.
Most likely the file just wasn't there when you ran the script.
Add a line to save a remote listing to a text file.
That file will help you see what the FTP server returned as a listing so you can determine if it is a parsing problem or if the file just wasn't there.
ftp.exe has no built-in means to skip a file because a local copy already exists.


edit: I've seen FTP servers scripted to delete a file after it is downloaded but those are almost always hosted by banks.
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Using dates in 'DOS' batch files has always been tricky as it has no built in ways of extracting date information.  There are many tricks using string manipulation to pull out parts of the %date% variable or output of the date /t command but these r…
This is a fine trick which I've found useful many times, when you just don't want to accidentally run a batch script or the commands needs administrator rights.
Monitoring a network: how to monitor network services and why? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the philosophy behind service monitoring and why a handshake validation is critical in network monitoring. Software utilized …
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question