Solved

use mget to copy newer files only.

Posted on 2014-01-14
3
665 Views
Last Modified: 2014-01-30
I use psftp to connect to a remote sftp server and copy files to a local server and is currently using mget * to copy everything.

Is it possible to copy just the newer files only?

Please advise.

Thanks.
0
Comment
Question by:nav2567
3 Comments
 
LVL 57

Expert Comment

by:giltjr
ID: 39782093
No, because there is no definition of "newer".

You would need to write a script that somehow identifies the files you want to download and download only them.
0
 
LVL 23

Expert Comment

by:David
ID: 39783124
One workaround:  DOS/Windows uses a simple flag (archive bit) on the file header to indicate whether the file has changed since its last backup.  That bit could be set off when a mget has successfully run.  Thereafter, if the source file hasn't changed, you know it can be passed over.

Variation:  copy the source file(s) to a zip archive, then ftp the zip to the target.  Unpack the zip if the zip copy is newer than the target copy.
0
 
LVL 16

Accepted Solution

by:
AlexPace earned 500 total points
ID: 39785807
I can think of three ways to do this with Robo-FTP.  Perhaps you could "translate" one of these techniques to your Powershell solution.

A. If you keep a local copy of the downloaded files use RCVFILE "*" /serverdate /ifnewer to download only files where the remote copy has a newer date than the local copy.  The /serverdate option is included to preserve the timestamp on downloaded files.


B. If you run the process on a fixed schedule every X days, hours, minutes etc then you could just download any file with a timestamp after the last run.  If I were you, I would update a  timestamp in a text file or registry setting each time the script runs, that way, if your computer were ever down for a full day, such that it completely missed the scheduled execution time, your script would read from the last ACTUAL run time instead of the last calculated run time.


C. Save a directory listing from the remote server every time the script runs.  The first time you'll download everything but on each subsequent time the script will compare the saved directory listing from the last execution with the current directory listing.  It will identify any changes and download only the things that are different.... well unless that difference is that the file was REMOVED from the server... if that happens you might want to delete the local orphan file or not, your choice.
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

I thought I'd write this up for anyone who has a request to create an anonymous whistle-blower-type submission form created using SharePoint 2010 (this would probably work the same for 2013). It's not 100% fool-proof but it's as close as you can get…
When it comes to security, there are always trade-offs between security and convenience/ease of administration. This article examines some of the main pros and cons of using key authentication vs password authentication for hosting an SFTP server.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Viewers will learn how to properly install and use Secure Shell (SSH) to work on projects or homework remotely. Download Secure Shell: Follow basic installation instructions: Open Secure Shell and use "Quick Connect" to enter credentials includi…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now