Solved

How to SFTP and pull daily files

Posted on 2013-06-27
16
660 Views
Last Modified: 2013-06-29
I have a need to log in daily from a windows 7 box to a cent0s box and retrieve files that are stored in folders with the current date (2013-06-27). Is there a simple method to do this through powershell or a vbs script that you could share? I have the login down but the logic to dynamically pick a folder and get contents is escaping me.
Thanks
0
Comment
Question by:MRS
  • 5
  • 5
  • 5
  • +1
16 Comments
 
LVL 26

Expert Comment

by:pony10us
ID: 39281510
0
 

Author Comment

by:MRS
ID: 39281520
I have used winscp previously however I would like to have it scripted to just run and handle the date changes on its own. Can winscp be embedded into a script and still handle this dynamic date changing?
0
 

Author Comment

by:MRS
ID: 39281538
as an example the path to my folders for yesterday /var/spool/asterisk/monitor/2013/06/26
so todays would be /var/spool/asterisk/monitor/2013/06/27
0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281541
Yes - take a look at:  http://winscp.net/eng/docs/script_download_most_recent_file

EDIT:

Sorry we posted at the same time.  I understand now that you are looking at the folder changing daily.  Take a look at the script above anyway but not sure if it will do what you are after.

You may need to set environment variables using batch and use those variables in the SCP script.
0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281623
Try setting variables using a batch file:

::-----------------------------------------------------------------------------
::  Set variables for the day abbreviation (DAY), month number (MM), 
::  day number (DD) and 4-digit year (YYYY). 
::-----------------------------------------------------------------------------

:setvar

for /F "tokens=1-4 delims=/ " %%i IN ('date /t') DO (
set DT_DAY=%%i
set DT_MM=%%j
set DT_DD=%%k
set DT_YYYY=%%l)

Open in new window


Then call WinSCP from the batch file with the following "get":

get /var/spool/asterisk/monitor/%DT_YYYY%/%DT_MM%/%DT_DAY%/*.* c:\downloadsfolder <this would be where you want it to go>

Open in new window


I have not fully tested this out yet since my Linux box is offline right now.
0
 
LVL 7

Expert Comment

by:BT15
ID: 39281631
The way i have tackled this in the past is to use WinSCP, and then script around its commandline functionality.

for example:

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs

Open in new window


where scripttemp.txt is a file containing the ftp commands.

for example:

option batch abort
option confirm off
open ftpsite
cd 2013-06-27
get -delete *.* c:\fromFTP\*.*
exit

Open in new window


in this example, ftpsite is the name of a saved connection in the WinSCP GUI.

Given the issue that you have presented, I would do something like this:

$today = Get-Date -UFormat %Y-%m-%d

$file = @("option batch abort","option confirm off","open ftpsite","cd $today","get -delete *.* c:\fromFTP\*.*","exit")

$file | sc scripttemp.txt

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs


del scripttemp.txt

Open in new window

0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281671
BT15:

That is a nice solution except that it looks at the date as a whole (2013-06-27) single folder. The author indicated that the folder structure was ../yyyy/mm/dd so you would need to split that out somehow.

I showed one way to do that in the first part of my reply.  If you incorporate that (or something similar) into your code then I think it should work.
0
 
LVL 7

Accepted Solution

by:
BT15 earned 500 total points
ID: 39281695
pony10us

i kept getting pulled away from this as i was typing it, so i didnt see any of the replies until i had posted. the -uFormat option of the get-date commandlet is extremely flexible, so changing the hypens is as simple as :

$today = Get-Date -UFormat %Y/%m/%d

Open in new window


so to revise:

$today = Get-Date -UFormat %Y/%m/%d
$folder = "/var/spool/asterisk/monitor/" + $today

$file = @("option batch abort","option confirm off","open ftpsite","cd $folder","get -delete *.* c:\fromFTP\*.*","exit")

$file | sc scripttemp.txt

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs


del scripttemp.txt

Open in new window

0
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
LVL 26

Expert Comment

by:pony10us
ID: 39281740
BT15:

Okay, I have not worked with uFormat before. Something for me to look into and learn.  

On another note.  I see that you delete the files from the site after you "get" them?
0
 
LVL 7

Expert Comment

by:BT15
ID: 39281970
pony10us

I have found uFormat to be so convenient that I rarely get-date without it anymore. you can do anything:

$date = get-date -uformat %YExperts%mExchange%d

Open in new window


The ftp script file can be populated with any valid FTP command, so in that example, yes I am deleting files after getting them. That can certainly be changed.
0
 
LVL 16

Expert Comment

by:AlexPace
ID: 39282159
This Robo-FTP script should do the it if you change the download destination folder, server address and login credentials:

SET today = %date 
SETEXTRACT month = yesterday "-" 1
SETEXTRACT day = yesterday "-" 2
SETEXTRACT year = yesterday "-" 3
SET TodayPath = "/var/spool/asterisk/monitor/20" + year + "/" + day + "/" + day
WORKINGDIR "c:\download\destination\folder"
FTPLOGON "sftp.mydomain.com" /user="UserID" /pw="Secret" /servertype=SFTP /trust=all 
FTPCD TodayPath 
IFERROR GOTO Done
RCVFILE "*"
:Done
FTPLOGOFF
EXIT

Open in new window

0
 

Author Comment

by:MRS
ID: 39282213
BT15
Can you please explain exactly how your recent addition would be implemented? Is this called via single batch script that I could schedule?
0
 
LVL 7

Expert Comment

by:BT15
ID: 39282283
sure.

so as to not leave this up to assumption, make sure you have WINSCP installed on your machine. Also, you will need to launch WINSCP from the GUI, set up a connection to your site, and save it as ftpsite (if you dont want to alter my script at all). be sure to check the option to have the credentials saved. manually connect to your ftp site to validate functionality. This is important so that we can rule this out if the script fails for some reason.

on your windows 7 box, save the script as a .PS1 file (c:\scripts\ftp.ps1). in the same directory, make a .CMD file (c:\scripts\ftp.cmd). inside the CMD file, use the following code:

powershell -ExecutionPolicy unrestricted -command .\ftp.ps1

then go into Windows Task Scheduler and have c:\scripts\ftp.cmd executed at whatever interval you prefer.

i know that is a very wide view of setting up a windows scheduled task, but i wasnt sure how granular you needed me to get there.

i have attached files for your convenience. just remove the .txt extension from them and place them in the folder of your choice (c:\scripts if you dont want to change anything)
ftp.cmd.zip
0
 

Author Comment

by:MRS
ID: 39282442
this is great how about if I wanted my local folders to mirror the remote?
0
 

Author Closing Comment

by:MRS
ID: 39284285
thank you great point to get me moving forward.
0
 
LVL 7

Expert Comment

by:BT15
ID: 39286772
you are welcome.

sorry for the delay on responding.

http://winscp.net/eng/docs/task_synchronize_full
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Set OWA language and time zone in Exchange for individuals, all users or per database.
A procedure for exporting installed hotfix details of remote computers using powershell
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now