[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now

x
?
Solved

How to SFTP and pull daily files

Posted on 2013-06-27
16
Medium Priority
?
719 Views
Last Modified: 2013-06-29
I have a need to log in daily from a windows 7 box to a cent0s box and retrieve files that are stored in folders with the current date (2013-06-27). Is there a simple method to do this through powershell or a vbs script that you could share? I have the login down but the logic to dynamically pick a folder and get contents is escaping me.
Thanks
0
Comment
Question by:MRS
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
  • 5
  • +1
16 Comments
 
LVL 26

Expert Comment

by:pony10us
ID: 39281510
0
 

Author Comment

by:MRS
ID: 39281520
I have used winscp previously however I would like to have it scripted to just run and handle the date changes on its own. Can winscp be embedded into a script and still handle this dynamic date changing?
0
 

Author Comment

by:MRS
ID: 39281538
as an example the path to my folders for yesterday /var/spool/asterisk/monitor/2013/06/26
so todays would be /var/spool/asterisk/monitor/2013/06/27
0
Veeam Task Manager for Hyper-V

Task Manager for Hyper-V provides critical information that allows you to monitor Hyper-V performance by displaying real-time views of CPU and memory at the individual VM-level, so you can quickly identify which VMs are using host resources.

 
LVL 26

Expert Comment

by:pony10us
ID: 39281541
Yes - take a look at:  http://winscp.net/eng/docs/script_download_most_recent_file

EDIT:

Sorry we posted at the same time.  I understand now that you are looking at the folder changing daily.  Take a look at the script above anyway but not sure if it will do what you are after.

You may need to set environment variables using batch and use those variables in the SCP script.
0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281623
Try setting variables using a batch file:

::-----------------------------------------------------------------------------
::  Set variables for the day abbreviation (DAY), month number (MM), 
::  day number (DD) and 4-digit year (YYYY). 
::-----------------------------------------------------------------------------

:setvar

for /F "tokens=1-4 delims=/ " %%i IN ('date /t') DO (
set DT_DAY=%%i
set DT_MM=%%j
set DT_DD=%%k
set DT_YYYY=%%l)

Open in new window


Then call WinSCP from the batch file with the following "get":

get /var/spool/asterisk/monitor/%DT_YYYY%/%DT_MM%/%DT_DAY%/*.* c:\downloadsfolder <this would be where you want it to go>

Open in new window


I have not fully tested this out yet since my Linux box is offline right now.
0
 
LVL 7

Expert Comment

by:BT15
ID: 39281631
The way i have tackled this in the past is to use WinSCP, and then script around its commandline functionality.

for example:

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs

Open in new window


where scripttemp.txt is a file containing the ftp commands.

for example:

option batch abort
option confirm off
open ftpsite
cd 2013-06-27
get -delete *.* c:\fromFTP\*.*
exit

Open in new window


in this example, ftpsite is the name of a saved connection in the WinSCP GUI.

Given the issue that you have presented, I would do something like this:

$today = Get-Date -UFormat %Y-%m-%d

$file = @("option batch abort","option confirm off","open ftpsite","cd $today","get -delete *.* c:\fromFTP\*.*","exit")

$file | sc scripttemp.txt

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs


del scripttemp.txt

Open in new window

0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281671
BT15:

That is a nice solution except that it looks at the date as a whole (2013-06-27) single folder. The author indicated that the folder structure was ../yyyy/mm/dd so you would need to split that out somehow.

I showed one way to do that in the first part of my reply.  If you incorporate that (or something similar) into your code then I think it should work.
0
 
LVL 7

Accepted Solution

by:
BT15 earned 2000 total points
ID: 39281695
pony10us

i kept getting pulled away from this as i was typing it, so i didnt see any of the replies until i had posted. the -uFormat option of the get-date commandlet is extremely flexible, so changing the hypens is as simple as :

$today = Get-Date -UFormat %Y/%m/%d

Open in new window


so to revise:

$today = Get-Date -UFormat %Y/%m/%d
$folder = "/var/spool/asterisk/monitor/" + $today

$file = @("option batch abort","option confirm off","open ftpsite","cd $folder","get -delete *.* c:\fromFTP\*.*","exit")

$file | sc scripttemp.txt

$command = 'c:\Program Files\WinSCP\WinSCP.com'
$commandargs ='/script=scripttemp.txt' 
& $command $commandargs


del scripttemp.txt

Open in new window

0
 
LVL 26

Expert Comment

by:pony10us
ID: 39281740
BT15:

Okay, I have not worked with uFormat before. Something for me to look into and learn.  

On another note.  I see that you delete the files from the site after you "get" them?
0
 
LVL 7

Expert Comment

by:BT15
ID: 39281970
pony10us

I have found uFormat to be so convenient that I rarely get-date without it anymore. you can do anything:

$date = get-date -uformat %YExperts%mExchange%d

Open in new window


The ftp script file can be populated with any valid FTP command, so in that example, yes I am deleting files after getting them. That can certainly be changed.
0
 
LVL 16

Expert Comment

by:AlexPace
ID: 39282159
This Robo-FTP script should do the it if you change the download destination folder, server address and login credentials:

SET today = %date 
SETEXTRACT month = yesterday "-" 1
SETEXTRACT day = yesterday "-" 2
SETEXTRACT year = yesterday "-" 3
SET TodayPath = "/var/spool/asterisk/monitor/20" + year + "/" + day + "/" + day
WORKINGDIR "c:\download\destination\folder"
FTPLOGON "sftp.mydomain.com" /user="UserID" /pw="Secret" /servertype=SFTP /trust=all 
FTPCD TodayPath 
IFERROR GOTO Done
RCVFILE "*"
:Done
FTPLOGOFF
EXIT

Open in new window

0
 

Author Comment

by:MRS
ID: 39282213
BT15
Can you please explain exactly how your recent addition would be implemented? Is this called via single batch script that I could schedule?
0
 
LVL 7

Expert Comment

by:BT15
ID: 39282283
sure.

so as to not leave this up to assumption, make sure you have WINSCP installed on your machine. Also, you will need to launch WINSCP from the GUI, set up a connection to your site, and save it as ftpsite (if you dont want to alter my script at all). be sure to check the option to have the credentials saved. manually connect to your ftp site to validate functionality. This is important so that we can rule this out if the script fails for some reason.

on your windows 7 box, save the script as a .PS1 file (c:\scripts\ftp.ps1). in the same directory, make a .CMD file (c:\scripts\ftp.cmd). inside the CMD file, use the following code:

powershell -ExecutionPolicy unrestricted -command .\ftp.ps1

then go into Windows Task Scheduler and have c:\scripts\ftp.cmd executed at whatever interval you prefer.

i know that is a very wide view of setting up a windows scheduled task, but i wasnt sure how granular you needed me to get there.

i have attached files for your convenience. just remove the .txt extension from them and place them in the folder of your choice (c:\scripts if you dont want to change anything)
ftp.cmd.zip
0
 

Author Comment

by:MRS
ID: 39282442
this is great how about if I wanted my local folders to mirror the remote?
0
 

Author Closing Comment

by:MRS
ID: 39284285
thank you great point to get me moving forward.
0
 
LVL 7

Expert Comment

by:BT15
ID: 39286772
you are welcome.

sorry for the delay on responding.

http://winscp.net/eng/docs/task_synchronize_full
0

Featured Post

Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you see single cell contains number and text, and you have to get any date out of it seems like cracking our heads.
My attempt to use PowerShell and other great resources found online to simplify the deployment of Office 365 ProPlus client components to any workstation that needs it, regardless of existing Office components that may be needing attention.
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…
Suggested Courses

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question