FTP - Powershell

http://www.experts-exchange.com/Programming/Languages/Scripting/Powershell/Q_28684693.html ( closed)


upload.bat

@echo off
setlocal enabledelayedexpansion
set FtpFile=%~dp0upload.ftp
set CsvFolder=F:\Purple_Top
set logfile=upload.log

set CsvFile=
for /f "delims=" %%a in ('dir /o:-d /b "%CsvFolder%\*.csv" ^| findstr.exe "^[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9]_[0-9][0-9]_[0-9][0-9]\.csv$"') do (set CsvFile=%CsvFolder%\%%a)
if not defined CsvFile (
	echo No csv file found in '%CsvFolder%'!
	exit /b 1
)
echo Using '%CsvFile%' as file to upload.
call :ExportDataSection FTP "%FtpFile%"

echo Content of the ftp file:
type "%FtpFile%"

ftp.exe -i -v -s:"%FtpFile%" >"%logfile%"



REM ================================================================================
REM Only functions after this line!
REM ================================================================================
goto :eof
:ExportDataSection
REM *** Reads all lines listed in %1 (section) and writes them to %2 (file name).
REM *** Environment variables will be expanded.
set Section=%~1
set FileName=%~2
if exist "%FileName%" del "%FileName%"
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| C:\Windows\system32\find.exe /n "[%Section%]"') do set DataStart=%%a
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	call :Expand Line "%%a"
	if "!Line:~0,1!"=="[" (goto :eof) else (>>"%FileName%" echo !Line!)
)
goto :eof

:Expand
set %1=%~2
goto :eof

REM ================================================================================
REM Only data sections after this line!
REM ================================================================================
[FTP]
open heart1
hbc_data
hbc_data
cd /home/hbc_data/data/purple_top_data
mput %CsvFile%
bye

Open in new window


It's currently going by "modified" date, because a time stamp in mm_dd_yyyy format doesn't really lend itself for alphabetic sort.

As you mentioned the file in purple_top directory

1. 06_03_2015_14_49.csv  Modified date - 6/4/2015 9:13 AM

2. 06_02_2015_15_45.csv  Modified date - 6/3/2015 7:37 AM

As per your code when I run the batch file should it not ftp No 1 file? Please confirm. I am asking this as it did not ftp based on modified date.


The user will put one file everyday in the morning. 1 File a day by date and time.

Need help in getting the most recent file to the server.
LVL 6
anumosesAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

oBdACommented:
That's the older version of which I mentioned that in line 8, the "/o:-d" should have been "/o:d".
Anyway, here's an updated version that will now pick the latest file in the source folder based on the time stamp in the file name, not on the modified date anymore.
It contains the PS code to re-process the csv file in it, and it will now strip off the header line.
As before, the ftp command is inactive, remove the ECHO in front of line 38 to run it for real.
@echo off
setlocal enabledelayedexpansion
set FtpFile=%~dp0upload.ftp
set CsvFolder=F:\Purple_Top
set logfile=upload.log

set TmpFile=%Temp%\%~n0.tmp
set CsvFile=
(
	for /f "delims=" %%a in ('dir /o:d /b "%CsvFolder%\*.csv" ^| findstr.exe "^[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9]_[0-9][0-9]_[0-9][0-9]\.csv$"') do (
		for /f "tokens=1-5 delims=_." %%d in ("%%a") do (
			echo %%f%%d%%e%%g%%h	%%a
		)
	)
) >"%TmpFile%"
for /f "tokens=1*" %%a in ('type "%TmpFile%" ^| sort.exe') do (
	set CsvFile=%CsvFolder%\%%b
)
del "%TmpFile%"

if not defined CsvFile (
	echo No csv file found in '%CsvFolder%'.
	exit /b 1
)
echo Using '%CsvFile%' as input file.
for %%a in ("%CsvFile%") do (set UploadFile=%%~dpna-converted.csv)

echo Filtering unused columns, writing output to '%UploadFile%'.
powershell.exe -ExecutionPolicy RemoteSigned -Command "& {Import-Csv -Path '%CsvFile%' | Select-Object -Property 'Sample ID No', 'HGB(g/L)', 'PLT(10^3/uL)' | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | ForEach-Object {$_.Replace([char]34 + ',' + [char]34 , ',').Trim([char]34)} | Out-File -FilePath '%UploadFile%' -Encoding ASCII}"

call :ExportDataSection FTP "%FtpFile%"

echo Content of the ftp file:
type "%FtpFile%"

ECHO ftp.exe -i -v -s:"%FtpFile%" >"%logfile%"



REM ================================================================================
REM Only functions after this line!
REM ================================================================================
goto :eof
:ExportDataSection
REM *** Reads all lines listed in %1 (section) and writes them to %2 (file name).
REM *** Environment variables will be expanded.
set Section=%~1
set FileName=%~2
if exist "%FileName%" del "%FileName%"
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| C:\Windows\system32\find.exe /n "[%Section%]"') do set DataStart=%%a
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	call :Expand Line "%%a"
	if "!Line:~0,1!"=="[" (goto :eof) else (>>"%FileName%" echo !Line!)
)
goto :eof

:Expand
set %1=%~2
goto :eof

REM ================================================================================
REM Only data sections after this line!
REM ================================================================================
[FTP]
open heart1
hbc_data
hbc_data
cd /home/hbc_data/data/purple_top_data
mput %UploadFile%
bye

Open in new window

0
anumosesAuthor Commented:
Log file

ftp> User (heart1.heartlandbc.org:(none)): open heart1
ftp>

ftp> cd /home/hbc_data/data/purple_top_data
ftp> mput F:\Purple_Top\_.
F:\Purple_Top\_.:File not found
bye

this is the file that is there

06_03_2015_14_49.csv
0
anumosesAuthor Commented:
_. %d in (06_03_2015_14_49.csv) do (echo %f%d%e%g%h 06_03_2015_14_49-converted.csv

This existed in the folder f:purple_top
0
Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

anumosesAuthor Commented:
Is there a way that you can add another line in the code to move this into the archive folder at f:purple_top that I will create. This way the code will always detect one csv file. Let me know and I will add the folder. Thanks
0
oBdACommented:
This now moves the source csv into an archive folder.
If you remove the REM in front of line 29, it will delete the converted csv file as well.
@echo off
setlocal enabledelayedexpansion
set FtpFile=%~dp0upload.ftp
set CsvFolder=F:\Purple_Top
set logfile=upload.log
set ArchiveFolder=%CsvFolder%\Archive
set CsvFile=
if not exist "%ArchiveFolder%" md "%ArchiveFolder%"
for /f "delims=" %%a in ('dir /o:d /b "%CsvFolder%\*.csv" ^| findstr.exe "^[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9]_[0-9][0-9]_[0-9][0-9]\.csv$"') do (set CsvFile=%CsvFolder%\%%a)
if not defined CsvFile (
	echo No csv file found in '%CsvFolder%'.
	exit /b 1
)

echo Using '%CsvFile%' as input file.
for %%a in ("%CsvFile%") do (set UploadFile=%%~dpna-converted.csv)

echo Filtering unused columns, writing output to '%UploadFile%'.
powershell.exe -ExecutionPolicy RemoteSigned -Command "& {Import-Csv -Path '%CsvFile%' | Select-Object -Property 'Sample ID No', 'HGB(g/L)', 'PLT(10^3/uL)' | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | ForEach-Object {$_.Replace([char]34 + ',' + [char]34 , ',').Trim([char]34)} | Out-File -FilePath '%UploadFile%' -Encoding ASCII}"

call :ExportDataSection FTP "%FtpFile%"

echo Content of the ftp file:
type "%FtpFile%"

ECHO ftp.exe -i -v -s:"%FtpFile%" >"%logfile%"
del "%FtpFile%"
move "%CsvFile%" "%ArchiveFolder%"
REM del "%UploadFile%"

REM ================================================================================
REM Only functions after this line!
REM ================================================================================
goto :eof
:ExportDataSection
REM *** Reads all lines listed in %1 (section) and writes them to %2 (file name).
REM *** Environment variables will be expanded.
set Section=%~1
set FileName=%~2
if exist "%FileName%" del "%FileName%"
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| C:\Windows\system32\find.exe /n "[%Section%]"') do set DataStart=%%a
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	call :Expand Line "%%a"
	if "!Line:~0,1!"=="[" (goto :eof) else (>>"%FileName%" echo !Line!)
)
goto :eof

:Expand
set %1=%~2
goto :eof

REM ================================================================================
REM Only data sections after this line!
REM ================================================================================
[FTP]
open heart1
hbc_data
hbc_data
cd /home/hbc_data/data/purple_top_data
mput %UploadFile%
bye

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
anumosesAuthor Commented:
Perfect solution. Awesome job. Thanks
0
anumosesAuthor Commented:
Just one question. I know I have to use a mv command on the server to move the converted file to archive that I have already created.
/home/hbc_data/data/purple_top_date/archive
 Can you tell me where to add the code after it is converted on the server?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.