Solved

Copying latest files in another folder

Posted on 2013-12-06
17
518 Views
Last Modified: 2013-12-11
I have these 2 scripts however it only copies the latest file into another folder.
I want it to copy any more recent files since the last time it ran last into the other folder.

Currently, it only copies the latest file, however let's say I ran this at 12:01, and it copied the latest file into the other folder, now that I run it at 12:29, if there are 2 more files it finds,  one from 12:25 pm and one at 12:26 pm, I want both files to be copied into the other folder, not just the latest one, which would of been the one at 12:26 pm.

setlocal enabledelayedexpansion
set SourceMask=*.txt
set SourceFolder=c:\Programs\A\P\CopyIn
set TargetFolder=c:\Programs\A\P\LatestIn
set IniFile=%~dpn0.ini
set LatestFile_Name=
set LatestFile_Size=
set LatestFile_Name=
for /f "delims=" %%a in ('dir /a:-d /b /o:-d "%SourceFolder%\%SourceMask%" 2^>NUL') do (
    set LatestFile_Name=%%~nxa
    set LatestFile_Size=%%~za
    set LatestFile_Date=%%~ta
    goto Break_LatestFile
)
:Break_LatestFile
if "%LatestFile_Name%"=="" (
    echo No files matching '%SourceMask%' found in '%SourceFolder%'.
    goto Leave
)
if exist "%IniFile%" (
    for /f "tokens=1* delims==" %%a in ('type "%IniFile%"') do (set %%a=%%b)
) else (
    set CopiedFile_Name=
    set CopiedFile_Size=
    set CopiedFile_Date=
)
if "%LatestFile_Name%_%LatestFile_Size%_%LatestFile_Date%"=="%CopiedFile_Name%_%CopiedFile_Size%_%CopiedFile_Date%" (
    echo File '%CopiedFile_Name%' has already been copied to '%TargetFolder%'.
    if exist "%TargetFolder%\%LatestFile_Name%" (
        del "%TargetFolder%\%LatestFile_Name%"
        echo Deleted file '%CopiedFile_Name%' from '%TargetFolder%'.
    )
    goto Leave
)
if exist "%TargetFolder%\%SourceMask%" del /q "%TargetFolder%\%SourceMask%"
copy "%SourceFolder%\%LatestFile_Name%" "%TargetFolder%" >NUL
echo Copied file '%LatestFile_Name%' to '%TargetFolder%'.
if exist "%IniFile%" del "%IniFile%"
for %%a in (Name Size Date) do (
    >>"%IniFile%" echo CopiedFile_%%a=!LatestFile_%%a!
)


ping -n 8 -w 1000 127.0.0.1 > nul

pushd "C:\Programs\A\P\CopyIn"
for /f "tokens=* delims= " %%G in ('dir/b/od') do (set newest=%%G)
copy %newest% C:\Programs\A\P\CopyofIncoming
popd

Open in new window

0
Comment
Question by:100questions
  • 8
  • 4
  • 2
  • +1
17 Comments
 
LVL 43

Expert Comment

by:Steve Knight
ID: 39701601
Do you keep the files in the new folder all the time then, or are they moved again etc?

i.e. how will it know that there are two new files, we would either need to know which files were copied by recording their names somewhere, or copying any files not already there say?

i.e. perhaps even an xcopy /d of anything changed today?

Also can we 'touch' the original area files, i.e. if we could reset the archive attribute or move them to a subdir etc. it would be much easier.

Steve
0
 

Author Comment

by:100questions
ID: 39701812
Thanks.  Actually you can ignore the last batch,...

pushd "C:\Programs\A\P\CopyIn"
for /f "tokens=* delims= " %%G in ('dir/b/od') do (set newest=%%G)
copy %newest% C:\Programs\A\P\CopyofIncoming
popd

When I run the .bat file it creates an .ini file where I believe a file name is stored.
Perhaps that .ini file can store the latest files it copied etc?

I just want it to copy the last newest files, compared to the last time it ran, into the LatestIn, actually is should be renamed to LatestOut infact, not to make it confusing.
Every time the script runs and it detects a newer file, or newer files, in the CopyIn, which should actually be called the CopyOut (sorry for the confusion), it should clear the LatestOut folder and copy the newer files in there.  Hope this helps.
0
 
LVL 43

Expert Comment

by:Steve Knight
ID: 39702171
Sorry off doing other things now and not going to have time for any EE bits until Monday.  Might want to press "request attention" if no-one else picks it up.

Steve
0
 
LVL 51

Expert Comment

by:Bill Prew
ID: 39703043
I think the XCOPY /D option *might* be useable here, but not sure yet.

I need to understand better how these two folders are used I think.  In general terms you are looking to copy "newer" files from the CopyOut folder to the LatestIn. But is newer defined as a file that exists in both folders and has a newer time stamp? Or a file that exists in the CopyOut folder but not the LatestIn folder? Of a file that was received after the last time this BAT script ran?

When a file is copied from CopyOut to Latest In, shouldn't it be removed from CopyOut? Or could it still be there next time, even after it was processed from LatestIn?

When the file is processed from LatestIn, is it deleted by that process? Or is the only way it will be deleted is by this script, when there are "newer" files found to process, and all files are removed from LatestIn first?

Will the same file name show up in future days, with new data, or are the file names always the same?

~bp
0
 

Author Comment

by:100questions
ID: 39703100
Hi Bill, sorry for the confusion.
Let me rename the paths so that I hope they make a little sense and I'll try to answer some questions too.

set SourceFolder=c:\Programs\A\P\OutgoingFiles
set TargetFolder=c:\Programs\A\P\LatestOutgoingFiles

The OutgoingFiles folder has many files, which accumulate from day to day since it contains a copy of Outgoing Files, files that are Sent.
When executing the script, I want it to copy the Newer files found in OutgoingFiles compared to the last time the script ran - and copy this file or files to the LatestOutgoingFiles folder.
Before it copies the latest/newer file or files to the LatestOutoingFiles folder, it needs to clear that folder of any files first.  So essentially, all I will see in the LatestOutgoingFiles folder are a copy of the latest/newest file or files it finds in the OutgoingFiles folder - compared to the last time it checked.  Please note that when the script runs, whether it finds a newer file or newest files in the OutgoingFiles folder or not, it needs to clear out the LatestOutgoingFiles folder, regardless if it it copies something newer in it or not.  If it does not find any changes, for instance if the same file or files, are in the OutoingFiles folder and there is nothing newer, it should not copy the same file or files back into the LatestOutgoingFiles folder again, but just clear the LatestOutoingFiles of any files and leave it empty.

Currently, the script does this in part but it only takes 1 file, the last, newest file it find in OutgoingFiles - however if there were 2 files for instance since the last time the script ran, it will only copy the latest/newst file but not both those newer files.

File names are constantly changing and different.



I hope this helps.
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 39705009
I have a question?

FileA  Created Date 12/8/2013 6:34 PM

Run your copy batch

\\ServerA\FolderA\FileA --> \\ServerB\FolderB\FileA  

FileA Modified Date 12/8/2013 8:40 PM

Run batch.  Wont it overwrite FileA with the newest one?
Are the name different?  Not sure how this would work unless there is a procedure in place to move/copy the previous file prior to overwriting it in the current location
0
 

Author Comment

by:100questions
ID: 39705084
set SourceFolder=c:\Programs\A\P\OutgoingFiles
set TargetFolder=c:\Programs\A\P\LatestOutgoingFile

The OutgoingFiles folder receives new files daily which accumulate.
If the script sees a new file or files since the last time the script ran, it will clear/remove/delete the file in the LatestOutgoingFile folder, and copy the newest file or files in there.

If the files are not newer then it still clears/removes/deletes the file or files in the LatestOugoingFolder.
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 39705087
But if the file is the same name wont it just copy over the older file?
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:100questions
ID: 39705100
The file which is copied into the OutgoingFiles will always be different.
0
 

Author Comment

by:100questions
ID: 39706185
@billprew - Hi Bill, any suggestions based on my last clarifying post to you?
Thanks in advance.
0
 
LVL 51

Accepted Solution

by:
Bill Prew earned 500 total points
ID: 39707669
Okay, I tried to keep some of the old structure so that this might be easier to digest.  This seems to do what you are looking for, hope it makes sense.  Save as a BAT file and adjust the source and dest folder names at the top to match your folders.

@echo off
setlocal enabledelayedexpansion

REM Define source and destination folders, and mask for files to process
set SourceFolder=C:\EE\EE28312389\source
set TargetFolder=C:\EE\EE28312389\dest
set SourceMask=*.txt

REM Define INI file in same location as BAT file
set IniFile=%~dpn0.ini

REM Load any existing INI settings into environment variables
set CopiedFile_Name=
if exist "%IniFile%" (
    for /f "tokens=1* delims==" %%A in ('type "%IniFile%"') do (set %%A=%%B)
)

REM Delete any existing files from destination folder
if exist "%TargetFolder%\%SourceMask%" del /q "%TargetFolder%\%SourceMask%"

REM Process the files in the source folder, in reverse chronological order (newest to oldest)
set Count=0
for /f "delims=" %%A in ('dir /a:-d /b /o:-d "%SourceFolder%\%SourceMask%" 2^>NUL') do (

    REM See if this file was the latest we copied last time, if so we're done
    if "%%~nxA" EQU "%CopiedFile_Name%" (
        goto :Break_LatestFile
    )

    REM If this is the first file (newest) save it's name for INI write later
    if !Count! EQU 0 (
        echo CopiedFile_Name=%%~nxA>"%IniFile%"
    )

    REM Copy the file, and display it    
    copy "%SourceFolder%\%%~A" "%TargetFolder%" > NUL
    echo Copied file '%SourceFolder%\%%~A' to '%TargetFolder%'.
    set /a Count+=1
)
:Break_LatestFile

REM See if there were any files in the source folder to be processed
echo '%Count%' new files matching '%SourceMask%' copied from '%SourceFolder%' to '%TargetFolder%'.

REM Done
:Leave

Open in new window

~bp
0
 

Author Closing Comment

by:100questions
ID: 39709851
Very well done, many thank Bill.
0
 
LVL 51

Expert Comment

by:Bill Prew
ID: 39710081
Welcome, thanks for the feedback.

~bp
0
 

Author Comment

by:100questions
ID: 39712336
@billprew - Hi bill, one last request on the above please, I think the location of the .ini file may conflict with the another similar script I have going, is there a way I can place the .ini file related to this script somewhere else?

Say if I wanted to place it at C:\Programs\Script... how can I do that.

Write now you wrote:  set IniFile=%~dpn0.ini
0
 
LVL 51

Expert Comment

by:Bill Prew
ID: 39712522
Sure, in the existing statement:

 set IniFile=%~dpn0.ini

the "d" will be replaced with the drive letter where the BAT file exists, the "p" will be replaced with the full path of the folder where the BAT file exists, and the "n" will be replaced by the name of the BAT file (no extension).

So you can place it in another folder easy enough like this:

 set IniFile=C:\Programs\Script\%~n0.ini

~bp
0
 

Author Comment

by:100questions
ID: 39712909
Thanks kindly.
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

When you upgrade from Windows 8 to 8.1 or to Windows 10 or if you are like me you are on the Insider Program you may find yourself with many 450MB recovery partitions.  With a traditional disk that may not be a problem but with relatively smaller SS…
A Bare Metal Image backup allows for the restore of an entire system to a similar or dissimilar hardware. They are highly useful for migrations and disaster recovery. Bare Metal Image backups support Full and Incremental backups. Differential backup…
Windows 8 comes with a dramatically different user interface known as Metro. Notably missing from the new interface is a Start button and Start Menu. Many users do not like it, much preferring the interface of earlier versions — Windows 7, Windows X…
The Task Scheduler is a powerful tool that is built into Windows. It allows you to schedule tasks (actions) on a recurring basis, such as hourly, daily, weekly, monthly, at log on, at startup, on idle, etc. This video Micro Tutorial is a brief intro…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now