batch file or some other automation script for simple-ish file operation

Hello,

I shoot RAW video, result is that each take is a separate directory filled with photos camera takes 24 photos per second(dependent on FPS setting), each one of the photos is named sequentialy in the filename.

so each take is a subdirectory in the root directory

I'm trying to create a script that would traverse the root directory and all it's subdirectories and simply copy files(photo/image) based on their filename, and just copy every 1440th file out of each subdirectory, effectively giving me stills from the video footage every 1 minute, to a designated directory.

Does that make sense? I'm just looking to have a batch file to perform that, this would give me a sampling of all the footage from the day in 1 minute increments, batch file or if there is another way to automate this copying setup with some amount of control as to the interval, that would be great.

here are example three sequential file names from one of the directories.

bmpcc   001_1_2015-09-17_0400_C0005_000000.dng
bmpcc   001_1_2015-09-17_0400_C0005_000001.dng
bmpcc   001_1_2015-09-17_0400_C0005_000002.dng

and so on, so the number of files in each directory varies, script would need to stop when the last file is considered and go to the next directory to copy files from there

could someone help me with a script or write it for me if it's simple or point me to an application that could be setup to do something like this?
mark reduxmotionAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

NVITEnd-user supportCommented:
This copies all files having a "001440" filename suffix.
Target folders are created based on source file folder.

Make a .bat of this. Change the value after the = for RootDir, CopyToDir, Sequence to your needs:
set RootDir=C:\Local\test\dir
set CopyToDir=C:\Local\test\copyto
set Sequence=001440
xcopy /s "%RootDir%\bmpcc   *_%Sequence%.dng" "%CopyToDir%"

Open in new window

mark reduxmotionAuthor Commented:
I think you misunderstood, I meant every 1440th file in directory, so for example, based on example filenames I posted...

bmpcc   001_1_2015-09-17_0400_C0005_000000.dng
bmpcc   001_1_2015-09-17_0400_C0005_001440.dng
bmpcc   001_1_2015-09-17_0400_C0005_002880.dng
bmpcc   001_1_2015-09-17_0400_C0005_003960.dng

and so on, as long as the file exists, I would also like to be able to easily change the increment from 1440 to something else, for example 6000

each directory will have vastly different number of files so before copying script needs to check if file with that name exists and if it doesn't skip to next directory and stop over, from 1440 for example...
Dan CraciunIT ConsultantCommented:
Not tested, but this should do what you want in Powershell:

gci "X:\path\to\img\folder" -File -Recurse | ?([int]$_.Name.Substring($_.Name.Length - 6, 6) % 1440 -eq 0) | Copy-Item $_.FullLength "X:\path\to\still\folder\"+$_.Name

Open in new window

Replace the paths and 1440 with what you want.

HTH,
Dan
OWASP: Avoiding Hacker Tricks

Learn to build secure applications from the mindset of the hacker and avoid being exploited.

mark reduxmotionAuthor Commented:
again, only a single file will match these answers, I'm looking for a batch file or whatever that may be that will copy files with multiples of the number 1440 in the name, or whatever other number I may set, plus it would need to traverse all subdirectories under specified root directory and in each directory do the same, copy all existing files with multiplied number 1440 in their name
Dan CraciunIT ConsultantCommented:
You do realize that
number modulo 1440 = 0
will give you all the multiples of 1440, right?

In Powershell, that's written as number % 1440 -eq 0

And
[int]$_.Name.Substring($_.Name.Length - 6, 6)
will get you an integer from the last 6 characters of the file name.

That one liner will go recursively through all the files in your picture folder, check if the last 6 chars are multiples of 1440 and if so copy the file to your still folder.
oBdACommented:
Here's a batch that copies every nth file (not necessarily multiplies, if the numbers aren't contiguous). It's currently in test mode and will only display the files it would copy; remove the REM in line 13 to run it for real.
@echo off
setlocal enabledelayedexpansion
set Source=C:\Temp\Takes
set Target=C:\Temp\Target
set Increment=1440
set Mask=*.dng
for /f "delims=" %%a in ('dir /b /a:d /o:n "%Source%\*.*"') do (
	echo Processing %%a ...
	set /a Index = %Increment%
	for /f "delims=" %%b in ('dir /b /a:-d /o:n "%Source%\%%a\%Mask%"') do (
		if !Index!==%Increment% (
			echo 	- '%%b'
			REM copy "%Source%\%%a\%%b" "%Target%"
			set /a Index = 1
		) else (
			set /a Index += 1
		)
	)
)

Open in new window

mark reduxmotionAuthor Commented:
I tried the powershell script and here is what happens

PS C:\Users\user> gci "I:\Rob day 1\bmpcc   001_1_2015-09-17_0341_C0001" -File -Recurse | ?([int]$_.Name.Substring($_.Na
me.Length - 6, 6) % 1440 -eq 0) | Copy-Item $_.FullLength "I:\stills\"+$_.Name
You cannot call a method on a null-valued expression.
At line:1 char:1
+ gci "I:\Rob day 1\bmpcc   001_1_2015-09-17_0341_C0001" -File -Recurse ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull
oBdACommented:
And here's a working PS version (in test mode; remove the -WhatIf argument of Copy-Item to run it for real).
Condensed:
gci "C:\temp\Takes\*.dng" -File -R | ? {!($_.BaseName.Split("_")[-1] % 1440)} | % {cp $_.FullName "C:\temp\Target" -WhatIf}

Open in new window

Fleshed out:
$Source = "C:\temp\Takes\*.dng"
$Target = "C:\temp\Target"
$Increment = 1440
Get-ChildItem -Path $Source -File -Recurse |
	Where-Object {
		($_.BaseName.Split("_")[-1] % $Increment) -eq 0
	} | 
	ForEach-Object {
		Copy-Item -Path $_.FullName -Destination $Target -WhatIf
	}

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
mark reduxmotionAuthor Commented:
$Source = "C:\temp\Takes\*.dng"
$Target = "C:\temp\Target"
$Increment = 1440
Get-ChildItem -Path $Source -File -Recurse |
      Where-Object {
            ($_.BaseName.Split("_")[-1] % $Increment) -eq 0
      } |
      ForEach-Object {
            Copy-Item -Path $_.FullName -Destination $Target -WhatIf
      }

will this one also traverse all subdirectiories under takes? so for example C:\temp\Takes\1\, C:\temp\Takes\2\, C:\temp\Takes\3\ and so on?
oBdACommented:
Yes, that's what the "-Recurse" does (or -R in the condensed version).
You can just run it as it is, it will only generate console output for the files it would copy as long as Copy-Item has the -WhatIf argument.
Or replace the "Copy-Item ..." line with Write-Host $_.FullName, which makes the output more readable than the -WhatIf.
mark reduxmotionAuthor Commented:
now you got me confused, sorry, could you post the full script with the modifications you mentioned? I've never used the PS script in my life...
oBdACommented:
Well, there's always the batch solution I posted earlier ...
Anyway, condensed with Write-Host instead of Copy-item:
gci "C:\temp\Takes\*.dng" -File -R | ? {!($_.BaseName.Split("_")[-1] % 1440)} | % {Write-Host $_.FullName}

Open in new window

Fleshed out with Write-Host instead of Copy-item:
$Source = "C:\temp\Takes\*.dng"
$Target = "C:\temp\Target"
$Increment = 1440
Get-ChildItem -Path $Source -File -Recurse |
	Where-Object {
		($_.BaseName.Split("_")[-1] % $Increment) -eq 0
	} | 
	ForEach-Object {
		Write-Host $_.FullName
	}

Open in new window

mark reduxmotionAuthor Commented:
I don't think it works, what am I doing wrong?

PS C:\Users\user> $Source = "I:\rob day 22\*.dng" |$Target = "I:\stills\" | $Increment = 1440 | Get-ChildItem -Path
$Source -File -Recurse | Where-Object { | ($_.BaseName.Split("_")[-1] % $Increment) -eq 0 | } | ForEach-Object { | C
opy-Item -Path $_.FullName -Destination $Target -WhatIf}
At line:1 char:34
+ $Source = "I:\rob day 22\*.dng" |$Target = "I:\stills\" | $Increment  ...
+                                  ~~~~~~~
Expressions are only allowed as the first element of a pipeline.
At line:1 char:59
+ ...  = "I:\rob day 22\*.dng" |$Target = "I:\stills\" | $Increment = 1440  ...
+                                                        ~~~~~~~~~~
Expressions are only allowed as the first element of a pipeline.
At line:1 char:139
+ ... Get-ChildItem -Path $Source -File -Recurse | Where-Object { | ($_.Bas ...
+                                                                 ~
An empty pipe element is not allowed.
At line:1 char:190
+ ... -Object { | ($_.BaseName.Split("_")[-1] % $Increment) -eq 0 | } | For ...
+                                                                  ~
An empty pipe element is not allowed.
At line:1 char:212
+ ... e.Split("_")[-1] % $Increment) -eq 0 | } | ForEach-Object { | Copy-It ...
+                                                                 ~
An empty pipe element is not allowed.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : ExpressionsMustBeFirstInPipeline

PS C:\Users\user>
Bill PrewIT / Software Engineering ConsultantCommented:
Here's a BAT script approach that should work, but you have to be patient, it does take a little bit of time to work...

@echo off
setlocal EnableDelayedExpansion
set BaseDir=B:\EE\EE28734759\Base
set DestDir=B:\EE\EE28734759\Dest
set Filter=*.dng
set Increment=1440

for /f "delims=" %%A in ('dir /s /b /a-d "%BaseDir%\%Filter%"') do (
  set Filename=%%~nA
  set /a Number=1!Filename:~-6! - 1000000
  set /a Mod=!Number! %% Increment
  if !Mod! == 0 (
    echo copy "%%~A" "%DestDir%"
  )
)

Open in new window

~bp
oBdACommented:
Two possibilities:
1. Click "Select all" under the code box, copy it into the clipboard, paste it into an open PS console, hit Enter; this will only display the files it would copy:
gci "I:\rob day 22\*.dng" -File -R | ? {!($_.BaseName.Split("_")[-1] % 1440)} | % {Write-Host $_.FullName}

Open in new window

2. Click "Select all" under the "fleshed out" script, copy and paste it into Notepad, change $Source, $Target, $Increment to your likings, save it for example as C:\Temp\whatever.ps1.
In a PS console, enter
C:\Temp\whatever.ps1
If you get an error message about unsigned scripts and the execution policy, enter
Set-ExecutionPolicy RemoteSigned -Force
and try again.
mark reduxmotionAuthor Commented:
but I don't see the destination folder here for matching files?

gci "I:\rob day 22\*.dng" -File -R | ? {!($_.BaseName.Split("_")[-1] % 1440)} | % {Write-Host $_.FullName}

Sorry to be a pain, it's been years since I did any code writing, I'm now  a DP... it's healthier then programming, but  this code looks like magic to me now =)
oBdACommented:
As I said: this will only list the files it would copy; it does not actually copy anything. I's just for testing.
The following versions are live and will actually copy (if they find anything):
gci "I:\rob day 22\*.dng" -File -R | ? {!($_.BaseName.Split("_")[-1] % 1440)} | % {cp $_.FullName "I:\stills"}

Open in new window

$Source = "I:\rob day 22\*.dng"
$Target = "I:\stills"
$Increment = 1440
Get-ChildItem -Path $Source -File -Recurse |
	Where-Object {
		($_.BaseName.Split("_")[-1] % $Increment) -eq 0
	} | 
	ForEach-Object {
		Copy-Item -Path $_.FullName -Destination $Target
	}

Open in new window

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VB Script

From novice to tech pro — start learning today.