asked on

Deleting large qty of files - based on File name filters - what program to use?

Hi,
I have several years of files from our security system. The names are in this type format, as an example:
Office.20150621_093154
Office.20150621_093034

name of room. year month date _ hour minutes seconds

What I'd like to do is find a way to go through and delete any file that has an 'hour' between, for instance, 01 and 06 (so any file between 1:00AM and 6:00AM). To do this I'd need to be able to somehow filter based on the right 5th and 6th digits, or the two digits to the right of the "_". Not sure the best way to do this.

There are over 500,000 of these files, so manually is not an option

Let me know if any suggestion

ASKER CERTIFIED SOLUTION

oBdA

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Mystical_Ice

ASKER

Files have a regular extension after the date.

One more thing - Is it possible to only delete where, for instance, there are more than 3 events for a particular hour and particular camera (Office, Entryway, Hallway, etc).

So for instance:
Office.20150621_093154
Office.20150621_093034
Office.20150621_092524
Office.20150621_092336
Office.20150621_092212
Office.20150621_092017
Office.20150621_071515

In the above example, because the "071515" is the only event in the hour, it's more than likely legitimate. Whereas the others that are all in the '09' hour are probably a spider crawling over the lens triggering the file, so can be deleted.

oBdA

Try this, in test mode again; note that there are two ECHOs that need to be removed to arm the script, in line 18 and line 28. Note that because of the need for sorting the results first, it might take a while for something to happen; test it on a smaller test folder first.
If it turns out to be too slow or just too much for batch with the 500000 files, would Powershell be an option? And are the files flat in one directory or spread out over several subfolders?

@echo off
setlocal enabledelayedexpansion
set Root=D:\Temp
set Mask=*.mov
set MaxEvents=3

set OldHash=
set /a FileCount = 0
for /f "tokens=1-3* delims=_." %%a in ('dir /s /b /a:-d "%Root%\%Mask%" ^| sort.exe') do (
	echo Processing '%%a.%%b_%%c' ...
	set FileTime=%%c
	set NewHash=%%a!FileTime:~0,2!
	if /i "!OldHash!"=="!NewHash!" (
		set /a FileCount += 1
	) else (
		if !FileCount! gtr %MaxEvents% (
			for /l %%i in (1, 1, !FileCount!) do (
				ECHO del "!DelFile[%%i]!"
			)
		)
		set OldHash=!NewHash!
		set /a FileCount = 1
	)
	set DelFile[!FileCount!]=%%a.%%b_%%c.%%d
)
if !FileCount! gtr %MaxEvents% (
	for /l %%i in (1, 1, !FileCount!) do (
		ECHO del "!DelFile[%%i]!"
	)
)

Open in new window

Mystical_Ice

ASKER

They are all in one big directory. And powershell I don't know much about

Mystical_Ice

ASKER

What I ended up doing, and for anyone viewing this question in the future, was just using the built-in windows search capability.

Searching for:

"_03*", "_04*", "_05*", etc. would find anything that contained _03, _04, _05 and so forth.

Thanks all for your help!