Link to home
Start Free TrialLog in
Avatar of jcwh
jcwh

asked on

Move files in a directory to another directory when the directory has a certain number of files in it

I'm doing a project that dealing with database.

I want to write a program that is able to check the number of files in a directory. This directory is used to store data files and therefore after a number of days, the directory will have too many files as NTFS can store 65536 files in a directory.

This program will monitor the directory and rename the directory to a BACKUP name if it has 65000 files in it.

eg.

Directory:  C:\ABC now has 65000 files in it.

Program will rename C:\ABC to C:\BACKABC1 and create the directory C:\ABC again.
Where now, C:\ABC will be an empty directory, ready to receive more data files.

and when C:\ABC is full again,

the program will automatically rename the directory, C:\ABC to C:\BACKABC2

The name of the backup directories will have an incremental numbers:
eg.
        BACKUP1
        BACKUP2
        BACKUP3...
        BACKUP10...
        BACKUP11
and so on...

So the program also needs to know what is the next directory it should create.

Please reply ASAP.
Thank you.
Avatar of shah1d1698
shah1d1698

Take a look at this site..

http://forums.devshed.com/archive/t-203777
ASKER CERTIFIED SOLUTION
Avatar of rstaveley
rstaveley
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Sorry - that link I gave you was a mis-paste. This is what I meant to paste: http://msdn.microsoft.com/library/en-us/script56/html/sgFileSystemObjects.asp
Here's a batch file that will do what you want.

** Note** if this is for a real-world application, where its IMPORTANT to not lose anything, you'd better add a LOT of error checking!!

-------------------------------------------------------------------------
Set TheMainDir=c:\test
Set TheBackupsDir=%TheMainDir%\BackupsGoHere
Set TheFilesDir=%TheMainDir%\FilesAccumulateHere
Set MaxFiles=2

:Again
rem  Count the files
Set /A  FTot = 0
For %%f in ( %TheFilesDir%\*.* ) do Set /A FTot = FTot + 1
echo ftot is %FTot%
if %FTot% LSS %MaxFiles%   goto :NoWork

rem count the dirs

Set /A  DTot = 0
For /D %%f in ( %TheBackupsDir%\*.* ) do Set /A DTot = DTot + 1
echo Dtot is %DTot%

Set /A DTot=DTot + 1

echo moving ....
move %TheFilesDir% %TheBackupsDir%\Backup%DTot%
echo Done ....
mkdir %TheFilesDir%
goto :Again

:NoWork
echo Only %FTot% files so far.....
pause

goto :Again


Avatar of jcwh

ASKER

BTW, rstaveley, how do I execute the .wsf file?
Can you give an example?

The website gave:
CScript //Job:MyFirstJob MyScripts.wsf

what should I replace //Job:MyFirstJob with?
You should just be able to double-click the .wsf. Since there is only one job in it, I left it nameless.

It should suffice to write:

    CScript MyScripts.wsf
If you want to have multiple jobs, give them IDs as indicated at http://msdn.microsoft.com/library/en-us/script56/html/wselejob.asp
Beware of my typo: archivname should be archivename (I've got biscuit crumbs in this keyboard)
The problem with renaming directories might be that it fails if any application has opened a file while renaming. So, instead of renaming i would suggest to frequently use a new subdirectory, e. g. any new day:

  c:\
      ABC\
             20041211\
             20041212\
             20041213\

These sub directories easily could be backuped. If you don't want the applications to have to care about directory names and current dates, you could map a drive letter to the current active directory:

        REM  unmap drive letter B
        subst b: /d
        REM get current date and set environment variable
        for /f "tokens=1,2*" %%i in ('date /t') do set TODAY=%%i%%j%%k
        REM create new directory
        if not exist c:\abc\%TODAY% md c:\abc\%TODAY%
        REM map new directroy
        subst B: "c:\abc\%TODAY%"
 

Now, applications always could create files using  "B:\\" as prefix.

Note, if you need directory in the format 'yyyymmdd' you may anaylize output of

     date /t

at the command line and create an appropiate  directory name for example  by

        set NEWDIR= %TODAY:~6,4%%TODAY:~3,2%%TODAY:~0,2%
       
where %TODAY:~6,4%   means 4 bytes at offset 6 of variable TODAY

Regards, Alex
That's ingenious, Alex. Nice to see that you can subst B: when B: is "busy" (I just tested it because I didn't believe you - oh me of little faith).

jcwh, if you want to run the subst from a WSH scripts, you need to use a shell object thus...
--------8<--------
// Shell object
var shell = new ActiveXObject("WScript.Shell");
shell.exec("%comspec% /c subst b: /d")
--------8<--------
On the other hand, perhaps I could humbly suggest that it's a real Kludge to use the file system as a database!

If this data is of anything more than trivial importance, it's a big risk to use NTFS as a database.  It's an even bigger risk to subject thousands of these files to the unreliable vagaries of batch files or scripting.  There are so many things that can go wrong!




>> it's a big risk to use NTFS as a database

Maybe it is simple log files that shouldn't be written to a DBMS. NTFS actually isn't  a bad choice for that. In a DBMS it's much easier to add new entries but getting rid of old ones ...

Using NTFS you simply have to

     format c:

and all data was *backuped* forever (it's a joke).


Regards, Alex
> it's a real Kludge to use the file system as a database!

ala WinFS
Avatar of jcwh

ASKER

Those files only add as a source of raw data for the database.
Thus there is other program to convert these raw data into usable data and stored in database such as Oracle.

And we may want to keep these raw data, just incase the Oracle database is corrupted.
Avatar of jcwh

ASKER

rstaveley, your code works, but I spotted a typo error:

// Find the next applicable archive name
     var archive_number = 1;
     for (;fso.FolderExists(archivname+archive_number);archive_number++)

should be:
     for (;fso.FolderExists(archivename+archive_number);archive_number++)

your archivename spelled wrongly.

Thanks for all your help.
> typo

Yes, I spotted that too http:Q_21240034.html#12809736 . Alex is quite right that the directory name will fail if the directory is busy. It this going to be on a scheduler when it is known not to be busy? If not, it is worth considering his subst idea.
Avatar of jcwh

ASKER

When the script runs, there won't be any program accessing the data files as I execute the script from my VC++ program. It is run after other functions.