We help IT Professionals succeed at work.

Sorting files in a given directory

scooter1
scooter1 asked
on
Medium Priority
1,247 Views
Last Modified: 2008-02-01
ok part of a program i'm working on needs to take a given directory say "C:\\myfiles" create a new sub directory say "C:\\myfiles\\group1\\" and then MOVE UP TO 795 MB of the files in "myfiles" to it. Then repeate untill the folder is completly sub divided. So if there were 1600 MB of files in "myfiles" you would have about 795 MB in "group1" and 795 MB in "group2" and about 10 MB in "group3". Any suggestions.
Comment
Watch Question

ekc

Commented:
This problem has two parts - finding and browsing the original file list, and copying the files to another location.
For the first problem, I recommend you the class CFileFind, which has many usefull operations for file manipulation.
For the second part of the problem, take a look to the MFC class CFile, or simply use the I/O basic functions like CreateFile(), DeleteFile(), CopyFile() etc. CopyFileEx() is doing the copy asyncronicly, and this can help you to avoid the thread programming and still have good performance...
Check SHFileOperation API, that can use for copy/move/delete all files in Directory
CERTIFIED EXPERT
Author of the Year 2009

Commented:
You would be crazy to actually copy this file data to the new directories -- unless you need duplicates.  Just use the rename function.  Size of files is unimportant since just a directory entry is changed -- you can "move" 100GB of data in 10ms that way.  

To create the new directory, use the mkdir() fn.

To move the files from one directoy to another, sue the rename() fn.

Yes, there are lots of pretty objects (such a CFile) and higher levels (such as SHFileOperation), but they all boil down to calling the low-level mkdir and rename code.

It is the most trivial of file operations.  Do you need some example code?

-- Dan
CERTIFIED EXPERT
Author of the Year 2009

Commented:
Your question's title mentions "sorting files," but you don't mention anything about sorting in the question itself.  Would you explain your needs?

I suspect that you need to create directories like this:

c:\myfiles\AtoM
c:\myfiles\NtoS
c:\myfiles\TtoZ

and move files whose filenames begin with A through M from c:\myfiles to c:\myfiles\AtoM

Is this correct?  Or do you have some other criteria for deciding which files go into which directory?.

Unless it is a very simple criteria, such as moving all fils with a certain extension or date, you may need to create and populate a list of all files in c:\myfiles and use a utility such as qsort to order them, then go down the list using rename with each one.

What is the criteria you will use to decide which directories to create and which files to put there?

-- Dan

Author

Commented:
oh my criteria isn't really criteria I didn't word it right. I really just want to like partition the directory. So it can take the first 795 MB and put this in a folder called "group1", then take the next 795 MB and move to folder "group2" The problem is creating a directory moving no more than 795 MB to it, then creating another directory moving up to 795 MB to it and so on till all the files are now in a new sub-directory.
CERTIFIED EXPERT
Author of the Year 2009
Commented:
Use FindFirstFile and FindNextFile followed by CreateDirectory and MoveFile.  Keep track of how may bytes you have moved and when you go over 795MB, create a new directory and resume moving files to there.

// note: assumes no file is larger than  4,294,967,295 bytes (4GB)

void main()
{
    char szBaseDir[]= "c:\\temp";
    char szWildFile[]= "c:\\temp\\*.*";
    char szNewDir[ MAX_PATH ];
    char szNewFile[ MAX_PATH ];
    char szOldFile[ MAX_PATH ];

    DWORD nMaxXferLen= 795000000;

    WIN32_FIND_DATA rFFD;
    HANDLE hFind= FindFirstFile( szWildFile, &rFFD );
    if ( hFind == INVALID_HANDLE_VALUE ) {
        // do error handling  and exit
    }
    int nDirsCreated= 0;

    DWORD nSizeSoFar= nMaxXferLen +1; // force new dir on first file
    do {
        if ( rFFD.cFileName[0]== '.' ) {
            continue;  // skip . and .. dirs
        }
        if ( rFFD.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY ) {
            continue;  // skip all other dirs
        }
        if ( nSizeSoFar > nMaxXferLen ) {
            sprintf( szNewDir, "%s\\%08d", szBaseDir, nDirsCreated+1 );
            if (! CreateDirectory( szNewDir, NULL ) ) {
                // do some error handling and exit
            }
            nDirsCreated++;
            nSizeSoFar= rFFD.nFileSizeLow;
        }
        sprintf( szOldFile, "%s\\%s", szBaseDir, rFFD.cFileName );
        sprintf( szNewFile, "%s\\%s", szNewDir, rFFD.cFileName );
        if (! MoveFile( szOldFile, szNewFile ) ) {
            // do some error handling and exit
        }
        nSizeSoFar += rFFD.nFileSizeLow;
    } while( FindNextFile( hFind, &rFFD ) );

    FindClose(hFind);
}

Watch out for one case:  If you have moved 794999999 bytes and then a really large file comes along, it will be moved into the current "new directory"  So, worst case, a new directory could have total bytes far greater than the desired max.  Unlikely but possible.  

The alternative is to check to see if the new total > max *before* moving and if so, create the dir them.  But in that case, you better guard against the chance that any single file > max.

With some effort, you could make this a lot smarter to avoid these scenarios.  Generate a list of files and try to find combinations of files that are nearly the desired max size.

-- Dan

Author

Commented:
dude sweet, thanks.

Explore More ContentExplore courses, solutions, and other research materials related to this topic.