Link to home
Start Free TrialLog in
Avatar of simondopickup
simondopickup

asked on

Scan all folders within a directory

Experts,

Does anyone know of a function that will enable me to scan all the files AND contents of subfolders of a directory. I am currently using FindFirst and FindNext functions, however these functions are only currently searching for files in the root directory - not all subfolders within it too.
string FullPath = DirPath;						// Store inputted DirPath Variable as FullPath before amending
FullPath = FullPath + "*.*";					// Add the .* extension to check for all files (notice + operator overload)
 
 
FndHnd = FindFirstFile(FullPath.c_str(), &FindDat);		// Use of FindFirstFile - handle returned

Open in new window

Avatar of evilrix
evilrix
Flag of United Kingdom of Great Britain and Northern Ireland image

I'm afraid you'll have to implement this yourself. It's not too hard, every time you come across a new folder start a new FindFirstFile/FindNextFile by recursively calling the current function encapsulating your scanning logic with the new folders details.. When you finish with the current folder return from the function and carry on where you was in the previous one. This will implement depth first scanning. Doing width first is a little trickier so if you don't need it then stick with depth first.
Avatar of simondopickup
simondopickup

ASKER

Okay - in JKR's example, at what point do i define a root directory path?  I am debugging JKR's code to see what it does, but i am returning at main() because my argument count is 1....

#include <windows.h>
#include <stdio.h>
 
void HandleFile ( WIN32_FIND_DATA* pw32fd);
void WalkTree ( char* pszPath,  char*   pszBase);
void main ( int argc, char** argv)
{
    if  (   argc    !=  2)  return;    <--- returning here when i run in console
 
    WalkTree ( *( argv + 1),    NULL);
}
>> at what point do i define a root directory path?
WalkTree ( *( argv + 1),    NULL); // This is the entry point so this would be where you pass in the root path.

>> but i am returning at main() because my argument count is 1....
That's because the first argv is the program name (as defined by the C standard), you need to provide the next arg as a path

walk.exe <path>

Of course, argv is just being used here for the purposes of the example.

In fact - a quick annotation of JKR's code below - would be amazing. It is taking an age looking at the msdn for every keyword i find.....this would really assist me if someone could take the time...

Many thanks
#include <windows.h>
#include <stdio.h>
 
void HandleFile ( WIN32_FIND_DATA* pw32fd);						
void WalkTree ( char* pszPath,  char*   pszBase);				
 
void main ( int argc, char** argv)
{
    if  (   argc    !=  2)  return;								 
    WalkTree ( *( argv + 1),    NULL);									
void WalkTree ( char* pszPath,  char*   pszBase)
{
WIN32_FIND_DATA w32fd;
HANDLE hFind;
DWORD dwAtt;
char acPath [ MAX_PATH];
char acBase [ MAX_PATH];
 
printf  (   "WalkTree():\tcalled with '%s' '%s'\n",   pszPath,    pszBase);		 
if ( '.' == * (pszPath + lstrlen (pszPath) - 1))								                return;
 
if  (   pszBase)													   sprintf (   acPath, "%s\\%s",   pszBase,    pszPath);		
else
    lstrcpy ( acPath, pszPath);										
printf ( "path is %s\n",    acPath);
lstrcpy ( acBase, acPath);
 
dwAtt = GetFileAttributes ( acPath);
 
if ( 0xffffffff == dwAtt)
{
 // error ...
}
 
if ( FILE_ATTRIBUTE_DIRECTORY & dwAtt)
{
    if  (   '\\'    ==  acPath  [   lstrlen (   acPath) -   1])
            lstrcat (   acPath, "*.*");
     else
            lstrcat (   acPath, "\\*.*");
 
    printf ( "path is now %s\n",    acPath);
}
 
 
hFind = FindFirstFile ( acPath, &w32fd);
 
if ( INVALID_HANDLE_VALUE == hFind)
{
 // error
 
printf ( "ERROR %d\n",  GetLastError    ());
 
return;
}
 
// recurse if directory...
if ( FILE_ATTRIBUTE_DIRECTORY == w32fd.dwFileAttributes)
{
 
     WalkTree ( w32fd.cFileName,    acBase);
}
else
 HandleFile ( &w32fd);
 
while ( FindNextFile ( hFind, &w32fd))
{
    // recurse if directory...
    if ( FILE_ATTRIBUTE_DIRECTORY == w32fd.dwFileAttributes)
    {
 
     WalkTree ( w32fd.cFileName,    acBase);
    }
    else
      HandleFile ( &w32fd);
}
 
if ( ERROR_NO_MORE_FILES != GetLastError())
{
 // error
}
FindClose ( hFind);
}
 
void HandleFile ( WIN32_FIND_DATA* pw32fd)
{
  // handle file here
}

Open in new window

But i dont get the opportunity to enter the path in the command line before the it exits. I get that i need to provide the argument for the *( argv + 1) pointer. But, if i run this - am i meant to enter the path at runtime?
>> But i dont get the opportunity to enter the path in the command line before the it exits
Do you understand how argv works? It's what's passed as part of the command line argument. Your opportunity to enter it is when you execute the process on the command line, it's just an argument.

http://www.d.umn.edu/~gshute/C/argv.html
Yes i understand that bit now - thanks. I dont execute my scripts from command line, i run them iin debug mode in Visual C++ - hence why no arguments being passed and the confusion.
>>Yes i understand that bit now - thanks
No worries :)

>> I dont execute my scripts from command line, i run them iin debug mode in Visual C++ - hence why no arguments being passed and the confusion.
That's ok, like I said jkr wrote it this way just as an example, of course you don't have to use argv you can pass in the root path anyway you want, included just as a hard coded string if that's a good solution for you.

I don't have time to annotate the code right now but now you have it working I'm sure if you step through it with a debugger you'll see it's actually not that complex... just remember it's recursive so you'll need to keep that in mind when stepping through it.

BTW: I don't usually "steal" other peoples code from other questions but in this case (a) I don't have the time to knock you up an example myself and (b) jkr's example is exactly what you need anyway so I saw no point in reinventing the wheel :)
This is what I have so far - trying to interpret JKR's solution to what i already had. Basically,  I want to call the ProcessDirectory function when the code encounters a file. I have tried to stick this in but i have commented it out because it is causing the system to crash.

Also, I know that there would be no way using this alone to bring the scan back out of the sub directory to continue where it left off. Therefore, i probably need some sort of base directory assignment and check in there.

Could anyone help and look at this snippet please
/* Script to parse all the audio files and store R/T communication information
for each of the actors within the simulation
FindFirstFile() & FindNextFile() may be used to scroll through a directory to find specific files within a directory*/
 
#include <windows.h> 
#include <conio.h> 
#include <time.h> 
#include <stdio.h> 
#include <string> 
#include <iostream> 
#include <fstream> 
using namespace std ; 
static int FileCount = 0;							// used to record number of files parsed and names stored in string vector
 
void WriteToLog(string Message)						// This a function that writes to the log whenever the WritetoLog function is called
{ 
 
string UtilityLog  = "C:\\Users\\SimonP\\Documents\\Visual Studio 2008\\Projects\\Development\\P7_ACE_04\\AudioParseLog.txt"; 
ofstream WriteFile; 
WriteFile.open(UtilityLog.c_str(), ios::app);		// Open the log file to write the input and append to the end of the file
if(WriteFile.fail())								// If the opening of the parselg fails then throw a message box to the user
{ 
	MessageBox(NULL,"Error writing to AudiParseLog!  Contact Support.", 
	   "ParsingInfo", MB_OK | MB_ICONERROR);
	return;
} 
 
else
{WriteFile << Message;}
WriteFile.close(); 
return; 
} 
 
 
void ProcessDirectory(string DirPath)			//Takes a directory as an input and processes it. Directory is passed as a string.
{ 
 
bool  Done;										// used for searching the directory for files 
HANDLE  FndHnd;									// used for FindFirstFile() 
WIN32_FIND_DATA FindDat;						// allocate a local structure for the file 
string LogMessage;								// holds the converted message to be written in the log 
char datebuf[9];								// stores the date obtained from _strdate() 
char timebuf[9];								// stores the time obtained from _strtime() 
char MessageBuffer[100];						// holds the message which is converted into a string 
int i;											// used as an index for 'MessageBuffer' array
int sc;											// String Search integer for comparing filepaths
 
 
 
string Fname;									// String to store filename that is found from the search
string FilePath;								// File path is stored also as a string when found
string searchterm = "audio";					// Search string to compare the filename that is discovered to
string FullPath = DirPath;						// Store inputted DirPath Variable as FullPath before amending
FullPath = FullPath + "*.*";					// Add the .* extension to check for all files (notice + operator overload)
 
 
FndHnd = FindFirstFile(FullPath.c_str(), &FindDat);		// Use of FindFirstFile - handle returned
Done = (FndHnd == INVALID_HANDLE_VALUE)?true:false;		// Done is true if an invalid handle is returned
string CurDate = _strdate(datebuf);						// get current date for log information
string CurTime = _strtime(timebuf);						// get current time for log information
 
while (!Done)											// keep looping until an invalid handle value is returned 
	{ 
	
	
	if ((FindDat.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0)	//  check directory attribute bit of WIN32_FIND_DATA structure 		
	{ 
		Fname = FindDat.cFileName;						// WIN32_FIND_DATA structure object cFilename is assigned to the found file. This is given to Fname
		FilePath = DirPath + Fname;						// Add the file path for a complete address
		sc= Fname.find(searchterm);						// Check for the correct string 'audio' within the filename			
		if(sc!=-1)										// If sc returned -1, then the string has not been found				
		{ 
		//Add the filename to the vector list and write to log that this has happened
		FileCount++;
		}	// end of if
		else
		{
		string fileprint =Fname;
		i =  sprintf(MessageBuffer, "Following file was not opened: %s", fileprint.c_str());
		i += sprintf(MessageBuffer + i,"  %s %s", CurDate.c_str(), CurTime.c_str());
		i += sprintf(MessageBuffer + i,"  Number of characters are %d\n", i);
		LogMessage = MessageBuffer; 
		WriteToLog(LogMessage); 
		}	// end of else 
	} 
 
	//else if ( FILE_ATTRIBUTE_DIRECTORY == FindDat.dwFileAttributes)	// If the file that is discovered is a directory then recurse the function
	//{
	//	ProcessDirectory (FindDat.cFileName);
	//}
 
	Done = !FindNextFile(FndHnd, &FindDat);							// Find next file. Includes handle that was specified in the FindFirst function 
} 
 
 
 
 
 
if (FndHnd)										//If handle is open 
FindClose(FndHnd);								//Then close it.
 
if (FileCount > 0)
{ 
	i =  sprintf(MessageBuffer, "Total files parsed = %d", FileCount); 
	i =+ sprintf(MessageBuffer + i,"  %s %s\n", CurDate.c_str(), CurTime.c_str()); 
	LogMessage = MessageBuffer; 
	WriteToLog(LogMessage); 
} 
else
{
	i =  sprintf(MessageBuffer, "No files parsed"); 
	i += sprintf(MessageBuffer + i,"  (%s %s)\n", CurDate.c_str(), CurTime.c_str()); 
	LogMessage = MessageBuffer; 
	WriteToLog(LogMessage); 
}
 
return;
} 
 
int main()											
{ 
DWORD NameLength = 100;								// specifies the character length of the buffer 
string FilePath;									// assigned the value returned from GetModuleFileName() 
char CurPath[100];									// holds the message which is converted into a string 
int PathLength;										// stores return value of length()
 
 
GetModuleFileName(NULL, CurPath, NameLength);		// Obtain file path of current program 
FilePath = CurPath; 
PathLength = FilePath.length(); 
FilePath = FilePath.substr(0, (PathLength - 13));	// This bit subtracts 13 elements from the string FilePath object P7_ACE_04.exe
string dir_1 = "C:\\Users\\SimonP\\Documents\\Visual Studio 2008\\Projects\\Development\\P7_ACE_04\\";  //Define Source paths for processing 
ProcessDirectory(dir_1);							// Process the following directories specified by dir_1 string 
return 0; 
} 

Open in new window

I have tried to emulate JKR's method to my situation. This code compiles, but simply isnt doing whats required.

A few lines of the code I simply dont understand what is going on, despite my reading and full day endeavours with this problem. I have attached 'my' code snippet to look at.

1) ( FILE_ATTRIBUTE_DIRECTORY == FindDat.dwFileAttributes)      -> is this set to true if the structure data attribute confirms a directory i.e 'if the file is a directory then this statement is true?'
2) if ( '.' == *(Path + lstrlen(Path) - 1)) <- what is this doing? What does a path returned as '.' indicate - this is happening a lot when i am debugging my code. I am also seeing '..' returned.
3) if (FILE_ATTRIBUTE_DIRECTORY & dwAtt)                                                // If NEW path is a directory
{
    if  ( '\\'== acPath[lstrlen(acPath)-1])                                          // If the end 2 characters of acPath are '\\' then
      {lstrcat (acPath, "*.*");}                                                            // append a *.* to the end of the filepath to indicate a path for all files
    else
      {lstrcat(acPath, "\\*.*");}                                                            // or this to ensure filepath for acpath ends in \\*.*
    printf ( "path is now %s\n",acPath);                                          // print the new file path to the output device.
}
This section of the code is also confusing me - could anyone give a quick explanation.

I really want to close this question and move on to other things. I know i am not far away - but i need a little assistance in looking at the detail of the snippet - rather than being passed links to more tutorials.

Many thanks



#include <windows.h> 
#include <conio.h> 
#include <time.h> 
#include <stdio.h> 
#include <string> 
#include <iostream> 
#include <fstream> 
 
using namespace std;
 
static int FileCount=0;									// Initialise parse counter to 0.
void HandleFile (WIN32_FIND_DATA *file_data);			// Declarator to handle file function
void WriteToLog (string Message);						// Declarator for the WritetoLog function							
void WalkTree (char *Path, char *Base);					// Declarator to the WalkTree Function
 
 
int main()											
{ 
char *filebase = "C:\\Users\\SimonP\\Documents\\Visual Studio 2008\\Projects\\Development\\P7_ACE_05\\";  
char *filepath = "";
 
char MessageBuffer[100];
char datebuf[9];												// stores the date obtained from _strdate() 
char timebuf[9];												// stores the time obtained from _strtime() 
string CurDate = _strdate(datebuf);								// get current date for log information
string CurTime = _strtime(timebuf);								// get current time for log information
string LogMessage; 
int i;
 
WalkTree(filepath,filebase);									// Start the process of scanning from the filebase above
 
if (FileCount > 0)
{ 
	i =  sprintf(MessageBuffer, "Total files parsed = %d", FileCount); 
	i =+ sprintf(MessageBuffer + i," %s %s\n", CurDate.c_str(), CurTime.c_str()); 
	LogMessage = MessageBuffer; 
	WriteToLog(LogMessage); 
} 
else
{
	i =  sprintf(MessageBuffer, "No files parsed"); 
	i += sprintf(MessageBuffer + i,"  (%s %s)\n", CurDate.c_str(), CurTime.c_str()); 
	LogMessage = MessageBuffer; 
	WriteToLog(LogMessage); 
}
 
return 0; 
}
 
void WalkTree (char *Path,char *Base)
{
WIN32_FIND_DATA FindDat;
HANDLE FndHnd;
DWORD dwAtt;
char acPath[MAX_PATH] ;
char acBase[MAX_PATH] ;
 
printf ( "WalkTree()was called with <%s> <%s>\n",Path,Base);		// Print Output that walktree was called with the path and base
 
if ( '.' == *(Path + lstrlen(Path) - 1))							// If last character of path is a '.' then return <--- WHY IS THIS HERE?!?!?!
return;
 
if(Base)															// Do if Base is occupied with characters
sprintf (acPath, "%s\\%s",Base,Path);								// If base path exists then append base and path to acpath buffer
else
lstrcpy (acPath, Path);												// Make acPath buffer a copy of Path should Base be empty
 
printf ( "path is %s\n",    acPath);								// Print the path to the output										
lstrcpy (acBase, acPath);											// Make acBase the current path
	
dwAtt = GetFileAttributes (acPath);									// Get file attributes of the new path
 
if ( 0xffffffff == dwAtt)											// If the path can not be found then signal an error and write to log.
{
 // Write an error to the log here
}
 
if (FILE_ATTRIBUTE_DIRECTORY & dwAtt)								// If NEW path is a directory
{
    if  ( '\\'== acPath[lstrlen(acPath)-1])							// If the end 2 characters of acPath are '\\' then
	{lstrcat (acPath, "*.*");}										// append a *.* to the end of the filepath to indicate a path for all files
    else
	{lstrcat(acPath, "\\*.*");}										// or this to ensure filepath for acpath ends in \\*.*
    printf ( "path is now %s\n",acPath);							// print the new file path to the output device.
}
 
FndHnd = FindFirstFile(acPath, &FindDat);							// Now look for the first file in the new file path. 
 
if (INVALID_HANDLE_VALUE == FndHnd)									// If no handle is found - or no files at all then throw an error
{
 // Write an error to the log here
printf ("ERROR %d\n",GetLastError());								// Error Printed and returned
return;
}
 
if ( FILE_ATTRIBUTE_DIRECTORY == FindDat.dwFileAttributes)			// IF the FindFirstFile FINDS a directory check (different from 
{																	// deciphering if the current path is a directory (above)
     WalkTree (FindDat.cFileName,acBase);							// Pass the filename of the handle to the top of the function to redefine the path for the search
}
else
 HandleFile (&FindDat);												// If file found, pass to the handler ( the IF function to examine the name
																	// and to enter into the logs. 
while ( FindNextFile(FndHnd, &FindDat))
{
    if ( FILE_ATTRIBUTE_DIRECTORY == FindDat.dwFileAttributes)		// IF the FindNextFile FINDS a directory check (different from
    {																// deciphering if the current path is a directory (above)
    WalkTree (FindDat.cFileName,acBase);							// then recurse the walk tree function 						
    }
    else
      HandleFile (&FindDat);										// If FILE_ATTRIBUTE_DIRECTORY is not equal to FileAttributes then you have 
}																	// found a viable file - so handle it.
 
if ( ERROR_NO_MORE_FILES != GetLastError())
{
 // error
}
FindClose (FndHnd);
}
 
void HandleFile (WIN32_FIND_DATA *file_data)						// Pass the win32_find_data structure by reference
{																	// to the log if the name viable against the check.
  string keyword = "audio";											// keyword to check each handle file before opening.
  char datebuf[9];													// stores the date obtained from _strdate() 
  char timebuf[9];													// stores the time obtained from _strtime() 
  char MessageBuffer[100];											// holds the message which is converted into a string
  string LogMessage;												// holds the converted message to be written in the log
  string file_name = file_data->cFileName;
  string CurDate = _strdate(datebuf);								// get current date for log information
  string CurTime = _strtime(timebuf);								// get current time for log information
  int sc,i;															// holds result of search term and character count
 
 
 
		sc=	file_name.find(keyword);								// Check for the correct string 'audio' within the filename			
		if(sc!=-1)													// If sc returned -1, then the string has not been found				
		{ 
		//Add the filename to the vector list and write to log that this has happened
		FileCount++;
		}	// end of if
		else
		{
		string fileprint =file_name;
		i =  sprintf(MessageBuffer, "Following file was not opened: %s", fileprint.c_str());
		i += sprintf(MessageBuffer + i,"  %s %s", CurDate.c_str(), CurTime.c_str());
		i += sprintf(MessageBuffer + i,"  Number of characters are %d\n", i);
		LogMessage = MessageBuffer; 
		WriteToLog(LogMessage); 
		}	// end of else
	return;
	} 
 
void WriteToLog(string Message)						// This a function that writes to the log whenever the WritetoLog function is called
{ 
string UtilityLog  = "C:\\Users\\SimonP\\Documents\\Visual Studio 2008\\Projects\\Development\\P7_ACE_05\\AudioParseLog.txt"; 
ofstream WriteFile; 
WriteFile.open(UtilityLog.c_str(), ios::app);		// Open the log file to write the input and append to the end of the file
if(WriteFile.fail())								// If the opening of the parselg fails then throw a message box to the user
{ 
	MessageBox(NULL,"Error writing to AudiParseLog!  Contact Support.", 
	   "ParsingInfo", MB_OK | MB_ICONERROR);
	return;
} 
else
{WriteFile << Message;}
WriteFile.close(); 
return; 
} 

Open in new window

Unfortunately, I can't test/debug this for you cos I don't currently have access to a windows machine so I'll let someone else (jkr probably if you can wait until later when he'll almsot certianly be online) take it from here. Meanwhile, I hope I've at least managed to get the principle across to you. If not, let me know and I'll be glad to try and elaborate on that.
You can take a look at the find_file function described here:

http://www.boost.org/doc/libs/1_39_0/libs/filesystem/doc/index.htm

I think it almost exactly matches your requirements
It does require adding a dependency on (at least a subset of) Boost, but the nice thing is that the code will  on platforms other than Windows too.
Okay thanks Evilrx - lets hope that JKR checks this thread out soon ;)
ASKER CERTIFIED SOLUTION
Avatar of evilrix
evilrix
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hurrah hurrah hurrah.

I have sorted it - and thanks for taking the time evilrx to explain those few bits for me - it enabled me to understand what was happening as i was debugging the script. It turns out that I was overloading the char message[100] buffer - which i have since set to  char message[MAX_PATH].

Thanks again. Points well deserved.
Glad to have helped :)
Dear simondopickup,
If I want to search using wildcard character like "?" along with "*" so how to implement this in this project?