Solved

Iterating through files in directories

Posted on 2000-03-18
5
539 Views
Last Modified: 2013-12-14
I need C/C++ code which will iterate through all files in a given directory (and possibly recursively for subdirectories) with the following extra requirements:
1. No MFC (I know MFC has file iterator classes)
2. Must equally work (compile) on Windows/Unix platforms.
AINSI C/C++ code would be ideal.
0
Comment
Question by:olegsp
5 Comments
 
LVL 22

Expert Comment

by:nietod
Comment Utility
>> 2. Must equally work (compile) on Windows/Unix
>> platforms.  AINSI C/C++ code would be ideal.
Impossible.  Standard C/C++ has no features for this.  Standard C/C++ doesn't even recognize the existance of directories  (Not all OSs have them).   The only way to do this is to use OS specific techniques and that means the code won't be portable.  (Youcan get around this by writting different versions for different OSes and then using conditional complation (#ifdef) to insure that only the right version is included in the OS you are compiling on.

In windows you need to use the FindFirst() and FindNext() procedures to do this.  

In Unix I bleive you need to use something called opendir() and probably another procedure or two.
0
 
LVL 5

Expert Comment

by:pitonyak
Comment Utility


Here is some code that I use... It is from a  multi-threaded text search program that I wrote some years back...
This will not be complete. If you desire the entire code set then tell me pitonyak@bigfoot.com

You are probably better off simply taking the idea rather than to use this code directly...

First there is the primary work to be done. I will edit it a bit..


This code is found in the header file... What this does is set in some wrapper code listed at the bottom

******************************


_MacroDefineList(ADPString, StringPtr, StringList, StringIterator);

******************************

And now for some modified code

******************************

//**************************************************************************
//**                                                                      **
//**   Here is where the magic happens, check the files and dirs...       **
//**                                                                      **
//**************************************************************************
int FileTSThread::doTheSearch(const char* path)
{

     // The entered path had better have a trailing '/' or '\\' as appropriate...

    ADPString searchName = path;            // I wrote my own string class some years back
    searchName += _filespec;                // Add in the file spec so searchName is like /path/*.txt
    StringList fileList;                    // A predefined list...

    // Build the list of matching files in this directory
    // Why do I keep building all these silly lists?  Oh yeah,
    // it is because I want to keep the handle free.
    // More important for the subdirectories where I will actually
    // be recursing and using the handle again.  Does it matter?
    // <heee heee heee>!

#if defined( _ADP_OS2_ )

    unsigned long  attr = FILE_SYSTEM | FILE_READONLY | FILE_ARCHIVED | FILE_HIDDEN;

#else

    unsigned long  attr = FA_RDONLY | FA_HIDDEN | FA_SYSTEM | FA_ARCH;

#endif
    //
    // Do not allow directory listings for this one. Not really needed!
    //
    getMatchingNames(fileList, attr, searchName.c_str(), -1);   // the real work happens here
    //
    // Search the files in the list.  Found sections are added to
    // the _itemList.  The parent process had better be reading
    // these things out and emptying the buffer.
    //
    _MacroDeclareIterator(StringIterator, strIt, fileList);
    _MacroIteratorRestart(strIt, fileList);
    //
    // For each file in the list
    //
      while (_MacorIteratorIsValid(strIt)){
        //
        // Get the next string (file name)
        //
        StringPtr currentStr = _MacroIteratorGetCurrent(strIt, fileList);
        //
        // Build full pathname to file, set status and build FileSearchBuffer
        //
        ADPString aName(path);      // This is the path
        aName += *currentStr;       // This is the file name
        //
        // Now I can do some work with aName which is the full path to the file
        // So now advance the iterator after doing the work
        //
        _MacroIteratorSetToNext(strIt);
    }
    //
    // clear the list of files
    //
    _MacroFlushList(StringIterator, fileList);
    //
    // Do directories if a recursion has been requested
    //
    if (_recurse) {

#if defined( _ADP_OS2_ )

        attr = FILE_SYSTEM | FILE_READONLY | FILE_ARCHIVED | FILE_HIDDEN | MUST_HAVE_DIRECTORY;

#else

    unsigned long  attr = FA_RDONLY | FA_HIDDEN | FA_SYSTEM | FA_ARCH | FA_DIREC;

#endif
        searchName = path;              // Get the path
        searchName.forceLastChar('*');  // Search for ALL directories
        //
        // force directory listings for this one. Not needed for OS/2
        // I do need it for Windows.
        //
        getMatchingNames(fileList, attr, searchName.c_str(), 1);
        //
        // Make certain that we do not loop forever, of course,
        // I modified getMatchingNames() to skip these anyway!
        //
        ADPString skip1(".");
        ADPString skip2("..");
        //
        // Now get ready to loop through all of the directories
        //
        _MacroIteratorRestart(strIt, fileList);
          while (_MacorIteratorIsValid(strIt)) {
            StringPtr currentStr = _MacroIteratorGetCurrent(strIt, fileList);
            //
            // Skip the directories "." and ".."
            //
            if (*currentStr != skip1 && *currentStr != skip2) {
                ADPString newPath(path);        // The current path
                newPath += *currentStr;         // The new directory
                newPath.forceLastChar('/');     // Force an ending '/'
                doTheSearch(newPath.c_str());   // Recursively call the routine
            }
            _MacroIteratorSetToNext(strIt);
        }
        _MacroFlushList(StringIterator, fileList);
    }
    return ok();
}

******************************

OK, so now for the routine that actually builds the list

******************************

//**************************************************************************
//**                                                                      **
//**   Get a list of names in the directory matching attributes and name. **
//**   For files in OS/2 consider                                         **
//**      attr = FILE_SYSTEM | FILE_READONLY | FILE_ARCHIVED | FILE_HIDDEN**
//**   and for Directories add (attr |= MUST_HAVE_DIRECTORY)              **
//**                                                                      **
//**   On return, names will have the list.  If you want it emptied, you  **
//**   must empty it before calling.  This will return a 0 if a serious   **
//**   error was encountered.                                             **
//**                                                                      **
//**   I included forceDir for those operating systems that do not allow  **
//**   me to allow or disallow directories from the list.                 **
//**   a value of -1 means do not allow, 0 means do not care and 1 means  **
//**   force it to be a dir.                                              **
//**                                                                      **
//**************************************************************************
int FileTSThread::getMatchingNames(StringList& fileList, unsigned long attr, const char* spec, int forceDir)
{
    //
    // Write a class to encapsulate this would ya!
    // This routine is in a mess!
    //
    unsigned long rc = 0;

#if defined( _ADP_OS2_ )

    HDIR handle = 0x0001;
    FILEFINDBUF3 buf3;
    unsigned long count = 1;    // Used in DosFindFirst and DosFindNext

    if (forceDir < 0)
        attr &= ~(FILE_DIRECTORY | MUST_HAVE_DIRECTORY); // do not allow dir
    else if (forceDir > 0)
        attr |= FILE_DIRECTORY | MUST_HAVE_DIRECTORY;    // force a dir

#if defined(__EMX__)
    rc  = DosFindFirst((const unsigned char *) spec, &handle, attr, (PVOID) &buf3, sizeof(buf3), &count, FIL_STANDARD);
#else
    rc  = DosFindFirst(spec, &handle, attr, (PVOID) &buf3, sizeof(buf3), &count, FIL_STANDARD);
#endif

#else

#if defined(_MSC_VER)

    struct _finddata_t dir;
    long win_find_handle = _findfirst(spec, &dir);
    rc = win_find_handle == -1;  // Done if nothing found

#elif defined(__EMX__)
    struct _find dir;
    rc = __findfirst(spec, attr, &dir);
#else
    struct ffblk dir;
    rc = findfirst(spec, &dir, attr);
#endif

#endif

    while (!rc){

        int skipThis = 0;
        ADPString* s = 0;

#if defined( _ADP_OS2_ )
        //
        // For OS/2, this has already been specified.
        //
        skipThis = 0;
        if (!skipThis)
            s = new ADPString(buf3.achName);

#else


#if defined(__EMX__)
        if (forceDir < 0)
            skipThis = dir.attr & FA_DIREC ? 1 : 0;
        else if (forceDir > 0)
            skipThis = dir.attr & FA_DIREC ? 0 : 1;

        if (!skipThis)
            s = new ADPString(dir.name);

#elif defined(_MSC_VER)

        if (forceDir < 0)
            skipThis = dir.attrib & FA_DIREC ? 1 : 0;
        else if (forceDir > 0)
            skipThis = dir.attrib & FA_DIREC ? 0 : 1;

        if (!skipThis)
            s = new ADPString(dir.name);

#else
        if (forceDir < 0)
            skipThis = dir.ff_attrib & FA_DIREC ? 1 : 0;
        else if (forceDir > 0)
            skipThis = dir.ff_attrib & FA_DIREC ? 0 : 1;

        if (!skipThis)
            s = new ADPString(dir.ff_name);
#endif

#endif
        if (s) {
            //
            // Perform a few sanity checks
            //
            s->trimAll();
            if (s->length() > 0 && *s != "." && *s != "..")
                _MacroAddToList(fileList, s, StringPtr);
         }

#if defined( _ADP_OS2_ )

        rc = DosFindNext(handle, (PVOID) &buf3, sizeof(buf3), &count);

#else

#if defined(__EMX__)

        rc = __findnext(&dir);

#elif defined(_MSC_VER)

    rc = _findnext(win_find_handle, &dir);

#else
        rc = findnext(&dir);
#endif

#endif
    }

#if defined( _ADP_OS2_ )

    DosFindClose(handle);

#elif defined(_MSC_VER)

    rc = _findclose(win_find_handle);

#else

    // So it seems that findfirst() and such do not need to be closed

#endif
    return ok();
}


******************************

OK, so how do these macros work which allow the list access?

******************************

#ifndef _WRAPPER_LIST_HPP
#define _WRAPPER_LIST_HPP

//**************************************************************************
//**                                                                      **
//**            Copyright 1998-1999 by Andrew D. Pitonyak                 **
//**                                                                      **
//**   This code is free for personal use. Use for profit requires prior  **
//**   permission. I appreciate bug reports and fixes.                    **
//**                                                                      **
//**   Andrew D. Pitonyak                                                 **
//**   4446 Mobile Drive #105          pitonyak@bigfoot.com  <-- here 1st **
//**   Columbus, OH 43220              apitonya@columbus.rr.com           **
//**   (614) 442-8615                  apitonya@oakland.edu               **
//**                                                                      **
//**************************************************************************
//**                                                                      **
//**  This file contains a set of macros which encapsulate a list of      **
//**  pointers. Actually, the code which is using this is storing a       **
//**  list of pointers to strings.                                        **
//**                                                                      **
//**   16 May 1998                                                        **
//**          Made changes to support EMX                                 **
//**                                                                      **
//**  07-19-1999 Pitonyak                                                 **
//**     Modified to work with Microsoft Visual C++ V6                    **
//**                                                                      **
//**  December 8, 1999                                                    **
//**       There seemed to be a problem with the Microsoft STL stuff and  **
//**       I was obtaining an object from the container class that was    **
//**       not yet allocated. As such, I now use my DynamicPtrArray.      **
//**                                                                      **
//**************************************************************************

#include "myconfig.hpp"
#include "DynamicPtrArray.h"

#define _MacroDefineList(thetype, ptrtype, listtype, itType) \
    typedef thetype* ptrtype; \
    typedef DynamicPtrArray<thetype> listtype; \
    typedef int itType

#define _MacroFlushList(itType, lst) lst.clear()
#define _MacroIsListEmpty(lst) lst.empty()
#define _MacroDeclareIterator(itType, itName, itList) itType itName
#define _MacroIteratorRestart(itName,itList) itName = itList.size() - 1
#define _MacroIteratorGetCurrent(itName,itList) itList[itName]
#define _MacorIteratorIsValid(itName) (itName >= 0)
#define _MacroIteratorSetToNext(itName) --itName
#define _MacroAddToList(lst, ent, entptr) lst.add(ent)


#if 0

#if defined( __BORLANDC__ )
#include <classlib/listimp.h>

#define _MacroDefineList(thetype, ptrtype, listtype, ittype) \
    typedef thetype* ptrtype; \
    typedef TISListImp<thetype> listtype; \
    typedef TISListIteratorImp<thetype> ittype

#define _MacroFlushList(itType, lst) lst.Flush(1)
#define _MacroIsListEmpty(lst) lst.IsEmpty()
#define _MacroDeclareIterator(itType, itName, itList) itType itName(itList)
#define _MacroIteratorRestart(itName,itList) itName.Restart()
#define _MacroIteratorGetCurrent(itName,itList) itName.Current()
#define _MacorIteratorIsValid(itName) (itName.Current() != 0)
#define _MacroIteratorSetToNext(itName) itName++
#define _MacroAddToList(lst, ent, entptr) lst.Add(ent)



#elif defined(__IBMCPP__)

#include <iptr.h>
#include <isrtbag.h>

#define _MacroDefineList(thetype, ptrtype, listtype, ittype) \
   typedef IMngElemPointer <thetype> ptrtype; \
   typedef ISortedBag <ptrtype> listtype; \
   typedef ISortedBag <ptrtype> ittype

#define _MacroFlushList(itType, lst) lst.removeAll()
#define _MacroIsListEmpty(lst) lst.isEmpty()
#define _MacroDeclareIterator(itType, itName, itList) itType::Cursor itName(itList)
#define _MacroIteratorRestart(itName,itList) itName.setToFirst()
#define _MacroIteratorGetCurrent(itName,itList) itList.elementAt(itName)
#define _MacorIteratorIsValid(itName) itName.isValid()
#define _MacroIteratorSetToNext(itName) itName.setToNext()
#define _MacroAddToList(lst, ent, entptr) entptr ptr(ent, IINIT); lst.add(ptr)


#elif defined(__EMX__)

//
// I really need to learn more about the STL stuff used here
// since I am not doing this in the best way possible.
// I do not know how to do this nicely with pointers so I define
// some of my own template stuff, yuck!
//

#include <set>
#include <iterator>

//
// First I must dereference the pointer while comparing
//
template <class T>
struct lessp : binary_function<T, T, bool> {
    bool operator()(const T& x, const T& y) const { return (*x) < (*y); }
};



#define _MacroDefineList(thetype, ptrtype, listtype, ittype) \
    typedef thetype* ptrtype; \
    typedef multiset<ADPString*, lessp<ADPString*> > listtype; \
    typedef multiset<ADPString*, lessp<ADPString*> >::iterator ittype
//
// User a smart pointer thingy and then this complicated flush macro goes away.
//
#define _MacroFlushList(itType, lst) {itType x1y1z1y1 = lst.begin(); while(*x1y1z1y1) {delete (*x1y1z1y1); ++x1y1z1y1;}} lst.erase(lst.begin(), lst.end())
#define _MacroIsListEmpty(lst) lst.empty()
#define _MacroDeclareIterator(itType, itName, itList) itType itName = itList.begin()
#define _MacroIteratorRestart(itName,itList) itName = itList.begin()
#define _MacroIteratorGetCurrent(itName,itList) (*itName)
#define _MacorIteratorIsValid(itName) ((*itName) != 0)
#define _MacroIteratorSetToNext(itName) ++itName
#define _MacroAddToList(lst, ent, entptr) lst.insert(ent)

#elif (defined(__EMX__) || defined(_MSC_VER))

//
// I really need to learn more about the STL stuff used here
// since I am not doing this in the best way possible.
// I do not know how to do this nicely with pointers so I define
// some of my own template stuff, yuck!
//

#if defined(_MSC_VER)

#include <functional>
#include <set>
#include <iterator>
//using namespace std;    // MSC hides this here, perhaps they all should

#else

#include <stl.h>
#include <multiset.h>
#include <iterator.h>

#endif
//
// First I must dereference the pointer while comparing
//
template <class T>
struct lessp : std::binary_function<T, T, bool> {
    bool operator()(const T& x, const T& y) const { return (*x) < (*y) ? true : false; }
};



#define _MacroDefineList(thetype, ptrtype, listtype, ittype) \
    typedef thetype* ptrtype; \
    typedef std::multiset<ADPString*, lessp<ADPString*> > listtype; \
    typedef std::multiset<ADPString*, lessp<ADPString*> >::iterator ittype
//
// User a smart pointer thingy and then this complicated flush macro goes away.
//
#define _MacroFlushList(itType, lst) {itType x1y1z1y1 = lst.begin(); while(*x1y1z1y1) {delete (*x1y1z1y1); ++x1y1z1y1;}} lst.erase(lst.begin(), lst.end())
#define _MacroIsListEmpty(lst) lst.empty()
#define _MacroDeclareIterator(itType, itName, itList) itType itName = itList.begin()
#define _MacroIteratorRestart(itName,itList) itName = itList.begin()
#define _MacroIteratorGetCurrent(itName,itList) (*itName)
#define _MacorIteratorIsValid(itName) ((*itName) != 0)
#define _MacroIteratorSetToNext(itName) ++itName
#define _MacroAddToList(lst, ent, entptr) lst.insert(ent)


#else

This will generate an error

#endif

#endif

#endif


******************************


0
 

Accepted Solution

by:
ori_b earned 50 total points
Comment Utility
This is a simple code that will work for the unix machines :

#include <dirent.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>


void main(int argc, char* argv[]) {
  int DirLoop(const char* path);
 
  DirLoop(argv[1]);

}
int     DirLoop(const char* path) {
 
  DIR *dirp;
  struct dirent *direntp;

  char file_name[256];
 
  printf("opening directory %s\n", path);
 
  dirp = opendir( path );
  if (dirp == NULL) {
        printf("Error %s : %s\n", path,strerror(errno));
        return -1;
  }
 
  while ( (direntp = readdir( dirp )) != NULL ) {
        if (direntp == NULL) {
          printf("Erorr %s : %s \n", path,strerror(errno));
          return -1;
        }


    // skip the ./ and ../ files  
    if (direntp->d_name[0] != '.') {
        // build the full path
          strcpy (file_name,path);
          strcat (file_name,"/");
          strcat (file_name,direntp->d_name);

          struct stat buf;
         
          // Find file's type
          if (stat(file_name, &buf) == 0) {
                if ((buf.st_mode & S_IFMT) == S_IFDIR) {
                  // it is a directory
                  DirLoop(file_name);
                }

                else {
                  // A regular file - Do What you want to ...
                  printf("%s\n",file_name);
                }
          }

          else {
                printf("Fail to stat file %s : %s\n",file_name, strerror(errno ));
                return -1;
          }

        }
  }

 
  if (closedir( dirp ) == -1) {
        printf("Erorr closedir of %s : %s \n", path,strerror(errno));
        return -1;
  }

  return 0;
}
0
 
LVL 11

Expert Comment

by:alexo
Comment Utility
>>     if (direntp->d_name[0] != '.') {

Incidently, this is a bug.  Under Unix etc., it is perfectly legitimate to start a directory/file name with a period.

Anyway, todd is right.  You'll have to check for the target platform (using #ifdef or similar) and conditionally compile either findfirst/findnext or opendir/readdir recursive search.
0
 
LVL 1

Author Comment

by:olegsp
Comment Utility
Thanks for the UNIX code.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Here is a helpful source code for C++ Builder programmers that allows you to manage and manipulate HTML content from C++ code, while also handling HTML events like onclick, onmouseover, ... Some objects defined and used in this source include: …
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
The viewer will learn how to use and create keystrokes in Netbeans IDE 8.0 for Windows.
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now