File filter program that writes a log for one directory layer but no more, I have code for doing extra layers although cannot encorpate it

For this program I need to search multiple directory layers and then if the file is in a directory then the file name will have the directory name added to it. This is the code for the directory scanning so you can get an idea of what I mean:
[code]
public void processDirectory(String directoryName)
    {
        try
        {
            File dir = new File(directoryName);

            File[] files = dir.listFiles();

            for (int i = 0; i < files.length; i++)
            {
                if (files[i].isDirectory())
                {
                    processDirectory(directoryName + "/" + files[i].getName());
                }
                  }
        } catch (Exception e)
        {
            System.err.println("Problem with " + directoryName);
            System.exit(0);
        }
    }
[/code]

This is what I have managed to do so far, it includes comments. This is obviously not all of the code - I can post all of the code if you think it will help.

[code]
public void process(String[] argStrings, File files)
{
   
    //lots of code for validation of command line (the directory, copydate, fileextension1 and fileexttesnion 2 are entered on the command line)
   
    File folder = new File(name);
    processDirectory(folder); // This passes the folder (argstrings0 to the processdirectory method)

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] htmlFiles = files.listFiles(multiFilter); //when this was folder.listFiles(multiFilter) it worked for one directory level

      multiFilter.setExtension(fileExtensionTwo);
    File[] phpFiles = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
    log(filesToLog, archiveLimit, copyLimit);
  }

  public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        { // once this method has found directories in the directories it recurses,
          processDirectory(directoryName + "\\" + files[i].getName()); // Add the directories to the filename so that they appear in the log

          return files[i];// here i am trying to get all the files in the directory (s) back to the method above so that they can be filtered
                          // i.e. either php or html as eneterd on the command line
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }
 
  // Also I use this to run the application, this might be wrong also:
 
  import java.io.*;
 
  public class CrawlerApp
  {
    public static void main(String[] argStrings, File files)
    {
        Crawler crawler = new Crawler();
      crawler.process(argStrings, files);
    }
}
 
//At the moment my program does not compile and I get these errors:

//C:\Documents and Settings\User\Desktop\Crawler.java:209: processDirectory(java.io.File) in Crawler cannot be applied to (java.lang.String)
//processDirectory(folder + "\\" + files[i].getName());

[/code]
Thank you
LVL 1
scottcwAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
>>processDirectory(folder + "\\" + files[i].getName());

should be

processDirectory(new File(folder,files[i].getName()));
0
scottcwAuthor Commented:
Ok this gets rid of that compile error, cheers.
Although now I get this compile error:

C:\Crawler.java:220: missing return statement
  }

This is at the end of the processDirectory method.

Thanks
0
CEHJCommented:
Follow

>>System.err.println("Problem with directory " + folder);

with

return null;
0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

scottcwAuthor Commented:
Yes this is what I tried a while ago, and it does compile although does not run now, I get an error like this at command line:

java.lang.nosuchmethoderror: main

When the method looks like this:

 public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(new File(folder,files[i].getName())); // Add the directories to the filename so that they appear in the log

          return files[i];
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }

Cheers
0
CEHJCommented:
>>public static void main(String[] argStrings, File files)

should be

public static void main(String[] argStrings)
0
scottcwAuthor Commented:
Making progress.

I get one compile error:

C:\Documents and Settings\User\Desktop\CrawlerApp.java:6: process(java.lang.String[],java.io.File) in Crawler cannot be applied to (java.lang.String[])
    crawler.process(argStrings);

When the App file looks like this:

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.process(argStrings);
  }
}

Thanks
0
CEHJCommented:
No, you need to pass the directory.Somehting like:

 Crawler crawler = new Crawler();
 crawler.process(argStrings, new File("."));
0
scottcwAuthor Commented:
I tried what you said and i get a compile error:

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.process(argStrings, new File("."));
  }
}

Gives:

C:\Documents and Settings\User\Desktop\CrawlerApp.java:6: cannot find symbol
symbol  : class File
location: class CrawlerApp
    crawler.process(argStrings, new File("."));
                                    ^
1 error

Tool completed with exit code 1

I have not done much work on App files before, I just use the same format for each program therefore I dont know what this does.
Cheers
0
scottcwAuthor Commented:
I did not import
sorry
0
scottcwAuthor Commented:
Well it compiles, it runs when I input the directory, copydate and extensions althrough it says it has finished and then I read the logs and they are empty for some reason.

Cheers
0
CEHJCommented:
Try starting at a different directory than the current directory
0
scottcwAuthor Commented:
The program should work when the directory is in the same directory as the program (this worked when it only did one level).

I tried somedifferent file extensions and i got some output in the log, but these were files on my desktop where the Crawler and CrawlerApp are.

So I think at the moment it is doing the files on the same level as the directory specified and not directories inside the directory.

Cheers
0
CEHJCommented:
I'm not sure what you're trying to do with this code
0
scottcwAuthor Commented:
This is some of the javadoc in my program:

/**
 * The Crawler class executes six methods that identify files which could potentially
 * be archived and copied in a directory specified on the command line. A file is to
 * be archived if the last modified date is more than 100 days ago. A file is to be copied
 * if it has been modified since a given time. Only nominated extensions are to be
 * considered (php and html).
 * <p>
 * The output from this class is two files (called archive.txt and copy.txt)
 * containing the names of files to be archived, and those which may be copied. The
 * names will follow the convention specified in the create application.
 * <p>
 * $ java CrawlerApp [directory name] [copy date] [extension one] [extension two]
 *
 * @author        John

Do you want me to post the whole code to you, or email it maybe, then you can compile it yourself to get a better understanding?
Cheers
John
0
scottcwAuthor Commented:
I am recursing over all the directories, but only reporting on the top level.

I am not using the argument "files" to process().

I went to the Java forums and they suggested this, I have been trying for a while to implement this but with no luck. Please could you have a go. Once this is done I think it will work:

1) start recursing (do processDirectory())
2) at each directory, call process() to process that directory.

Don't call processDirectory() from process(). It should be the other way around.

This is the code that needs to be changed:

public void process(String[] argStrings, File files)
  {
// validation code for the command line
    File folder = new File(name);
    processDirectory(folder);
 
    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] htmlFiles = files.listFiles(multiFilter);
 
      multiFilter.setExtension(fileExtensionTwo);
    File[] phpFiles = files.listFiles(multiFilter);
 
    File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
    log(filesToLog, archiveLimit, copyLimit);
  }
 
  public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();
 
      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(new File(folder,files[i].getName())); // Add the directories to the filename so that they appear in the log
 
          return files[i];
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }


Thanks
0
CEHJCommented:
Your code looks more or less alright, but the sole functionality of processDirectory, IMO is to recurse subdirectories. If so,  all it needs to be is

public void processDirectory(File folder)
  {
    try
    {

        if (folder.isDirectory())
        {
          process(folder); // Do your processing
        }

      File[] files = folder.listFiles();
 
      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i]); // Add the directories to the filename so that they appear in the log
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }
0
scottcwAuthor Commented:
I changed to what you said, and I only get one compile error now which is good. Is this something to do with not passing enough values back to process?

java:184: process(java.lang.String[],java.io.File) in Crawler cannot be applied to (java.io.File)
        process(folder); // Do your processing
        ^
1 error

Tool completed with exit code 1

Cheers
0
CEHJCommented:
Yes. What's the String parameter meant to be
0
scottcwAuthor Commented:
The process method takes arguements from the command line e.g. directory = argstrings[0], copydate = argstrings[1], extension 1 and 2 = argstrings[2] and [3]:

public void process(String[] argStrings, File files)
{
Thanks
0
CEHJCommented:
>>Don't call processDirectory() from process(). It should be the other way around.

If you want the above to be the case, you will have to forward that String[] *through* processDirectory *to* process
0
scottcwAuthor Commented:
Something like this?

public void process(String[] argStrings, File files)
  {
//code not shown
  }

 public void processDirectory(File folder, String[] argStrings)
  {
    try
    {
      if (folder.isDirectory())
      {
        process(argStrings, folder); // Do your processing
      }

      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }

It compiles, but still has the same problem, it only does one level.
Cheers
0
scottcwAuthor Commented:
Oh and I forgot to say that this has been removed from process:
processDirectory(folder);
0
CEHJCommented:
Put some debug code in before

>>processDirectory(files[i], argStrings);

such as

System.out.println("Calling processDirectory for directory " + files[i]);

and tell me what it does
0
scottcwAuthor Commented:
It does nothing.
0
scottcwAuthor Commented:
This must mean that it is still only doing oe level as indicated by the logs.
cheers
0
CEHJCommented:
Oh - please put the same sort of thing before

>>process(argStrings, folder); // Do your processing
0
scottcwAuthor Commented:
I get these errors with this code:

C:\Documents and Settings\User\Desktop\Computing\Crawler.java:184: cannot find symbol
symbol  : variable files
location: class Crawler
            System.out.println("Calling processDirectory for directory " + files[i]);
                                                                               ^
C:\Documents and Settings\User\Desktop\Computing\Crawler.java:184: cannot find symbol
symbol  : variable i
location: class Crawler
            System.out.println("Calling processDirectory for directory " + files[i]);
                                                                                     ^
2 errors
Code:
public void process(String[] argStrings, File files)
  {
//more code
    File folder = new File(directory);
      process(argStrings, folder);

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] extensionOne = files.listFiles(multiFilter);

      multiFilter.setExtension(fileExtensionTwo);
    File[] extensionTwo = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
    log(filesToLog, archiveLimit, copyLimit);
  }

  public void processDirectory(File folder, String[] argStrings)
  {
    try
    {
      if (folder.isDirectory())
      {
            System.out.println("Calling processDirectory for directory " + files[i]);
        process(argStrings, folder); // Do your processing
      }

      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }

Cheers
0
scottcwAuthor Commented:
I tried this also:

System.out.println("Calling processDirectory for directory " + folder);

And still nothing.
Cheers
0
CEHJCommented:
>>I tried this also:

That's the one. Are you saying you are getting *no output at all*?
0
scottcwAuthor Commented:
On the command line nothing prints i.e. the ""Calling processDirectory for directory " + folder" does not show.

Get one level of directories in the logs though - still the same.
0
CEHJCommented:
Then that method is not being called at all. Got to go for a while
0
scottcwAuthor Commented:
Ok cheers for the help
0
CEHJCommented:
>>Then that method is not being called at all.

(Unless of course it's not being called in the first place with a directory - which would be quite usesless)
0
scottcwAuthor Commented:
Right now I have got it to scan through every directory and file (each is displayed on the command line).

Some of the logs are being written properly also (deeper directories).

This is my latest code:

App file:

import java.io.*;

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.processDirectory(new File("."), argStrings);
    //crawler.process(argStrings, new File("."));
  }
}

This is process and processDirectory:
public void process(String[] argStrings, File files)
  {
    File folder = new File(directory);

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] extensionOne = files.listFiles(multiFilter);

      multiFilter.setExtension(fileExtensionTwo);
    File[] extensionTwo = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
    log(filesToLog, archiveLimit, copyLimit);

      processDirectory(folder, argStrings);
  }

  public File processDirectory(File folder, String[] argStrings)
  {
    try
    {
      File[] files = folder.listFiles();

      for(int i = 0; i < files.length; i++)
      {
            if(files[i].isFile())
        {
              System.out.println("Files " + files[i]);
            }
        else if(files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
          System.out.println("Directories " + files[i]);
          process(argStrings, files[i]);
          }
      }
    }
    catch(Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }
Cheers
0
CEHJCommented:
Well it looks like you need to implement 'author John's' functionality

e.g. 'argStrings' is currently completely unused ...
0
scottcwAuthor Commented:
Sorry its taken a while, but I have actually managed to get the program working perfectly with HTML formatted output.

I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.

So far for functional testing I have tried different parameters on the command line (the program takes input from the command line) e.g. java CrawlerApp [directory name] [copy date] [extension one] [extension two] would be input or the actual input would be java CrawlerApp knuth 2005-12-28 php html.

The program then searches all files in that directroy even if they are in lower levels of directories and outputs the php and html files that have a last modified date of 2005-12-28.

I was wondering if there are any more functional tests that i could do for a program like this?

I am very new to structural testing - would I test try and catch blocks and for for loops would i just show that it printed every iteration to the command line for instance? Any tips on this would be appreciated.

Thank you
0
CEHJCommented:
>>I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.

Let me say we are not obliged to answer additional questions - i have already helped you quite extensively in this question. However, this is what you need for testing - see documentation

http://www.junit.org/
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CEHJCommented:
:-)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.