Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

File filter program that writes a log for one directory layer but no more, I have code for doing extra layers although cannot encorpate it

Posted on 2006-04-26
38
Medium Priority
?
185 Views
Last Modified: 2010-03-31
For this program I need to search multiple directory layers and then if the file is in a directory then the file name will have the directory name added to it. This is the code for the directory scanning so you can get an idea of what I mean:
[code]
public void processDirectory(String directoryName)
    {
        try
        {
            File dir = new File(directoryName);

            File[] files = dir.listFiles();

            for (int i = 0; i < files.length; i++)
            {
                if (files[i].isDirectory())
                {
                    processDirectory(directoryName + "/" + files[i].getName());
                }
                  }
        } catch (Exception e)
        {
            System.err.println("Problem with " + directoryName);
            System.exit(0);
        }
    }
[/code]

This is what I have managed to do so far, it includes comments. This is obviously not all of the code - I can post all of the code if you think it will help.

[code]
public void process(String[] argStrings, File files)
{
   
    //lots of code for validation of command line (the directory, copydate, fileextension1 and fileexttesnion 2 are entered on the command line)
   
    File folder = new File(name);
    processDirectory(folder); // This passes the folder (argstrings0 to the processdirectory method)

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] htmlFiles = files.listFiles(multiFilter); //when this was folder.listFiles(multiFilter) it worked for one directory level

      multiFilter.setExtension(fileExtensionTwo);
    File[] phpFiles = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
    log(filesToLog, archiveLimit, copyLimit);
  }

  public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        { // once this method has found directories in the directories it recurses,
          processDirectory(directoryName + "\\" + files[i].getName()); // Add the directories to the filename so that they appear in the log

          return files[i];// here i am trying to get all the files in the directory (s) back to the method above so that they can be filtered
                          // i.e. either php or html as eneterd on the command line
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }
 
  // Also I use this to run the application, this might be wrong also:
 
  import java.io.*;
 
  public class CrawlerApp
  {
    public static void main(String[] argStrings, File files)
    {
        Crawler crawler = new Crawler();
      crawler.process(argStrings, files);
    }
}
 
//At the moment my program does not compile and I get these errors:

//C:\Documents and Settings\User\Desktop\Crawler.java:209: processDirectory(java.io.File) in Crawler cannot be applied to (java.lang.String)
//processDirectory(folder + "\\" + files[i].getName());

[/code]
Thank you
0
Comment
Question by:scottcw
  • 21
  • 17
38 Comments
 
LVL 86

Expert Comment

by:CEHJ
ID: 16545861
>>processDirectory(folder + "\\" + files[i].getName());

should be

processDirectory(new File(folder,files[i].getName()));
0
 
LVL 1

Author Comment

by:scottcw
ID: 16545999
Ok this gets rid of that compile error, cheers.
Although now I get this compile error:

C:\Crawler.java:220: missing return statement
  }

This is at the end of the processDirectory method.

Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16546022
Follow

>>System.err.println("Problem with directory " + folder);

with

return null;
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 1

Author Comment

by:scottcw
ID: 16546077
Yes this is what I tried a while ago, and it does compile although does not run now, I get an error like this at command line:

java.lang.nosuchmethoderror: main

When the method looks like this:

 public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(new File(folder,files[i].getName())); // Add the directories to the filename so that they appear in the log

          return files[i];
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }

Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16546119
>>public static void main(String[] argStrings, File files)

should be

public static void main(String[] argStrings)
0
 
LVL 1

Author Comment

by:scottcw
ID: 16546196
Making progress.

I get one compile error:

C:\Documents and Settings\User\Desktop\CrawlerApp.java:6: process(java.lang.String[],java.io.File) in Crawler cannot be applied to (java.lang.String[])
    crawler.process(argStrings);

When the App file looks like this:

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.process(argStrings);
  }
}

Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16546216
No, you need to pass the directory.Somehting like:

 Crawler crawler = new Crawler();
 crawler.process(argStrings, new File("."));
0
 
LVL 1

Author Comment

by:scottcw
ID: 16546312
I tried what you said and i get a compile error:

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.process(argStrings, new File("."));
  }
}

Gives:

C:\Documents and Settings\User\Desktop\CrawlerApp.java:6: cannot find symbol
symbol  : class File
location: class CrawlerApp
    crawler.process(argStrings, new File("."));
                                    ^
1 error

Tool completed with exit code 1

I have not done much work on App files before, I just use the same format for each program therefore I dont know what this does.
Cheers
0
 
LVL 1

Author Comment

by:scottcw
ID: 16546374
I did not import
sorry
0
 
LVL 1

Author Comment

by:scottcw
ID: 16546438
Well it compiles, it runs when I input the directory, copydate and extensions althrough it says it has finished and then I read the logs and they are empty for some reason.

Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16546481
Try starting at a different directory than the current directory
0
 
LVL 1

Author Comment

by:scottcw
ID: 16546643
The program should work when the directory is in the same directory as the program (this worked when it only did one level).

I tried somedifferent file extensions and i got some output in the log, but these were files on my desktop where the Crawler and CrawlerApp are.

So I think at the moment it is doing the files on the same level as the directory specified and not directories inside the directory.

Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16547589
I'm not sure what you're trying to do with this code
0
 
LVL 1

Author Comment

by:scottcw
ID: 16547727
This is some of the javadoc in my program:

/**
 * The Crawler class executes six methods that identify files which could potentially
 * be archived and copied in a directory specified on the command line. A file is to
 * be archived if the last modified date is more than 100 days ago. A file is to be copied
 * if it has been modified since a given time. Only nominated extensions are to be
 * considered (php and html).
 * <p>
 * The output from this class is two files (called archive.txt and copy.txt)
 * containing the names of files to be archived, and those which may be copied. The
 * names will follow the convention specified in the create application.
 * <p>
 * $ java CrawlerApp [directory name] [copy date] [extension one] [extension two]
 *
 * @author        John

Do you want me to post the whole code to you, or email it maybe, then you can compile it yourself to get a better understanding?
Cheers
John
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554058
I am recursing over all the directories, but only reporting on the top level.

I am not using the argument "files" to process().

I went to the Java forums and they suggested this, I have been trying for a while to implement this but with no luck. Please could you have a go. Once this is done I think it will work:

1) start recursing (do processDirectory())
2) at each directory, call process() to process that directory.

Don't call processDirectory() from process(). It should be the other way around.

This is the code that needs to be changed:

public void process(String[] argStrings, File files)
  {
// validation code for the command line
    File folder = new File(name);
    processDirectory(folder);
 
    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] htmlFiles = files.listFiles(multiFilter);
 
      multiFilter.setExtension(fileExtensionTwo);
    File[] phpFiles = files.listFiles(multiFilter);
 
    File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
    log(filesToLog, archiveLimit, copyLimit);
  }
 
  public File processDirectory(File folder)
  {
    try
    {
      File[] files = folder.listFiles();
 
      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(new File(folder,files[i].getName())); // Add the directories to the filename so that they appear in the log
 
          return files[i];
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }


Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554270
Your code looks more or less alright, but the sole functionality of processDirectory, IMO is to recurse subdirectories. If so,  all it needs to be is

public void processDirectory(File folder)
  {
    try
    {

        if (folder.isDirectory())
        {
          process(folder); // Do your processing
        }

      File[] files = folder.listFiles();
 
      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i]); // Add the directories to the filename so that they appear in the log
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554369
I changed to what you said, and I only get one compile error now which is good. Is this something to do with not passing enough values back to process?

java:184: process(java.lang.String[],java.io.File) in Crawler cannot be applied to (java.io.File)
        process(folder); // Do your processing
        ^
1 error

Tool completed with exit code 1

Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554386
Yes. What's the String parameter meant to be
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554426
The process method takes arguements from the command line e.g. directory = argstrings[0], copydate = argstrings[1], extension 1 and 2 = argstrings[2] and [3]:

public void process(String[] argStrings, File files)
{
Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554477
>>Don't call processDirectory() from process(). It should be the other way around.

If you want the above to be the case, you will have to forward that String[] *through* processDirectory *to* process
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554528
Something like this?

public void process(String[] argStrings, File files)
  {
//code not shown
  }

 public void processDirectory(File folder, String[] argStrings)
  {
    try
    {
      if (folder.isDirectory())
      {
        process(argStrings, folder); // Do your processing
      }

      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }

It compiles, but still has the same problem, it only does one level.
Cheers
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554537
Oh and I forgot to say that this has been removed from process:
processDirectory(folder);
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554558
Put some debug code in before

>>processDirectory(files[i], argStrings);

such as

System.out.println("Calling processDirectory for directory " + files[i]);

and tell me what it does
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554569
It does nothing.
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554616
This must mean that it is still only doing oe level as indicated by the logs.
cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554632
Oh - please put the same sort of thing before

>>process(argStrings, folder); // Do your processing
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554695
I get these errors with this code:

C:\Documents and Settings\User\Desktop\Computing\Crawler.java:184: cannot find symbol
symbol  : variable files
location: class Crawler
            System.out.println("Calling processDirectory for directory " + files[i]);
                                                                               ^
C:\Documents and Settings\User\Desktop\Computing\Crawler.java:184: cannot find symbol
symbol  : variable i
location: class Crawler
            System.out.println("Calling processDirectory for directory " + files[i]);
                                                                                     ^
2 errors
Code:
public void process(String[] argStrings, File files)
  {
//more code
    File folder = new File(directory);
      process(argStrings, folder);

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] extensionOne = files.listFiles(multiFilter);

      multiFilter.setExtension(fileExtensionTwo);
    File[] extensionTwo = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
    log(filesToLog, archiveLimit, copyLimit);
  }

  public void processDirectory(File folder, String[] argStrings)
  {
    try
    {
      if (folder.isDirectory())
      {
            System.out.println("Calling processDirectory for directory " + files[i]);
        process(argStrings, folder); // Do your processing
      }

      File[] files = folder.listFiles();

      for (int i = 0; i < files.length; i++)
      {
        if (files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
        }
      }
    }
    catch (Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
  }

Cheers
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554763
I tried this also:

System.out.println("Calling processDirectory for directory " + folder);

And still nothing.
Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554777
>>I tried this also:

That's the one. Are you saying you are getting *no output at all*?
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554795
On the command line nothing prints i.e. the ""Calling processDirectory for directory " + folder" does not show.

Get one level of directories in the logs though - still the same.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16554821
Then that method is not being called at all. Got to go for a while
0
 
LVL 1

Author Comment

by:scottcw
ID: 16554848
Ok cheers for the help
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16556305
>>Then that method is not being called at all.

(Unless of course it's not being called in the first place with a directory - which would be quite usesless)
0
 
LVL 1

Author Comment

by:scottcw
ID: 16558652
Right now I have got it to scan through every directory and file (each is displayed on the command line).

Some of the logs are being written properly also (deeper directories).

This is my latest code:

App file:

import java.io.*;

public class CrawlerApp
{
  public static void main(String[] argStrings)
  {
      Crawler crawler = new Crawler();
    crawler.processDirectory(new File("."), argStrings);
    //crawler.process(argStrings, new File("."));
  }
}

This is process and processDirectory:
public void process(String[] argStrings, File files)
  {
    File folder = new File(directory);

    MultiFilter multiFilter = new MultiFilter(fileExtensionOne);
      File[] extensionOne = files.listFiles(multiFilter);

      multiFilter.setExtension(fileExtensionTwo);
    File[] extensionTwo = files.listFiles(multiFilter);

    File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
    log(filesToLog, archiveLimit, copyLimit);

      processDirectory(folder, argStrings);
  }

  public File processDirectory(File folder, String[] argStrings)
  {
    try
    {
      File[] files = folder.listFiles();

      for(int i = 0; i < files.length; i++)
      {
            if(files[i].isFile())
        {
              System.out.println("Files " + files[i]);
            }
        else if(files[i].isDirectory())
        {
          processDirectory(files[i], argStrings);
          System.out.println("Directories " + files[i]);
          process(argStrings, files[i]);
          }
      }
    }
    catch(Exception e)
    {
      System.err.println("Problem with directory " + folder);
      System.exit(0);
    }
    return null;
  }
Cheers
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16560234
Well it looks like you need to implement 'author John's' functionality

e.g. 'argStrings' is currently completely unused ...
0
 
LVL 1

Author Comment

by:scottcw
ID: 16569521
Sorry its taken a while, but I have actually managed to get the program working perfectly with HTML formatted output.

I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.

So far for functional testing I have tried different parameters on the command line (the program takes input from the command line) e.g. java CrawlerApp [directory name] [copy date] [extension one] [extension two] would be input or the actual input would be java CrawlerApp knuth 2005-12-28 php html.

The program then searches all files in that directroy even if they are in lower levels of directories and outputs the php and html files that have a last modified date of 2005-12-28.

I was wondering if there are any more functional tests that i could do for a program like this?

I am very new to structural testing - would I test try and catch blocks and for for loops would i just show that it printed every iteration to the command line for instance? Any tips on this would be appreciated.

Thank you
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 1600 total points
ID: 16572024
>>I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.

Let me say we are not obliged to answer additional questions - i have already helped you quite extensively in this question. However, this is what you need for testing - see documentation

http://www.junit.org/
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16681198
:-)
0

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java contains several comparison operators (e.g., <, <=, >, >=, ==, !=) that allow you to compare primitive values. However, these operators cannot be used to compare the contents of objects. Interface Comparable is used to allow objects of a cl…
In this post we will learn different types of Android Layout and some basics of an Android App.
The viewer will learn how to implement Singleton Design Pattern in Java.
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
Suggested Courses
Course of the Month13 days, 21 hours left to enroll

581 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question