scottcw
asked on
File filter program that writes a log for one directory layer but no more, I have code for doing extra layers although cannot encorpate it
For this program I need to search multiple directory layers and then if the file is in a directory then the file name will have the directory name added to it. This is the code for the directory scanning so you can get an idea of what I mean:
[code]
public void processDirectory(String directoryName)
{
try
{
File dir = new File(directoryName);
File[] files = dir.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(directory Name + "/" + files[i].getName());
}
}
} catch (Exception e)
{
System.err.println("Proble m with " + directoryName);
System.exit(0);
}
}
[/code]
This is what I have managed to do so far, it includes comments. This is obviously not all of the code - I can post all of the code if you think it will help.
[code]
public void process(String[] argStrings, File files)
{
//lots of code for validation of command line (the directory, copydate, fileextension1 and fileexttesnion 2 are entered on the command line)
File folder = new File(name);
processDirectory(folder); // This passes the folder (argstrings0 to the processdirectory method)
MultiFilter multiFilter = new MultiFilter(fileExtensionO ne);
File[] htmlFiles = files.listFiles(multiFilte r); //when this was folder.listFiles(multiFilt er) it worked for one directory level
multiFilter.setExtension(f ileExtensi onTwo);
File[] phpFiles = files.listFiles(multiFilte r);
File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
log(filesToLog, archiveLimit, copyLimit);
}
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{ // once this method has found directories in the directories it recurses,
processDirectory(directory Name + "\\" + files[i].getName()); // Add the directories to the filename so that they appear in the log
return files[i];// here i am trying to get all the files in the directory (s) back to the method above so that they can be filtered
// i.e. either php or html as eneterd on the command line
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
}
// Also I use this to run the application, this might be wrong also:
import java.io.*;
public class CrawlerApp
{
public static void main(String[] argStrings, File files)
{
Crawler crawler = new Crawler();
crawler.process(argStrings , files);
}
}
//At the moment my program does not compile and I get these errors:
//C:\Documents and Settings\User\Desktop\Craw ler.java:2 09: processDirectory(java.io.F ile) in Crawler cannot be applied to (java.lang.String)
//processDirectory(folder + "\\" + files[i].getName());
[/code]
Thank you
[code]
public void processDirectory(String directoryName)
{
try
{
File dir = new File(directoryName);
File[] files = dir.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(directory
}
}
} catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
}
[/code]
This is what I have managed to do so far, it includes comments. This is obviously not all of the code - I can post all of the code if you think it will help.
[code]
public void process(String[] argStrings, File files)
{
//lots of code for validation of command line (the directory, copydate, fileextension1 and fileexttesnion 2 are entered on the command line)
File folder = new File(name);
processDirectory(folder); // This passes the folder (argstrings0 to the processdirectory method)
MultiFilter multiFilter = new MultiFilter(fileExtensionO
File[] htmlFiles = files.listFiles(multiFilte
multiFilter.setExtension(f
File[] phpFiles = files.listFiles(multiFilte
File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
log(filesToLog, archiveLimit, copyLimit);
}
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{ // once this method has found directories in the directories it recurses,
processDirectory(directory
return files[i];// here i am trying to get all the files in the directory (s) back to the method above so that they can be filtered
// i.e. either php or html as eneterd on the command line
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
}
// Also I use this to run the application, this might be wrong also:
import java.io.*;
public class CrawlerApp
{
public static void main(String[] argStrings, File files)
{
Crawler crawler = new Crawler();
crawler.process(argStrings
}
}
//At the moment my program does not compile and I get these errors:
//C:\Documents and Settings\User\Desktop\Craw
//processDirectory(folder + "\\" + files[i].getName());
[/code]
Thank you
ASKER
Ok this gets rid of that compile error, cheers.
Although now I get this compile error:
C:\Crawler.java:220: missing return statement
}
This is at the end of the processDirectory method.
Thanks
Although now I get this compile error:
C:\Crawler.java:220: missing return statement
}
This is at the end of the processDirectory method.
Thanks
Follow
>>System.err.println("Prob lem with directory " + folder);
with
return null;
>>System.err.println("Prob
with
return null;
ASKER
Yes this is what I tried a while ago, and it does compile although does not run now, I get an error like this at command line:
java.lang.nosuchmethoderro r: main
When the method looks like this:
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(new File(folder,files[i].getNa me())); // Add the directories to the filename so that they appear in the log
return files[i];
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
return null;
}
Cheers
java.lang.nosuchmethoderro
When the method looks like this:
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(new File(folder,files[i].getNa
return files[i];
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
return null;
}
Cheers
>>public static void main(String[] argStrings, File files)
should be
public static void main(String[] argStrings)
should be
public static void main(String[] argStrings)
ASKER
Making progress.
I get one compile error:
C:\Documents and Settings\User\Desktop\Craw lerApp.jav a:6: process(java.lang.String[] ,java.io.F ile) in Crawler cannot be applied to (java.lang.String[])
crawler.process(argStrings );
When the App file looks like this:
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.process(argStrings );
}
}
Thanks
I get one compile error:
C:\Documents and Settings\User\Desktop\Craw
crawler.process(argStrings
When the App file looks like this:
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.process(argStrings
}
}
Thanks
No, you need to pass the directory.Somehting like:
Crawler crawler = new Crawler();
crawler.process(argStrings , new File("."));
Crawler crawler = new Crawler();
crawler.process(argStrings
ASKER
I tried what you said and i get a compile error:
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.process(argStrings , new File("."));
}
}
Gives:
C:\Documents and Settings\User\Desktop\Craw lerApp.jav a:6: cannot find symbol
symbol : class File
location: class CrawlerApp
crawler.process(argStrings , new File("."));
^
1 error
Tool completed with exit code 1
I have not done much work on App files before, I just use the same format for each program therefore I dont know what this does.
Cheers
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.process(argStrings
}
}
Gives:
C:\Documents and Settings\User\Desktop\Craw
symbol : class File
location: class CrawlerApp
crawler.process(argStrings
^
1 error
Tool completed with exit code 1
I have not done much work on App files before, I just use the same format for each program therefore I dont know what this does.
Cheers
ASKER
I did not import
sorry
sorry
ASKER
Well it compiles, it runs when I input the directory, copydate and extensions althrough it says it has finished and then I read the logs and they are empty for some reason.
Cheers
Cheers
Try starting at a different directory than the current directory
ASKER
The program should work when the directory is in the same directory as the program (this worked when it only did one level).
I tried somedifferent file extensions and i got some output in the log, but these were files on my desktop where the Crawler and CrawlerApp are.
So I think at the moment it is doing the files on the same level as the directory specified and not directories inside the directory.
Cheers
I tried somedifferent file extensions and i got some output in the log, but these were files on my desktop where the Crawler and CrawlerApp are.
So I think at the moment it is doing the files on the same level as the directory specified and not directories inside the directory.
Cheers
I'm not sure what you're trying to do with this code
ASKER
This is some of the javadoc in my program:
/**
* The Crawler class executes six methods that identify files which could potentially
* be archived and copied in a directory specified on the command line. A file is to
* be archived if the last modified date is more than 100 days ago. A file is to be copied
* if it has been modified since a given time. Only nominated extensions are to be
* considered (php and html).
* <p>
* The output from this class is two files (called archive.txt and copy.txt)
* containing the names of files to be archived, and those which may be copied. The
* names will follow the convention specified in the create application.
* <p>
* $ java CrawlerApp [directory name] [copy date] [extension one] [extension two]
*
* @author John
Do you want me to post the whole code to you, or email it maybe, then you can compile it yourself to get a better understanding?
Cheers
John
/**
* The Crawler class executes six methods that identify files which could potentially
* be archived and copied in a directory specified on the command line. A file is to
* be archived if the last modified date is more than 100 days ago. A file is to be copied
* if it has been modified since a given time. Only nominated extensions are to be
* considered (php and html).
* <p>
* The output from this class is two files (called archive.txt and copy.txt)
* containing the names of files to be archived, and those which may be copied. The
* names will follow the convention specified in the create application.
* <p>
* $ java CrawlerApp [directory name] [copy date] [extension one] [extension two]
*
* @author John
Do you want me to post the whole code to you, or email it maybe, then you can compile it yourself to get a better understanding?
Cheers
John
ASKER
I am recursing over all the directories, but only reporting on the top level.
I am not using the argument "files" to process().
I went to the Java forums and they suggested this, I have been trying for a while to implement this but with no luck. Please could you have a go. Once this is done I think it will work:
1) start recursing (do processDirectory())
2) at each directory, call process() to process that directory.
Don't call processDirectory() from process(). It should be the other way around.
This is the code that needs to be changed:
public void process(String[] argStrings, File files)
{
// validation code for the command line
File folder = new File(name);
processDirectory(folder);
MultiFilter multiFilter = new MultiFilter(fileExtensionO ne);
File[] htmlFiles = files.listFiles(multiFilte r);
multiFilter.setExtension(f ileExtensi onTwo);
File[] phpFiles = files.listFiles(multiFilte r);
File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
log(filesToLog, archiveLimit, copyLimit);
}
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(new File(folder,files[i].getNa me())); // Add the directories to the filename so that they appear in the log
return files[i];
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
return null;
}
Thanks
I am not using the argument "files" to process().
I went to the Java forums and they suggested this, I have been trying for a while to implement this but with no luck. Please could you have a go. Once this is done I think it will work:
1) start recursing (do processDirectory())
2) at each directory, call process() to process that directory.
Don't call processDirectory() from process(). It should be the other way around.
This is the code that needs to be changed:
public void process(String[] argStrings, File files)
{
// validation code for the command line
File folder = new File(name);
processDirectory(folder);
MultiFilter multiFilter = new MultiFilter(fileExtensionO
File[] htmlFiles = files.listFiles(multiFilte
multiFilter.setExtension(f
File[] phpFiles = files.listFiles(multiFilte
File[] filesToLog = mergeFiles(htmlFiles, phpFiles);
log(filesToLog, archiveLimit, copyLimit);
}
public File processDirectory(File folder)
{
try
{
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(new File(folder,files[i].getNa
return files[i];
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
return null;
}
Thanks
Your code looks more or less alright, but the sole functionality of processDirectory, IMO is to recurse subdirectories. If so, all it needs to be is
public void processDirectory(File folder)
{
try
{
if (folder.isDirectory())
{
process(folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i]) ; // Add the directories to the filename so that they appear in the log
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
}
public void processDirectory(File folder)
{
try
{
if (folder.isDirectory())
{
process(folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i])
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
}
ASKER
I changed to what you said, and I only get one compile error now which is good. Is this something to do with not passing enough values back to process?
java:184: process(java.lang.String[] ,java.io.F ile) in Crawler cannot be applied to (java.io.File)
process(folder); // Do your processing
^
1 error
Tool completed with exit code 1
Cheers
java:184: process(java.lang.String[]
process(folder); // Do your processing
^
1 error
Tool completed with exit code 1
Cheers
Yes. What's the String parameter meant to be
ASKER
The process method takes arguements from the command line e.g. directory = argstrings[0], copydate = argstrings[1], extension 1 and 2 = argstrings[2] and [3]:
public void process(String[] argStrings, File files)
{
Thanks
public void process(String[] argStrings, File files)
{
Thanks
>>Don't call processDirectory() from process(). It should be the other way around.
If you want the above to be the case, you will have to forward that String[] *through* processDirectory *to* process
If you want the above to be the case, you will have to forward that String[] *through* processDirectory *to* process
ASKER
Something like this?
public void process(String[] argStrings, File files)
{
//code not shown
}
public void processDirectory(File folder, String[] argStrings)
{
try
{
if (folder.isDirectory())
{
process(argStrings, folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i], argStrings);
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
}
It compiles, but still has the same problem, it only does one level.
Cheers
public void process(String[] argStrings, File files)
{
//code not shown
}
public void processDirectory(File folder, String[] argStrings)
{
try
{
if (folder.isDirectory())
{
process(argStrings, folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i],
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
}
It compiles, but still has the same problem, it only does one level.
Cheers
ASKER
Oh and I forgot to say that this has been removed from process:
processDirectory(folder);
processDirectory(folder);
Put some debug code in before
>>processDirectory(files[i ], argStrings);
such as
System.out.println("Callin g processDirectory for directory " + files[i]);
and tell me what it does
>>processDirectory(files[i
such as
System.out.println("Callin
and tell me what it does
ASKER
It does nothing.
ASKER
This must mean that it is still only doing oe level as indicated by the logs.
cheers
cheers
Oh - please put the same sort of thing before
>>process(argStrings, folder); // Do your processing
>>process(argStrings, folder); // Do your processing
ASKER
I get these errors with this code:
C:\Documents and Settings\User\Desktop\Comp uting\Craw ler.java:1 84: cannot find symbol
symbol : variable files
location: class Crawler
System.out.println("Callin g processDirectory for directory " + files[i]);
^
C:\Documents and Settings\User\Desktop\Comp uting\Craw ler.java:1 84: cannot find symbol
symbol : variable i
location: class Crawler
System.out.println("Callin g processDirectory for directory " + files[i]);
^
2 errors
Code:
public void process(String[] argStrings, File files)
{
//more code
File folder = new File(directory);
process(argStrings, folder);
MultiFilter multiFilter = new MultiFilter(fileExtensionO ne);
File[] extensionOne = files.listFiles(multiFilte r);
multiFilter.setExtension(f ileExtensi onTwo);
File[] extensionTwo = files.listFiles(multiFilte r);
File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
log(filesToLog, archiveLimit, copyLimit);
}
public void processDirectory(File folder, String[] argStrings)
{
try
{
if (folder.isDirectory())
{
System.out.println("Callin g processDirectory for directory " + files[i]);
process(argStrings, folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i], argStrings);
}
}
}
catch (Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
}
Cheers
C:\Documents and Settings\User\Desktop\Comp
symbol : variable files
location: class Crawler
System.out.println("Callin
^
C:\Documents and Settings\User\Desktop\Comp
symbol : variable i
location: class Crawler
System.out.println("Callin
^
2 errors
Code:
public void process(String[] argStrings, File files)
{
//more code
File folder = new File(directory);
process(argStrings, folder);
MultiFilter multiFilter = new MultiFilter(fileExtensionO
File[] extensionOne = files.listFiles(multiFilte
multiFilter.setExtension(f
File[] extensionTwo = files.listFiles(multiFilte
File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
log(filesToLog, archiveLimit, copyLimit);
}
public void processDirectory(File folder, String[] argStrings)
{
try
{
if (folder.isDirectory())
{
System.out.println("Callin
process(argStrings, folder); // Do your processing
}
File[] files = folder.listFiles();
for (int i = 0; i < files.length; i++)
{
if (files[i].isDirectory())
{
processDirectory(files[i],
}
}
}
catch (Exception e)
{
System.err.println("Proble
System.exit(0);
}
}
Cheers
ASKER
I tried this also:
System.out.println("Callin g processDirectory for directory " + folder);
And still nothing.
Cheers
System.out.println("Callin
And still nothing.
Cheers
>>I tried this also:
That's the one. Are you saying you are getting *no output at all*?
That's the one. Are you saying you are getting *no output at all*?
ASKER
On the command line nothing prints i.e. the ""Calling processDirectory for directory " + folder" does not show.
Get one level of directories in the logs though - still the same.
Get one level of directories in the logs though - still the same.
Then that method is not being called at all. Got to go for a while
ASKER
Ok cheers for the help
>>Then that method is not being called at all.
(Unless of course it's not being called in the first place with a directory - which would be quite usesless)
(Unless of course it's not being called in the first place with a directory - which would be quite usesless)
ASKER
Right now I have got it to scan through every directory and file (each is displayed on the command line).
Some of the logs are being written properly also (deeper directories).
This is my latest code:
App file:
import java.io.*;
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.processDirectory(n ew File("."), argStrings);
//crawler.process(argStrin gs, new File("."));
}
}
This is process and processDirectory:
public void process(String[] argStrings, File files)
{
File folder = new File(directory);
MultiFilter multiFilter = new MultiFilter(fileExtensionO ne);
File[] extensionOne = files.listFiles(multiFilte r);
multiFilter.setExtension(f ileExtensi onTwo);
File[] extensionTwo = files.listFiles(multiFilte r);
File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
log(filesToLog, archiveLimit, copyLimit);
processDirectory(folder, argStrings);
}
public File processDirectory(File folder, String[] argStrings)
{
try
{
File[] files = folder.listFiles();
for(int i = 0; i < files.length; i++)
{
if(files[i].isFile())
{
System.out.println("Files " + files[i]);
}
else if(files[i].isDirectory())
{
processDirectory(files[i], argStrings);
System.out.println("Direct ories " + files[i]);
process(argStrings, files[i]);
}
}
}
catch(Exception e)
{
System.err.println("Proble m with directory " + folder);
System.exit(0);
}
return null;
}
Cheers
Some of the logs are being written properly also (deeper directories).
This is my latest code:
App file:
import java.io.*;
public class CrawlerApp
{
public static void main(String[] argStrings)
{
Crawler crawler = new Crawler();
crawler.processDirectory(n
//crawler.process(argStrin
}
}
This is process and processDirectory:
public void process(String[] argStrings, File files)
{
File folder = new File(directory);
MultiFilter multiFilter = new MultiFilter(fileExtensionO
File[] extensionOne = files.listFiles(multiFilte
multiFilter.setExtension(f
File[] extensionTwo = files.listFiles(multiFilte
File[] filesToLog = mergeFiles(extensionOne, extensionTwo);
log(filesToLog, archiveLimit, copyLimit);
processDirectory(folder, argStrings);
}
public File processDirectory(File folder, String[] argStrings)
{
try
{
File[] files = folder.listFiles();
for(int i = 0; i < files.length; i++)
{
if(files[i].isFile())
{
System.out.println("Files " + files[i]);
}
else if(files[i].isDirectory())
{
processDirectory(files[i],
System.out.println("Direct
process(argStrings, files[i]);
}
}
}
catch(Exception e)
{
System.err.println("Proble
System.exit(0);
}
return null;
}
Cheers
Well it looks like you need to implement 'author John's' functionality
e.g. 'argStrings' is currently completely unused ...
e.g. 'argStrings' is currently completely unused ...
ASKER
Sorry its taken a while, but I have actually managed to get the program working perfectly with HTML formatted output.
I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.
So far for functional testing I have tried different parameters on the command line (the program takes input from the command line) e.g. java CrawlerApp [directory name] [copy date] [extension one] [extension two] would be input or the actual input would be java CrawlerApp knuth 2005-12-28 php html.
The program then searches all files in that directroy even if they are in lower levels of directories and outputs the php and html files that have a last modified date of 2005-12-28.
I was wondering if there are any more functional tests that i could do for a program like this?
I am very new to structural testing - would I test try and catch blocks and for for loops would i just show that it printed every iteration to the command line for instance? Any tips on this would be appreciated.
Thank you
I think if you were to give me some info on functional and structural testing then it would be appropriate to give the 200 points.
So far for functional testing I have tried different parameters on the command line (the program takes input from the command line) e.g. java CrawlerApp [directory name] [copy date] [extension one] [extension two] would be input or the actual input would be java CrawlerApp knuth 2005-12-28 php html.
The program then searches all files in that directroy even if they are in lower levels of directories and outputs the php and html files that have a last modified date of 2005-12-28.
I was wondering if there are any more functional tests that i could do for a program like this?
I am very new to structural testing - would I test try and catch blocks and for for loops would i just show that it printed every iteration to the command line for instance? Any tips on this would be appreciated.
Thank you
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
:-)
should be
processDirectory(new File(folder,files[i].getNa