Reading file name and compare them take out the difference

Hi,

I have file something like bellow and there is many files coming on daily bases:

fil1.20130121000108.20130121000608.txt.gz
fil2.20130121000608.20130121001108.txt.gz
etc...

so from the first file the bold section that you see it starts from that and on the second file the bold section would be the end of the file so what I want to compare those two files based on the section I bold and if there is any difference it should write it in a filed.

what I want is the following:

- read the file name and compare them  and I want something like bellow a table where I can check quickly:

Processed date: filename: size:difference

thanks for help
LVL 1
hi4pplAsked:
Who is Participating?
 
CEHJCommented:
This is i think what you need. You need to add moving files to it

	File[] listOfFiles = folder.listFiles();
	Arrays.sort(listOfFiles);

	long lastTime = 0;
	String lastName = null;
	for (int i = 0; i < listOfFiles.length; i++) {
	    File f = listOfFiles[i];
	    if (f.isFile()) {
		String[] atoms = f.getName().split("\\.");
		if(lastTime == 0) {
		    lastTime = Long.parseLong(atoms[2]);
		    lastName = f.getName();
		    continue;
		}
		else {
		    long latestTime = Long.parseLong(atoms[1]);
		    long diff = latestTime - lastTime;
		    if (diff != 0) {
			System.out.printf("Gap of %d found between predecessor file %s and current file %s\n", diff, lastName, f.getName());
		    }
		    lastTime = Long.parseLong(atoms[2]);
		    lastName = f.getName();
		}
	    } else if (listOfFiles[i].isDirectory()) {
		System.out.println("Directory " + listOfFiles[i].getName());
	    }
	}

Open in new window

0
 
ozoCommented:
What would
Processed date: filename: size:difference
be in the case of the example you showed above?
0
 
CEHJCommented:
and if there is any difference it should write it in a filed.
Any difference in what? The contents? If so, you need to take the checksum each time, as even if the size if the same as the last one, it doesn't mean the contents are the same. If you want to know the difference in terms of content, you need to use a diff algo (assuming text contents, not binary)
0
Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

 
krakatoaCommented:
With these two experts asking you, it's unlikely you are talking about something as simple as this. But here it is anyway:

class Difference {

static String f1 = "fil1.20130121000108.20130121000608.txt.gz";
static String f2 = "fil2.20130121000608.20130121001108.txt.gz";


public static void main(String[] args){

String[] partsf1 = f1.split("\\.");
String[] partsf2 = f2.split("\\.");

if(partsf1[2].equals(partsf2[1])){System.out.println("Contiguous content");}else{System.out.println("Non-contiguous content");}

}


}

Open in new window

0
 
hi4pplAuthor Commented:
Well Kraka that worked for me but what I want the files to be dynamic and grab the files dynamically and the example I give was two files to be understandable in the real data there is hunder thousand files:

so it would be helpful to grab the data sort them and then compare them as that way it would be more accurate if do it like that with hundred files then  without sorting the data this class will give wrong result.

second about:
Processed date|filename size |difference

I want this in text files or in a in database so I can check when this class run the time of that and the filesize is that sometimes the files come blank without any content or the files don't have enough that that it suppose to have so can track that also.

thanks for all helps :D

one more thing to add is the files is

20130121000608
YYYYMMDDHHMMSS

so if there is any difference in the SS part it should tell me how many SEC there is difference.

thanks
0
 
CEHJCommented:
Sounds like you would benefit from a database approach where you had

id | from_time | to_time | path

columns, so you could do lookups
0
 
hi4pplAuthor Commented:
Well yeah that could also be good anything that help me identified the size, and difference in Second on the files if any or else null if there is no difference in the time of those files
0
 
CEHJCommented:
You still haven't made yourself very clear on what functionality you require. You would make things clearer if you stated your actual GOALS
0
 
hi4pplAuthor Commented:
Well my Goal is clear I have already explained in my question and details:

those two files is example (krakatoa) created that class which does the job but I need addition to that to do the following as well as that script only do two files but in my case there is hundred files coming in daily bases so:

1- I need to read those files and sort them out make sure the files are in order
2- compare the those values in SS and
3- Create a text files of load it in database the following values

processed date (system date) | size of the files | Difference if any or else put zero

regards
0
 
krakatoaCommented:
Unfortunately I was right first time about not understanding your aims. I am now completely confused.
0
 
hi4pplAuthor Commented:
krakatoa your example is good but what it does is for two files what I want this to be dynamic and do this for files that existent in a directory
0
 
krakatoaCommented:
Look you are throwing away help from a couple of major experts, myself excluded, because your explanation is not clear at all.

The question is going to drift still further if you ignore CEHJ's recent link so please read it.

For myself, I now don't know if you want to :

a) sort files *internally* - i.e. their contents;
b) sort them on disk;
c) sort their filenames;
d) search the filenames and return a 1 or 0 if some sequence is missing.

Can't spend all day on this.
0
 
hi4pplAuthor Commented:
Well bellow is the class I manage to create that list the files form that directory and your script does the comparison like check if the sequence meet and if not it gives another error

my script
		
		File folder = new File("D:files");
		File[] listOfFiles = folder.listFiles(); {
		
		    for (int i = 0; i < listOfFiles.length; i++) {
		      if (listOfFiles[i].isFile()) {
		        System.out.println("File " + listOfFiles[i].getName());
		      } else if (listOfFiles[i].isDirectory()) {
		        System.out.println("Directory " + listOfFiles[i].getName());
		      }
		    }
		}

Open in new window



so I want to combine your script that check the sequence and mine that list the files and both in once single out put I hope it's clear now.

thanks
0
 
krakatoaCommented:
Let's see if CEHJ or ozo comment from here if they will, and give you the best options.
0
 
krakatoaCommented:
In the meantime, there is something else I don't understand, and that is why you don't just grab lines 9,10 and 12 above, tidy them up a bit, shoot the probable next couple of Exceptions I've overlooked, and get the result? If you are entrusted with twiddling around with hundreds of thousands of files, I'd imagine this little bit of code integration won't cause you too much difficulty. ? .
0
 
CEHJCommented:
System.out.println("File " + listOfFiles[i].getName());

Open in new window

You see - this is one of the things causing confusion - that's not 'reading files' - it's reading file names

So please now clear up the confusion between the two. Is any of your required functionality actually concerned with the contents of the file? If so, what is it?
0
 
hi4pplAuthor Commented:
Hi,
thanks for the help, no sorry about confusion all I need is the file name itself I don't care about what is inside but I want to make sure to list all filename and if there is any missing in sequence that is all.

and secondly im not that advance in java just a little I know that I created that statesmen and after that I stuck so I ask help here to extended that script. and compare those file name if sequence is missing .

the file formate that I menioned above is like bellow :

20130121000608
YYYYMMDDHHMMSS

and that bold part I want to either do a divided and show the difference if there is any so I know how many second delayed in file sequence

thanks for help
0
 
CEHJCommented:
OK, can we have, please, a longer and more realistic directory listing.

Also bear in mind that you really need to mv (move) the files from that directory, or you'll just be repeating work
0
 
mcnuteCommented:
FolderA=/XXX/XXX
FolderB=/YYY/YYY
Difffolder=/ZZZ/ZZZ

cd $FolderA

for file in $(ls *)
do
   diff $FolderA/file $FolderB/file > $Difffolder/diff.patc
done

Open in new window

0
 
CEHJCommented:
diff $FolderA/file $FolderB/file > $Difffolder/diff.patc

Open in new window

I don't care about what is inside
0
 
mcnuteCommented:
So you want the timestamp difference in days or minutes or seconds? What do you want? Which format?
0
 
hi4pplAuthor Commented:
Hi, about the directory listing if you mean where my files located then im flexible about that I can manage to change the directory if required. and yeah that I need to do to move today file when the comparison done and empty the directory for new files to come and do that in the new files

thanks for the help
0
 
CEHJCommented:
OK, can we have, please, a longer and more realistic directory listing.
0
 
hi4pplAuthor Commented:
sorry here is the details

File.20130121000108.20130121000608.txt.gz
File.20130121000608.20130121001108.txt.gz
File.20130121001108.20130121001608.txt.gz
File.20130121001608.20130121002108.txt.gz
File.20130121002108.20130121002608.txt.gz
File.20130121002608.20130121003108.txt.gz
File.20130121003108.20130121003608.txt.gz
File.20130121003608.20130121004108.txt.gz
File.20130121004108.20130121004608.txt.gz
File.20130121004608.20130121005108.txt.gz
File.20130121005108.20130121005608.txt.gz

Open in new window

0
 
hi4pplAuthor Commented:
Hi, thank you very much it does the job I really appreciate that, however can I take the out put of files to a text file and also move them to other folder in t his script?

and also on split section I have other batch of files that have two level of character how can I modify this to use it in other batch of files those files are as bellow:

File..sim.20130121000108.20130121000608.txt.gz
File.sim.20130121000608.20130121001108.txt.gz
File.sim.20130121001108.20130121001608.txt.gz
File.sim.20130121001608.20130121002108.txt.gz
File.sim.20130121002108.20130121002608.txt.gz
File.sim.20130121002608.20130121003108.txt.gz
File.sim.20130121003108.20130121003608.txt.gz
File.sim.20130121003608.20130121004108.txt.gz
File.sim.20130121004108.20130121004608.txt.gz
File.sim.20130121004608.20130121005108.txt.gz
File.sim.20130121005108.20130121005608.txt.gz

Open in new window


thanks again for your help appreciate your time you spend on this :D
0
 
CEHJCommented:
however can I take the out put of files to a text file and also move them to other folder in t his script?
In general you can do

java YourApp | tee somefile.log

Open in new window


I'm afraid i don't have the time to continue doing individual alterations
0
 
hi4pplAuthor Commented:
Hi thanks I found the solution and fix that extra (.) now when I run the programm in eclipse eveyrthing is working well but when I go and run the .class from CMD it throw an error :


Exception in thread "main" java.lang.NoClassDefFoundError: readFiles (wrong name
: com/test/compare/readFiles)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$100(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Open in new window


I have added java path and class path as well in my system and still same error: I have try running it like bellow

java -classpath . classname
no luck and still it throws error, the program compiled without any error I try it both from CMD and from Eclipse it goes without any error but when running it from CMD it gives that error


Bellow is the final changes I did and it runs okay in eclipse but not in cmd

import java.io.File;
import java.util.Arrays;

public class readme {

	public static void main(String[] args) {
		
		
		File folder = new File("D:files");
		File[] listOfFiles = folder.listFiles();
		Arrays.sort(listOfFiles);

		long lastTime = 0;
		String lastName = null;
		for (int i = 0; i < listOfFiles.length; i++) {
		    File f = listOfFiles[i];
		   // System.out.println("List of Files" + f);
		    if (f.isFile()) {
			String[] atoms = f.getName().split("\\.");
			if(lastTime == 0) {
			    lastTime = Long.parseLong(atoms[3]);
			    lastName = f.getName();
			    continue;
			    
			}
			else {
			    long latestTime = Long.parseLong(atoms[2]);
			    long diff = latestTime - lastTime;
			    if (diff != 0) {
				System.out.printf("Gap of %d Seconds found between predecessor file %s and current file %s\n", diff, lastName, f.getName());
			    }
			    lastTime = Long.parseLong(atoms[3]);
			    lastName = f.getName();
			}
			System.out.println(f);
//		    } else if (listOfFiles[i].isDirectory()) {
//			System.out.println("Directory " + listOfFiles[i].getName());
		    }
		}
	}
}

Open in new window

0
 
hi4pplAuthor Commented:
Hi fix it never mind
0
 
CEHJCommented:
:)
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.