Solved

Reading file name and compare them take out the difference

Posted on 2013-01-22
29
320 Views
Last Modified: 2013-01-24
Hi,

I have file something like bellow and there is many files coming on daily bases:

fil1.20130121000108.20130121000608.txt.gz
fil2.20130121000608.20130121001108.txt.gz
etc...

so from the first file the bold section that you see it starts from that and on the second file the bold section would be the end of the file so what I want to compare those two files based on the section I bold and if there is any difference it should write it in a filed.

what I want is the following:

- read the file name and compare them  and I want something like bellow a table where I can check quickly:

Processed date: filename: size:difference

thanks for help
0
Comment
Question by:hi4ppl
  • 11
  • 10
  • 5
  • +2
29 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 38805439
What would
Processed date: filename: size:difference
be in the case of the example you showed above?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38805517
and if there is any difference it should write it in a filed.
Any difference in what? The contents? If so, you need to take the checksum each time, as even if the size if the same as the last one, it doesn't mean the contents are the same. If you want to know the difference in terms of content, you need to use a diff algo (assuming text contents, not binary)
0
 
LVL 16

Expert Comment

by:krakatoa
ID: 38805687
With these two experts asking you, it's unlikely you are talking about something as simple as this. But here it is anyway:

class Difference {

static String f1 = "fil1.20130121000108.20130121000608.txt.gz";
static String f2 = "fil2.20130121000608.20130121001108.txt.gz";


public static void main(String[] args){

String[] partsf1 = f1.split("\\.");
String[] partsf2 = f2.split("\\.");

if(partsf1[2].equals(partsf2[1])){System.out.println("Contiguous content");}else{System.out.println("Non-contiguous content");}

}


}

Open in new window

0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38806042
Well Kraka that worked for me but what I want the files to be dynamic and grab the files dynamically and the example I give was two files to be understandable in the real data there is hunder thousand files:

so it would be helpful to grab the data sort them and then compare them as that way it would be more accurate if do it like that with hundred files then  without sorting the data this class will give wrong result.

second about:
Processed date|filename size |difference

I want this in text files or in a in database so I can check when this class run the time of that and the filesize is that sometimes the files come blank without any content or the files don't have enough that that it suppose to have so can track that also.

thanks for all helps :D

one more thing to add is the files is

20130121000608
YYYYMMDDHHMMSS

so if there is any difference in the SS part it should tell me how many SEC there is difference.

thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38806216
Sounds like you would benefit from a database approach where you had

id | from_time | to_time | path

columns, so you could do lookups
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38806290
Well yeah that could also be good anything that help me identified the size, and difference in Second on the files if any or else null if there is no difference in the time of those files
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38806329
You still haven't made yourself very clear on what functionality you require. You would make things clearer if you stated your actual GOALS
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38806422
Well my Goal is clear I have already explained in my question and details:

those two files is example (krakatoa) created that class which does the job but I need addition to that to do the following as well as that script only do two files but in my case there is hundred files coming in daily bases so:

1- I need to read those files and sort them out make sure the files are in order
2- compare the those values in SS and
3- Create a text files of load it in database the following values

processed date (system date) | size of the files | Difference if any or else put zero

regards
0
 
LVL 16

Expert Comment

by:krakatoa
ID: 38806503
Unfortunately I was right first time about not understanding your aims. I am now completely confused.
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38806519
krakatoa your example is good but what it does is for two files what I want this to be dynamic and do this for files that existent in a directory
0
 
LVL 16

Expert Comment

by:krakatoa
ID: 38806548
Look you are throwing away help from a couple of major experts, myself excluded, because your explanation is not clear at all.

The question is going to drift still further if you ignore CEHJ's recent link so please read it.

For myself, I now don't know if you want to :

a) sort files *internally* - i.e. their contents;
b) sort them on disk;
c) sort their filenames;
d) search the filenames and return a 1 or 0 if some sequence is missing.

Can't spend all day on this.
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38806566
Well bellow is the class I manage to create that list the files form that directory and your script does the comparison like check if the sequence meet and if not it gives another error

my script
		
		File folder = new File("D:files");
		File[] listOfFiles = folder.listFiles(); {
		
		    for (int i = 0; i < listOfFiles.length; i++) {
		      if (listOfFiles[i].isFile()) {
		        System.out.println("File " + listOfFiles[i].getName());
		      } else if (listOfFiles[i].isDirectory()) {
		        System.out.println("Directory " + listOfFiles[i].getName());
		      }
		    }
		}

Open in new window



so I want to combine your script that check the sequence and mine that list the files and both in once single out put I hope it's clear now.

thanks
0
 
LVL 16

Expert Comment

by:krakatoa
ID: 38806614
Let's see if CEHJ or ozo comment from here if they will, and give you the best options.
0
 
LVL 16

Expert Comment

by:krakatoa
ID: 38807471
In the meantime, there is something else I don't understand, and that is why you don't just grab lines 9,10 and 12 above, tidy them up a bit, shoot the probable next couple of Exceptions I've overlooked, and get the result? If you are entrusted with twiddling around with hundreds of thousands of files, I'd imagine this little bit of code integration won't cause you too much difficulty. ? .
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 86

Expert Comment

by:CEHJ
ID: 38807754
System.out.println("File " + listOfFiles[i].getName());

Open in new window

You see - this is one of the things causing confusion - that's not 'reading files' - it's reading file names

So please now clear up the confusion between the two. Is any of your required functionality actually concerned with the contents of the file? If so, what is it?
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38809281
Hi,
thanks for the help, no sorry about confusion all I need is the file name itself I don't care about what is inside but I want to make sure to list all filename and if there is any missing in sequence that is all.

and secondly im not that advance in java just a little I know that I created that statesmen and after that I stuck so I ask help here to extended that script. and compare those file name if sequence is missing .

the file formate that I menioned above is like bellow :

20130121000608
YYYYMMDDHHMMSS

and that bold part I want to either do a divided and show the difference if there is any so I know how many second delayed in file sequence

thanks for help
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38809500
OK, can we have, please, a longer and more realistic directory listing.

Also bear in mind that you really need to mv (move) the files from that directory, or you'll just be repeating work
0
 
LVL 11

Expert Comment

by:mcnute
ID: 38809668
FolderA=/XXX/XXX
FolderB=/YYY/YYY
Difffolder=/ZZZ/ZZZ

cd $FolderA

for file in $(ls *)
do
   diff $FolderA/file $FolderB/file > $Difffolder/diff.patc
done

Open in new window

0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38809750
diff $FolderA/file $FolderB/file > $Difffolder/diff.patc

Open in new window

I don't care about what is inside
0
 
LVL 11

Expert Comment

by:mcnute
ID: 38809886
So you want the timestamp difference in days or minutes or seconds? What do you want? Which format?
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38810045
Hi, about the directory listing if you mean where my files located then im flexible about that I can manage to change the directory if required. and yeah that I need to do to move today file when the comparison done and empty the directory for new files to come and do that in the new files

thanks for the help
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38810060
OK, can we have, please, a longer and more realistic directory listing.
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38810376
sorry here is the details

File.20130121000108.20130121000608.txt.gz
File.20130121000608.20130121001108.txt.gz
File.20130121001108.20130121001608.txt.gz
File.20130121001608.20130121002108.txt.gz
File.20130121002108.20130121002608.txt.gz
File.20130121002608.20130121003108.txt.gz
File.20130121003108.20130121003608.txt.gz
File.20130121003608.20130121004108.txt.gz
File.20130121004108.20130121004608.txt.gz
File.20130121004608.20130121005108.txt.gz
File.20130121005108.20130121005608.txt.gz

Open in new window

0
 
LVL 86

Accepted Solution

by:
CEHJ earned 500 total points
ID: 38811682
This is i think what you need. You need to add moving files to it

	File[] listOfFiles = folder.listFiles();
	Arrays.sort(listOfFiles);

	long lastTime = 0;
	String lastName = null;
	for (int i = 0; i < listOfFiles.length; i++) {
	    File f = listOfFiles[i];
	    if (f.isFile()) {
		String[] atoms = f.getName().split("\\.");
		if(lastTime == 0) {
		    lastTime = Long.parseLong(atoms[2]);
		    lastName = f.getName();
		    continue;
		}
		else {
		    long latestTime = Long.parseLong(atoms[1]);
		    long diff = latestTime - lastTime;
		    if (diff != 0) {
			System.out.printf("Gap of %d found between predecessor file %s and current file %s\n", diff, lastName, f.getName());
		    }
		    lastTime = Long.parseLong(atoms[2]);
		    lastName = f.getName();
		}
	    } else if (listOfFiles[i].isDirectory()) {
		System.out.println("Directory " + listOfFiles[i].getName());
	    }
	}

Open in new window

0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38811853
Hi, thank you very much it does the job I really appreciate that, however can I take the out put of files to a text file and also move them to other folder in t his script?

and also on split section I have other batch of files that have two level of character how can I modify this to use it in other batch of files those files are as bellow:

File..sim.20130121000108.20130121000608.txt.gz
File.sim.20130121000608.20130121001108.txt.gz
File.sim.20130121001108.20130121001608.txt.gz
File.sim.20130121001608.20130121002108.txt.gz
File.sim.20130121002108.20130121002608.txt.gz
File.sim.20130121002608.20130121003108.txt.gz
File.sim.20130121003108.20130121003608.txt.gz
File.sim.20130121003608.20130121004108.txt.gz
File.sim.20130121004108.20130121004608.txt.gz
File.sim.20130121004608.20130121005108.txt.gz
File.sim.20130121005108.20130121005608.txt.gz

Open in new window


thanks again for your help appreciate your time you spend on this :D
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38812012
however can I take the out put of files to a text file and also move them to other folder in t his script?
In general you can do

java YourApp | tee somefile.log

Open in new window


I'm afraid i don't have the time to continue doing individual alterations
0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38812440
Hi thanks I found the solution and fix that extra (.) now when I run the programm in eclipse eveyrthing is working well but when I go and run the .class from CMD it throw an error :


Exception in thread "main" java.lang.NoClassDefFoundError: readFiles (wrong name
: com/test/compare/readFiles)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$100(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Open in new window


I have added java path and class path as well in my system and still same error: I have try running it like bellow

java -classpath . classname
no luck and still it throws error, the program compiled without any error I try it both from CMD and from Eclipse it goes without any error but when running it from CMD it gives that error


Bellow is the final changes I did and it runs okay in eclipse but not in cmd

import java.io.File;
import java.util.Arrays;

public class readme {

	public static void main(String[] args) {
		
		
		File folder = new File("D:files");
		File[] listOfFiles = folder.listFiles();
		Arrays.sort(listOfFiles);

		long lastTime = 0;
		String lastName = null;
		for (int i = 0; i < listOfFiles.length; i++) {
		    File f = listOfFiles[i];
		   // System.out.println("List of Files" + f);
		    if (f.isFile()) {
			String[] atoms = f.getName().split("\\.");
			if(lastTime == 0) {
			    lastTime = Long.parseLong(atoms[3]);
			    lastName = f.getName();
			    continue;
			    
			}
			else {
			    long latestTime = Long.parseLong(atoms[2]);
			    long diff = latestTime - lastTime;
			    if (diff != 0) {
				System.out.printf("Gap of %d Seconds found between predecessor file %s and current file %s\n", diff, lastName, f.getName());
			    }
			    lastTime = Long.parseLong(atoms[3]);
			    lastName = f.getName();
			}
			System.out.println(f);
//		    } else if (listOfFiles[i].isDirectory()) {
//			System.out.println("Directory " + listOfFiles[i].getName());
		    }
		}
	}
}

Open in new window

0
 
LVL 1

Author Comment

by:hi4ppl
ID: 38812889
Hi fix it never mind
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 38813966
:)
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
This article discusses four methods for overlaying images in a container on a web page
This video teaches viewers about errors in exception handling.
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now