Algorithm of files comparison

Posted on 2001-08-06
Last Modified: 2008-02-26
I am now writing a java program which compares 10 files for producing the difference, so that the user can change the difference easily. This is similar to a software WinMerge which can compare 2 files only. I want to ask whether there is any similar algorithm that some people have met and found some in web.
Moreover, this program needs the comparison of different language, it seems that different language should have different algorithm.?At this time, I need to support the English and Chinese (in unicode). However, I want to know whether I need to implement in 2 algorithm for these 2 languages. How about the other?

Moreover, if there is Chinese and English at the same time in the file, how to deal with the comparison?

Question by:fungho
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 10

Expert Comment

ID: 6355379

Author Comment

ID: 6355968

Expert Comment

ID: 6358338
rajesh doesn't know the answer, but is interested in the answer...

Rajesh, this is not neccessary, just press on the subscribe button to receive e-mail notifications.
Salesforce Made Easy to Use

On-screen guidance at the moment of need enables you & your employees to focus on the core, you can now boost your adoption rates swiftly and simply with one easy tool.


Expert Comment

ID: 6358346
And, fungho, as for the different languages in Unicode, there's not really a problem. Each "letter" or better said, a "symbol" is represented by a char value. Just to the comparison on char-values. This way, the Latin letter "a" will be recognized being different from a Chinese symbol.

Say, you have two files:

--- file one ---
Hello, World!
--- file one ---


--- file two ---
Hello, <Chinese Symbol for Chinese>World!
--- file two ---

then the symbol in front of the big Latin letter 'W' (the chinese symbol for "Chinese") is treated as a single "letter". So, this way, the second file just differs in the additional letter <Chinese Symbol for Chinese>.

Expert Comment

ID: 6358494
1. what exactly are you trying to achive?
2. why different algorithns for different languages?

Author Comment

ID: 6367421
It needs different algorithm is that for English, there is space between each word, however, for the Chinese, there is no space. So I need to cut the Chinese word by characters.

Accepted Solution

dnoelpp earned 200 total points
ID: 6367610
Now I understand your problem a little better. Do you want to do sort of an overview: number of each word, so say in a text the word "I" is counted 4 times, "a" 5 times, "times" 3 times, etc, do you understand? To achieve this, we can use a sorted hashmap for the words, but that's not the problem. You want to know how to extract words out of the text and consider the differences in punctuation in Chinese and English.

There's the class Character.UnicodeBlock. One of the instances is CJK_UNIFIED_IDEOGRAPHS. I think that this is the instance for all Chinese symbols.

So, to find out whether a character is a Chinese symbol, use this:

Character.UnicodeBlock.of(c) == Character.UnicodeBlock.CJK_UNIFIED_IDEOGRAPHS.

When a character is chinese, just dump it in the hashmap for counting.

Then, to find out whether the character is a Latin letter, use this:

Character.UnicodeBlock.of(c) == Character.UnicodeBlock.BASIC_LATIN.

I am not sure, however, whether BASIC_LATIN covers accented letters like a e i ? u c, that's up to you to find out.

When you have a latin letter, it can be a part of a word. How to find out? I suggest to partition the Unicode character set into three parts: 1. Chinese symbols 2. Latin letters 3. None of the first two.


Read the file character-wise. Start with an empty word (for English). For each character do:

1. Chinese symbol? Dump it into the hash map!
2. Letter? Add it to the word.
3. None of the two? Is the word emtpy, don't do anything. Else dump the word to the hash map and make it empty.


Author Comment

ID: 6368652
Thanks! dnoelpp! In fact, I have solved the problem by myself. But I check the character whether it is english or chinese by using this method: ( < "\u00FF")... I will use your method tomorrow. Besides I have a question also, if there is a language which starts from right to left, at the same time, there is number and english embeded in it, so some words needs to be read from left to right? how to deal with this? This does not mind whether you know or not... Thanks!

Expert Comment

ID: 6369332
That's a problem for Swing. Some Swing components, say, JLabel have to cope with it. Imagine an editor where you edit arabic and english text at the same time. What happens if you move the cursor from the english part to the arabic part.

Strings or Unicode sequence, however, they follow the principle that the "reading" order is important, not the placing. An example. Two words, "english" and "arabic" are in a string, and "arabic" is printed from right to left. Then the string would be displayed as such:

english cibara

But the string sequence is as follows: 'e' 'n' 'g' 'l' 'i' 's' 'h' ' ' 'a' 'r' 'a' 'b' 'i' 'c'

Author Comment

ID: 6370886
Then does the java swing deal with the situation that the arabic comes from right to left? or I need to deal with it?

Expert Comment

ID: 6371280
Swing does. Please read this article:

The keyword is "bidi" (short for bi-directional text)

By the way, this could be interesting for you (collating and sorting in other languages than English):

Author Comment

ID: 6371417
Thanks for your help! I will read it later!

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
web application structure 18 130
spring jars download 1 67
ejb entity bean example issue 2 40
Github api to fetch the number of collaborators in a git repo 1 66
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
Viewers will learn about the different types of variables in Java and how to declare them. Decide the type of variable desired: Put the keyword corresponding to the type of variable in front of the variable name: Use the equal sign to assign a v…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.
Suggested Courses

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question