Best performance duplication prevention a collection of objects?

Posted on 2005-04-20
Medium Priority
Last Modified: 2010-03-31
I've created an application which scans through a log file line by line and creates an object representing each line... I.e the object will contain date strings line number etc...

 I wish to prevent duplicates of a certain type. i.e duplicate objects would have the same date but different string and line numbers....

I've been using a ArrayList of objects and each time before I add an object I check if the object is in the arraylist already.. I've overloaded the equals() function and I'm using the arraylist.contains(object) to check for the duplicate...

My problem is that the arraylist can have up to 60,000 elements, so I have to check for duplicates each time before I add an element..

Anyone know of a more efficient way of preventing duplicates because this is realy inefficient.
Question by:conorocallaghan
  • 3
  • 2
LVL 86

Expert Comment

ID: 13826239
You can use a Set. The Set cannot contain duplicates
LVL 15

Expert Comment

ID: 13826252
Yes, as mentioned above, HashSet would be your best bet.
HashSet set = new HashSet();

if (set.contains(key))
// skip this
continue logic...
LVL 86

Accepted Solution

CEHJ earned 1000 total points
ID: 13826299
>>if (set.contains(key))

Is not necessary to prevent duplicates - they won't occur anyway. It would only be necessary if you wanted to know 'key' is going to be skipped
LVL 15

Expert Comment

ID: 13826416
I assume that set is not pre populated but get populated as it goes thru the log file.
And in that case an action might be taken when line already exists... (and not necessarily override the old line with the new one).
LVL 86

Expert Comment

ID: 13925901
:-) but why a 'C'?

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Java can be integrated with native programs using an interface called JNI(Java Native Interface). Native programs are programs which can directly run on the processor. JNI is simply a naming and calling convention so that the JVM (Java…
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
Suggested Courses
Course of the Month9 days, 20 hours left to enroll

569 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question