Import Database into existing database, without the duplicates!

My working database contains nearly 300,000 records and one of my clients gave me their database of 40,000 records, but I know that there are many, many duplicates. I got great help the last time I did this and was able to fix a duplicate problem AFTER I imported the data. This time, I would like to get rid of the duplicates before I import them. First Name, Last Name and Company could be a good way to compare the records in both databases, but many of these people have left their companies, or I have the old information. For now, I would just like to see which names in my clients database DO NOT exist in my database and import them. Of course, this will not help with the Mikes and Michaels, or the Susans and Sues, or the misspelled names, but it's a start...

Any suggestions?

Thanks,
John
johnmoedAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

lesouefCommented:
create a calculated field in both data bases called "check" for instance.
check = name&firstname&company (and any other if you'd like to improve dup detection.

make a link in between the bases using this field

create a calculated field, say 'link_true' in the small database (the 40k lines one) = if (database1::check ; "ok" ; "")
search all records in small database where link_true="ok"
go to main database, import the found set you just did from the small database.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
johnmoedAuthor Commented:
I tried a couple different relationships and was able to delete thousands of exact duplicates. E-mail address is a really good one to use for this and I also used direct dial number.

Thanks again!
John
0
lesouefCommented:
the more fields you use, the more accurate the dups will be!
the real problem is to take into account trailing spaces, phone nbs with different formats, so it is also a good idea to remove all double, leading and trailing spaces from all text fields and keep the deleted ones in a corner for a while just in case.
till next, thanks for the points, at least 3 weeks I had not got anything, and bill is chasing me!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
FileMaker Pro

From novice to tech pro — start learning today.