Validate Duplicates While Performing a Bulk Insert

Posted on 2007-08-09
Medium Priority
Last Modified: 2013-11-05
I have a flat table that i use to stage CSV Lead Imports in.  It has four fields: home,work ,cell,alt phone.  Home will always have a number in it. Cell, Work and Alt could be 0.  I'd like to be able to bulk insert this data, but not insert rows from this table where ANY of these four fields matches to a phone number i have in my PhoneNumber table AND don't match to any other record in my staging table.  

For instance, if Lead A has a home of 5555555555 and B has a Cell of 5555555555, then only A would go in, etc, etc, etc.  

I really don't want to use a cursor, but I'm thinking that the code I would need to write to only select a distinct lead (based on those numbers and considering some could be 0 for many rows) from this table could get ugly.  If i cursor, then lead A will be in the phone number table by the time I get to lead B.  HOWEVER, I was wondering if these is any way to validate as i'm doing the bulk insert.  Probably Not.  I'm taking suggestions.
Question by:davidcahan
  • 2

Expert Comment

ID: 19687473
You could have the package execute a stored procedure to filter the data in the table before you do you transfer into your storage table.

Expert Comment

ID: 19687495
You can find more information via this link: http://www.sqlis.com/58.aspx

Author Comment

ID: 19687939
I figured it out...i was just hoping to use to some really really dynamic code.  Instead, i just filter out dupes down each column between rows.  Then i filter between columns for each row.  Then I'm good to go.

Accepted Solution

Vee_Mod earned 0 total points
ID: 19729538
Closed, 500 points refunded.
Community Support Moderator

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I've encountered valid database schemas that do not have a primary key.  For example, I use LogParser from Microsoft to push IIS logs into a SQL database table for processing and analysis.  However, occasionally due to user error or a scheduled task…
In this article I will describe the Backup & Restore method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
We’ve all felt that sense of false security before—locking down external access to a database or component and feeling like we’ve done all we need to do to secure company data. But that feeling is fleeting. Attacks these days can happen in many w…
Please read the paragraph below before following the instructions in the video — there are important caveats in the paragraph that I did not mention in the video. If your PaperPort 12 or PaperPort 14 is failing to start, or crashing, or hanging, …

807 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question