Validate Duplicates While Performing a Bulk Insert

Posted on 2007-08-09
Last Modified: 2013-11-05
I have a flat table that i use to stage CSV Lead Imports in.  It has four fields: home,work ,cell,alt phone.  Home will always have a number in it. Cell, Work and Alt could be 0.  I'd like to be able to bulk insert this data, but not insert rows from this table where ANY of these four fields matches to a phone number i have in my PhoneNumber table AND don't match to any other record in my staging table.  

For instance, if Lead A has a home of 5555555555 and B has a Cell of 5555555555, then only A would go in, etc, etc, etc.  

I really don't want to use a cursor, but I'm thinking that the code I would need to write to only select a distinct lead (based on those numbers and considering some could be 0 for many rows) from this table could get ugly.  If i cursor, then lead A will be in the phone number table by the time I get to lead B.  HOWEVER, I was wondering if these is any way to validate as i'm doing the bulk insert.  Probably Not.  I'm taking suggestions.
Question by:davidcahan
    LVL 5

    Expert Comment

    You could have the package execute a stored procedure to filter the data in the table before you do you transfer into your storage table.
    LVL 5

    Expert Comment

    You can find more information via this link:

    Author Comment

    I figured it out...i was just hoping to use to some really really dynamic code.  Instead, i just filter out dupes down each column between rows.  Then i filter between columns for each row.  Then I'm good to go.
    LVL 1

    Accepted Solution

    Closed, 500 points refunded.
    Community Support Moderator

    Featured Post

    Enabling OSINT in Activity Based Intelligence

    Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

    Join & Write a Comment

    Recently, when I was asked to create a new SQL 2005 cluster, Microsoft released a new service pack for MS SQL 2005 what is Service Pack 3. When I finished the installation of MS SQL 2005 I found myself troubled why the installation of SP3 failed …
    INTRODUCTION: While tying your database objects into builds and your enterprise source control system takes a third-party product (like Visual Studio Database Edition or Red-Gate's SQL Source Control), you can achieve some protection using a sing…
    Hi everyone! This is Experts Exchange customer support.  This quick video will show you how to change your primary email address.  If you have any questions, then please Write a Comment below!
    This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor ( If you're looking for how to monitor bandwidth using netflow or packet s…

    728 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    21 Experts available now in Live!

    Get 1:1 Help Now