Handling unwanted records during bcp

I get a file which i will attempt to bcp into a table.  I got an error because there is a unique index.  I know it would probably be better to drop the index before copying, but i still want to know about the record(s) that are violating my integrity.  Do i need to load the table first, then do something like selecting a distinct or unique into another table or is there a way to check during the bcp copy process, identify the offending records and continue copying?
Who is Participating?
JHausmannConnect With a Mentor Commented:
Being new to this, I assume this is the question I answered. If not, sorry.
I don't know of a way to check in the bcp process. What I've done, in the past, is drop unique indexes, bcp the data in, check the fields for uniqueness and fix any problems, then re-index the table.

You can use the following select statement to identify any duplicate keys that might exist in your table (assuming the key is a single field):

Select keyfield, count(*) from table group by keyfield

Any field that has a count > 1 is suspect and needs to be investigated.
BCP is just a Bulk Copy, you don't get any control on individual records that get inserted...

For one of my project I had to do such a thing (importing dayly data from COBOL text files) I used temporary tables and a stored procedure to insert data from the temp table to 'normalize' data.

If you're using MSSQL V7, you might have a look at the Data Transformation Services, this is a nice feature that's more integrated in MSSQL and gives you more control on the whole process...

Hope this helps.

This question was awarded, but never cleared due to the JSP-500 errors of that time.  It was "stuck" against userID -1 versus the intended expert whom you awarded.  This corrects the problem and the expert will now receive these points; points verified.

Please click on your Member Profile and select "View Question History" to navigate through any open or locked questions you may have to update and finalize them.  If you are an EE Pro user, you can also choose Power Search to find all your open questions.

This is the Community Support link, if help is needed, along with the link to All Topics which reflects many TAs recently added.

Thank you,
Moderator @ Experts Exchange
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.