using SSIS to import a CSV file to a table while 3 x field must be unique`

I see two possible conditions here:

1)  If there is a duplicate, no records are to be imported.

2)  If there is a duplicate, import only the first record (or the last record, depending on specification).


In either case, the issue is that you do not know if there are duplicates until the second duplicate record is loaded.  However, you don't want the duplicate data loaded.

In case (2), then importing with an error handler for duplicate rows should work fine.  The first insert works, but subsequent inserts of duplicate rows (based on the unique columns) fails.  The import continues.

In case (1), I would use a different process.  I would import into a copy of the destination table first (a staging table), then delete any duplicated records, then finally copy records from the staging table to the final destination.

it seems 2 is better, but does SSIS provide any tools for us that allow we specify the 3 x colume is unique that based on this, no row can be duplicate?

should I use select distinct column A, B and C on that target table using SSIS? anyway to do it ?

How about adding one more unique column in the target table, this column must not exist in ithe original csv file?

please close this question and have the point refund.