We help IT Professionals succeed at work.

Flat file to Insert Update

Hi I have a flat file of 400,000 records.

I have an existing table that matches the flat file. This wil be the second time I read the data in. The first time the table was empty. This time the table is full and about 10% of the fields will be new, 10% will need to be update. How can I read through the flat file and decide if the key already exists. If it does I want to update the table if it does not I want to insert the data into the table.  I do not know ahead of time which frows will have changes data so if I need to do every record that exists and update it thats ok.

This is a once every quarter update speed is not an issue.
Comment
Watch Question

Database Consultant
BRONZE EXPERT
Top Expert 2009
Commented:
You can probably copy the data into a temptable first, then you can update the values which are there in the master table and insert the new values from the temptable using an 'EXISTS' clause
Guy Hengel [angelIII / a3]Billing Engineer
GOLD EXPERT
Most Valuable Expert 2014
Top Expert 2009
Commented:
I agree: load the flat file into a "staging" table, and from there (after creating eventual indexes), update the production table.
if you need help on how to formulate the update/insert queries accordingly, or how to load the file into a table (BULK INSERT), ask.
Charles BaldoSoftware Developer

Author

Commented:
Hi

I understand its difficult to know exactly what the issue is and whats in a programmers mind. But ocassional it happens. Below is a link of almost exactly my issue and it was solved. The steps suggested by the experts were used. Thank both I will split the points.

http://blogs.conchango.com/jamiethomson/archive/2006/09/12/SSIS_3A00_-Checking-if-a-row-exists-and-if-it-does_2C00_-has-it-changed.aspx
Charles BaldoSoftware Developer

Author

Commented:
Thank You all I did give an excellent grade because I appreciate the quick response. However the solution was only a partial.

Explore More ContentExplore courses, solutions, and other research materials related to this topic.