I have an ERP system with an ORD_HDR(about a million rows) table and an ORD_DTL(about 7 million rows) table. Initially there was an ODBC connection to the ERP system and it was a query on the ORD_HDR table with an INNER JOIN to the details table for the last 10 days with a Slowly Changing dimension to insert or update the table and then a bunch of Lookups and then loading a Fact table. This took over an hour and a half to run. So I loaded all the data into SQL tables and then want to use those tables to do the UPSERT. My initial problem is that I have to check the ERP data against my tables that are loaded already and do an Insert or Update if it exists already or not. I wanna do this the fastest way possible so Im open to suggestions. I have to query the ERP tables, match the records against the tables I loaded today and do an Insert if it doesnt exist and an Update if it does. I dont even beleive the Update is necessary but I guess it was necessary on the SCD I guess? Please help!!!
Have you tried a MERGE statement, which can perform inserts, updates, and deletes in a single statement?
Scoll down for examples. Assumes 2008 and higher.