johnojohno
asked on
Data Ware House - Daily CSV Extracts into Microsoft SQL 2008 R2 Server
Hi,
I've recently been give a task to import daily change files into an Microsoft SQL Server. There are 156 files/tables that are Extracted & downloaded on a daily base that need to be bulk insert into an SQL database. What is the best way to approach this?
There are a few tools that I've been looking at, which are SSIS and BCP.
I've recently been give a task to import daily change files into an Microsoft SQL Server. There are 156 files/tables that are Extracted & downloaded on a daily base that need to be bulk insert into an SQL database. What is the best way to approach this?
There are a few tools that I've been looking at, which are SSIS and BCP.
You should include Powershell as another viable tool. Like BCP, it is a 'push' model. SSIS would be the pull model.
ASKER
Thanks aikmark I'll check it out. I've started to use SSIS to import the data. I've created a foreach loop container for each of my table extracts, which are daily CSV files.
Unfortunately non of the data extracts have a record creation date so I need to come up with a way to understand which records are the most up to date.
Does anyone have any ideals how I can do this and remove the duplicate records of about 117 tables on a daily bases?
Unfortunately non of the data extracts have a record creation date so I need to come up with a way to understand which records are the most up to date.
Does anyone have any ideals how I can do this and remove the duplicate records of about 117 tables on a daily bases?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.