?
Solved

Data Ware House - Daily CSV Extracts into Microsoft SQL 2008 R2 Server

Posted on 2014-03-07
3
Medium Priority
?
240 Views
Last Modified: 2016-02-10
Hi,
I've recently been give a task to import daily change files into an Microsoft SQL Server. There are 156 files/tables that are Extracted & downloaded on a daily base that need to be bulk insert into an SQL database. What is the best way to approach this?

There are a few tools that I've been looking at, which are SSIS and BCP.
0
Comment
Question by:johnojohno
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 46

Expert Comment

by:aikimark
ID: 39914134
You should include Powershell as another viable tool.  Like BCP, it is a 'push' model.  SSIS would be the pull model.
0
 

Author Comment

by:johnojohno
ID: 39925878
Thanks aikmark I'll check it out. I've started to use SSIS to import the data. I've created a foreach loop container for each of my table extracts, which are daily CSV files.
Unfortunately non of the data extracts have a record creation date so I need to come up with a way to understand which records are the most up to date.  

Does anyone have any ideals how I can do this and remove the duplicate records of about 117 tables on a daily bases?
0
 
LVL 46

Accepted Solution

by:
aikimark earned 1500 total points
ID: 39926415
Files have both a creation date and a modified date properties.  Why couldn't you use that date?
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Suggested Courses

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question