Solved

Data Ware House - Daily CSV Extracts into Microsoft SQL 2008 R2 Server

Posted on 2014-03-07
3
238 Views
Last Modified: 2016-02-10
Hi,
I've recently been give a task to import daily change files into an Microsoft SQL Server. There are 156 files/tables that are Extracted & downloaded on a daily base that need to be bulk insert into an SQL database. What is the best way to approach this?

There are a few tools that I've been looking at, which are SSIS and BCP.
0
Comment
Question by:johnojohno
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 45

Expert Comment

by:aikimark
ID: 39914134
You should include Powershell as another viable tool.  Like BCP, it is a 'push' model.  SSIS would be the pull model.
0
 

Author Comment

by:johnojohno
ID: 39925878
Thanks aikmark I'll check it out. I've started to use SSIS to import the data. I've created a foreach loop container for each of my table extracts, which are daily CSV files.
Unfortunately non of the data extracts have a record creation date so I need to come up with a way to understand which records are the most up to date.  

Does anyone have any ideals how I can do this and remove the duplicate records of about 117 tables on a daily bases?
0
 
LVL 45

Accepted Solution

by:
aikimark earned 500 total points
ID: 39926415
Files have both a creation date and a modified date properties.  Why couldn't you use that date?
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why is this different from all of the other step by step guides?  Because I make a living as a DBA and not as a writer and I lived through this experience. Defining the name: When I talk to people they say different names on this subject stuff l…
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

737 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question