Solved

Automating using SSIS (DTXS files) to import FLAT FILES

Posted on 2008-06-16
4
782 Views
Last Modified: 2013-11-30
I have 4 flat files which I am receiving updates for on daily basis.  I have currently used SSIS to import these files into SQL Server 2005 and have created dtxs files for each one.  

Questions:

Is it possible after the first run of the configed dtxs files (each import a flat file to a SQL Server 2005 DB) to automate this process so that imports can occur on a daily basis even if columns are removed/added to future flat files?  If so please explain how this is done (In detail, pictures would be nice).

Just to clarify my vendor has told me that they will remove or added different columns to these flat files at any time.  I do not want to recreate my orginal dtxs files, each time this occurs.
0
Comment
Question by:RunnerNwizard
4 Comments
 
LVL 5

Accepted Solution

by:
fesnyng earned 250 total points
ID: 21801487
Besides getting another vendor?  

Data structure reflects the business structure.   If it is changing every data load, then the business is changing.  And, unfortunately, you will need to change your ETL each time the data structure changes.

What I have done in the face of changing data is to make the import as simple as possible -- i.e. use the "Import Data..." task ( in SSMS, right click the data base, select Tasks, Select Import Data ...).  Let SSIS figure out what the columns are.  Then use stored procedures (which will be modified each time the vendor changes the data structure) to extract and transform the data.  This is more along the lines of ELT (Extract Load Transform) but is "the least effort" that I have found in the face of constantly changing data structure.  

Not the answer you were looking for, I know.   Hopefully, I am wrong and there is a better solution among the experts here.
0
 
LVL 30

Assisted Solution

by:nmcdermaid
nmcdermaid earned 250 total points
ID: 21862581
Well you could probably automate adding a new column to a staging table and adding a new column to the connection manager but I agree with fesnyq: its more fundamental than that - where does the data in your staging table go after a new column is added??? You certainly can't automate that.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article I will describe the Copy Database Wizard method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Via a live example, show how to extract information from SQL Server on Database, Connection and Server properties
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now