Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 790
  • Last Modified:

Automating using SSIS (DTXS files) to import FLAT FILES

I have 4 flat files which I am receiving updates for on daily basis.  I have currently used SSIS to import these files into SQL Server 2005 and have created dtxs files for each one.  

Questions:

Is it possible after the first run of the configed dtxs files (each import a flat file to a SQL Server 2005 DB) to automate this process so that imports can occur on a daily basis even if columns are removed/added to future flat files?  If so please explain how this is done (In detail, pictures would be nice).

Just to clarify my vendor has told me that they will remove or added different columns to these flat files at any time.  I do not want to recreate my orginal dtxs files, each time this occurs.
0
RunnerNwizard
Asked:
RunnerNwizard
2 Solutions
 
fesnyngCommented:
Besides getting another vendor?  

Data structure reflects the business structure.   If it is changing every data load, then the business is changing.  And, unfortunately, you will need to change your ETL each time the data structure changes.

What I have done in the face of changing data is to make the import as simple as possible -- i.e. use the "Import Data..." task ( in SSMS, right click the data base, select Tasks, Select Import Data ...).  Let SSIS figure out what the columns are.  Then use stored procedures (which will be modified each time the vendor changes the data structure) to extract and transform the data.  This is more along the lines of ELT (Extract Load Transform) but is "the least effort" that I have found in the face of constantly changing data structure.  

Not the answer you were looking for, I know.   Hopefully, I am wrong and there is a better solution among the experts here.
0
 
nmcdermaidCommented:
Well you could probably automate adding a new column to a staging table and adding a new column to the connection manager but I agree with fesnyq: its more fundamental than that - where does the data in your staging table go after a new column is added??? You certainly can't automate that.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now