[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

SSIS import multiple CSVs into associated tables

Posted on 2016-11-22
3
Medium Priority
?
112 Views
Last Modified: 2016-11-23
Hi

We currently have 3 files that are uploaded through SFTP to us on a daily bases.  Filename examples:

 - Invoices_20161122.csv
 - Agencies_20161122.csv
-  Sales_20161122.csv

The file names will change the date stamp every day when they are uploaded to the SFTP site

The files need to be uploaded in their associated tables

- Invoices
- Agencies
- Sales

The tables would need to be dropped and recreated when the files are imported.  The first row in the files contains the column names.

How can I create an SSIS package to:

- Drop the tables
- Recreate the tables when importing the CSV file and name them appropriately
- Move the old files an archive folder


Thank you for all of your help
0
Comment
Question by:thomasm1948
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 32

Expert Comment

by:Pawan Kumar
ID: 41898350
Have you tried anything? With which part you need help ?
0
 
LVL 13

Accepted Solution

by:
Arifhusen Ansari earned 2000 total points
ID: 41898541
Hi,

Based on your explanation. I can give you below solution.
You can create a single package to achieve this. But rather i would suggest, to create three packages. So that it would be easy to maintain.

1) Rather than drop a table and recreate the same. you can truncate that table.

For this you can use Execute script task and in that write a script to truncate table.

2) If you only have one file per type per day. You can use Data flow task directly to load the file in the table.

  e.g Invoices_20161122.csv will be only one for that day.

3) You have to use Flat File Connection as your file is of type csv.
4) when you create the package initially use the static connection to csv file. Develop the package if it's loaded perfectly.
5) You need to create the dynamic connection so that from next time it will automatically pick a file and load it.

For this you have to declare 3 variable.
1) FolderPath: path for your folder.
e.g. "E:\\FlatFileLoadOrCSV\\
2) FileName: this will be generated based on expression as we have to dynamically pick a file.
e.g. Expression would be like:  "Invoices_"+(DT_WSTR,4)  year(getdate())+(DT_WSTR,2)  month(getdate())+(DT_WSTR,2)  day(getdate())+".csv"

Evaluated value: Invoices_20161123.csv

3) FilePath: This is exact file path from where package will pick a file.
e.g. Expression is @[User::FolderPath]+@[User::FileName]

Evaluated value: E:\FlatFileLoadOrCSV\Invoices_20161123.csv

4) After this you just need to configure you connection manager to pick a file dynamically.

Click on File Connection go to property. Go to expression option click on ellipsis button and set the connection string property.

Refer below screenshot.

2016-11-23_10-30-05.png

Hope it will help you.

I have attached the ssis package as well. I have changed the extension to .txt. Download the file and just change it to .dtsx.

Hope it will help you.
FlatFileLoadOrCSV.txt
0
 

Author Closing Comment

by:thomasm1948
ID: 41899069
Thank this works out perfectly
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
Ready to get certified? Check out some courses that help you prepare for third-party exams.
Familiarize people with the process of retrieving data from SQL Server using an Access pass-thru query. Microsoft Access is a very powerful client/server development tool. One of the ways that you can retrieve data from a SQL Server is by using a pa…
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed

656 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question