Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

How can I use parameters in a connection file manager and source in ssis 2015 to load 2 different file definitions

Posted on 2016-10-31
16
Medium Priority
?
89 Views
Last Modified: 2016-11-04
How can I use parameters in a connection file manager and source in ssis 2015 to load 2 different file definitions with variable names from the same directory into sql server.

I want to define connection managers, parameter(s), file sources, and for each loop to load 2 different .txt file definitions into sql server.

the 2 different defintions are a seperate files named with a date time stamp at the trailing end of the file name:
def1 = posdef<dtstamp>.txt
def2 = posdef2<dtstatmp>.txt

both files are loaded to the same directory periodically.
I don't understand how to implement a combination(s) of params to account for varaible file names and different definitions to load using for each loop(s) to load into different tables in an ssis package
0
Comment
Question by:conardb
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 3
  • 3
16 Comments
 
LVL 32

Expert Comment

by:Pawan Kumar
ID: 41867907
Well in this case you have to use string Variable say FileName & Flat File Connection Manager. Below Blog URL will help you in step by step -

http://beyondrelational.com/modules/2/blogs/106/posts/11113/ssis-dynamically-set-flat-file-connection-manager.aspx

You just have to follow these simple steps.
0
 

Author Comment

by:conardb
ID: 41868243
Thanks Pawan, I don't think that will work.  The two different definitions will be in two different files.  Each will have a different name.  The difference will be a time stamp when the file was generated so the name can't be stored in a table and they will have the same unique leading characters for each file def.  file1<timestamp>.txt. filename2<timestamp>.txt.

I'm thinking I need to be able to do a for reach loop through directory that has a seperate connection to each of the two file def by unique leading characters.
0
 

Author Comment

by:conardb
ID: 41868252
I was thinking use the fully qualified path+file name and process the files based on a substring of the path that uses the leading file name characters
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 32

Accepted Solution

by:
Pawan Kumar earned 1000 total points
ID: 41868253
Okies, if you want to go head with foreach loop and dynamic connections below can help.

http://www.sqlerudition.com/example-of-ssis-foreach-loop-item-enumerator-and-dynamic-oledb-connection/
0
 

Author Comment

by:conardb
ID: 41868281
Thanks, this looks to take table data in a database and write to text files.  I'm need to take text files from a directory and load into sql server table and I need to determine the connection or source in the ssis package by the leading characters of the text file name.
0
 
LVL 52

Assisted Solution

by:Vitor Montalvão
Vitor Montalvão earned 1000 total points
ID: 41868288
I'm not a SSIS expert but already created some packages before.
Are you using a Flat File Connection Manager? If so you can set a variable that will produce dynamically the file name by using T-SQL only in the expression. Example:
'File1'+YEAR(GETDATE())+MONTH(GETDATE())+DAY(GETDATE())+'.txt'
0
 

Author Comment

by:conardb
ID: 41868452
Yes, I'm using the flat file connection manager.  The flat files are the source, so I have the connection manager, for each loop container, dataflow, source in the data flow.  There may be 10 files for example with two file defintion / file types.  Each file def/type is indicated the the prefix or starting char of the file name (file1<timestamp>.txt, file2<timestame>.txt)   The record length and columns are different for each file def
0
 
LVL 52

Expert Comment

by:Vitor Montalvão
ID: 41868469
Good. Do you have any variable set?
For your case you'll need to have a variable for each file.
In the example I gave above you'll keep the file name as File1YYYYMMDD.txt so you can work from there.
0
 

Author Comment

by:conardb
ID: 41868757
what are the settings for the connection manager and source.  That's where I keep having issues is with the connection mgr and source specifying the record length
0
 
LVL 52

Expert Comment

by:Vitor Montalvão
ID: 41869703
what are the settings for the connection manager and source.
What do you mean? Can you show the settings that you're having now?
0
 

Author Comment

by:conardb
ID: 41873876
I have a connection manager and flat file source for each file definition.
A-connection-manager-for-each.docx
0
 

Author Comment

by:conardb
ID: 41873905
here's the loop settings and var settings
ForEachLoop-Properties.docx
0
 

Author Comment

by:conardb
ID: 41873915
cannot find specified file
cannot-find-file.docx
0
 

Author Comment

by:conardb
ID: 41874306
I think I got it now...
0
 
LVL 32

Expert Comment

by:Pawan Kumar
ID: 41874367
Great..
0
 

Author Closing Comment

by:conardb
ID: 41874447
used a single var for folder path, 2 vars for each file def: one for filename and one for folder path and file combination for a total of 5 vars and 2 for each loops... initially, I had tried to use the multi file connection in an attempt to avoid multiple connections and variables per file def, I'd be interested in knowing if there is someway to use one connection manager for multiple sources or would a connection manager and source always be required for each file def.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
A couple of weeks ago, my client requested me to implement a SSIS package that allows them to download their files from a FTP server and archives them. Microsoft SSIS is the powerful tool which allows us to proceed multiple files at same time even w…
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question