Solved

Import Flat File in SSIS

Posted on 2011-02-15
11
1,031 Views
Last Modified: 2012-05-11
My flat file is ragged right with no column headings but has multiple headers, detail, footers.
header
detail
footer

header
detail
footer

I need to import the detail only and not the header/footers.  I tried to import all and later delete the header and footers but not all the records are getting into my sql table.  Does anyone know how this can be accomplished.  I am new to SSIS and do not know how to set up a conditional split since I've read that this may be a solution.  Appreciate any help ... ps this is urgent.
0
Comment
Question by:bar0822
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 3
  • 2
11 Comments
 
LVL 5

Expert Comment

by:jijeesh
ID: 34901410
Are your headr, details and footer info  repeating?  
Can you send a sample input file and what are you expect to extract.

0
 
LVL 40

Expert Comment

by:lcohan
ID: 34901423
If you can do it directly in SQL query the example below should help just keep in mind the path and file is relative on the SQL server box not the client where you run the query:

--Assumes:Usage : exec sp_readTextFile 'c:\autoexec.bat'
--**************************************

Create proc sp_readTextFile @filename sysname
as


    begin
    set nocount on
    Create table #tempfile (line varchar(8000))
    exec ('bulk insert #tempfile from "' + @filename + '"')
    select * from #tempfile
    drop table #tempfile
End
go

0
 

Author Comment

by:bar0822
ID: 34909821
here is a sample
0
Get free NFR key for Veeam Availability Suite 9.5

Veeam is happy to provide a free NFR license (1 year, 2 sockets) to all certified IT Pros. The license allows for the non-production use of Veeam Availability Suite v9.5 in your home lab, without any feature limitations. It works for both VMware and Hyper-V environments

 
LVL 5

Accepted Solution

by:
jijeesh earned 500 total points
ID: 34912877
Attached a sample dtsx package. (you need to rename the extension from .txt to .dtsx). Hope this will be useful.
Copy-of-Package.txt
0
 

Author Comment

by:bar0822
ID: 34928722
I will try it and let you know. thanks
0
 

Author Comment

by:bar0822
ID: 34928809
Can you send me the solution file so I can take a look at it?
0
 

Author Comment

by:bar0822
ID: 34928842
I am running this from SSIS - how do I use your proc sp_readTextFile @filename sysname.

How does that omit the header and footer lines from the text file?
0
 
LVL 40

Expert Comment

by:lcohan
ID: 34929640
SQLCMD has many switches and I only gave an exmaple for you. you should look for more details about SQLCMD at http://msdn.microsoft.com/en-us/library/ms162773.aspx and use the switches you need for your purpose like:

-hheaders
Specifies the number of rows to print between the column headings. The default is to print headings one time for each set of query results. This option sets the sqlcmd scripting variable SQLCMDHEADERS. Use -1 to specify that headers must not be printed. Any value that is not valid causes sqlcmd to generate an error message and then exit.

-scol_separator
Specifies the column-separator character. The default is a blank space. This option sets the sqlcmd scripting variable SQLCMDCOLSEP. To use characters that have special meaning to the operating system such as the ampersand (&), or semicolon (;), enclose the character in quotation marks ("). The column separator can be any 8-bit character.

0
 

Author Comment

by:bar0822
ID: 34932963
thank you so much for all your time and help - I did not know this existed but there will def be a time when I will need to use.  sorry, I did not explain myself correctly - The text file does not have column headers - it is a flat ragged right file - I was looking to extract information about the data in the flat file e.g.
Header000 20110101
Detail information here
Trailer000 4,222 records
Header0000 20110101
Detail information here
Trailer000 3,111 records
I need to extract only the Detail information and send to SQL table through a SQL Execute Task, need to add a count to those records.  I was thinking of a conditional split - extract header/trailer data and send to a file, then count the rows in detail.  I am new to ssis and not sure how to write this.
0
 

Author Comment

by:bar0822
ID: 35097615
here is new file posted revised
Revised-Text-Document.txt
0
 
LVL 40

Expert Comment

by:lcohan
ID: 35097705
--Ok so let's leave the SP for now and just try in SQL to get what you need

Create table #tempfile (line varchar(8000))
exec ('bulk insert #tempfile from "' + 'C:\FolderName\FileName.txt' + '"')
select * from #tempfile where line NOT LIKE 'Header%' or line NOT LIKE 'Trailer%'
drop table #tempfile


if this is what you want then you can put the code in a SSIS T-SQL step and add a INSERT statement into your DB table from the temp table. You obviously can adjust the datatype,number of columns ETC now is all in SQL and easy to work with.
0

Featured Post

Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The new Microsoft OS looks great, is easier than ever to upgrade to, it is even free.  So what's the catch?  If you don't change the privacy settings, Microsoft will, in accordance with the (EULA) you clicked okay to without reading, collect all the…
Whether you've completed a degree in computer sciences or you're a self-taught programmer, writing your first lines of code in the real world is always a challenge. Here are some of the most common pitfalls for new programmers.
The viewer will learn how to simulate a series of sales calls dependent on a single skill level and learn how to simulate a series of sales calls dependent on two skill levels. Simulating Independent Sales Calls: Enter .75 into cell C2 – “skill leve…
The view will learn how to download and install SIMTOOLS and FORMLIST into Excel, how to use SIMTOOLS to generate a Monte Carlo simulation of 30 sales calls, and how to calculate the conditional probability based on the results of the Monte Carlo …

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question