?
Solved

SSIS, Source is text file with matrixed data, Destination needs to be flat

Posted on 2014-01-13
5
Medium Priority
?
368 Views
Last Modified: 2016-02-10
Hi guys

I have a text file with 'matrixed' data like this:
10/11/13      22     420     131             810     813     440
     555  255.21  255.76  257.74                                        *
     559  296.21  296.76  298.74                                        *
     551
     560
     562
     550
     660
     650  300.23  300.78  302.87                                        *
     640  301.13  301.68  303.77                                        *
     620  309.30  309.85  311.94                                        *
     600
     595
     810                                  110.30  110.30                *

Open in new window

I need to write a data flow task that takes this 'matrixed' data and does...
Delete all rows where there are no values in the 2nd and 3rd column
Delete all columns after the 3rd column
Writes what's left to a flat table like this..
555         22       255.21
555        420       255.76
559         22       296.21 
559        420       296.76
650         22       300.23
650        420       300.78
640         22       301.13
640        420       301.68
620         22       309.30
620        420       309.85

Open in new window

Points for any useful design ideas on how to pull this off.
SSIS 2008 R2 + Task Factory, can also write in 2012 if needed.

Thanks in advance.
Jim
0
Comment
Question by:Jim Horn
  • 2
  • 2
5 Comments
 
LVL 84

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 200 total points
ID: 39777224
Silly question... how many rows?  Is it a repetitive task that needs to be programmed?
0
 
LVL 66

Author Comment

by:Jim Horn
ID: 39777250
>Silly question... how many rows?
Couple hundred.  It's a pricing file for commodities by location.

>Is it a repetitive task that needs to be programmed?
Yes, it will be scheduled to execute daily at oh-dark-thirty.
0
 
LVL 84

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 200 total points
ID: 39777297
What programming language does it need to be?  It looks like it wouldn't be too hard in command line PHP.  Do you have a sample file (other than what you posted because what you posted may be reformatted)?
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 400 total points
ID: 39785660
Hey Jim,

I assume you'd prefer to get this done using SSIS, right?  Did you try out the Flat File Source with Fixed Width format?  AFAIK that should work okay-ish.

That first line looks like a header, if you don't need that you'll probably want to skip it because the FF source can only handle one line format.  That's done through the Header rows to skip setting.

If you do need it then you might want to investigate into using the Script Component as source. Or use a separate data flow (I guess two FF sources using the same file in the same data flow won't work but haven't tried it out to know for sure).

VV.
0
 
LVL 66

Author Comment

by:Jim Horn
ID: 39912800
The ultimate solution ended up being to import the flat file as a source, and lots of derived columns to slice-dice.  

The gig ended, and I can't elaborate more on that, so I'll lower and split points for helpful advise.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A couple of weeks ago, my client requested me to implement a SSIS package that allows them to download their files from a FTP server and archives them. Microsoft SSIS is the powerful tool which allows us to proceed multiple files at same time even w…
Recently we ran in to an issue while running some SQL jobs where we were trying to process the cubes.  We got an error saying failure stating 'NT SERVICE\SQLSERVERAGENT does not have access to Analysis Services. So this is a way to automate that wit…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question