Solved

SSIS, Source is text file with matrixed data, Destination needs to be flat

Posted on 2014-01-13
5
347 Views
Last Modified: 2016-02-10
Hi guys

I have a text file with 'matrixed' data like this:
10/11/13      22     420     131             810     813     440
     555  255.21  255.76  257.74                                        *
     559  296.21  296.76  298.74                                        *
     551
     560
     562
     550
     660
     650  300.23  300.78  302.87                                        *
     640  301.13  301.68  303.77                                        *
     620  309.30  309.85  311.94                                        *
     600
     595
     810                                  110.30  110.30                *

Open in new window

I need to write a data flow task that takes this 'matrixed' data and does...
Delete all rows where there are no values in the 2nd and 3rd column
Delete all columns after the 3rd column
Writes what's left to a flat table like this..
555         22       255.21
555        420       255.76
559         22       296.21 
559        420       296.76
650         22       300.23
650        420       300.78
640         22       301.13
640        420       301.68
620         22       309.30
620        420       309.85

Open in new window

Points for any useful design ideas on how to pull this off.
SSIS 2008 R2 + Task Factory, can also write in 2012 if needed.

Thanks in advance.
Jim
0
Comment
Question by:Jim Horn
  • 2
  • 2
5 Comments
 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 50 total points
ID: 39777224
Silly question... how many rows?  Is it a repetitive task that needs to be programmed?
0
 
LVL 65

Author Comment

by:Jim Horn
ID: 39777250
>Silly question... how many rows?
Couple hundred.  It's a pricing file for commodities by location.

>Is it a repetitive task that needs to be programmed?
Yes, it will be scheduled to execute daily at oh-dark-thirty.
0
 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 50 total points
ID: 39777297
What programming language does it need to be?  It looks like it wouldn't be too hard in command line PHP.  Do you have a sample file (other than what you posted because what you posted may be reformatted)?
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 100 total points
ID: 39785660
Hey Jim,

I assume you'd prefer to get this done using SSIS, right?  Did you try out the Flat File Source with Fixed Width format?  AFAIK that should work okay-ish.

That first line looks like a header, if you don't need that you'll probably want to skip it because the FF source can only handle one line format.  That's done through the Header rows to skip setting.

If you do need it then you might want to investigate into using the Script Component as source. Or use a separate data flow (I guess two FF sources using the same file in the same data flow won't work but haven't tried it out to know for sure).

VV.
0
 
LVL 65

Author Comment

by:Jim Horn
ID: 39912800
The ultimate solution ended up being to import the flat file as a source, and lots of derived columns to slice-dice.  

The gig ended, and I can't elaborate more on that, so I'll lower and split points for helpful advise.
0

Featured Post

Ransomware: The New Cyber Threat & How to Stop It

This infographic explains ransomware, type of malware that blocks access to your files or your systems and holds them hostage until a ransom is paid. It also examines the different types of ransomware and explains what you can do to thwart this sinister online threat.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SQL Server Designer 19 45
T-SQL Default value in Select? 5 27
abs operation in sql server 15 15
delete the first occurence of a duplicate row in sql 5 26
In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question