Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

SSIS, Source is text file with matrixed data, Destination needs to be flat

Posted on 2014-01-13
5
348 Views
Last Modified: 2016-02-10
Hi guys

I have a text file with 'matrixed' data like this:
10/11/13      22     420     131             810     813     440
     555  255.21  255.76  257.74                                        *
     559  296.21  296.76  298.74                                        *
     551
     560
     562
     550
     660
     650  300.23  300.78  302.87                                        *
     640  301.13  301.68  303.77                                        *
     620  309.30  309.85  311.94                                        *
     600
     595
     810                                  110.30  110.30                *

Open in new window

I need to write a data flow task that takes this 'matrixed' data and does...
Delete all rows where there are no values in the 2nd and 3rd column
Delete all columns after the 3rd column
Writes what's left to a flat table like this..
555         22       255.21
555        420       255.76
559         22       296.21 
559        420       296.76
650         22       300.23
650        420       300.78
640         22       301.13
640        420       301.68
620         22       309.30
620        420       309.85

Open in new window

Points for any useful design ideas on how to pull this off.
SSIS 2008 R2 + Task Factory, can also write in 2012 if needed.

Thanks in advance.
Jim
0
Comment
Question by:Jim Horn
  • 2
  • 2
5 Comments
 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 50 total points
ID: 39777224
Silly question... how many rows?  Is it a repetitive task that needs to be programmed?
0
 
LVL 65

Author Comment

by:Jim Horn
ID: 39777250
>Silly question... how many rows?
Couple hundred.  It's a pricing file for commodities by location.

>Is it a repetitive task that needs to be programmed?
Yes, it will be scheduled to execute daily at oh-dark-thirty.
0
 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 50 total points
ID: 39777297
What programming language does it need to be?  It looks like it wouldn't be too hard in command line PHP.  Do you have a sample file (other than what you posted because what you posted may be reformatted)?
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 100 total points
ID: 39785660
Hey Jim,

I assume you'd prefer to get this done using SSIS, right?  Did you try out the Flat File Source with Fixed Width format?  AFAIK that should work okay-ish.

That first line looks like a header, if you don't need that you'll probably want to skip it because the FF source can only handle one line format.  That's done through the Header rows to skip setting.

If you do need it then you might want to investigate into using the Script Component as source. Or use a separate data flow (I guess two FF sources using the same file in the same data flow won't work but haven't tried it out to know for sure).

VV.
0
 
LVL 65

Author Comment

by:Jim Horn
ID: 39912800
The ultimate solution ended up being to import the flat file as a source, and lots of derived columns to slice-dice.  

The gig ended, and I can't elaborate more on that, so I'll lower and split points for helpful advise.
0

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
T-SQL 10 33
SQL Activity Monitor detail 2 23
SSRS Deployment problem 5 63
MSSQL - Lock Row from reading by other programs 9 34
Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.
Viewers will learn how to use the SELECT statement in SQL to return specific rows and columns, with various degrees of sorting and limits in place.

792 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question