importing data using SSIS

Hi there,
I am looking to import some files using SSIS 2005 -
the format in which I receive the files is what is throwing me off.
Rather than a normal table schema (rows and columns) the formatting is like follows:

{u'23456789': {u'2014-08-19': {u'date': u'2014-08-19'
                                  u'downloads': 50
                                  u'net_downloads': 49
                                  u'product_id': 39794100070
                                  u'returns': 1
                                  u'revenue': u'49.00'
                                  u'updates': 0}
                  u'2014-08-20': {u'date': u'2014-08-20'
                                  u'downloads': 40
                                  u'net_downloads': 40
                                  u'product_id': 39794100070
                                 u'returns': 0
                                  u'revenue': u'40.00'
                                  u'updates': 0}}}


I can receive this info in csv or txt file. if I open in CSV everything is on the first column A.

Can anyone advise me on how to import this sort of formatting into a normal database.
do I need to use .net to call each of the fields and split the data out?

if you have any reference sites I can look at please let me know,
thanks,p
PutochAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Vitor MontalvãoMSSQL Senior EngineerCommented:
Will be useful if you could find a pattern and fields divisors ( { , u' or : )
0
Jim HornMicrosoft SQL Server Developer, Architect, and AuthorCommented:
Agreed.  What is the source of this file?

I don't see any row/column delimeters, and it doesn't appear to be XML, so afaik unless it's in a format that is covered by a third-party tool such as Pragmatic Works Task Factory or CozyRoc, you're going to have to import it all as one column, then parse the hell out of it in T-SQL to extract values.
0
ValentinoVBI ConsultantCommented:
I don't think this will be possible using the standard Flat File source but besides the option that Jim mentions there's another one: the .NET Script component.  You can use it as Source in a Data Flow.

But it does indeed mean that you'll need to write some string manipulation code to extract the field names and values out of it.  The Split function should help you there.  It also means that you'll need to set up a data flow that uses the Key/Value pattern, this way you don't have dynamic field names going down the flow (which wouldn't work anyway).

So you'll end up with a table that contains some identifiers (file name? or that '23456789' number?  or both?), a key (date, downloads, ...) and a value ('2014-08-19', '50', ...)

Once you've got that and your end goal is a regular flat table you'll need to pivot the values somehow, should be possible through T-SQL.

Setting this up won't be trivial but it should be possible...
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
PutochAuthor Commented:
thank you all for the comments.
I may get to work!
I will repost my solution if/when I get it done.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server 2005

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.