SSIS Importing CSV file

I have set up a Flat File Source in SSIS to load in a CSV file. The trouble that I am having is that some rows have 10 columns, some have 15. When I set up the column structure, the first row had 10 columns, so I added the missing columns to the data structure, but when I do that, it does not appear to be picking up the CR/LF characters, so in the column that I create, it looks like its pulling in the CR/LF control characters followed by the first value of the new line and so on.

I've tried all the variations of the row delimiter, but nothing seems to work as the line feed and properly load the new line of data.

Could it be that the csv source file has an unknown CR/LF character or am I not setting up the parameters of the file connection properly?
wppiexpertsAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

vdr1620Commented:
it might be possible that when you added the extra columns the CR/LF on that line might have been messed up..if you added all the extra columns needed ..i would say, open the CSV file in Excel format and then add an extra blank column to the end (using insert column to right in excel).. Doing that, will add an extra column to the end with correct CR/LF.. In SSIS, Source just select the columns that you need..

If this is a recurring process where the file has In Variable columns.i would suggest you to take a dynamic approach by using script task where you can split the data as suggested in link below , so that you don't have to do any changes to the file manually

http://agilebi.com/cs/blogs/jwelch/archive/2007/05/08/handling-flat-files-with-varying-numbers-of-columns.aspx

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
wppiexpertsAuthor Commented:
as an additional note, I threw together a sample file with 3 columns and a couple rows of data, the import worked fine. However, when one of the rows only had 2 columns, the import went haywire and didn't load properly. So it looks like when using the flat file connection manager, each row of data has to be uniform, otherwise this issue will arise.
vdr1620Commented:
Yes the issue will definitely arise.. Thats the reason i suggested you to take a dynamic approach as suggested in the link where you treat the row as one complete column and then in the script you split the columns. please check the link above
wppiexpertsAuthor Commented:
wow - you link was the perfect solution! Thanks!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.