Solved

SSIS Importing CSV file

Posted on 2010-08-25
4
1,010 Views
Last Modified: 2013-11-10
I have set up a Flat File Source in SSIS to load in a CSV file. The trouble that I am having is that some rows have 10 columns, some have 15. When I set up the column structure, the first row had 10 columns, so I added the missing columns to the data structure, but when I do that, it does not appear to be picking up the CR/LF characters, so in the column that I create, it looks like its pulling in the CR/LF control characters followed by the first value of the new line and so on.

I've tried all the variations of the row delimiter, but nothing seems to work as the line feed and properly load the new line of data.

Could it be that the csv source file has an unknown CR/LF character or am I not setting up the parameters of the file connection properly?
0
Comment
Question by:wppiexperts
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
4 Comments
 
LVL 16

Accepted Solution

by:
vdr1620 earned 50 total points
ID: 33522969
it might be possible that when you added the extra columns the CR/LF on that line might have been messed up..if you added all the extra columns needed ..i would say, open the CSV file in Excel format and then add an extra blank column to the end (using insert column to right in excel).. Doing that, will add an extra column to the end with correct CR/LF.. In SSIS, Source just select the columns that you need..

If this is a recurring process where the file has In Variable columns.i would suggest you to take a dynamic approach by using script task where you can split the data as suggested in link below , so that you don't have to do any changes to the file manually

http://agilebi.com/cs/blogs/jwelch/archive/2007/05/08/handling-flat-files-with-varying-numbers-of-columns.aspx
0
 

Author Comment

by:wppiexperts
ID: 33523655
as an additional note, I threw together a sample file with 3 columns and a couple rows of data, the import worked fine. However, when one of the rows only had 2 columns, the import went haywire and didn't load properly. So it looks like when using the flat file connection manager, each row of data has to be uniform, otherwise this issue will arise.
0
 
LVL 16

Expert Comment

by:vdr1620
ID: 33523723
Yes the issue will definitely arise.. Thats the reason i suggested you to take a dynamic approach as suggested in the link where you treat the row as one complete column and then in the script you split the columns. please check the link above
0
 

Author Closing Comment

by:wppiexperts
ID: 33525101
wow - you link was the perfect solution! Thanks!
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Show RTF format in an SSRS report 3 41
SQL Get Store Procedure Column Name As Row 5 41
Connect Gridview column to Textbox in C# 2 40
SQL Distinct Question 3 15
Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
Via a live example, show how to extract insert data into a SQL Server database table using the Import/Export option and Bulk Insert.
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question