Solved

SSIS: Flat File Data Source: Text qualifier (") blows up for lines with internal two double quotes

Posted on 2011-03-23
3
1,012 Views
Last Modified: 2012-05-11
I'm trying to import a .csv file whose author chose this quoting system: two double quotes are used internally to quote an item inside a field.  Example input file:
"Song Name", "Composer"
"Yellow Submarine","Beatles"
"Symphony no 94 ""Surprise""","Haydn"     <--- SSIS doesn't like ""Surprise""
I have these set:
    Format: Delimited
    Text qualifier: "

All rows import except the lines that look ""like this""

How can I get around this problem without writing C# to manually clean up those lines before I import using SSIS?

SQL Server is an enterprise-level data solution... there must be a built-in way to attack this problem.
0
Comment
Question by:ZuZuPetals
  • 2
3 Comments
 
LVL 17

Expert Comment

by:Barry Cunney
ID: 35200668
Add a Derived Column Transformation task
Set up 2 variables one to store "" the other to store ''
in the expression of Derived column do a REPLACE function something like this
REPLACE( [yourfield], @[User::q1], @[User::q2])

so as to replace 2 double quotes with 2 single quotes -
0
 
LVL 2

Author Comment

by:ZuZuPetals
ID: 35203192
Do you do that stuff before or after (or inside?) the "Flat File Source" component?
Thanks, but could you please provide more detail?
0
 
LVL 17

Accepted Solution

by:
Barry Cunney earned 500 total points
ID: 35205481
Hi ZuZuPetals,
You do that stuff in separate step between your flat file source and your destinatation
Derived Column is one of the Transformation Tasks in the Toolbox

Please see print screen with example


SSISDerivedColumn.jpg
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
Via a live example, show how to shrink a transaction log file down to a reasonable size.
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now