Link to home
Start Free TrialLog in
Avatar of ZuZuPetals
ZuZuPetalsFlag for United States of America

asked on

SSIS: Flat File Data Source: Text qualifier (") blows up for lines with internal two double quotes

I'm trying to import a .csv file whose author chose this quoting system: two double quotes are used internally to quote an item inside a field.  Example input file:
"Song Name", "Composer"
"Yellow Submarine","Beatles"
"Symphony no 94 ""Surprise""","Haydn"     <--- SSIS doesn't like ""Surprise""
I have these set:
    Format: Delimited
    Text qualifier: "

All rows import except the lines that look ""like this""

How can I get around this problem without writing C# to manually clean up those lines before I import using SSIS?

SQL Server is an enterprise-level data solution... there must be a built-in way to attack this problem.
Avatar of Barry Cunney
Barry Cunney
Flag of Ireland image

Add a Derived Column Transformation task
Set up 2 variables one to store "" the other to store ''
in the expression of Derived column do a REPLACE function something like this
REPLACE( [yourfield], @[User::q1], @[User::q2])

so as to replace 2 double quotes with 2 single quotes -
Avatar of ZuZuPetals

ASKER

Do you do that stuff before or after (or inside?) the "Flat File Source" component?
Thanks, but could you please provide more detail?
ASKER CERTIFIED SOLUTION
Avatar of Barry Cunney
Barry Cunney
Flag of Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial