?
Solved

SSIS / BIDS Nvarchar Flat File Export Issue - Text was truncated or one or more characters had no match in the target code page

Posted on 2012-09-07
6
Medium Priority
?
2,349 Views
Last Modified: 2013-11-18
I'm having an issue with SQL Server 2008R2 64-bit with a BIDS Data Flow Task specifically.

I'm trying to export data from a SQL Server database to a flat file. I have narrowed the problem down to one specific field, which is type NVARCHAR(100) .

I altered my query to only include records that are a problem:

SELECT PoNum FROM SALES WHERE CAST(PoNum AS VARCHAR) != PoNum;

This allowed me to lower the record count to 160 records.
Then I used the 'TOP' command to get as many records to work as I could, which narrowed it further to the following 3 records which cause the problems (I pasted them and included a screen shot).

170608356¿
¿¿¿
пѿсѿовой - Innovate
recordset

My flat file settings look like this:
flat file settings
And the column in question is set as such in the flat file connection manager:

DataType: Unicode String [DT_WSTR]
width 100
textQualifed: True

When I hit run, I get this message:

[Flat File Destination [19]] Error: Data conversion failed. The data conversion for column "NumAtCard" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".

I have tried the following:
1. Setting the column width to 250 in the advanced tab in the flat file manager
2. Setting TextQualified to False
3. Changing the column type to DT_String , DT_Text, and DT_NText

If I ignore these 3 records, everything works perfectly for all records in the table (100,000+). It's obviously an encoding problem, but I can't figure out how to fix it.

Thanks!
0
Comment
Question by:Candidochris
  • 3
  • 3
6 Comments
 
LVL 16

Accepted Solution

by:
DcpKing earned 1500 total points
ID: 38380336
You have some weird character in there somewhere! I'd recommend this:

Go to IDM solutions - https://www.ultraedit.com/store/home.php - and download a copy of UltraEdit for your OS (there are versions for Windows, Mac, and Linux).

Install it (there's a trial period of a few weeks), and load in your CSV file

Once you have the file up and you've found the problem line, switch to Hex mode - either Ctrl+H or Shift+Ctrl+H, depending on the version) and look at what you're <i>really</i> sending to SSIS, as opposed to what your normal text editor tells you it thinks you want to see!

I think you'll see some strange characters in there that you really don't want to have! That's my opinion because your screenshot and paste are different!

hth

Mike
0
 

Author Comment

by:Candidochris
ID: 38382985
Thanks, I will try UltraEdit - but the problem is that the characters are in the database - I'm looking for a way to ignore/convert these characters, regardless of what they are to generate the flat file.
0
 
LVL 16

Expert Comment

by:DcpKing
ID: 38414540
Ah! Sorry - for some reason I was thinking you were importing.

Try doing something like this:

select * from tablename where field like '%[^!-}]%'

which means "like any number of characters, a special char, and any number more, where the special char is outside the range from "!" (21 hex, 33 decimal) to "}" (7D hex)"

For the character ranges check http://en.wikipedia.org/wiki/ASCII

I didn't try, but you should be able to use the char function, so replace "!" with char(33), or use char(32) if you have spaces you want to ignore.

hth

Mike
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 

Author Comment

by:Candidochris
ID: 38429946
Thanks Mike, that's what I needed.
0
 
LVL 16

Expert Comment

by:DcpKing
ID: 38430215
Great. I'm sorry for the long delay!

So what's next? :)

Mike
0
 

Author Comment

by:Candidochris
ID: 38430732
Mike,

 Hopefully everything just works - but like always, I'll post in here if not. I did get this project going and working consistently.
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
An alternative to the "For XML" way of pivoting and concatenating result sets into strings, and an easy introduction to "common table expressions" (CTEs). Being someone who is always looking for alternatives to "work your data", I came across this …
The viewer will learn how to count occurrences of each item in an array.
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.
Suggested Courses

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question