• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 358
  • Last Modified:

Special characters in DB2 won't import to SQL via CSV file

I have a large table in DB2 (>1GB) that I need to export to a CSV and then import to SQL Server 2008.  Security rules prohibit a direct connection to DB2 from SQL, so I must export to the CSV file.

There are special characters in the DB2 table that prevent the SQL import to succeed.  Is there a way to identify these characters (I have even seen an arrow character), and replace them with something that will import properly?
0
pansophy
Asked:
pansophy
  • 3
  • 3
2 Solutions
 
Jim HornMicrosoft SQL Server Developer, Architect, and AuthorCommented:
>Is there a way to identify these characters
Give this a whirl..

1.

Download a trial of the super-dooper text editor Ultra Edit.

2.

Open it, load your file.

3.

Hit ctrl-H to go into Hex mode, which shows two windows.
Left side is your file, including any characters that would not be visible in a .csv.  
Right side is the Hex number of every character on the left side.

4.

Do a google search for 'ANSI character set' and eventually you'll find one that maps out every character's value, ANSI, and HEX.

5.

Now you know what characters you'll need to weed out, and edit your ETL package accordingly.
0
 
pansophyAuthor Commented:
Jim, I got to step 4 without a problem.  But how do I know which characters will give SQL a problem?

Also am I correct that once I identify these characters i am doing a HEX find/replace?
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Hi pansophy,

What character sets (code pages) are you using for DB2 and SQL Server?
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
pansophyAuthor Commented:
It appears that DB2 is using this character set "IBM All Character 437 Set (ANSI)" to create the CSV file.  

http://whitefiles.org/b1_s/1_free_guides/fg2cd/pgs/c03b.htm
0
 
Kent OlsenData Warehouse Architect / DBACommented:
What version of DB2 are you running?  Z/OS, UDB on Windows, UDB on *nix, or AS400?

On most systems you can run the db2 command to find the code page setting

     db2 get db cfg for {databasename}

The code page, code set, and collating information are in the first 8 or 10 lines.
0
 
pansophyAuthor Commented:
Database configuration release level                    = 0x0f00
Database release level                                  = 0x0f00
 
Database territory                                      = US
Database code page                                      = 819
Database code set                                       = ISO8859-1
Database country/region code                            = 1
Database collating sequence                             = UNIQUE
Alternate collating sequence              (ALT_COLLATE) =
 Number compatibility                                    = OFF
Varchar2 compatibility                                  = OFF
Date compatibility                                      = OFF
0
 
Kent OlsenData Warehouse Architect / DBACommented:
Ok.

819 is the "normal" code page used when DB2 is running on AIX.  The key to making completing this conversion is to knowing what character set the SQL Server database is using.  If it's utf-8 (a reasonable choice) you can get the correct export with something like this:

 db2 "export to mytable.csv of del modified by CODEPAGE=1208 select * from mytable"


Kent
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

  • 3
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now