Solved

SqlBulkCopy

Posted on 2010-11-30
8
1,372 Views
Last Modified: 2012-05-10
I seem to have stumbled across an issue with c# and the SqlBulkCopy import when bulk importing thing inside asp.net

When we import the data from a console application it works fine.
When we import the data from inside asp.net and certain char's with accents and the like seem to be converted to a ?

We are struggling to get to the bottom of this.
Or is there any way to set the encoding type to avoid this problem from happen?





0
Comment
Question by:Mistralol
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
8 Comments
 
LVL 96

Expert Comment

by:Bob Learned
ID: 34239826
How are you importing the data that is passed to the SqlBulkCopy?  I don't know of any encoding that can be applied to the SqlBulkCopy, so my thinking is that it is an importing problem, and not an SqlBulkCopy problem...
0
 
LVL 6

Author Comment

by:Mistralol
ID: 34240310

Its using a fixed with idatareader we wrote. Its reading this with a forced encoding type of ASCII.
However this work when it is in the console app.

0
 
LVL 6

Author Comment

by:Mistralol
ID: 34240562

I think we may have found the problem. Its something i was not aware of. and the issue is because the text file is being opened from insider a zip file :/

0
Database Solutions Engineer FAQs

In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller single-server environments.

 
LVL 6

Accepted Solution

by:
Mistralol earned 0 total points
ID: 34247231

Ok so i guess we did't find the problem.

But here is a bit more information.


This works ... (with reduced dataset)

            IDataReader Reader = new Forth.Data.Standard.FixedDataReader("data.txt",
                new int[] { 40 },
                Forth.Data.Standard.FixedDataReaderOptions.RecordTerminatedCarriageReturnLineFeed);

            Conn.ImportNoMapping(Reader, "CharTest");

This Also works .... (with reduced dataset)

            byte[] b = File.ReadAllBytes("data.zip");
            ZipFile Zip = new ZipFile(new MemoryStream(b));
            Reader = new Forth.Data.Standard.FixedDataReader(Zip.GetInputStream(Zip.GetEntry("data.txt")),
                new int[] { 40 },
                    Forth.Data.Standard.FixedDataReaderOptions.RecordTerminatedCarriageReturnLineFeed);

            Conn.ImportNoMapping(Reader, "CharTest2");

However when run across the full datafile either method fails in console or web envirorment!
The encoded text seems to fail again.



0
 
LVL 96

Assisted Solution

by:Bob Learned
Bob Learned earned 500 total points
ID: 34247829
Are you saying that the Unicode characters are not getting translated to question marks during the reading process?  I don't remember ever seeing that problem with bcp and SqlBulkCopy, so it would most likely be a mysterious problem under the hood.   That kind of problem is difficult to solve for myself, let alone for someone else...
0
 
LVL 6

Author Comment

by:Mistralol
ID: 34248665

Yeah we did get to the bottom of the problem. Involving the FixedDataReader. It was reading the file in the wrong mode. I switched it to binary and the issue magicly vanished. The interesting part is that the data in the file was changing the bug somehow.



0
 
LVL 96

Assisted Solution

by:Bob Learned
Bob Learned earned 500 total points
ID: 34249065
That would make sense if the data wasn't always Unicode, since those are the characters that get translated to ? with ASCII representation...
0
 
LVL 6

Author Closing Comment

by:Mistralol
ID: 34636486
We did get it fixed.
0

Featured Post

Do you have a plan for Continuity?

It's inevitable. People leave organizations creating a gap in your service. That's where Percona comes in.

See how Pepper.com relies on Percona to:
-Manage their database
-Guarantee data safety and protection
-Provide database expertise that is available for any situation

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Via a live example, show how to shrink a transaction log file down to a reasonable size.
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

615 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question