Bulk Insert Fails Error

I am getting this error when trying to do a bulk insert of a text file

Bulk Insert fails. Column is too long in the data file for row 1, column 4.
Make sure the field terminator and row terminator are specified correctly.


Here is my SQL statement:

BULK INSERT MPI
FROM 'd:\inetpub\wwwroot\mpi\load\data\unit.fil'
WITH (FIELDTERMINATOR = '|', ROWTERMINATOR = '\n', DATAFILETYPE = 'CHAR', BATCHSIZE = 800000, ORDER( unit_num ASC), TABLOCK)


What's wierd is that if I go an open the file in wordpad and save it, then try the bulk insert again it succeeds with no problems.  The file is initially transferred from a remote server via ftp.  I'm thinking that the rowterminator is different than \n initially before I open it in wordpad, but I don't know how to determine what it is.

LVL 1
dspentAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ptjcbCommented:
I would suspect that someone is using the '|' in the third column - that would terminate the column, and probably make the fourth column too large. Just a guess.
0
Aneesh RetnakaranDatabase AdministratorCommented:
post the table structure and the some of the file entries
0
dspentAuthor Commented:
Here's the first few lines from the text file (*'s are there because it is hospital patient data).  In total there are approx 700,000 rows.

A000000001 |*******,PETER R             |10-08-**|***-**-****
A000000002 |*******,LINDA S             |        |***-**-****
A000000003 |*******,TIM                 |        |          
A000000004 |**********,BETTY L          |12-29-**|***-**-****
A000000006 |*******,JOHN L              |02-25-**|***-**-****
A000000008 |******,JANIS D              |07-15-**|***-**-****

here is the create table sql:

CREATE TABLE MPI
(unit_num varchar(11) NOT NULL PRIMARY KEY,  pat_name varchar(28) NOT NULL,  dob varchar(8),  ssn varchar(12))


The reason ssn is set to 12 instead of 11 is that many rows include a trailing space after the 11 digit ssn and others don't.  I have no control over the creation of the text file.
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

ptjcbCommented:
Ok - you are right - I didn't know that the table was only 4 columns.

I would guess that this was saved in a different format and they do not recognize the \n as the newline character. That's why it worked when you opened the file in Notepad, then saved it, and ran the job.

Line terminators by OS

Windows/DOS - <CR><LF>
Unix <LF>
Apple <CR>

Has the provider assured you that \n will be the line terminator?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
dspentAuthor Commented:
well I am basically updating this process to work in a coldfusion template.....
previously it was done in a batch file using BCP.

In the BCP statement they used \n as the row terminator and it works without a hitch.  So I can't understand why it would not work with bulk insert.

In windows/dos , do I include the < and > characters???
0
ptjcbCommented:
No...

In VB - if I remember (and it's been awhile) vbCrLf = Chr(13) & Chr(10) = Carriage return/linefeed combination

Use char(13)+ char(10) in SQL.
0
dspentAuthor Commented:
no go.... however if I use \CR or \LF

I get a different error....
for example using ROWTERMINATOR = '\LF' I now get:

Bulk insert data conversion error (truncation) for row 1, column 4 (ssn).
0
dspentAuthor Commented:
Well I have a workaround...  Coldfusion has a cffile tag that allows you to read the contents of a file into a variable, and then output that variable into a new file.
In doing so, you can add an attribute called fixnewline which will replace the line end of the file with the OS specific line end (in this case windows).

So what I have done is after the FTP transfer from the remote server, I read the file contents into a variable, delete the file then rewrite it using the fixnewline attriubute.  This now allows me to use the ROWTERMINATOR = '\n' and all is now at peace with the world.

Seeing as the problem lies with rowterminator, I will award the points to you as everything you mentioned did lead me in the direction of either getting the right terminator to use or fixing the file so I could use the default.

Thanks for your assistance.
0
ptjcbCommented:
No problem. I just wrestled with the same problem last month (it was a Unix file) so I knew to look in that direction.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.