Solved

Best way to import txt-file to MS-SQL ?

Posted on 2012-03-19
14
403 Views
Last Modified: 2012-04-11
Hi!

Have a textfile that locks like:

1,"Kelly 2","Reynold","kelly@reynold.com"
22,"John 2","Smith","bill@smith.com"
37,"Sara 2","Parker","sara@parker.com"

Have a example code, that remove Double quotes (lock below)
But cant figer out how to use the same sourcecode to use the abow txt-file
The sourcode use today this txt-file:

"Kelly 2","Reynold","kelly@reynold.com"
"John 2","Smith","bill@smith.com"
"Sara 2","Parker","sara@parker.com"



Need help to make the modification

ALTER PROCEDURE "dbo"."ps_StudentList_Import"
@PathFileName varchar(100),
@OrderID integer,
@FileType tinyint
AS


DECLARE @SQL varchar(2000)
IF @FileType = 1
 BEGIN
  
  SET @SQL = "BULK INSERT TmpStList FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
 END
ELSE
 BEGIN
  
  SET @SQL = "BULK INSERT TmpStList FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = ',') "
 END


EXEC (@SQL)


INSERT StudentList (StFName,StLName,StEmail,OrderID)
SELECT  CASE WHEN @FileType = 1 THEN SUBSTRING(StFName,2,DATALENGTH(StFName)-1)
             ELSE StFName
        END,
        SUBSTRING(StLName,1,DATALENGTH(StLName)-0),
        CASE WHEN @FileType = 1 THEN SUBSTRING(StEmail,1,DATALENGTH(StEmail)-1)
             ELSE StEmail
        END,
        @OrderID
FROM tmpStList


TRUNCATE TABLE TmpStList

Open in new window

0
Comment
Question by:team2005
  • 7
  • 5
  • 2
14 Comments
 
LVL 42

Expert Comment

by:EugeneZ
ID: 37736934
create new or recreate TmpStList table with just 3 columns
instead of 4
0
 
LVL 2

Author Comment

by:team2005
ID: 37737002
Hi!

Tryed that, but test1 field is emty ?
0
 
LVL 2

Author Comment

by:team2005
ID: 37737380
Hi!

Need help solving this issue ASAP.

I am stuck...
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 39

Expert Comment

by:lcohan
ID: 37738812
"Tryed that, but test1 field is emty "

What is test1? Is it a column of tmpStList table? can you post tmpStList table structure?
0
 
LVL 2

Author Comment

by:team2005
ID: 37738865
Hi!

test1 is a colum in table tmpStlist

tmpStlist

stFName  varchar(50)
stLName varchar(50)
stEmail varchar(100)

Other table > StudentList

StID int identity
test1 int
stFName  varchar(50)
stLName varchar(50)
stEmail varchar(100)
OrderID int
0
 
LVL 42

Expert Comment

by:EugeneZ
ID: 37739824
if you need to keep
test1 is a colum in table tmpStlist
create new table and do bulk insert there
if not delete test1 column from tmpStlist table

tmpStlist

stFName  varchar(50)
stLName varchar(50)
stEmail varchar(100)
0
 
LVL 2

Author Comment

by:team2005
ID: 37739892
Hi!

Can't delete test1. Have a textfile like:

1,"Kelly 2","Reynold","kelly@reynold.com"
22,"John 2","Smith","bill@smith.com"
37,"Sara 2","Parker","sara@parker.com"

Maby better to import textfile to tmbtable
and loop and insert data into a new table. And use
 SUBSTRING in the loop.

Another problem is that if i import another textfile.
That contains of same test1 value. The BULK insert wil fail...

Must be able to yust update, and not insert.

Realy need to solve this issue
0
 
LVL 42

Expert Comment

by:EugeneZ
ID: 37740002
ok
I got it now

you have no problems with

"Kelly 2","Reynold","kelly@reynold.com"
"John 2","Smith","bill@smith.com"
"Sara 2","Parker","sara@parker.com"

but have problem to bulk this

1,"Kelly 2","Reynold","kelly@reynold.com"
22,"John 2","Smith","bill@smith.com"
37,"Sara 2","Parker","sara@parker.com"
--

please confirm that TmpStList has 3 columns?

also can you please post the error when you import the "bad "file data?
0
 
LVL 42

Accepted Solution

by:
EugeneZ earned 500 total points
ID: 37740009
looks like it is the source
Using BULK INSERT to Load a Text File
http://www.sqlteam.com/article/using-bulk-insert-to-load-a-text-file
0
 
LVL 42

Expert Comment

by:EugeneZ
ID: 37740030
it means due to import 4 parts file data

you need to create new table TmpStList and use it

CREATE TABLE TmpStList
(
ID varchar (50) NOT NULL,   ----<<<<<new
 stFName varchar (50) NOT NULL,
 stLName varchar (50) NOT NULL,
 stEmail varchar (100) NOT NULL
)
go
0
 
LVL 2

Author Comment

by:team2005
ID: 37741283
Hi!

Tryed to make this Stored procedure now:

ALTER PROCEDURE "dbo"."ps_StudentList_Import6"
@PathFileName varchar(100),
@OrderID integer,
@FileType tinyint
AS


DECLARE @SQL varchar(2000)
IF @FileType = 1
 BEGIN
  
  SET @SQL = "BULK INSERT TmpStList6 FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
 END
ELSE
 BEGIN
  
  SET @SQL = "BULK INSERT TmpStList6 FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = ',') "
 END


EXEC (@SQL)


INSERT StudentList6 (test1,StFName,StLName,StEmail,OrderID)
SELECT  CASE WHEN @FileType = 1 THEN test1 END,
        CASE WHEN @FileType = 1 THEN SUBSTRING(StFName,2,DATALENGTH(StFName)-1)
             ELSE StFName
        END,
        SUBSTRING(StLName,1,DATALENGTH(StLName)-0),
        CASE WHEN @FileType = 1 THEN SUBSTRING(StEmail,1,DATALENGTH(StEmail)-1)
             ELSE StEmail
        END,
        @OrderID
FROM tmpStList6


TRUNCATE TABLE TmpStList6

Open in new window


Get this error message:

10:13:58  [ALTER - 0 row(s), 0.000 secs]  1) [Error Code: 207, SQL State: 42S22]  Invalid column name 'BULK INSERT TmpStList6 FROM ''. 2) [Error Code: 207, SQL State: 42S22]  Invalid column name '' WITH (FIELDTERMINATOR = '","') '. 3) [Error Code: 207, SQL State: 42S22]  Invalid column name 'BULK INSERT TmpStList6 FROM ''. 4) [Error Code: 207, SQL State: 42S22]  Invalid column name '' WITH (FIELDTERMINATOR = ',') '.
... 1 statement(s) executed, 0 row(s) affected, exec/fetch time: 0.000/0.000 sec  [0 successful, 0 warnings, 1 errors]


test1 is a int field, i used this code to make tables:

CREATE TABLE StudentList
(
 StID int IDENTITY NOT NULL,
 test1 int,
 StFName varchar(50) NOT NULL,
 StLName varchar(50) NOT NULL,
 StEmail varchar(100) NOT NULL,
 OrderID int NOT NULL
)
go
CREATE TABLE TmpStList
(
 test1 int,
 stFName varchar (50) NOT NULL,
 stLName varchar (50) NOT NULL,
 stEmail varchar (100) NOT NULL
)
go

Open in new window

0
 
LVL 2

Author Comment

by:team2005
ID: 37741358
Hi!

Make this stored procedure (create procedure):

SET QUOTED_IDENTIFIER OFF
go
CREATE PROCEDURE ps_StudentList_Import6
@PathFileName varchar(100),
@OrderID integer,
@FileType tinyint
AS

--Step 1: Build Valid BULK INSERT Statement
DECLARE @SQL varchar(2000)
IF @FileType = 1
 BEGIN
  -- Valid format: "John","Smith","john@smith.com"
  SET @SQL = "BULK INSERT TmpStList6 FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = '"",""') "
 END
ELSE
 BEGIN
  -- Valid format: John,Smith,john@smith.com
  SET @SQL = "BULK INSERT TmpStList6 FROM '"+@PathFileName+"' WITH (FIELDTERMINATOR = ',') "
 END

--Step 2: Execute BULK INSERT statement
EXEC (@SQL)

--Step 3: INSERT data into final table
INSERT StudentList6 (Stteste,StFName,StLName,StEmail,OrderID)
SELECT  CASE WHEN @FileType = 1 THEN Stteste
            ELSE Stteste
        END,
        CASE WHEN @FileType = 1 THEN SUBSTRING(StFName,2,DATALENGTH(StFName)-1)
             ELSE StFName
        END,
        SUBSTRING(StLName,1,DATALENGTH(StLName)-0),
        CASE WHEN @FileType = 1 THEN SUBSTRING(StEmail,1,DATALENGTH(StEmail)-1)
             ELSE StEmail
        END,
        @OrderID
FROM tmpStList6

--Step 4: Empty temporary table
TRUNCATE TABLE TmpStList6
go

Open in new window


But when i run the stored procedure, (the new created procedure)
i get this error message:

10:39:43  [@CALL - 0 row(s), 0.000 secs]  1) [Error Code: 4864, SQL State: S1000]  Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 1 (Stteste). 2) [Error Code: 4864, SQL State: S1000]  Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 2, column 1 (Stteste). 3) [Error Code: 4832, SQL State: S1000]  Bulk load: An unexpected end of file was encountered in the data file. 4) [Error Code: 7399, SQL State: S1000]  The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. 5) [Error Code: 7330, SQL State: S1000]  Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
0
 
LVL 39

Expert Comment

by:lcohan
ID: 37744325
You said
"Hi!

test1 is a colum in table tmpStlist"

but then you said

"tmpStlist

stFName  varchar(50)
stLName varchar(50)
stEmail varchar(100"

so my question remains (just check above) - "Tryed that, but test1 field is emty "

What is test1? Is it a column of tmpStList table? If not it should be as you try to get data from the text field like 1,22,37 ... values right? Should those go into StudentList.test1 int??

If yes add that column to your tmpStlist and change its structure like below:

test1 int,
stFName  varchar(50),
stLName varchar(50),
stEmail varchar(100)
0
 
LVL 2

Author Closing Comment

by:team2005
ID: 37832222
thanks
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SQL Replication question 9 43
Query Syntax 17 36
Passing Parameter to Stored Procedure 4 24
How to resolve SQL Server DB deadlock which makes my application hangs ? 6 30
Introduction SQL Server Integration Services can read XML files, that’s known by every BI developer.  (If you didn’t, don’t worry, I’m aiming this article at newcomers as well.) But how far can you go?  When does the XML Source component become …
Everyone has problem when going to load data into Data warehouse (EDW). They all need to confirm that data quality is good but they don't no how to proceed. Microsoft has provided new task within SSIS 2008 called "Data Profiler Task". It solve th…
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question