Solved

How do I create a Stored Procedure to produce a table from a tab delimited text file

Posted on 2009-05-19
5
472 Views
Last Modified: 2012-05-07
We have a report that creates a .iif file for Quickbooks import into a Journal Entry.

The file conents look like:
------------------------------------------------------------------------------------
!TRNS[\t]TRNSTYPE[\t]DATE[\t]ACCNT[\t]CLASS[\t]AMOUNT[\t]NAME[\t]DOCNUM[cr][lf]
!SPL[\t]TRNSTYPE[\t]DATE[\t]ACCNT[\t]CLASS[\t]AMOUNT[\t]NAME[\t]DOCNUM[cr][lf]
!ENDTRNS[cr][lf]
TRNS[\t]GENERAL JOURNAL[\t]01/01/2100[\t]PrimaryAcct[\t][\t]-55942.36[\t]EntryName[\t]EntryNum[cr][lf]
SPL[\t]GENERAL JOURNAL[\t]01/01/2100[\t]DetailAcct[\t][\t]663.33[\t]EntryName[\t]EntryNum[cr][lf]
SPL[\t]GENERAL JOURNAL[\t]01/01/2100[\t]DetailAcct[\t][\t]358.06[\t]EntryName[\t]EntryNum[cr][lf]
ENDTRNS[cr][lf]
------------------------------------------------------------------------------------

The file name is usually: filename.iif

Due to a issue with a 3rd party software occassionally the file is created with a Detail Account number missing.

I am trying to build a stored procedure on a SQL Express database that pulls this file into a temp table that can then be queried for null values.

Each time I create the procedure and run it I get zero rows returned.  Also selecting all from the temp table gives me an invalid object name error.  The procedure command completes successfully, yet there are no records in the temp table.

What should I do to get this procedure working?  OR should I use a different approach?


So far this is my code:
 
------------------------------------------------------------------------------
--=====================================================================
SET QUOTED_IDENTIFIER OFF
go
CREATE PROCEDURE QBIMport
-- Parameters input into the procedure
@PathFileName varchar(100)
AS
 
CREATE TABLE #QBExport
(
TRNS VARCHAR(2000),
TRNSTYPE VARCHAR(2000),
DATE VARCHAR(2000),
ACCNT VARCHAR(2000),
CLASS VARCHAR(2000),
AMOUNT VARCHAR(2000),
NAME VARCHAR(2000),
DOCNUM VARCHAR(2000)
)
 
--Step 1: Build Valid BULK INSERT Statement
DECLARE @SQL varchar(2000)
DECLARE @FIELDTERMINATOR varchar(2000)
DECLARE @ROWTERMINATOR varchar(2000)
 
SET @FIELDTERMINATOR = "\t"
SET @ROWTERMINATOR = "\r\n"
 
BEGIN
  -- Valid format:
  SET @SQL = "BULK INSERT #QBExport FROM '"+@PathFileName+"' 
WITH (FIRSTROW = 4,MAXERRORS = 0,FIELDTERMINATOR = '"+@FIELDTERMINATOR+"',ROWTERMINATOR = '"+@ROWTERMINATOR+"') "
END
 
--Step 2: Execute BULK INSERT statement
EXEC (@SQL)
--=====================================================================
--Execute procedure
 
EXEC QBIMport 'c:\testqbglexport.iif'
 
//////////////////////////////////////////////////////////////////////////////////////////////

Open in new window

0
Comment
Question by:DarinAlred
  • 3
  • 2
5 Comments
 
LVL 4

Expert Comment

by:TimSledge
ID: 24425929
You might find the plug-in offered by SpreadsheetGear.com helpful.  I use it to create SQL tables from Excel spreadsheets.  www.spreadhseetgear.com is the url.
0
 

Author Comment

by:DarinAlred
ID: 24433300
Tim,

Thanks for the suggestion.  However, I am looking for something I can automate without having to make a purchase.

BTW- the link is misspelled.

0
 
LVL 4

Expert Comment

by:TimSledge
ID: 24433957
No problem.  Sorry about the link spelling.

0
 

Author Comment

by:DarinAlred
ID: 24503072
I am currently experimenting with the following:

1st I am using a BCP format file
--bcp FORMAT FILE CREATION

9.0
8
1       SQLCHAR       0       2000       "\t"     1     TRNS     ""
2       SQLCHAR       0       2000     "\t"     2     TRNSTYPE             ""
3       SQLDATETIME       0       2000     "\t"     3     DATE        ""
4       SQLCHAR       0       2000      "\t"   4     ACCNT     ""
5       SQLCHAR       0       2000      "\t"   5     CLASS     ""
6       SQLDECIMAL       0       2000      "\t"   6     AMOUNT     ""
7       SQLCHAR       0       2000      "\t"   7     NAME     ""
8       SQLCHAR       0       2000      "\r\n"   8     DOCNUM     ""

Next

I am using a Global Temp Table structure:

CREATE TABLE ##QBExport
(
TRNS VARCHAR(2000),
TRNSTYPE VARCHAR(2000),
DATE NVARCHAR(10),
ACCNT VARCHAR(2000),
CLASS VARCHAR(2000),
AMOUNT MONEY,
NAME VARCHAR(2000),
DOCNUM VARCHAR(2000)
)

BULK INSERT ##QBExport
FROM 'C:\testqbglexport.txt'
WITH ( FIRSTROW = 4, LASTROW FORMATFILE = 'c:\TestFormatFileNew.Fmt');

However, because my file ends with a footer I am now inserting a counter format file which I got off of another EE answer ( http://www.experts-exchange.com/Database/Miscellaneous/Q_23349870.html) mainly because I get an "unexpected end of file" error.

--define a format file for single line per row
--9.0
--1
--1      SQLCHAR       0      8000     "\r\n"   1     Myrecord            ""
--
-- now count the records
--
declare @lastrow int
set     @lastrow = (SELECT count(*) FROM OPENROWSET( BULK 'c:\MyData.txt', FORMATFILE = 'c:\MyFormat_Counting.fmt', MAXERRORS=10) AS a ) - 1
--
-- now use that count to define last record, have to build up dynamic sql, load into staging area
--
if exists (select * from information_schema.tables where table_name = 'mystagingtable') drop table mystagingtable
declare @sqlcmd varchar(max)
set     @sqlcmd = 'SELECT a.* into MyStagingTable FROM OPENROWSET( BULK ''c:\MyData.txt'', FORMATFILE = ''c:\MyFormat.fmt'', MAXERRORS=10, FIRSTROW = 1, LASTROW = '+convert(varchar,@lastrow)+') AS a '
--
exec   (@sqlcmd)
--
select * from MyStagingTable

I will be fiddling with this to see if it can work today.
0
 

Accepted Solution

by:
DarinAlred earned 0 total points
ID: 24528665
This is the current code.

It works, only if I remove the last line of the file which hold the ENDTRNS.

If I do not I get:


Msg 4832, Level 16, State 1, Line 1
Bulk load: An unexpected end of file was encountered in the data file.
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".

This file cannot have the last line taken off and still work when the file is sent to the client.

Is there a way to get the last line recoginzed?
-- START ''C:\TestFormatFileNew.Fmt''
 
--9.0
--8
--1	SQLCHAR	0	2000	"\t"	1	TRNS	""
--2	SQLCHAR	0	2000	"\t"	2	TRNSTYPE	""
--3	SQLNCHAR	0	10	"\t"	3	DATE	""
--4	SQLCHAR	0	2000	"\t"	4	ACCNT	""
--5	SQLCHAR	0	2000	"\t"	5	CLASS	""
--6	SQLMONEY	0	2000	"\t"	6	AMOUNT	""
--7	SQLCHAR	0	2000	"\t"	7	NAME	""
--8	SQLCHAR	0	2000	"\r\n"	8	DOCNUM	""
--
-- END ''C:\TestFormatFileNew.Fmt''
 
--define a format file for single line per row
--9.0
--1
--1      SQLCHAR       0      8000     "\r\n"   1     Myrecord            ""
--
-- now count the records
--
declare @lastrow int
set     @lastrow = (SELECT count(*) FROM OPENROWSET( BULK 'C:\testqbglexport.txt', FORMATFILE = 'C:\TestFormatFileNew_Count.Fmt', MAXERRORS=10) AS a ) - 1
--
-- now use that count to define last record, have to build up dynamic sql, load into staging area
--
if exists (select * from information_schema.tables where table_name = '##QBExport') drop table ##QBExport
declare @sqlcmd varchar(max)
set     @sqlcmd = 'SELECT a.* into ##QBExport FROM OPENROWSET( BULK ''C:\testqbglexport.txt'', FORMATFILE = ''C:\TestFormatFileNew.Fmt'', MAXERRORS=10, FIRSTROW = 1, LASTROW = '+convert(varchar,@lastrow)+') AS a '
--
exec   (@sqlcmd)
--
select * from ##QBExport

Open in new window

0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Union 2 queries to a cte (temp table perhaps) 9 41
CSV How to add columns based on existing column(s)? 20 32
Perl Versus AWK? 7 51
Parse this column 6 27
Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Many companies are looking to get out of the datacenter business and to services like Microsoft Azure to provide Infrastructure as a Service (IaaS) solutions for legacy client server workloads, rather than continuing to make capital investments in h…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question