Comments in dat file for BCP import

Posted on 2008-10-22
Last Modified: 2013-11-30
I have a dat file that is imported into sql server via bcp.  It's format is:

1  33 this is text into a varchar() field
2  43 this is more text
3  44 here is some more text

I'd like do something like this:

1  33 this is text into a varchar() field
//this is part of question3 group
2  43 this is more text
//this is part of the question6 group
3  44 here is some more text

which allows people to look at the dat file data and realize what's going on.
Question by:brettr
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
LVL 51

Expert Comment

by:Mark Wills
ID: 22787933
Cannot do it in BCP, but might be able to do it via openrowset :

select * from OpenRowset('MSDASQL', 'Driver={Microsoft Text Driver (*.txt; *.csv)}; DefaultDir=C:\mypath;','select * from  mytext.dat')

Then you can inspect each row and append a comment accordingly.

Can you share the file ? Or are you OK with the above ?

Author Comment

ID: 22788166
Don't want to append the comment.  It needs to already be in the file so someone can open it up and see comments.   Thanks.
LVL 51

Expert Comment

by:Mark Wills
ID: 22788312
Sorry, misunderstood thought it was being added... so, you are saying there is variable record layouts in the dat file ? If not, then would you mind explaining what the problem is ? That technique above will handle variable inputs... Might like to see if it reads the dat file.
Transaction Monitoring Vs. Real User Monitoring

Synthetic Transaction Monitoring Vs. Real User Monitoring: When To Use Each Approach? In this article, we will discuss two major monitoring approaches: Synthetic Transaction and Real User Monitoring.


Author Comment

ID: 22788958
Sorry.  I don't know how better to explain it than the example I gave.  Where are you confused?
LVL 51

Accepted Solution

Mark Wills earned 500 total points
ID: 22981983
So, you haven't tried the import ?

Answer is still "no" it cannot handle embedded comments - it can be there, but you will have to code for it...

either import as single lines, then manually unpack the lines of interest (or delete the // prefixed lines), or write that back out to import just the lines of interest, or use a DOS command to strip out the lines that do not contain //  (e.g.  find /v "//" mydata.dat >> myimport_data.dat ) and then import that.

There are a number of possibilities to accommodate, it is just that BCP does not know about content other than what it is told as to how to read (such as with format files).

Can certainly use format files to skip columns, but not rows - somewhere it catches you out (e.g. ignore errors with maxerrors=999 - doesn't ignore in batch mode) e.g. can strip out comment lines :

select * from OpenRowset('MSDASQL', 'Driver={Microsoft Text Driver (*.txt; *.csv)}; DefaultDir=C:\ee;','select * from  mytable.txt') where <column heading> not like '//%'

but the catch here is you have to do your own unpacking of the lines of interest...

So, I am not confused - just thought you wanted to add comments to begin with (which we sometimes want to do to flag any errors)...

Author Closing Comment

ID: 31508843
Good call with find /v "//" mydata.dat >> myimport_data.dat and additional sql syntax.  Thanks.

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question