steven
asked on
SQL Csv export has End of line character
We are using the BCP command to export data from a 2012 r2 sql database to a CSV file. There seems to be a end of line character on the last line of data that is causing issues [] . Is there a way to remove this?
Command:
set @body = 'bcp ' + db_name() + '.dbo.##ENSORDERPROD out ' + @outfile + ' -c -t, -T -S EINCUtility05'
exec master..xp_cmdshell @body
Command:
set @body = 'bcp ' + db_name() + '.dbo.##ENSORDERPROD out ' + @outfile + ' -c -t, -T -S EINCUtility05'
exec master..xp_cmdshell @body
What are you opening the output file with?
If I use Notepad, it does "appear" there is an extra line but in any other editor it doesn't show it.
If you don't have another editor, use FIND to count the lines (I used a file called q.txt):
find /c /v "" q.txt
Open up powershell and do a hex dump:
type q.txt | Format-Hex
count the lines in Powershell:
type q.txt | Measure-Object
If I use Notepad, it does "appear" there is an extra line but in any other editor it doesn't show it.
If you don't have another editor, use FIND to count the lines (I used a file called q.txt):
find /c /v "" q.txt
Open up powershell and do a hex dump:
type q.txt | Format-Hex
count the lines in Powershell:
type q.txt | Measure-Object
Maybe some BCP versions do not add CRLF or whatever character is defined in -r parameter but I still see CRLF at the end of each line and this is correct behavior.
If the software does not process the CRLF in the last line properly then its removal seems to the only way.
If the software does not process the CRLF in the last line properly then its removal seems to the only way.
>>at the end of each line and this is correct behavior.
At the end of each line is correct. The question asked is about the last line of the file and an "extra" one.
At the end of each line is correct. The question asked is about the last line of the file and an "extra" one.
No, I did not understand question this way. There is nothing extra, just the "EOL character on the last line of data".
ASKER
The question is the software that opens the file to import it has issues with the "end of line" character on the last line. The software that opens is from a vendor of ours. They want us to remove the character, I only see it when I open the file in excel as it is a CSV file. I have attached a screen shot.
Capture.PNG
Capture.PNG
OK, this is not extra EOL but something else… and such character does not generate BCP obviously. Could you open the CSV file in some editor which can display invisible characters? Possibilities are Notepad++ or some hexadecimal editor.
ASKER
I opened in notepad ++ has SUB at the end. I have attached the CSV file.
Is that data OK to post in an open forum?
I don't see anything "extra" ad the end.
I don't see anything "extra" ad the end.
That image still shows what might be considered "sensitive" data. You might want to take a look and remove it if it does.
The SUB is an ascii 26:
http://www.asciitable.com/
I don't believe bcp by itself is creating it. If you run the bcp command from the command line, is it there?
http://www.asciitable.com/
I don't believe bcp by itself is creating it. If you run the bcp command from the command line, is it there?
ASKER
you are correct when run from command line straight to file nothing, when the below is run: It appears.
exec master..xp_cmdshell 'copy \\eincftp\m$\Invoices\ENSI NVOICEHEAD ER.csv + \\eincftp\m$\Invoices\ENSI NVOICEBODY .csv \\eincftp\m$\Invoices\INCI NVOICE.csv '
exec master..xp_cmdshell 'copy \\eincftp\m$\Invoices\ENSI
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thank you very much this took care of it. THANKS
LOL, we could tell it directly if you would mention you are using COPY command to generate the output...
ASKER
I should have posted that with my question but I didn't sorry, again thanks to all! great job.
In the future: NEVER try to debug the entire process at once.
Break it down to the smallest components:
Test bcp command line first: Removes stored procedure stuff.
Then test bcp inside the procedure.
Then you would have seen it wasn't in bcp so it HAD to be external to that.
Break it down to the smallest components:
Test bcp command line first: Removes stored procedure stuff.
Then test bcp inside the procedure.
Then you would have seen it wasn't in bcp so it HAD to be external to that.
ASKER
Will Do thanks for all the help. I jumped straight to bcp.
The same work can do FSUTIL DOS command. First retrieve the current file length by
FSUTIL file queryValidData YourFileName.ext
and then set the file size
FSUTIL file setValidData YourFileName.ext newSize
UPDATE:
FSUTIL requires administrative rights for setting the new file size... but even with admin rights it does not work for me.
Additional tool is SFK: http://stahlworks.com/dev/?tool=partcopy