Solved

Batch script to remove blank Carriage Return lines

Posted on 2008-10-22
11
4,029 Views
Last Modified: 2011-10-19
Batch script to remove blank Carriage Return lines

I currently have the below script which parses a file and if any lines exist which are empty and contain:

CR
CR/LF
data blah blah CR/LF
CR
CR/LF
more data blah blah CR/LF

it will strip away the empty data and produce:

data blah blah CR/LF
more data blah blah CR/LF

The problem is if an input file is like

CR
data blah blah CR/LF
CR
more data blah blah CR/LF

it fails....  

I would like it to work in either scenario or if not possible then just a new script / modification to handle this new scenario


Many thanks



@echo off

setlocal

rem This script strips out any empty lines terminated with a CR or a CR/LF

rem usage is process_feeds.bat "C:\inputfilename.txt" "C:\outputfilepath"

rem it also makes a backup of the input files
 

if [%2] NEQ [] (call :process %1 %2) else (

 for /f "tokens=*" %%a in ('dir /b /a-d "%source%\*.txt"') do call :process "%source%\%%a" "%dest%"

)

 

goto :eof

 

:process

set input=%~1

set output=%~2\%~nx1

echo Working on %input%

findstr /B /I /R "[!-z]" "%input%" > "%output%"

REM remove the REM from the next line to have it remove the 'source' files.

move "%input%" C:\backup\

Open in new window

0
Comment
Question by:arundelr
  • 4
  • 4
  • 2
  • +1
11 Comments
 
LVL 28

Expert Comment

by:Bill Bach
ID: 22776598
If you're not opposed to a simple C program, then the process is fairly simple.

Compile and call this program like this:
   REMBLANKS <inputfile >outputfile

Not quite a batch script, but I can build you the EXE (for use in a batch script) if you need it.
#include <stdio.h>

#include <string.h>
 

int main(int argc,char *argv[])

{

        char Buff[512],TempBuff[512];

	int i;
 

	while(!feof(stdin))

	{

		/* Read file, one line at a time, make a copy of each line  */

		if(fgets(Buff,510,stdin)==NULL)

			break;

		strcpy(TempBuff,Buff);

                // Get rid of trailing CR and LF bytes

                for(i=strlen(TempBuff);i>0;i--)

                    if(TempBuff[i]=='\r' || TempBuff[i]=='\n')

                        TempBuff[i]='\0';

                    else

                        break;  

                //If there's anything left, output original string unchanged

		if(strlen(TempBuff))

			fprintf(stdout,"%s",Buff);

	}

        return(0);

}

Open in new window

0
 
LVL 38

Expert Comment

by:Shift-3
ID: 22776637
Can you upload a sample of a text file (using the Attach File checkbox) which causes your current script to fail?  How does it fail?

Would a vbscript solution be acceptable?
0
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 22776675
Try MUNGE http://www.ss64.com/nt/munge.html

Or use the 'tr' or 'sed' unix commands provided by www.cygwin.com

0
 

Author Comment

by:arundelr
ID: 22777104
Hi Experts,

Attached is an example of the file

I would be OK to use a VBS because I can just call it from the batch
web.txt
0
 
LVL 38

Expert Comment

by:Shift-3
ID: 22777944
Paste the script below into a text file with a .vbs extension.  Customize the value of the strInput variable with the location of a file to process.  Running the script will create an output file as defined in the strOutput variable.  

If this works correctly then additional code to handle arguments and multiple files can be added.  

If you test a file which does not process correctly, please upload the file along with a description of how it should work differently.


Const ForReading = 1

Const ForWriting = 2

Const TriStateUseDefault = -2
 

strInput = "web.txt"

strOutput = "weboutput.txt"
 

Set objFSO = CreateObject("Scripting.FileSystemObject")
 

Set objTextFile = objFSO.OpenTextFile(strInput, ForReading, False, TriStateUseDefault)

strText = objTextFile.ReadAll

strText = Replace(strText, vbCr, "")

objTextFile.Close
 

Set objTextFile = objFSO.OpenTextFile(strOutput, ForWriting, True)

objTextFile.Write strText

objTextFile.Close

Open in new window

0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 

Author Comment

by:arundelr
ID: 22778307
Hi Shift-3,

Thanks for your hard work, the only issue is that the file names are hard coded, what I need to be able to do is pass the inputfilename and the output path when I call the script

i.e. Currently I would use

process_feeds.bat "C:\inputfilename.txt" "C:\outputfilepath"

0
 
LVL 38

Expert Comment

by:Shift-3
ID: 22779128
Yes, that was just a proof of concept.  As I said, additional code to handle arguments and multiple files can be added.

Paste the script below into a text file with a .vbs extension.  Customize the value of the strSource variable with the default folder to search (equivalent to the source variable in your batch script).  Customize the value of the strDest variable with the default folder to search (equivalent to the dest variable).  

Running the script should now work in the same manner as your batch script, with or without arguments.


Const ForReading = 1

Const ForWriting = 2

Const TriStateUseDefault = -2
 

On Error Resume Next
 

strSource = "c:\files"

strDest = "c:\output"
 

Set objFSO = CreateObject("Scripting.FileSystemObject")
 

If WScript.Arguments.Count > 1 Then

    Process WScript.Arguments(0), WScript.Arguments(1)

Else

    Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")
 

    Set FileList = objWMIService.ExecQuery _

        ("ASSOCIATORS OF {Win32_Directory.Name='" & strSource & "'} Where " _

            & "ResultClass = CIM_DataFile")
 

    For Each objFile In FileList

        If objFile.Extension = "txt" Then

            Process objFile.Name, strDest

        End If

    Next

End If
 

Sub Process(strInput, strOutput)

    Set objTextFile = objFSO.OpenTextFile(strInput, ForReading, False, TriStateUseDefault)

    strText = objTextFile.ReadAll

    strText = Replace(strText, vbCr, "")

    objTextFile.Close

    

    Set objInput = objFSO.GetFile(strInput)

    strOutputFile = strOutput & "\" & objInput.Name

    Set objTextFile = objFSO.OpenTextFile(strOutputFile, ForWriting, True)

    objTextFile.Write strText

    objTextFile.Close

End Sub

Open in new window

0
 

Author Comment

by:arundelr
ID: 22796118
Hr Shift-3,

"As I said, additional code to handle arguments and multiple files can be added."
Ah, sorry I speed read your original notes ;-)

I have done as suggested and it does now behave correctly being called from the batch and setting the source/destinations

The only issue is that its stripping away all the CR

so the file starts

CR
X      0014116908 CR/LF
CR
1      000000000000000000CR/LF
CR
2      000000000000026301263551      HECR/LF
CR
3      000000 kgCR/LF
CR
3      0000000006 kgCR/LF

and ends up

X      0014116908 LF
1      000000000000000000 LF
2      000000000000026301263551      HE LF
3      000000 kg LF
3      0000000006 kg LF

and should be

X      0014116908 CR/LF
1      000000000000000000 CR/LF
2      000000000000026301263551      HECR/LF
3      000000 kg CR/LF
3      0000000006 kg LF

is it possible to change that ?





web.txt
0
 
LVL 28

Expert Comment

by:Bill Bach
ID: 22798181
FYI -- I did rebuild by RemFile tool to handle this for you, too, if you don't mind calling a separate application.  It is a free tool available from www.goldstarsoftware.com/tools.asp.  The command will be:
    REMFILE Blanks /E <inputfile >outputfile
0
 
LVL 38

Accepted Solution

by:
Shift-3 earned 500 total points
ID: 22803559
Ok, try this.


Const ForReading = 1

Const ForWriting = 2

Const TriStateUseDefault = -2

 

On Error Resume Next

 

strSource = "c:\files"

strDest = "c:\output"

 

Set objFSO = CreateObject("Scripting.FileSystemObject")

 

If WScript.Arguments.Count > 1 Then

    Process WScript.Arguments(0), WScript.Arguments(1)

Else

    Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")

 

    Set FileList = objWMIService.ExecQuery _

        ("ASSOCIATORS OF {Win32_Directory.Name='" & strSource & "'} Where " _

            & "ResultClass = CIM_DataFile")

 

    For Each objFile In FileList

        If objFile.Extension = "txt" Then

            Process objFile.Name, strDest

        End If

    Next

End If

 

Sub Process(strInput, strOutput)

    Set objTextFile = objFSO.OpenTextFile(strInput, ForReading, False, TriStateUseDefault)

    strText = objTextFile.ReadAll

    strText = Replace(strText, vbCr, "")

	strText = Replace(strText, vbLf, vbCrLf)

    objTextFile.Close

    

    Set objInput = objFSO.GetFile(strInput)

    strOutputFile = strOutput & "\" & objInput.Name

    Set objTextFile = objFSO.OpenTextFile(strOutputFile, ForWriting, True)

    objTextFile.Write strText

    objTextFile.Close

End Sub

Open in new window

0
 

Author Closing Comment

by:arundelr
ID: 31508691
Spot on Shift-3 - thanks ;o)
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
Introduction: Recently, I got a requirement to zip all files individually with batch file script in Windows OS. I don't know much about scripting, but I searched Google and found a lot of examples and websites to complete my task. Finally, I was ab…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now