Replace New Page Character in multiple text files - want something I can schedule

Note... the previous comment shows the @ symbol... the actual symbol is the symbol for "male"... a circle with a cross dangling straight down from it.  You can see that symbol in a command prompt if you hold down the ALT key and press the 1 key on the numeric keypad, release the 1, then press the 2, then release all keys.

I tried the following. Thought I could modify it to look for the character but may work only for text.

@echo off
   setlocal
   set SEARCHDIR=R:\LPIReporting\TempImportFiles
   set FILETYPE=*.txt
   set TRIGGER=TEST
   set OLDTEXT=?
   set NEWTEXT=
   ECHO "%OLDTEXT%" "%NEWTEXT% > Replace.ini
   FOR /F "usebackq" %%X IN (`findstr /S /I /M "%trigger%" "%SEARCHDIR%\%FILETYPE%"`)  DO Munge.exe R:\LPIReporting\TempImportFiles\Replace.ini %%X

Attached is a portion of the file. And yes, the symbol is as Jester states in command prompt window.

sorry to spam you... forgot to add that the sample file you attached had no formfeed characters in it.

This is a pure batch to strip From Feed char

Bye Gastone

@echo off
::: process each txt file in SourcePath directory
:: pure batch to remove formfeed character writing
:: updated file to TargetPath directory.
:: by Gastone Canali
setlocal
set SourcePath=C:\Documents and Settings\g
set TargetPath=c:\temp
cd /D %targetPath%   || goto :ERR
cd /D "%SourcePath%" || goto :ERR1
 
for /f "tokens=* delims=" %%F in ('dir /a-d /b "%SourcePath%\x*.txt"') do set file=%%F&call :parseFile
      
endlocal 
goto :EOF 
     
:parseFile
        type nul >"%TargetPath%\%file%" 2>nul
        for /f " tokens=*" %%a in ('type "%file%"') do set value=%%a&call :parseLine 
goto :eof
:: strip the from feed
::  the fromfeed char is in  between : and =
:parseLine
         set newvalue=%value:=%
         echo %newvalue% >>"%TargetPath%\%file%"
goto :eof
:ERR
echo Target dir not found!
:ERR1
echo Source dir not found!
:end
:eof

                                              

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:

2nd solution is really, really close. I appreciate your work. As I am not proficient with batch files, I am having a little trouble gettig my exact solution from your proposal. I just want to remove every instance of  while leaving the rest of the file intact. I have played with it, but cannot figure out how to isolate the character. Thanks so much.

RR345

Have you tried the script I provided?

Since it uses SED you can obtain it here:  http://www.physionet.org/physiotools/cygwin/

Install the CygWin utilities (there are many!), then either copy sed.exe to somewere in your path or add the CygWin\bin location to your path.  (I recommend the former just to be safe since I don't know what other executables you may already have on your machine and there could be a conflict between sam-named files).

Then, just run the script above and check the output.  As written, it never alters the original input files.

explain better what u want.
My script remove all FF...

These are really good suggestions. I am going on vacation for a week and will need to revisit when I return. Hang in there with me. Thanks

Jester, I downloaded the CygWin and copied everything accordingly including the script you kindly provided. What is happening now, is that I get entirely blank text files. I am reattaching one of the text files. Please look at the characters prior to the text AMSBC510. Looking forward to your respons.

The posted script worked perfectly for me.  It found/removed 3 occurrences of a FormFeed from the original file (size was 2658 bytes) and produced an output file without the 3 characters (size was 2655 bytes) but with all other data intact.

If you made any modifications to the script, please repost it so I can check it.

Thanks,
Lynn

Okay, here's the code--

::@echo off
:: process each txt file in SourcePath directory
:: via SED to remove formfeed character writing
:: updated file to TargetPath directory.
::
:: NOTE:  Bypasses any files which already exist in TargetPath
 
setlocal
set SourcePath=R:\LPIReporting\TempImportFiles
set TargetPath=R:\LPIReporting\TempImportFiles\Temp
 
for /f "tokens=* delims=" %%V in ('dir /a-d /b %SourcePath%\*.txt') do (
   if not exist %TargetPath%\%%V (
      sed "s/\x0C//g"  <%SourcePath%\%%V >%TargetPath%\%%V
   )
)

Maybe there is a problem with sed on my machine?

I got sed to run by placing all of the files in bin into the source path. So now I have data in my files. It is replacing other "hidden" characters that I do not want replaced as I want to keep the format. The attached files show what I mean SEDInput1 to SEDOutput1 is results using your code. Output2_ShouldBe are the results I want. Both Output files use the same Input file. Is this going to be possible. Thanks again.

The character being dropped is the "Carriage Return".  *nix uses only a Line Feed character to terminate lines while Windows/DOS uses a Carriage Return and Line Feed pair.

Are you using the version of SED from the Cygwin package?  It follows the WIndows convention while some other ports of SED follow the *nix convention.

>>> I got sed to run by placing all of the files in bin into the source path.  <<<

I'm not sure if that means you put all of Cygwin's bin folder files into your path or whether you are referring to just your data files.  The way my system is configured is that I have a "util" folder that is one of the earliest folders listed in my PATH environment variable.  Then, I only put the exe files I intend to use in the util folder.  Thus I only have SED, TEE, and a few other of the Cygwin files that are readily accessible along with some other utility programs from other sources.  I don't know if there might be some Cygwin file that is causing your copy to differ in behavior from my copy... I looked for a "configuration" file that might provide an option for line termination style preferences but did not find one.

I put the sed.exe in the file path but it kept looking for some .dlls in the bin file, so I copied everything to the file path. As far as I know, I am using SED from the Cygwin package - I used the link you provided. So when you run the same code, your file looks like Output1_ShouldBe.txt that I posted in my previous comment?

Here's the solution:
Add -b to provided code
 sed -b "s/\x0C//g"
--binary
This option is available on every platform, but is only effective where the operating system makes a distinction between text files and binary files. When such a distinction is madeas is the case for MS-DOS, Windows, Cygwintext files are composed of lines separated by a carriage return and a line feed character, and sed does not see the ending CR. When this option is specified, sed will open input files in binary mode, thus not requesting this special processing and considering lines to end at a line feed.

Thanks