Solved

Linux Script to validate the file exists ?? and to filter the CR from the file.

Posted on 2015-01-21
12
490 Views
Last Modified: 2015-01-21
I get .txt file in my Linux. File is exported to oracle database through some oracle scripts.

I have to validate whether the file exists or not and if the file has any CR or CR LF,
i need to filter from the input .txt file.

I am very new to Linux Shell scripting and having a tough time to getting into it.
0
Comment
Question by:chokka
  • 6
  • 5
12 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40562658
Use the "-e" test to check if the file exists.
Use "cat -v filename" to display the CR characters as "^M" (if any) and use "grep -F" to find them.
Finally, use "tr -d" to delete the unwanted CR charactes ("\r"). I think you want to leave the linefeed characters (LF, "\n") in place, right?

FILE="/path/to/file.txt"
if [[ -e "$FILE" ]]; then
   if cat -v "$FILE" | grep -q -F "^M"; then
       tr -d '\r' < "$FILE" > "${FILE}.new"
   fi
fi

If the file exists and contains CR the the above will create a file "/path/to/file.txt.new".
If you're satisfied with the content of this new file rename it to its previous name, thus overwriting the old file:

FILE="/path/to/file.txt"
mv "${FILE}.new" "$FILE"

Note: I used quotation marks around the filenames to provide for the case that the names might contain spaces.
0
 

Author Comment

by:chokka
ID: 40562679
Thank you @woolmilkporc,

I am so thank ful. I am not pushing you hard.

However, i need to get this syntax in shape to run and make the file to work.

I have attached the Sample Data - TestData.txt ( This is how we get our Input Source File ).

On opening with Notepad++, I am able to see the CR in each line.

I dont need CR or LR. Yes , CR goes for \n

I can remove manually by replacing \n with empty .

I have attached the screenshot of Notepad++
D--LINUX-File-Validation-TESTDATA.TXT
C--Users-ZHQ8HPV-Desktop-TESTDATA-CR.jpg
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40562713
CR ("carriage return") is "\r", not "\n"! The latter is LF ("linefeed").

Your file contains Windows-style CR characters at some line ends. The code I gave you will remove these.

You said you're on Linux, so you can use this (shorter) code:

FILE="TESTDATA.txt"
if [[ -e "$FILE" ]]; then
   if grep -q $'\r' "$FILE"; then
      sed -i.bak 's/\r$//' "$FILE"
   fi
fi

The above will create an unchanged backup file TESTDATA.txt.bak and change TESTDATA.txt in place.
0
 

Author Comment

by:chokka
ID: 40562804
I ran the shell script but file is not changed.
Still i can see the CR.

Do i made any mistake in my script

I tried both by giving only file  /home/chokka/file.txt and filename as file.txt.
D--LINUX-File-Validation-TestFileValidat
D--LINUX-File-Validation-TestFilewithCR.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40562851
Did you run the script from the directory where the file "TestFilewithCR.TXT" is in?
You must provide the full absolute path (with "/" on the left) if the file is not in your currrent directory.

Are you aware that case matters in Linux? For example ".TXT" is not the same as ".txt"!
0
 

Author Comment

by:chokka
ID: 40562897
Yes, i changed the file path as

/home/chokka/TestFileValidation.sh

script creates another file as

TestFilewithCR.TXT.bak


When i checked both .bak file and original file, i still have the CR

How should i take off the CR ?
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40562918
Could you please post the output of

od -tcx TestFilewithCR.TXT
0
 

Author Comment

by:chokka
ID: 40563027
Thank you for your help, I have attached the output from running the script -

od -tcx TestFilewithCR.TXT


Please find the attachment in a notepad. It has \r
C--Users-ZHQ8HPV-Desktop-Output.txt
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 40563045
OK, we definitely have CR (\r) in that file.

Now we could try to find out why "grep" or "sed" or both obviously don't support
$'\r' or '\r$', but you could also try my first version.

FILE="/path/to/file.txt"
if [[ -e "$FILE" ]]; then
   if cat -v "$FILE" | grep -q -F "^M"; then
       tr -d '\r' < "$FILE" > "${FILE}.new"
   fi
fi

Please don't forget that the changed file gets a ".new" suffix with this code!

What is your OS? (uname -a; cat /proc/version), what is your shell (echo $0)?
0
 
LVL 48

Expert Comment

by:Tintin
ID: 40563197
Most Linux systems have the dos2unix command installed, so you could simply do

[ -e $FILE ] && dos2unix $FILE
0
 

Author Comment

by:chokka
ID: 40563210
Woolmilkporc,

Comment ID: 40563045 - Works

I am closing this posting. I have to do few more testing !!

I am so thank ful for your efforts !!
0
 

Author Closing Comment

by:chokka
ID: 40563570
Great !! Thanks !!
0

Featured Post

VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
1. Introduction As many people are interested in Linux but not as many are interested or knowledgeable (enough) to install Linux on their system, here is a safe way to try out Linux on your existing (Windows) system. The idea is that you insta…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now