Solved

sed -y and grep

Posted on 2013-06-05
6
466 Views
Last Modified: 2013-06-05
Here is  my shell script. I wanted to understand the following from the below shell

grep command -
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat

and

sed command

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

Since I am new to unix, could you please explain

opds:/export/home/opdu/cts/sh > cat cts_fltrt_intra.sh

#!/bin/ksh -xv
date
echo 'start loading  FLTRT_INTRA '
#

ORACLE_SID="$PAPP_SID"
export ORACLE_SID
echo "Started Deleting OMR Records in fltrt_intra \n"
sqlplus -s $appcts_user/$appcts_pass <<!
whenever oserror  exit failure
whenever sqlerror exit failure
set time on
set timing on
--
delete from fltrt_intra
where source_system_id = 1
/
commit
/
--
!
echo 'Deleting OMR Records is done in fltrt_intra'
date
echo 'start loading  fltrt_INTRA '
#
cd $tradehome/$ctsdata
unzip -o cts_fltrt_intra.zip
mv  cts_fltrt_intra.inx cts_fltrt_intra.dat
echo 'fltrt unzipped '

echo  $appcts_pass   | sqlldr cts@papp.composers.caxton.com  control=$tradehome/$ctsctl/cts_fltrt_intra.ctl log=$tradehome/$ctslog/cts_fltrt_intra.log
date
!
date
cd $tradehome/$ctslog
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/cts/log/discard_fltrt_1.log

grep 'reject'    cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_discard.dat
sed   y/:/,/     $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/$ctslog/discard_fltrt_1.log
date
$tradehome/$ctssh/exception_check.sh 'fltrt_intra'
rm  $tradehome/$ctsdata/cts_fltrt_intra.dat
rm  $tradehome/$ctsdata/fltrt_discard.txt
rm  $tradehome/$ctsdata/fltrt_discard.dat
echo 'FLTRT loading Completed......'
date
opds:/export/home/opdu/cts/sh > ^C
opds:/export/home/opdu/cts/sh > cat $tradehome/$ctsctl/cts_fltrt_discard.ctl
load data
infile '$tradehome/$ctsdata/cts_fltrt_discard.dat'
append into table discard
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
   (
   table_name,
   file_ext,
   mes_txt,
   no_of_rec
   )

Open in new window

0
Comment
Question by:d27m11y
  • 3
  • 2
6 Comments
 
LVL 7

Expert Comment

by:msifox
ID: 39223101
grep 'discarded'
  lists only the lines that contain the word discarded

sed y/:/,/
  copies everything but replaces all colon signs with commas

Both read from the file that is specified as argument, and write their output to the file that is specified behind the > sign
0
 
LVL 7

Accepted Solution

by:
msifox earned 300 total points
ID: 39223120
examples:
if you have an input file like this:
a discarded
b included
c included
d discarded

and let the grep command process it, you get this as output:
a discarded
d discarded

If you have an input file like this
a:b:c
1:2:3

and let the sed command process it, you get this as output:
a,b,c
1,2,3
0
 

Author Comment

by:d27m11y
ID: 39223123
is there any difference between

sed "s/:/," $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt


Also I see that grep is looking for two files here

One is cts_fltrt_intra.log and the other is empty.fil, am confused, please suggest

grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
0
Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

 
LVL 7

Assisted Solution

by:msifox
msifox earned 300 total points
ID: 39223142
The y command replaces one character with another.
The s command is capable of replacing strings and doing regular expressions.
But these options are not used in this case, so both do the same.

If you specify more than one file as input for the grep command, it will scan both, one after the other, so you get all lines with 'discarded' from both files.
0
 

Author Closing Comment

by:d27m11y
ID: 39223147
Thank you for your help !
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39224021
The reason for havng "empty.fil" on the grep command line (presumably an empty file!) is that if you give grep one file name to search it just prints the matching lines - if you give it two or more, it precedes the line with the name of the file it found the line in.

So, "grep discarded input_file" might return
  line with discarded in it
  second line with discarded as one of the words

while "grep discarded input_file empty.fil" will return
  input_file: line with discarded in it
  input_file: second line with discarded as one of the words

This can make it easier to work out where data has come from later on.

BTW, I usually use /dev/null as the second file - it saves you haveing to create an empty file specially for this purpose.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question