Solved

sed -y and grep

Posted on 2013-06-05
6
464 Views
Last Modified: 2013-06-05
Here is  my shell script. I wanted to understand the following from the below shell

grep command -
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat

and

sed command

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

Since I am new to unix, could you please explain

opds:/export/home/opdu/cts/sh > cat cts_fltrt_intra.sh

#!/bin/ksh -xv
date
echo 'start loading  FLTRT_INTRA '
#

ORACLE_SID="$PAPP_SID"
export ORACLE_SID
echo "Started Deleting OMR Records in fltrt_intra \n"
sqlplus -s $appcts_user/$appcts_pass <<!
whenever oserror  exit failure
whenever sqlerror exit failure
set time on
set timing on
--
delete from fltrt_intra
where source_system_id = 1
/
commit
/
--
!
echo 'Deleting OMR Records is done in fltrt_intra'
date
echo 'start loading  fltrt_INTRA '
#
cd $tradehome/$ctsdata
unzip -o cts_fltrt_intra.zip
mv  cts_fltrt_intra.inx cts_fltrt_intra.dat
echo 'fltrt unzipped '

echo  $appcts_pass   | sqlldr cts@papp.composers.caxton.com  control=$tradehome/$ctsctl/cts_fltrt_intra.ctl log=$tradehome/$ctslog/cts_fltrt_intra.log
date
!
date
cd $tradehome/$ctslog
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/cts/log/discard_fltrt_1.log

grep 'reject'    cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_discard.dat
sed   y/:/,/     $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/$ctslog/discard_fltrt_1.log
date
$tradehome/$ctssh/exception_check.sh 'fltrt_intra'
rm  $tradehome/$ctsdata/cts_fltrt_intra.dat
rm  $tradehome/$ctsdata/fltrt_discard.txt
rm  $tradehome/$ctsdata/fltrt_discard.dat
echo 'FLTRT loading Completed......'
date
opds:/export/home/opdu/cts/sh > ^C
opds:/export/home/opdu/cts/sh > cat $tradehome/$ctsctl/cts_fltrt_discard.ctl
load data
infile '$tradehome/$ctsdata/cts_fltrt_discard.dat'
append into table discard
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
   (
   table_name,
   file_ext,
   mes_txt,
   no_of_rec
   )

Open in new window

0
Comment
Question by:d27m11y
  • 3
  • 2
6 Comments
 
LVL 7

Expert Comment

by:msifox
ID: 39223101
grep 'discarded'
  lists only the lines that contain the word discarded

sed y/:/,/
  copies everything but replaces all colon signs with commas

Both read from the file that is specified as argument, and write their output to the file that is specified behind the > sign
0
 
LVL 7

Accepted Solution

by:
msifox earned 300 total points
ID: 39223120
examples:
if you have an input file like this:
a discarded
b included
c included
d discarded

and let the grep command process it, you get this as output:
a discarded
d discarded

If you have an input file like this
a:b:c
1:2:3

and let the sed command process it, you get this as output:
a,b,c
1,2,3
0
 

Author Comment

by:d27m11y
ID: 39223123
is there any difference between

sed "s/:/," $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt


Also I see that grep is looking for two files here

One is cts_fltrt_intra.log and the other is empty.fil, am confused, please suggest

grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 7

Assisted Solution

by:msifox
msifox earned 300 total points
ID: 39223142
The y command replaces one character with another.
The s command is capable of replacing strings and doing regular expressions.
But these options are not used in this case, so both do the same.

If you specify more than one file as input for the grep command, it will scan both, one after the other, so you get all lines with 'discarded' from both files.
0
 

Author Closing Comment

by:d27m11y
ID: 39223147
Thank you for your help !
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39224021
The reason for havng "empty.fil" on the grep command line (presumably an empty file!) is that if you give grep one file name to search it just prints the matching lines - if you give it two or more, it precedes the line with the name of the file it found the line in.

So, "grep discarded input_file" might return
  line with discarded in it
  second line with discarded as one of the words

while "grep discarded input_file empty.fil" will return
  input_file: line with discarded in it
  input_file: second line with discarded as one of the words

This can make it easier to work out where data has come from later on.

BTW, I usually use /dev/null as the second file - it saves you haveing to create an empty file specially for this purpose.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

943 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

5 Experts available now in Live!

Get 1:1 Help Now