?
Solved

sed -y and grep

Posted on 2013-06-05
6
Medium Priority
?
476 Views
Last Modified: 2013-06-05
Here is  my shell script. I wanted to understand the following from the below shell

grep command -
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat

and

sed command

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

Since I am new to unix, could you please explain

opds:/export/home/opdu/cts/sh > cat cts_fltrt_intra.sh

#!/bin/ksh -xv
date
echo 'start loading  FLTRT_INTRA '
#

ORACLE_SID="$PAPP_SID"
export ORACLE_SID
echo "Started Deleting OMR Records in fltrt_intra \n"
sqlplus -s $appcts_user/$appcts_pass <<!
whenever oserror  exit failure
whenever sqlerror exit failure
set time on
set timing on
--
delete from fltrt_intra
where source_system_id = 1
/
commit
/
--
!
echo 'Deleting OMR Records is done in fltrt_intra'
date
echo 'start loading  fltrt_INTRA '
#
cd $tradehome/$ctsdata
unzip -o cts_fltrt_intra.zip
mv  cts_fltrt_intra.inx cts_fltrt_intra.dat
echo 'fltrt unzipped '

echo  $appcts_pass   | sqlldr cts@papp.composers.caxton.com  control=$tradehome/$ctsctl/cts_fltrt_intra.ctl log=$tradehome/$ctslog/cts_fltrt_intra.log
date
!
date
cd $tradehome/$ctslog
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/cts/log/discard_fltrt_1.log

grep 'reject'    cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_discard.dat
sed   y/:/,/     $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/$ctslog/discard_fltrt_1.log
date
$tradehome/$ctssh/exception_check.sh 'fltrt_intra'
rm  $tradehome/$ctsdata/cts_fltrt_intra.dat
rm  $tradehome/$ctsdata/fltrt_discard.txt
rm  $tradehome/$ctsdata/fltrt_discard.dat
echo 'FLTRT loading Completed......'
date
opds:/export/home/opdu/cts/sh > ^C
opds:/export/home/opdu/cts/sh > cat $tradehome/$ctsctl/cts_fltrt_discard.ctl
load data
infile '$tradehome/$ctsdata/cts_fltrt_discard.dat'
append into table discard
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
   (
   table_name,
   file_ext,
   mes_txt,
   no_of_rec
   )

Open in new window

0
Comment
Question by:d27m11y
  • 3
  • 2
6 Comments
 
LVL 7

Expert Comment

by:msifox
ID: 39223101
grep 'discarded'
  lists only the lines that contain the word discarded

sed y/:/,/
  copies everything but replaces all colon signs with commas

Both read from the file that is specified as argument, and write their output to the file that is specified behind the > sign
0
 
LVL 7

Accepted Solution

by:
msifox earned 1200 total points
ID: 39223120
examples:
if you have an input file like this:
a discarded
b included
c included
d discarded

and let the grep command process it, you get this as output:
a discarded
d discarded

If you have an input file like this
a:b:c
1:2:3

and let the sed command process it, you get this as output:
a,b,c
1,2,3
0
 

Author Comment

by:d27m11y
ID: 39223123
is there any difference between

sed "s/:/," $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt


Also I see that grep is looking for two files here

One is cts_fltrt_intra.log and the other is empty.fil, am confused, please suggest

grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
0
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

 
LVL 7

Assisted Solution

by:msifox
msifox earned 1200 total points
ID: 39223142
The y command replaces one character with another.
The s command is capable of replacing strings and doing regular expressions.
But these options are not used in this case, so both do the same.

If you specify more than one file as input for the grep command, it will scan both, one after the other, so you get all lines with 'discarded' from both files.
0
 

Author Closing Comment

by:d27m11y
ID: 39223147
Thank you for your help !
0
 
LVL 20

Expert Comment

by:simon3270
ID: 39224021
The reason for havng "empty.fil" on the grep command line (presumably an empty file!) is that if you give grep one file name to search it just prints the matching lines - if you give it two or more, it precedes the line with the name of the file it found the line in.

So, "grep discarded input_file" might return
  line with discarded in it
  second line with discarded as one of the words

while "grep discarded input_file empty.fil" will return
  input_file: line with discarded in it
  input_file: second line with discarded as one of the words

This can make it easier to work out where data has come from later on.

BTW, I usually use /dev/null as the second file - it saves you haveing to create an empty file specially for this purpose.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you do backups in the Solaris Operating System, the file system must be inactive. Otherwise, the output may be inconsistent. A file system is inactive when it's unmounted or it's write-locked by the operating system. Although the fssnap utility…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

588 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question