Solved

sed -y and grep

Posted on 2013-06-05
6
470 Views
Last Modified: 2013-06-05
Here is  my shell script. I wanted to understand the following from the below shell

grep command -
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat

and

sed command

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

Since I am new to unix, could you please explain

opds:/export/home/opdu/cts/sh > cat cts_fltrt_intra.sh

#!/bin/ksh -xv
date
echo 'start loading  FLTRT_INTRA '
#

ORACLE_SID="$PAPP_SID"
export ORACLE_SID
echo "Started Deleting OMR Records in fltrt_intra \n"
sqlplus -s $appcts_user/$appcts_pass <<!
whenever oserror  exit failure
whenever sqlerror exit failure
set time on
set timing on
--
delete from fltrt_intra
where source_system_id = 1
/
commit
/
--
!
echo 'Deleting OMR Records is done in fltrt_intra'
date
echo 'start loading  fltrt_INTRA '
#
cd $tradehome/$ctsdata
unzip -o cts_fltrt_intra.zip
mv  cts_fltrt_intra.inx cts_fltrt_intra.dat
echo 'fltrt unzipped '

echo  $appcts_pass   | sqlldr cts@papp.composers.caxton.com  control=$tradehome/$ctsctl/cts_fltrt_intra.ctl log=$tradehome/$ctslog/cts_fltrt_intra.log
date
!
date
cd $tradehome/$ctslog
grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/cts/log/discard_fltrt_1.log

grep 'reject'    cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_discard.dat
sed   y/:/,/     $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt
sed  y/./,/      $tradehome/$ctsdata/cts_fltrt_discard.txt  > $tradehome/$ctsdata/cts_fltrt_discard.new
mv      $tradehome/$ctsdata/cts_fltrt_discard.new     $tradehome/$ctsdata/cts_fltrt_discard.dat
echo  $appcts_pass  | sqlldr cts@papp.composers.caxton.com control=$tradehome/$ctsctl/cts_fltrt_discard.ctl log=$tradehome/$ctslog/discard_fltrt_1.log
date
$tradehome/$ctssh/exception_check.sh 'fltrt_intra'
rm  $tradehome/$ctsdata/cts_fltrt_intra.dat
rm  $tradehome/$ctsdata/fltrt_discard.txt
rm  $tradehome/$ctsdata/fltrt_discard.dat
echo 'FLTRT loading Completed......'
date
opds:/export/home/opdu/cts/sh > ^C
opds:/export/home/opdu/cts/sh > cat $tradehome/$ctsctl/cts_fltrt_discard.ctl
load data
infile '$tradehome/$ctsdata/cts_fltrt_discard.dat'
append into table discard
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
   (
   table_name,
   file_ext,
   mes_txt,
   no_of_rec
   )

Open in new window

0
Comment
Question by:d27m11y
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 7

Expert Comment

by:msifox
ID: 39223101
grep 'discarded'
  lists only the lines that contain the word discarded

sed y/:/,/
  copies everything but replaces all colon signs with commas

Both read from the file that is specified as argument, and write their output to the file that is specified behind the > sign
0
 
LVL 7

Accepted Solution

by:
msifox earned 300 total points
ID: 39223120
examples:
if you have an input file like this:
a discarded
b included
c included
d discarded

and let the grep command process it, you get this as output:
a discarded
d discarded

If you have an input file like this
a:b:c
1:2:3

and let the sed command process it, you get this as output:
a,b,c
1,2,3
0
 

Author Comment

by:d27m11y
ID: 39223123
is there any difference between

sed "s/:/," $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt

sed   y/:/,/      $tradehome/$ctsdata/cts_fltrt_discard.dat > $tradehome/$ctsdata/cts_fltrt_discard.txt


Also I see that grep is looking for two files here

One is cts_fltrt_intra.log and the other is empty.fil, am confused, please suggest

grep 'discarded'  cts_fltrt_intra.log empty.fil > $tradehome/$ctsdata/cts_fltrt_intra_discard.dat
0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 7

Assisted Solution

by:msifox
msifox earned 300 total points
ID: 39223142
The y command replaces one character with another.
The s command is capable of replacing strings and doing regular expressions.
But these options are not used in this case, so both do the same.

If you specify more than one file as input for the grep command, it will scan both, one after the other, so you get all lines with 'discarded' from both files.
0
 

Author Closing Comment

by:d27m11y
ID: 39223147
Thank you for your help !
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39224021
The reason for havng "empty.fil" on the grep command line (presumably an empty file!) is that if you give grep one file name to search it just prints the matching lines - if you give it two or more, it precedes the line with the name of the file it found the line in.

So, "grep discarded input_file" might return
  line with discarded in it
  second line with discarded as one of the words

while "grep discarded input_file empty.fil" will return
  input_file: line with discarded in it
  input_file: second line with discarded as one of the words

This can make it easier to work out where data has come from later on.

BTW, I usually use /dev/null as the second file - it saves you haveing to create an empty file specially for this purpose.
0

Featured Post

Get HTML5 Certified

Want to be a web developer? You'll need to know HTML. Prepare for HTML5 certification by enrolling in July's Course of the Month! It's free for Premium Members, Team Accounts, and Qualified Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

628 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question