Solved

Need help decyphering ksh script

Posted on 2013-06-07
27
502 Views
Last Modified: 2013-06-14
Here is a piece of ksh script I am trying to understand. I understand that the script reads a file line by line and puts the first part of each line into a variable,  whenever third part of each line equals to "Y".

I just have a few questions about parts of the script. Script is as follows:

cat $FILE1 | while read LINE

do
	
case $LINE in
	
\#*)	# Comment-Line in FILE1
	
;;
	
*) 	# Setup MYVAR 		
# if third field in FILE1 is 'Y'
        
if [ "`echo $LINE | awk -F: '{print $3}' -`" = "Y" ]
	
then
		
MYVAR=`echo $LINE | awk -F: '{print $1}' -`
		
if [ "$MYVAR" = '*' ]
		
then
			
MYVAR=""
		
fi
		

...

esac # End case $LINE

done    # End Loop read FILE1

Open in new window


some questions I have are:

1) in line 5 "case $LINE in", what does it check? Ususally, in other languages, case statement checks multiple cases if a variable equals to some value. here I do not completely understand it

2) In line 7 there is "\#*)" - what does it mean?

3) In line 9 - ";;" what are the two semicolons?

4) in line 11 "*)" ?

5) In line 14:

if [ "`echo $LINE | awk -F: '{print $3}' -`" = "Y" ]

what is the  ` in front of echo do, what is awk-F:
and why '{print $3}' cannot be replaced by just $3?

6) In line 20 what does the if statement check for?

if [ "$MYVAR" = '*' ]
      

I'd really appreciate some help, I am new to ksh scripting
0
Comment
Question by:YZlat
  • 13
  • 8
  • 4
  • +2
27 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 39230240
man ksh
...
       case word in [ [(]pattern [ | pattern ] ... ) list ;; ] ... esac
              A case command executes the list associated with the first pattern that matches word.  The form of the patterns  is
              the  same as that used for file-name generation (see File Name Generation below).  The ;; operator causes execution
              of case to terminate.  If ;& is used in place of ;; the next subsequent list, if any,  is executed.
...
   Command Substitution.
       The  standard  output  from a command enclosed in parentheses preceded by a dollar sign ( $() ) or a pair of grave accents
       (``) may be used as part or all of a word; trailing new-lines are removed.  In the  second  (obsolete)  form,  the  string
       between  the  quotes  is processed for special quoting characters before the command is executed (see Quoting below).  The
       command substitution $(cat file) can be replaced by the equivalent but faster $(<file).  The command  substitution  $(n<#)
       will expand to the current byte offset for file descriptor n.
0
 
LVL 84

Expert Comment

by:ozo
ID: 39230270
man awk
NAME
       awk - pattern-directed scanning and processing language

SYNOPSIS
       awk [ -F fs ] [ -v var=value ] [ 'prog' | -f progfile ] [ file ...  ]

DESCRIPTION
       Awk  scans  each  input  file  for lines that match any of a set of patterns specified literally in prog or in one or more
       files specified as -f progfile.  With each pattern there can be an associated action that will be performed when a line of
       a file matches the pattern.  Each line is matched against the pattern portion of every pattern-action statement; the asso-
       ciated action is performed for each matched pattern.  The file name - means the standard input.   Any  file  of  the  form
       var=value  is treated as an assignment, not a filename, and is executed at the time it would have been opened if it were a
       filename.  The option -v followed by var=value is an assignment to be done before prog  is  executed;  any  number  of  -v
       options may be present.  The -F fs option defines the input field separator to be the regular expression fs.

       An input line is normally made up of fields separated by white space, or by regular expression FS.  The fields are denoted
       $1, $2, ..., while $0 refers to the entire line.  If FS is null, the input line is split into one field per character.

       A pattern-action statement has the form

              pattern { action }

       A missing { action } means print the line; a missing pattern always matches.  Pattern-action statements are  separated  by
       newlines or semicolons.
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230295
ozo, could you tell me what exactly each line here does?

case $LINE in      
\#*)            
;;
0
 
LVL 84

Expert Comment

by:ozo
ID: 39230300
man [
NAME
     test, [ -- condition evaluation utility

SYNOPSIS
     test expression
     [ expression ]

DESCRIPTION
     The test utility evaluates the expression and, if it evaluates to true, returns a zero (true) exit status; otherwise it
     returns 1 (false).  If there is no expression, test also returns 1 (false).

     All operators and flags are separate arguments to the test utility.

     The following primaries are used to construct expressions:
...
     s1 = s2       True if the strings s1 and s2 are identical.
0
 
LVL 84

Expert Comment

by:ozo
ID: 39230368
case $LINE in  
\#*)            
;;
terminates the case statement if $LINE matches the pattern \#*
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230431
and what is this pattern?

based on help files I've read, \ makes following characters insignificant

# ia a comment and * I didn't find

what it the pattern "\#*"??

I have read the help file but it was not helpful enough to help me understand the code, that's why i asked the question here. please do not post anymore quotes from help files. Thank you
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230444
As I understand, if [ "$MYVAR" = '*' ] checks if any value is assigned to MYVAR. Is that correct? If so, why not just say if $MYVAR is not null?
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230450
Also I think I got this part:

case $LINE in
      
\#*)      # Comment-Line in FILE1
      
;;
      
*)

it checks if content of $LINE is a comment - contains #, where \ ensures that compiler does not take the rest of the line for a comment, and if the $LINE is, in fact a comment, it breaks out of the case statement.
*) means if it is anything else, but the comment
0
 
LVL 84

Expert Comment

by:ozo
ID: 39230467
Without quoting the documentation, I can't tell you that  *  matches  any string
If you read the documentation but still don't understand the code, that suggests that there could be something you didn't understand about * matches  any string.  
If so, we may be able to clarify if you can help us determine the nature of the misunderstanding.
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230471
Looks like I figured most of it out with the help of this tutorial:

http://b62.tripod.com/doc/docksh.htm

Now if you could only help me break this one down:

if [ "`echo $LINE | awk -F: '{print $3}' -`" = "Y" ]

echo $LINE would normally print contents of $LINE to the screen but that does not seem to be the case here

Also awk supposedly scans a file for a pattern but what is -F switch? man awk did not tell me much about switches, -F in particular

And why '{print $3}' -`" = "Y" is used to check if the third part of the string is "Y"? why not just use if[ $3 = "Y"] ??
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 168 total points
ID: 39230473
[ "$MYVAR" = '*' ]
is true if the strings "$MYVAR" and '*' are identical.
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 168 total points
ID: 39230490
in awk, an input line is normally made up of fields separated by white space, or by regular expression FS.
in this case, :
in the shell, $3 is the third argument given in the invocation of the shell.
in awk, $3 is the third field of the input line
0
 
LVL 35

Author Comment

by:YZlat
ID: 39230564
But why not just use $3="Y"??

And what does -F switch mean in awk command?
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 84

Assisted Solution

by:ozo
ozo earned 168 total points
ID: 39230585
Because [ $3 = "Y"] would the shells $3, not awk's $3

in awk, the -F fs option defines the input field separator
0
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 222 total points
ID: 39230877
That code is written in a very roundabout way.

The whole code can be replaced with:

grep "^\*:.*:Y:" $FILE1 | while read LINE
do
  ...
done  

Open in new window

0
 
LVL 35

Author Comment

by:YZlat
ID: 39234663
Tintin, what does the carrot do?

Also could you please explain to me what does

`echo $LINE

do in line: if [ "`echo $LINE | awk -F: '{print $3}' -`" = "Y" ]

and also why

if [ "$MYVAR" = '*' ]
            
then
                  
   MYVAR=""
            
fi

cannot e re-written as

if [ $MYVAR != ' ' ]
            
then
                  
   MYVAR=""
            
fi
0
 
LVL 35

Author Comment

by:YZlat
ID: 39235813
Tintin, I tried

grep "^\*:.*:Y" $FILE1 | while read LINE
do
     echo $LINE
done 

Open in new window


and didn't get any output at all

I also tried

grep "^\*:*:Y" $FILE1 | while read LINE
do
     echo $LINE
done 

Open in new window

and
grep "*:*:Y" $FILE1 | while read LINE
do
     echo $LINE
done 

Open in new window

but nothing gets returned

The exact format of the lines in the input file is

#DBName:/u01/app/oracle/product/11.2.0.3.0:Y
DBName2:/u01/app/oracle/product/11.2.0.3.0:N
DBName3:/u01/app/oracle/product/11.2.0.3.0:Y

from the above I only want

DBName3:/u01/app/oracle/product/11.2.0.3.0:Y

to be returned since it is not commented out and ends with "Y"
0
 
LVL 48

Accepted Solution

by:
Tintin earned 222 total points
ID: 39236152
Ah, I see what you are trying to do now.

Change the grep to

grep "^[A-Z]*:.*:Y"


The expression means match any line starting with a uppercase letter,  anything between the colons and ending with a 'Y'.
0
 
LVL 35

Author Comment

by:YZlat
ID: 39237547
Tintin, but it does not necsessary starts with an upper case
0
 
LVL 35

Author Comment

by:YZlat
ID: 39237550
I just need to ensure that it does not start with # hash sign

grep -v "^\#"  ensures the line does not start with # sign, but I also need the line to be in the format "dbname:path:Y"

grep "^[a-z]*:.*:Y" $FILE1

worked for me but only for lowercase and grep "^[A-Z]*:.*:Y" $FILE1
worked only for all uppercase. I need it to be case independent
0
 
LVL 27

Assisted Solution

by:serialband
serialband earned 55 total points
ID: 39238248
You should be able to specify both.

grep "^[a-zA-Z]*:.*:Y" $FILE1
0
 
LVL 35

Author Comment

by:YZlat
ID: 39238745
but why is

grep "^\*:.*:Y" $FILE1

not working?

and what is the purpose of the dot (.) in the string above, between : and *
0
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 222 total points
ID: 39239440
grep "^\*:.*:Y" $FILE1

means match any line starting with a *.

* by itself means zero or more matches of the previous expression
\* means literally match the * character


.* means zero or more matches of any character (specified by the .)
0
 
LVL 35

Author Comment

by:YZlat
ID: 39240836
Tintin, what would I use to match a line that can start with any character (except #) but contain colon twice and end with Y?

Example:  xxx:xxxxxx:Y

I tried

grep "^[!#*]:.*:Y" $FILE1

and

grep "^[!#]*:.*:Y" $FILE1

but neither worked
0
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 222 total points
ID: 39247084
grep "^[A-z]*:.*:Y" $FILE1
0
 
LVL 19

Assisted Solution

by:simon3270
simon3270 earned 55 total points
ID: 39248386
You use "^" within square brackets, not "!", to specify characters *not* to be found, so your last example would be:

     grep "^[^#]*:.*:Y" $FILE1

This would match the start of the line (the first ^), any number of not-hashes (the [^#]* bit), then the colon, any characters, colon and Y sequence).

This would match lines such as
    ::Y
If that is not acceptable, and you also only care about the first character being a hash, the pattern becomes:

     grep "^[^#].*:..*:Y" $FILE1
0
 
LVL 35

Author Closing Comment

by:YZlat
ID: 39248857
Thank you all!
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now