Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Bash shell variable not passed into AWK reg expression

Posted on 2009-07-05
12
Medium Priority
?
870 Views
Last Modified: 2012-05-07
Hello,

I am able to succesfully pass a shell variable into a AWK reg expression  if the shell variable is explicitely defined. (In the code below if I uncomment  #pattern="INTEGER: 4"  everything works)  

However when the variable is implicitely defined (via the code : pattern=`awk ....)  [and  "pattern="INTEGER: 4" is commented]  I  am not getting any match.

Can this be done?

Thank you

PA

echo "Regular Expression:"
d="4/1"
pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }'`
echo $pattern # to check that the reg expression is formed correctly. This is working as expected.
 
#pattern="INTEGER: 4"
#  If I uncomment, the filter below works. If I leave it commented, the filter below does not return any match.
 
echo "Filter:"
awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' /home/pguanel/MIBDump
# I am using the variable "pattern" defined earlier to do my reg expression
 
 
!  The releavant section of the file "MIBDump" 
 
# Interface Description (ifDescr):
ifDescr.1 = STRING: Ethernet0/0
ifDescr.2 = STRING: GigabitEthernet0/0
ifDescr.3 = STRING: FastEthernet4/0
ifDescr.4 = STRING: FastEthernet4/1
ifDescr.6 = STRING: Null0
ifDescr.7 = STRING: E1 2/0
ifDescr.8 = STRING: E1 2/1
ifDescr.9 = STRING: E1 2/2
ifDescr.10 = STRING: E1 2/3
ifDescr.11 = STRING: E1 2/4
enterprises.9.9.166.1.1.1.1.4.68683 = INTEGER: 27
enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  
enterprises.9.9.166.1.1.1.1.4.103221 = INTEGER: 2
 
! Result should be : 102973

Open in new window

0
Comment
Question by:pierre-alex
  • 6
  • 4
  • 2
12 Comments
 
LVL 16

Expert Comment

by:Peter Kwan
ID: 24779357
1. Please check whether your files /home/pguanel/MIBDump  (line 10) and MIBDump (line 3) are referring to the same file.
2. Please see the following works:

awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' MIBDump



0
 

Author Comment

by:pierre-alex
ID: 24779375
pkwan, I have changed the path on line 10 as suggested, but no result.

All the files are in my  home directory (script and MIBDump file) so the discrepency did not matter here.

thank you

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779592
its better to show your input file as well as  show how your final output file will look like. From your code, i don't see where the comma delimiter is.
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 

Author Comment

by:pierre-alex
ID: 24779734
ghostdog74,

there is no comma delimiters in my input file, but since I am doing the reg expression on the first collumn any way , it does not matter.  

Here is the output when I remove the comment  on #pattern="INTEGER: 4". You can see that the bash variable is correctly used by the filter.

Regular Expression:
INTEGER: 4    # This is the result of the first Regular expresssion
Filter:
enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  # this is the result of the filter


Here is the output when I leave the comment

Regular Expression:
INTEGER: 4   # This is the result of the first Regular expresssion
Filter:         # nothing there ....


Input file and script attached as per your request.

It seems as though when the bash variable is the result of a previous awk, it does not work.

Also I tried to set the bash variable interractively using the "read" command,  it also worked correctly ...


PA
MIBDump
attempt02
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779862
i am not interested in how you get your results. I am only interested in how your actual output file will look like . ie what do you exactly want to get from your input file. Show it exactly.
0
 

Author Comment

by:pierre-alex
ID: 24780041
Hi ghostdog74

What I want to get is the value : 102973 as seen from line 28 of the snipet : enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  

The process should be as follows: start from a known interface description (here it is d$), then awk to find the index value of the interface  (here it is 4) , store that value into a variable after adding some characters (my variable is "pattern"), then use that variable in another reg expression to derive the final MIB index value (102973). ...

This needs to be done dynamically ...

Thanks

PA

0
 
LVL 16

Accepted Solution

by:
Peter Kwan earned 1000 total points
ID: 24780157
This is because you are getting a space at the end of the variable (you are trying to get 2 characters out of the string, but since "4" is a single character, the substr function pad an extra space at the end). Please try this:
pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }' | sed -e "s/ $//"`

Open in new window

0
 
LVL 9

Assisted Solution

by:ghostdog74
ghostdog74 earned 1000 total points
ID: 24780175
just do it with awk


awk 'BEGIN{
 pat="4/1"
 FS="[=: ]"
}
/ifDescr/ && $0~pat {
     gsub(/.*\./,"",$1)     
     pattern="INTEGER: "$1
     val=$1     
     f=1
}
f && /INTEGER/ && $NF==val {    
    m=split($1,t,".")
    print t[m]
    f=0
}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24781409
pkwan and ghostdog74, your solutions both work.

pkwan's solution is easier to understand given my novice level. Just a point of curiosity though, is there a way in shell to see spaces and other hidden characters  (like in word you can toggle a special mode to see spaces, end of line , carriage return etc ...) ?

ghostdog74, your solution looks very compact but a bit over my head (I am just starting to learn awk). If you could give me a brief explanation of your code to get me started I will be most grateful...

Anyway I am accepting both of your solutions.

Thanks again for your help.

PA
0
 

Author Comment

by:pierre-alex
ID: 24781431
Please see comments on the solution  above.

Many thanks

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24781702
@OP
Please read the GAwk manual on how awk works.


awk 'BEGIN{
 #initialize variables
 pat="4/1"
 #set field separator as "=" , ":" and space
 FS="[=: ]"
}
/ifDescr/ && $0~pat {
     # if ifDescr is found and contains "4/1"
     # substitute from start of line till the last "." of column 1
     gsub(/.*\./,"",$1)     
     pattern="INTEGER: "$1
     #val should be 4
     val=$1     
     f=1 #set flag to indicate that "4" is found
}
f && /INTEGER/ && $NF==val {    
    #if "found" and line contain the word INTEGER and the last column($NF) is "4"
    m=split($1,t,".") #split the first column on dots "."
    #then get the last element which is what you want
    print t[m]
    f=0
}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24783724
Thanks ghostdog74 !
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

by Batuhan Cetin Regular expression is a language that we use to edit a string or retrieve sub-strings that meets specific rules from a text. A regular expression can be applied to a set of string variables. There are many RegEx engines for u…
Do you hate spam? I do, and I am willing to bet you do as well. I often wonder, though, "if people hate spam so much, why do they still post their email addresses on the web?" I'm not talking about a plain-text posting here. I am referring to the fa…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

577 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question