Solved

Bash shell variable not passed into AWK reg expression

Posted on 2009-07-05
12
831 Views
Last Modified: 2012-05-07
Hello,

I am able to succesfully pass a shell variable into a AWK reg expression  if the shell variable is explicitely defined. (In the code below if I uncomment  #pattern="INTEGER: 4"  everything works)  

However when the variable is implicitely defined (via the code : pattern=`awk ....)  [and  "pattern="INTEGER: 4" is commented]  I  am not getting any match.

Can this be done?

Thank you

PA

echo "Regular Expression:"

d="4/1"

pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }'`

echo $pattern # to check that the reg expression is formed correctly. This is working as expected.
 

#pattern="INTEGER: 4"

#  If I uncomment, the filter below works. If I leave it commented, the filter below does not return any match.
 

echo "Filter:"

awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' /home/pguanel/MIBDump

# I am using the variable "pattern" defined earlier to do my reg expression
 
 

!  The releavant section of the file "MIBDump" 
 

# Interface Description (ifDescr):

ifDescr.1 = STRING: Ethernet0/0

ifDescr.2 = STRING: GigabitEthernet0/0

ifDescr.3 = STRING: FastEthernet4/0

ifDescr.4 = STRING: FastEthernet4/1

ifDescr.6 = STRING: Null0

ifDescr.7 = STRING: E1 2/0

ifDescr.8 = STRING: E1 2/1

ifDescr.9 = STRING: E1 2/2

ifDescr.10 = STRING: E1 2/3

ifDescr.11 = STRING: E1 2/4

enterprises.9.9.166.1.1.1.1.4.68683 = INTEGER: 27

enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  

enterprises.9.9.166.1.1.1.1.4.103221 = INTEGER: 2
 

! Result should be : 102973

Open in new window

0
Comment
Question by:pierre-alex
  • 6
  • 4
  • 2
12 Comments
 
LVL 16

Expert Comment

by:Peter Kwan
ID: 24779357
1. Please check whether your files /home/pguanel/MIBDump  (line 10) and MIBDump (line 3) are referring to the same file.
2. Please see the following works:

awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' MIBDump



0
 

Author Comment

by:pierre-alex
ID: 24779375
pkwan, I have changed the path on line 10 as suggested, but no result.

All the files are in my  home directory (script and MIBDump file) so the discrepency did not matter here.

thank you

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779592
its better to show your input file as well as  show how your final output file will look like. From your code, i don't see where the comma delimiter is.
0
 

Author Comment

by:pierre-alex
ID: 24779734
ghostdog74,

there is no comma delimiters in my input file, but since I am doing the reg expression on the first collumn any way , it does not matter.  

Here is the output when I remove the comment  on #pattern="INTEGER: 4". You can see that the bash variable is correctly used by the filter.

Regular Expression:
INTEGER: 4    # This is the result of the first Regular expresssion
Filter:
enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  # this is the result of the filter


Here is the output when I leave the comment

Regular Expression:
INTEGER: 4   # This is the result of the first Regular expresssion
Filter:         # nothing there ....


Input file and script attached as per your request.

It seems as though when the bash variable is the result of a previous awk, it does not work.

Also I tried to set the bash variable interractively using the "read" command,  it also worked correctly ...


PA
MIBDump
attempt02
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779862
i am not interested in how you get your results. I am only interested in how your actual output file will look like . ie what do you exactly want to get from your input file. Show it exactly.
0
 

Author Comment

by:pierre-alex
ID: 24780041
Hi ghostdog74

What I want to get is the value : 102973 as seen from line 28 of the snipet : enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  

The process should be as follows: start from a known interface description (here it is d$), then awk to find the index value of the interface  (here it is 4) , store that value into a variable after adding some characters (my variable is "pattern"), then use that variable in another reg expression to derive the final MIB index value (102973). ...

This needs to be done dynamically ...

Thanks

PA

0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 16

Accepted Solution

by:
Peter Kwan earned 250 total points
ID: 24780157
This is because you are getting a space at the end of the variable (you are trying to get 2 characters out of the string, but since "4" is a single character, the substr function pad an extra space at the end). Please try this:
pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }' | sed -e "s/ $//"`

Open in new window

0
 
LVL 9

Assisted Solution

by:ghostdog74
ghostdog74 earned 250 total points
ID: 24780175
just do it with awk


awk 'BEGIN{

 pat="4/1"

 FS="[=: ]"

}

/ifDescr/ && $0~pat {

     gsub(/.*\./,"",$1)     

     pattern="INTEGER: "$1

     val=$1     

     f=1

}

f && /INTEGER/ && $NF==val {    

    m=split($1,t,".")

    print t[m]

    f=0

}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24781409
pkwan and ghostdog74, your solutions both work.

pkwan's solution is easier to understand given my novice level. Just a point of curiosity though, is there a way in shell to see spaces and other hidden characters  (like in word you can toggle a special mode to see spaces, end of line , carriage return etc ...) ?

ghostdog74, your solution looks very compact but a bit over my head (I am just starting to learn awk). If you could give me a brief explanation of your code to get me started I will be most grateful...

Anyway I am accepting both of your solutions.

Thanks again for your help.

PA
0
 

Author Comment

by:pierre-alex
ID: 24781431
Please see comments on the solution  above.

Many thanks

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24781702
@OP
Please read the GAwk manual on how awk works.


awk 'BEGIN{

 #initialize variables

 pat="4/1"

 #set field separator as "=" , ":" and space

 FS="[=: ]"

}

/ifDescr/ && $0~pat {

     # if ifDescr is found and contains "4/1"

     # substitute from start of line till the last "." of column 1

     gsub(/.*\./,"",$1)     

     pattern="INTEGER: "$1

     #val should be 4

     val=$1     

     f=1 #set flag to indicate that "4" is found

}

f && /INTEGER/ && $NF==val {    

    #if "found" and line contain the word INTEGER and the last column($NF) is "4"

    m=split($1,t,".") #split the first column on dots "."

    #then get the last element which is what you want

    print t[m]

    f=0

}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24783724
Thanks ghostdog74 !
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
As most anyone who uses or has come across them can attest to, regular expressions (regex) are a complicated bit of magic. Packed so succinctly within their cryptic syntax lies a great deal of power. It's not the "take over the world" kind of power,…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now