Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Bash shell variable not passed into AWK reg expression

Posted on 2009-07-05
12
Medium Priority
?
865 Views
Last Modified: 2012-05-07
Hello,

I am able to succesfully pass a shell variable into a AWK reg expression  if the shell variable is explicitely defined. (In the code below if I uncomment  #pattern="INTEGER: 4"  everything works)  

However when the variable is implicitely defined (via the code : pattern=`awk ....)  [and  "pattern="INTEGER: 4" is commented]  I  am not getting any match.

Can this be done?

Thank you

PA

echo "Regular Expression:"
d="4/1"
pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }'`
echo $pattern # to check that the reg expression is formed correctly. This is working as expected.
 
#pattern="INTEGER: 4"
#  If I uncomment, the filter below works. If I leave it commented, the filter below does not return any match.
 
echo "Filter:"
awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' /home/pguanel/MIBDump
# I am using the variable "pattern" defined earlier to do my reg expression
 
 
!  The releavant section of the file "MIBDump" 
 
# Interface Description (ifDescr):
ifDescr.1 = STRING: Ethernet0/0
ifDescr.2 = STRING: GigabitEthernet0/0
ifDescr.3 = STRING: FastEthernet4/0
ifDescr.4 = STRING: FastEthernet4/1
ifDescr.6 = STRING: Null0
ifDescr.7 = STRING: E1 2/0
ifDescr.8 = STRING: E1 2/1
ifDescr.9 = STRING: E1 2/2
ifDescr.10 = STRING: E1 2/3
ifDescr.11 = STRING: E1 2/4
enterprises.9.9.166.1.1.1.1.4.68683 = INTEGER: 27
enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  
enterprises.9.9.166.1.1.1.1.4.103221 = INTEGER: 2
 
! Result should be : 102973

Open in new window

0
Comment
Question by:pierre-alex
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
  • 2
12 Comments
 
LVL 16

Expert Comment

by:Peter Kwan
ID: 24779357
1. Please check whether your files /home/pguanel/MIBDump  (line 10) and MIBDump (line 3) are referring to the same file.
2. Please see the following works:

awk -F',' -v test2="$pattern" '$0 ~ test2 { print $0 }' MIBDump



0
 

Author Comment

by:pierre-alex
ID: 24779375
pkwan, I have changed the path on line 10 as suggested, but no result.

All the files are in my  home directory (script and MIBDump file) so the discrepency did not matter here.

thank you

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779592
its better to show your input file as well as  show how your final output file will look like. From your code, i don't see where the comma delimiter is.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:pierre-alex
ID: 24779734
ghostdog74,

there is no comma delimiters in my input file, but since I am doing the reg expression on the first collumn any way , it does not matter.  

Here is the output when I remove the comment  on #pattern="INTEGER: 4". You can see that the bash variable is correctly used by the filter.

Regular Expression:
INTEGER: 4    # This is the result of the first Regular expresssion
Filter:
enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  # this is the result of the filter


Here is the output when I leave the comment

Regular Expression:
INTEGER: 4   # This is the result of the first Regular expresssion
Filter:         # nothing there ....


Input file and script attached as per your request.

It seems as though when the bash variable is the result of a previous awk, it does not work.

Also I tried to set the bash variable interractively using the "read" command,  it also worked correctly ...


PA
MIBDump
attempt02
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24779862
i am not interested in how you get your results. I am only interested in how your actual output file will look like . ie what do you exactly want to get from your input file. Show it exactly.
0
 

Author Comment

by:pierre-alex
ID: 24780041
Hi ghostdog74

What I want to get is the value : 102973 as seen from line 28 of the snipet : enterprises.9.9.166.1.1.1.1.4.102973 = INTEGER: 4  

The process should be as follows: start from a known interface description (here it is d$), then awk to find the index value of the interface  (here it is 4) , store that value into a variable after adding some characters (my variable is "pattern"), then use that variable in another reg expression to derive the final MIB index value (102973). ...

This needs to be done dynamically ...

Thanks

PA

0
 
LVL 16

Accepted Solution

by:
Peter Kwan earned 1000 total points
ID: 24780157
This is because you are getting a space at the end of the variable (you are trying to get 2 characters out of the string, but since "4" is a single character, the substr function pad an extra space at the end). Please try this:
pattern=`awk '/ifDescr/ {print $0}' MIBDump |  awk -F',' -v test1="$d" '$0 ~ test1 { print "INTEGER: " substr($1,9,2) }' | sed -e "s/ $//"`

Open in new window

0
 
LVL 9

Assisted Solution

by:ghostdog74
ghostdog74 earned 1000 total points
ID: 24780175
just do it with awk


awk 'BEGIN{
 pat="4/1"
 FS="[=: ]"
}
/ifDescr/ && $0~pat {
     gsub(/.*\./,"",$1)     
     pattern="INTEGER: "$1
     val=$1     
     f=1
}
f && /INTEGER/ && $NF==val {    
    m=split($1,t,".")
    print t[m]
    f=0
}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24781409
pkwan and ghostdog74, your solutions both work.

pkwan's solution is easier to understand given my novice level. Just a point of curiosity though, is there a way in shell to see spaces and other hidden characters  (like in word you can toggle a special mode to see spaces, end of line , carriage return etc ...) ?

ghostdog74, your solution looks very compact but a bit over my head (I am just starting to learn awk). If you could give me a brief explanation of your code to get me started I will be most grateful...

Anyway I am accepting both of your solutions.

Thanks again for your help.

PA
0
 

Author Comment

by:pierre-alex
ID: 24781431
Please see comments on the solution  above.

Many thanks

PA
0
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24781702
@OP
Please read the GAwk manual on how awk works.


awk 'BEGIN{
 #initialize variables
 pat="4/1"
 #set field separator as "=" , ":" and space
 FS="[=: ]"
}
/ifDescr/ && $0~pat {
     # if ifDescr is found and contains "4/1"
     # substitute from start of line till the last "." of column 1
     gsub(/.*\./,"",$1)     
     pattern="INTEGER: "$1
     #val should be 4
     val=$1     
     f=1 #set flag to indicate that "4" is found
}
f && /INTEGER/ && $NF==val {    
    #if "found" and line contain the word INTEGER and the last column($NF) is "4"
    m=split($1,t,".") #split the first column on dots "."
    #then get the last element which is what you want
    print t[m]
    f=0
}' MIBDump

Open in new window

0
 

Author Comment

by:pierre-alex
ID: 24783724
Thanks ghostdog74 !
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This Windows batch file is useful for organizing image files from a digital camera or other source, but can have many other uses.  It simply renames the file(s) to match their create date.  For example, if you took a picture today at 1:40pm and the …
We are witnesses that everyone is saying that our children shouldn't "play" with a technology because it is dangerous. This article is going to prove that they are wrong.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

721 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question