Solved

How to interprete this grep pattern?

Posted on 2000-03-31
20
594 Views
Last Modified: 2012-06-27
# ps -ef | grep -E '[/ ]rds$|[/ ]rds'
    root  1260     1  0  Mar 11  ?        160:03 /opt/omni/lbin/rds -d

 What does it mean by [/ ] and | inside the grep pattern?

If I use the following command, then I will get different result,
# ps -ef|grep rds
    root  1171     1  0  Mar 20  ?        699:19 /opt/omni/lbin/rds -d
    root 25634 25071  2 17:15:43 ttyp3     0:00 grep rds
0
Comment
Question by:kslzzg
  • 7
  • 6
  • 3
  • +2
20 Comments
 
LVL 40

Accepted Solution

by:
jlevie earned 20 total points
Comment Utility
The "[]" are used to specify a set of characters, any one of which is a valid match. Your example starts with [/ ], which says that the first char can be either a "/" or a space. The "|" is an alternation flag that joins two regular expressions allowing either of the subexpressions to match. So what we've got are two regex's.  The first matches a string anchored at the end of the line (that's what "$" does) that has either a space or a "/" follwed by "rds". The second matches the same string but isn't anchored to the end of the line.
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
Umm...I was trying to work this one out, but I got confused by the fact the "[/ ]rds" construct wasn't matching "grep rds" as well.

Care to expand on that one, Jim? I may even invest some of my own points for the education ;)
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
"[/ ]rds" does match "grep rds", but when `ps -ef | grep -E '[/ ]rds$|[/ ]rds'` is executed, there is no  "grep rds" to match
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
The regex grep won't match "grep rds" because that pattern won't appear anywhere in the output of ps -ef. If you "tee'd" the operation you'd see "grep -E '[/ ]rds$|[/ ]rds'" in the process list and you can see that there isn't "/rds" or " rds" in the "grep" command to match. The [/ ] match operator says that either of the characters can preceed "rds", but that one of them must be present.

The second form of the command "ps -ef | grep rds" will show both the line from the process list containg the invocation of "rds" and the grep command because "rds" appears as a process in the list and in the "grep command in the process list.

The cheap way to get rid of the "grep line" is to use "ps -ef | grep rds | grep -v grep". The last part of the pipe tells grep to discard any matches of "grep". I've used that for years either as an alias or as a shell function (bash). It looks like:

psg () { ps -ef | fgrep $1 | fgrep -v fgrep; }

allowing me to to psg some-prog to get the line(s) where "some-prog" occurs, or to do "psg some-user" to find out everything run by "some-user"

Does that explain it?
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Before anybody gets excited... no I didn't "steal" ozo's comment. I started composing my last at about 12:30, but what with interruptions I didn't get to finish it until 2:20
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
ozo/jlevie:It's been a long week - Thanks for being gentle with me.

Now you just have to collect your rewards elsewhere in this topic area.

kslzzg: Sorry for hijacking your question - I hope you got something out of it too ;)

0
 

Author Comment

by:kslzzg
Comment Utility
I still can't understand why the pattern doesn't
 match "grep rds".

I tried "ps -ef | grep -E '[/ ]rds' " and I also got the
 same result as the first one.  If [/ ] means either
 "/" or space, then when it choose space, there
should be a match for "grep rds",  right?  

I can also gain the same result if I run this command,
# ps -ef | grep -E '[/]rds$|[/]rds'
    root  1260     1  0  Mar 11  ?        169:12 /opt/omni/lbin/rds -d


I think it is really a trick confusing many people.
  The riddle here is how come there is one line output  for  the first pattern,   but there are two line output including "grep" process  for the
 second patten.    
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Let me see if I can't make it a little more clear as to why you get only one match one time, two matches another time, apparently the same result the last time. I have to use dummy data to illustrate it, but the meaning should be clear.

Lets take the first case from the original question, that of "ps -ef | grep -E '[/ ]rds$|[/ ]rds'". While that command is running, the process list contains (amoung all the other processes):

root      1260     1  0 Mar 11  ?        169:12 /opt/omni/lbin/rds -d
levie     1007   933  0 22:31 pts/0    00:00:00 ps -ef
levie     1010   933  0 22:31 pts/0    00:00:00 grep -E '[/ ]rds$|[/ ]rds'

You'll notice that the grep command and it's arguments are there but you don't see "/rds" or " rds" in the process line for the grep command. And we've told grep that it can match either of those two patterns (but only those patterns), so it doesn't output the line for process ID 1010.

However, in the second case, "ps -ef | grep rds" the process list would like:

root      1260     1  0 Mar 11  ?        169:12 /opt/omni/lbin/rds -d
levie     1007   933  0 22:31 pts/0    00:00:00 ps -ef
levie     1010   933  0 22:31 pts/0    00:00:00 grep rds

This time grep has been told to match "rds" and it occurs twice in that process list. Once for the process ID 1007 and once for process ID 1010, thus two lines of output, one for process ID 1007 and one for 1010.

And finally taking the last case from your last comment, that of "ps -ef | grep -E '[/]rds$|[/]rds'", which seems to give the same result, but won't give the correct result in all cases. Suppose I had done:

root> cd /opt/omni/lbin
root> rds -d

The process list while running the example that omitted the space characters would look like:

root      1260     1  0 Mar 11  ?        169:12 /opt/omni/lbin/rds -d
root      1360     1  0 Mar 11  ?         10:13 rds -d
levie     1007   933  0 22:31 pts/0    00:00:00 ps -ef
levie     1010   933  0 22:31 pts/0    00:00:00 grep -E '[/]rds$|[/]rds'

With the space characters omitted from the regex expression, grep has been told to match only "/rds". Notice that it would still match the data in process ID 1260, but would miss the rds instance in process 1360 because there's not "/" directly in front of "rds". There is a space preceeding rds and the first regex expression would have matched it, producing two lines of output, one for process ID 1260 and one for 1360.

Does that help?
0
 

Author Comment

by:kslzzg
Comment Utility

But how come I don't see the line
"levie     1010   933  0 22:31 pts/0    00:00:00 grep -E '[/]rds$|[/]rds' "  
as you mentioned above in the output of the command "ps -ef|grep -E '[/]rds$|[/]rds " ?

For example,

# ps -ef|grep -E  "[/ ]rds$|[/ ]rds"
    root  1260     1  0  Mar 11  ?        186:58 /opt/omni/lbin/rds -d
#

Why there is no grep line in the previous process-list ?
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
because the "rds" in "levie     1010   933  0 22:31 pts/0    00:00:00 grep -E '[/]rds$|[/]rds' "
is preceded by a "]" which is not a " " or a "/"
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 20

Expert Comment

by:tfewster
Comment Utility
ozo: I guess I could have saved myself a few points, just by waiting for  kslzzg to ask for clarification - Never mind, your points Q is here if you want them :

http://www1.experts-exchange.com/bin/Q.10321042
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Take a look careful look at the first example from my last comment, keeping ozo's comment in mind. His explanation is very direct and quite to the point.
0
 

Author Comment

by:kslzzg
Comment Utility
Let me clarify my doubt.
As we can expect,
# ps -ef|grep rds
    root  1260     1  0  Mar 11  ?        196:15 /opt/omni/lbin/rds -d
    root 13558 13542  1 09:24:20 ttyp2     0:00 grep rds
#

Similarly, we expect the output as below,
# ps -ef|grep -E  "[/ ]rds$|[/ ]rds"
    root  1260     1  0  Mar 11  ?        196:15 /opt/omni/lbin/rds -d
    root 13559 13542  1 09:24:20 ttyp2     0:00 grep -E '[/ ]rds$|[/ ]rds'
#

But actually it is wrong, we don't get the second line which is the process "grep -E '[/ ]rds$|[/ ]rds' " ,  why?
 From my understanding, and also from what jlevie
 has mentioned earlier, there should be a process
 called "grep -E ..."  during the time when you run
 the command "ps -ef|grep -E ...",  right?

 So what's the reason behind the actual output?

0
 

Author Comment

by:kslzzg
Comment Utility
when `ps -ef | grep -E '[/ ]rds$|[/ ]rds'` is
                    executed, there is no  "grep rds" to match, it is true.   But there should be "grep -E '[/ ]rds$|[/ ]rds'" to match,  right?
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Yes the "grep" process is there while the command is running. You won't normally be able to see it because it will complete too quickly. Since nothing in the "grep" line matches the grep regex search pattern the command doesn't output the process line for grep. It does output the process line for rds becae it does match the regex pattern.

And yes there is a "grep -E '[/ ]rds$|[/ ]rds'" that could be matched, but only if we told grep to look for something that's in that string.


0
 

Author Comment

by:kslzzg
Comment Utility
It is rather funny  as you see the output below,

# ps -ef|grep grep
    root 28107 27786  1 14:21:28 ttyp2     0:00 grep grep
#
# ps -ef|grep -E '[/ ]grep$|[/ ]grep'
    root 28039 27786  1 14:20:28 ttyp2     0:00 grep -E [/ ]grep$|[/ ]grep

    Just as you mentioned above, I told grep to grep "grep" and it did work.  It goes by  the same rule for grep to grep "grep -E '[/ ]grep$|[/ ]grep' " itself.   It this case, do you think the grep regex search pattern still works?   It seems rather strange  to me that the grep process can match itself and  it does not go into an endless loop searching for the grep process itself,  why?
 
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Yes the regex pattern still worked. If it hadn't, it would not have have found a match and there wouldn't have been any output (it matched " grep" in this case).

Well it can't go into an endless loop. There's only one copy of the executable running and all it's doing is looking at each line data being presented and deciding whether the search pattern is present in that data or not. If the search pattern is present, it emits the line to stdout, if not it goes to the next line.
0
 
LVL 4

Expert Comment

by:Nazdor
Comment Utility
Maybe a new perspective can help - but I'm not saying anything new here...just explaining it a different way.

First off: "and it does not go into an endless loop ".  Remember that 'ps' takes a single 'snapshot' of the processes, it doesn't keep looking.  If you think of it as 'ps' effectively executing FIRST, then the output passed into the grep command, it's a bit like doing:
  ps -ef > tmp.1
  cat tmp.1 | grep ps
(but, of course, this wont find the grep) (it's not exactly like that, but similar, which is why it wont go into a loop - the grep can't pass anything 'back' to the ps to tell it to start again).


To answer kslzzg's comment at 2:30: Similarly, we expect the output as below,
# ps -ef|grep -E  "[/ ]rds$|[/ ]rds"
    root  1260     1  0  Mar 11  ?        196:15 /opt/omni/lbin/rds -d
    root 13559 13542  1 09:24:20 ttyp2     0:00 grep -E '[/ ]rds$|[/ ]rds'
#

But actually it is wrong, we don't get the second line which is the process "grep -E '[/ ]rds$|[/ ]rds' " ,  why?
---

The regexp in the grep says to match one of the following:
  /rds$
  <space>rds$
  /rds
  <space>rds

But the line you think is 'missing' has ']rds' in, NOT '/rds' or '<space>rds', so it wont show that line because none of the 4 expressions actually appears in that line.

So you wouldn't 'expect' to find the grep line and... you don't :)

If you need to show the grep, then you need to look for something that is in the command line, ie:

# ps -ef|grep -E  "[]/ ]rds$|[]/ ]rds"

Just a quick explanation of []/ ] - normally, to use an alternative you put it in [], eg [ab] (either a or b) - to say 'either a or ]' you make sure the ] appears straight after the opening [, ie:  []a]

Now it is searching for
   ]rds
   /rds
or
  <space>rds
which will match "[]/ ]rds"

Incidently, the '$' part is irrelevant here because /rds matches /rds$ and <space>rds matches <space>rds$.


Another way to look at it:  As '|' means "or" and '[/ ]' also means "or", you could write "[/ ]rds" as " rds|/rds", or with '$' if you need it:

# ps -ef|grep -E  " rds$|/rds$| rds|/rds"

Now " rds" appears in the grep command, because there are no square brackets in there.


To summarise, assuming you do want the grep command in (which most people don't) then use either:

# ps -ef|grep -E  "[]/ ]rds$|[]/ ]rds"
or
# ps -ef|grep -E  " rds$|/rds$| rds|/rds"


Hope that helps.
0
 

Author Comment

by:kslzzg
Comment Utility
I got it.    Thank you very much.
0
 
LVL 84

Expert Comment

by:ozo
Comment Utility
which leads to the common trick of using
ps -ef | grep '[s]tring'
instead of
ps -ef | grep 'string'
when you don't want the grep to match itself.

and by the way, `grep -E '[/ ]rds$|[/ ]rds'`
could be written more simply as `grep -E '[/ ]rds'`
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now