Solved

How to add lsof with pipe and awk to a array variable?

Posted on 2014-03-23
13
566 Views
Last Modified: 2014-04-02
I have to populate a array which will be commands and I would run them in a loop to execute the commands:

for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

Populating the command-array:

cmd_array=("cat /proc/${process_pid}/status")
cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

I get the following error when i execute the cmd_array[1]:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100

lsof: status error on |: No such file or directory
lsof: status error on awk: No such file or directory
lsof: status error on '{: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on =: No such file or directory
lsof: status error on $4: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on if: No such file or directory
lsof: status error on (: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on ==: No such file or directory
lsof: status error on mem: No such file or directory
lsof: status error on ): No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on print: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on }': No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on sort: No such file or directory
lsof: status error on -nrk: No such file or directory
lsof: status error on 7: No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on head: No such file or directory
lsof: status error on -100: No such file or directory

How to process lsof commands this way?
0
Comment
Question by:pvinodp
  • 6
  • 5
  • 2
13 Comments
 
LVL 62

Assisted Solution

by:gheist
gheist earned 167 total points
ID: 39949846
First try to not escape pipe ( | ) characters...
0
 

Author Comment

by:pvinodp
ID: 39949872
i do not have a backslash at the pipe!..
What do u mean by escaping pipe command.

cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

Above is my oroginal command.
But when i try to execute it by :
for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

I get the following error:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39949896
Try

eval ${i} >> $logFile

And wouldn't the user appear in column 3, thus requiring "$3" instead of "$4" in awk?

Next, you're missing (escaped) double quotes here:

if ( user == \"$fileType\" )

Strings must be quoted in awk so that they can be distinguished from variables.


cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$3 } { if ( user == \"$fileType\" ) { print } }' | sort -nrk 7 |  head -100"

eval ${cmd_array[1]}


In your (and the above) version process_pid and fileType must be set before populating the array. Is this desired?
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 

Author Comment

by:pvinodp
ID: 39950035
is there anything wrong or is it not the optmized way?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950047
Once the suggested corrections have been made there's nothing wrong anymore. Using "eval" the command should run as desired (see my note about populating variables, however).

To be able to ponder what the "optimitzed" way might be I'll need much more info what you're actually trying to achieve, and in which context.

I don't understand what this

cmd_array=("cat /proc/${process_pid}/status")

should be good for.
The statement will fill ${cmd_array[0]} with all the stuff from the status part of the process entry in /proc, which is not at all related to the lsof command you posted. ${cmd_array[0]} will not contain a command, so running it will always produce errors, even with "eval".
0
 

Author Comment

by:pvinodp
ID: 39950295
I am doing a performance analysis of a particular process id .

i need to execute "cat /proc/pid/status" and "lsof ****" .
how can i proceed?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950909
Do you want to run the analysis against a specific pid/user combo, or how should pid and user be selected?

What do you actually mean with "user" in your script? The username as displayed by lsof in column 3, or perhaps something different?
0
 

Author Comment

by:pvinodp
ID: 39951131
user is just a placeholder .. The variable can have any name.
i want to measure against particular pid
I am given the pid for whcih i have to measure.

The code to run the commands is comon. i have a array of commands for certain type of pids.
For one such pid there  is 2 commands:
1. cat /proc/pid/status
2. lsof ****

the actual problem is that i am unable to append the eval output to a file.
THat is my next question. Can someone answer with that perspective?
0
 
LVL 62

Expert Comment

by:gheist
ID: 39951146
You know that lsof can output just couple of fields without header?
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 333 total points
ID: 39951324
Sure, a variable name is just a placeholder, that's trivial.

What I actually wanted to ask for is the desired content of the variable, in a substantial perspective.

OK, you have a PID, that's good.

The posted "awk" would filter field # 4 of lsof's output against a variable whose name in the shell environment is "fileType" (setting it to "user" in the awk environment is not a problem).

Where in the script is this variable filled, and should it really be used as a filter against field # 4 which is "FD" (File Descriptor)?
FD can sometimes well be "mem" (yet followed by two more characters) as in the posted command in comment #39949872.

Anyway, I can't see why you should use arrays here, so there is no need for "eval" either:

process_pid=5573
fileType="mem"
logFile="/path/to/logfile.txt"

cat /proc/${process_pid}/status >> $logFile
lsof -s -p $process_pid | awk '{ user = $4 } { if ( user ~"'$fileType'" ) { print } }' | sort -nrk 7 |  head -100 >> $logFile

Note: Use double plus single quotes around "'$fileType'"!  
Here it's shown with spaces for illustration, but don't use the spaces: " ' $fileType ' "

Once the above has run you will find a file "/path/to/logfile.txt" which you can analyze whatever way.

Try both statements without redirection first ( omit ">> $logFile") so you can check the outcome on your terminal.
Reason: "mem" in the FD (field #4) means "memory-mapped file" and there might well be no such files held open by the process in question - so the "awk" output will be empty.
0
 

Author Comment

by:pvinodp
ID: 39952309
I changed the command from using a variable to use a static constant.
if ( user ~"'$fileType'" )  is changed to if ( user ~"mem" )

what is the difference between == and ~
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 333 total points
ID: 39952618
~ is for a pattern/regex match, == is for an exact match.

I used ~ because field 4 does not contain exactly "mem" but can contain something like "memw " or "memrR".

~"mem"  will match all of them.
0
 

Author Closing Comment

by:pvinodp
ID: 39971965
Thanks for your response
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Fine Tune your automatic Updates for Ubuntu / Debian
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question