Solved

How to add lsof with pipe and awk to a array variable?

Posted on 2014-03-23
13
579 Views
Last Modified: 2014-04-02
I have to populate a array which will be commands and I would run them in a loop to execute the commands:

for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

Populating the command-array:

cmd_array=("cat /proc/${process_pid}/status")
cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

I get the following error when i execute the cmd_array[1]:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100

lsof: status error on |: No such file or directory
lsof: status error on awk: No such file or directory
lsof: status error on '{: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on =: No such file or directory
lsof: status error on $4: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on if: No such file or directory
lsof: status error on (: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on ==: No such file or directory
lsof: status error on mem: No such file or directory
lsof: status error on ): No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on print: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on }': No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on sort: No such file or directory
lsof: status error on -nrk: No such file or directory
lsof: status error on 7: No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on head: No such file or directory
lsof: status error on -100: No such file or directory

How to process lsof commands this way?
0
Comment
Question by:pvinodp
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 2
13 Comments
 
LVL 62

Assisted Solution

by:gheist
gheist earned 167 total points
ID: 39949846
First try to not escape pipe ( | ) characters...
0
 

Author Comment

by:pvinodp
ID: 39949872
i do not have a backslash at the pipe!..
What do u mean by escaping pipe command.

cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

Above is my oroginal command.
But when i try to execute it by :
for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

I get the following error:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39949896
Try

eval ${i} >> $logFile

And wouldn't the user appear in column 3, thus requiring "$3" instead of "$4" in awk?

Next, you're missing (escaped) double quotes here:

if ( user == \"$fileType\" )

Strings must be quoted in awk so that they can be distinguished from variables.


cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$3 } { if ( user == \"$fileType\" ) { print } }' | sort -nrk 7 |  head -100"

eval ${cmd_array[1]}


In your (and the above) version process_pid and fileType must be set before populating the array. Is this desired?
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 

Author Comment

by:pvinodp
ID: 39950035
is there anything wrong or is it not the optmized way?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950047
Once the suggested corrections have been made there's nothing wrong anymore. Using "eval" the command should run as desired (see my note about populating variables, however).

To be able to ponder what the "optimitzed" way might be I'll need much more info what you're actually trying to achieve, and in which context.

I don't understand what this

cmd_array=("cat /proc/${process_pid}/status")

should be good for.
The statement will fill ${cmd_array[0]} with all the stuff from the status part of the process entry in /proc, which is not at all related to the lsof command you posted. ${cmd_array[0]} will not contain a command, so running it will always produce errors, even with "eval".
0
 

Author Comment

by:pvinodp
ID: 39950295
I am doing a performance analysis of a particular process id .

i need to execute "cat /proc/pid/status" and "lsof ****" .
how can i proceed?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950909
Do you want to run the analysis against a specific pid/user combo, or how should pid and user be selected?

What do you actually mean with "user" in your script? The username as displayed by lsof in column 3, or perhaps something different?
0
 

Author Comment

by:pvinodp
ID: 39951131
user is just a placeholder .. The variable can have any name.
i want to measure against particular pid
I am given the pid for whcih i have to measure.

The code to run the commands is comon. i have a array of commands for certain type of pids.
For one such pid there  is 2 commands:
1. cat /proc/pid/status
2. lsof ****

the actual problem is that i am unable to append the eval output to a file.
THat is my next question. Can someone answer with that perspective?
0
 
LVL 62

Expert Comment

by:gheist
ID: 39951146
You know that lsof can output just couple of fields without header?
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 333 total points
ID: 39951324
Sure, a variable name is just a placeholder, that's trivial.

What I actually wanted to ask for is the desired content of the variable, in a substantial perspective.

OK, you have a PID, that's good.

The posted "awk" would filter field # 4 of lsof's output against a variable whose name in the shell environment is "fileType" (setting it to "user" in the awk environment is not a problem).

Where in the script is this variable filled, and should it really be used as a filter against field # 4 which is "FD" (File Descriptor)?
FD can sometimes well be "mem" (yet followed by two more characters) as in the posted command in comment #39949872.

Anyway, I can't see why you should use arrays here, so there is no need for "eval" either:

process_pid=5573
fileType="mem"
logFile="/path/to/logfile.txt"

cat /proc/${process_pid}/status >> $logFile
lsof -s -p $process_pid | awk '{ user = $4 } { if ( user ~"'$fileType'" ) { print } }' | sort -nrk 7 |  head -100 >> $logFile

Note: Use double plus single quotes around "'$fileType'"!  
Here it's shown with spaces for illustration, but don't use the spaces: " ' $fileType ' "

Once the above has run you will find a file "/path/to/logfile.txt" which you can analyze whatever way.

Try both statements without redirection first ( omit ">> $logFile") so you can check the outcome on your terminal.
Reason: "mem" in the FD (field #4) means "memory-mapped file" and there might well be no such files held open by the process in question - so the "awk" output will be empty.
0
 

Author Comment

by:pvinodp
ID: 39952309
I changed the command from using a variable to use a static constant.
if ( user ~"'$fileType'" )  is changed to if ( user ~"mem" )

what is the difference between == and ~
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 333 total points
ID: 39952618
~ is for a pattern/regex match, == is for an exact match.

I used ~ because field 4 does not contain exactly "mem" but can contain something like "memw " or "memrR".

~"mem"  will match all of them.
0
 

Author Closing Comment

by:pvinodp
ID: 39971965
Thanks for your response
0

Featured Post

Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

695 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question