Solved

How to add lsof with pipe and awk to a array variable?

Posted on 2014-03-23
13
558 Views
Last Modified: 2014-04-02
I have to populate a array which will be commands and I would run them in a loop to execute the commands:

for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

Populating the command-array:

cmd_array=("cat /proc/${process_pid}/status")
cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

I get the following error when i execute the cmd_array[1]:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100

lsof: status error on |: No such file or directory
lsof: status error on awk: No such file or directory
lsof: status error on '{: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on =: No such file or directory
lsof: status error on $4: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on if: No such file or directory
lsof: status error on (: No such file or directory
lsof: status error on user: No such file or directory
lsof: status error on ==: No such file or directory
lsof: status error on mem: No such file or directory
lsof: status error on ): No such file or directory
lsof: status error on {: No such file or directory
lsof: status error on print: No such file or directory
lsof: status error on }: No such file or directory
lsof: status error on }': No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on sort: No such file or directory
lsof: status error on -nrk: No such file or directory
lsof: status error on 7: No such file or directory
lsof: status error on |: No such file or directory
lsof: status error on head: No such file or directory
lsof: status error on -100: No such file or directory

How to process lsof commands this way?
0
Comment
Question by:pvinodp
  • 6
  • 5
  • 2
13 Comments
 
LVL 61

Assisted Solution

by:gheist
gheist earned 167 total points
ID: 39949846
First try to not escape pipe ( | ) characters...
0
 

Author Comment

by:pvinodp
ID: 39949872
i do not have a backslash at the pipe!..
What do u mean by escaping pipe command.

cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$4 } { if ( user == $fileType ) { print } }' | sort -nrk 7 |  head -100"

Above is my oroginal command.
But when i try to execute it by :
for i in "${cmd_array[@]}"
        do
                ${i} >> $logFile
        done

I get the following error:
lsof -s -p 5573 '|' awk ''\''{' user = '$4' '}' '{' if '(' user == mem ')' '{' print '}' '}'\''' '|' sort -nrk 7 '|' head -100
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39949896
Try

eval ${i} >> $logFile

And wouldn't the user appear in column 3, thus requiring "$3" instead of "$4" in awk?

Next, you're missing (escaped) double quotes here:

if ( user == \"$fileType\" )

Strings must be quoted in awk so that they can be distinguished from variables.


cmd_array[1]="lsof -s -p $process_pid | awk '{ user = \$3 } { if ( user == \"$fileType\" ) { print } }' | sort -nrk 7 |  head -100"

eval ${cmd_array[1]}


In your (and the above) version process_pid and fileType must be set before populating the array. Is this desired?
0
 

Author Comment

by:pvinodp
ID: 39950035
is there anything wrong or is it not the optmized way?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950047
Once the suggested corrections have been made there's nothing wrong anymore. Using "eval" the command should run as desired (see my note about populating variables, however).

To be able to ponder what the "optimitzed" way might be I'll need much more info what you're actually trying to achieve, and in which context.

I don't understand what this

cmd_array=("cat /proc/${process_pid}/status")

should be good for.
The statement will fill ${cmd_array[0]} with all the stuff from the status part of the process entry in /proc, which is not at all related to the lsof command you posted. ${cmd_array[0]} will not contain a command, so running it will always produce errors, even with "eval".
0
 

Author Comment

by:pvinodp
ID: 39950295
I am doing a performance analysis of a particular process id .

i need to execute "cat /proc/pid/status" and "lsof ****" .
how can i proceed?
0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39950909
Do you want to run the analysis against a specific pid/user combo, or how should pid and user be selected?

What do you actually mean with "user" in your script? The username as displayed by lsof in column 3, or perhaps something different?
0
 

Author Comment

by:pvinodp
ID: 39951131
user is just a placeholder .. The variable can have any name.
i want to measure against particular pid
I am given the pid for whcih i have to measure.

The code to run the commands is comon. i have a array of commands for certain type of pids.
For one such pid there  is 2 commands:
1. cat /proc/pid/status
2. lsof ****

the actual problem is that i am unable to append the eval output to a file.
THat is my next question. Can someone answer with that perspective?
0
 
LVL 61

Expert Comment

by:gheist
ID: 39951146
You know that lsof can output just couple of fields without header?
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 333 total points
ID: 39951324
Sure, a variable name is just a placeholder, that's trivial.

What I actually wanted to ask for is the desired content of the variable, in a substantial perspective.

OK, you have a PID, that's good.

The posted "awk" would filter field # 4 of lsof's output against a variable whose name in the shell environment is "fileType" (setting it to "user" in the awk environment is not a problem).

Where in the script is this variable filled, and should it really be used as a filter against field # 4 which is "FD" (File Descriptor)?
FD can sometimes well be "mem" (yet followed by two more characters) as in the posted command in comment #39949872.

Anyway, I can't see why you should use arrays here, so there is no need for "eval" either:

process_pid=5573
fileType="mem"
logFile="/path/to/logfile.txt"

cat /proc/${process_pid}/status >> $logFile
lsof -s -p $process_pid | awk '{ user = $4 } { if ( user ~"'$fileType'" ) { print } }' | sort -nrk 7 |  head -100 >> $logFile

Note: Use double plus single quotes around "'$fileType'"!  
Here it's shown with spaces for illustration, but don't use the spaces: " ' $fileType ' "

Once the above has run you will find a file "/path/to/logfile.txt" which you can analyze whatever way.

Try both statements without redirection first ( omit ">> $logFile") so you can check the outcome on your terminal.
Reason: "mem" in the FD (field #4) means "memory-mapped file" and there might well be no such files held open by the process in question - so the "awk" output will be empty.
0
 

Author Comment

by:pvinodp
ID: 39952309
I changed the command from using a variable to use a static constant.
if ( user ~"'$fileType'" )  is changed to if ( user ~"mem" )

what is the difference between == and ~
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 333 total points
ID: 39952618
~ is for a pattern/regex match, == is for an exact match.

I used ~ because field 4 does not contain exactly "mem" but can contain something like "memw " or "memrR".

~"mem"  will match all of them.
0
 

Author Closing Comment

by:pvinodp
ID: 39971965
Thanks for your response
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now