Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

formatting output of iperf in bash script

Posted on 2014-11-19
21
Medium Priority
?
1,470 Views
Last Modified: 2014-11-26
I need a bash function which does an iperf test and formats the output as follows;
When running iperf, the results can be from Mb/s down to Kb/s so to get a uniform output, I need all of the output in Kb/s and into variables which I can then store into MySQL.

For example, the two following tests show what I mean about Mb/s or Kb/s.

# iperf -c 3.4.5.6
------------------------------------------------------------
Client connecting to 3.4.5.6 TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.103 port 39793 connected with 3.4.5.6 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.4 sec  5.25 MBytes  4.25 Mbits/sec

# iperf -c 6.7.8.9
------------------------------------------------------------
Client connecting to 6.7.8.9, TCP port 5001
TCP window size: 20.9 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.103 port 35086 connected with 6.7.8.9 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-13.2 sec  1.25 MBytes   794 Kbits/sec

I need a bash function which will take the results on line 7, tuning them into the appropriate variables to be stored into Mysql.

The Mb/s results should be converted to Kb/s which I will convert back to proper result when reading the database.

The variables should be;

INTERVAL
TRANSFER (in Kb/s)
BANDWIDTH (in Kb/s)

I have used the -f Kbits option but it convers it to KBytes so am not sure how to get past that either.
Can someone help me with this please. I've been struggling to build the code for a while now and getting nowhere.

Thanks.
0
Comment
Question by:projects
20 Comments
 
LVL 85

Expert Comment

by:ozo
ID: 40454027
iperf -c 6.7.8.9 |  perl -pe 's#([\d.]+)\s*((M)|K)((Byte)|bit)#@{[$1 * ($3?1024:1)*($5?8:1)]} Kbit#g'
0
 

Author Comment

by:projects
ID: 40454083
Sorry, I don't have perl on the device. It is a very basic Linux device.
I also need to turn the output into the three variables mentioned.

Thanks.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40461893
This awk script will generate the three values, comma-separated.  If the first character of the transfer descriptor is M (for MBytes), then the transfer value is multiplied by 1024 to get KBytes.  If it is G (for GBytes), it is multiplied by 1048576 to get KBytes.  The same is done for the Bandwidth, to convert to kbytes/sec.  For the interval, it splits the two values around the "-", and subtracts the first from the second (in case the first isn't 0.0)

#!/bin/sh

awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print iv "," tf "," bw
    }'

Open in new window


To use, filter the iperf output through this script:

    iperf -c 1.2.3.4 | iperf_fmt.sh
0
Learn Veeam advantages over legacy backup

Every day, more and more legacy backup customers switch to Veeam. Technologies designed for the client-server era cannot restore any IT service running in the hybrid cloud within seconds. Learn top Veeam advantages over legacy backup and get Veeam for the price of your renewal

 
LVL 14

Expert Comment

by:Otto_N
ID: 40461961
I don't have the complete answer for you, but here are some pointers:
"iperf -c -f k" should cause all results to be reported in Kbits, "iperf -c -f K" (which you tried) reports in Kbytes.  This can simplify simon3270's script above
The Ubuntu iperf man page report options -x (to exclude certain fields from the report) and -y (to set output to CSV).  It seems that these options can format the output to provide a much reduced text field as output, if you happen to run Ubuntu.  You could check the man page for your specific version to see if these options are also applicable to your platform.
0
 

Author Comment

by:projects
ID: 40462158
>To use, filter the iperf output through this script:
>    iperf -c 1.2.3.4 | iperf_fmt.sh

I made it executable but keep getting the error that the filter is not found.

# iperf -c server.com | perft.sh (the name of the script)
-ash: perft.sh: not found
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40462242
Is the directory of perf.sh in your $PATH? If not, use the full path to it, or use ./perf.sh if it is in the current directory.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40462251
Or add "." to your PATH, though that can be less secure (you can more easily run undesirable code, e.g. if you are in /tmp.
0
 

Author Comment

by:projects
ID: 40463001
Seems to work with the ./ added.

Regular;
# iperf -c server.com
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.4 sec  5.25 MBytes  4.22 Mbits/sec

With Filter;
# iperf -c server.com | ./perft.sh
10.2,5376,4403.2

Two questions.
Assuming output meaning;
10.2 = how long the test took
5376 = 5.3Mb.s
4403.2 = 4.4Mb/s

So, this is an external filter so how do I put all this into one script? The server is always the same so it can be a variable at the top of IP1 for example.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40463198
Yes, the output is the number of seconds the test took, the transfer amount in kbytes, and the transfer speed in kbits/sec.

To get it all in one script, just roll the iperf command into the script text and pipe its output into the awk program:
#!/bin/sh

IP1=1.2.3.4

iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print iv "," tf "," bw
    }'

Open in new window

0
 

Author Comment

by:projects
ID: 40464573
Works, thanks.
Only remaining item is how do I get the results into three variables so I can store them in mysql?

I tried
echo "Seconds: $iv Transfer: $tf  Speed: $bw"
for example because I need those three variables filled but that doesn't work.

Sorry, I've never used a filter like this.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40464839
The variables only exist within the "awk" script, so you need to change the "print" statement within the awk script.

If the echo above shows the format you want, replace the print line with:

    print "Seconds: " iv " Transfer: " tf " Speed: " bw

I'm not sure whether that's the format you want, so that you can use the data directly as input to mysq - if not, let me know the exact format and I'll construct a print command for you!
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40464877
Then once the line is in the right format, we can use the output of the iperf+awk as input to mysql, so like:

 
   invals=$(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
          .. and the rest of the awk script...
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print "Seconds: " iv " Transfer: " tf " Speed: " bw
    }')

   echo "sql load $invals" | mysql

Open in new window

I know that the mysql line is wrong, but it's just there to show how the entire output of the awk script can be used in another command.
0
 

Author Comment

by:projects
ID: 40465090
I don't need the mysql part, I just need to convert the three outputs into three separate variables.
This will be a script running now and then, sending it's output to a php page which is already set up but I need the outputs as $SECONDS, $TRANSFER and $SPEED
0
 
LVL 20

Accepted Solution

by:
simon3270 earned 2000 total points
ID: 40465336
There are two ways to do this.  They both start the same way. Let's call the script doiperf.sh.
#!/bin/sh

IP1=1.2.3.4

set $(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        printf("%.1f %.1f %.1f\n", iv, tf, bw)
    }')

PSECONDS=$1
TRANSFER=$2
SPEED=$3

Open in new window

Then it depends on how you want to use the variables.  If you call some sort of script to pass the values to your PHP page, you can append the code to do that, like:
add_to_php.sh $PSECONDS $TRANSFER $SPEED

Open in new window


An alternative version is to pass the values back to the calling shell.  You do this by sourcing the script rather than running it, then using the values after the call:
source doiperf.sh

Open in new window

.

NOTE: In general, I wouldn't use $SECONDS as a shell variable (note that I use $PSECONDS above)  - the bash shell (and probably others) use $SECONDS as the number of seconds that the shell had been running.  If you set this to another value, the shell will simply increment that new value every second, so it won't be what you set it to!
0
 

Author Comment

by:projects
ID: 40465585
Good point about $SECONDS.

I would like to understand why the variables are $1, $2, $3 though and change those also because there are other things in the main script which could conflict.

Thanks for your help.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40465856
The "set" command (in the way I have used it here) sets $1 to the first value in the string following it, $2 to the second, and so on.

If you are using positional parameters in the rest of your script, it is generally a good idea to save them in named variables early on, to avoid clashes with this sort of code, and to make their usage more obvious (better to have "target_dir=$2" when a script starts, so that it is much clearer later on to have "cd $target_dir" than "cd $2")

If this is within a function,  then it only affects $1 $2 and so on within that function - parameters in the main script are unaffected.
0
 

Author Comment

by:projects
ID: 40466778
If repeated variables are inside of their own functions in a multi-function script, is there still a chance that variables could conflict?

I don't have enough experience to really understand your code to be honest.

#!/bin/sh

IP1=server.com

set $(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        printf("%.1f %.1f %.1f\n", iv, tf, bw)
    }')

PSECONDS=$1
TRANSFER=$2
SPEED=$3

echo "$PSECONDS - $TRANSFER - $SPEED"

Open in new window


Are you saying something along the lines of I would need to do the following at the top of the script?

PSECONDS=$1
TRANSFER=$2
SPEED=$3

I'm guessing not since you say you are using set for $1.
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40466854
No, I'm saying that parameters to a script should be stored in other variables so that they are safe from calls like mine (to "set"), and to make their use more obvious later in the script  (my "cd $target_dir" rather than "cd $2"). In the script here, if you passed the IP address in as a parameter, you might have
    IP1=$1

As for named variables,  they are usually global to the entire script - many people will set a variable within a function then use it in the main body of the script, which can cause confusion. To make sure that a named variable within a function does not affect a variable with the same name in the main script or another function, put "local" in front of the name , as in:
    function myfunc {
        local IP1
        IP1=1.2.3.4
    }

Open in new window

The IP1 variable there will not clash with an IP1 in the main script.

Positional parameter variables  ($1, $2 etc) are always local to a function - this means that you can modify positional parameter within a function without affecting the main script's parameters, and that within a function you have no access to the main script's parameters.

To understand the script,  you can follow its development throughout this question. The "awk" bit converts the data to a known format and prints out the three values  separated by spaces. The "iperf" bit runs the iperf command and feeds the output into the awk. This is then run in its own shell (the "$()" bit), and
0
 
LVL 20

Expert Comment

by:simon3270
ID: 40466857
(pressed "submit too soon!)

The "$()" bit returns the output of the awk. This output is then used as the arguments for the "set" command to set $1, $2 and $3.
0
 

Author Comment

by:projects
ID: 40467686
Thanks for all of this information. Very useful.
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In part one, we reviewed the prerequisites required for installing SQL Server vNext. In this part we will explore how to install Microsoft's SQL Server on Ubuntu 16.04.
Tech spooks aren't just for those who are tech savvy, it also happens to those of us running a business. Check out the top tech spooks for business owners.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…

824 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question