Solved

formatting output of iperf in bash script

Posted on 2014-11-19
21
781 Views
Last Modified: 2014-11-26
I need a bash function which does an iperf test and formats the output as follows;
When running iperf, the results can be from Mb/s down to Kb/s so to get a uniform output, I need all of the output in Kb/s and into variables which I can then store into MySQL.

For example, the two following tests show what I mean about Mb/s or Kb/s.

# iperf -c 3.4.5.6
------------------------------------------------------------
Client connecting to 3.4.5.6 TCP port 5001
TCP window size: 19.6 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.103 port 39793 connected with 3.4.5.6 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.4 sec  5.25 MBytes  4.25 Mbits/sec

# iperf -c 6.7.8.9
------------------------------------------------------------
Client connecting to 6.7.8.9, TCP port 5001
TCP window size: 20.9 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.103 port 35086 connected with 6.7.8.9 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-13.2 sec  1.25 MBytes   794 Kbits/sec

I need a bash function which will take the results on line 7, tuning them into the appropriate variables to be stored into Mysql.

The Mb/s results should be converted to Kb/s which I will convert back to proper result when reading the database.

The variables should be;

INTERVAL
TRANSFER (in Kb/s)
BANDWIDTH (in Kb/s)

I have used the -f Kbits option but it convers it to KBytes so am not sure how to get past that either.
Can someone help me with this please. I've been struggling to build the code for a while now and getting nowhere.

Thanks.
0
Comment
Question by:projects
21 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 40454027
iperf -c 6.7.8.9 |  perl -pe 's#([\d.]+)\s*((M)|K)((Byte)|bit)#@{[$1 * ($3?1024:1)*($5?8:1)]} Kbit#g'
0
 

Author Comment

by:projects
ID: 40454083
Sorry, I don't have perl on the device. It is a very basic Linux device.
I also need to turn the output into the three variables mentioned.

Thanks.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40461893
This awk script will generate the three values, comma-separated.  If the first character of the transfer descriptor is M (for MBytes), then the transfer value is multiplied by 1024 to get KBytes.  If it is G (for GBytes), it is multiplied by 1048576 to get KBytes.  The same is done for the Bandwidth, to convert to kbytes/sec.  For the interval, it splits the two values around the "-", and subtracts the first from the second (in case the first isn't 0.0)

#!/bin/sh

awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print iv "," tf "," bw
    }'

Open in new window


To use, filter the iperf output through this script:

    iperf -c 1.2.3.4 | iperf_fmt.sh
0
 
LVL 14

Expert Comment

by:Otto_N
ID: 40461961
I don't have the complete answer for you, but here are some pointers:
"iperf -c -f k" should cause all results to be reported in Kbits, "iperf -c -f K" (which you tried) reports in Kbytes.  This can simplify simon3270's script above
The Ubuntu iperf man page report options -x (to exclude certain fields from the report) and -y (to set output to CSV).  It seems that these options can format the output to provide a much reduced text field as output, if you happen to run Ubuntu.  You could check the man page for your specific version to see if these options are also applicable to your platform.
0
 

Author Comment

by:projects
ID: 40462158
>To use, filter the iperf output through this script:
>    iperf -c 1.2.3.4 | iperf_fmt.sh

I made it executable but keep getting the error that the filter is not found.

# iperf -c server.com | perft.sh (the name of the script)
-ash: perft.sh: not found
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40462242
Is the directory of perf.sh in your $PATH? If not, use the full path to it, or use ./perf.sh if it is in the current directory.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40462251
Or add "." to your PATH, though that can be less secure (you can more easily run undesirable code, e.g. if you are in /tmp.
0
 

Author Comment

by:projects
ID: 40463001
Seems to work with the ./ added.

Regular;
# iperf -c server.com
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.4 sec  5.25 MBytes  4.22 Mbits/sec

With Filter;
# iperf -c server.com | ./perft.sh
10.2,5376,4403.2

Two questions.
Assuming output meaning;
10.2 = how long the test took
5376 = 5.3Mb.s
4403.2 = 4.4Mb/s

So, this is an external filter so how do I put all this into one script? The server is always the same so it can be a variable at the top of IP1 for example.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40463198
Yes, the output is the number of seconds the test took, the transfer amount in kbytes, and the transfer speed in kbits/sec.

To get it all in one script, just roll the iperf command into the script text and pipe its output into the awk program:
#!/bin/sh

IP1=1.2.3.4

iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print iv "," tf "," bw
    }'

Open in new window

0
 

Author Comment

by:projects
ID: 40464573
Works, thanks.
Only remaining item is how do I get the results into three variables so I can store them in mysql?

I tried
echo "Seconds: $iv Transfer: $tf  Speed: $bw"
for example because I need those three variables filled but that doesn't work.

Sorry, I've never used a filter like this.
0
VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

 
LVL 19

Expert Comment

by:simon3270
ID: 40464839
The variables only exist within the "awk" script, so you need to change the "print" statement within the awk script.

If the echo above shows the format you want, replace the print line with:

    print "Seconds: " iv " Transfer: " tf " Speed: " bw

I'm not sure whether that's the format you want, so that you can use the data directly as input to mysq - if not, let me know the exact format and I'll construct a print command for you!
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40464877
Then once the line is in the right format, we can use the output of the iperf+awk as input to mysql, so like:

 
   invals=$(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
          .. and the rest of the awk script...
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        print "Seconds: " iv " Transfer: " tf " Speed: " bw
    }')

   echo "sql load $invals" | mysql

Open in new window

I know that the mysql line is wrong, but it's just there to show how the entire output of the awk script can be used in another command.
0
 

Author Comment

by:projects
ID: 40465090
I don't need the mysql part, I just need to convert the three outputs into three separate variables.
This will be a script running now and then, sending it's output to a php page which is already set up but I need the outputs as $SECONDS, $TRANSFER and $SPEED
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
ID: 40465336
There are two ways to do this.  They both start the same way. Let's call the script doiperf.sh.
#!/bin/sh

IP1=1.2.3.4

set $(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        printf("%.1f %.1f %.1f\n", iv, tf, bw)
    }')

PSECONDS=$1
TRANSFER=$2
SPEED=$3

Open in new window

Then it depends on how you want to use the variables.  If you call some sort of script to pass the values to your PHP page, you can append the code to do that, like:
add_to_php.sh $PSECONDS $TRANSFER $SPEED

Open in new window


An alternative version is to pass the values back to the calling shell.  You do this by sourcing the script rather than running it, then using the values after the call:
source doiperf.sh

Open in new window

.

NOTE: In general, I wouldn't use $SECONDS as a shell variable (note that I use $PSECONDS above)  - the bash shell (and probably others) use $SECONDS as the number of seconds that the shell had been running.  If you set this to another value, the shell will simply increment that new value every second, so it won't be what you set it to!
0
 

Author Comment

by:projects
ID: 40465585
Good point about $SECONDS.

I would like to understand why the variables are $1, $2, $3 though and change those also because there are other things in the main script which could conflict.

Thanks for your help.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40465856
The "set" command (in the way I have used it here) sets $1 to the first value in the string following it, $2 to the second, and so on.

If you are using positional parameters in the rest of your script, it is generally a good idea to save them in named variables early on, to avoid clashes with this sort of code, and to make their usage more obvious (better to have "target_dir=$2" when a script starts, so that it is much clearer later on to have "cd $target_dir" than "cd $2")

If this is within a function,  then it only affects $1 $2 and so on within that function - parameters in the main script are unaffected.
0
 

Author Comment

by:projects
ID: 40466778
If repeated variables are inside of their own functions in a multi-function script, is there still a chance that variables could conflict?

I don't have enough experience to really understand your code to be honest.

#!/bin/sh

IP1=server.com

set $(iperf -c $IP1 | awk 'NR == 7{mul=1
        if ($6 ~ /^M/) {mul=1024}
        if ($6 ~ /^G/) {mul=1048576}
        tf=$5*mul
        mul=1
        if ($8 ~ /^M/) {mul=1024}
        if ($8 ~ /^G/) {mul=1048576}
        bw=$7*mul
        ic = split($3,ia,"-")
        if (ic == 2) {iv = ia[2] - ia[1]} else {iv = 0}
        printf("%.1f %.1f %.1f\n", iv, tf, bw)
    }')

PSECONDS=$1
TRANSFER=$2
SPEED=$3

echo "$PSECONDS - $TRANSFER - $SPEED"

Open in new window


Are you saying something along the lines of I would need to do the following at the top of the script?

PSECONDS=$1
TRANSFER=$2
SPEED=$3

I'm guessing not since you say you are using set for $1.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40466854
No, I'm saying that parameters to a script should be stored in other variables so that they are safe from calls like mine (to "set"), and to make their use more obvious later in the script  (my "cd $target_dir" rather than "cd $2"). In the script here, if you passed the IP address in as a parameter, you might have
    IP1=$1

As for named variables,  they are usually global to the entire script - many people will set a variable within a function then use it in the main body of the script, which can cause confusion. To make sure that a named variable within a function does not affect a variable with the same name in the main script or another function, put "local" in front of the name , as in:
    function myfunc {
        local IP1
        IP1=1.2.3.4
    }

Open in new window

The IP1 variable there will not clash with an IP1 in the main script.

Positional parameter variables  ($1, $2 etc) are always local to a function - this means that you can modify positional parameter within a function without affecting the main script's parameters, and that within a function you have no access to the main script's parameters.

To understand the script,  you can follow its development throughout this question. The "awk" bit converts the data to a known format and prints out the three values  separated by spaces. The "iperf" bit runs the iperf command and feeds the output into the awk. This is then run in its own shell (the "$()" bit), and
0
 
LVL 19

Expert Comment

by:simon3270
ID: 40466857
(pressed "submit too soon!)

The "$()" bit returns the output of the awk. This output is then used as the arguments for the "set" command to set $1, $2 and $3.
0
 

Author Comment

by:projects
ID: 40467686
Thanks for all of this information. Very useful.
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

If your business is like most, chances are you still need to maintain a fax infrastructure for your staff. It’s hard to believe that a communication technology that was thriving in the mid-80s could still be an essential part of your team’s modern I…
When it comes to security, there are always trade-offs between security and convenience/ease of administration. This article examines some of the main pros and cons of using key authentication vs password authentication for hosting an SFTP server.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now