Solved

AIX 5.3 vmstat iostat output, signs of a overloaded system?

Posted on 2010-11-15
4
1,208 Views
Last Modified: 2012-06-27
Hi,

New to the AIX world here, we are running AIX 5.3 64BIT ML-9. Below is a sample output of the various performance monitoring tools:

vmstat 1 10
procs                      memory      swap          io     system         cpu
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 2  5 375912  19548  17556 477472    0    1     0     0    1     1  1  0  0  1
 0  4 375912  18700  17556 478264    0    0  1044     0  774  1329  8  1  0 91
 0  5 375912  17664  17556 479168    0    0  1160     0  764  1110  8  1  0 91
 1  8 375912  15836  17568 479796    0    0  1144   840  751  1622 16  7  0 78
 0  7 375912  19340  17576 480224    0    0  1224   148  587  1958 17 18  0 65
 2  0 375912  18288  17588 481036    0    0   812     0  845  1732 18  3 21 59
 0  2 375912  15868  17588 481528    0    0  1012     0  588   941  4  1  5 90 

Open in new window


sar -u 2 3
AIX van-oppy 3 5 00001501D600    11/15/10

System configuration: lcpu=8  mode=Capped 

17:26:01    %usr    %sys    %wio   %idle   physc
17:26:03      17       8       9      66    3.98
17:26:05      14       4       6      76    4.00
17:26:07      19       4       2      75    4.00

Average       16       5       6      72    4.00

Open in new window


# mpstat 1 1
System configuration: lcpu=8 mode=Capped 

cpu  min  maj  mpc  int   cs  ics   rq  mig lpa sysc us sy wt id   pc
  0 4287   21    0 1152 1248  692    0  136 100 25903 34 14 29 23 0.68
  1    0    1    0  305   22   11    0    0 100  124  0  2  5 93 0.31
  2 1719   18    0 1342 1671  861    0  144 100 10558 23  8 27 42 0.61
  3    3    0    0  339   45   19    0    7 100   61  0  1  0 99 0.39
  4 3747    0    0 1047 1248  685    0   90  86 23355 14 13  1 72 0.60
  5    0    0    0  585  296  129    0    2  99  632  0  2  0 98 0.40
  6 2634    0    0 1014 1410  770    0  109  85 11946  8  9  0 82 0.50
  7    0    0    0  538    3    1    0    0 100 1302 27  3  0 69 0.51
ALL 12390   40    0 6322 5943 3168    0  488  94 73881 16  8 10 67 3.99

Open in new window


iostat 10 1
System configuration: lcpu=8 drives=43 paths=42 vdisks=0

tty:      tin         tout    avg-cpu: % user % sys % idle % iowait
          0.9        1732.7               13.6   4.0   79.6      2.8

Disks:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
hdisk14          0.6       2.4       0.6         24         0
hdisk15          0.0       0.0       0.0          0         0
hdisk39          0.0       0.0       0.0          0         0
hdisk13          0.0       0.0       0.0          0         0
hdisk41          0.0       0.0       0.0          0         0
hdisk37          0.0       0.0       0.0          0         0
hdisk11          0.0       0.0       0.0          0         0
hdisk38          0.0       0.0       0.0          0         0
hdisk36          0.0       0.0       0.0          0         0
hdisk10         17.6     206.1      51.5          0      2060
hdisk12         19.1     206.1      51.5          0      2060
hdisk40          0.0       0.0       0.0          0         0
hdisk17          7.2      73.4      18.2          0       734
hdisk19         11.6     125.4      28.5          0      1254
hdisk16         10.9     128.4      23.6          0      1284
hdisk18         11.9     145.4      26.0          0      1453
hdisk21         10.4      98.6      23.9          0       986
hdisk23          0.0       0.0       0.0          0         0
hdisk22          0.0       0.0       0.0          0         0
hdisk27         10.5     100.2      23.9         16       986
hdisk25         11.5     125.4      28.5          0      1254
hdisk29          8.4     140.6      19.6          0      1405
hdisk24         11.5     145.4      25.9          0      1453
hdisk35          0.0       0.0       0.0          0         0
hdisk26          8.8      73.4      18.2          0       734
hdisk20          8.3     140.6      19.6          0      1405
hdisk28         12.5     128.4      23.6          0      1284
hdisk31          3.5      57.3       8.4          0       573
hdisk4          12.0     145.4      26.0          0      1453
hdisk1          10.3     128.4      23.6          0      1284
hdisk0          10.1      98.6      23.9          0       986
hdisk2           8.0      73.4      18.2          0       734
hdisk5           7.8     140.6      19.6          0      1405
hdisk33          0.0       0.0       0.0          0         0
hdisk34          0.0       0.0       0.0          0         0
hdisk32          0.0       0.0       0.0          0         0
hdisk3          12.4     125.4      28.5          0      1254
hdisk7           3.7      58.1       8.5          8       573
hdisk6           4.7      98.3      10.3          0       983
hdisk30          4.5      99.1      10.4          8       983
hdisk9           0.0       0.0       0.0          0         0
hdisk8           0.0       0.0       0.0          0         0
cd0              0.0       0.0       0.0          0         0

Open in new window


What I am trying to do is ensure system is not overloaded, and that buffer memory is not trashing the disk because our OLTP database would grind to a crawl in such cases.

Any hints or techniques are appreciated :)

If you require more output from different commands, please post and I will post results.
0
Comment
Question by:mirde
  • 2
4 Comments
 
LVL 2

Expert Comment

by:ExpertSteve
Comment Utility
Well, from that short set of data, it was not overloaded during the times you captured that output.
Your % iowait is low, and no disk is active much of the time. You have no swapping, and plenty of idle CPU. (Which, given it is not blocked waiting for IO, means you have lots of headroom.)

But you really need to trend these things over time, not just with point in time snapshots.
Checkout something like http://www.logicmonitor.com/monitoring/servers/aix/
0
 
LVL 61

Accepted Solution

by:
gheist earned 250 total points
Comment Utility
huge wa in vmstat comes from active swapping.
configure your processes to use less memory and set vmo thresholds to page in bigger chunks.
0
 
LVL 61

Assisted Solution

by:gheist
gheist earned 250 total points
Comment Utility
wa is IO wait.
i'd blindly suggest reorgvg...
0

Featured Post

Zoho SalesIQ

Hassle-free live chat software re-imagined for business growth. 2 users, always free.

Join & Write a Comment

The purpose of this article is to demonstrate how we can use conditional statements using Python.
You might have come across a situation when you have Exchange 2013 server in two different sites (Production and DR). After adding the Database copy in ECP console it displays Database copy status unknown for the DR exchange server. Issue is strange…
This tutorial will walk an individual through locating and launching the BEUtility application and how to execute it on the appropriate database. Log onto the server running the Backup Exec database. In a larger environment, this would generally be …
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now