Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

AIX 5.3 vmstat iostat output, signs of a overloaded system?

Posted on 2010-11-15
4
Medium Priority
?
1,244 Views
Last Modified: 2012-06-27
Hi,

New to the AIX world here, we are running AIX 5.3 64BIT ML-9. Below is a sample output of the various performance monitoring tools:

vmstat 1 10
procs                      memory      swap          io     system         cpu
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 2  5 375912  19548  17556 477472    0    1     0     0    1     1  1  0  0  1
 0  4 375912  18700  17556 478264    0    0  1044     0  774  1329  8  1  0 91
 0  5 375912  17664  17556 479168    0    0  1160     0  764  1110  8  1  0 91
 1  8 375912  15836  17568 479796    0    0  1144   840  751  1622 16  7  0 78
 0  7 375912  19340  17576 480224    0    0  1224   148  587  1958 17 18  0 65
 2  0 375912  18288  17588 481036    0    0   812     0  845  1732 18  3 21 59
 0  2 375912  15868  17588 481528    0    0  1012     0  588   941  4  1  5 90 

Open in new window


sar -u 2 3
AIX van-oppy 3 5 00001501D600    11/15/10

System configuration: lcpu=8  mode=Capped 

17:26:01    %usr    %sys    %wio   %idle   physc
17:26:03      17       8       9      66    3.98
17:26:05      14       4       6      76    4.00
17:26:07      19       4       2      75    4.00

Average       16       5       6      72    4.00

Open in new window


# mpstat 1 1
System configuration: lcpu=8 mode=Capped 

cpu  min  maj  mpc  int   cs  ics   rq  mig lpa sysc us sy wt id   pc
  0 4287   21    0 1152 1248  692    0  136 100 25903 34 14 29 23 0.68
  1    0    1    0  305   22   11    0    0 100  124  0  2  5 93 0.31
  2 1719   18    0 1342 1671  861    0  144 100 10558 23  8 27 42 0.61
  3    3    0    0  339   45   19    0    7 100   61  0  1  0 99 0.39
  4 3747    0    0 1047 1248  685    0   90  86 23355 14 13  1 72 0.60
  5    0    0    0  585  296  129    0    2  99  632  0  2  0 98 0.40
  6 2634    0    0 1014 1410  770    0  109  85 11946  8  9  0 82 0.50
  7    0    0    0  538    3    1    0    0 100 1302 27  3  0 69 0.51
ALL 12390   40    0 6322 5943 3168    0  488  94 73881 16  8 10 67 3.99

Open in new window


iostat 10 1
System configuration: lcpu=8 drives=43 paths=42 vdisks=0

tty:      tin         tout    avg-cpu: % user % sys % idle % iowait
          0.9        1732.7               13.6   4.0   79.6      2.8

Disks:        % tm_act     Kbps      tps    Kb_read   Kb_wrtn
hdisk14          0.6       2.4       0.6         24         0
hdisk15          0.0       0.0       0.0          0         0
hdisk39          0.0       0.0       0.0          0         0
hdisk13          0.0       0.0       0.0          0         0
hdisk41          0.0       0.0       0.0          0         0
hdisk37          0.0       0.0       0.0          0         0
hdisk11          0.0       0.0       0.0          0         0
hdisk38          0.0       0.0       0.0          0         0
hdisk36          0.0       0.0       0.0          0         0
hdisk10         17.6     206.1      51.5          0      2060
hdisk12         19.1     206.1      51.5          0      2060
hdisk40          0.0       0.0       0.0          0         0
hdisk17          7.2      73.4      18.2          0       734
hdisk19         11.6     125.4      28.5          0      1254
hdisk16         10.9     128.4      23.6          0      1284
hdisk18         11.9     145.4      26.0          0      1453
hdisk21         10.4      98.6      23.9          0       986
hdisk23          0.0       0.0       0.0          0         0
hdisk22          0.0       0.0       0.0          0         0
hdisk27         10.5     100.2      23.9         16       986
hdisk25         11.5     125.4      28.5          0      1254
hdisk29          8.4     140.6      19.6          0      1405
hdisk24         11.5     145.4      25.9          0      1453
hdisk35          0.0       0.0       0.0          0         0
hdisk26          8.8      73.4      18.2          0       734
hdisk20          8.3     140.6      19.6          0      1405
hdisk28         12.5     128.4      23.6          0      1284
hdisk31          3.5      57.3       8.4          0       573
hdisk4          12.0     145.4      26.0          0      1453
hdisk1          10.3     128.4      23.6          0      1284
hdisk0          10.1      98.6      23.9          0       986
hdisk2           8.0      73.4      18.2          0       734
hdisk5           7.8     140.6      19.6          0      1405
hdisk33          0.0       0.0       0.0          0         0
hdisk34          0.0       0.0       0.0          0         0
hdisk32          0.0       0.0       0.0          0         0
hdisk3          12.4     125.4      28.5          0      1254
hdisk7           3.7      58.1       8.5          8       573
hdisk6           4.7      98.3      10.3          0       983
hdisk30          4.5      99.1      10.4          8       983
hdisk9           0.0       0.0       0.0          0         0
hdisk8           0.0       0.0       0.0          0         0
cd0              0.0       0.0       0.0          0         0

Open in new window


What I am trying to do is ensure system is not overloaded, and that buffer memory is not trashing the disk because our OLTP database would grind to a crawl in such cases.

Any hints or techniques are appreciated :)

If you require more output from different commands, please post and I will post results.
0
Comment
Question by:mirde
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 2

Expert Comment

by:ExpertSteve
ID: 34142653
Well, from that short set of data, it was not overloaded during the times you captured that output.
Your % iowait is low, and no disk is active much of the time. You have no swapping, and plenty of idle CPU. (Which, given it is not blocked waiting for IO, means you have lots of headroom.)

But you really need to trend these things over time, not just with point in time snapshots.
Checkout something like http://www.logicmonitor.com/monitoring/servers/aix/
0
 
LVL 62

Accepted Solution

by:
gheist earned 1000 total points
ID: 34144578
huge wa in vmstat comes from active swapping.
configure your processes to use less memory and set vmo thresholds to page in bigger chunks.
0
 
LVL 62

Assisted Solution

by:gheist
gheist earned 1000 total points
ID: 34211390
wa is IO wait.
i'd blindly suggest reorgvg...
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to install and use the NTBackup utility that comes with Windows Server.
Resolving an irritating Remote Desktop connection that stops your saved credentials from being used.
This tutorial will give a short introduction and overview of Backup Exec 2012 and how to navigate and perform basic functions. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as conne…
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
Suggested Courses

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question