Solved

Netapp FAS 3140 - help interpreting lots of cache hits

Posted on 2012-03-28
2
1,159 Views
Last Modified: 2012-03-31
We have been having some serious disk performance problems.   During certain operations in VMWare like storage vmotion or other the ESX hosts and their guest OS's lose access to the hard drive which are reached via iSCSI.  Cisco UCS VMware 5 communicating with the 3140 over 10Gbps links.  No errors on the ethernet path.  But seeing cache hits at or near 100% all the time.  What does this indicate?

heraklesb> sysstat -u 1
 CPU   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk
       ops/s      in    out    read  write    read  write    age    hit  time  ty  util
 10%     915    8719    864     724      0       0      0    26s   100%    0%  -     9%
  7%     955    7625   1181     544      8       0      0    26s    98%    0%  -     9%
 12%     819    8833   1074   19501    631       0      0    26s   100%    6%  T    15%
 49%    1130    9214   1858   82880  79888       0      0    26s   100%  100%  :   100%
 17%     812    7228    423   32576  11368       0      0    26s    99%  100%  :    33%
 17%    1083    9123    774   16732  40248       0      0    26s    99%  100%  :    53%
 38%    3557    9541 297324    3772  23548       0      0    26s   100%   72%  :    35%
 22%    2656    5399 237799     936      0       0      0    26s   100%    0%  -    11%
  7%     851    6582   3580     940     24       0      0    26s    94%    0%  -     7%
  6%     512    4146    557     920      0       0      0    26s    97%    0%  -    10%
  5%     384    2641    309     440      0       0      0    26s    97%    0%  -     6%
  9%     858    7569    449     396     32       0      0    26s    98%    0%  -     7%
  8%     876    6573   1601     576      0       0      0    26s    98%    0%  -     8%
  7%     669    6293    426    1096      0       0      0    26s    95%    0%  -    11%
  5%     408    4215    602     492     24       0      0    26s    89%    0%  -     7%
  4%     388    3145    442     268      0       0      0    26s   100%    0%  -     7%
 23%    1472   13173   1425   17052  12556       0      0    26s   100%   26%  T    25%
 38%     973    9131    386   80056  46725       0      0    26s   100%  100%  :    98%
 14%     618    5120    257   17252  18124       0      0    26s   100%  100%  :    35%
 CPU   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk
       ops/s      in    out    read  write    read  write    age    hit  time  ty  util
 14%     568    4759    725   17104  35692       0      0    26s    98%  100%  :    46%
heraklesb> sysstat -u 1
 CPU   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk
       ops/s      in    out    read  write    read  write    age    hit  time  ty  util
 39%     325    2805    740     436      0       0      0    13s    86%    0%  -     6%
 10%    1166    9126   3180    1256     24       0      0    13s   100%    0%  -    10%
  8%    1178    6791   4929    1676      0       0      0    13s    99%    0%  -    12%
  4%     291    3147    360     232      0       0      0    13s   100%    0%  -     4%
 16%     334    2213   1011   18440  15380       0      0    13s    99%   43%  T    42%
 19%     497    8507    385   55052  32376       0      0    13s   100%  100%  :    83%
 18%     986    9137   1125   25400  27236       0      0    13s   100%  100%  :    43%
 12%     507    4202    467    3700  35208       0      0    13s   100%  100%  :    46%
  9%    1068    8792   1809     748   4136       0      0    13s    99%   22%  :    21%
  9%     980    6072  41322     208      8       0      0    13s   100%    0%  -     5%
 10%     998    2014  77318     412     24       0      0    13s   100%    0%  -    11%
  5%     561    5180    478     256      0       0      0    13s    99%    0%  -     5%
 16%    1633    3090 149573     188      0       0      0    13s   100%    0%  -     5%
  9%     946    8522  20868     672     24       0      0    13s   100%    0%  -    10%
  7%     801    8548    518     504      8       0      0    13s   100%    0%  -     8%
  6%     499    4292    300     148      0       0      0    13s   100%    0%  -     5%
  7%     678    6066    486     444     24       0      0    13s    98%    0%  -     6%
  6%     604    6306    745     420      0       0      0    38s   100%    0%  -     7%
 32%     448    2958    947   67600  46508       0      0    39s   100%   77%  T    75%
 CPU   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk
       ops/s      in    out    read  write    read  write    age    hit  time  ty  util
 23%     332    2346    161   58888  18884       0      0    39s   100%  100%  :    51%
 15%     770    6636    875     404  17880       0      0    39s   100%  100%  :    21%
 16%    1009   10627    530    3272  44408       0      0    39s   100%   97%  :    51%
  4%     338    2424    126     692     24       0      0    39s    99%    0%  -    10%
  4%     465    3745    292     140      0       0      0    39s   100%    0%  -     3%
  7%     685    7148   1934     856      8       0      0    39s    99%    0%  -    12%
  7%     749    8642   2026    1708     24       0      0    39s    98%    0%  -    15%
  9%     856    2762  58752     160      0       0      0    11s   100%    0%  -     6%
  6%     575    2047  38734     456      0       0      0    11s    99%    0%  -     5%
  6%     458    1996  18820     632     24       0      0    11s    96%    0%  -    25%
0
Comment
Question by:amigan_99
2 Comments
 
LVL 5

Accepted Solution

by:
cdfs earned 500 total points
ID: 37780404
Cache hit means, that blocks that should be read where found in cache, so it was not necessary to load them from disk. A high Cache Hit-rate indicates, that most reads are the same, so they can be served directly from cache, that is faster than reading from disk.
This should have nothing to do with your problem, hosts and VMs losing connection to the filer. When you are using iSCSI or FC-LUNs with VMware, you should be aware, that you shouldn't put to much VMs on a single LUN, as this can lead to a degradation of performance. In general you should take a deep look at the NetApp Best Practices for VMware, found at http://media.netapp.com/documents/tr-3749.pdf
There is described, what the benefits of the different protocols are and how to fine-tune VMware and your filer to get the most out of it. I for myself am now using NFS over 10GBit with our FAS3210 and I'm very happy with it.
0
 
LVL 1

Author Closing Comment

by:amigan_99
ID: 37792260
Thank you.
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Lets start to have a small explanation what is VAAI(vStorage API for Array Integration ) and what are the benefits using it. VAAI is an API framework in VMware that enable some Storage tasks. It first presented in ESXi 4.1, but only after 5.x sup…
In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to install log collectors and how to configure ESXi 5.5 for remote logging Open console session and mount vCenter Server installer: Install vSphere Core Dump Collector: Install vSphere Syslog Collector: Open vSphere Client: Config…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now