Solved

NetApp Snapshot Volume is over 100%

Posted on 2013-06-18
10
1,954 Views
Last Modified: 2013-07-12
We received an alert from LogicMonitor.com at 5:50AM, with the below message. About 2 hours prior to that (at 3:37AM), Veeam ran a backup of our Exchange server where vCenter triggered a "VM with snapshots" alert as well (below), but the vCenter alert was cleared at 5:36AM.

I understand Veeam creates snapshots to backup a VM, but then it deletes it. I checked the Snapshot manager of our Exchange VM and it is showing there are no snapshots right now (good). So why is LogicMonitor still giving us the alert.

vCenter Alert at 3:37 AM
Target: i1whlmail01
Previous Status: Green
New Status: Red
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 50,261,657KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Green to Red


vCenter Alert at 5:36 AM
Target: i1whlmail01
Previous Status: Red
New Status: Green
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 0KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Red to Green


LogicMonitor Alert at 5:50 AM
The NetApp i1whlnetapp01, volume /vol/Napp01_vol2/.snapshot , is now at 166 percent of space capacity, putting it in a state of warn.
Thresholds are > 100.
This state has existed since 2013-06-18 05:37:09 PDT - or for 0h 12m.
When the snapshot volume is over 100%, it means that deleting a file will not free up space in the file system until any referencing snapshot is deleted. This makes managing disk space harder.
Recommendation: check to ensure that the snapshot policy in effect is appropriate for the kind of data being stored on this filer. If so, increase the snapshot reserve.
Otherwise, change the snapshot policy or delete snapshots.
0
Comment
Question by:pzozulka
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 121

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 250 total points
ID: 39256978
Firstly, sometimes the VMware Snapshot Alert is "faulty!".

Check my EE Article to check your VM is not on a snapshot!

HOW TO: VMware Snapshots :- Be Patient

The Snapshot Alert you have received is because of Snapshots on your SAN for the volume/LUN.

Do you use NetApp Snapshots or SnapMirror?
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39257038
I've read your HOW TO article in the past and it has saved me many times before. Very well written. To answer you question, we use both snapshots and SnapMirror. This volume is snapmirrored every 3 hours at the 15 min mark.

I believe the vCenter alert about snapshots, and the storage device snapshot alert are related.

snapshotsvolume2 snapshot
0
 
LVL 121
ID: 39257267
If you use SnapMirror, it creates a SAN Snapshot.

Is your VM running on a snapshot?
0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 250 total points
ID: 39257560
The snapshots that are busy are the snapmirror snapshots and can only be deleted if/when the snapmirror is finished or you can queisce/break snapmirror, delete snapshot and review if you have a common snapshot on both sides you can continue to replicate from.

Snapshot reserve is just for accounting so that if you have space in the volume it will not do anything to your data, on the other hand if your snapshots start to encroash into your data your volume may go offline as you'll run out of space.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259882
I'm just trying to figure out what is causing this to occur:

I was under the impression that snapshots start out at zero, and then grow based on changes in the data compared to the previous snapshot. Having said that, based on my previous post, what does 93.91GB Cumalitve Total Size mean? Does that mean that there has been 93.91GB of changes since the last snapshot?

And lastly, what's the difference between the two columns, Total Size vs. Cumalitive Total Size?
0
 
LVL 121
ID: 39259932
sizes of snapshots will be based on the "delta" changes between each snapshot.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259965
Correct, so does that mean that on 6/18/2013 03:15:03 (from above image), there was 92.65GB of changes?

I'm not sure how to read the above image and Total Size vs. Cumalitive Total Size.
0
 
LVL 121
ID: 39259985
total size is the size of the snapshot, (delta change, e.g. changes between snapshots)

cumulative all the snapshots.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 39261954
The Netapp snapshots work a bit different than your copy on write snapshots in that they're pointer to the data.  So let's say you've added a VM, no snapshot space is being used but if you have a snapshot with a VM in place and you delete it then the snapshot will be holding the data and the snapshot will grow.  It can also be changes inside of the VMs, let's say a database that will look as different data if it's encrypted or binary where there is no way to dedupe it on the WAFL.
0
 
LVL 20

Expert Comment

by:compdigit44
ID: 39278745
Have you checked your datastore or orphan snapshots? Sometime not all snapshots are removed completed and are not listed in snapshot manager.
0

Featured Post

Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article we will learn how to backup a VMware farm using Nakivo Backup & Replication. In this tutorial we will install the software on a Windows 2012 R2 Server.
Many businesses neglect disaster recovery and treat it as an after-thought. I can tell you first hand that data will be lost, hard drives die, servers will be hacked, and careless (or malicious) employees can ruin your data.
Teach the user how to configure vSphere Replication and how to protect and recover VMs Open vSphere Web Client: Verify vsphere Replication is enabled: Enable vSphere Replication for a virtual machine: Verify replicated VM is created: Recover replica…
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …

690 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question