Solved

NetApp Snapshot Volume is over 100%

Posted on 2013-06-18
10
1,879 Views
Last Modified: 2013-07-12
We received an alert from LogicMonitor.com at 5:50AM, with the below message. About 2 hours prior to that (at 3:37AM), Veeam ran a backup of our Exchange server where vCenter triggered a "VM with snapshots" alert as well (below), but the vCenter alert was cleared at 5:36AM.

I understand Veeam creates snapshots to backup a VM, but then it deletes it. I checked the Snapshot manager of our Exchange VM and it is showing there are no snapshots right now (good). So why is LogicMonitor still giving us the alert.

vCenter Alert at 3:37 AM
Target: i1whlmail01
Previous Status: Green
New Status: Red
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 50,261,657KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Green to Red


vCenter Alert at 5:36 AM
Target: i1whlmail01
Previous Status: Red
New Status: Green
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 0KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Red to Green


LogicMonitor Alert at 5:50 AM
The NetApp i1whlnetapp01, volume /vol/Napp01_vol2/.snapshot , is now at 166 percent of space capacity, putting it in a state of warn.
Thresholds are > 100.
This state has existed since 2013-06-18 05:37:09 PDT - or for 0h 12m.
When the snapshot volume is over 100%, it means that deleting a file will not free up space in the file system until any referencing snapshot is deleted. This makes managing disk space harder.
Recommendation: check to ensure that the snapshot policy in effect is appropriate for the kind of data being stored on this filer. If so, increase the snapshot reserve.
Otherwise, change the snapshot policy or delete snapshots.
0
Comment
Question by:pzozulka
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 119

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 250 total points
ID: 39256978
Firstly, sometimes the VMware Snapshot Alert is "faulty!".

Check my EE Article to check your VM is not on a snapshot!

HOW TO: VMware Snapshots :- Be Patient

The Snapshot Alert you have received is because of Snapshots on your SAN for the volume/LUN.

Do you use NetApp Snapshots or SnapMirror?
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39257038
I've read your HOW TO article in the past and it has saved me many times before. Very well written. To answer you question, we use both snapshots and SnapMirror. This volume is snapmirrored every 3 hours at the 15 min mark.

I believe the vCenter alert about snapshots, and the storage device snapshot alert are related.

snapshotsvolume2 snapshot
0
 
LVL 119
ID: 39257267
If you use SnapMirror, it creates a SAN Snapshot.

Is your VM running on a snapshot?
0
Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 250 total points
ID: 39257560
The snapshots that are busy are the snapmirror snapshots and can only be deleted if/when the snapmirror is finished or you can queisce/break snapmirror, delete snapshot and review if you have a common snapshot on both sides you can continue to replicate from.

Snapshot reserve is just for accounting so that if you have space in the volume it will not do anything to your data, on the other hand if your snapshots start to encroash into your data your volume may go offline as you'll run out of space.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259882
I'm just trying to figure out what is causing this to occur:

I was under the impression that snapshots start out at zero, and then grow based on changes in the data compared to the previous snapshot. Having said that, based on my previous post, what does 93.91GB Cumalitve Total Size mean? Does that mean that there has been 93.91GB of changes since the last snapshot?

And lastly, what's the difference between the two columns, Total Size vs. Cumalitive Total Size?
0
 
LVL 119
ID: 39259932
sizes of snapshots will be based on the "delta" changes between each snapshot.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259965
Correct, so does that mean that on 6/18/2013 03:15:03 (from above image), there was 92.65GB of changes?

I'm not sure how to read the above image and Total Size vs. Cumalitive Total Size.
0
 
LVL 119
ID: 39259985
total size is the size of the snapshot, (delta change, e.g. changes between snapshots)

cumulative all the snapshots.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 39261954
The Netapp snapshots work a bit different than your copy on write snapshots in that they're pointer to the data.  So let's say you've added a VM, no snapshot space is being used but if you have a snapshot with a VM in place and you delete it then the snapshot will be holding the data and the snapshot will grow.  It can also be changes inside of the VMs, let's say a database that will look as different data if it's encrypted or binary where there is no way to dedupe it on the WAFL.
0
 
LVL 19

Expert Comment

by:compdigit44
ID: 39278745
Have you checked your datastore or orphan snapshots? Sometime not all snapshots are removed completed and are not listed in snapshot manager.
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
I cannot Import a vmdk file into a new VM host 3 52
NETAPP rename a datastore 2 43
vmsn files 11 49
Choosing a virtualization platform, Hyper-V or VMWare? 5 23
Concerto Cloud Services, a provider of fully managed private, public and hybrid cloud solutions, announced today it was named to the 20 Coolest Cloud Infrastructure Vendors Of The 2017 Cloud  (http://www.concertocloud.com/about/in-the-news/2017/02/0…
When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computers…
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

785 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question