[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

NetApp Snapshot Volume is over 100%

Posted on 2013-06-18
10
Medium Priority
?
2,002 Views
Last Modified: 2013-07-12
We received an alert from LogicMonitor.com at 5:50AM, with the below message. About 2 hours prior to that (at 3:37AM), Veeam ran a backup of our Exchange server where vCenter triggered a "VM with snapshots" alert as well (below), but the vCenter alert was cleared at 5:36AM.

I understand Veeam creates snapshots to backup a VM, but then it deletes it. I checked the Snapshot manager of our Exchange VM and it is showing there are no snapshots right now (good). So why is LogicMonitor still giving us the alert.

vCenter Alert at 3:37 AM
Target: i1whlmail01
Previous Status: Green
New Status: Red
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 50,261,657KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Green to Red


vCenter Alert at 5:36 AM
Target: i1whlmail01
Previous Status: Red
New Status: Green
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 0KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Red to Green


LogicMonitor Alert at 5:50 AM
The NetApp i1whlnetapp01, volume /vol/Napp01_vol2/.snapshot , is now at 166 percent of space capacity, putting it in a state of warn.
Thresholds are > 100.
This state has existed since 2013-06-18 05:37:09 PDT - or for 0h 12m.
When the snapshot volume is over 100%, it means that deleting a file will not free up space in the file system until any referencing snapshot is deleted. This makes managing disk space harder.
Recommendation: check to ensure that the snapshot policy in effect is appropriate for the kind of data being stored on this filer. If so, increase the snapshot reserve.
Otherwise, change the snapshot policy or delete snapshots.
0
Comment
Question by:pzozulka
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 1000 total points
ID: 39256978
Firstly, sometimes the VMware Snapshot Alert is "faulty!".

Check my EE Article to check your VM is not on a snapshot!

HOW TO: VMware Snapshots :- Be Patient

The Snapshot Alert you have received is because of Snapshots on your SAN for the volume/LUN.

Do you use NetApp Snapshots or SnapMirror?
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39257038
I've read your HOW TO article in the past and it has saved me many times before. Very well written. To answer you question, we use both snapshots and SnapMirror. This volume is snapmirrored every 3 hours at the 15 min mark.

I believe the vCenter alert about snapshots, and the storage device snapshot alert are related.

snapshotsvolume2 snapshot
0
 
LVL 123
ID: 39257267
If you use SnapMirror, it creates a SAN Snapshot.

Is your VM running on a snapshot?
0
Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 1000 total points
ID: 39257560
The snapshots that are busy are the snapmirror snapshots and can only be deleted if/when the snapmirror is finished or you can queisce/break snapmirror, delete snapshot and review if you have a common snapshot on both sides you can continue to replicate from.

Snapshot reserve is just for accounting so that if you have space in the volume it will not do anything to your data, on the other hand if your snapshots start to encroash into your data your volume may go offline as you'll run out of space.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259882
I'm just trying to figure out what is causing this to occur:

I was under the impression that snapshots start out at zero, and then grow based on changes in the data compared to the previous snapshot. Having said that, based on my previous post, what does 93.91GB Cumalitve Total Size mean? Does that mean that there has been 93.91GB of changes since the last snapshot?

And lastly, what's the difference between the two columns, Total Size vs. Cumalitive Total Size?
0
 
LVL 123
ID: 39259932
sizes of snapshots will be based on the "delta" changes between each snapshot.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259965
Correct, so does that mean that on 6/18/2013 03:15:03 (from above image), there was 92.65GB of changes?

I'm not sure how to read the above image and Total Size vs. Cumalitive Total Size.
0
 
LVL 123
ID: 39259985
total size is the size of the snapshot, (delta change, e.g. changes between snapshots)

cumulative all the snapshots.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 39261954
The Netapp snapshots work a bit different than your copy on write snapshots in that they're pointer to the data.  So let's say you've added a VM, no snapshot space is being used but if you have a snapshot with a VM in place and you delete it then the snapshot will be holding the data and the snapshot will grow.  It can also be changes inside of the VMs, let's say a database that will look as different data if it's encrypted or binary where there is no way to dedupe it on the WAFL.
0
 
LVL 20

Expert Comment

by:compdigit44
ID: 39278745
Have you checked your datastore or orphan snapshots? Sometime not all snapshots are removed completed and are not listed in snapshot manager.
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The question appears often enough, how do I transfer my data from my old server to the new server while preserving file shares, share permissions, and NTFS permisions.  Here are my tips for handling such a transfer.
August and September have been big months for VMware—from VMworld last month to our new Course of the Month in VMware Professional - Data Center Virtualization. We reached out to Andrew Hancock, resident VMware vExpert, to have a more in-depth discu…
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…

656 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question