?
Solved

NetApp Snapshot Volume is over 100%

Posted on 2013-06-18
10
Medium Priority
?
1,982 Views
Last Modified: 2013-07-12
We received an alert from LogicMonitor.com at 5:50AM, with the below message. About 2 hours prior to that (at 3:37AM), Veeam ran a backup of our Exchange server where vCenter triggered a "VM with snapshots" alert as well (below), but the vCenter alert was cleared at 5:36AM.

I understand Veeam creates snapshots to backup a VM, but then it deletes it. I checked the Snapshot manager of our Exchange VM and it is showing there are no snapshots right now (good). So why is LogicMonitor still giving us the alert.

vCenter Alert at 3:37 AM
Target: i1whlmail01
Previous Status: Green
New Status: Red
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 50,261,657KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Green to Red


vCenter Alert at 5:36 AM
Target: i1whlmail01
Previous Status: Red
New Status: Green
 
Alarm Definition:
([Yellow Metric Is above 2,097,152KB; Red Metric Is above 3,145,728KB])
 
Current values for metric/state:
 Metric Overhead due to delta disk backings = 0KB
 
Description:
Alarm 'VM with snapshots' on i1whlmail01 changed from Red to Green


LogicMonitor Alert at 5:50 AM
The NetApp i1whlnetapp01, volume /vol/Napp01_vol2/.snapshot , is now at 166 percent of space capacity, putting it in a state of warn.
Thresholds are > 100.
This state has existed since 2013-06-18 05:37:09 PDT - or for 0h 12m.
When the snapshot volume is over 100%, it means that deleting a file will not free up space in the file system until any referencing snapshot is deleted. This makes managing disk space harder.
Recommendation: check to ensure that the snapshot policy in effect is appropriate for the kind of data being stored on this filer. If so, increase the snapshot reserve.
Otherwise, change the snapshot policy or delete snapshots.
0
Comment
Question by:pzozulka
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 1000 total points
ID: 39256978
Firstly, sometimes the VMware Snapshot Alert is "faulty!".

Check my EE Article to check your VM is not on a snapshot!

HOW TO: VMware Snapshots :- Be Patient

The Snapshot Alert you have received is because of Snapshots on your SAN for the volume/LUN.

Do you use NetApp Snapshots or SnapMirror?
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39257038
I've read your HOW TO article in the past and it has saved me many times before. Very well written. To answer you question, we use both snapshots and SnapMirror. This volume is snapmirrored every 3 hours at the 15 min mark.

I believe the vCenter alert about snapshots, and the storage device snapshot alert are related.

snapshotsvolume2 snapshot
0
 
LVL 123
ID: 39257267
If you use SnapMirror, it creates a SAN Snapshot.

Is your VM running on a snapshot?
0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 1000 total points
ID: 39257560
The snapshots that are busy are the snapmirror snapshots and can only be deleted if/when the snapmirror is finished or you can queisce/break snapmirror, delete snapshot and review if you have a common snapshot on both sides you can continue to replicate from.

Snapshot reserve is just for accounting so that if you have space in the volume it will not do anything to your data, on the other hand if your snapshots start to encroash into your data your volume may go offline as you'll run out of space.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259882
I'm just trying to figure out what is causing this to occur:

I was under the impression that snapshots start out at zero, and then grow based on changes in the data compared to the previous snapshot. Having said that, based on my previous post, what does 93.91GB Cumalitve Total Size mean? Does that mean that there has been 93.91GB of changes since the last snapshot?

And lastly, what's the difference between the two columns, Total Size vs. Cumalitive Total Size?
0
 
LVL 123
ID: 39259932
sizes of snapshots will be based on the "delta" changes between each snapshot.
0
 
LVL 8

Author Comment

by:pzozulka
ID: 39259965
Correct, so does that mean that on 6/18/2013 03:15:03 (from above image), there was 92.65GB of changes?

I'm not sure how to read the above image and Total Size vs. Cumalitive Total Size.
0
 
LVL 123
ID: 39259985
total size is the size of the snapshot, (delta change, e.g. changes between snapshots)

cumulative all the snapshots.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 39261954
The Netapp snapshots work a bit different than your copy on write snapshots in that they're pointer to the data.  So let's say you've added a VM, no snapshot space is being used but if you have a snapshot with a VM in place and you delete it then the snapshot will be holding the data and the snapshot will grow.  It can also be changes inside of the VMs, let's say a database that will look as different data if it's encrypted or binary where there is no way to dedupe it on the WAFL.
0
 
LVL 20

Expert Comment

by:compdigit44
ID: 39278745
Have you checked your datastore or orphan snapshots? Sometime not all snapshots are removed completed and are not listed in snapshot manager.
0

Featured Post

Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Each year, investment in cloud platforms grows more than 20% (https://www.immun.io/hubfs/Immunio_2016/Content/Marketing/Cloud-Security-Report-2016.pdf?submissionGuid=a8d80a00-6fee-4b85-81db-a4e28f681762) as an increasing number of companies begin to…
Giving access to ESXi shell console is always an issue for IT departments to other Teams, or Projects. We need to find a way so that teams can use ESXTOP for their POCs, or tests without giving them the access to ESXi host shell console with a root …
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:
This video shows you how to use a vSphere client to connect to your ESX host as the root user. Demonstrates the basic connection of bypassing certification set up. Demonstrates how to access the traditional view to begin managing your virtual mac…
Suggested Courses

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question