Solved

Virtual Disk Space - understanding 2T-4T data.

Posted on 2013-10-27
7
365 Views
Last Modified: 2013-11-12
I've been asked to build a server that will house 2T to 4T of data, couple questions come to mind:
1. Is there any significance in going the virtual route? I'd prefer to go this route rather than creating more hardware.
2. To elaborate on the above question is there any compression when virtualizing?
3. Regarding hardware there may be other projects that large amounts of data are needed, one reason virtualization seems like a better route.
0
Comment
Question by:WORKS2011
7 Comments
 
LVL 119

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 167 total points
ID: 39604555
1. Physical or virtual can host large data sets virtual has larger benefits Disaster Recovery Replication Backup and Restore.

2. There is compression available in a Windows OS but not at the virtual layer.

3. We prefer to use NAS devices rather than file servers for large data sets no Windows OS required SAN snapshots for backup and restore and the benefit of faster access to files and the ability to use DeDuplication to reduce storage required.
0
 
LVL 13

Assisted Solution

by:Daniel Helgenberger
Daniel Helgenberger earned 167 total points
ID: 39604576
Most of my points below are from the storage perspective:
1. Virtualization adds complexity while yielding flexibility. In my opinion the gained flexibility outweighs this disadvantage by far, but this is yours to decide. This flexibility includes being able to overprovision and doing live migrations if you need high availability.

2. Virtual Machines can utilize any number of disk types, even whole devices and complete physical disks. This is why it is not so much a question of compression but rather overhead. Any virtual environment creates performance overheads which may be considerable. If disk performance is an issue, you need fast disks as with physical machines. Also, the use of paravirtualization is essential here, as it keeps the mentioned overhead in check. For instance KVM/libvirt runs VMs at about 90% of the storage performance with virtio drivers.
But for your question, there are compressed thin provisioned formats as virtual disk; in reality a file, which takes up only as much space as there is real data but the VM 'sees' a huge disk.

3. If you have these large amounts of data and want to virtualize, consider investing in a 'virtualized' storage solution which supports easy LUN management, thin  provisioning, HSM and snapshotting; also converged networking may be a key. These features play really well with virtual machines and help you to isolate the (virtual) machine from your precious data.
This enables you to easily recreate a new VM with the 'old' data and move it around physical machines.
0
 
LVL 17

Author Comment

by:WORKS2011
ID: 39604628
Thank you, let me know if I need to create another question I have no problem doing so, do you recommend NAS or SAN to expand network storage with a virtual environment?
0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 
LVL 17

Author Comment

by:WORKS2011
ID: 39604632
Actually I see hanccocka already answered in on this, thanks
0
 
LVL 95

Assisted Solution

by:Lee W, MVP
Lee W, MVP earned 166 total points
ID: 39604700
Just a note - when installing a new server, the question shouldn't be "is virtualization a good idea" it should be "are there any reasons not to?"

I prefer SANs for storage.  To me, they are more flexible... if your requirements change you can use that storage for other things and windows supports Dedupe features.
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39605250
For your virtual environment environment you do not need a SAN per se, though it might be beneficial if you have other needs for that storage, like file servers. For your setup, if both servers will need to access the same data, you might be much better off with a SAN.

it should be "are there any reasons not to?"
leew makes a good point here. But this highly depends on your environment. If you need to virtualize only one or two machines, then you may be better off with physical servers. But from experience once you have this infrastructure, you are adding a lot more VMs; making it worthwhile and your hardware better working to capacity - saving space and energy.
A good read, the NetApp best practices with KVM. Most of the things are is true for all hypervisors and storage components:
http://solutionconnection.netapp.com/Core/DownloadDoc.aspx?documentID=3302
0
 
LVL 17

Author Closing Comment

by:WORKS2011
ID: 39642506
thank you everyone, I took all these to heart not just dividing out points.
0

Featured Post

Use Case: Protecting a Hybrid Cloud Infrastructure

Microsoft Azure is rapidly becoming the norm in dynamic IT environments. This document describes the challenges that organizations face when protecting data in a hybrid cloud IT environment and presents a use case to demonstrate how Acronis Backup protects all data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

When we have a dead host and we lose all connections to the ESXi, and we need to find a way to move all VMs from that dead ESXi host.
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
In this video tutorial I show you the main steps to install and configure  a VMware ESXi6.0 server. The video has my comments as text on the screen and you can pause anytime when needed. Hope this will be helpful. Verify that your hardware and BIO…

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now