Additional storage space on ESX servers
Hello,
I have an unusual problem how to upgrade storage space on our VMWare ESX servers.
We own two ESX servers ver 3.5 u3, Std license and identical HW on both HPDL380 G5 servers (RAM- 32 GB, CPU- 2xDualCore, HDD- 2x72 GB RAID1 with ESX OS + 6x146 GB RAID5 for VMachine storage). This was intented to be a TESTING configuration and not for production purposes until we get some adequate LUN space to store VM. New LUN will not be accessible in the next six or more months, and in the mean time several production servers (for different reasons) have been migrated to VM. Now I have a insufficient storage space for VM and 12 new 300 GB HDDs planned for upgrade .
The real problem here is the downtime. Many of this servers (now VM) has been intended to work 24h per day and any interuption of them is a potential problem (I was aware of this problem before this happened but it wasnt my decision...not important anymore) .
These are some of the posibilities that I considered :
1. Backup/Restore: backup all servers from node, change HDDs, create RAID, restore servers.. It takes too many hours of downtime, I would rather try some other option.
2. RAID5: I could change one-by-one HDD that works in RAID5 with bigger one. But this way it will be recognized only 146 GB per disk of 300 GB size.
3. FreeNAS: Prepare 3rd DL380G5 server with some FreeNAS (or similar NAS)program and transfer VM. It looks good but I dont have experience with these programs and not sure how they work with.
Any new suggestions or help for the 3rd ? Thank you.
I have an unusual problem how to upgrade storage space on our VMWare ESX servers.
We own two ESX servers ver 3.5 u3, Std license and identical HW on both HPDL380 G5 servers (RAM- 32 GB, CPU- 2xDualCore, HDD- 2x72 GB RAID1 with ESX OS + 6x146 GB RAID5 for VMachine storage). This was intented to be a TESTING configuration and not for production purposes until we get some adequate LUN space to store VM. New LUN will not be accessible in the next six or more months, and in the mean time several production servers (for different reasons) have been migrated to VM. Now I have a insufficient storage space for VM and 12 new 300 GB HDDs planned for upgrade .
The real problem here is the downtime. Many of this servers (now VM) has been intended to work 24h per day and any interuption of them is a potential problem (I was aware of this problem before this happened but it wasnt my decision...not important anymore) .
These are some of the posibilities that I considered :
1. Backup/Restore: backup all servers from node, change HDDs, create RAID, restore servers.. It takes too many hours of downtime, I would rather try some other option.
2. RAID5: I could change one-by-one HDD that works in RAID5 with bigger one. But this way it will be recognized only 146 GB per disk of 300 GB size.
3. FreeNAS: Prepare 3rd DL380G5 server with some FreeNAS (or similar NAS)program and transfer VM. It looks good but I dont have experience with these programs and not sure how they work with.
Any new suggestions or help for the 3rd ? Thank you.
Additional option: Prepare 3rd DL380 G5 as an ESX host with the 300GB disks, and install the new VMs on there. Involves extra cost, but this could be a good thing as it demonstrates clearly that VMs are not "free"
if you are going with option 3, take great care with your network infrastructure, as this will be a single point of failure for all your VMs. Ensure you have a dedicated physical network (or separate VLAN at least) for the NFS traffic, and multiple paths (NICs on hosts, switches, NICs on NAS). Also consider enabling jumbo frames for the NFS traffic.
Make sure you are comfortable with whatever you implement, as the SAN may be delayed further and you'll end up running like this for a year or more!
Good luck,
D
if you are going with option 3, take great care with your network infrastructure, as this will be a single point of failure for all your VMs. Ensure you have a dedicated physical network (or separate VLAN at least) for the NFS traffic, and multiple paths (NICs on hosts, switches, NICs on NAS). Also consider enabling jumbo frames for the NFS traffic.
Make sure you are comfortable with whatever you implement, as the SAN may be delayed further and you'll end up running like this for a year or more!
Good luck,
D
Maybe a better option:
- Does can the server support or have and external scsi port?
- if so, get a cheap external scsi box for Server A and scsi card(that is supported by VMWare)
- configure it with additional space and create another vmfs store on server A
- migrate the dtat from Server A to this additional storage on Server B
- Upgrade the hardware on Server A
- Move all of the data from Server B's external storage back to Server A
- Copy the data from Server B to the External storage
- Connect the external storage and scsi card to server A and startup
- Upgrade Server B and move the data back
For temp solution, I suggest the 3rd option but I'm not familiar with FreeNAS so if you are open to other solution i suggest Openfiler http://www.openfiler.com/
The only thing that you need to understand for this 3rd option is to have at least redundant gigabit network for best performance and prefarably its own dedicated network
You can present local disks from your 3rd server via iscsi/nfs to create additional datastore to store less critical VMs to free up space on the existing datastores
Its better to down the VM for a while and move to this temp datastore. I dont really recommend to use converter at this stage as you end up wasting more space plus for VMs with DB, it is strongly recommended to stop app/DB while doing V2V to prevent DB inconsistency so basically downtime is still required, its better to use the dowtime to move VMs off existing datastore, you can do this quickly 1 by 1 until you feel there is enough space on the existing datastore
If you need help with Openfiler dont hesitate to ask
Cheers!
The only thing that you need to understand for this 3rd option is to have at least redundant gigabit network for best performance and prefarably its own dedicated network
You can present local disks from your 3rd server via iscsi/nfs to create additional datastore to store less critical VMs to free up space on the existing datastores
Its better to down the VM for a while and move to this temp datastore. I dont really recommend to use converter at this stage as you end up wasting more space plus for VMs with DB, it is strongly recommended to stop app/DB while doing V2V to prevent DB inconsistency so basically downtime is still required, its better to use the dowtime to move VMs off existing datastore, you can do this quickly 1 by 1 until you feel there is enough space on the existing datastore
If you need help with Openfiler dont hesitate to ask
Cheers!
ASKER
Hello everyone,
Thank you for your responses. I read them carefully and try to evaluate each of them as possible scenario. Before that I would like to emphasise the importance of downtime in this case and the preferable solution should minimize it.
This is what I do in the meantime: I configured VMKernel ports on both ESX servers, and started clonning VM from one ESX node to another. If everything goes well Im planning to boot up clone_VM on other ESX but it will be connected to the network only after the source_VM is disconnected (avoiding IP address conflict in this way).
After HDD upgrade on ESX server I could re-clone or migrate VM to upgraded server.
It looks that downtime could be minimized this way, but Im not sure what is going to happen with applications, logs ... until I try.
What do you think about?
Bad news I'm afraid!
This method will mean you lose any changes that are made to the VM while it is being cloned. If the application running on this VM is "stateful" (eg Exchange, SQL or any other database, or even file servers) you will lose data and likely corrupt your application. Even if your application is read-only, you will lose any events that are logged on the server (eg Audit events) during the clone process.
The only method of migrating a VM from one ESX host to another is VMotion, and that isn't possible unless you have shared storage and VirtualCenter
This method will mean you lose any changes that are made to the VM while it is being cloned. If the application running on this VM is "stateful" (eg Exchange, SQL or any other database, or even file servers) you will lose data and likely corrupt your application. Even if your application is read-only, you will lose any events that are logged on the server (eg Audit events) during the clone process.
The only method of migrating a VM from one ESX host to another is VMotion, and that isn't possible unless you have shared storage and VirtualCenter
Correction: "The only method of migrating a running VM from one ESX host to another is VMotion..."
yep, no vmotion no live migration. I think you will need to plan downtime if your company is giving you a "beer budget".
Take note, even if you have vCenter you cant do vmotion with only standard license
So no enterprise license means no vmotion so you need downtime
So no enterprise license means no vmotion so you need downtime
ASKER
Ok, Im going to boot up the server with Openfiler (ISO file is downloading) and to config NAS.
Could you help me with OF and ESX server configuration (some useful link maybe or guide, just to speed up the search on the Internet. I know the logic of this sw I played with freeNAS but couldnt make to work with VMware).
Concerning the budget this year was difficult the next one will be better :-)
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
It worked.
I have installed Openfiler as NAS on a separate server, and set up doublednetwork connections, separate VLAN and separate fast switches for NICs. VMotion is functionning also and it is completing VM transfer from one ESX node to another in app 1min.
Didnt try HA yet but Im already satisfied with obtained results. This way downtime (migration VM to NAS) is reduced on less than 10 min /machine and I improved complete ESX system.
I used first link in the recent answer and it was very useful.
Many thanks for your help!
Glad to help :)
We use this strategy successfully when we upgrading storages and purchasing new servers, there's no downtime.