How to perform a proper sizing for multiple Terminal servers?
s,
We have a customer with 6 terminal VM servers on a single ESXi 5.5 host. The server conf is: Dell R520, 2x E5-2620 (12 cores in total), 192GB RAM, 3x local SSD.
During the busy hours of the day all 6 VM’s performs poorly, and when digging into it I’ve found the below (`amazing`) findings:
1) Each one of the 6x VM's was allocated with 2x vCPU's and 6 cores (!) – 12 cores in total, which IMHO means that each TS VM thinks that he's eligible to take ownership of the entire physical CPU's resources, right?
2) The ESXi host shows more than 100% CPU usage (see attached screenshot - R520 6xTS, fully loaded CPU's.jpg)
3) Running ESXTOP, I've witnessed even a greater pleasure. Hint: watch the %RDY + %CSTP per VM (see attached screenshot - R520 6xTS - ESXTOP.jpg). Amazing ah?
The customer told me that when he tried to allocate less vCPU’s the each VM was stuck on 100% CPU usage, so lowering the amount of vCPU’s per VM doesn’t seems like an option.
BTW, the host is currently a standalone host with no VC which manage it. There’re additional 3x ESXi hosts which are ~50% loaded each – in the organization which are being managed by a VC with an Essentials plus license (which is limited to 3x hosts).
In the bottom line, I need to do sizing for the customer for 170 terminal users and allow him to scale while he grow in the future. Currently there are 70 users and the system is dying.
What's the best way to size a TS session; Should I understand which applications the user is running? I guess there's a smarter way right, especially that the system is already configured, up and running?
The issue is urgent, and your prompt reply will be more than appreciated!
Thanks in advance
R520-6xTS--fully-loaded-CPU-s.jpg
R520-6xTS---ESXTOP.jpg
We have a customer with 6 terminal VM servers on a single ESXi 5.5 host. The server conf is: Dell R520, 2x E5-2620 (12 cores in total), 192GB RAM, 3x local SSD.
During the busy hours of the day all 6 VM’s performs poorly, and when digging into it I’ve found the below (`amazing`) findings:
1) Each one of the 6x VM's was allocated with 2x vCPU's and 6 cores (!) – 12 cores in total, which IMHO means that each TS VM thinks that he's eligible to take ownership of the entire physical CPU's resources, right?
2) The ESXi host shows more than 100% CPU usage (see attached screenshot - R520 6xTS, fully loaded CPU's.jpg)
3) Running ESXTOP, I've witnessed even a greater pleasure. Hint: watch the %RDY + %CSTP per VM (see attached screenshot - R520 6xTS - ESXTOP.jpg). Amazing ah?
The customer told me that when he tried to allocate less vCPU’s the each VM was stuck on 100% CPU usage, so lowering the amount of vCPU’s per VM doesn’t seems like an option.
BTW, the host is currently a standalone host with no VC which manage it. There’re additional 3x ESXi hosts which are ~50% loaded each – in the organization which are being managed by a VC with an Essentials plus license (which is limited to 3x hosts).
In the bottom line, I need to do sizing for the customer for 170 terminal users and allow him to scale while he grow in the future. Currently there are 70 users and the system is dying.
What's the best way to size a TS session; Should I understand which applications the user is running? I guess there's a smarter way right, especially that the system is already configured, up and running?
The issue is urgent, and your prompt reply will be more than appreciated!
Thanks in advance
R520-6xTS--fully-loaded-CPU-s.jpg
R520-6xTS---ESXTOP.jpg
ASKER
Andrew, thanks,
2 vCPUs per VM has been tried, 100% flat in the OS.
I'm not sure yet which application the users are using, it's part of the sizing we're about to do.
What else can we do to get a proper sizing?
Thanks
2 vCPUs per VM has been tried, 100% flat in the OS.
I'm not sure yet which application the users are using, it's part of the sizing we're about to do.
What else can we do to get a proper sizing?
Thanks
Proper sizing is performed by Testing and Proof of Concept, we have never managed to get more than 25 user per 2 vCPU Server!
It is difficult for anyone to tell you how many concurrent users, you will get per virtual server.
Compared to 55 concurrent users per physical server of the same specification used by the hypervisor, with less memory.
It is difficult for anyone to tell you how many concurrent users, you will get per virtual server.
Compared to 55 concurrent users per physical server of the same specification used by the hypervisor, with less memory.
the sizing depends greatly on what apps the users will run
note that in such a case, running a single TS on the physical host would perform much better. if the guests provide the same functionality, there is little to no reason not to do so.
also note that it is possible that a handful of users eat up all the resources. most likely applying per user cpu limitations would be a great help, and you might not need to use different hardware for that number of users if you do so.
note that in such a case, running a single TS on the physical host would perform much better. if the guests provide the same functionality, there is little to no reason not to do so.
also note that it is possible that a handful of users eat up all the resources. most likely applying per user cpu limitations would be a great help, and you might not need to use different hardware for that number of users if you do so.
ASKER
Guys, thanks alot.
I've got a couple of questions:
1) Why virtual TS servers will perform poorly compared to physical servers?
2) Given that today 70 users on 6x TS servers are killing the server - but users are ABLE to work, how much improvement do you think we'll benefit from a single TS (Physical OR virtual)? How many concurrent users?
Thanks
I've got a couple of questions:
1) Why virtual TS servers will perform poorly compared to physical servers?
2) Given that today 70 users on 6x TS servers are killing the server - but users are ABLE to work, how much improvement do you think we'll benefit from a single TS (Physical OR virtual)? How many concurrent users?
Thanks
1. Because the hypervisor takes away too much performance, and TS servers are generally maxed out at 90%-100% which makes them unsuitable candidates for viurtualisation.
2. You would probably be able to reduce the number of servers by half, e.g. 2 Servers for that load, but a third for resilience, should a server fail. 70 concurrent users on a server, I've not seen for a long time, we used to be able to get 77 concurrent users on a DL360G5, Windows 2003, Citrix MetaFRame 1.8, 4GB, Dual Processor, Quad Core.
3. Or, just scale out the number of TS, you require virtually. I would think another 6.
2. You would probably be able to reduce the number of servers by half, e.g. 2 Servers for that load, but a third for resilience, should a server fail. 70 concurrent users on a server, I've not seen for a long time, we used to be able to get 77 concurrent users on a DL360G5, Windows 2003, Citrix MetaFRame 1.8, 4GB, Dual Processor, Quad Core.
3. Or, just scale out the number of TS, you require virtually. I would think another 6.
i'm not too sure that there is an easy answer for 2).
like mentioned above, it is quite likely that a few users are using 100% of their available CPU. if 5 of your users are in this case, you can use 15 servers and end up in the same situation because 3 or 4 of your servers will still host at least one such user so there will still be a significant number of users to complain
i don't think it is reasonable to try to guess about sizing before looking at which user and/or app eats up the resources. also alternate solutions might emerge if you detail what are your needs. for example it may be worth a shot to virtualize a single app rather than a full desktop if that's all that is needed, or use restrained profiles that are not cluttered with various useless bloatware the users tend to install on their desktops.
like mentioned above, it is quite likely that a few users are using 100% of their available CPU. if 5 of your users are in this case, you can use 15 servers and end up in the same situation because 3 or 4 of your servers will still host at least one such user so there will still be a significant number of users to complain
i don't think it is reasonable to try to guess about sizing before looking at which user and/or app eats up the resources. also alternate solutions might emerge if you detail what are your needs. for example it may be worth a shot to virtualize a single app rather than a full desktop if that's all that is needed, or use restrained profiles that are not cluttered with various useless bloatware the users tend to install on their desktops.
You may also want to consider using tools...and these are the tools we use for large Citrix/TS farms we build, e.g. 1,500 - 26,000 concurrent users.
Login VSI
http://www.loginvsi.com/product-overview
CitraTest
http://www.tevron.com/citratest.aspx
Login VSI
http://www.loginvsi.com/product-overview
CitraTest
http://www.tevron.com/citratest.aspx
2 vCPUs per VM has been tried, 100% flat in the OS.How are you measuring the CPU usage? Are you looking at the stats on the host or the stats within the VM?
Using performance metrics for CPU from within the guest is not reliable. The hypervisor will only offer the number of CPU cycles requested, so often the CPU looks maxed out on the guest while the host still has resources in reserve.
To determine whether you're running into resource contention, look at the CPU_Ready performance counter on the host. This will tell you how long the guest has to wait after requesting CPU.
Another place to look for a performance bottleneck is disk. You typically need 150 IOPS per user in order to guarantee performance. Average IOPS will be lower, but you need to guarantee performance even during peak times.
ASKER
Thank you all.
asavener - The IOPS are SSD based, so I think it's not the issue.
In the bottom line - what would YOU do with the current situation (physical/virtual/more VM's/extra server etc..)
Thanks
asavener - The IOPS are SSD based, so I think it's not the issue.
In the bottom line - what would YOU do with the current situation (physical/virtual/more VM's/extra server etc..)
Thanks
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks for the effort guys!
However virtual servers with TS can perform really poorly compared to physical servers.
How many concurrent users, and what applications are they running?