Sizing Exchange for Very Large Implementation

Posted on 2009-04-23
Last Modified: 2012-05-06
We are being asked to upgrade an exchange 2003 system to Exchange 2007. The current system has 250,000 users in 4 Active/Passive Cluster servers. This seems to work ok, since most of the users accounts are not very active. (about 30,000 are very active).

We want to see if the plan for upgrade is comparable to other plans for systems of this size. Can you guys give me advice about how many servers, virtual or non-virtual, memory, etc, for this engagement?

We think we have a good solution, but want to see what others say.
Question by:sunnylowe
    LVL 11

    Expert Comment

    With 30.000 active users and 250.000 you already have quite a large exchange organisation, i assume your address books alone already produce quite some noticable traffic ;)

    Well to come to the point - i find it quite "brave" to spread these among just 4 servers, but hey after all it is not numbers where i would say "impossible", but having growth in mind it would be the more economic solution if you at least gave a thought about having more servers.

    So for your current setup you would "just" need 8 servers. And when i say server i dont mean 1U pizza box style servers, i mean real monsters. For your Exchange 2003 i would have NOT recommended it, but with Windows 2008 and Exchange 2007 you would benefit from more processors. In 2003 you would have had not much fun with having too many cores, but now here's your chance. Even 32 core systems are no problem for exchange 2007, so finally a fun machine to administrate. How many cores you really need depends on your average load, your current system should give you an impression about what you should calculate with.

    If you are really having a 4/4 cluster is a question better not to discuss - one could write books about it and not have said everything. I would say the more economic solution would be to double the number of virtual exchange servers, with this you could use smaller scale single servers. At first glance this does not sound like a win, but in the long run 2 affordable midrange servers are way cheaper than the 4U monster model. This depends a bit on your supplier, but we have Fujitsu and they tend to be a pharmacy when its about "the big ones". We have one of these beasts where a single replacement backplane costs more than a small server.

    However, this is much of a strategic/economic solution, after all you also double administrative overhead - on the other side if you expect growth it is much easier to avoid bottlenecks with more servers. You can also double virtual servers but not double physical servers ofr the cost of failover safety, but in your scale it is unlikely its a company that loves to save on safety. I assume you have a "Two server locations" concept, so that will most probably not be an option anyways.

    I am too conservative for a virtual setup here. Yes virtual servers will give you more flexibility, but in a dedicated environment i think with good planning virtual servers would do more harm than benefit as you need to have aditional administrators or skill, have an additional "point of failure" and additional licensing cost and planning overhead. But as before - this is heavily dependant on the company strategy. My personal oppinion is that well planned dedicated systems are not a typical candidate for virtualisation.

    Memory: Well with Windows 2008, 64 Bit and loads of Gigabytes of memory life can be so much more fun. Goodbye /3GB switch. Generally the more the better. For your "4 virtual servers" i would recommend at least 32 GB per machine to have fun with these. Unlike several years ago memory became so cheap it is no longer a strategic value, Good rule: If unsure just double it, unlike old 32 bit versions with the /3GB switch windows and exchange can finally handle memory in a way it should always have been.

    In the end this is all my oppinion. In the end there are many possible solutions and if well planned they all lead to a happy end ;)

    LVL 1

    Accepted Solution

    My current design is 9 dual Quad servers with 128gigs of RAM each on replicating SANS between 2 sites. We will set up 5 in the primary and 4 in the secondary, then allow the SAN replication to work for DR, and bring up all the VMs on the other server set, in case of a disaster.

    Microsoft and VM ware say Exchange can't handle more than 32 gigs of ram, and that 8k heavy users per server is a lot.

    VM says they can get 16k heavy users by going virtual, and maxing memory out at 128gb. But, they want single store servers, with lots of VMs in the machines. (500 users per).  (see attached).

    This probably means I can get tons of mild and regular users using this method, and take advantage of the RAM. Exchange is not all that processor intensive, so will throwing a bunch of CPUs at the problem help?

    What I really need is reflections from you on a setup this large that you may have been involved with. We are looking at probably 25 VMs on these 9 servers.

    LVL 11

    Assisted Solution

    If you're really into "more CPUs > more servers"  then Windows 2008/Exchange 2007 are a good team. Keep in mind that the enterprise edition of Win2008 only supports up to 8 processors at once, for more you need the datacenter edition which may be a strategic decision due to the notable license pricing (If you have an enterprise agreement it is a bit more attractive).

    I already said my oppinion about virtualisation for dedicated servers. imho 1 big physical server running 1 big Virtual exchange server is just easier to maintain, but thats just from an administrator who suddenly has 25 Servers to maintain which really DOES increase the work. However if they want it that way they probably know what they are doing.

    However i have looked through the supplied white paper and from the planning it is not just a valid but a good scenario, i have to underline the great flexibility in this setup. Exchange is generally not that CPU hungry, but it depends a bit on the scenario, user behaviour and special features. In your case RAM should not be a problem, with 128 GB of Ram a machine should be able to easily host 10 of your "500 user servers". The number of CPUscan however help in such a large setup. Exchange is generally not CPU hungry, but peaks can occur that you need to be able to compensate. Although a 16 Core (4x quadcore) could be sufficient i would tend to rather go for 8 x quadcore per machine if the budget allows it to offer a greater range of flexibility. I'd rate disc I/O as the biggest problem for these monsters, but the EMC from your white paper should be able to take it.

    All in all I'd rate it a very convincing white paper.
    LVL 1

    Author Comment

    I will be only allocating 1 or 2 cpu's per virtual Machine, so the numbers of CPUs in the Physical machine won't be that important when considering what kind of 2k8 license to buy.

    Featured Post

    IT, Stop Being Called Into Every Meeting

    Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

    Join & Write a Comment

    Local Continuous Replication is a cost effective and quick way of backing up Exchange server data. The following article describes the steps required to configure Local Continuous Replication. Also, the article tells you how to restore from a backup…
    To show how to generate a certificate request in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.:  First we need to log into the Exchange Admin Center. Navigate to the Servers >> Certificates…
    To add imagery to an HTML email signature, you have two options available to you. You can either add a logo/image by embedding it directly into the signature or hosting it externally and linking to it. The vast majority of email clients display l…

    729 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now