Exchange transaction logs advance configuration on two compaq controller cards

Posted on 2004-08-04
Last Modified: 2010-05-18
This question is for someone who has knowledge as well as tried this:

I want to optimize an Exchange 2000 server as follows:
Server: Compaq DL380 G3
Controller 1(5300 or 642): attached  3 x scsi disks for RAID5
Contoller 2 (imbedded 5i) : attached 2 x scsi disks for RAID1

RAID1 will hold transaction logs ONLY
RAID5 will hold all the rest (database, OS, pagefile)
Note: I know the above can be improved further but pls...that is not my question

Has anyone done this type of setup? If so the 5i does not come with write but only read cache, can someone pls comfirm that I should fully disable the cache for the 5i? And give reasons pls

The 5300 and 642 both come with read and write cache, I will leave this alone, due to the nature of RAID5, again has anyone done this and if so, please give me your thoughts

Also. I'm notsure on the boot order as it will now be 2 controller cards, I'm thinking this is obivious as controller 1 will have to be first, seeing it has the OS.

Please let me know if you have done something of this nature with advantages and disadvantages?  I'm just a bit confused with all this write/read cache

Many Thanks
Question by:cybernaz67
  • 4
  • 2
LVL 15

Assisted Solution

Cyber-Dude earned 200 total points
ID: 11724171
There is a great report about write back cache. Read cache shoud not be disabled as well for the reason it helps inject data to the Computing unit on the controller thus helping data to reach faster for reading.
Here are the intresting points the report suggested:

How does Exchange use write-back caching?

To satisfy most client read requests to the Exchange Server Information Store, Exchange Server uses the Information Store buffer pool, a data structure that resides in memory. When a client requests a particular database element, such as a message text, folder, or attachment, the Information Store attempts to satisfy the request from the Information Store buffer pool before requesting the data from disk and incurring I/O overhead. This mechanism bypasses the NT system cache and provides substantial performance improvements for Exchange Server.

For write activity, Exchange Server uses a transacted mechanism that first applies the database write to the transaction log. This mechanism lets you recover the database or roll back transactions, depending on I/O success. The Exchange Server database engine then applies the writes to the database file (via the NT storage mechanisms) by moving through the transaction log, committing the transactions, and moving a pointer (the checkpoint) to identify the last transaction committed to the database.
Theoretically, if a write transaction to the database residing on disk is not successful, the Information Store must roll back the transaction and make the necessary alerts and log entries. Read and write activities occur in 4KB blocks, or database pages. A database engine must verify each page to ensure that a robust and recoverable database remains available to all clients.
On servers that use a hardware-based RAID controller with write-back caching, NT posts the Exchange Server engine's database write transactions to a write-back cache that the controller manages. The controller accepts responsibility for completing the write to the database and notifies the disk driver and operating system that it has completed the write. This process allows high-performance processing of disk write requests to the write-back cache and alleviates the need for the disk driver or OS to wait for the write operation to be completed to the slower mechanical rotating disk media. Even today's 10,000rpm drives can't match the speed of a write-back cache with 60­nanosecond RAM. All these actions greatly improve the speed of write activity to the Information Store.

Why is using write-back caching on RAID disk controllers with Exchange Server controversial?

Although Exchange can survive a hardware or software event that causes the system to fail, I have seen cases in which the Exchange Server Information Store or the Directory Service has become corrupt. Microsoft Knowledge Base article "XADM: EDB Event ID 23 Error ­1018" ( articles/q151/7/89.asp) discusses possible causes of Exchange Server database corruption.
This article suggests that write-back caching on hard disks and hardware-based RAID controllers is one possible cause for corruption of the Exchange Server database. The article offers no hard evidence or documentation for its claims. Microsoft Product Support Services wrote the article to address concerns about enabling write-back caching on hard disks and never intended to recommend against using hardware-based RAID controllers that use write-back caching.
In practice, hard disks that most server vendors ship with their products already have the write-back caching function disabled. Without the benefit of Error Checking and Correction (ECC) RAM or battery backup, write-back caching on hard disks is a dangerous feature to implement. Unfortunately, recommendations from Microsoft based on the Knowledge Base article have alarmed users about the use of write-back caching on hardware-based RAID controllers with Exchange Server.
The uncertainty around this issue stems from the confusion between write-back caching on hard disks and write-back caching on hardware-based RAID controllers. In short, don't enable write-back caching on hard disks. For hardware-based RAID controllers, write-back caching is safe if you have a controller with properly designed data integrity features.

What features does a RAID controller need for write-back caching?

A RAID controller must have a conservative design that includes ECC memory and battery-backup protection. You need to implement the cache memory architecture with some ECC mechanism that ensures protection of the cache data in the event of a RAM component failure. For example, the Compaq SMART-2/DH Array controller implements its cache in a RAID5 memory configuration. With parity information, the 20MB cache yields 16MB of usable cache.
In addition to protecting memory for the cache, a hardware-based RAID controller that supports a write-back cache must also protect that cache from power failure or interruption. In case of power loss, the controller must have battery power available to maintain posted writes that might be in the cache. When you restore power, the controller must ensure that it has applied the writes in the cache to the disk subsystem.
Other desirable features for RAID controllers include the ability to dynamically (i.e., while the server is online) configure the cache for read vs. write and the ability to designate which volumes to cache. Highly desirable is the ability to remove the cache with data intact (via a daughtercard) in the event of a controller failure, and to install the cache on a new controller and still maintain data integrity. In my group's performance testing, we have achieved the best results when we've configured controller cache for 100 percent write-back and cached only the Exchange Server Information Store volume.
Vendors that manufacture hardware-based RAID controllers contend that no evidence shows that these controllers cause database corruption with any Microsoft BackOffice products, including Exchange Server. These vendors recommend that companies use the technology to its fullest capability, but companies must follow the proper design criteria. Be careful, however, because not all hardware-based RAID controllers support a conservative design. Some use cache sizes as large as 64MB with no ECC or power protection. A thorough understanding of your hardware and of how Exchange Server interacts with it will reduce confusion surrounding this complex issue.

Though I know you may already know those issues, I think it is a nice report and there are many things you should go through when doing some of the configuration steps.

I have a question for you though; what was the guide line that guided you to use RAID5 for all OS, Exchange and Paging File while designate RAID1 for the transaction logs rather than RAID5 to the Exchange and RAID1 for the OS, Paging File and Transaction Logs?


Author Comment

ID: 11724631
Thanks, that is a very good report and has filled in some gaps.

regarding your question, the reason I came to use this setup is as follows:

The above was an example setup of what I thought would be good vs cost, as well as a possible setup for new client.

Server I'm working on has a 5300 controller using RAID 5 with 4 disks
Everything is on this and the customer is using a product called CMS which works heavily with the public folders, this has resulted in a slow response to exchange activity, mainly on the disk subsystem
I thought it would be good to use the 5i and  add two more disks for RAID1 and place the trans logs on it to help with the disk performace?
do you think  this is ok, taking cost into the matter.

On another point, ref the setup in my first question, how would you set it up if you had 5 scsi disks, a 5300 controller and a 5i, would be great if you have a better way

many thanks
LVL 55

Accepted Solution

andyalder earned 300 total points
ID: 11729899
I would buy a write cache enabler for the 5i, it's a battery pack that goes in the server at the front, you definitly want write cache turned on for the transaction logs, the users get an egg-timer until the data is written to the log. Plus buy another disk and extra RAM. Far better way to spend the money than an extra RAID controller. This is assuming you haven't bought an external storage box and actually have 7 drives available - you don't mention disks for the OS in your question.
Normal exchange setup if you only have 6 drive bays is a pair for the OS inc pagefile, a pair for the transaction logs and since there are only two bays left a pair for the data. All RAID 1.

If you only have 5 disks then the poor old transaction logs usually go on the same disks as the OS and pagefile and the data as RAID 5 on the other 3  but you shove lots of RAM in it to stop it paging. Never seen the data shared with the OS before, for exchange or SQL and similar databases but it's an interesting thought, theoretically unless you do anything with the OS like read the event log or touch the mouse or keyboard (real ones or virtual ones through iLO) the OS part would not access the disks leaving them free for the data to run on. But then you can't make it a DC or global catalog server since GC exercises the OS disks quite well.

Do you want a tape drive with that? it could explain 5 drives rather than 6.

Smartstart will probably put the additional controller first in the boot order but you can change it, also disable hotplug reservation unless you have hotplug cards, it is on by default and just wastes resources.

Do they have an external disk shelf or not? - half a one would be useful. How many users and how much mail are we taking about? Is there a tape or do you already have a backup server? Do the  perfmon logs show mainly reads or mainly writes?
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.


Author Comment

ID: 11731497
The server that I am currently amending, has a 5i, and I have purchased the battery pack for it.
Ref the OS, I mentioned in the original question that the RAID5 will hold OS: Here's the extract:  "RAID5 will hold all the rest (database, OS, pagefile)"
No external disks, permon showed high writes.

I'm not a fan of RAID1 purely for the amount of disk space that gets wasted, however I do know that the transaction logs are very write intensive and hence adding the additional RAID1 and placing the logs onto it should improve the performance....don't you agree?
Although the client site is fairly small, approx 70users, they are using a product called CMS which is putting a heavy burden on the trans logs, and hence I came up with this design.  I am implementing this all on Saturday morning and will let you all know the results.

Here is my setup again, but in detail:

Currently I have the DL380 with a 5300 controller with 4 disks on RAID5.  5i not being used (this was already configured like this when I took it on)

On Staurday:
Enable 5i, insert battery pack and build 2 new disks as RAID1
Move transaction logs to the new array.

I'm hoping with the amount of overhead this CMS product is causing on the trans logs, will immensly improve the performance, once the trans logs have there own RAID card as well as RAID1 disk subsystem.

You have an interesting point on having the database,OS and pagefile on a RAID1, however I had the understanding that seeing as the  transaction logs (being synchronous as well as write only)  perform better on RAID1 on there own, which is better write performance than RAID5, and the database being read and write would be better off on a RAID5....with all the rest.  Have you had a setup of Exchange the way you mention, and if so, rought how big was the site.

I will be purchasing a whole new system for another client after the results of this, however I will take your design into consideration
Thanks and look forward to your feedback

Author Comment

ID: 11744097
OK, didn't go too well, mainly my fault, as I forgot the main basic thing:  Could not find any documentation on how to actuall y setup both the 5300 and 5i controllers together.  At the moment all the 6 disks plugged in the front (DL380 G3) are connected to the 5300.  Cannot find any information as to how I am going to connect 2 of them so that I can then use the 5i to create a mirror set.
Either of you guys have anythign on this, as I will continue this on Monday night GMT.

Important things I learnt today which may be useful to know about the real world experiences  :0)
1. The ACU will show a warning sign that the Cache Enabler Battery (CEB) for 5i is very weak. Tip: Best to connect the CEB 1 day before you do the configuration to fully charge better.
2. On my RAID5 I had a C and D drive.  C was Basic (boot partition), D was the additional (dynamic partition), During my config of the 5i card, upon reboot, the D drive disapeared and all I saw was "foreign drive" shoiwing in Disk mgt.  To resolve this you have to add the foreign disk back into the system via Disk mgt and as soon as you do that it will reappear showing all your data.

Please: if anyone has any documentation as to how I get 2 disks on the 5i with the existing 4 disks as RAID5 staying on the 5300 controller, will be cool.
Many Thanks
LVL 55

Assisted Solution

andyalder earned 300 total points
ID: 11762589
Can't be done to split 2 disks off. You can split 3 off, there's a duplex cable in the middle of the disk assy backplane. If you look at the numbering on the front you'll see the 3+3 option from the bay numbering scheme.

You mentioned upgrading the box to a faster one but it's not CPU speed that's slowing it down by the looks of it but the fact that you only have 6 disks. People often buy these large capacity disks and expect systems to perform well but in reality lots of small disks are much faster than one or two big disks. The G4 series is out now but the I/O is still based on the 10K (aprox 110 I/O per sec) or 15K (aprox 160 I/O per sec) so I doubt that would be an improvement. If money is available for a faster box I would spend that money on an external shelf instead, then you can have 2 internal for the OS, 2 internal for the logs and up to 14 external for the data. Alternatively you could have 7 external bays for this server and the other 7 could be used by another server since the MSA30-DB has a split bus.

What speed are your disks? If they are 10K you can easily upgrade them to 15K, just pull one out and replace with a 15K and let it rebuild overnight, then replace the other one the next night. Turn rebuild priority in the ACU to high or it may still be rebuilding the next morning. Not worth upgrading the OS disks, 10K is fine for them.

You're right about the batteries being shipped uncharged, it's the same with the external storage boxes and the PCI controllers.

I've only set up small exchange boxes with the logs sharing the OS drives, it's not good practice but there's little option if there aren't enough disks to have 2 for the OS, 2 for the logs and the rest as RAID 0+1 or 5 for the data. I asked the 6 disk question in the SQL area (and there's very little difference between how Exchange and SQL work with the drives) and everyone said 3 mirrored pairs so that's how I set them up, and I confirmed with my colleagues it's how they set theirs up as well. I know it's a waste of disk space but then so is RAID 10 which is commonly used for databases for speed and I've even seen it used once for the transaction logs.

Author Comment

ID: 11793504
Thanks for all your feedback.
I got some documentation from HP which lets you split the backplane.  The only way it can be split to two controller cards in duplex mode is as follows: SCSI IDs 0,1 for one card.  SCSI IDS 2,3,4,5 for the other.  With this in mind, I cannot use two cards, since the first card has already been configured with IDs 0,1,2,4.  To change this will mean a rebuild, and so we decided to use the 5300 controller for both the existing RAID5 and the new RAID1.
Although this has helped, we have found out the application is still slow, and that the delveloper is working to improve it.

Thanks again

Closing ticket

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question