This question is more for my learning than it is to find a "fix it" solution to a problem, so all helpful responses will receive points.
I have a Dell Poweredge 2600 server with a perc raid controller. I have always liked to use raid 5 for my servers, so in case of single drive failure I won't lose anything. Recently I have read about Solaris' raid-z and how it eliminates the problem of the "write hole" for raid 5.
Does anyone have experience with raid-z (and ZFS)? Is it really all it's cracked up to be? Is it really more for read-intensive applications because any time you write you have to write a whole stripe? Is the raid-5 write hole really a substantial risk (does someone have an experience with data corruption in raid 5?)
I realize these are a lot of questions, but they are all related to the topic and I'm wondering whether or not to move my file server over to Solaris and raid-z.
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
This is exactly the kind of experience and feedback I was hoping for. I looked up RAID 10, and wikipedia says it's short for RAID 1+0. If I understand correctly, I'd need a minimum of 4 identical drives, and lose the capacity of half of them for redundancy.
mrjoltcola, I appreciate what you said about parity. I think now I'm beginning to understand the risks involved with RAID arrays with parity, thanks to your explanation.
My goals are "all of the above." I suppose if I had to give priority to them, it would be:
1) data integrity/redundancy
2) space (I'm on a limited budget right now, trying to make the best of what I have... or else space wouldn't be a consideration)
3) performace (even though I list it last, it's still important!)
Right now our office is using a limping Windows 2000 Server for its file server. So my questions about moving to Solaris really revolve around the potential usability and integrity of ZFS, which seems to offer
a) a safer enivornment for our data than our current RAID 5 setup,
and b) the ability to "grow" the file system as needed (which you can do in ZFS, I think, right?)
Yes, RAID-10 = 1/2 total disk capacity, and striped mirrors. RAID-10 is not RAID 0+1, there is a critical difference. In RAID-10, each mirror can lose a disk. If you have 20 mirrors, all striped, you could lose up to 10 disks, though with each disk loss, the RAID is in more jeopardy.
Example RAID-10 below with 8 disks, each letter represents a mirror pair. So each mirror pair, is a disk unit that can be striped.
Consider each pair a mirror (RAID-1)
A1 <-> A2
B1 <-> B2
C1 <-> C2
D1 <-> D2
Then RAID-10 is the striped across all A-D, so as you see you can lose one of each pair and still keep trucking with little performance degradation. RAID-20, etc. just adds more redundancy (ie. more mirrors)
RAID 0+1 should not be considered, it is not redundant enough. It is a mirror of stripes, which means you can lose max 1 disk in each stripe, so for RAID 0+1 you can lose max 2 disks, period.
Regarding filesystems, you can grow many other filesystems dynamically, so that is not a reason to move to ZFS / Solaris.
mrjoltcola,
Yes, Wikipedia explained the difference between RAID 1+0 and RAID 0+1. I will be careful not to confuse them.
I understand that there are other file systems that can be grown dynamically: FreeBSD has Vinum, Linux has LVM, but those options did not look like they had the other benefits of Solaris' ZFS.
As long as I'm standing "in a yellow wood" (Robert Frost) looking down diverging roads, I thought I'd ask some questions about them and see which one I'd like to travel by.
I welcome any suggestions. Thanks.
It depends on your skillset (who is your UNIX admin) and your goals.
For mission critical, gigantic fileservers, Linux will work great, but until Linux ext4 and/or btrfs is a reality, a crash / fsck will take a while on bootup. I've never run any giant servers on Linux, usually 500GB to 1TB Oracle servers is my thing, and Linux has always worked great.
But if you are in many many terabytes of storage, a commercial, enterprise quality filesystem might be considered. Note that doesn't mean Solaris is better, you can still get VeritasFS for Linux as far as I know. I worked with vxfs and vxvm many times on Solaris / Sparc and its great, and according to my research recently they provide it for Linux as well.
You won't really go wrong either way.
Would not go the FreeBSD route, in your case. I would stick with a commercially supported Linux (Redhat or Oracle Enterprise Linux) or go with Solaris (which is now also Oracle. :) ).
http://en.wikipedia.org/wi
Interesting discussion, my 2c
Raid6 vs Raid 10
With Raid6 (Raid5 + Additional parity disk), you can loose any 2 disks, when loosing 1 disks gives you redundancy while rebuilding failed disk.
With Raid10 you can loose 2 disk, if they are not in the same mirror set, if you are unlucky 2nd disk could be the end of the raid set.
Summary Raid6 vs Raid10
1. Space Raid6 - only looses 2 disks to parity in set, Raid10 looses 50%
2. Speed Raid10 faster, striped over multiple pairs. Raid6 writes much slower
3. Redundancy - Raid6, loose any 2 disks still work, Raid10, vulnerable to loosing 2 disks in same mirror set, can loose 50% of disk if you are lucky and it is always just one of the mirrored pairs.
Thank you all for your input.
You guys have to check out this site: http://www.baarf.com/
Like I said earlier, I'm beginning to understand the fundamental risks of any parity-checking RAID, although RAID6 seems like the best of them. Thank you, diepes, for pointing out the vulnerability inherent in RAID10.
I'm still waiting for someone to comment who has any experience with ZFS or RAID-Z. There are many people who have a lot of good things to say about it, but it doesn't seem like many people are actually using it.
We've just finished a rollout of the latest solaris 10 across the estate. Part of that rollout was to create a local zfs storage pool on each and every server, just mirrors of space on a couple of disks. We're using these pools for home drives, quick retrieval archive logs, temporary storage, ad-hoc requests and quick and dirty provisioning of small amounts of disk space. We use ufs for all root disks and vxfs for all database volumes. I see it being many years before we might consider using zfs for anything mission critical, purely because of the wealth of knowledge and experience supporting ufs and vxfs far outweighs that of zfs.
It's easy to administer, intuitive almost. It's got some lovely features that we use such as compression, quotas, and replication/cloning. Given time, it should and could be a viable and excellent alternative to the current mainstay of vxfs and such. It really is a powerful piece of technology that is being given away for *free*. You can find yourself paying through the nose for equivalent hardware based solutions.
We would so love to have zfs everywhere as it makes file systems a comparative joy to administer but ... we're scared.
We are comfortable with what we know and its going to take a sysadmin/manager/cto with cojones grandes to migrate away from entrenched standards. Hence the gently-bently approach by ourselves and the vast majority of other Sun shops. We just don't have the time to go headlong into working with a new fs, but if you have the time and the inclination, you'd be silly not to at least consider it. In your case though, you have a hardware raid controller, so you'd leverage that with whatever raid level you can afford/is appropriate and have the advantages of the zfs file system on top. If you have a lot of storage to throw at it, then you've got an excellent volume manager too.
my ha'pennies worth anyway, hth.
Business Accounts
Answer for Membership
by: mrjoltcolaPosted on 2009-08-07 at 12:54:15ID: 25046332
I have experience losing two RAID 5 volumes in my career, both due to multiple disk failures in a volume before the administrator caught it. In one instance, the admin worked for me. :|
As such, I do not recommend RAID-5 or any other parity RAID with modern disk capacity and cost. There is no point anymore. Years ago it was the way to get great savings on disk, but nowadays, RAID-10 (or 0+1 which is different) is mostly superior. I've seen a comple of comparisons that indicated RAID-5 and other parity levels with a slight edge on reads, but it still loses in all other categories:
1) Redundancy
2) Performance including read and write in a healthy volume
3) Performance in degraded state (parity RAIDs suffer badly in degraded state)
4) Speed of recovery / rebuild (parity RAIDs are horribly slow in rebuild)
RAID-10 wins all of these. You can theoretically lose half your disks with RAID-10, though you'd have to get lucky.
Since RAID-Z is non-standard, and proprietary to Sun's ZFS, and it still is not as redundant as a RAID-10, I've never used it, nor wanted to. I've seen many arguments for the various parity RAIDs over the years, most of them are hanging by some small thread, and ignoring the rest of the benefits of RAID-10/20, etc. but I like the simplicity and cleanliness of RAID-10, and this is why I also feel it is the standard RAID chosen in many corporate datacenters for critical data and is also the one recommended by Microsoft for Exchange servers. I feel with SSD, huge caches and higher platter capacities, all parity RAID is a dead end technology, except where data integrity is concerned.
The one nice thing RAID-Z has is the filesystem integrity / self-healing data, due to involvement of the ZFS checksum, which filesystem independant hardware RAID cannot do. Thats nice.
So my question to you, what is your goal?
To achieve maximum disk space with available spindles?
To achieve better performance than RAID-5?
A combination of both?
Or to get the integrity advantage of ZFS + RAID-Z?
If the former two, I wouldn't move to Solaris just to get that, there are other alternatives.