Building a iSCSI-SAN?

Regarding "big SATA disks", I recommend "Enterprise class" drives that are still way cheaper than all SAS HDD but can benefit of a UBE/BER of 1 per 10E15 (Means a 0.88% probability of 1 unreadable sector per 1TB read)
As an example, one can check that the Seagate Constellation 500GB 2.5 HDD exists in both SAS 2.0 and SATA 2.0 interfaces and has a 1 per 10E15 UBE.

Regarding "white boxes" sorry my french, I was not thinking of building it yourself (cf "on site next day warranty" above)...but to look for other less known brands.
An example at icc-usa (using the SuperMicro SC216 24x 2.5" hot swap 2U rack)
==> If you have the money to pay for the brand : GO for Dell, HP, IBM or EMC of course...

My current experiences regarding high availability is :

• Too complex configurations have more unplanned time than others and the unavailability takes longuer to get fixed 
• The key points are monitoring tools and production guys competencies 
• Spare servers and storage means every production guy nows how to efficiently execute the disaster recovery plan 
• Backup on disk, backup offsite 

@andyalder : if your "dedicated highly available storage server" has its "motherboards fails", you are having the same problem that you were complaining about above...but I have a spare ready, do you ? The point is that dual controllers means dual path drives whose are way too expensive for their low availability benefit. Most of the time, some RAIDed drives or a redunded PSU fails, with no serious impact...this is a matter of risk vs investment and not one of my "rules of thumb"

I did not mention a "dedicated highly available storage server" since that implies something that has a motherboard. I said "dedicated highly available storage box".

There is no such thing as a motherboard in dual controller SAN boxes, just two controllers. I suppose you could say they have *two* motherboards in them if you want to call the controller a "motherboard with in-built interfaces and interconnect for heartbeat and cache mirroring" There's no SPOF in such boxes unless you count the passive interconnect between controllers.

Why all the bold rich text stuff? It just makes what you say unreadable.

Hello again.
Thank you very much for you really very interesting information.
As i also read a lot in the last days i learned a lot about the howtos of SANs and the interaction with hypervisors...
With all this info i came to the conclusion, that AT THE MOMENT a full virtualized storage solution is in every case way beyond our financial possibilities (also with the fact in mind, that such a storage has to be fully redundant).
In the meantime we will stick with Xenserver and local Savvio SAS raid 10 installations.
With this config, we are able to collect the right number of drives in advance and then make a switch to SAN somewhere in the future with these disks.
THEN we will build two Quad-Core Servers with each 96gig of RAM, a 12port Raid60 Controller and 10gbase T Ethernetcontrollers. 1 Conroller leading to the Switch for the VMservers and one controller leading to the second stroage for redundancy.
With this config it should to keep the whole storage mirrored in realtime and to have the right performance for the VMservers.
At last there is only the question how many machines and users can run with such a storage and how much users and machines can run parallel on one vmserver?
Do you have any clue on this? (for simplicity we assume that 10 users have one sql/fileserver one Terminalserver and one Scalix-Server)

I thank you two guys again very much as you helped me to get my understanding of "enterprise" virtualization to a higher level :-D

I give a little addition of my hardware plans:

Storage:
Quadcore Xeon with 96Gig ram, 12 SAS Savvio with 300Gig, Adaptec 51245 SAS Controller, 2x 10GbaseT Controller

mirrored to a second one with the same config.

three VMservers :
dual Quadcore Xeons with 96Gig RAM (expandable up to 144gig) and Xenserver, no local storage. 1 10Gbase T Controller

Each FileSQLserver is either Win2008X64 with 8Gig RAM or (a few old installations) Win2003 std. 32bit with 4Gig RAM.
Terminalservers are mostly Win2008 X64 with 6 Gig RAM (10 -20Users) or Win2003 32bit with 4Gig (max 10 users)
The Scalix Servers are Centos VMs with 4 (<15 Users) or 6-8gig RAM (>15/20 Users)

All users are commonly using applications for financial- tax- and payrollaccounting (MS SQL Backend), Word, Excel, Outlook.

@andyalder: English is not my native language...I felt agressed reading some of your arguments...I am sorry about that

Dual controllers with embedded interconnection interfaces, drives and redunded PSU : I agree that those should render a reliable storage system with no SPOF....but can you ask DELL why they only offer their MD3000 (non iSCSI) to their PowerEdge servers only...
Those "do-all-the-storage-task" controllers are more complex than a classic raid hba and not in a high volume market so they may not be as reliable as marketed. Any serious study about this reliability ? If not, I can tell you the story of a large french retailer that was using a SAN and had TWO major (1h to 3h) intraday failures last year due to the SAN FC switch firmware.

Regarding the SAS Savvio 10K 300GB and your other hw choices, I would recommend :

• to mix some SATA HDD and SSD to get more capacity and more IOPS at a same or lower price - for example, getting 4x Ocz Vertex 250GB + 8x Seagate Constellation 500GB should offer both capacity (500GB r10 + 2TB r10) and IOPS (1600iops + 700iops)...definitely more than 12x SAS 10K 300GB (1.8TB r10 1400iops) capability 
• to expect not being able to reuse your SAS drives in a future SAN as they may not comply with it - for example, many SAN requires drives with dual port SAS or FC interfaces where $500 per a 300GB 10K drive is a "very good" price 
• to buy TWO raid hba instead of one ... and please, no raid 5/6/50/60 without a write back cache backed by a battery 

@BigSchmuh, sorrt if I came over as agressive, I just don't think a mix of SATA and SSD disks will work in many environments unless there is some automatic tiering software involved, It's just too much work to administer if not.

I can see how a firmware problem could knock out one fibre channel fabric, strange that it would knock both out at the same time though, presumably they did have redundant fabric just like alexciric is presumably going to buy two switches for his iSCSI SAN to run on and not connect them together.

Thank you guys for your input.
Now i am thinking of the following config:

SAN Hardware: (2x )
12x SAS drives (SSD i think, i dont want to mix drive models for higher compatibilit/reliability/managementcapabilities...) in a Supermicro Sc216 Case with an Adaptec 51205 Controller. Then 96gig ram, Quadcore CPU, Openfiler. 2x Dualport 10Gbase T Controller. (1 to the VMs, the 2nd to the second SAN). The 10gbt ports will be trunked to have 20GbaseT.

Also we will get two 10Gbase T Switches to avoid the redundancy traffic on the connection to the VMservers.

For the VMservers working with Xenserver i think about dual Quadcore machines with plenty of ram and also 10gbaset interfaces. (dual? trunked???)

Two of these devices should be able to  mirror the data in realtime and tto have enough performance to handle around 150 Users in several VMs.  (We are thinking of ~20-30 VMs on 4-6  dual Quadcore machines)

With this config it should be possible to reuse existing drives for the inital phase with low utilization and to scale to larger dimensions in the future.

What do you think?

Also i am a bit confused as an acquaintance gave said that virtual Terminalservers lose around 40% performance compared to native machines??? I cannot believe this. What is the point with this statement?