@Raudi: my question pointed to your issue :-) i think we will test some scenarios with block and with file IO
@netsyphon: yes, whoah :-) we decided to use as many spindles as we can due to high IO requirements. we discussed rebuilding times and we decided that a rebuild up to 30h is OK. right now we have a 12 disk 5 TB raid6 on a slower controller and it takes up to 10h for a complete rebuild with normal workload.
if this big block is going to be critical we can run all our VMs on an alternative iscsi path with our secondary dss which has replicated volumes from out primary dss
if anything else is going wrong we have an additional backup server with snapshots of all VMs , so data-loss shouldn't be any problem
our vms are mostly development systems, so downtime is not really critical for us.
@shon: we will do some tests with 2x quadcore vs. 2x dualcore xeons, i think its interesting which kind of processor does more IO in multithreaded applications like ESX farming