In my experience, the key is spindles. It's easy to get terabytes of storage very cheaply now, but if you don't have the iops to keep up with your workload, then all of that space is wasted. I would recommend that in regards to storage, think about your workload as if it were physical. If you would set up a database server with four drives in a RAID10 for performance reasons, you should dedicate a LUN with four drives for your virtual database server to get the same performance. Also, make sure you have enough iops for all of your virtual guest's OS partitions. I don't know how many guests you have and how you plan on splitting up your LUNs across your drives, so that will of course make a difference. 16 drives split between 5 guests is a much different load than 16 drives split between 60 guests. I've found that oversubscribing your "OS partition" LUN leads to poor performance even on guests that don't seem like they're doing much. Windows has a lot of background i/o and if your disks get saturated, they'll stutter and appear to hang for seconds at a time.Originally Posted by asingh
With regards to an iSCSI HBA vs. a quad-port NIC - I don't have a strong opinion as far as performance goes. I have some VMWare servers with HBA's and some with software iSCSI through an Intel NIC. VMWare has said that they see future development in the software iSCSI stack, so there may be more performance gains in the future by using software iSCSI. vSphere now does MPIO with the software iSCSI stack, so that puts it on par with the HBAs.
Hope that helps,
James