Hardware Recommendations?

Hey Guys,

I see a lot of people post their specs on these forums and a lot of good performance numbers coming from some of them.

I'm interested in building a homegrown machine, preferably 12+ hot swap SAS drive bays.

Anyone have any hardware recommendations? (ie. RAID Controllers, Mainboards, Drives, Chassis) I'm looking for good DSS hardware support, not really interested in experimental hardware.

I will be using Intel 10GbE NICs to connect this new Storage to my existing storage network.

If you have benchmarks to back up the numbers that would be great.

Thanks for your input!

Drew

Hi,

we have build our system in april-may.
Hardware is redundant:
Mainboard Supermicro X8DT3
Chasis SC836
CPU 1xIntel Xeon 5550W
RAM 24GB (6x4GB)
Raid Controller Adaptec ICP Vortex 8165BR with BB 256 Cache
HDD 16x Western Digital 1TB Raid Edition 3
NIC Intel 10gbe CX4 Dual Port

Switch
2x HP 6400cl CX4

we have 2 Citrix XEN Server as "Client"
Hardware 2xIntel Xeon 5550W with 32GB RAM and Intel 10gbe CX4 Dual

Performance Windows 2008

with write cache on: ca 90000 io/s
with write cache off: ca 25000 io/s

Performance with Citrix XEN (windows 2008 guest):

with write cache on: ca 66000 io/s
with write cache off: ca 15000 io/s

there was no differenz between raid 10 and raid 6 !

so we use raid 6 iscsi for our production system.

the perfomance measure was taken with sqlio and the folowing parameters

sqlio -kW -s60 -frandom -o64 -b4 -BH -LS -Fparam.txt
d:\testfile.dat 2 0x0 10

for xen we make some special config.

we also tested dss with 2x Xeon X550w and 48GB RAM , but the system seems to be slower with that config.


so i hope this will help you.

regards
roger

Thank you, I will investigate the cost of this setup,

Which SuperMicro board did you get exactly? I see there are many models, X8DT3, X8DT3-O, X8DT3-F-O, X8DT3-F???

Also roger, I was not able to find the Adaptec 8165BR, did you mean the 5165BR? http://www.adaptec.com/en-US/support...aid/ICP5165BR/

hi drew,

you are right. i mean the 5165br and the board is the x8dt3.

rogerk, what sort of performance have you been getting with larger test file sizes? Like, twice or thrice as big as your ram?

1 thread writing for 20 secs to file d:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use hardware disk cache (but not file cache)
size of file d:\testfile.dat needs to be: 41943040000 bytes
current file size: 4194304000 bytes
need to expand by: 37748736000 bytes
expanding d:\testfile.dat ... done.
using specified size: 40000 MB for file: d:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 147.68
MBs/sec: 0.57
latency metrics:
Min_Latency(ms): 5
Avg_Latency(ms): 369
Max_Latency(ms): 7260
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

this was a 40GB File from our exchange 2003 on citrix xen.
what a performance do you aspect?

regards
roger

Ouch, those results seem fairly bad to me... Especially the latency.

i made another test with a non production vitual machine
testfile 40GB

C:\Program Files (x86)\SQLIO>sqlio.exe -kW -s20 -frandom -o64 -b4 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 1 thread (0) using mask 0x0 (0)
1 thread writing for 20 secs to file g:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 64 outstanding
size of file g:\testfile.dat needs to be: 41943040000 bytes
current file size: 10485760 bytes
need to expand by: 41932554240 bytes
expanding g:\testfile.dat ... done.
using specified size: 40000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 33119.70
MBs/sec: 129.37
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 1
Max_Latency(ms): 403
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 84 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

so i hope this help

i think the test above was made on the wrong machine.
our exchange has a 300GB database...

Quote:

---

Originally Posted by rogerk

i made another test with a non production vitual machine
testfile 40GB

C:\Program Files (x86)\SQLIO>sqlio.exe -kW -s20 -frandom -o64 -b4 -LS -Fparam.tx
t
using specified size: 40000 MB for file: g:\testfile.dat
IOs/sec: 33119.70
MBs/sec: 129.37
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 1
Max_Latency(ms): 403

---

That's more like it!

We need to get a 10GbE card to test with...

C:\Program Files (x86)\SQLIO>sqlio.exe -kW -s20 -frandom -o64 -b4 -BH -LS -Fpara
m.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 1 thread (0) using mask 0x0 (0)
1 thread writing for 20 secs to file g:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use hardware disk cache (but not file cache)
size of file g:\testfile.dat needs to be: 83886080000 bytes
current file size: 41943040000 bytes
need to expand by: 41943040000 bytes
expanding g:\testfile.dat ... done.
using specified size: 80000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 30826.99
MBs/sec: 120.41
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 1
Max_Latency(ms): 787
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 78 21 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

this was with a 80GB file

in a non virtual system this should be 30% better.

regards
roger

How are reads?