10GB Performance Disaster

[rogerk]

hi,

do you have already done this settings?

http://forum.open-e.com/faq.php?faq=...please_give_me

do you activate direct cache access in bios on your mainboard for dss and vmware?
(in some mainboards you find crystal beach to set up direct cache access)

greetings
rogerk

[dwright1542]

hi,

do you have already done this settings?

http://forum.open-e.com/faq.php?faq=...please_give_me

do you activate direct cache access in bios on your mainboard for dss and vmware?
(in some mainboards you find crystal beach to set up direct cache access)

greetings
rogerk

yes I did.

[rogerk]

can you use insteed of VMWare an Windows 2003 or 2008 installtion to access DSS?

We use Citrix XEN Server and DDS6 with 10GB from Intel with good performance.


have tested with sqlio and these parameters?
sqlio.exe -kW -s20 -frandom -o64 -b4 -BH -LS -Fparam.txt
size of testfile 10MB
you should get roundabout : 50k IO/s and 200MB/s

if not i think there is an Hardware Error or some misconfig on VMWare site.

greetings
rogerk

[1parkplace]

If you read the release notes, Chelsio and Netxen are iffy choices for 10gbe NICs. I'm using intel based 2-port 10gig cards and they work great. I had major problems with NetXen cards which I was using before these.

Are you using Block I/O or File I/O for your Volumes? Either way, check the following and retest:

From the Console: Ctrl + Alt + W -> Enter Admin Password -> Tuning Options -> iSCSI Daemon Options -> Target Options -> Pick your target

MaxRecvDataSegmentLength: 8192
MaxBurstLength 262144
MaxXmitDataSegmentLength: 8192
FirstBurstLength: 65536
DataDigest: None
MaxOutstandingR2T: 1
InitialR2T: Yes
ImmediateData: No
HeaderDigest: None
Wthreads: 8

Have you tried to enable jumbo frames on the switch / DSS / ESX? I found this article relating to this : http://www.interworks.com/blogs/kcul...s-vmware-esx-4

I am not using Jumbo Frames, however you can try it...

[dwright1542]

If you read the release notes, Chelsio and Netxen are iffy choices for 10gbe NICs. I'm using intel based 2-port 10gig cards and they work great. I had major problems with NetXen cards which I was using before these.

-----------------------
Intel on the DSS side, Chelsio on the VM side. (They were NetXen)
----------------------------

Are you using Block I/O or File I/O for your Volumes? Either way, check the following and retest:
------------------------------
File I/O
-------------------------------

From the Console: Ctrl + Alt + W -> Enter Admin Password -> Tuning Options -> iSCSI Daemon Options -> Target Options -> Pick your target

MaxRecvDataSegmentLength: 8192
MaxBurstLength 262144
MaxXmitDataSegmentLength: 8192
FirstBurstLength: 65536
DataDigest: None
MaxOutstandingR2T: 1
InitialR2T: Yes
ImmediateData: No
HeaderDigest: None
Wthreads: 8

Have you tried to enable jumbo frames on the switch / DSS / ESX? I found this article relating to this : http://www.interworks.com/blogs/kcul...s-vmware-esx-4

I am not using Jumbo Frames, however you can try it...

No Jumbo frames....I should still be able to push 300meg /sec without.

[dweberwr]

My DSS box will be here in a few days (finally!). I am going the 10 gb route myself on my dss box and found this thread quite interesting. What I don't understand is why would you NOT enable jumbo frames? The investement to 10 gbe stuff is still enough over 1 gbe, why would you not want to eek out every ounce of performance that 10 gbe has to offer, especially when all it takes is a few keystrokes and mouse clicks?

[1parkplace]

Block I/O or File I/O?

Did you apply those tuning options and retest?

[dwright1542]

My DSS box will be here in a few days (finally!). I am going the 10 gb route myself on my dss box and found this thread quite interesting. What I don't understand is why would you NOT enable jumbo frames? The investement to 10 gbe stuff is still enough over 1 gbe, why would you not want to eek out every ounce of performance that 10 gbe has to offer, especially when all it takes is a few keystrokes and mouse clicks?

Aha well, my DSS can't pass enough data to saturate the link, even with non-jumbo frames.

The other issue is that I have a tap into the iSCSI vlan, and I'd rather not have to set jumbo frames if I don;t have to on any management box that may need to be on there.

There's also the ease of migration....with little downtime.

-D