¿Why my iscsi speed is like limited?

Hi! i´m using DSS V6 Lite (6.0up50.8401.4786 64bit) with VMWare esxi 4.0 (171294).

I have made this guide (documents.open-e.com/Open-_DSS_V6_MPIO_with_ESX4i_4.1.pdf) step by step but i can´t get a good iscsi speed.
I have saw this video too and I followed it step by step: http://www.vimeo.com/moogaloop.swf?clip_id=18113062

I have configured jumbo frames to 7000 and 9000.
I have used a dedicated switch with jumbo frames enabled.
Now I ´m using crossover patchs but i´m still getting the same performance.

Also i have configure this: (based on this post: http://forum.open-e.com/showthread.p...ght=slow+iscsi)
On DSS V6 Lite:
Here are the settings I made:
MaxRecvDataSegmentLength=65536
MaxBurstLength=1047552 (the recommended 16776192 knocked out my connection)
MaxXmitDataSegmentLength=65536
FirstBurstLength=523776
MaxOutstandingR2T=8
InitialR2T=No
ImmediateData=Yes

On ESXi 4:
iscovery.sendtargets.iscsi.MaxRecvDataSegmentLeng ht = 65536
node.session.iscsi.FirstBurstLength = 523776
node.session.iscsi.MaxBurstLength = 1047552
node.conn[0].iscsi.MaxRecvDataSegmentLength = 65536
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmedateData = Yes
node.conn[0].tcp.window_size = 65536

but i cannot improve the speed.
I don´t know what i´m doing wrong.

In DSS V6 Lite i have 2 dedicated intel PRO 1000 Gt (PCI) nics.
H/w raid 3ware 9650SE in RAID 10 configuration.
In esxi the same configuration, 2 dedicated intel PRO 1000 Gt (PCI) nics.

IOmeter shows a maximun of 25 Mb

here are some pics:

http://descargas.fiesa.com.ar/descar.../dssv6lite.JPG
http://descargas.fiesa.com.ar/descargas/varios/esxi.jpg
http://descargas.fiesa.com.ar/descar...os/esxinet.JPG
http://descargas.fiesa.com.ar/descar...ios/esxirr.JPG

Sorry for my english, thank you!!!

Two more pics:

http://descargas.fiesa.com.ar/descar...os/esxconf.JPG
http://descargas.fiesa.com.ar/descar.../esxdevice.JPG

Thanks!

From the console whats your output for the following

esxcfg-vswitch -l
esxcfg-vmknic -l
esxcfg-nics -l
esxcli swiscsi nic list -d vmhba32

Cheers
Adam

Thanks for you help AdStar

Here are the commands

~ # esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 64 10 64 1500 vmnic1,vmnic0

PortGroup Name VLAN ID Used Ports Uplinks
VM Network 0 6 vmnic0,vmnic1
Management Network 0 1 vmnic0,vmnic1

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 64 3 64 9000 vmnic2

PortGroup Name VLAN ID Used Ports Uplinks
MPIO-1 0 1 vmnic2

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch2 64 3 64 9000 vmnic3

PortGroup Name VLAN ID Used Ports Uplinks
MPIO-2 0 1 vmnic3

~ # esxcfg-vmknic -l
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk0 Management Network IPv4 192.168.0.60 255.255.255.0 192.168.0.255 00:25:90:15:d3:6e 1500 65535 true STATIC
vmk1 MPIO-1 IPv4 192.168.2.200 255.255.255.0 192.168.2.255 00:50:56:72:b6:bd 9000 65535 true STATIC
vmk2 MPIO-2 IPv4 192.168.3.200 255.255.255.0 192.168.3.255 00:50:56:7d:2a:83 9000 65535 true STATIC

~ # esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0d:00.00 e1000e Up 1000Mbps Full 00:25:90:15:d3:6e 1500 Intel Corporation 82573E Gigabit Ethernet Controller
vmnic1 0f:00.00 e1000e Up 1000Mbps Full 00:25:90:15:d3:6f 1500 Intel Corporation 82573L Gigabit Ethernet Controller
vmnic2 11:02.00 e1000 Up 1000Mbps Full 00:0e:0c:d8:22:e6 9000 Intel Corporation PRO/1000 GT Desktop Adapter
vmnic3 11:03.00 e1000 Up 1000Mbps Full 00:1b:21:86:fd:fc 9000 Intel Corporation PRO/1000 GT Desktop Adapter

If i do esxcli swiscsi nic list -d vmhba32 i get this:

~ # esxcli swiscsi nic list -d vmhba32
Could not find valid OIDs for adapter vmhba32
Errors:
List nic failed in IMA.

But if i do esxcli swiscsi nic list -d vmhba34 i get this:

~ # esxcli swiscsi nic list -d vmhba34
vmk1
pNic name: vmnic2
ipv4 address: 192.168.2.200
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:0e:0c:d8:22:e6
mtu: 9000
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 70
packets sent: 11
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

vmk2
pNic name: vmnic3
ipv4 address: 192.168.3.200
ipv4 net mask: 255.255.255.0
ipv6 addresses:
mac address: 00:1b:21:86:fd:fc
mtu: 9000
toe: false
tso: true
tcp checksum: false
vlan: true
link connected: true
ethernet speed: 1000
packets received: 70
packets sent: 11
NIC driver: e1000
driver version: 8.0.3.1-NAPI
firmware version: N/A

Thanks!!!!

an update:
i have setup raid 0 for speed test but i still have the same performance, I do not know which is the bottleneck in my setup.

thanks

Another update:

The previous tests were made with the target configured in File I/O mode with WB enabled.
I ran test with WB disable, also in the controller WB was disabled but the performance was too slow.

Recently i have configured the DSS V6 Lite with Block I/O with WB enabled. Now i have a better performance but it still slow.

Esxi says that the nics are transmitting about 30.000 Kbps each.

What other test I can do to figure out what is the problem?

Thanks!

try upgrading to latest V6 b5087, and switch to SCST1.0

Gr-R:


I´m already using DSS V6: 6.0up55.8101.5087 64bit and SCST1.0.

The strange thing is that i can move a 41 Gb archive from the esxi to the DSS V6 in less than 10 minutes but, in a VM, IOmeter shows 25 Mb maximun.

I don´t know if is the processor (athlonII x2 245, 2.9 ghz) is my bottleneck:

http://descargas.fiesa.com.ar/descargas/varios/load.JPG

Thanks!!!

Hmm your using a desktop adapter for your MPIO I would carefully check your specs on these adapters. Not saying there is an issue here, just I wouldn't use desktop cards for anything intensive and iSCSI MPIO, the rest of your esxi settings look right.

What is your switch equipment? Have you check packet through put on your switches etc

Cheers
Ad

AdStar:

Now i am using crossover cable between the Open-E and the esxi.
But if u want to know this is my switch:
http://www.intellinet-network.com/en...b-smart-switch

I will try to upgrade the processor and see if the performance improves.

Thanks!

OK.
I have upgraded to a Quad Core CPU (intel Q8300) with 4 GB of ram and a mobo with intel chipset (Don´t remember wich one).

The performance is even worst!!!

I only have a maximun of 15.000 Kbps on each Nic and a total usage of 28647 kbps (according the esxi graphics)

I give up with this. Tomorrow i think that i will go back to Openfiler.

Thank you all!!!

On DSS side:

1. From the console, press CTRL+ALT+W,
2. Select Tuning options -> iSCSI daemon options -> Target options,
3. Select the target in question,
4. change the MaxRecvDataSegmentLength and MaxXmitDataSegmentLength values to the
maximal required data size (check w/ the initiator to match).

maxRecvDataSegmentLen=262144
Maxxmitdatasegment=262144
maxoutstandingr2t=8
InitialR2T=No
ImmediateData=Yes

Set allow and deny IP's for each target.
Enable WB on targets
On VMWare side, set static targets, not discovery method.

Gr-R:

I have done what you askme. The speed has improove but i don´t know if the results are good or it can be greater.

Here are some pics:
Speed on esxi:
http://descargas.fiesa.com.ar/descar...s/esxisays.JPG
HD Tach on a VM:
http://descargas.fiesa.com.ar/descar...ios/hdtach.JPG
Configuration on iscsi initiator: (static targets configured)
http://descargas.fiesa.com.ar/descar.../initiator.JPG
Configuration on iscsi Nics on esxi:
http://descargas.fiesa.com.ar/descar.../iscsinics.JPG

If you see something wrong please telme.
Thanks for your help

The last pic:
Configuration on OPEN-E:
http://descargas.fiesa.com.ar/descar...s/openessh.JPG

An update:

Testing the transfer speed between the open-e and the esxi i have saw the horrible speed that i have, :confused:

Don´t know why iscsi is so dificult to use.
One parameter here, one parameter there but everybody has diferent results with the same configuration.

http://descargas.fiesa.com.ar/descar...sxirecieve.JPG

Send me PM, and I'll tell ya how to send me logs.
Speeds could be better, but maybe it isnt the NIC's...............

I moved the post over here:

http://forum.open-e.com/showthread.php?t=2376