iSCSI R3 Enterprise performance tuning

[Raudi]

Hello,

i still have performance problems. Yesterday i updated to the current version and deleted the complete unit to made new iSCSI volumes. After this i copyed the vmdk files from my local VMFS back to the VMFS on the iSCSI Volume. A 6 GB file takes 11:15 minutes, this will be 9 MB/Sec.

If i remember right, the copy of the file in the other direction to a local single S-ATA drive was much faster. At this time the restore from tape is running so i think i got no useable value when i test it now. The backup to the LTO1 tape runs with 900MB/Min and now the restore only with 550MB/Min. (2 days remaining to restore all the data.)

At the performance chart on the esx host for the network card i used for the ESX iSCSI initaor i got never more than 30 MBps. The current value during restore is 10 MBps.

When i use IOMeter in a VM on a RAW mapped 2 TB iSCSI Volume i got for read and write a value of 35 MB/Sec.

What can i do to optimise the performance?

My enviroment:

Storage
Supermicro PDSM4 Mainboard with a Pentium D 930 (3 GHz Dual Core) and 4 GB memory.
3Ware 9550SXU 16 Ch. S-ATA II (with BBU in performance mode) with 6x Seagate 750 GB HDD's

2x ESX Host
Supermicro X7DBE Mainboard with two Xeon E5320 (1,86 GHz Quad Core) and 6 GB memory.

For iSCSI communication i use direct cables from the two on board NIC's of the storage to each ESX host.

Best regards
Stefan

[Raudi]

Note:
The Virtual Infrastructure Client seem to show the wrong value. There is a "0" missing.

The current Data Transmit Rate is shown as 12000 KBps, this will be 1,5 MB/s and can't be correct. This must be 120000 KBps corresponding to the value i read on the check_sys page on the Open-E.

Regards
Stefan

[Raudi]

Argh... The KBps are "KB per Second" and not "KBit per Second"!

But this doesn't change anything at my problem....

[To-M]

Can you test without the VM iscsi connection and just use direct with a MS initiator (MS XP or 2003) to see if it is the iSCSI-R3. Also check the tests.log in the log file from Status > Hardware > Download Logs then look for the hdparm -t /dev/sdx to see what speeds we are getting from the RAID set.

[Raudi]

Now i canceled my restore to made some tests:

To copy a 8 GB file from a single 500 GB drive (connected to a MegaRaid SATA 150-6) to iSCSI it take 13 min and 23 sec. The other direction only 5 min. and 43 sec.

The other informations:

I made a new volume and mounted it from my Windows XP with the MS Initiator. With IOMeter i got the same values i got in the VM. 30-35 while reading and 30-40 during write... (32K 100% seq. read or write)

From the tests.log:

--- Physical volume ---
PV Name /dev/sdb
VG Name vg+3ware
PV Size 3.41 TB / not usable 0
Allocatable yes
PE Size (KByte) 32768
Total PE 111757
Free PE 12
Allocated PE 111745

hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 1036 MB in 3.00 seconds = 345.31 MB/sec

This looks good. Is there a way to test only the network connection? (like with iperf)

Regards
Stefan

[Raudi]

It must be a local problem on my iSCSI box. Now i restarted the restore and during this i tested again with IOMeter from my Workstation. The value for read is now between 0 and 5 MB/s...

The workstation is connected to a Intel GigaBit Server Adapter which is placed in a PCI-X slot. The ESX Hosts are connected to the both on board Intel GigaBit NIC's.

Can i made some other tests? Where is the bottleneck?

Regards
Stefan