my iscsi debacle

[vkeven]

I cant seem to saturate my gb network!

I am getting realllllly bad performance out of our new iscsi system..

15 disk raid6 on a 3ware 9650 with BBC = 94mb read, 9mb write.
http://www.3ware.com/KB/article.aspx...2&cNode=1R6H2L (this is the document for benchmarking im using)

our server is supermicro 836-tq with X7DBE+ mainboard and 500gb seagate drives with NCQ enabled.

Any ideas?

If i wanted to tune some settings (receive descriptors for example) for my e1000 driver.. is there anyway i can do this?

http://www.intel.com/support/network/sb/cs-009209.htm

thanks in advance for any insight!

Could you describe more your system,memory CPU, how drives are connected to 9650?

[netsyphon]

Supermicro 836-TQ chasis + backplane
4gb memory
dual 5130 xeon procs
9650se-16 port with a raid6 (64k stripe) 15drive +1 spare with write cache enabled, storsave mode set to protection.
each disk is a ST3500630NS (seagate 500gb nearline model) set at 3.0gbps with NCQ enabled.

i have done all the updates:
Model: iSCSI-R3
Version: 2.21.DB00000010.2753
Release date: 2007-07-03
+0080-iscsi-r3_e.upd

[netsyphon]

what kernel version is the default kernel and what kernel is the TESTING?

[vkeven]

Supermicro 836-TQ chasis + backplane
4gb memory
dual 5130 xeon procs
9650se-16 port with a raid6 (64k stripe) 15drive +1 spare with write cache enabled, storsave mode set to protection.
each disk is a ST3500630NS (seagate 500gb nearline model) set at 3.0gbps with NCQ enabled.

i have done all the updates:
Model: iSCSI-R3
Version: 2.21.DB00000010.2753
Release date: 2007-07-03
+0080-iscsi-r3_e.upd

I saw on Supermicro website that this board is having onboard SATA so if I was in your position this is what I would did, take one sata drive and plug it directly on board, create temporary vg and a temporary volume on it,enable ISCSI target on this volume and now benchmark it , benchmark with software but also with real world file copy , ex. take 1gb single file and caculate how much seconds it take to write it on your iscsi target. At least we will no if your problem come from 3ware driver or somewhere else.I know it take times but I'm facing a similar issue so post your result and I will try to help you

Keven

[To-M]

Default is 2.6.17.13
TESTING 2.6.20.7

[netsyphon]

ok i have managed to get 101mb/s with iometer with both read and write. turns out my storsave profile for the 3ware was set to protection (which disables WB cache and forces data written to disk to be verified).

its really interesting because i have a FAS270 im comparing it to and its losing in the 4-512 range but killing the fas270 in anything larger (1024-16384). may be acceptable due to the 65k difference in price! not to mention this is 15k FC drives vs 7.2k Sata

the only thing not OOB so far is that i have changed the "receive descriptors" to 2048. soon i will be able to jumbo up the MTU but i predict ill just get better performance on the top end than im already getting.

i think that there is more tweaking that can be done at the ietd.conf level.

is the open-e iscsi target file based or blockio? if its file based what filesystem is being used?

and why does open-e not like iscsi-r3 as much as they like dss! (no update in the ftp folder yet).

[To-M]

New release for iSCSI-R3 will have BlockIO. Please be patient all updates follow DSS. The reason is that once DSS is completed then the iSCSI-R3 and NAS-R3 follows, as the R-3 family is DSS combined. We hope to have this ready next week.

Please remember that if you have iSCSI Logical Volumes in pre-release and you want to use the new version that is blockIO, you will need to create new iSCSI Logical Volumes and delete the existing iSCSI volumes.

Also for Replication as well you cannot have mixed versions for replication.

[netsyphon]

cool, ill be waiting for that update