Are my iscsi performance results normal?

[zeki893]

my bonnie++ results
http://98.126.34.42/disk.htm

[zeki893]

i forgot to mention i'm using balance-rr bonded with 2x Intel Corporation PRO/1000 PT Dual Port Server Adapter and 2xBroadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)

on the router i have a netiron mlx configured a link aggregation with the 4 ports.
i see about 200-300Mbps on each port when i run the bonnie++ with a max at around 750Mbps so i think that translates to about 93MBps

i ran hd tach and i'm averaging about 45MBps.

When i have bonnie++ running on one target and hd tach on another i see the performance on hd tach drop to average 10-20MBps

So are these results I'm getting normal? Is this the hardware limit of my setup?

I just want to make sure I'm not underperforming. If I am, what I can do to improve its performance?

[beng]

i forgot to mention i'm using balance-rr bonded with 2x Intel Corporation PRO/1000 PT Dual Port Server Adapter and 2xBroadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)

on the router i have a netiron mlx configured a link aggregation with the 4 ports.
i see about 200-300Mbps on each port when i run the bonnie++ with a max at around 750Mbps so i think that translates to about 93MBps

i ran hd tach and i'm averaging about 45MBps.

When i have bonnie++ running on one target and hd tach on another i see the performance on hd tach drop to average 10-20MBps

So are these results I'm getting normal? Is this the hardware limit of my setup?

I just want to make sure I'm not underperforming. If I am, what I can do to improve its performance?

G'day,
One way to review the "Raw" speed of your array, is to download the logs and look for the tests.log, in there you should see a hdparm test, what is the result?
On one system we have 12x 1TB Seagates on an Adaptec 5405 all in a Raid6 and the raw speed was 135.01 MB/sec, yet the Network speed was <10MB/s (fault was with the Intel drivers in DSS)
Also, with balance-rr, as far as I know this is switch independent(as opposed to 802.3ad), so I don't know the affect if you also bond the switch ports?
Also, when you combine cards with different chipsets into a bonded interface you can get issues with incompatibilities there too. (Fault tolerance works, but not load balance.)
Try putting all the traffic through a single card and see if it makes a difference, as we had a problem with the Intel Pro1000's try using just one and compare with the Broadcom.

Rgds Ben

[zeki893]

How did you resolve your driver issue?

is sda the usb module?


hdparm -t /dev/sda
*-----------------------------------------------------------------------------*


/dev/sda:
Timing buffered disk reads: 58 MB in 3.02 seconds = 19.20 MB/sec

*-----------------------------------------------------------------------------*
hdparm -t /dev/sdb
*-----------------------------------------------------------------------------*


/dev/sdb:
Timing buffered disk reads: 498 MB in 3.00 seconds = 165.88 MB/sec

*-----------------------------------------------------------------------------*
hdparm -t /dev/sdc
*-----------------------------------------------------------------------------*


/dev/sdc:
Timing buffered disk reads: 174 MB in 3.01 seconds = 57.82 MB/sec

*

[Sh-J]

Dear zeki893,

Yes, the sda is the module. So your read performance is 165.88MB/s

/dev/sdb:
Timing buffered disk reads: 498 MB in 3.00 seconds = 165.88 MB/sec


Regards,
SJ

[beng]

How did you resolve your driver issue?

is sda the usb module?


hdparm -t /dev/sda
*-----------------------------------------------------------------------------*


/dev/sda:
Timing buffered disk reads: 58 MB in 3.02 seconds = 19.20 MB/sec

*-----------------------------------------------------------------------------*
hdparm -t /dev/sdb
*-----------------------------------------------------------------------------*


/dev/sdb:
Timing buffered disk reads: 498 MB in 3.00 seconds = 165.88 MB/sec

*-----------------------------------------------------------------------------*
hdparm -t /dev/sdc
*-----------------------------------------------------------------------------*


/dev/sdc:
Timing buffered disk reads: 174 MB in 3.01 seconds = 57.82 MB/sec

*

We created a support request for it, and received a custom update to try. This seemed to work well until we updated to the current version and speeds are now back to <10Mb/s.

RE Your HDParm, are your two MD1000's identical? or do they have different numbers of drives? I would have expected your figures to be almost the same. (assuming theer was nothing else happening on the arrays at the same time.

Rgds Ben

[zeki893]

Where can I which device these device names are?
I actually have another local raid5 volume
so sdb i think might be the local raid5 volume on a perc5 with 6x1tb drives
and sdc is the 2xmd1000 30x1tb drives raid6

i have made some changes though since my last post. i now have each md1000 with raid5 15x1tb, instead of one 30x1tb raid6

[zeki893]

here is an updated hdparm

hdparm -t /dev/sda
*-----------------------------------------------------------------------------*


/dev/sda:
Timing buffered disk reads: 58 MB in 3.03 seconds = 19.16 MB/sec

*-----------------------------------------------------------------------------*


hdparm -t /dev/sdb
*-----------------------------------------------------------------------------*


/dev/sdb:
Timing buffered disk reads: 492 MB in 3.00 seconds = 163.82 MB/sec

*-----------------------------------------------------------------------------*


hdparm -t /dev/sdc
*-----------------------------------------------------------------------------*


/dev/sdc:
Timing buffered disk reads: 304 MB in 3.01 seconds = 101.09 MB/sec

*-----------------------------------------------------------------------------*


hdparm -t /dev/sdd
*-----------------------------------------------------------------------------*


/dev/sdd:
Timing buffered disk reads: 264 MB in 3.00 seconds = 87.86 MB/sec

*