slow replication speed 10G

[kellogs]

the following was changed

MTU changed to 9000

Tested tasked speed 300M 400M and 600M

Speed is still the same

both units have 16 2TB drives on raid 6 using Adaptec 5805 MAXIQ

[kellogs]

DRDB is set as following
max-buffers 8000
max-epoch-size 8000
unplug-watermark 128
sndbuf-size 2048
al-extents 127
no-disk-barrier off
no-disk-flushes off

[maygyver]

You have a combined Speed of 300MB per Second. For me it seems, that the raid is the bottleneck. Please check Write cache. ( Controller and HDD)

[kellogs]

write cache is enabled for both raid controller and HDD

Please take note that the HDD is 3G/s spec.

If i am getting 300MB/sec ... i think that sounds about right ... but does it matter if I have 16 disks (spindles) thus the speed should be higher considering it is dedicated 10G link?

[maygyver]

Yes the HDD is 3GB/s spec.
But a single HDD will never do it.

You have 16 disk with RAID6. I think 300 MB/s is not so bad. Should be a little bit more. May be up to 500MB/s.

Sven

[maygyver]

Is the read Cache enabled?
Is the write mode, Write back or Write through.

[Jerky_san]

Type of 10Gbe nics? RPM rating of hard drive? Make/Model of raid card? Raid 6 with 16 hard drives will cause a bottleneck on the raid card.. Raid cards have major problems getting over the 600mb hurdle with raid 6. If you want more performance consider raid 10 or raid 5. I run raid 10 for my 20 disk VM system but i also replicate to a second 20 disk san both open-e. This ensures i have dual fail over encase of disk loss.