slow replication speed 10G

6 volumes each with 4000GB

replicating over a CX4 - 10G direct connect

Each of those 6 replication only runs at 41668 kB/s (72340 kB/s)

This is shocking. Any tips?

MTU for the interface has been set to 15000

set MTU to 9000

and what speed did you set the tasks to?

What kind of hard disks and how many hard disks are you using? Which raid lvl? H/W Raid or S/W raid?

Some additional infos might be good for a us to see if youre performance is competitive.
How many MB/s did you choose for each replication task?

open-e support is too fast for me... ^^

the following was changed

MTU changed to 9000

Tested tasked speed 300M 400M and 600M

Speed is still the same

both units have 16 2TB drives on raid 6 using Adaptec 5805 MAXIQ

DRDB is set as following
max-buffers 8000
max-epoch-size 8000
unplug-watermark 128
sndbuf-size 2048
al-extents 127
no-disk-barrier off
no-disk-flushes off

You have a combined Speed of 300MB per Second. For me it seems, that the raid is the bottleneck. Please check Write cache. ( Controller and HDD)

write cache is enabled for both raid controller and HDD

Please take note that the HDD is 3G/s spec.

If i am getting 300MB/sec ... i think that sounds about right ... but does it matter if I have 16 disks (spindles) thus the speed should be higher considering it is dedicated 10G link?

Yes the HDD is 3GB/s spec.
But a single HDD will never do it.

You have 16 disk with RAID6. I think 300 MB/s is not so bad. Should be a little bit more. May be up to 500MB/s.

Sven

Is the read Cache enabled?
Is the write mode, Write back or Write through.

Type of 10Gbe nics? RPM rating of hard drive? Make/Model of raid card? Raid 6 with 16 hard drives will cause a bottleneck on the raid card.. Raid cards have major problems getting over the 600mb hurdle with raid 6. If you want more performance consider raid 10 or raid 5. I run raid 10 for my 20 disk VM system but i also replicate to a second 20 disk san both open-e. This ensures i have dual fail over encase of disk loss.