Using bonded nics for replication

Printable View

05-03-2010, 04:47 PM
webguyz

Using bonded nics for replication

I just redid my DSS for autofailover and I have 2 bonded 1gb nics for replication (Using 802.3as point to point, no switch). I fired up 4 large initial replication tasks and its going to take several hours.

I looked at the network statistics for the 2 bonded 1Gb nics and saw the bond was getting 100 mbs. looked at the individual ethernet ports that were in the bond and one was doing 100 mbs and the other was doing nothing.

Does bonding work with replication or is it a waste of resources?

I have all the 4 replication tasks set to 40 mbs and it appears each is using about 25-30 mbs when I look at the replication tasks.

Would increasing that bandwidth number have it start using that second nic?

Thanks!
05-03-2010, 05:12 PM
transparent

I had the same problem when bonding 2 NICs from our 4 port 3com. Our onboard NICs would bond and achieve full bandwidth (or 40MB/s as the upper limit in the settings). Any of those ports on that 3com card would not go over 100Mbit (bonded or not).

I suspect it was a driver issue. We have replaced those cards with others and solved the problem. Those same cards work at 1Gbit in Windows boxes.

Since the recommended limit is 40MB/sec for replication, bonding won't give you more speed, just redundancy. Because of that we opted to have a single NIC dedicated to replication traffic and it easily hits the 40MB/sec limit.
05-03-2010, 05:15 PM
transparent

I should also add that I was specifically told by Open-E support to not increase the 40MB/sec limit, unless using 10Gbit cards. Some sort of limitation of the underlying sync protocol (can't remember the exact explanation)
05-03-2010, 05:18 PM
jmo

Quote:

Originally Posted by webguyz

[...](Using 802.3as point to point, no switch)[...]

I looked at the network statistics for the 2 bonded 1Gb nics and saw the bond was getting 100 mbs. looked at the individual ethernet ports that were in the bond and one was doing 100 mbs and the other was doing nothing.

Does bonding work with replication or is it a waste of resources?
[·..]

webguyz,

have a look at

http://www.mjmwired.net/kernel/Docum...ng/bonding.txt

especially mode (802.3ad) and xmit_hash_policy parameter. As you are transmitting between the same nodes, 802.3ad will not give you load distribution across the links.

Maybe balance-rr would be a better choice in your situation?

With regards,
jmo