802.3ad bonding Intel X520-DA2 with direct attached doesn't work

[Gr-R]

I could be wrong, but I think the 10GBe needs switch to do it.

[DIESOFTWARE-RS]

Taken from http://www.kernel.org/doc/Documentat...ng/bonding.txt

5. Switch Configuration
=======================
For this section, "switch" refers to whatever system the bonded devices are directly connected to (i.e., where the other end of the cable plugs into). This may be an actual dedicated switch device, or it may be another regular system (e.g., another computer running Linux),

The active-backup, balance-tlb and balance-alb modes do not require any specific configuration of the switch.

The 802.3ad mode requires that the switch have the appropriate ports configured as an 802.3ad aggregation. The precise method used to configure this varies from switch to switch, but, for example, a Cisco 3550 series switch requires that the appropriate ports first be grouped together in a single etherchannel instance, then that etherchannel is set to mode "lacp" to enable 802.3ad (instead of standard EtherChannel).

So "switch" could be whatever (my understanding...).

Regards,
Ralph

[jmo]

Hi Ralph,

what's in the logs of the other server? Does it look similar?

802.3ad between two Linux servers, directly connected, should work (as long as you have only a single bond per server - there's no way to set partner IDs with Linux ).

There's a lot of ups & downs of the interfaces going on during boot - I take it that's caused by setting the interface parameters. This happens after bonding the interfaces, maybe that's confusing the bonding driver?

I suggest that you start both servers from a Linux boot medium and try to set up the LACP link manually. That way you can confirm that it indeed works and can spot potential pitfalls.

Regards,
jmo

[DIESOFTWARE-RS]

Great stuff !

Bonding works, after deleting bond on both systems, reboot both systems and then
recreate the bond.
Those up/downs were caused by booting both systems simultanously.
Maybe that would prevent bond from coming up correctly...
Just performing a reboot of ONE system now to see, wether it's a general reboot issue:

And it is.

Rebooting one system causes the bond to fail.

Works again when we remove bonding and recreate the bond on the booted system via webgui.
Also when we setup jumbo frames to 9000 on both systems via console while bond is running,
we are losing connectivity.
Only way to get it back ist remove bond and recreate via gui, this time on both systems.
Unfortunately then our jumboframes are gone to 1500. So no way to use jumbos.
Edit: And no proper reboot of the system also....

Regards,
Ralph

[Gr-R]

set jumbo frames on each NIC, then create the bond, and set the frame size for the bond.

[DIESOFTWARE-RS]

set jumbo frames on each NIC, then create the bond, and set the frame size for the bond.

Yep, we have done this before...
(without success...)

Regards

[DIESOFTWARE-RS]

Should we open a ticket regarding this ?

regards
Ralph