mscooper and Robotbeat and everyone else!

Finally we have Infiniband support for CM (Connected mode).

We would like to inform you about new "small update" available for all DSS V6. Please send in a support ticket asking for this this small update for the DSS V6 only!.

This is a new option "Connected Mode" for IPoIB. Until now we were supporting only Datagram mode.

After applying small update you may change the desired mode under hardware console tools (ALT+CTRL+W) -> Infiniband tuning.

In order to get the best performance for IPoIB please to switch to "Connected Mode" and change Jumbo Frames to 65520.

Jumbo Frames can be changed under Hardware Console Tools (ALT+VTRL+W) -> Tuning Options -> Jumbo Frames.


Datagram vs Connected modes

In datagram mode, the IB UD (Unreliable Datagram) transport is used
and so the interface MTU has is equal to the IB L2 MTU minus the
IPoIB encapsulation header (4 bytes). For example, in a typical IB
fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 = 2044 bytes.

In connected mode, the IB RC (Reliable Connected) transport is used.
Connected mode takes advantage of the connected nature of the IB
transport and allows an MTU up to the maximal IP packet size of 64K,
which reduces the number of IP packets needed for handling large UDP
datagrams, TCP segments, etc and increases the performance for large
messages.

In connected mode, the interface's UD QP is still used for multicast
and communication with peers that don't support connected mode. In
this case, RX emulation of ICMP PMTU packets is used to cause the
networking stack to use the smaller UD MTU for these neighbors