Failover with bonding - Ping Nodes goes up and down

Hello,

my environment is 2x Open-E DSS, 2x ESX Hosts and 2 Switches. All Servers are connected to both switches and they are linked together using Spanning Tree (STP). The iSCSI Backend uses 172.16.* IP's and the Management LAN is 10.1.* - separated by VLANs.

Like this: http://www.dropmocks.com/miB2n (red are my IPs)

IP Configuration:
DSS Node 1
eth0 = WebGUI (10.1.1.1)
eth1 = Storage Replication (172.16.1.101)
eth2 = bond0 (172.16.2.101)
eth3 = bond0 (172.16.3.101)
bond0 = balancerr (172.16.2.101)

DSS Node 2
eth0 = WebGUI (10.1.1.2)
eth1 = Storage Replication (172.16.1.102)
eth2 = bond0 (172.16.2.102)
eth3 = bond0 (172.16.3.102)
bond0 = balancerr (172.16.2.102)

The iSCSI Failover uses 172.16.100.100 as Virtual IP and both Switches as Ping Nodes (172.16.2.1;172.16.2.2)


If I check the DSS WebGUI I can see that the ping nodes goes up and down. If I enable Email Warnings I get a lot of mails with the message "Failover:Failover: Ping node 172.16.2.2 is down."

I can ping 172.16.2.1 and .2 (the Switch Interfaces) without packet loss from both DSS.

I habe 4 implementations like this, everytime the same problem. I use Version 6.0up65.8101.5217 (64bit) with a small patch from the support to disable the Email-Warnings.




Is the bond the problem?
Any ideas?
From my side I would say that the network part looks good.



Best regards,
Manuel

It could be a few things:
1. make sure your nics are the ones you think they are. what you assume is eth0 could really be eth3. and they may not be consistent across both servers

2. Verify your vlans are setup right

3. This stands out to me: is all traffic on 172.16 on the same vlan? If possible you should get a separate switch for the iscsi traffic. I would use a crossover cable for the replication instead of a switch.
if you can't get another switch make a separate vlan for each subnet

4.************please tell me the picture is wrong, you have 172.16.1.101 on two different hosts.

Hi,

about your concerns:

1. NICs are ok and the right ones. Checked after installation on both nodes.

2. VLAN setup is ok

3. all traffic of the iSCSI (172.*) is on the same VLAN and I use a Crossover Cable for storage replication, not the switch

4. yes the drawing is wrong (oooops!) - the second node use 172.16.1.102 for the storage replication.


Thanks for your help.
Manuel

The only thing that seems strange to me is to bond two nics and them have the connect to two different Switches.

I guess this is possible, but I'm not sure the advantage, especialy if you are using MPIO.
I believe in MPIO you set two different Virtual IP's. If you can, check the video.

http://www.vimeo.com/moogaloop.swf?clip_id=18113062

if you use DSS servers only for ESX, try to use VMkernel ports with address 172.16.2.xx (add new one to vSwitch, where are 172.16.100.xx ports).
I think, that "controling" final destination address is better, than controling something in the middle.

what type of devices are your ping nodes and why are they on your iscsi subnet?
I'm guessing it could be that iscsi traffic is drowning out your pings to your ping nodes. Try finding some switches or routers on your management vlan to use as ping nodes.