Wired Intelligent Edge

Hi Community

I recently came across a very strange behavior of a ArubaOS-CX core (4x8360 acting as routers, 4x8325 acting as switches).
We migrated a network more or less at it was (logically) to the new Aruba platform. For that reason, there is some historical leftovers which, however, should not cause any major problems in the network. Still they need to get cleaned up later on.

The network consists of two datacenter locations with 2 8360s (VSX cluster) acting as routers for a couple of VLANs plus a 8325 VSX cluster acting as 25G server access switch. This switch hold all connections to the servers (mostly VMware ESXi), the firewall and the storage system. The 8325s are connected through a full-mesh (4 links) lag to the two 8360 routers. The firewall holds a couple of DMZ networks which are routed there and of course the WAN link towards the internet. All VLANs on the core have an active-gateway configured with the same MAC address across all VLANs and all routers.
VLAN2 acts as transit network between the network core and the firewall. However, and that's the historical part, it also has servers connected to it. Furthermore the connection between the two DCs is made through a full-mesh IP backbone between the 8360s and EVPN VXLAN. This, however, does in my opinion not play a relevant role for the behavior I observe. I'm running firmware 10.08.1050

Note: the drawing is slightly simplified as DMZs on the firewall are not relevant.

Now, what's my issue: When I send out packets from that server on VLAN2 (let's call it "Server01") towards a network behind the firewall (e.g. the Internet), they eventually get duplicated before they reach the firewall. It affects ICMP traffic as well as UDP and perhaps also TCP (didn't check it). The Server01 has its gateway set to the ActiveGateway living on the core. As my test traffic is destined for the Internet (let it be a ping to 1.1.1.1), a "bounce" happens on the core as the traffic enters VLANIF2 and also leaves VLANIF2 to reach the firewall. An ICMP redirect message is generated. But strangely, also the traffic gets duplicated. So capturing the traffic before it enters VLANIF2 on the routers is normal, after getting routed it is sent out twice with the MAC address of one of the routers (the same for all packets of a single flow).
Traffic which crosses the routers from a different VLAN (e.g. VLAN16 from Server02) towards the same destination is also processed normally.
While investigating I found a way to "disable" this behavior. Once "no ip igmp redirect" is configured, that behavior immediately stopped. The systems are still able to communicate but there is no more duplication.

What I'm wondering is whether you would see that behavior as normal or whether you think this is a bug of ArubaOS-CX.



Thank you for your feedback.
Regards,
Thomas

Hi Community

I recently came across a very strange behavior of a ArubaOS-CX core (4x8360 acting as routers, 4x8325 acting as switches).
We migrated a network more or less at it was (logically) to the new Aruba platform. For that reason, there is some historical leftovers which, however, should not cause any major problems in the network. Still they need to get cleaned up later on.

The network consists of two datacenter locations with 2 8360s (VSX cluster) acting as routers for a couple of VLANs plus a 8325 VSX cluster acting as 25G server access switch. This switch hold all connections to the servers (mostly VMware ESXi), the firewall and the storage system. The 8325s are connected through a full-mesh (4 links) lag to the two 8360 routers. The firewall holds a couple of DMZ networks which are routed there and of course the WAN link towards the internet. All VLANs on the core have an active-gateway configured with the same MAC address across all VLANs and all routers.
VLAN2 acts as transit network between the network core and the firewall. However, and that's the historical part, it also has servers connected to it. Furthermore the connection between the two DCs is made through a full-mesh IP backbone between the 8360s and EVPN VXLAN. This, however, does in my opinion not play a relevant role for the behavior I observe. I'm running firmware 10.08.1050

Note: the drawing is slightly simplified as DMZs on the firewall are not relevant.

Now, what's my issue: When I send out packets from that server on VLAN2 (let's call it "Server01") towards a network behind the firewall (e.g. the Internet), they eventually get duplicated before they reach the firewall. It affects ICMP traffic as well as UDP and perhaps also TCP (didn't check it). The Server01 has its gateway set to the ActiveGateway living on the core. As my test traffic is destined for the Internet (let it be a ping to 1.1.1.1), a "bounce" happens on the core as the traffic enters VLANIF2 and also leaves VLANIF2 to reach the firewall. An ICMP redirect message is generated. But strangely, also the traffic gets duplicated. So capturing the traffic before it enters VLANIF2 on the routers is normal, after getting routed it is sent out twice with the MAC address of one of the routers (the same for all packets of a single flow).
Traffic which crosses the routers from a different VLAN (e.g. VLAN16 from Server02) towards the same destination is also processed normally.
While investigating I found a way to "disable" this behavior. Once "no ip igmp redirect" is configured, that behavior immediately stopped. The systems are still able to communicate but there is no more duplication.

What I'm wondering is whether you would see that behavior as normal or whether you think this is a bug of ArubaOS-CX.



Thank you for your feedback.
Regards,
Thomas