Wired Intelligent Edge

 View Only
last person joined: 3 days ago 

Bring performance and reliability to your network with the Aruba Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of the ArubaOS-Switch and ArubaOS-CX devices, and find ways to improve security across your network to bring together a mobile first solution.
Expand all | Collapse all

Traffic duplication on core router

Jump to Best Answer
This thread has been viewed 11 times
  • 1.  Traffic duplication on core router

    Posted 23 days ago

    Hi Community

    I recently came across a very strange behavior of a ArubaOS-CX core (4x8360 acting as routers, 4x8325 acting as switches).
    We migrated a network more or less at it was (logically) to the new Aruba platform. For that reason, there is some historical leftovers which, however, should not cause any major problems in the network. Still they need to get cleaned up later on.

    The network consists of two datacenter locations with 2 8360s (VSX cluster) acting as routers for a couple of VLANs plus a 8325 VSX cluster acting as 25G server access switch. This switch hold all connections to the servers (mostly VMware ESXi), the firewall and the storage system. The 8325s are connected through a full-mesh (4 links) lag to the two 8360 routers. The firewall holds a couple of DMZ networks which are routed there and of course the WAN link towards the internet. All VLANs on the core have an active-gateway configured with the same MAC address across all VLANs and all routers.
    VLAN2 acts as transit network between the network core and the firewall. However, and that's the historical part, it also has servers connected to it. Furthermore the connection between the two DCs is made through a full-mesh IP backbone between the 8360s and EVPN VXLAN. This, however, does in my opinion not play a relevant role for the behavior I observe. I'm running firmware 10.08.1050

    Note: the drawing is slightly simplified as DMZs on the firewall are not relevant.

    Now, what's my issue: When I send out packets from that server on VLAN2 (let's call it "Server01") towards a network behind the firewall (e.g. the Internet), they eventually get duplicated before they reach the firewall. It affects ICMP traffic as well as UDP and perhaps also TCP (didn't check it). The Server01 has its gateway set to the ActiveGateway living on the core. As my test traffic is destined for the Internet (let it be a ping to, a "bounce" happens on the core as the traffic enters VLANIF2 and also leaves VLANIF2 to reach the firewall. An ICMP redirect message is generated. But strangely, also the traffic gets duplicated. So capturing the traffic before it enters VLANIF2 on the routers is normal, after getting routed it is sent out twice with the MAC address of one of the routers (the same for all packets of a single flow).
    Traffic which crosses the routers from a different VLAN (e.g. VLAN16 from Server02) towards the same destination is also processed normally.
    While investigating I found a way to "disable" this behavior. Once "no ip igmp redirect" is configured, that behavior immediately stopped. The systems are still able to communicate but there is no more duplication.

    What I'm wondering is whether you would see that behavior as normal or whether you think this is a bug of ArubaOS-CX.

    Thank you for your feedback.

    Thomas Siegenthaler

  • 2.  RE: Traffic duplication on core router
    Best Answer

    Posted 23 days ago
    Hi Thomas,

    It's not a surprise why ICMP redirect messages are being generated by the 8360 since the next-hop for the route entry is located on the same subnet as the source, 8360 just tries to be helpful ;-) The question why traffic gets duplicated in this case is more interesting, but actually I'd keep things simple and just disable ICMP Redirects without digging too much into this matter because:

    1. Having ICMP redirects in such case is not desirable.
    2. The most important reason. VSX Active Gateway feature that you use requires ICMP redirects to be disabled as per VSX Guide for 10.08 that can be found here - https://www.arubanetworks.com/techdocs/AOS-CX/10.08/PDF/vsx.pdf  On page 60 in the "Requirements" section for the "Active Gateway over VSX" feature it says "Disable ICMP redirect when routing is enabled through an active gateway SVI where egress port belongs to same VLAN as ingress."

    Hope this helps!

    Ivan Bondar

  • 3.  RE: Traffic duplication on core router

    Posted 23 days ago
    Hi Ivan

    thank you so much for your reply.

    Indeed, I overlooked that requirement to disable ICMP redirects in cases where ingress and egress interfaces are the same while using a VSX configuration.

    So thanks again for making me aware of it.

    Best regards,

    Thomas Siegenthaler