Wired Intelligent Edge

 View Only
last person joined: yesterday 

Bring performance and reliability to your network with the Aruba Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of the ArubaOS-Switch and ArubaOS-CX devices, and find ways to improve security across your network to bring together a mobile first solution.
Expand all | Collapse all

VSX to VSX LAG

Jump to Best Answer
This thread has been viewed 48 times
  • 1.  VSX to VSX LAG

    Posted Dec 16, 2021 12:10 AM
    I have a client with 2 40Gb dark fiber connections between their corporate office and their data center.  This connection is purely layer 2.  Layer 3 just isn't an option here.

    They currently have Arista switches and we were going to just do an MLAG between the locations but I learned that MLAGs (Arista, Nexus and possibly others) work as a triangle meaning a normal switch or switch stack on one side with an MLAG on the other.  You can't do an a connection that is an MLAG to MLAG.  I'm including an image of what we're trying to accomplish.

    The reason we don't want to just do a 2 member switch stack on each side is mainly for firmware update reasons.  The client would prefer not to have the network go completely down during an upgrade.  That leaves either MLAG or a stack that supports ISSU or in 1 vendor's case, NSSU.

    My question:  Can this be done with VSX?  Would it operate like an LACP trunk?



    ------------------------------
    Matthew Ausmus
    ------------------------------


  • 2.  RE: VSX to VSX LAG

    MVP GURU
    Posted Dec 16, 2021 02:05 AM
    Hi Matthew, sure that you can.

    VSX Stack (or whatever) to VSX Stack and vice-versa.

    Say VSX LAG (MC-LAG) on one end and the same in the other, the point is that you have just two physical links between sites so the interconnection isn't properly resilient (the ideal would be to deal with four links so you can cross them between sites).

    ------------------------------
    Davide Poletto
    ------------------------------



  • 3.  RE: VSX to VSX LAG

    EMPLOYEE
    Posted Dec 17, 2021 05:46 AM
    Hello,

    You'll find an example (more complex that your use-case as dealing with L3 also) here:
    https://support.hpe.com/hpesc/public/docDisplay?docId=a00094242en_us
    Page 106- Appendix F


    ------------------------------
    Vincent Giles
    ------------------------------



  • 4.  RE: VSX to VSX LAG

    Posted Dec 17, 2021 06:54 PM
    Vincent, thanks for the response.  That setup is the optimal it just isn't an option in this environment.

    Davide, my question is really about the resiliency of this configuration. I included an updated diagram that I hope helps.
    If an IDF switch stack has a LAG to Switch1 & Switch2 and the link between Switch1 & Switch3 goes down, will the traffic entering switch1 from the Switch Stack LAG be sent to Switch2 to get across to the remote location? Would the MC-LAG operate as an LACP LAG even though it is only 2 links instead of the recommended 4 links?

    ------------------------------
    Matthew Ausmus
    ------------------------------



  • 5.  RE: VSX to VSX LAG

    MVP GURU
    Posted Dec 18, 2021 05:17 AM
    Hi Matthew,

    "If an IDF switch stack has a LAG to Switch1 & Switch2 and the link between Switch1 & Switch3 goes down, will the traffic entering switch1 from the Switch Stack LAG be sent to Switch2 to get across to the remote location?"

    Yes, a Stack LACP connected Switch Stack (say a VSF or a IRF or another VSX or a Backplane) or a standalone LACP connected Switch (or any other LACP connected peer) on the left of your network diagram...if properly connected to (via a Link Aggregation, LACP is preferred) the VSX (Switch 1+2) will be able to continue to reach its destination party on the far right of the same diagram even if - with that VSX "A" to VSX "B" two links interconnection - the link between VSX "A" Switch 1 and VSX "B" Switch 3 goes down...and that will happen because VSX "A" will flow that messages via VSX ISL logical interface to VSX "A" Switch 2 to VSX "B" Switch 4.

    What I said before was...what will happen if CONCURRENTLY you will lose the physical link between VSX "A" Switch 1 and VSX "B" Switch 3 and - let me say you will be very unlucky - the VSX "A" Switch 2 OR/AND the VSX "B" Switch 4 goes/go down? that I mean about "resiliency" of VSX "A" to VSX "B" interconnection...clearly if you have only two physical links you must accept what you can achieve with those two links...and just to be prepared to the "black swan" event (if it happens).

    "Would the MC-LAG operate as an LACP LAG even though it is only 2 links instead of the recommended 4 links?"

    I don't understand the question...but Yes, a two links LAG is like a four links LAG even if it has less member links (less resiliency/redundancy). In the end you're just assessing if a VSX-to-VSX (back to back) could work...it works! but the point is really the resiliency level of this back-to-back interconnection...and this level is directly proportional to the number of physical links you use to build this interconnection (and the way you do it).


    ------------------------------
    Davide Poletto
    ------------------------------



  • 6.  RE: VSX to VSX LAG

    Posted Dec 19, 2021 05:30 PM
    Hi Davide

    "I don't understand the question...but Yes, a two links LAG is like a four links LAG even if it has less member links (less resiliency/redundancy). In the end you're just assessing if a VSX-to-VSX (back to back) could work...it works! but the point is really the resiliency level of this back-to-back interconnection...and this level is directly proportional to the number of physical links you use to build this interconnection (and the way you do it)."

    Is that design supported for an MC-LAG ? My understanding is if the switch is dual homed the MC-LAG/VSX LAG applies however here we have single links between each VSX-PAIR. Can the single links still be configured as a MC-LAG  ?


    ------------------------------
    John Bockarie
    ------------------------------



  • 7.  RE: VSX to VSX LAG
    Best Answer

    MVP GURU
    Posted Dec 20, 2021 03:59 AM
    Yes, that's enough to setup a VSX LAG (Multi-Chassis LAG) between the two VSX clusters.

    Let's say you want to setup a VSX LAG lag21 on the VSX Stack "A" and a VSX LAG lag34 on the VSX Stack "B": in your specific scenario you're forced to assign only one interface (per VSX Member of that VSX Stack) which means that a VSX LAG (on each VSX Stack) will be made of just two interfaces (exactly your two 40Gbps links) and those two interfaces will be "stretched" along both VSX Members appearing to the other peer device (the other VSX Stack in your case) as a classical LACP LAG (the same applies when you setup a VSX LAG to the IDF Stack represented on the left of your Network Diagram).

    On VSX Stack "A" Primary you need:

    interface lag 21 multi-chassis
    no shutdown
    description "This is VSX LAG 21 on VSX A Primary"
    no routing
    vlan trunk native <vlan-id>
    vlan trunk allowed <vlan-id>
    lacp mode active
    loop-protect
    loop-protect vlan <vlan-id>

    interface <40Gbps-interface-id-on-VSX-A-Primary>
    no shutdown
    mtu 9198
    low-control rx
    description <description-of-40Gbps-interface-which-is-part-of-a-VSX-LAG-lag21-on-VSX-A-Primary>
    lag 21

    On VSX Stack "A" Secondary you need:

    interface lag 21 multi-chassis
    no shutdown
    description "This is VSX LAG 21 on VSX A Secondary"
    no routing
    vlan trunk native <vlan-id>
    vlan trunk allowed <vlan-id>
    lacp mode active
    loop-protect
    loop-protect vlan <vlan-id>

    interface <40Gbps-interface-id-on-VSX-A-Secondary>
    no shutdown
    mtu 9198
    low-control rx
    description <description-of-40Gbps-interface-which-is-part-of-a-VSX-LAG-lag21-on-VSX-A-Secondary>
    lag 21

    now it should be clear that on VSX Stack "B" you need to define the corresponding VSX LAG (lag34) on both members and to bind the correct type of 40Gbps interfaces to that VSX LAG.

    On VSX Stack "B" Primary you need:

    interface lag 34 multi-chassis
    no shutdown
    description "This is VSX LAG 34 on VSX B Primary"
    no routing
    vlan trunk native <vlan-id>
    vlan trunk allowed <vlan-id>
    lacp mode active
    loop-protect
    loop-protect vlan <vlan-id>

    interface <40Gbps-interface-id-on-VSX-B-Primary>
    no shutdown
    mtu 9198
    low-control rx
    description <description-of-40Gbps-interface-which-is-part-of-a-VSX-LAG-lag34-on-VSX-B-Primary>
    lag 34

    On VSX Stack "B" Secondary you need:

    interface lag 34 multi-chassis
    no shutdown
    description "This is VSX LAG 34 on VSX B Secondary"
    no routing
    vlan trunk native <vlan-id>
    vlan trunk allowed <vlan-id>
    lacp mode active
    loop-protect
    loop-protect vlan <vlan-id>

    interface <40Gbps-interface-id-on-VSX-B-Secondary>
    no shutdown
    mtu 9198
    low-control rx
    description <description-of-40Gbps-interface-which-is-part-of-a-VSX-LAG-lag34-on-VSX-B-Secondary>
    lag 34

    Change lag21 and lag34, involved interface(s)-id and VLAN id(s) with desired values...also consider the above as VSX LAGs between peer switches...so you would need also to allow other VLAN id(s) over the VSX LAG, that's to say it is just a basic example and other adjustments are necessary.

    There are other considerations...vsx-sync, spanning tree...etc...but the above is just to start and understand the mechanism.

    ------------------------------
    Davide Poletto
    ------------------------------



  • 8.  RE: VSX to VSX LAG

    Posted Dec 20, 2021 05:21 AM
    Thanks Davide makes sense

    Regarding the loop protect, assuming you were allowing all vlans on the lags between the vsx-pairs A and B i.e "vlan trunk allowed all". Would loop protect be effective on its own ? or would you need to create all vlans on switch (1-4094) and add loop protect-vlan 1-4094

    Also I presume the same lag ID can be used on both vsx-pairs e.g 21 for VSX STACK A and 21 for VSX STACK B or is it best practice to use different IDs like you have ?

    ------------------------------
    John Bockarie
    ------------------------------



  • 9.  RE: VSX to VSX LAG

    MVP GURU
    Posted Dec 20, 2021 07:41 AM
    Yes, I admit that the loop protect could be used effectively when you're dealing with a "dual homed" peer device instead (where "dual homed" peer means a peer connected thorough that particular VSX LAG to both VSX members) and - for sure - there are other considerations to be done and other settings that would fit your scenario better (basically you have two L2 VSX back-to-back interconnected via VSX LAG made of two 40Gbps interfaces...so it's like a chain of two access - or two distribution - switches).

    The VSX LAG id has no particular requirement: it's a best practice trying to keep the VSX LAG member interfaces paired (say interface 1/1/10 on VSX LAG lag 1 on the VSX Primary member and corresponding interface 1/1/10 on VSX LAG lag 1 on the VSX Secondary member) but nobody will impose you to use lag1 instead of, example, lag10 or whatever you like (within the current VSX LAG lag<id> numbering restrictions various AOS-CX based platforms have -> generally latest lag<id> is used for VSX LAG and best practice reports lag1 for VSX LAG made of interface 1/1/1 on Primary and 1/1/1 on Secondary).

    Mine with lag21 and lag34 was an example made exactly to show there are no restrictions: you can configure a VSX LAG 1 on first VSX Stack connected back-to-back to a VSX LAG 100 on second VSX Stack...since those Id(s) have a meaning only for the VSX Stack they are configured into (clearly it's nice to have an equal VSX LAG <id> on both ends so you can easily troubleshoot in case of issues...because you know that a VSX LAG has the same name on both ends...but, again, it's not mandatory).

    I suggest you to use a VSX LAG id on the high range (say 100) so all other VSX LAGs under that value can be used for access peers (if you have them).

    Also there is the VSX Best Practice Guide that will explain all in great detail (also VLAN 1 considerations bound to STP and VLAN allowed over Trunk Interfaces), the most important part is to set the VSX correctly, then VSX LAGs (with their various options) can be fine tuned with various options depending on the particular scenario you're dealing with.

    References (basic literature):
    AOS-CX 10.09 Virtual Switching Extension (VSX) Guide for Aruba CX 6400, 8320, 8325, 8360, 8400, 10000 Switch Series (9 December 2021)
    VSX Configuration Best Practices for Aruba CX 6400, 8320, 8325, 8360, 8400 (Edition 1.3 December 2020) 


    ------------------------------
    Davide Poletto
    ------------------------------



  • 10.  RE: VSX to VSX LAG

    Posted Dec 20, 2021 09:27 AM
    great, thanks Davide for the explanation

    ------------------------------
    John Bockarie
    ------------------------------



  • 11.  RE: VSX to VSX LAG

    Posted Jan 14, 2022 02:37 PM
    Sorry for being such a flake and not getting back.

    Davide, that was a great description and reinforced what I already thought about how this system would work.

    The client that has this setup is currently using another vendor that utilizes VX-LAN.  The infrastructure is all end of support ('EOS').  When we were setting this up the head engineer on this project didn't setup this VXLAN stating the existing vendor didn't support it (I disagreed but it wasn't my decision).  Well, it didn't go well for him and currently one of those links is in a down state to resolve the STP storm.

    Since this hear is all EOS we're talking to them about upgrading all it and Aruba-CX line is one that we're considering suggesting.  When we do this, we'll be re-architecting the environment to fully utilize VXLAN to eliminate STP as a failover mechanism and this particular part is the most critical piece and what will define what switches we recommend.

    Thank you again for such an in-depth answer.

    ------------------------------
    Matthew Ausmus
    ------------------------------