If the two VSX Clusters (BAY001 and BAY002), which are Layer 2 interconnected together in your last design, are each one connected to an upstream Router (with routed interfaces) operating at Layer 3, I don't see necessarily a loop forming in such design. It's quite different when the upstream device (your CPE) is a Layer 2 device, in this case a loop will form and the STP will kick in (considering the STP is well configured).
Original Message:
Sent: Nov 01, 2024 06:29 PM
From: Malinda Rathnayake
Subject: VSX Square topology
Im curious what would happen if the CPE was a L3 switch with this topology? would RPVST kick in and block one of the links?
Original Message:
Sent: Nov 01, 2024 06:13 AM
From: parnassus
Subject: VSX Square topology
No, it doesn't (shouldn't), indeed.
Original Message:
Sent: 11/1/2024 12:38:00 AM
From: Malinda Rathnayake
Subject: RE: VSX Square topology
It's a FortiGate, I don't think STP will come in to play with Redundant interfaces (I'm calling them to confirm), I just wanted to confirm the behavior if it was a L3 Nexus switch as a thought experiment, so I can plan for that situation
In the VSX Best Practice guide there is a similar example at the end of the document where two VSX Clusters, placed in two DCs, square interconnected, share VLAN IDs for the benefit of downstream peers...is a sort of this design you want to achieve?
Exactly, instead of 2 DCs this will be in 2 Banks in the same DC with a shared upstream firewall using redundant link (One link stays active, one link is in slave mode)
Which seems it might be tricky if STP needs to be involved
Original Message:
Sent: Oct 31, 2024 03:03 AM
From: parnassus
Subject: VSX Square topology
Hi Malinda, with the last topology diagram things are way better (loop free on the downstream side, where the two VSF stacks are placed).
What is the CPE exactly?
A thing to be clarified is the fact that you are trying - correct me if I' m wrong - to deploy a sort of "seamless routing" with one Routing vIP valid for all downstream peers (peers placed at north and south of the VSX Clusters) and shared by (and so equal on) both VSX Clusters (thus in total you are dealing with 4 SVI IPs + 1 AG IP per VLAN, VLAN shared by both VSX Clusters)...so the vIP (the AG IP) floats between (or is operating on) VSX Clusters, from east to west and vice-versa, transparently.
In the VSX Best Practice guide there is a similar example at the end of the document where two VSX Clusters, placed in two DCs, square interconnected, share VLAN IDs for the benefit of downstream peers...is a sort of this design you want to achieve?
So, to explain, the way you want will let you to transparently move a virtual workload (a VM) between east and west ESXi Clusters, left to right and vice-versa.
Original Message:
Sent: 10/31/2024 2:00:00 AM
From: Malinda Rathnayake
Subject: RE: VSX Square topology
Thats a good point
if it can create a L2 Loop that means the RPVST should work and block the port based on the priority
Would this be possible?
Original Message:
Sent: Oct 31, 2024 12:58 AM
From: R0h0LM
Subject: VSX Square topology
Hi Malinda.
I agree with @parnassus that it looks like you might be creating a L2 loop with the CPE.
The active-gateway part should be work, it has been part of the guides for quite some years.
If we forget the CPE, then that part should be fine it there VLAN is present between the two VSX.
You need to think about how to connect the CPE without a loop.
Original Message:
Sent: Oct 30, 2024 08:32 PM
From: Malinda Rathnayake
Subject: VSX Square topology
I pasted the wrong diagram Sorry about that
I based this off - Page 108
VSX Configuration Best Practices for Aruba CX 6400, 8320, 8325, 8360, 8400 v1.3
goal was to be able to take down the VSX stack for re-cabling and relocating it to optimize space on the rack without a long lights-out Maintenace window,
- move everything to BAY01 shutdown everything on BAY02
- Do what we need to do without rushing
- Bring everything up and let DRS re-balance everything.
Also......
- if a power surge or hardware/Firmware failure bricks one VSX Pairs (for this example lets say BAY001). we can take the pair out of prod, uplink the Downstream Switches of BAY001 to the Available BAY002 VSX stack and continue operations until we can remediate the issues,
Just want to do a sanity check and confirm these statements are true, that's why posted here US TAC wasn't really helpful with answers since they dont have experience with this topology
- If VM001 on BAY001 to wants to talk to VM111 in BAY002, Traffic will hit the BAY002 VSX Stack and then traverse the MLAG between the stacks to reach the VM001
- If VM001 on BAY001 needs to access internet via the CPE, one of the BAY001 VSX stack members will route it to the CPE based on the L3 configuration
Original Message:
Sent: Oct 30, 2024 06:06 PM
From: parnassus
Subject: VSX Square topology
This second diagram, to me, looks a little bit different by the first one you originally posted: the two VSX Clusters interconnected "front-to-front" are one thing, the two VSX Clusters interconnected together "front-to-front" with both being also concurrently connected with a peer Switch (either it would be a standalone chassis switch or any other type of Cluster/Stack solution like your Aruba 6300 VSF Stack <- in any case the peer is seen by each VSX Cluster as a single logical entity) is another thing.
I see a non-loop free design here. A triangle (the VSX Clusters are two vertices, the peer VSF Stack is the remaining one), with all the consequences a loop produces.
This as first comment.
Then, independently by this problematic form of connectivity to downstream peers, you should pay attention to deploy each VSX Cluster thinking as it is very unique on your network...so Virtual MAC Addresses for Active Gateway and System MAC Addresses should not be the same (shared) by both VSX Clusters, don't you want them to be two "IP Routers" routing each others through a Transit VLAN?
So, given the above considerations, the second diagram - to me - looks a little bit problematic exactly because it seems you want to deploy two VSX Clusters - fuse them together (in terms of Layer 3 routing and redundant Layer 2 connectivity features provided to downstream peers) - and so use them in a way it "makes them acting" as a sort of super VRRP (deployed over two VSX Clusters)...and this to have downstream peers routed by this sort of VSX Clusters in HA mode (is it a VSX Super-Cluster?)...isn't it?
Is this you're trying to achieve?
Have you read the VSX Configuration Best Practice Guide Technical Whitepaper (look for it on ASP)? there are examples, restrictions and recommendations about how (and scenarios) to deploy VSX Cluster(s)? It should clarify you some best practices and approaches.
Original Message:
Sent: Oct 30, 2024 12:00 PM
From: Malinda Rathnayake
Subject: VSX Square topology
Thanks for the reply,
Yes, the idea is to be able to able to Service each routing rack one at a time, without any downtime for the gateway
We should be able to take routing Rack01 down without any impact to the gateway reachability
What are your thoughts?
Original Message:
Sent: Oct 30, 2024 11:09 AM
From: parnassus
Subject: VSX Square topology
Hello, at very first sight it looks like your desire is just to interconnect two VSX Clusters (necessarily) by means of VSX LAGs (on each side) where each VSX Cluster has the Active Gateway role enabled to serve both its downstream peers (VLAN SVIs) and also the upstream (over Layer 2 VSX LAGs or over Layer 3 VSX LAGs) Transit VLAN (point to point to the other VSX Cluster) which will route across the VSX to VSX (front-to-front) interconnection (the square).
I haven't read all the links you posted yet...but...that's my very first impression...two IP routers (the VSX Clusters) routing through the square.
Original Message:
Sent: 10/29/2024 4:24:00 PM
From: Malinda Rathnayake
Subject: VSX Square topology
https://www.reddit.com/r/ArubaNetworks/comments/1fqubku/vrrp_between_2_vsx_stacks/
End Goal
192.168.1.1 Span 2 VSX clusters and use the active gateway functionality
What are the Gotchas and things i need to lookout for based on the comments i saw on the reedit post this looks like a very commonly deployed topology