Hi Clint, personally - If I were you - I'd not drop the Loop Protect approach (at least not generally): please read the
document.
I've a question in my head: if your VSX is so "basic and plain" (no Layer 3 routing is performed to downstream peers through the Active Gateway featur...basically no nothing) why the complexity with so many separated VSX LAGs to upstream Firewalls' Cluster? where are the real benefits? physical separation of broadcast domains? IP Routing is happening on the Firewalls' Cluster...are you really OK with that (bottlenecks)? are you accepting this approach only because Firewalls are configured in a particular way and you don't want to change that?
Again, if I were you (and along with your deployment restrictions/requirements) I would totally rethink how the VSX Cluster <-> Firewall Cluster are currently physically interconnected.
Two VSX LAGs with just two physical ports to each Firewall node should be enough (each Firewall node <- 2x10Gbps -> each VSX node):
Reconfigure the Firewall Cluster to adapt/bind its Access Policies to the (only two) new Firewall's node 1 LAG 1 and node 2 LAG 2 logical interfaces shouldn't be too difficult...consider how much you will simplify your design now and maintenance in the future.
In other words I don't understand the (unnecessary?) complexity built into/within the upstream layer of connectivity if then the VSX cluster acts as a very very basic Layer 2 device (no IP Routing, no ACLs, etc.) from the PoV of all the downstream peers it will be connected with.
Original Message:
Sent: Oct 07, 2021 05:12 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Robert - That is exactly the advice I was looking/hoping for!!! I will drop the loop protect and take a look at those command outputs.
Thank you!!!
------------------------------
Clint
Original Message:
Sent: Oct 07, 2021 05:05 PM
From: Robert Großmann
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Hi Clint,
For my opinion I would not use loop-protect on uplink/downlink Ports (to other switches), even Davide has it in its config example. But it is not bad at all.
There I would rely on the spanning tree mechanism and that you does not connect rogue devices to this ports.
For example LATER you can use the spanning tree root guard function on all downlink ports of your vsx-pair to prevent a unwanted switch to be the root in the network.
You can think about using the spanning tree loop-guard function on switch Uplink Ports, and bpdu-guard and tcn-guard on access ports (even to firewall). In regular cases we dont want to configure to much features on a core, but I would say your network is very flat and is like a collapsed core.
Take a look at the best practices design guide and search for "best practice on".
Core is not listed, but aggregation and access are. And you're free to combine the subset of possibilities for your needs.
I'm not sure if the parameter for spanning-tree on your old and new switches are the same.
Aruba-CX uses MSTP per Default, Aruba-OS on 3810M I dont know. On MSTP any config parameter must be the same (name, revision, instances). And your aruba-cx config-name is as an MAC?? If any parameter differs you do have an STP Boundary using CST between the "MSTP Topologies".
I would suggest to type on Aruba-CX:
show spanning-tree
show spanning-tree mst
and on Aruba-OS:
show spanning-tree
show spanning-tree instance ist
and to examine if old and new switches are in the same "MSTP topology". But perhaps you want them to be in different "topologies", then yiu are right with choosing another config name or revision or an other spanning-tree mode.
Good luck with the network migration!
Kind Regards
Robert
------------------------------
Robert Großmann
Original Message:
Sent: Oct 07, 2021 03:34 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
First, I want to thank everyone for all their time and help, I wouldn't have been able to get all this done without you guys!!!
Second, I have pulled pretty much all the complexity out of the configs, I believe removing the need for any routing or anything special, just basic MC-LAGs and VLANs, allowing the firewall to do any routing necessary between any of the network segments.
With all that said, I am posting what I believe to be the final configs (minus any suggestions anyone has) for the switches. Please take a look if/when you have time. I'm especially interested in making sure I have spanning tree / loop back protection configured properly.
Primary 8320
!!Version ArubaOS-CX TL.10.08.1010!export-password: defaulthostname Swt-8320-01profile L3-corevrf KeepAliventp server pool.ntp.org minpoll 4 maxpoll 4 iburstntp enablentp vrf mgmtcli-session timeout 0!!!!!aruba-central disablessh server vrf defaultssh server vrf mgmtvlan 1vlan 100 name Mgmt VLAN vsx-sync description Mgmt VLAN - VLAN 100vlan 101 name Private VLAN vsx-sync description Private VLAN - VLAN 101vlan 202 name DMZ VLAN vsx-sync description DMZ VLAN - VLAN 202vlan 212 name DMZ-LB VLAN vsx-sync description DMZ-LB VLAN - VLAN 212vlan 217 name Corp217 VLAN vsx-sync description Corp 217 VLAN - VLAN 217vlan 250 name Public VLAN vsx-sync description Public VLAN - VLAN 250spanning-treespanning-tree config-name 38:10:f0:1b:70:a0interface mgmt no shutdown ip static 10.10.0.3/24 default-gateway 10.10.0.1 nameserver 10.1.1.20 10.1.1.21interface lag 99 multi-chassis no shutdown description Mgmt/MLAG 99 - Swt-8320 <-> FW-02 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 100 multi-chassis no shutdown description Mgmt/MLAG 100 - Swt-8320 <-> FW-01 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 101 multi-chassis no shutdown description Private/MLAG 101 - Swt-8320 <-> FW-01 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 102 multi-chassis no shutdown description Private/MLAG 102 - Swt-8320 <-> FW-02 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 144 multi-chassis no shutdown description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fast loop-protect loop-protect vlan 1,100-101,202,212,217,250interface lag 202 multi-chassis no shutdown description DMZ/MLAG 202 - Swt-8320 <-> FW-01 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 203 multi-chassis no shutdown description DMZ/MLAG 203 - Swt-8320 <-> FW-02 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 212 multi-chassis no shutdown description DMZ-LB/MLAG 212 - Swt-8320 <-> FW-01 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 213 multi-chassis no shutdown description DMZ-LB/MLAG 213 - Swt-8320 <-> FW-02 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 217 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320 <-> FW-01 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 218 multi-chassis no shutdown description Corp217/MLAG 218 - Swt-8320 <-> FW-02 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 250 multi-chassis no shutdown description Public/MLAG 250 - Swt-8320 <-> FW-01 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 251 multi-chassis no shutdown description Public/MLAG 251 - Swt-8320 <-> FW-02 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 256 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface 1/1/2 no shutdown description *** VLAN 250 Test Port *** no routing vlan access 250interface 1/1/5 no shutdown description *** VLAN 202 Test Port *** no routing vlan access 202interface 1/1/6 no shutdown description *** VLAN 202 Test Port *** no routing vlan access 202interface 1/1/13 no shutdown description *** VLAN 212 Test Port *** no routing vlan access 212interface 1/1/14 no shutdown description *** VLAN 212 Test Port *** no routing vlan access 212interface 1/1/21 no shutdown description *** VLAN 217 Test Port *** no routing vlan access 217interface 1/1/22 no shutdown description *** VLAN 217 Test Port *** no routing vlan access 217interface 1/1/25 no shutdown description *** VLAN 101 Test Port *** no routing vlan access 101interface 1/1/26 no shutdown description *** VLAN 101 Test Port *** no routing vlan access 101interface 1/1/31 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/32 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/35 no shutdown description Public/MLAG 250 - Swt-8320 <-> FW-01 lag 250interface 1/1/36 no shutdown description Public/MLAG 251 - Swt-8320 <-> FW-02 lag 251interface 1/1/37 no shutdown description DMZ/MLAG 202 - Swt-8320 <-> FW-01 lag 202interface 1/1/38 no shutdown description DMZ/MLAG 203 - Swt-8320 <-> FW-02 lag 203interface 1/1/39 no shutdown description DMZ-LB/MLAG 212 - Swt-8320 <-> FW-01 lag 212interface 1/1/40 no shutdown description DMZ-LB/MLAG 213 - Swt-8320 <-> FW-02 lag 213interface 1/1/41 no shutdown description Corp217/MLAG 217 - Swt-8320 <-> FW-01 lag 217interface 1/1/42 no shutdown description Corp217/MLAG 218 - Swt-8320 <-> FW-02 lag 218interface 1/1/43 no shutdown description Mgmt/MLAG 100 - Swt-8320 <-> FW-01 lag 100interface 1/1/44 no shutdown description Mgmt/MLAG 99 - Swt-8320 <-> FW-02 lag 99interface 1/1/45 no shutdown mtu 9198 flow-control rx description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M lag 144interface 1/1/46 no shutdown mtu 9198 flow-control rx description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M lag 144interface 1/1/48 no shutdown vrf attach KeepAlive description VSX-ISL-KeepAlive-Link ip address 192.168.168.1/24interface 1/1/51 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface 1/1/52 no shutdown description Private/MLAG 101 - Swt-8320 <-> FW-01 lag 101interface 1/1/53 no shutdown description Private/MLAG 102 - Swt-8320 <-> FW-02 lag 102interface 1/1/54 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface vlan 100 vsx-sync active-gateways description Mgmt VLAN - VLAN 100interface vlan 101 vsx-sync active-gateways description Private VLAN - VLAN 101 ip address 10.10.1.3/24interface vlan 202 vsx-sync active-gateways description DMZ VLAN - VLAN 202interface vlan 212 vsx-sync active-gateways description DMZ-LB VLAN - VLAN 212interface vlan 217 vsx-sync active-gateways description Corp 217 VLAN - VLAN 217interface vlan 250 vsx-sync active-gateways description Public VLAN - VLAN 250snmp-server community IntraVexvsx system-mac 02:01:00:00:01:00 inter-switch-link lag 256 role primary keepalive peer 192.168.168.2 source 192.168.168.1 vrf KeepAlive linkup-delay-timer 30 vsx-sync copp-policy dhcp-relay dns icmp-tcp loop-protect-global mclag-interfaces qos-global snmp ssh static-routes stp-global time vsx-globalip dns server-address 10.1.1.20ip dns server-address 10.1.1.21!!!!!ip source-interface all 10.10.1.3https-server session-timeout 480https-server vrf defaulthttps-server vrf mgmt
Secondary 8320
!!Version ArubaOS-CX TL.10.08.1010!export-password: defaulthostname Swt-8320-02profile L3-corevrf KeepAliventp server pool.ntp.org minpoll 4 maxpoll 4 iburstntp enablentp vrf mgmtcli-session timeout 0!!!!!aruba-central disablessh server vrf defaultssh server vrf mgmtvlan 1vlan 100 name Mgmt VLAN vsx-sync description Mgmt VLAN - VLAN 100vlan 101 name Private VLAN vsx-sync description Private VLAN - VLAN 101vlan 202 name DMZ VLAN vsx-sync description DMZ VLAN - VLAN 202vlan 212 name DMZ-LB VLAN vsx-sync description DMZ-LB VLAN - VLAN 212vlan 217 name Corp217 VLAN vsx-sync description Corp 217 VLAN - VLAN 217vlan 250 name Public VLAN vsx-sync description Public VLAN - VLAN 250spanning-treespanning-tree config-name 38:10:f0:1b:70:a0interface mgmt no shutdown ip static 10.10.0.4/24 default-gateway 10.10.0.1 nameserver 10.1.1.20 10.1.1.21interface lag 99 multi-chassis no shutdown description Mgmt/MLAG 99 - Swt-8320 <-> FW-02 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 100 multi-chassis no shutdown description Mgmt/MLAG 100 - Swt-8320 <-> FW-01 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 101 multi-chassis no shutdown description Private/MLAG 101 - Swt-8320 <-> FW-01 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 102 multi-chassis no shutdown description Private/MLAG 102 - Swt-8320 <-> FW-02 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 144 multi-chassis no shutdown description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fast loop-protect loop-protect vlan 1,100-101,202,212,217,250interface lag 202 multi-chassis no shutdown description DMZ/MLAG 202 - Swt-8320 <-> FW-01 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 203 multi-chassis no shutdown description DMZ/MLAG 203 - Swt-8320 <-> FW-02 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 212 multi-chassis no shutdown description DMZ-LB/MLAG 212 - Swt-8320 <-> FW-01 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 213 multi-chassis no shutdown description DMZ-LB/MLAG 213 - Swt-8320 <-> FW-02 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 217 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320 <-> FW-01 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 218 multi-chassis no shutdown description Corp217/MLAG 218 - Swt-8320 <-> FW-02 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 250 multi-chassis no shutdown description Public/MLAG 250 - Swt-8320 <-> FW-01 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 251 multi-chassis no shutdown description Public/MLAG 251 - Swt-8320 <-> FW-02 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 256 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface 1/1/2 no shutdown description *** VLAN 250 Test Port *** no routing vlan access 250interface 1/1/5 no shutdown description *** VLAN 202 Test Port *** no routing vlan access 202interface 1/1/6 no shutdown description *** VLAN 202 Test Port *** no routing vlan access 202interface 1/1/13 no shutdown description *** VLAN 212 Test Port *** no routing vlan access 212interface 1/1/14 no shutdown description *** VLAN 212 Test Port *** no routing vlan access 212interface 1/1/21 no shutdown description *** VLAN 217 Test Port *** no routing vlan access 217interface 1/1/22 no shutdown description *** VLAN 217 Test Port *** no routing vlan access 217interface 1/1/25 no shutdown description *** VLAN 101 Test Port *** no routing vlan access 101interface 1/1/26 no shutdown description *** VLAN 101 Test Port *** no routing vlan access 101interface 1/1/31 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/32 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/35 no shutdown description Public/MLAG 250 - Swt-8320 <-> FW-01 lag 250interface 1/1/36 no shutdown description Public/MLAG 251 - Swt-8320 <-> FW-02 lag 251interface 1/1/37 no shutdown description DMZ/MLAG 202 - Swt-8320 <-> FW-01 lag 202interface 1/1/38 no shutdown description DMZ/MLAG 203 - Swt-8320 <-> FW-02 lag 203interface 1/1/39 no shutdown description DMZ-LB/MLAG 212 - Swt-8320 <-> FW-01 lag 212interface 1/1/40 no shutdown description DMZ-LB/MLAG 213 - Swt-8320 <-> FW-02 lag 213interface 1/1/41 no shutdown description Corp217/MLAG 217 - Swt-8320 <-> FW-01 lag 217interface 1/1/42 no shutdown description Corp217/MLAG 218 - Swt-8320 <-> FW-02 lag 218interface 1/1/43 no shutdown description Mgmt/MLAG 100 - Swt-8320 <-> FW-01 lag 100interface 1/1/44 no shutdown description Mgmt/MLAG 99 - Swt-8320 <-> FW-02 lag 99interface 1/1/45 no shutdown mtu 9198 flow-control rx description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M lag 144interface 1/1/46 no shutdown mtu 9198 flow-control rx description Switch Trunk/MLAG 144 - Swt-8320 <-> 3810M lag 144interface 1/1/48 no shutdown vrf attach KeepAlive description VSX-ISL-KeepAlive-Link ip address 192.168.168.2/24interface 1/1/51 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface 1/1/52 no shutdown description Private/MLAG 101 - Swt-8320 <-> FW-01 lag 101interface 1/1/53 no shutdown description Private/MLAG 102 - Swt-8320 <-> FW-02 lag 102interface 1/1/54 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface vlan 100 vsx-sync active-gateways description Mgmt VLAN - VLAN 100interface vlan 101 vsx-sync active-gateways description Private VLAN - VLAN 101 ip address 10.10.1.4/24interface vlan 202 vsx-sync active-gateways description DMZ VLAN - VLAN 202interface vlan 212 vsx-sync active-gateways description DMZ-LB VLAN - VLAN 212interface vlan 217 vsx-sync active-gateways description Corp 217 VLAN - VLAN 217interface vlan 250 vsx-sync active-gateways description Public VLAN - VLAN 250snmp-server community IntraVexvsx system-mac 02:01:00:00:01:00 inter-switch-link lag 256 role secondary keepalive peer 192.168.168.1 source 192.168.168.2 vrf KeepAlive linkup-delay-timer 30 vsx-sync copp-policy dhcp-relay dns icmp-tcp loop-protect-global mclag-interfaces qos-global snmp ssh static-routes stp-global time vsx-globalip dns server-address 10.1.1.20ip dns server-address 10.1.1.21!!!!!ip source-interface all 10.10.1.4https-server session-timeout 480https-server vrf defaulthttps-server vrf mgmt
3810M Stack
stacking member 1 type "JL074A" mac-address 8c85c1-ebe480 member 1 priority 200 member 1 flexible-module A type JL083A member 2 type "JL074A" mac-address 8c85c1-ea1e00 member 2 priority 100 member 2 flexible-module A type JL083A exithostname "Swt-3810M-Stack"trunk 1/A3-1/A4,2/A3-2/A4 trk144 lacptimesync ntpntp server 10.1.1.20 iburstntp server 10.1.1.21 iburstntp enableip dns server-address priority 1 10.1.1.20ip dns server-address priority 2 10.1.1.21ip route 0.0.0.0 0.0.0.0 10.10.1.1interface 1/A3 name "8320 <-> 3810M Trunk - LAG 144" exitinterface 1/A4 name "8320 <-> 3810M Trunk - LAG 144" exitinterface 2/A3 name "8320 <-> 3810M Trunk - LAG 144" exitinterface 2/A4 name "8320 <-> 3810M Trunk - LAG 144" exitsnmp-server community "IntraVex" unrestrictedoobm ip address 10.10.0.7 255.255.255.0 member 1 ip address 10.10.0.8 255.255.255.0 exit member 2 ip address 10.10.0.9 255.255.255.0 exit exitvlan 1 name "DEFAULT_VLAN" no untagged 1/1-1/2,1/5-1/8,1/13-1/16,1/21-1/22,1/25-1/26,1/29-1/44,2/1-2/2,2/5-2/8,2/13-2/16,2/21-2/22,2/25-2/26,2/29-2/44 untagged 1/3-1/4,1/9-1/12,1/17-1/20,1/23-1/24,1/27-1/28,1/45-1/48,1/A1-1/A2,2/3-2/4,2/9-2/12,2/17-2/20,2/23-2/24,2/27-2/28,2/45-2/48,2/A1-2/A2,Trk144 no ip address exitvlan 100 name "Mgmt" untagged 1/29-1/44,2/29-2/44 tagged Trk144 no ip address exitvlan 101 name "Private LAN" untagged 1/25-1/26,2/25-2/26 tagged Trk144 ip address 10.10.1.7 255.255.255.0 exitvlan 202 name "DMZ LAN" untagged 1/5-1/8,2/5-2/8 tagged Trk144 no ip address exitvlan 212 name "DMZ-LB LAN" untagged 1/13-1/16,2/13-2/16 tagged Trk144 no ip address exitvlan 217 name "Corp 217 VLAN" untagged 1/21-1/22,2/21-2/22 tagged Trk144 no ip address exitvlan 250 name "Public WAN" untagged 1/1-1/2,2/1-2/2 tagged Trk144 no ip address exitspanning-treespanning-tree Trk144 priority 4password manager
Please let me know if you see anything I've overlooked.
Thank you all again so very much!!!
------------------------------
Clint
Original Message:
Sent: Oct 06, 2021 06:04 AM
From: Vincent Giles
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Ack, and agree this would be fantastic :-)
Not sure that we would be able to maintain such guide with all current and future releases of FW vendors: PA, Fortinet, Checkpoint to name a few...
So current guidance: if any LACP pre-negotiation is enabled is 2 VSX LAGs. I can add this comment/guidance in the VSX best practices of future revision of documentation user-guide to, at least, capture the point and provide clarity.
------------------------------
Vincent Giles
Original Message:
Sent: Oct 06, 2021 05:38 AM
From: Davide Poletto
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Super! so know we know what to ask/check/verify in order to correctly configure the VSX against such Firewall's deployments.
My take is: it would be fantastic to see Aruba providing some "VSX-to-Firewall-Clusters" validated scenarios (say against major players like Cisco, PaloAlto, Fortinet, Forcepoint, etc.) or, at least, create a KB Article or a tiny guide where to place what pre-checks should be done against a particular "bulk" Firewall Cluster (treating scenarios instead of vendors) to select the most appropriate VSX configuration and what performance drawbacks/plus one should expect...sort of pro versus cons (as you said...conservatively...the mainstream approach where one VSX LAG against a physical Firewall node of a Firewall Cluster...seems to be the recommended way to go).
------------------------------
Davide Poletto
Original Message:
Sent: Oct 06, 2021 04:47 AM
From: Vincent Giles
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
ok. For that one, this is very dependent on the FW behavior
I didn't check all FW vendors, and the one that I look at for this pre-negotiation was PaloAlto:
https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/high-availability/ha-concepts/lacp-and-lldp-pre-negotiation-for-activepassive-ha.html
Best would be to test (I don't have one handy) or to dive into the documentation to understand was is the bridge-ID use for this pre-negotiation.
1) If the bridge-ID is the same than on the active FW, a deeper look is required to understand how PaloAlto sends LACP out-of-sync so that LACP neighbor (here our Aruba CX switch) does not send traffic on the links connecting the standby FW (this would lead to drops).
2) if bridge-ID is different than on the active FW, then the same check is required in order to guarantee that the LACP neighbor (CX switch) will select the right links for the active bridge-ID. Again this is very much dependent on the FW side.
Bottom-line, not knowing those level of details, it is recommended to have a conservative approach and have separate VSX LAG for active FW and for standby FW when LACP pre-negotiation is enabled on FW.
Does it clarify ?
------------------------------
Vincent Giles
Original Message:
Sent: Oct 06, 2021 04:32 AM
From: Davide Poletto
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Hi Vincent! well what was asked on 4th reply about the pre-negotiating LLDP and LACP capabilities on the Standby member of the Firewall Cluster. That would be a nice clarification to understand the better way to connect a VSX to a Firewall Cluster depending on its deployed architecture (Active/Active versus Active/Standby) and its capabilities.
------------------------------
Davide Poletto
Original Message:
Sent: Oct 06, 2021 04:25 AM
From: Vincent Giles
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Hi Davide,
Let me know if there is still something to clarify as there are too many high-frequency messages in this thread for me to follow-up :-)
------------------------------
Vincent Giles
Original Message:
Sent: Oct 05, 2021 05:34 PM
From: Davide Poletto
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Clint, Robert...I was observing this thread with a lot of curiosity since beginning (and indeed, at some point, I asked Vincent to clarify a real doubt I had once I've read his notes about stretching a VSX LAG against both Firewall nodes hoping his answer could shed some light also on how Clint saw his Firewall cluster from the PoV of his VSX...but things were rolling too fast!): this thread contains good questions and it shows how some of the VSX basic concepts - over all the VSX LAG side by side with equirements/restrictions/method of operation and deployment - that can't be always immediately "given for granted" and this assumption is especially true if a Network Administrator - newbie or not he/she is - is carrying a different background (example: VSF/IRF know-how).
------------------------------
Davide Poletto
Original Message:
Sent: Oct 05, 2021 04:28 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
The light bulbs are slowly getting brighter, and it's all thanks to you guys!!! I'm just trying to do too much, too quickly, all at once.
This is the first of many projects back to back to back that I have been working on for a couple months. We have a this upgrade this weekend, followed up next weekend by a 3PAR & Servers move, followed by a 3PAR & Servers upgrade, all while trying to move another, much smaller DC footprint. Oh, I forgot to mention, we are building out a new HQ, so all the demo and rebuilding that's associated with a new office build out is in progress, and my responsibility. And, we have to be out of the old HQ by November 1st. Fun Fun all around!!!
Thank you both so much!!!
------------------------------
Clint
Original Message:
Sent: Oct 05, 2021 04:17 PM
From: Robert Großmann
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
no, its not the firewall, the firewall does not know about mclag or lag on switchside...
the switches need to have the same mc-lag configuration to work properly.
Perhaps a native english speaker could explain it better.
here the VSX Best practice guide (cx 10.6): Document Display | HPE Support Center
Edit:
Davide was faster ;-)
------------------------------
Robert Großmann
Original Message:
Sent: Oct 05, 2021 03:47 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
So the target (the firewall) device denotes the same lag-key and I've been using the source (switch), right?
Let me make those changes and I'll repost. Thank you!!!!
------------------------------
Clint
Original Message:
Sent: Oct 05, 2021 03:19 PM
From: Robert Großmann
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
no, no...
The first diagram is right. The second one only, if the passive interface shuts down its data interfaces.
I'm sorry. I only have looked at the first mclag interfaces, and thats what im wondering about:
provided config says:
sw-01 1/1/43 --> fw-01 == mclag 100
sw-01 1/1/44 --> fw-02 == mclag 100
sw-01 1/1/52 --> fw-01 == mclag 101
sw-01 1/1/53 --> fw-02 == mclag 101
and
sw-02 1/1/43 --> fw-01 == mclag 99
sw-02 1/1/44 --> fw-02 == mclag 99
sw-02 1/1/52 --> fw-01 == mclag 102
sw-02 1/1/53 --> fw-02 == mclag 102
Thats not right.
even your config does correspondend with diagram 1 regarding to mclag 250/251:
sw-01 1/1/35 --> fw-01 == mclag 250
sw-01 1/1/36 --> fw-02 == mclag 250
and
sw-02 1/1/35 --> fw-01 == mclag 251
sw-02 1/1/36 --> fw-02 == mclag 251
It ever should be like:
sw-01 1/1/35 --> fw-01 == mclag 250
sw-01 1/1/36 --> fw-02 == mclag 251
and
sw-02 1/1/35 --> fw-01 == mclag 250
sw-02 1/1/36 --> fw-02 == mclag 251
same lag interface with same lag-key for multi-chassis lag on both switches!
------------------------------
Robert Großmann
Original Message:
Sent: Oct 05, 2021 02:40 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Trust me, I am thoroughly confused as well at this point! But...
I think if I am understanding you correctly, I had it setup that way originally. For instance MC-LAG 101 was the same on both switches and cables from both switches went to both firewalls. The reason I changed it is because that was my interpretation of what Aruba TAC meant by separate MC-LAGs for each one, going to each switch.
Here is a diagram of what I "thought" A-TAC was suggesting:
Am I to assume this is what you're suggesting?
I don't think you've misunderstood the topology, I think I've torn down and rebuilt so many variations over the last week or so, I've become my own worst enemy, constantly second guessing myself. And you hit a big part of my issues right on the head. I can't get my head out of VSF land (or IRF land for that matter).
Routing – The clients on the Private LAN have the current switches as their default gateway, that is an issue I still need to address, along with the active gateway.
I will start sounding like a broken record, but I have to say thank you Robert so very much for all your time & help!!!
------------------------------
Clint
Original Message:
Sent: Oct 05, 2021 01:29 PM
From: Robert Großmann
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
I am confused...
Why do you have configured 2 MC-LAG Interfaces per function?
The MC-LAG Interface should be the same on both side.
csw-rz-r08# sh run int lag 36interface lag 36 multi-chassis no shutdown description Core-2 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active exitcsw-rz-r08# sh run int 1/1/36interface 1/1/36 no shutdown description Core-2_P_A2 lag 36 exitcsw-rz-r08# sh lag 36System-ID : b8:d4:e7:de:1c:00System-priority : 65534Aggregate lag36 is up Admin state is up Description : Core-2 Type : multi-chassis Lacp Fallback : Disabled MAC Address : 0a:00:0c:01:01:00 Aggregated-interfaces : 1/1/36 Aggregation-key : 36 Aggregate mode : active Hash : l3-src-dst LACP rate : slow Speed : 10000 Mb/s Mode : trunkcsw-rz-r08# sh lag 36 vsx-peerSystem-ID : 64:e8:81:19:32:00System-priority : 65534Aggregate lag36 is up Admin state is up Description : Core-2 Type : multi-chassis Lacp Fallback : Disabled MAC Address : 0a:00:0c:01:01:00 Aggregated-interfaces : 1/1/36 Aggregation-key : 36 Aggregate mode : active Hash : l3-src-dst LACP rate : slow Speed : 10000 Mb/s Mode : trunkcsw-rz-r08#
csw-rz-r09# sh run int lag 36interface lag 36 multi-chassis no shutdown description Core-2 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active exitcsw-rz-r09# sh run int 1/1/36interface 1/1/36 no shutdown description Core-2_P_A6 lag 36 exitcsw-rz-r09# sh lag 36System-ID : 64:e8:81:19:32:00System-priority : 65534Aggregate lag36 is up Admin state is up Description : Core-2 Type : multi-chassis Lacp Fallback : Disabled MAC Address : 0a:00:0c:01:01:00 Aggregated-interfaces : 1/1/36 Aggregation-key : 36 Aggregate mode : active Hash : l3-src-dst LACP rate : slow Speed : 10000 Mb/s Mode : trunkcsw-rz-r09# sh lag 36 vsSystem-ID : b8:d4:e7:de:1c:00System-priority : 65534Aggregate lag36 is up Admin state is up Description : Core-2 Type : multi-chassis Lacp Fallback : Disabled MAC Address : 0a:00:0c:01:01:00 Aggregated-interfaces : 1/1/36 Aggregation-key : 36 Aggregate mode : active Hash : l3-src-dst LACP rate : slow Speed : 10000 Mb/s Mode : trunkcsw-rz-r09#
In your case there must be one mclag interface (the same) on both VSX-Pair Switches for connecting to fw1 and another mclag interface (the same) for connecting to fw2. Or If the interfaces on the passive firewall are shutted down, then you could use one (the same) mclag interface from both switches to both firewalls. Solution 2 could be faster, but Solution 1 is safer (would be bad if traffic goes over physical link to passive firewall).
I guess without upstream routing active-gateway configuration is not necessary, but it should not be the problem.
Or did I have misunderstood the topology?
And one more thing: I guess the IP-Adress on the vlan interface (SVI) must be different, the active-gateway IP-adress is the same. Like VRRP and HSRP... The Best Practice Guide shows different IPs, too.
Think about that it is VSX and not VSF, so for Management and Routing the switches are independent. Not like VSF, where it is a logical Stack...
The Clients do have the firewall VIP IP as the default-gateway? I do not see any routing config.
Ping from switch SVI to client in another VLAN should go through firewall because you have configured VRF.
ping <IP-Client-VLAN-101> vrf VRF100 source vlan100
or through firewall from switch to switch:
ping 10.10.1.1 vrf VRF100 source vlan100
Be patient of the firewall security rules, and if the the used interfaces allow ICMP (directed to the firewall litself).
------------------------------
Robert Großmann
Original Message:
Sent: Oct 05, 2021 11:54 AM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
I forgot to answer one of the questions asked, the version is ArubaOS-CX TL.10.08.0001
Here are the configs as of today and most things so far are acting as expected. There are a few routing issues, I need to look at loop protection and other house cleaning items, but I feel much better today about this than I did yesterday, that's for sure!
Side Note - I'm guessing I don't need all the MC-LAGs in both the Primary & Secondary configs, only the ones specific to each switch, it was more of a cut paste thing than anything else.
------------------------------------------------------------------------------------------
8320 - Primary
------------------------------------------------------------------------------------------
!!Version ArubaOS-CX TL.10.08.0001!export-password: defaulthostname Swt-8320-01profile L3-corevrf KeepAlivevrf VRF100vrf VRF101vrf VRF202vrf VRF212vrf VRF217vrf VRF250ntp server pool.ntp.org minpoll 4 maxpoll 4 iburstntp enablentp vrf mgmtcli-session timeout 0!!!!!aruba-central disablessh server vrf defaultssh server vrf mgmtvlan 1vlan 100 name Mgmt VLAN vsx-sync description Mgmt VLAN - VLAN 100vlan 101 name Private VLAN vsx-sync description Private VLAN - VLAN 101vlan 202 name DMZ VLAN vsx-sync description DMZ VLAN - VLAN 202vlan 212 name DMZ-LB VLAN vsx-sync description DMZ-LB VLAN - VLAN 212vlan 217 name Corp217 VLAN vsx-sync description Corp 217 VLAN - VLAN 217vlan 250 name Public VLAN vsx-sync description Public VLAN - VLAN 250interface mgmt no shutdown ip static 10.10.0.2/24 default-gateway 10.10.0.254 nameserver 10.1.1.20 10.1.1.21interface lag 99 multi-chassis no shutdown description Mgmt/MLAG 99 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 100 multi-chassis no shutdown description Mgmt/MLAG 100 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 101 multi-chassis no shutdown description Private/MLAG 101 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 102 multi-chassis no shutdown description Private/MLAG 102 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 143 multi-chassis no shutdown description Switch Trunk/MLAG 143 - Swt-8320-02 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface lag 144 multi-chassis no shutdown description Switch Trunk/MLAG 144 - Swt-8320-01 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface lag 202 multi-chassis no shutdown description DMZ/MLAG 202 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 203 multi-chassis no shutdown description DMZ/MLAG 203 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 212 multi-chassis no shutdown description DMZ-LB/MLAG 212 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 213 multi-chassis no shutdown description DMZ-LB/MLAG 213 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 217 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 218 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 250 multi-chassis no shutdown description Public/MLAG 250 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 251 multi-chassis no shutdown description Public/MLAG 251 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 256 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface 1/1/26 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/35 no shutdown description Public/MLAG 250 - Swt-8320-01 <-> FW-01 lag 250interface 1/1/36 no shutdown description Public/MLAG 250 - Swt-8320-01 <-> FW-02 lag 250interface 1/1/37 no shutdown description DMZ/MLAG 202 - Swt-8320-01 <-> FW-01 lag 202interface 1/1/38 no shutdown description DMZ/MLAG 202 - Swt-8320-01 <-> FW-02 lag 202interface 1/1/39 no shutdown description DMZ-LB/MLAG 212 - Swt-8320-01 <-> FW-01 lag 212interface 1/1/40 no shutdown description DMZ-LB/MLAG 212 - Swt-8320-01 <-> FW-02 lag 212interface 1/1/41 no shutdown description Corp217/MLAG 217 - Swt-8320-01 <-> FW-01 lag 217interface 1/1/42 no shutdown description Corp217/MLAG 217 - Swt-8320-01 <-> FW-02 lag 217interface 1/1/43 no shutdown description Mgmt/MLAG 100 - Swt-8320-01 <-> FW-01 lag 100interface 1/1/44 no shutdown description Mgmt/MLAG 100 - Swt-8320-01 <-> FW-02 lag 100interface 1/1/45 no shutdown description Switch Trunk/MLAG 143 - Swt-8320-01 <-> 3810M lag 143interface 1/1/46 no shutdown description Switch Trunk/MLAG 143 - Swt-8320-01 <-> 3810M lag 143interface 1/1/48 no shutdown vrf attach KeepAlive description VSX-ISL-KeepAlive-Link ip address 192.168.168.1/24interface 1/1/51 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface 1/1/52 no shutdown description Private/MLAG 101 - Swt-8320-01 <-> FW-01 lag 101interface 1/1/53 no shutdown description Private/MLAG 101 - Swt-8320-01 <-> FW-02 lag 101interface 1/1/54 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface vlan 100 vsx-sync active-gateways vrf attach VRF100 description Mgmt VLAN - VLAN 100 ip address 10.10.0.1/24 active-gateway ip mac 02:01:00:00:01:01 active-gateway ip 10.10.0.254interface vlan 101 vsx-sync active-gateways vrf attach VRF101 description Private VLAN - VLAN 101 ip address 10.10.1.1/24 active-gateway ip mac 02:01:00:00:01:02 active-gateway ip 10.10.1.254interface vlan 202 vsx-sync active-gateways vrf attach VRF202 description DMZ VLAN - VLAN 202 ip address 10.20.20.1/24 active-gateway ip mac 02:01:00:00:01:03 active-gateway ip 10.20.20.254interface vlan 212 vsx-sync active-gateways vrf attach VRF212 description DMZ-LB VLAN - VLAN 212 ip address 10.21.21.254/24 active-gateway ip mac 02:01:00:00:01:04 active-gateway ip 10.21.21.1interface vlan 217 vsx-sync active-gateways vrf attach VRF217 description Corp 217 VLAN - VLAN 217 ip address 10.10.217.1/24 active-gateway ip mac 02:01:00:00:01:05 active-gateway ip 10.10.217.254interface vlan 250 vsx-sync active-gateways vrf attach VRF250 description Public VLAN - VLAN 250 ip address 10.20.20.1/24 active-gateway ip mac 02:01:00:00:01:06 active-gateway ip 10.20.20.254vsx system-mac 02:01:00:00:01:00 inter-switch-link lag 256 role primary keepalive peer 192.168.168.2 source 192.168.168.1 vrf KeepAlive linkup-delay-timer 30 vsx-sync copp-policy dhcp-relay dns icmp-tcp loop-protect-global mclag-interfaces qos-global snmp ssh static-routes stp-global time vsx-globalip dns server-address 10.1.1.20ip dns server-address 10.1.1.21!!!!!ip source-interface all 10.10.1.2https-server session-timeout 480https-server vrf defaulthttps-server vrf mgmt
------------------------------------------------------------------------------------------
8320 - Secondary
------------------------------------------------------------------------------------------
!!Version ArubaOS-CX TL.10.08.0001!export-password: defaulthostname Swt-8320-02profile L3-corevrf KeepAlivevrf VRF100vrf VRF101vrf VRF202vrf VRF212vrf VRF217vrf VRF250ntp server pool.ntp.org minpoll 4 maxpoll 4 iburstntp enablentp vrf mgmtcli-session timeout 0!!!!!aruba-central disablessh server vrf defaultssh server vrf mgmtvlan 1vlan 100 name Mgmt VLAN vsx-sync description Mgmt VLAN - VLAN 100vlan 101 name Private VLAN vsx-sync description Private VLAN - VLAN 101vlan 202 name DMZ VLAN vsx-sync description DMZ VLAN - VLAN 202vlan 212 name DMZ-LB VLAN vsx-sync description DMZ-LB VLAN - VLAN 212vlan 217 name Corp217 VLAN vsx-sync description Corp 217 VLAN - VLAN 217vlan 250 name Public VLAN vsx-sync description Public VLAN - VLAN 250interface mgmt no shutdown ip static 10.10.0.3/24 default-gateway 10.10.0.254 nameserver 10.1.1.20 10.1.1.21interface lag 99 multi-chassis no shutdown description Mgmt/MLAG 99 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 100 multi-chassis no shutdown description Mgmt/MLAG 100 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 100 vlan trunk allowed 100 lacp mode active lacp rate fastinterface lag 101 multi-chassis no shutdown description Private/MLAG 101 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 102 multi-chassis no shutdown description Private/MLAG 102 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 101 vlan trunk allowed 101 lacp mode active lacp rate fastinterface lag 143 multi-chassis no shutdown description Switch Trunk/MLAG 143 - Swt-8320-02 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface lag 144 multi-chassis no shutdown description Switch Trunk/MLAG 144 - Swt-8320-01 <-> 3810M no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface lag 202 multi-chassis no shutdown description DMZ/MLAG 202 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 203 multi-chassis no shutdown description DMZ/MLAG 203 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 202 vlan trunk allowed 202 lacp mode active lacp rate fastinterface lag 212 multi-chassis no shutdown description DMZ-LB/MLAG 212 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 213 multi-chassis no shutdown description DMZ-LB/MLAG 213 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 212 vlan trunk allowed 212 lacp mode active lacp rate fastinterface lag 217 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 218 multi-chassis no shutdown description Corp217/MLAG 217 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 217 vlan trunk allowed 217 lacp mode active lacp rate fastinterface lag 250 multi-chassis no shutdown description Public/MLAG 250 - Swt-8320-01 <-> FW-01 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 251 multi-chassis no shutdown description Public/MLAG 251 - Swt-8320-02 <-> FW-02 no routing vlan trunk native 250 vlan trunk allowed 250 lacp mode active lacp rate fastinterface lag 256 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 no routing vlan trunk native 1 vlan trunk allowed all lacp mode active lacp rate fastinterface 1/1/26 no shutdown description *** VLAN 100 Test Port *** no routing vlan access 100interface 1/1/35 no shutdown description Public/MLAG 251 - Swt-8320-02 <-> FW-01 lag 251interface 1/1/36 no shutdown description Public/MLAG 251 - Swt-8320-02 <-> FW-02 lag 251interface 1/1/37 no shutdown description DMZ/MLAG 203 - Swt-8320-02 <-> FW-01 lag 203interface 1/1/38 no shutdown description DMZ/MLAG 203 - Swt-8320-02 <-> FW-02 lag 203interface 1/1/39 no shutdown description DMZ-LB/MLAG 213 - Swt-8320-02 <-> FW-01 lag 213interface 1/1/40 no shutdown description DMZ-LB/MLAG 213 - Swt-8320-02 <-> FW-02 lag 213interface 1/1/41 no shutdown description Corp217/MLAG 218 - Swt-8320-02 <-> FW-01 lag 218interface 1/1/42 no shutdown description Corp217/MLAG 218 - Swt-8320-02 <-> FW-02 lag 218interface 1/1/43 no shutdown description Mgmt/MLAG 99 - Swt-8320-02 <-> FW-01 lag 99interface 1/1/44 no shutdown description Mgmt/MLAG 99 - Swt-8320-02 <-> FW-02 lag 99interface 1/1/45 no shutdown description Switch Trunk/MLAG 144 - Swt-8320-02 <-> 3810M lag 144interface 1/1/46 no shutdown description Switch Trunk/MLAG 144 - Swt-8320-02 <-> 3810M lag 144interface 1/1/48 no shutdown vrf attach KeepAlive description VSX-ISL-KeepAlive-Link ip address 192.168.168.2/24interface 1/1/51 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface 1/1/52 no shutdown description Private/MLAG 101 - Swt-8320-02 <-> FW-01 lag 102interface 1/1/53 no shutdown description Private/MLAG 101 - Swt-8320-02 <-> FW-02 lag 102interface 1/1/54 no shutdown description VSX-ISL-LAG/MLAG 256 - Swt-8320-01 <-> Swt-8320-02 lag 256interface vlan 100 vsx-sync active-gateways vrf attach VRF100 description Mgmt VLAN - VLAN 100 ip address 10.10.0.1/24 active-gateway ip mac 02:01:00:00:01:01 active-gateway ip 10.10.0.254interface vlan 101 vsx-sync active-gateways vrf attach VRF101 description Private VLAN - VLAN 101 ip address 10.10.1.1/24 active-gateway ip mac 02:01:00:00:01:02 active-gateway ip 10.10.1.254interface vlan 202 vsx-sync active-gateways vrf attach VRF202 description DMZ VLAN - VLAN 202 ip address 10.20.20.1/24 active-gateway ip mac 02:01:00:00:01:03 active-gateway ip 10.20.20.254interface vlan 212 vsx-sync active-gateways vrf attach VRF212 description DMZ-LB VLAN - VLAN 212 ip address 10.21.21.254/24 active-gateway ip mac 02:01:00:00:01:04 active-gateway ip 10.21.21.1interface vlan 217 vsx-sync active-gateways vrf attach VRF217 description Corp 217 VLAN - VLAN 217 ip address 10.10.217.1/24 active-gateway ip mac 02:01:00:00:01:05 active-gateway ip 10.10.217.254interface vlan 250 vsx-sync active-gateways vrf attach VRF250 description Public VLAN - VLAN 250 ip address 10.20.20.1/24 active-gateway ip mac 02:01:00:00:01:06 active-gateway ip 10.20.20.254vsx system-mac 02:01:00:00:01:00 inter-switch-link lag 256 role secondary keepalive peer 192.168.168.1 source 192.168.168.2 vrf KeepAlive linkup-delay-timer 30 vsx-sync copp-policy dhcp-relay dns icmp-tcp loop-protect-global mclag-interfaces qos-global snmp ssh static-routes stp-global time vsx-globalip dns server-address 10.1.1.20ip dns server-address 10.1.1.21!!!!!ip source-interface all 10.10.1.3https-server session-timeout 480https-server vrf defaulthttps-server vrf mgmt
Again, thank you all for your time & help!
------------------------------
Clint
Original Message:
Sent: Oct 05, 2021 11:37 AM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Robert - I went with the multiple MC-LAG's primarily to keep things as close as possible to the current config, for the most part meaning keep it "simple" and our firewall rules are based on ports, not VLANs. I have a very limited maintenance window this weekend to get everything installed, and I didn't want to worry about having to re-configuring everything else to fit a new topology. Because we are splitting everything into separate MC-LAGs, the firewall has no idea about any of the VLANs, all our rules are port based/physically separate.
I am finding that may be a poor way to go for the long run, we seem to be at the point where we have crossed over and trying to keep everything physically separated (port based rules) is actually becoming more complex than using less, but larger MC-LAGs with VLANs. Unfortunately, converting to that methodology would have to be a separate project, I do not have that large of a window this weekend.
Vincent – Our firewalls do support ECMP for Static Routing, OSPF, & BGP, but I fear that is getting a bit outside my comfort zone, once you get away from basic static routing, I have almost zero experience. I have looked at page 104 of the VSX best practice, in fact that's what drove me to where I am today with the configs.
I am extremely intrigued by your statement:
"having 1 single MCLAG with 4 links is fine: 2 links will appear with LACP-block, as the standby FW will not send LACPDU and will not negotiate"
That was my original config, and traffic was traversing the links as expected, including when I failed over the firewalls, and I thought I was good to go at that point. But, Aruba TAC stated that would not work as expected and might not failover as expected at times, my guess is due to the LACP-Block State those ports would be in. So the question becomes, what are the pros & cons of each? Obviously 4 cables in a single MC-LAG together greatly simplifies the config, but 2 separate "2 cable" MC-LAGs appears to be the more supported config.
I'm curious, why the "no ip icmp redirect"? I've had it both ways and I can't quit determine the differences. I know I need to wrap my head around the active forwarding & active gateways better, and is it even necessary for what I'm trying to do?
I will post the configs as they are today in a few minutes. I'm sure there's plenty wrong with them, so my apologies in advance if they are way off base.
Thank you all for your help so far, you have all been most gracious with your time and it is greatly appreciated!!!
------------------------------
Clint
Original Message:
Sent: Oct 05, 2021 06:29 AM
From: Davide Poletto
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Hi Vincent, as usual what you wrote must be read very carefully since it's always quite important and clarifier, so let me just to ask you a clarification since I have a doubt (my case is going to be Active/Standby scenario and I planned the two VSX LAGs approach before reading about the capabilities you cited above).
When you wrote:
"For active/standby, some FW vendors have also a capability for pre-negotiating LLDP and LACP on standby. In such a case, it might be recommended to have one VSX LAG per FW."
I understand that the capabilities for pre-negotiating LLDP and LACP on the Standby you're referring to (in a Active/Standby Firewall scenario, I add) are not necessarily mandatory requirements permitting us to work with just one VSX LAG made of physical links coming from both Firewall nodes (those nodes should indeed support these capabilities and they also should act as a single logical virtualized device as per LAG forming requirements) so in their absence one can still try the single VSX LAG approach targeting both Firewall nodes.
It's to understand that in presence of an Active/Standby Firewall scenario - pre-negotiating LLDP and LACP capabilities or not, in any case - that scenario will force us to work with separate VSX LAGs in any case (and each VSX LAG will receive physical links coming from a Firewall node, not from both nodes) in contrast with the alternative Active/Active Firewall scenario where instead a single VSX LAG could be used or not (and, if it used, that VSX LAG will receive physical links coming from both Firewall nodes in a concurrent fashion).
"Otherwise, having 1 single MCLAG with 4 links is fine: 2 links will appear with LACP-block, as the standby FW will not send LACPDU and will not negotiate."
Given this last sentence it seems that - in case of no capability for pre-negotiating LLDP and LACP on the Standby Firewall node - we can still try the single VSX LAG approach (as per Active/Active Firewall scenario) BUT we should expect that half of the physical member links of the VSX LAG will appear in LACP-Block state (the Standby Firewall node will indeed not send LACPDU).
Am I correct or wrong? that's very interesting!
------------------------------
Davide Poletto
Original Message:
Sent: Oct 05, 2021 04:04 AM
From: Vincent Giles
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Few points:
- what you try to achieve (as far as I can understand without the configuration) is perfectly possible and done by other people.
- for the routing aspect, if your FW does not support ECMP, you may use recommendation of VSX best practice (p104):
https://support.hpe.com/hpsc/doc/public/display?docId=a00094242en_us
- the choice between 4 links in one big MCLAG (Aruba name is VSX LAG) or 2 MCLAGs of 2 links each really depends on the MCLAG ability of the FW as well, if you target active/active. For active/standby, some FW vendors have also a capability for pre-negotiating LLDP and LACP on standby. In such a case, it might be recommended to have one VSX LAG per FW. Otherwise, having 1 single MCLAG with 4 links is fine: 2 links will appear with LACP-block, as the standby FW will not send LACPDU and will not negotiate.
- I recommend to turn-off ip icmp redirect on VSX priary and secondary (no ip icmp redirect)
- what AOS-CX version ?
------------------------------
Vincent Giles
Original Message:
Sent: Oct 04, 2021 01:35 PM
From: Clint Anderson
Subject: 8320's Multiple MLAG's to Active/Passive Firewalls
Hi All,
I'm looking for some serious help configuring our new switches, I fear I'm either in over my head, or, it's not possible to do what I'm envisioning for this deployment. I've read the AOS-CX Multi Chassis Link Aggregation Guide and I've seen the posts referring to Page 104 of the "VSX Configuration Best Practices for Aruba CX 6400, 8320, 8325, 8360, 8400", as well as the "VSX Stack to firewall" community post here, but I just can't seem to get it right.
I will pose the questions first, so whilst you're reading you can determine just how far off base or crazy I am...
1) Is implementing multiple MLAG's for different network segments between the Firewalls & Aruba 8320's possible?
2) Is implementing multiple MLAG's for different network segments between the Firewalls & Aruba 8320's the recommended way to accomplish what we need?
2a) If so, is there anyone out there that can help me from darn near the ground up get going in the right direction?
2a) If not, what is the best way to accomplish this with 8320's, and, can someone help get these up and running?
With all that said, on to the issue...
Currently we have a pair of FortiGate 800C's and a pair of HPE Flex Fabric 5700's (JG898A) in place, which we are replacing with a pair of FortiGate 1801F's and a pair of Aruba 8320's. I'm having an issue both conceptually and literally implementing the MLAG between the Active/Passive Firewalls & Switches.
Currently we have the old school A Side / B Side failover where if one piece of gear in the "A" stack fails, pretty much everything just shifts over to the "B" stack. I'd like to change that to what I "believe" is the better solution and use VSX (MLAG) between the 8320's & FW's.
To be completely honest, I'm not sure of the best way to implement the new gear, so I may be way off base. I am at the point, due to extreme frustration and a severe time constraint, where I am ready to bail and just mirror the current A/B topology, but I'm hoping someone here can help me so it doesn't come to that.
Oversimplified, here is what I am trying to do:
The ultimate config though needs to have multiple MLAG's for each "segment" of our network (Mgmt LAN, Private LAN, DMZ LAN, DMZ-DB LAN, and Public WAN).
I haven't been able to confidently implement the first diagram, let alone the second. I did have a configuration with 4 MLAG's (2 to each firewall – Mgmt LAN / Private LAN) setup, and was able to get traffic to traverse the switches and firewalls, and failover, but certain things just didn't "seem" right. Traffic would traverse between two external devices, but the 8320's themselves couldn't ping anything, and neither could the firewalls.
Two more little tidbits. First, our current switches handle the routing for traffic between certain segments so the traffic does not need to flow through the firewall. Second, there is a stack of 3810M's that needs to sit downstream of the 8320's and have all the different network segments available there as well.
I know this was a tough read, and for anyone that has made it this far, I greatly appreciate your time! As I mentioned before, I am under a severe time constraint, so if anyone out there can at least tell me with some certainty that Multiple MLAG's is the way to go or not, that would be a huge hue place to start.
Thank you all very much for your time!!!
------------------------------
Clint
------------------------------