Active gateway in VSX Cluster ISSUE in svi disable

8. RE: Active gateway in VSX Cluster ISSUE in svi disable

Kudos

thomasbnc

Posted Feb 14, 2022 07:16 AM
Edited by thomasbnc Feb 14, 2022 07:32 AM

Hi Daniele, Davide

Well that "square" topology is something which you may have if you have 2 datacenters with 2 routing VSX clusters connected L2 to each other. So in fact each VLAN can have 4 potential routers who forward traffic to other VLANs. With such a topology it is mandatory to have a symmetric configuration on both sides (e.g. the same list of VLANs which get routed using an AG). If not, that you need to configure the asymmetry with a different AG MAC. See here for details:

So anyway, shutting down an SVI with an AG configured on it is not a good idea. If you have use cases who require such a thing it is better to go with VRRP. Alternatively, use a different MAC address for the ones you want to test which may work if not more AGs with that MAC are running on a particular node.
However, it is not recommended to have each VLAN configured with a different MAC by default as you run into scalability limits and usual cases do not have a benefit of this. So just you it there where you have asymmetry if any kind as the problem is that a switch "consumes" a frame destined for a AG MAC no matter whether it is on that VLAN or not as long as it is configured on the system. This is perhaps to avoid MAC flapping in "square topology" scenarios.

------------------------------
Thomas Siegenthaler
------------------------------

Original Message

Original Message:
Sent: Feb 14, 2022 06:22 AM
From: Davide Poletto
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Given that your second VSX is acting as Layer 2 only, is that not enough to move your scenario out from the "Square Topology" corner case? I mean, your second VSX (CX8325-AGGR-A and CX8325-AGGR-B) doesn't route if it is just a Layer 2 extension of the VSX at the top (CX8325-CORE-A and CX8325-CORE-B)...or have I misunderstood the "Square (-routing) Topology" concept?

------------------------------
Davide Poletto

Original Message:
Sent: Feb 14, 2022 05:30 AM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Tomas,
yes i've other active gateway configured on switch with save VMAC " active gateway ip mac 12:01:00:00:01:00" . see the attached file.

Aruba best practice tells: https://www.arubanetworks.com/techdocs/AOS-CX/10.07/HTML/5200-7888/Content/Chp_Pre_tra_loss/act-gat-ove-vsx-10.htm

"Having same VMAC and different active gateway IP addresses on different VSX segments in a square topology is not supported. Ensure that you have either same VMAC and same active gateway IP addresses or different VMAC and different active gateway IP addresses configured on two different VSX segments. For 8320 and 8325 switch series, when VMAC and active gateway IP addresses are same, make sure that the SVI status is identical on both the VSX segments."

maybe it is my case since I use cx8325 ... the only limitation in this case is that I could not have more than 16 vlans with active gtw since at most aruba supports 16 VMAC, right?

With VRRP the problem does not exist. i loss some packet because change mastership, but not drop all packet like AG when shutdown one svi on primary.The reason why I tried to disable SVI on the primiry is to test the AG functionality for that vlan and see if there was no connectivity loss, it was a simple test.Because I don't fully understand how AG works compared to VRRP.

------------------------------
daniele dedda

Original Message:
Sent: Feb 14, 2022 02:00 AM
From: Thomas Siegenthaler
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi

Do you perhaps have other active gateways configured on the same switch with the same MAC address? If yes, depending on the model, the switch consumes the frame with this MAC address (instead of forwarding it to the remaining active SVI) and eventually drops it as there is no active-gateway enabled locally on that VLAN.

So if you have any particular reason to disable the AG on one side, make sure you use a unique MAC address for the particular VLAN. Be aware that there is scalability limits and you cannot have endless MAC addresses in place for all your VLANs.

Regards,
Thomas

------------------------------
Thomas Siegenthaler

Original Message:
Sent: Feb 10, 2022 06:55 PM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi,
i've this test infrastructure

CORE A and CORE B are configured in VSX with active gateway for each vlan svi interface. follow the config example for one vlan:

CX-8325-CORE-A
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.34/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

CX-8325-CORE-B
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.35/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

A server 1 is in vlan 2001 and server 2 in other vlan,i do a continuos ping between them.
CASE 1 - i make shutdown the interface svi 2001 on CORE A , the ping is stopping and it resumes only if i make no shutdown on svi. the svi 2001 on core b is always active
CASE 2 - when i make shutdown the interface svi on CORE B , the ping isn't stopping. the svi 2001 on CORE A is always active.

Is the behavior correct? i would have expected that even in case 1 the ping would not stop, otherwise what is the advantage of active gtw? it is not a virtual gtw like in vrrp?

thanks

------------------------------
daniele dedda
------------------------------

9. RE: Active gateway in VSX Cluster ISSUE in svi disable

Kudos

parnassus

Posted Feb 14, 2022 09:43 AM
Edited by parnassus Feb 14, 2022 02:13 PM

Hi Thomas,

"Well that "square" topology is something which you may have if you have 2 datacenters with 2 routing VSX clusters connected L2 to each other."

Yes, exactly. The topology example you posted (from "APPENDIX F – VLAN extension between two VSX clusters" at page 106 of "VSX Configuration Best Practices for Aruba CX 6400, 8320, 8325, 8360, 8400" Edition 1.3 December 2020) represents indeed another case - DC1 L3 VSX with AG <- VSX LAG (back-to-back) -> DC2 L3 VSX with AG - compared to (if I've correctly understood what Daniele represented above) the one we're probably dealing here - L3 VSX with AG <- VSX LAG (back-to-back) -> L2 VSX without AG - so my doubts about being really in a case where the "Square (-routing) Topology" is effective (with all its AG related restrictions/requirements).

To me, Daniele's scenario looks like quite "standard" (at least from the point of view of the involved Active Gateway features) so IMHO it's really a matter of understanding/clarifying the what-happen-if case when a particular SVI (with AG enabled and correctly configured as per best practices) is administratively shut down (on VSX Primary or on VSX Secondary) considering the current "Core" L3 VSX (CX8325-CORE-A and CX8325-CORE-B) running configuration.

Indeed I agree with your sentence "So anyway, shutting down an SVI with an AG configured on it is not a good idea.", this thread would be a great opportunity to clarify this particular scenario (also to correctly understand not only the AG implementations requirements/restrictions but also to highlight AG feature properties).

------------------------------
Davide Poletto
------------------------------

Original Message

Original Message:
Sent: Feb 14, 2022 07:16 AM
From: Thomas Siegenthaler
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Daniele, Davide

Well that "square" topology is something which you may have if you have 2 datacenters with 2 routing VSX clusters connected L2 to each other. So in fact each VLAN can have 4 potential routers who forward traffic to other VLANs. With such a topology it is mandatory to have a symmetric configuration on both sides (e.g. the same list of VLANs which get routed using an AG). If not, that you need to configure the asymmetry with a different AG MAC. See here for details:

So anyway, shutting down an SVI with an AG configured on it is not a good idea. If you have use cases who require such a thing it is better to go with VRRP. Alternatively, use a different MAC address for the ones you want to test which may work if not more AGs with that MAC are running on a particular node.
However, it is not recommended to have each VLAN configured with a different MAC by default as you run into scalability limits and usual cases do not have a benefit of this. So just you it there where you have asymmetry if any kind as the problem is that a switch "consumes" a frame destined for a AG MAC no matter whether it is on that VLAN or not as long as it is configured on the system. This is perhaps to avoid MAC flapping in "square topology" scenarios.

------------------------------
Thomas Siegenthaler

Original Message:
Sent: Feb 14, 2022 06:22 AM
From: Davide Poletto
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Given that your second VSX is acting as Layer 2 only, is that not enough to move your scenario out from the "Square Topology" corner case? I mean, your second VSX (CX8325-AGGR-A and CX8325-AGGR-B) doesn't route if it is just a Layer 2 extension of the VSX at the top (CX8325-CORE-A and CX8325-CORE-B)...or have I misunderstood the "Square (-routing) Topology" concept?

------------------------------
Davide Poletto

Original Message:
Sent: Feb 14, 2022 05:30 AM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Tomas,
yes i've other active gateway configured on switch with save VMAC " active gateway ip mac 12:01:00:00:01:00" . see the attached file.

Aruba best practice tells: https://www.arubanetworks.com/techdocs/AOS-CX/10.07/HTML/5200-7888/Content/Chp_Pre_tra_loss/act-gat-ove-vsx-10.htm

"Having same VMAC and different active gateway IP addresses on different VSX segments in a square topology is not supported. Ensure that you have either same VMAC and same active gateway IP addresses or different VMAC and different active gateway IP addresses configured on two different VSX segments. For 8320 and 8325 switch series, when VMAC and active gateway IP addresses are same, make sure that the SVI status is identical on both the VSX segments."

maybe it is my case since I use cx8325 ... the only limitation in this case is that I could not have more than 16 vlans with active gtw since at most aruba supports 16 VMAC, right?

With VRRP the problem does not exist. i loss some packet because change mastership, but not drop all packet like AG when shutdown one svi on primary.The reason why I tried to disable SVI on the primiry is to test the AG functionality for that vlan and see if there was no connectivity loss, it was a simple test.Because I don't fully understand how AG works compared to VRRP.

------------------------------
daniele dedda

Original Message:
Sent: Feb 14, 2022 02:00 AM
From: Thomas Siegenthaler
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi

Do you perhaps have other active gateways configured on the same switch with the same MAC address? If yes, depending on the model, the switch consumes the frame with this MAC address (instead of forwarding it to the remaining active SVI) and eventually drops it as there is no active-gateway enabled locally on that VLAN.

So if you have any particular reason to disable the AG on one side, make sure you use a unique MAC address for the particular VLAN. Be aware that there is scalability limits and you cannot have endless MAC addresses in place for all your VLANs.

Regards,
Thomas

------------------------------
Thomas Siegenthaler

Original Message:
Sent: Feb 10, 2022 06:55 PM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi,
i've this test infrastructure

CORE A and CORE B are configured in VSX with active gateway for each vlan svi interface. follow the config example for one vlan:

CX-8325-CORE-A
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.34/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

CX-8325-CORE-B
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.35/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

A server 1 is in vlan 2001 and server 2 in other vlan,i do a continuos ping between them.
CASE 1 - i make shutdown the interface svi 2001 on CORE A , the ping is stopping and it resumes only if i make no shutdown on svi. the svi 2001 on core b is always active
CASE 2 - when i make shutdown the interface svi on CORE B , the ping isn't stopping. the svi 2001 on CORE A is always active.

Is the behavior correct? i would have expected that even in case 1 the ping would not stop, otherwise what is the advantage of active gtw? it is not a virtual gtw like in vrrp?

thanks

------------------------------
daniele dedda
------------------------------

10. RE: Active gateway in VSX Cluster ISSUE in svi disable

Kudos

d.dedda@sirti.it

Posted Feb 15, 2022 04:14 AM

Hi David and Thomas,

Hi David and Thomas,
ok I understand the Square infrastructure, that will be the next step. For now I have limited myself to a standard configuration as David says,(PoC with core L3 vsx and AGGR vsx L2) just to understand and test the features.
maybe it doesn't make sense to shutdown the svi in AG, in fact as aruba says you will have packet loss/drop .... I'd like to understand why. While
(always with a continuous ping between end point) rebooting the primary this does not happen- no packet loss.
The next step is to understand well the square typology because I will have to replace cisco equipment with aruaba CX of an infrastructure consisting of two datacenters connected in L2, like this:



is there any more in-depth documentation regarding VSX and Square topology? besides the best practices I have not found anything
Thanks

Daniele

------------------------------
daniele dedda
------------------------------

Original Message

Original Message:
Sent: Feb 14, 2022 09:42 AM
From: Davide Poletto
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Thomas,

"Well that "square" topology is something which you may have if you have 2 datacenters with 2 routing VSX clusters connected L2 to each other."

Yes, exactly. The topology example you posted (from "APPENDIX F – VLAN extension between two VSX clusters" at page 106 of "VSX Configuration Best Practices for Aruba CX 6400, 8320, 8325, 8360, 8400" Edition 1.3 December 2020) represents indeed another case - DC1 L3 VSX with AG <- VSX LAG (back-to-back) -> DC2 L3 VSX with AG - compared to (if I've correctly understood what Daniele represented above) the one we're probably dealing here - L3 VSX with AG <- VSX LAG (back-to-back) -> L2 VSX without AG - so my doubts about being really in a case where the "Square (-routing) Topology" is effective (with all its AG related restrictions/requirements).

To me, Daniele's scenario looks like quite "standard" (at least from the point of view of the involved Active Gateway features) so IMHO it's really a matter of understanding/clarifying the what-happen-if case when a particular SVI (with AG enabled and correctly configured as per best practices) is administratively shut down (on VSX Primary or on VSX Secondary) considering the current "Core" L3 VSX (CX8325-CORE-A and CX8325-CORE-B) running configuration.

Indeed I agree with your sentence "So anyway, shutting down an SVI with an AG configured on it is not a good idea.", this thread would be a great opportunity to clarify this particular scenario (also to correctly understand not only the AG implementations requirements/restrictions but also to highlight AG feature properties).

------------------------------
Davide Poletto

Original Message:
Sent: Feb 14, 2022 07:16 AM
From: Thomas Siegenthaler
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Daniele, Davide

Well that "square" topology is something which you may have if you have 2 datacenters with 2 routing VSX clusters connected L2 to each other. So in fact each VLAN can have 4 potential routers who forward traffic to other VLANs. With such a topology it is mandatory to have a symmetric configuration on both sides (e.g. the same list of VLANs which get routed using an AG). If not, that you need to configure the asymmetry with a different AG MAC. See here for details:

So anyway, shutting down an SVI with an AG configured on it is not a good idea. If you have use cases who require such a thing it is better to go with VRRP. Alternatively, use a different MAC address for the ones you want to test which may work if not more AGs with that MAC are running on a particular node.
However, it is not recommended to have each VLAN configured with a different MAC by default as you run into scalability limits and usual cases do not have a benefit of this. So just you it there where you have asymmetry if any kind as the problem is that a switch "consumes" a frame destined for a AG MAC no matter whether it is on that VLAN or not as long as it is configured on the system. This is perhaps to avoid MAC flapping in "square topology" scenarios.

------------------------------
Thomas Siegenthaler

Original Message:
Sent: Feb 14, 2022 06:22 AM
From: Davide Poletto
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Given that your second VSX is acting as Layer 2 only, is that not enough to move your scenario out from the "Square Topology" corner case? I mean, your second VSX (CX8325-AGGR-A and CX8325-AGGR-B) doesn't route if it is just a Layer 2 extension of the VSX at the top (CX8325-CORE-A and CX8325-CORE-B)...or have I misunderstood the "Square (-routing) Topology" concept?

------------------------------
Davide Poletto

Original Message:
Sent: Feb 14, 2022 05:30 AM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi Tomas,
yes i've other active gateway configured on switch with save VMAC " active gateway ip mac 12:01:00:00:01:00" . see the attached file.

Aruba best practice tells: https://www.arubanetworks.com/techdocs/AOS-CX/10.07/HTML/5200-7888/Content/Chp_Pre_tra_loss/act-gat-ove-vsx-10.htm

"Having same VMAC and different active gateway IP addresses on different VSX segments in a square topology is not supported. Ensure that you have either same VMAC and same active gateway IP addresses or different VMAC and different active gateway IP addresses configured on two different VSX segments. For 8320 and 8325 switch series, when VMAC and active gateway IP addresses are same, make sure that the SVI status is identical on both the VSX segments."

maybe it is my case since I use cx8325 ... the only limitation in this case is that I could not have more than 16 vlans with active gtw since at most aruba supports 16 VMAC, right?

With VRRP the problem does not exist. i loss some packet because change mastership, but not drop all packet like AG when shutdown one svi on primary.The reason why I tried to disable SVI on the primiry is to test the AG functionality for that vlan and see if there was no connectivity loss, it was a simple test.Because I don't fully understand how AG works compared to VRRP.

------------------------------
daniele dedda

Original Message:
Sent: Feb 14, 2022 02:00 AM
From: Thomas Siegenthaler
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi

Do you perhaps have other active gateways configured on the same switch with the same MAC address? If yes, depending on the model, the switch consumes the frame with this MAC address (instead of forwarding it to the remaining active SVI) and eventually drops it as there is no active-gateway enabled locally on that VLAN.

So if you have any particular reason to disable the AG on one side, make sure you use a unique MAC address for the particular VLAN. Be aware that there is scalability limits and you cannot have endless MAC addresses in place for all your VLANs.

Regards,
Thomas

------------------------------
Thomas Siegenthaler

Original Message:
Sent: Feb 10, 2022 06:55 PM
From: daniele dedda
Subject: Active gateway in VSX Cluster ISSUE in svi disable

Hi,
i've this test infrastructure

CORE A and CORE B are configured in VSX with active gateway for each vlan svi interface. follow the config example for one vlan:

CX-8325-CORE-A
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.34/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

CX-8325-CORE-B
interface vlan 2001
vsx-sync active-gateways
ip mtu 9100
ip address 10.46.170.35/28
active-gateway ip mac 12:01:00:00:01:00
active-gateway ip 10.46.170.33

A server 1 is in vlan 2001 and server 2 in other vlan,i do a continuos ping between them.
CASE 1 - i make shutdown the interface svi 2001 on CORE A , the ping is stopping and it resumes only if i make no shutdown on svi. the svi 2001 on core b is always active
CASE 2 - when i make shutdown the interface svi on CORE B , the ping isn't stopping. the svi 2001 on CORE A is always active.

Is the behavior correct? i would have expected that even in case 1 the ping would not stop, otherwise what is the advantage of active gtw? it is not a virtual gtw like in vrrp?

thanks

------------------------------
daniele dedda
------------------------------

Wired Intelligent Edge