Wired Intelligent Edge

 View Only
last person joined: yesterday 

Bring performance and reliability to your network with the HPE Aruba Networking Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of your switching devices, and find ways to improve security across your network to bring together a mobile-first solution
Back to discussions
Expand all | Collapse all

ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

This thread has been viewed 92 times

  • 1.  ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 18, 2019 11:45 AM

    Hi all,

     

    I'm interested in learning opinions/pro/cons about using static (so non LACP) VSX LAG (available since ArubaOS-CX 10.02 and also on latest ArubaOS-CX 10.03) against VMware vSphere host's bonded ports (the vSphere host I'm referring to will use VSS Virtual Standard Switch - not VDS Virtual Distributed Switch - so usage of IEEE 803.3ad LACP is not possible and we need to rely on static Non-Protocol Ports Aggregation).

     

    Thanks for sharing, Davide.



  • 2.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 07:31 AM

    When the vmware switches are running in switch independent mode, i would not built in any "ïntelligence" into the network, vmware load-balances per vm on the connected interfaces will do the trick in that cases.

     

    If 1 VM would require more bandwith then 1 interface can offer, then i would implement LACP on vmware (MC-LAG on the core).



  • 3.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 20, 2019 08:07 AM

    Hi, thank you for answering...that's basically true...but:

     

    (1) the balancing is valid for traffic leaving the ESXi host (considering the physical ports connected to the considered vSwitch configured in Active/Active mode)...what happen to incoming traffic to ESXi host on that very same vSwitch if no aggregation (neither LACP nor Static) is deployed?

     

    (2) LACP is supported only on vNetwork with VDS Virtual Distributed Switch...which we can't actually implement (we haven't the supporting licensing for implementing VDS so we work only with VSS Virtual Standard Switch which doesn't support LACP but only static port aggregation without any protocol).

     

    I'm worried to understand if there are pro/cons in using static port trunking on VSS to VSX (with VSX LAG static as per AOS-CX 10.02/10.03) instead of simply using two active/active ports on VSS to VSX.

     

    Thanks!



  • 4.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 08:57 AM

    (1) For vmware load balanced, the incoming traffic wil be received on the same port for that particular vm. So always per-vm load balancing.

     

    (2) I'm not sure what type of load balancing mechanism is used when using static port aggregation, does it even load balance accros more then 1 interface, or is active-standby?

     

    The vmware load-balancing is pretty solid, but load-balancing on per-vm basis is not the most effective way IMO.



  • 5.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 20, 2019 10:32 AM

    Hi!

     

    (1) OK...still I don't understand if a Static LAG (so in a scenario, as you said, with some "intelligence" on Host's physical network ports side and on the peer Switch side) is better or worst than leaving the vSphere Host's physical network ports in Active/Active and letting the vSwitch to balance egress traffic (generated by VM) to physical network...is using Static LAG overcomplicating the scenario (considering that we have VSX as peer) or not?

     

    (2) As far as I understood Static LAG member ports are, once aggregated, all partecipating but, with regards to Static LAG versus LACP LAP, the difference is that Static LAG has no control protocol to manage member ports status and relationship with peer's LACP LAG (so this explain the term "Non Protocol" used by HP/HPE)...this main difference means less control-related features (like no Failover, no Hot-Standby port, no canditate aggregable port configuration verification...illuminating blog articles here and here)...the interesting point is that Static LAG versus LACP LAG doesn't necessarily mean that load balancing algorithms are different in favour of LACP LAG (they are, indeed, the same)...it's just that vSphere requires specifically to use "Route based on IP Hash" with Static LAG (so operating on VSS)...which basically means using the well known "src-dst IP" load balancing algorithm (as it often happens using LACP LAG).



  • 6.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 24, 2019 06:41 PM

    for me avoid static LAG (and also LACP LAG) with VMware ESXi (or only for very specific use case...)



  • 7.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 24, 2019 06:54 PM
    Hi Alexis, thanks! probably I'll step down(or not?) on multichassis static (VSX LAG Static) until I understand how to check if my setup is working or not...I tried yesterday with a 2 10Gbps ports member of a multichassis static (1 port on each VSX Member) against a ESXi 6.7 U3 set with 2 10 Gbps ports in Load Balancing (Route Based on IP Hash)...but, despite all involved ports are up, apparently ArubaOS-CX doesn't provide any feedback via the show lacp interface multi-chassis command I'm used to: I see the lag30(mc) line (local/remote) - lag 30 is my ststic one - totally empty (is this expected because it is not an usual LACP one?)...so, at first, I was a little bit in doubt about what I've set...show events on both peers reports that involved interfaces are up and lag30 is formed...also ESXi seems good on that. Today I'll check it better, I was too much in a hurry yesterday.


  • 8.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 25, 2019 06:14 AM

    The more I dive deeply into VSX Static LAG the more this configuration become interesting from the management standpoint...below my findings about VSX diagnostic done against my configured lag30 and its member interfaces 1/1/33.

     

    ArubaOS-CX GUI reports the VSX (Multi-Chassis) Static LAG lag30 as Up (both interfaces are Up on both nodes):

     

    ESXi host reported Up state for involved NICs' ports:

    ESXi01_s1p2_and_s3p2_connected_to_VSX.png

     

     

    ESXi vSwitch2 used to test VSX Static LAG (lag30):

     

    Properties

Network label			DF Backend (1)
VLAN ID				None (0)

Security

Promiscuous mode		Reject
MAC address changes 		Accept
Forged transmits		Accept

Traffic shaping

Average bandwidth		--
Peak bandwidth			--
Burst size			--

Teaming and failover

Load balancing			Route based on IP hash

INFO: In the IP hash load balancing policy all physical switch ports connected to the active uplinks must be in link aggregation mode.
INFO: IP hash load balancing should be set for all port groups using the same set of uplinks.

Network failure detection	Link status only
Notify switches			Yes
Failback			Yes
Active adapters			vmnic1, vmnic7
Standby adapters		--
Unused adapters			--

    And visually:ESXi01_vSwitch2_connected_to_VSX.png

    Aruba 8320 VSX Primary VSX Static LAG (lag30) status:

    VSX_Static_LAG_lag30_Aruba_8320-1_Primary_01.png

    VSX_Static_LAG_lag30_Aruba_8320-1_Primary_02.png

     

    Aruba 8320 VSX Secondary VSX Static LAG (lag30) status:VSX_Static_LAG_lag30_Aruba_8320-2_Secondary_01.png

    VSX_Static_LAG_lag30_Aruba_8320-2_Secondary_02.png

     

    Now the CLI part:

     

    === Aruba 8320 Node 1 (Primary) VSX Static LAG configuration ===

interface lag 30 multi-chassis static
    vsx-sync vlans
    description 8320-1-VSX-LAG-Static-lag30-ESXi01
    no shutdown
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,20,30,40
    loop-protect
    loop-protect vlan 1,20,30,40

interface 1/1/33 
    description Leg_to_s1p2_ESXi01_vmnic1_vSwitch2
    no shutdown
    mtu 9198
    flow-control rx
    lag 30

=== Aruba 8320 Node 2 (Secondary) VSX Static LAG configuration ===
interface lag 30 multi-chassis static
    vsx-sync vlans
    description 8320-2-VSX-LAG-Static-lag30-ESXi01
    no shutdown
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,20,30,40
    loop-protect
    loop-protect vlan 1,20,30,40

interface 1/1/33 
    description Leg_to_s3p2_ESXi01_vmnic7_vSwitch2
    no shutdown
    mtu 9198
    flow-control rx
    lag 30

=== Aruba 8320 VSX Static LAG (lag30) diagnostic on Primary and Secondary ===
Aruba-8320-1# show lacp aggregates

Aggregate name   : lag30 (multi-chassis)             
Interfaces       : 1/1/33                            
Peer interfaces  : 1/1/33                            
Heartbeat rate   : N/A                               
Hash             : l3-src-dst                        
Aggregate mode   : Off

Aruba-8320-1# show lacp aggregates vsx-peer

Aggregate name   : lag30 (multi-chassis)             
Interfaces       : 1/1/33                            
Peer interfaces  : 1/1/33                            
Heartbeat rate   : N/A                               
Hash             : l3-src-dst                        
Aggregate mode   : Off

Aruba-8320-1# show vsx lacp aggregates lag30   
                    Local-peer                        Remote-peer
-------------------------------------------------------------------------------

Aggregate name   : lag30 (multi-chassis)             lag30 (multi-chassis)
Interfaces       : 1/1/33                            1/1/33 
Peer interfaces  : 1/1/33                            1/1/33 
Heartbeat rate   : N/A                               N/A
Hash             : l3-src-dst                        l3-src-dst
Aggregate mode   : Off                               Off

Aruba-8320-2# show vsx lacp aggregates lag30
                    Local-peer                        Remote-peer
-------------------------------------------------------------------------------

Aggregate name   : lag30 (multi-chassis)             lag30 (multi-chassis)
Interfaces       : 1/1/33                            1/1/33 
Peer interfaces  : 1/1/33                            1/1/33 
Heartbeat rate   : N/A                               N/A
Hash             : l3-src-dst                        l3-src-dst
Aggregate mode   : Off                               Off

Aruba-8320-1# show lacp interfaces

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
        Name       Id    Pri                             Pri    Key  State     
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                    up        

Partner details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr           
        Name       Id    Pri                             Pri    Key            
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                              

Aruba-8320-1# show lacp interfaces vsx-peer 

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
        Name       Id    Pri                             Pri    Key  State     
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                    up        

Partner details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr           
        Name       Id    Pri                             Pri    Key            
------------------------------------------------------------------------------
1/1/33  lag30(mc)

    And configuration related events generated during lag30 setup yesterday (Errors on VSX Secondary should be probably related to the fact I was saving VSX Primary before lag30 was configured on VSX Secondary):

     

    === Aruba 8320 Node 1 (Primary) VSX Static LAG events related to 1/1/33 and lag30 configuration ===

2019-09-24:16:43:17.115485|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/33 is deleted from router discovery
2019-09-24:16:47:17.218517|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/34 is deleted from router discovery
2019-09-24:17:03:11.653192|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:10:02.902801|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:16:59.661158|lacpd|1303|LOG_INFO|AMM|-|Interface 1/1/33 added to LAG 30. Existing configuration on interface 1/1/33 will be removed.
2019-09-24:17:16:59.696098|ops-switchd|4803|LOG_INFO|AMM|-|All dynamic MAC addresses on port 1/1/33 were flushed
2019-09-24:17:16:59.699173|ops-switchd|1401|LOG_DEBUG|AMM|-|Trunk set succeeds unit 0 lag_id 4
2019-09-24:17:16:59.699216|ops-switchd|1301|LOG_INFO|AMM|-|Dynamic LAG lag30 created
2019-09-24:17:16:59.699255|ops-switchd|1409|LOG_INFO|AMM|-|LAG lag30, set to load balance mode to l3-src-dst-hash
2019-09-24:17:16:59.701674|ops-switchd|1404|LOG_DEBUG|AMM|-|Trunk member add port succeeds on unit 0 hw_port 33 tid 4
2019-09-24:17:17:35.273114|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
2019-09-24:17:17:44.802017|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:29:02.232570|pmd|3804|LOG_INFO|AMM|-|Transceiver hot-swap insert for interface 1/1/33
2019-09-24:17:33:40.949961|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
2019-09-24:17:33:41.026283|hpe-vsxd|7012|LOG_INFO|AMM|-|VSX 30 state local down, remote up
2019-09-24:17:34:25.107870|lldpd|110|LOG_INFO|AMM|-|Configured LLDP reinit-delay to 2
2019-09-24:17:34:25.124641|intfd|403|LOG_INFO|AMM|-|Link status for interface 1/1/33 is up
2019-09-24:17:34:25.152520|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up
2019-09-24:17:34:25.156592|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up
2019-09-24:17:34:33.320050|lldpd|104|LOG_INFO|AMM|-|LLDP neighbor f8:f2:1e:59:e7:91 added on 1/1/33

=== Aruba 8320 Node 2 (Secondary) VSX Static LAG events related to 1/1/33 and lag30 configuration ===

2019-09-24:16:50:15.097019|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/33 is deleted from router discovery
2019-09-24:17:03:00.228386|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:03:16.174430|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on primary VSX device.
2019-09-24:17:03:16.580494|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:03:23.817133|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
2019-09-24:17:07:00.282175|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/34 is deleted from router discovery
2019-09-24:17:09:52.080474|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:10:07.285763|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on primary VSX device.
2019-09-24:17:10:07.683633|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:10:14.894096|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
2019-09-24:17:16:39.026350|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
2019-09-24:17:16:39.218306|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
2019-09-24:17:16:43.328284|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
2019-09-24:17:16:43.527611|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
2019-09-24:17:16:47.196342|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
2019-09-24:17:16:47.507171|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
2019-09-24:17:17:03.590207|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
2019-09-24:17:17:19.850505|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
2019-09-24:17:17:22.585726|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : VSX configuration-sync updated database
2019-09-24:17:17:35.231058|lacpd|1303|LOG_INFO|AMM|-|Interface 1/1/33 added to LAG 30. Existing configuration on interface 1/1/33 will be removed.
2019-09-24:17:17:35.327042|ops-switchd|4803|LOG_INFO|AMM|-|All dynamic MAC addresses on port 1/1/33 were flushed
2019-09-24:17:17:35.332017|ops-switchd|1401|LOG_DEBUG|AMM|-|Trunk set succeeds unit 0 lag_id 6
2019-09-24:17:17:35.332083|ops-switchd|1301|LOG_INFO|AMM|-|Dynamic LAG lag30 created
2019-09-24:17:17:35.332150|ops-switchd|1409|LOG_INFO|AMM|-|LAG lag30, set to load balance mode to l3-src-dst-hash
2019-09-24:17:17:35.335095|ops-switchd|1404|LOG_DEBUG|AMM|-|Trunk member add port succeeds on unit 0 hw_port 33 tid 6
2019-09-24:17:17:40.764550|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
2019-09-24:17:17:56.876722|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
2019-09-24:17:29:22.147653|pmd|3804|LOG_INFO|AMM|-|Transceiver hot-swap insert for interface 1/1/33
2019-09-24:17:33:40.963502|lldpd|110|LOG_INFO|AMM|-|Configured LLDP reinit-delay to 2
2019-09-24:17:33:40.979534|intfd|403|LOG_INFO|AMM|-|Link status for interface 1/1/33 is up
2019-09-24:17:33:41.020537|hpe-vsxd|7011|LOG_INFO|AMM|-|VSX 30 state local up, remote down
2019-09-24:17:33:41.029885|hpe-vsxd|7011|LOG_INFO|AMM|-|VSX 30 state local up, remote down
2019-09-24:17:33:49.295089|lldpd|104|LOG_INFO|AMM|-|LLDP neighbor e4:43:4b:7a:f9:12 added on 1/1/33
2019-09-24:17:34:25.163042|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up

    So...in the end...the setup I've done looks correct from a pure configuration standpoint, at least looking at what Aruba nodes' GUIs report...BUT the usual command show lacp interface multi-chassis apparently fails to report lag30(mc) status correctly.

     

    At this point my main question is:

     

    Provided that one want to deploy a VSX Static LAG because no LAG LACP is supported by downlinked peer (here the ESXi)...is there on ArubaOS-CX 10.2/10.3 an equivalent to the well known ArubaOS-Switch's show trunks in order to diagnose the VSX Static LAG status since the show lacp interface multi-chassis (exactly because is heavily LACP biased?) seems to refuse to report anything about a Multi-Chassis Static LAG? or...Am I missing some other documented alternative about how to diagnose a VSX Static LAG? forgot to specify I'm on ArubaOS-CX 10.02.0031...not yet on 10.03.



  • 9.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Sep 25, 2019 11:00 AM

    With a Static VSX LAG (lag30(mc) and lag31(mc) in my scenario), it looks like the output of show lacp interfaces multi-chassis (redacted below to show only lag30(mc) and lag31(mc)) is inferior (in terms of information provided) if compared to the output of the command show lacp interfaces (redacted below to show only lag30(mc) and lag31(mc)) typically used in case of non-VSX LAGs!

     

    Documentation:

    Displaying_Static_non-VSX_LAG.png

     

    ..so, at this point, it looks like that using a command born to display non-VSX Static LAGs' status is the only (and best) way we can use to understand if a Static VSX LAG is (or is not) in Forwarding State (See above the "Up" on the last column).

     

    Info reported by the command one think it should not be the most appropriate:

     

    Aruba-8320-1# show lacp interfaces 

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
        Name       Id    Pri                             Pri    Key  State     
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                    up        
1/1/34  lag31(mc)                                                    up        

Partner details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggr       Port  Port  State   System-ID         System Aggr           
        Name       Id    Pri                             Pri    Key            
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                              
1/1/34  lag31(mc)

    Versus no info provided by the command one think it should be the most appropriate:

     

    Aruba-8320-1# show lacp interfaces multi-chassis 

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggregate  Port    Port     State   System-ID         System   Aggr    
        name       id      Priority                           Priority Key     
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                              
1/1/34  lag31(mc)                                                              


Partner details of all interfaces:
------------------------------------------------------------------------------
Intf    Aggregate  Partner Port     State   System-ID         System   Aggr    
        name       Port-id Priority                           Priority Key     
------------------------------------------------------------------------------
1/1/33  lag30(mc)                                                              
1/1/34  lag31(mc)                                                              


Remote Actor details of all interfaces:
------------------------------------------------------------------------------
Intf   Aggregate  Port    Port     State   System-ID         System   Aggr    
       name       id      Priority                           Priority Key     
------------------------------------------------------------------------------
1/1/33 lag30(mc)                                                              
1/1/34 lag31(mc)                                                              


Remote Partner details of all interfaces:
------------------------------------------------------------------------------
Intf   Aggregate  Partner Port     State   System-ID         System   Aggr    
       name       Port-id Priority                           Priority Key     
------------------------------------------------------------------------------
1/1/33 lag30(mc)                                                              
1/1/34 lag31(mc)

    I think it's crystal clear at this point that this CLI approach is very counter-intuitive because - primarily - Static LAGs can be implemented as VSX LAGs other than as non-VSX LAGs...so the proper command to check their status - having lag(mc) in our VSX scenario - must be the show lacp interfaces multi-chassis...and not the show lacp interfaces command.

     

    In any case those show lacp based commands are (and were, since very beginning), IMHO, all way too counter-inutitive for the task (the grammar used is counter-intuitive[*]).

     

    Why? because:

     

    • To check a non-VSX LAGs status (Static or LACP, it doesn't matter) a user should use a command such show lag [<lag-interface-id>|<all>] eventually with options.
    • To check a VSX LAG status (Static or LACP, it doesn't matter) a user should use a command such show vsx lag [<lag-interface-id>|<all>] eventually with options.
    • The differentiating factor is (and always have been) - at least in a ArubaOS-CX driven environment - only the fact that a LAG could be implemented as a non-VSX LAG or that it could be implemented as a VSX LAG (clearly is reasonable considering a VSX scenario) and not that a LAG is consequently using Static (Non Control Protocol) or LACP.

    [*] Simply consider that the show lacp interfaces command actually - at minimum - shows:

     

    • VSX LACP LAGs = VSX LAGs using LACP
    • VSX Static LAGs = VSX LAGs not using LACP
    • non-VSX LACP LAGs = indeed LACP LAG used for VSX ISL is listed

    Sorry for being so long...I started this thread requesting an opinion about VSX Static LAG coupling with vSphere VSS Port Aggregation...and ended opinioning about ArubaOS-CX grammar and some of its counter-intuitive commands...but this was simply discovered along the way...and I just pick up the challenge.

     

    I think a similar strange behaviour should be checked (and, if possible, addressed) by Product Development...personally I found that the ArubaOS-CX CLI is too raw on specific areas...in other terms IMHO there is room for improvements.



  • 10.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    EMPLOYEE
    Posted Oct 06, 2020 02:42 PM

    Hi Davide,

     

    I have got a similar setup with one of my customers, may I know if you have tested the connectivity from both VSX primary and secondary switch. 

    In my case, only one link was passing the traffic at a single point of time.

    We tried to ping from VM host to both the IPs on primary and secondary but only one was alive. If we disable the primary switch link other was coming up.

    Please let me know if you have found a similar behavior.

     

    Regards,

    Sahil Raina



  • 11.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Oct 06, 2020 03:02 PM

    Hi Sahil Raina, in my scenario pings flow from a Guest OS of a VM hosted on a ESXi node and tied to a particular tagged VLAN (Port Group with a Tagged VLAN id) to:

     

    • VSX-1 SVI Address for that VLAN
    • VSX-2 SVI Address for that VLAN
    • VSX Virtual Gateway SVI Address for that VLAN

    and vice-versa from both VSX-1 node and VSX-2 node to the IP of that Guest OS.

     

    I suppose that when I ping from VM to VSX-1 SVI (.251) or VSX-2 SVI (.252) the packets flow directly to respectively VSX-1 or VSX-2 (and not to VSX-1 or 2 and traversing the ISL to reach 2 or 1)...in other terms I believe traffic uses both links (both links are up).

     

    The physical interfaces used on the ESXi side are aggregated in static mode (no LACP) and so were set the corresponding interfaces of the VSX LAG (Static) on Aruba side.

     

    Just for reference, below my VSX LAG 30 (Static) settings:

     

     

    interface lag 30 multi-chassis static
   description 8320-1-VSX-LAG-Static-XXXXXXX-esxi01
   no shutdown
   no routing
    vlan trunk native 1
    vlan trunk allowed 1,1000,1050,1100

vsx-1# show running-config interface 1/1/33
interface 1/1/33 
    no shutdown 
    mtu 9198 
    flow-control rx
    description DELL_R740_XXXXXXX_s1p2_esxi01_vmnic1_vSwitch2
    lag 30
    exit

vsx-1# show running-config interface 1/1/33 vsx-peer
interface 1/1/33 
    no shutdown 
    mtu 9198 
    flow-control rx
    description DELL_R740_XXXXXXX_s3p2_esxi01_vmnic7_vSwitch2
    lag 30
    exit

     

     

    vSwitch2 is a VSS on ESXi01 made of two 10Gbps SFP+ links (originating on Port 2 of NIC in Slot 1 Port 2 and Port 2 of NIC on Slot 3).

     

    This vSwitch2 has four VLANs (one Untagged with VLAN id 1 and three VLAN Id(s) Tagged, 1000, 1050 and 1100) which mean four Port Groups (one with VLAN id 1 Untagged and three with VLAN Id(s) Tagged).

     

    The VM I referred above is hosted on VLAN 1000 (it has IP 10.254.0.1 and it is able to ping the 10.254.0.251 of VSX-1 SVI on VLAN 1000, .252 of VSX-2 SVI on VLAN 1000 and .254 Active Gateway for VLAN 1000).



  • 12.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    EMPLOYEE
    Posted Oct 07, 2020 04:34 AM

    Thank You so much that answers my question in my case LACP is showing block on the switch side we have a similar config so, with your explanation and confirmation I am suspecting he is doing something wrong on VM side. 

    Would you mind to share your VM side config and setup as well.

     

    Regards,

    Sahil Raina 



  • 13.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?
    Best Answer

    MVP GURU
    Posted Oct 07, 2020 10:36 AM

    Hi! VM side - since the Guest OS running on that VM is totally VLAN unaware - the network is configured as usual (IP, Subnet Mask, Default Gateway, etc. ...nothing less nothing more).

     

    The tricky part is on the VSS side - I have a dedicated vSwitch, the vSwitch2 - on the ESXi:

     

     

    Properties
Standard switch 	        vSwitch2
MTU 	                        9000

Security
Promiscuous mode 	        Reject
MAC address changes 	        Accept
Forged transmits 	        Accept

Traffic shaping
Average bandwidth 	        --
Peak bandwidth 	                --
Burst size 	                --

Teaming and failover
Load balancing 	                Route based on IP hash
Network failure detection 	Link status only
Notify switches 	        Yes
Failback 	                Yes
Active adapters 	        vmnic1, vmnic7
Standby adapters 	        --
Unused adapters 	        --

notes:
In the IP hash load balancing policy all physical switch ports connected to the active uplinks must be in link aggregation mode.

IP hash load balancing should be set for all port groups using the same set of uplinks.

     

     

    Please note that VSS (Standard vSwitch) doesn't support LACP as Link Aggregation Control Protocol...this means that, like me, you are dealing with a VSS then you can only aggregate physical links of your ESXi node by using what it's called a Non-Protocol trunking method (I prefer this description to highlight the fact this aggregation doesn't use a protocol between involved peers' logical interfaces)...and this is done exactly by using the "Route based on IP hash" as Load Balancing method under the Teaming and Failover. Acting that way is like to form what is known as a static channel-group (Cisco, for example, calls this a static-EtherChannel, HP - now Aruba - on ProVision/ArubaOS-Switch OS based switches calls it simply a port trunk operating in trunk mode instead of lacp mode). So no LACP here (at least until you can deploy a VDS Virtual Distributed Switch).

     

    On the Port Group simply define the VLAN id you want to be tagged (incoming tag accepted and outgoing tagged) and this need to cope the settings on the VSX Static LAG on the Aruba VSX side (clearly there you pass more VLAN tags, not just one).

     

    If you are so lucky to work with a VDS then you can setup a Team with LACP (IEEE 802.3ad) and thus use VSX LAG with LACP...with VSS you can't...I never tried to setup Teams with LACP as Load Balancing control protocol on ESXi vSwitches due to licensing restrictions of the ESXi editions I work with.



  • 14.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?
    Best Answer

    EMPLOYEE
    Posted Oct 08, 2020 12:26 AM

    Thank you so much for sharing the server config, i was able to fix the problem as in our case one NIC was showing active, and the other was on standby mode.

     

    Regards,

    Sahil Raina



  • 15.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Oct 08, 2020 04:41 AM

    Hi! Glad you solved!



  • 16.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Apr 13, 2021 09:23 AM
    Thanks Parnassus for sharing this.
    Did you try testing Split brain senarios ? Will the secondary bring down the STATIC VSX LAG members just as it would for the dynamic lag members ?

    Regards.

    ------------------------------
    Achraf Harkati
    ------------------------------

    Original Message


  • 17.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 02:56 AM
    I have come across this thread after running into the same problem on a similar setup (2xAruba8302 in VSX running 10.02 and 1 VMware with 2x10Gbps NIC's in teaming only - no LACP license).
    Before going any further with this, I would like to ask you how did you manage to create Static Mclag's, because you are not allowed to do that - by design...at least on my equipments, when creating a mclag, it will be automatically lacp enabled without the possibility to turn this off.

    ------------------------------
    Alex Vrabie
    ------------------------------

    Original Message


  • 18.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Dec 04, 2020 07:45 AM
    Hi @darthy_sanchez, "static" VSX LAGs (non LACP to be clear) were supported since the ArubaOS-CX 10.2.0001 software release, so since the very first public build of the ArubaOS-CX 10.2 software branch (the name MCLAG belongs to the old nomenclature...legacy of ArubaOS-CX 10.0​ era IIRC).

    I did exactly as written above in this thread.

    I don't recall exactly if I deployed that configuration when the ArubaOS-CX was already at 10.3 or when our VSX was still running ArubaOS-CX 10.2 (during September last year we upgraded from 10.2 to 10.3...in any case I'm quite sure we weren't running a too old ArubaOS-CX 10.02 at that time...probably 10.02.031 or so)...as I documented above the setup is working quite nicely.

    ------------------------------
    Davide Poletto
    ------------------------------

    Original Message


  • 19.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 08:25 AM
    I have somehow managed to find out how to do that: simply by issuing the command: "interface lag 100 multi-chasis static", with "static" at the end of the line. That's what I wanted to know...Tx anyway

    ------------------------------
    Alex Vrabie
    ------------------------------

    Original Message


  • 20.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Dec 04, 2020 08:52 AM
    Correct, it was cited above...pretty simple. Glad you found it.

    ------------------------------
    Davide Poletto
    ------------------------------

    Original Message


  • 21.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 12:02 PM
    Yep, just saw it now! sorry..didn't read through the whole thread line ny line, as this was my only real problem here! static or lacp, my lags behave the same (balance or failover wise)! Cheers!

    ------------------------------
    Alex Vrabie
    ------------------------------

    Original Message


  • 22.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    EMPLOYEE
    Posted Apr 15, 2021 04:10 AM
    Can you share in this thread what NIC teaming setting to you use on ESXi server ? That could be useful for others for later reference.

    ------------------------------
    Vincent Giles
    ------------------------------

    Original Message


  • 23.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Mar 31, 2022 11:53 AM
    Hello,

    We have try static lag with Virtual switch standard and we are having somes issues.
    Lag seems to be up but no mac address are learned on the lag., half of the ping are lost. Arp is working but with no port associated to the mac address.

    Does anyone have sucessfully configured static lag  with Virtual switch standard in vmware.

    Thanks for your help

    ------------------------------
    Romain Ricaud
    ------------------------------

    Original Message


  • 24.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Mar 31, 2022 11:58 AM
    Hello,

    The configuration is a classic 2 ports aggregate 
    We have followed this one :  https://kb.vmware.com/s/article/1004048

    We had a session with Aruba TAC and we will try to upgrade to see if it fix the issue our current version is 10.08.1021
    More to follow

    ------------------------------
    Romain Ricaud
    ------------------------------

    Original Message


  • 25.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    EMPLOYEE
    Posted Apr 01, 2022 09:39 AM
    Not sure to understand why you refer to static VSX LAG as you mentioned https://kb.vmware.com/s/article/1004048
    which is all about LACP.
    If you have vds and LACP, just use interface lag1 multi-chassis without static option.
    LACP on VSX and LACP on vds works for sure, most customers use this config.

    ------------------------------
    Vincent Giles
    ------------------------------

    Original Message


  • 26.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    MVP GURU
    Posted Mar 31, 2022 01:49 PM
    Hello Romain,

    "Does anyone have sucessfully configured static lag  with Virtual switch standard in vmware."

    Yes, we did: VSX non-protocol LAGs against two ESXi nodes running on VMware 6.7 U3 with VSS (Route based on IP Hash as Load Balancing method).




    Original Message