Wired

last person joined: 18 hours ago 

Bring performance and reliability to your network with the Aruba Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of the ArubaOS-Switch and ArubaOS-CX devices, and find ways to improve security across your network to bring together a mobile first solution.
Expand all | Collapse all

ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

Jump to Best Answer
This thread has been viewed 51 times
  • 1.  ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 18, 2019 11:45 AM

    Hi all,

     

    I'm interested in learning opinions/pro/cons about using static (so non LACP) VSX LAG (available since ArubaOS-CX 10.02 and also on latest ArubaOS-CX 10.03) against VMware vSphere host's bonded ports (the vSphere host I'm referring to will use VSS Virtual Standard Switch - not VDS Virtual Distributed Switch - so usage of IEEE 803.3ad LACP is not possible and we need to rely on static Non-Protocol Ports Aggregation).

     

    Thanks for sharing, Davide.



  • 2.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 07:31 AM

    When the vmware switches are running in switch independent mode, i would not built in any "ïntelligence" into the network, vmware load-balances per vm on the connected interfaces will do the trick in that cases.

     

    If 1 VM would require more bandwith then 1 interface can offer, then i would implement LACP on vmware (MC-LAG on the core).



  • 3.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 08:07 AM

    Hi, thank you for answering...that's basically true...but:

     

    (1) the balancing is valid for traffic leaving the ESXi host (considering the physical ports connected to the considered vSwitch configured in Active/Active mode)...what happen to incoming traffic to ESXi host on that very same vSwitch if no aggregation (neither LACP nor Static) is deployed?

     

    (2) LACP is supported only on vNetwork with VDS Virtual Distributed Switch...which we can't actually implement (we haven't the supporting licensing for implementing VDS so we work only with VSS Virtual Standard Switch which doesn't support LACP but only static port aggregation without any protocol).

     

    I'm worried to understand if there are pro/cons in using static port trunking on VSS to VSX (with VSX LAG static as per AOS-CX 10.02/10.03) instead of simply using two active/active ports on VSS to VSX.

     

    Thanks!



  • 4.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 08:57 AM

    (1) For vmware load balanced, the incoming traffic wil be received on the same port for that particular vm. So always per-vm load balancing.

     

    (2) I'm not sure what type of load balancing mechanism is used when using static port aggregation, does it even load balance accros more then 1 interface, or is active-standby?

     

    The vmware load-balancing is pretty solid, but load-balancing on per-vm basis is not the most effective way IMO.



  • 5.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 20, 2019 10:32 AM

    Hi!

     

    (1) OK...still I don't understand if a Static LAG (so in a scenario, as you said, with some "intelligence" on Host's physical network ports side and on the peer Switch side) is better or worst than leaving the vSphere Host's physical network ports in Active/Active and letting the vSwitch to balance egress traffic (generated by VM) to physical network...is using Static LAG overcomplicating the scenario (considering that we have VSX as peer) or not?

     

    (2) As far as I understood Static LAG member ports are, once aggregated, all partecipating but, with regards to Static LAG versus LACP LAP, the difference is that Static LAG has no control protocol to manage member ports status and relationship with peer's LACP LAG (so this explain the term "Non Protocol" used by HP/HPE)...this main difference means less control-related features (like no Failover, no Hot-Standby port, no canditate aggregable port configuration verification...illuminating blog articles here and here)...the interesting point is that Static LAG versus LACP LAG doesn't necessarily mean that load balancing algorithms are different in favour of LACP LAG (they are, indeed, the same)...it's just that vSphere requires specifically to use "Route based on IP Hash" with Static LAG (so operating on VSS)...which basically means using the well known "src-dst IP" load balancing algorithm (as it often happens using LACP LAG).



  • 6.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 24, 2019 06:41 PM

    for me avoid static LAG (and also LACP LAG) with VMware ESXi (or only for very specific use case...)



  • 7.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 24, 2019 06:54 PM
    Hi Alexis, thanks! probably I'll step down(or not?) on multichassis static (VSX LAG Static) until I understand how to check if my setup is working or not...I tried yesterday with a 2 10Gbps ports member of a multichassis static (1 port on each VSX Member) against a ESXi 6.7 U3 set with 2 10 Gbps ports in Load Balancing (Route Based on IP Hash)...but, despite all involved ports are up, apparently ArubaOS-CX doesn't provide any feedback via the show lacp interface multi-chassis command I'm used to: I see the lag30(mc) line (local/remote) - lag 30 is my ststic one - totally empty (is this expected because it is not an usual LACP one?)...so, at first, I was a little bit in doubt about what I've set...show events on both peers reports that involved interfaces are up and lag30 is formed...also ESXi seems good on that. Today I'll check it better, I was too much in a hurry yesterday.


  • 8.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 25, 2019 06:14 AM

    The more I dive deeply into VSX Static LAG the more this configuration become interesting from the management standpoint...below my findings about VSX diagnostic done against my configured lag30 and its member interfaces 1/1/33.

     

    ArubaOS-CX GUI reports the VSX (Multi-Chassis) Static LAG lag30 as Up (both interfaces are Up on both nodes):

     

    ESXi host reported Up state for involved NICs' ports:

    ESXi01_s1p2_and_s3p2_connected_to_VSX.png

     

     

    ESXi vSwitch2 used to test VSX Static LAG (lag30):

     

    Properties
    
    Network label			DF Backend (1)
    VLAN ID				None (0)
    
    Security
    
    Promiscuous mode		Reject
    MAC address changes 		Accept
    Forged transmits		Accept
    
    Traffic shaping
    
    Average bandwidth		--
    Peak bandwidth			--
    Burst size			--
    
    Teaming and failover
    
    Load balancing			Route based on IP hash
    
    INFO: In the IP hash load balancing policy all physical switch ports connected to the active uplinks must be in link aggregation mode.
    INFO: IP hash load balancing should be set for all port groups using the same set of uplinks.
    
    Network failure detection	Link status only
    Notify switches			Yes
    Failback			Yes
    Active adapters			vmnic1, vmnic7
    Standby adapters		--
    Unused adapters			--

    And visually:ESXi01_vSwitch2_connected_to_VSX.png

    Aruba 8320 VSX Primary VSX Static LAG (lag30) status:

    VSX_Static_LAG_lag30_Aruba_8320-1_Primary_01.png

    VSX_Static_LAG_lag30_Aruba_8320-1_Primary_02.png

     

    Aruba 8320 VSX Secondary VSX Static LAG (lag30) status:VSX_Static_LAG_lag30_Aruba_8320-2_Secondary_01.png

    VSX_Static_LAG_lag30_Aruba_8320-2_Secondary_02.png

     

    Now the CLI part:

     

    === Aruba 8320 Node 1 (Primary) VSX Static LAG configuration ===
    
    interface lag 30 multi-chassis static
        vsx-sync vlans
        description 8320-1-VSX-LAG-Static-lag30-ESXi01
        no shutdown
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,20,30,40
        loop-protect
        loop-protect vlan 1,20,30,40
    
    interface 1/1/33 
        description Leg_to_s1p2_ESXi01_vmnic1_vSwitch2
        no shutdown
        mtu 9198
        flow-control rx
        lag 30
    
    === Aruba 8320 Node 2 (Secondary) VSX Static LAG configuration ===
    interface lag 30 multi-chassis static
        vsx-sync vlans
        description 8320-2-VSX-LAG-Static-lag30-ESXi01
        no shutdown
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,20,30,40
        loop-protect
        loop-protect vlan 1,20,30,40
    
    interface 1/1/33 
        description Leg_to_s3p2_ESXi01_vmnic7_vSwitch2
        no shutdown
        mtu 9198
        flow-control rx
        lag 30
    
    === Aruba 8320 VSX Static LAG (lag30) diagnostic on Primary and Secondary ===
    Aruba-8320-1# show lacp aggregates
    
    Aggregate name   : lag30 (multi-chassis)             
    Interfaces       : 1/1/33                            
    Peer interfaces  : 1/1/33                            
    Heartbeat rate   : N/A                               
    Hash             : l3-src-dst                        
    Aggregate mode   : Off
    
    Aruba-8320-1# show lacp aggregates vsx-peer
    
    Aggregate name   : lag30 (multi-chassis)             
    Interfaces       : 1/1/33                            
    Peer interfaces  : 1/1/33                            
    Heartbeat rate   : N/A                               
    Hash             : l3-src-dst                        
    Aggregate mode   : Off
    
    Aruba-8320-1# show vsx lacp aggregates lag30   
                        Local-peer                        Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate name   : lag30 (multi-chassis)             lag30 (multi-chassis)
    Interfaces       : 1/1/33                            1/1/33 
    Peer interfaces  : 1/1/33                            1/1/33 
    Heartbeat rate   : N/A                               N/A
    Hash             : l3-src-dst                        l3-src-dst
    Aggregate mode   : Off                               Off
    
    Aruba-8320-2# show vsx lacp aggregates lag30
                        Local-peer                        Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate name   : lag30 (multi-chassis)             lag30 (multi-chassis)
    Interfaces       : 1/1/33                            1/1/33 
    Peer interfaces  : 1/1/33                            1/1/33 
    Heartbeat rate   : N/A                               N/A
    Hash             : l3-src-dst                        l3-src-dst
    Aggregate mode   : Off                               Off
    
    Aruba-8320-1# show lacp interfaces
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
            Name       Id    Pri                             Pri    Key  State     
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                    up        
    
    Partner details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr           
            Name       Id    Pri                             Pri    Key            
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                              
    
    Aruba-8320-1# show lacp interfaces vsx-peer 
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
            Name       Id    Pri                             Pri    Key  State     
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                    up        
    
    Partner details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr           
            Name       Id    Pri                             Pri    Key            
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                              

    And configuration related events generated during lag30 setup yesterday (Errors on VSX Secondary should be probably related to the fact I was saving VSX Primary before lag30 was configured on VSX Secondary):

     

    === Aruba 8320 Node 1 (Primary) VSX Static LAG events related to 1/1/33 and lag30 configuration ===
    
    2019-09-24:16:43:17.115485|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/33 is deleted from router discovery
    2019-09-24:16:47:17.218517|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/34 is deleted from router discovery
    2019-09-24:17:03:11.653192|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:10:02.902801|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:16:59.661158|lacpd|1303|LOG_INFO|AMM|-|Interface 1/1/33 added to LAG 30. Existing configuration on interface 1/1/33 will be removed.
    2019-09-24:17:16:59.696098|ops-switchd|4803|LOG_INFO|AMM|-|All dynamic MAC addresses on port 1/1/33 were flushed
    2019-09-24:17:16:59.699173|ops-switchd|1401|LOG_DEBUG|AMM|-|Trunk set succeeds unit 0 lag_id 4
    2019-09-24:17:16:59.699216|ops-switchd|1301|LOG_INFO|AMM|-|Dynamic LAG lag30 created
    2019-09-24:17:16:59.699255|ops-switchd|1409|LOG_INFO|AMM|-|LAG lag30, set to load balance mode to l3-src-dst-hash
    2019-09-24:17:16:59.701674|ops-switchd|1404|LOG_DEBUG|AMM|-|Trunk member add port succeeds on unit 0 hw_port 33 tid 4
    2019-09-24:17:17:35.273114|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
    2019-09-24:17:17:44.802017|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:29:02.232570|pmd|3804|LOG_INFO|AMM|-|Transceiver hot-swap insert for interface 1/1/33
    2019-09-24:17:33:40.949961|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
    2019-09-24:17:33:41.026283|hpe-vsxd|7012|LOG_INFO|AMM|-|VSX 30 state local down, remote up
    2019-09-24:17:34:25.107870|lldpd|110|LOG_INFO|AMM|-|Configured LLDP reinit-delay to 2
    2019-09-24:17:34:25.124641|intfd|403|LOG_INFO|AMM|-|Link status for interface 1/1/33 is up
    2019-09-24:17:34:25.152520|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up
    2019-09-24:17:34:25.156592|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up
    2019-09-24:17:34:33.320050|lldpd|104|LOG_INFO|AMM|-|LLDP neighbor f8:f2:1e:59:e7:91 added on 1/1/33
    
    === Aruba 8320 Node 2 (Secondary) VSX Static LAG events related to 1/1/33 and lag30 configuration ===
    
    2019-09-24:16:50:15.097019|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/33 is deleted from router discovery
    2019-09-24:17:03:00.228386|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:03:16.174430|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on primary VSX device.
    2019-09-24:17:03:16.580494|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:03:23.817133|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
    2019-09-24:17:07:00.282175|hpe-rdiscd|3910|LOG_INFO|AMM|-|Interface: 1/1/34 is deleted from router discovery
    2019-09-24:17:09:52.080474|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:10:07.285763|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on primary VSX device.
    2019-09-24:17:10:07.683633|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:10:14.894096|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
    2019-09-24:17:16:39.026350|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
    2019-09-24:17:16:39.218306|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
    2019-09-24:17:16:43.328284|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
    2019-09-24:17:16:43.527611|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
    2019-09-24:17:16:47.196342|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : Missing Reference Error: Could not find Port lag30 on secondary VSX database.
    2019-09-24:17:16:47.507171|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
    2019-09-24:17:17:03.590207|vsx-syncd|7601|LOG_ERR|AMM|-|Configuration sync error : Missing reference in database while syncing configuration. Use "show running-config vsx-sync peer-diff" to help identify the missing reference.
    2019-09-24:17:17:19.850505|hpe-vsxd|7014|LOG_INFO|AMM|-|VSX 30 state local down, remote down
    2019-09-24:17:17:22.585726|vsx-syncd|7602|LOG_INFO|AMM|-|Configuration sync update : VSX configuration-sync updated database
    2019-09-24:17:17:35.231058|lacpd|1303|LOG_INFO|AMM|-|Interface 1/1/33 added to LAG 30. Existing configuration on interface 1/1/33 will be removed.
    2019-09-24:17:17:35.327042|ops-switchd|4803|LOG_INFO|AMM|-|All dynamic MAC addresses on port 1/1/33 were flushed
    2019-09-24:17:17:35.332017|ops-switchd|1401|LOG_DEBUG|AMM|-|Trunk set succeeds unit 0 lag_id 6
    2019-09-24:17:17:35.332083|ops-switchd|1301|LOG_INFO|AMM|-|Dynamic LAG lag30 created
    2019-09-24:17:17:35.332150|ops-switchd|1409|LOG_INFO|AMM|-|LAG lag30, set to load balance mode to l3-src-dst-hash
    2019-09-24:17:17:35.335095|ops-switchd|1404|LOG_DEBUG|AMM|-|Trunk member add port succeeds on unit 0 hw_port 33 tid 6
    2019-09-24:17:17:40.764550|hpe-config|6801|LOG_INFO|AMM|-|Copying configs from: running-config to: startup-config
    2019-09-24:17:17:56.876722|vsx-syncd|7603|LOG_INFO|AMM|-|Configuration-persistence : configuration saved to startup-configuration on secondary VSX device.
    2019-09-24:17:29:22.147653|pmd|3804|LOG_INFO|AMM|-|Transceiver hot-swap insert for interface 1/1/33
    2019-09-24:17:33:40.963502|lldpd|110|LOG_INFO|AMM|-|Configured LLDP reinit-delay to 2
    2019-09-24:17:33:40.979534|intfd|403|LOG_INFO|AMM|-|Link status for interface 1/1/33 is up
    2019-09-24:17:33:41.020537|hpe-vsxd|7011|LOG_INFO|AMM|-|VSX 30 state local up, remote down
    2019-09-24:17:33:41.029885|hpe-vsxd|7011|LOG_INFO|AMM|-|VSX 30 state local up, remote down
    2019-09-24:17:33:49.295089|lldpd|104|LOG_INFO|AMM|-|LLDP neighbor e4:43:4b:7a:f9:12 added on 1/1/33
    2019-09-24:17:34:25.163042|hpe-vsxd|7013|LOG_INFO|AMM|-|VSX 30 state local up, remote up

    So...in the end...the setup I've done looks correct from a pure configuration standpoint, at least looking at what Aruba nodes' GUIs report...BUT the usual command show lacp interface multi-chassis apparently fails to report lag30(mc) status correctly.

     

    At this point my main question is:

     

    Provided that one want to deploy a VSX Static LAG because no LAG LACP is supported by downlinked peer (here the ESXi)...is there on ArubaOS-CX 10.2/10.3 an equivalent to the well known ArubaOS-Switch's show trunks in order to diagnose the VSX Static LAG status since the show lacp interface multi-chassis (exactly because is heavily LACP biased?) seems to refuse to report anything about a Multi-Chassis Static LAG? or...Am I missing some other documented alternative about how to diagnose a VSX Static LAG? forgot to specify I'm on ArubaOS-CX 10.02.0031...not yet on 10.03.



  • 9.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Sep 25, 2019 11:00 AM

    With a Static VSX LAG (lag30(mc) and lag31(mc) in my scenario), it looks like the output of show lacp interfaces multi-chassis (redacted below to show only lag30(mc) and lag31(mc)) is inferior (in terms of information provided) if compared to the output of the command show lacp interfaces (redacted below to show only lag30(mc) and lag31(mc)) typically used in case of non-VSX LAGs!

     

    Documentation:

    Displaying_Static_non-VSX_LAG.png

     

    ..so, at this point, it looks like that using a command born to display non-VSX Static LAGs' status is the only (and best) way we can use to understand if a Static VSX LAG is (or is not) in Forwarding State (See above the "Up" on the last column).

     

    Info reported by the command one think it should not be the most appropriate:

     

    Aruba-8320-1# show lacp interfaces 
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr Forwarding
            Name       Id    Pri                             Pri    Key  State     
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                    up        
    1/1/34  lag31(mc)                                                    up        
    
    Partner details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggr       Port  Port  State   System-ID         System Aggr           
            Name       Id    Pri                             Pri    Key            
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                              
    1/1/34  lag31(mc)                                                              

    Versus no info provided by the command one think it should be the most appropriate:

     

    Aruba-8320-1# show lacp interfaces multi-chassis 
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggregate  Port    Port     State   System-ID         System   Aggr    
            name       id      Priority                           Priority Key     
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                              
    1/1/34  lag31(mc)                                                              
    
    
    Partner details of all interfaces:
    ------------------------------------------------------------------------------
    Intf    Aggregate  Partner Port     State   System-ID         System   Aggr    
            name       Port-id Priority                           Priority Key     
    ------------------------------------------------------------------------------
    1/1/33  lag30(mc)                                                              
    1/1/34  lag31(mc)                                                              
    
    
    Remote Actor details of all interfaces:
    ------------------------------------------------------------------------------
    Intf   Aggregate  Port    Port     State   System-ID         System   Aggr    
           name       id      Priority                           Priority Key     
    ------------------------------------------------------------------------------
    1/1/33 lag30(mc)                                                              
    1/1/34 lag31(mc)                                                              
    
    
    Remote Partner details of all interfaces:
    ------------------------------------------------------------------------------
    Intf   Aggregate  Partner Port     State   System-ID         System   Aggr    
           name       Port-id Priority                           Priority Key     
    ------------------------------------------------------------------------------
    1/1/33 lag30(mc)                                                              
    1/1/34 lag31(mc)

    I think it's crystal clear at this point that this CLI approach is very counter-intuitive because - primarily - Static LAGs can be implemented as VSX LAGs other than as non-VSX LAGs...so the proper command to check their status - having lag(mc) in our VSX scenario - must be the show lacp interfaces multi-chassis...and not the show lacp interfaces command.

     

    In any case those show lacp based commands are (and were, since very beginning), IMHO, all way too counter-inutitive for the task (the grammar used is counter-intuitive[*]).

     

    Why? because:

     

    • To check a non-VSX LAGs status (Static or LACP, it doesn't matter) a user should use a command such show lag [<lag-interface-id>|<all>] eventually with options.
    • To check a VSX LAG status (Static or LACP, it doesn't matter) a user should use a command such show vsx lag [<lag-interface-id>|<all>] eventually with options.
    • The differentiating factor is (and always have been) - at least in a ArubaOS-CX driven environment - only the fact that a LAG could be implemented as a non-VSX LAG or that it could be implemented as a VSX LAG (clearly is reasonable considering a VSX scenario) and not that a LAG is consequently using Static (Non Control Protocol) or LACP.

    [*] Simply consider that the show lacp interfaces command actually - at minimum - shows:

     

    • VSX LACP LAGs = VSX LAGs using LACP
    • VSX Static LAGs = VSX LAGs not using LACP
    • non-VSX LACP LAGs = indeed LACP LAG used for VSX ISL is listed

    Sorry for being so long...I started this thread requesting an opinion about VSX Static LAG coupling with vSphere VSS Port Aggregation...and ended opinioning about ArubaOS-CX grammar and some of its counter-intuitive commands...but this was simply discovered along the way...and I just pick up the challenge.

     

    I think a similar strange behaviour should be checked (and, if possible, addressed) by Product Development...personally I found that the ArubaOS-CX CLI is too raw on specific areas...in other terms IMHO there is room for improvements.



  • 10.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Oct 06, 2020 02:42 PM

    Hi Davide,

     

    I have got a similar setup with one of my customers, may I know if you have tested the connectivity from both VSX primary and secondary switch. 

    In my case, only one link was passing the traffic at a single point of time.

    We tried to ping from VM host to both the IPs on primary and secondary but only one was alive. If we disable the primary switch link other was coming up.

    Please let me know if you have found a similar behavior.

     

    Regards,

    Sahil Raina



  • 11.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Oct 06, 2020 03:02 PM

    Hi Sahil Raina, in my scenario pings flow from a Guest OS of a VM hosted on a ESXi node and tied to a particular tagged VLAN (Port Group with a Tagged VLAN id) to:

     

    • VSX-1 SVI Address for that VLAN
    • VSX-2 SVI Address for that VLAN
    • VSX Virtual Gateway SVI Address for that VLAN

    and vice-versa from both VSX-1 node and VSX-2 node to the IP of that Guest OS.

     

    I suppose that when I ping from VM to VSX-1 SVI (.251) or VSX-2 SVI (.252) the packets flow directly to respectively VSX-1 or VSX-2 (and not to VSX-1 or 2 and traversing the ISL to reach 2 or 1)...in other terms I believe traffic uses both links (both links are up).

     

    The physical interfaces used on the ESXi side are aggregated in static mode (no LACP) and so were set the corresponding interfaces of the VSX LAG (Static) on Aruba side.

     

    Just for reference, below my VSX LAG 30 (Static) settings:

     

     

    interface lag 30 multi-chassis static
       description 8320-1-VSX-LAG-Static-XXXXXXX-esxi01
       no shutdown
       no routing
        vlan trunk native 1
        vlan trunk allowed 1,1000,1050,1100
    
    vsx-1# show running-config interface 1/1/33
    interface 1/1/33 
        no shutdown 
        mtu 9198 
        flow-control rx
        description DELL_R740_XXXXXXX_s1p2_esxi01_vmnic1_vSwitch2
        lag 30
        exit
    
    vsx-1# show running-config interface 1/1/33 vsx-peer
    interface 1/1/33 
        no shutdown 
        mtu 9198 
        flow-control rx
        description DELL_R740_XXXXXXX_s3p2_esxi01_vmnic7_vSwitch2
        lag 30
        exit

     

     

    vSwitch2 is a VSS on ESXi01 made of two 10Gbps SFP+ links (originating on Port 2 of NIC in Slot 1 Port 2 and Port 2 of NIC on Slot 3).

     

    This vSwitch2 has four VLANs (one Untagged with VLAN id 1 and three VLAN Id(s) Tagged, 1000, 1050 and 1100) which mean four Port Groups (one with VLAN id 1 Untagged and three with VLAN Id(s) Tagged).

     

    The VM I referred above is hosted on VLAN 1000 (it has IP 10.254.0.1 and it is able to ping the 10.254.0.251 of VSX-1 SVI on VLAN 1000, .252 of VSX-2 SVI on VLAN 1000 and .254 Active Gateway for VLAN 1000).



  • 12.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Oct 07, 2020 04:34 AM

    Thank You so much that answers my question in my case LACP is showing block on the switch side we have a similar config so, with your explanation and confirmation I am suspecting he is doing something wrong on VM side. 

    Would you mind to share your VM side config and setup as well.

     

    Regards,

    Sahil Raina 



  • 13.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?
    Best Answer

    Posted Oct 07, 2020 10:36 AM

    Hi! VM side - since the Guest OS running on that VM is totally VLAN unaware - the network is configured as usual (IP, Subnet Mask, Default Gateway, etc. ...nothing less nothing more).

     

    The tricky part is on the VSS side - I have a dedicated vSwitch, the vSwitch2 - on the ESXi:

     

     

    Properties
    Standard switch 	        vSwitch2
    MTU 	                        9000
    
    Security
    Promiscuous mode 	        Reject
    MAC address changes 	        Accept
    Forged transmits 	        Accept
    
    Traffic shaping
    Average bandwidth 	        --
    Peak bandwidth 	                --
    Burst size 	                --
    
    Teaming and failover
    Load balancing 	                Route based on IP hash
    Network failure detection 	Link status only
    Notify switches 	        Yes
    Failback 	                Yes
    Active adapters 	        vmnic1, vmnic7
    Standby adapters 	        --
    Unused adapters 	        --
    
    notes:
    In the IP hash load balancing policy all physical switch ports connected to the active uplinks must be in link aggregation mode.
    
    IP hash load balancing should be set for all port groups using the same set of uplinks.

     

     

    Please note that VSS (Standard vSwitch) doesn't support LACP as Link Aggregation Control Protocol...this means that, like me, you are dealing with a VSS then you can only aggregate physical links of your ESXi node by using what it's called a Non-Protocol trunking method (I prefer this description to highlight the fact this aggregation doesn't use a protocol between involved peers' logical interfaces)...and this is done exactly by using the "Route based on IP hash" as Load Balancing method under the Teaming and Failover. Acting that way is like to form what is known as a static channel-group (Cisco, for example, calls this a static-EtherChannel, HP - now Aruba - on ProVision/ArubaOS-Switch OS based switches calls it simply a port trunk operating in trunk mode instead of lacp mode). So no LACP here (at least until you can deploy a VDS Virtual Distributed Switch).

     

    On the Port Group simply define the VLAN id you want to be tagged (incoming tag accepted and outgoing tagged) and this need to cope the settings on the VSX Static LAG on the Aruba VSX side (clearly there you pass more VLAN tags, not just one).

     

    If you are so lucky to work with a VDS then you can setup a Team with LACP (IEEE 802.3ad) and thus use VSX LAG with LACP...with VSS you can't...I never tried to setup Teams with LACP as Load Balancing control protocol on ESXi vSwitches due to licensing restrictions of the ESXi editions I work with.



  • 14.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?
    Best Answer

    Posted Oct 08, 2020 12:26 AM

    Thank you so much for sharing the server config, i was able to fix the problem as in our case one NIC was showing active, and the other was on standby mode.

     

    Regards,

    Sahil Raina



  • 15.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Oct 08, 2020 04:41 AM

    Hi! Glad you solved!



  • 16.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted 24 days ago
    Thanks Parnassus for sharing this.
    Did you try testing Split brain senarios ? Will the secondary bring down the STATIC VSX LAG members just as it would for the dynamic lag members ?

    Regards.

    ------------------------------
    Achraf Harkati
    ------------------------------



  • 17.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 02:56 AM
    I have come across this thread after running into the same problem on a similar setup (2xAruba8302 in VSX running 10.02 and 1 VMware with 2x10Gbps NIC's in teaming only - no LACP license).
    Before going any further with this, I would like to ask you how did you manage to create Static Mclag's, because you are not allowed to do that - by design...at least on my equipments, when creating a mclag, it will be automatically lacp enabled without the possibility to turn this off.

    ------------------------------
    Alex Vrabie
    ------------------------------



  • 18.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 07:45 AM
    Hi @darthy_sanchez, "static" VSX LAGs (non LACP to be clear) were supported since the ArubaOS-CX 10.2.0001 software release, so since the very first public build of the ArubaOS-CX 10.2 software branch (the name MCLAG belongs to the old nomenclature...legacy of ArubaOS-CX 10.0​ era IIRC).

    I did exactly as written above in this thread.

    I don't recall exactly if I deployed that configuration when the ArubaOS-CX was already at 10.3 or when our VSX was still running ArubaOS-CX 10.2 (during September last year we upgraded from 10.2 to 10.3...in any case I'm quite sure we weren't running a too old ArubaOS-CX 10.02 at that time...probably 10.02.031 or so)...as I documented above the setup is working quite nicely.

    ------------------------------
    Davide Poletto
    ------------------------------



  • 19.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 08:25 AM
    I have somehow managed to find out how to do that: simply by issuing the command: "interface lag 100 multi-chasis static", with "static" at the end of the line. That's what I wanted to know...Tx anyway

    ------------------------------
    Alex Vrabie
    ------------------------------



  • 20.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 08:52 AM
    Correct, it was cited above...pretty simple. Glad you found it.

    ------------------------------
    Davide Poletto
    ------------------------------



  • 21.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted Dec 04, 2020 12:02 PM
    Yep, just saw it now! sorry..didn't read through the whole thread line ny line, as this was my only real problem here! static or lacp, my lags behave the same (balance or failover wise)! Cheers!

    ------------------------------
    Alex Vrabie
    ------------------------------



  • 22.  RE: ArubaOS-CX 10.02/10.03: anybody tried static (non LACP) VSX LAG against VMware vSphere host?

    Posted 22 days ago
    Can you share in this thread what NIC teaming setting to you use on ESXi server ? That could be useful for others for later reference.

    ------------------------------
    Vincent Giles
    ------------------------------