Wired Intelligent Edge

 View Only
last person joined: 16 hours ago 

Bring performance and reliability to your network with the HPE Aruba Networking Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of your switching devices, and find ways to improve security across your network to bring together a mobile-first solution
Expand all | Collapse all

Aruba CX in EVE-NG VSX LAG: lacp blocking

This thread has been viewed 71 times
  • 1.  Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 01:18 PM
    I'm working through the ACSP study guide labs using EVE-NG and running into what seems like a basic issue in the VSX lab.

    Here's the lab topology:

    VSX looks good and sync is working between Core-1 and Core-2. I've added the VSX LAG (multichassis) on the cores and all seems good there. Configured the LAG on the access switches and looks good there. However, the links are getting blocked by lacp:


    I see this in the logs (similar on core):
    2021-03-03T12:46:07.516897-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/9: Actor state: ALFOE, Partner state PLFOEX
    2021-03-03T12:46:07.497691-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/9: Actor state: ALFO, Partner state PSFO
    2021-03-03T12:46:07.469832-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/8: Actor state: ALFOE, Partner state PLFOEX
    2021-03-03T12:46:07.422924-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/8: Actor state: ALFO, Partner state PSFO
    2021-03-03T12:46:07.421809-05:00 ICX-T1-Access-1 lacpd[755]: Event|1311|LOG_WARN|AMM|1/1|Partner is lost (timed out) for interface 1/1/9 LAG sport: 1. State: Expired State -> Defaulted
     State 
    2021-03-03T12:46:07.418302-05:00 ICX-T1-Access-1 lacpd[755]: Event|1311|LOG_WARN|AMM|1/1|Partner is lost (timed out) for interface 1/1/8 LAG sport: 1. State: Expired State -> Defaulted
     State ​
    Configs:
    Core Side:
    ICX-T1-Core-1# show running-config 
    Current configuration:
    !
    !Version ArubaOS-CX Virtual.10.06.0001
    !export-password: default
    hostname ICX-T1-Core-1
    user admin group administrators password ciphertext AQBapSQBB2dn5FuH5eRuT0V2gECncjlnn8sN7cl7kY+1U0xcYgAAAPm7vxpZKQZqFiLNEP4GttzmQZbDhyxdKVGU4h50B23js6PYJwiKCyLUfV1UPofxC0/GpTSmwaRWMlqLcn7M/yRLfHmmN0Gy
    8H/d+n0Ys7ljyXiswHhDhFZ4SvKg2Riudw4m
    led locator on
    clock timezone us/eastern
    vrf ka
    ntp server 10.253.1.15 iburst prefer
    ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
    ntp enable
    ntp vrf mgmt
    cli-session
        timeout 43200
    !
    !
    !
    !
    ssh server vrf mgmt
    vlan 1
    vlan 11
        vsx-sync
    vlan 12                                                        
        vsx-sync
    vlan 13
        vsx-sync
    interface mgmt
        no shutdown
        ip static 10.251.1.2/24
        default-gateway 10.251.1.254
    interface lag 1 multi-chassis
        no shutdown
        description access-1
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,11-13
        lacp mode active
        lacp rate fast
    interface lag 2 multi-chassis
        no shutdown
        description access-2
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,11-13
        lacp mode active
    interface lag 256                                              
        no shutdown
        no routing
        vlan trunk native 1 tag
        vlan trunk allowed all
        lacp mode active
    interface 1/1/1
        no shutdown
        mtu 9100
        description access-1
        lag 1
    interface 1/1/2
        no shutdown
        mtu 9100
        description access-2
        lag 2
    interface 1/1/7
        no shutdown
        mtu 9198
        lag 256
    interface 1/1/8
        no shutdown
        mtu 9198
        lag 256                                                    
    interface 1/1/9
        no shutdown
        vrf attach ka
        ip address 192.168.0.0/31
    interface vlan 1
        vsx-sync active-gateways policies
        ip address 10.1.1.2/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.1.1
    interface vlan 11
        vsx-sync active-gateways policies
        ip address 10.1.11.2/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.11.1
        ip helper-address 10.1.1.6
        l3-counters
    interface vlan 12
        vsx-sync active-gateways policies
        ip address 10.1.12.2/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.12.1
        ip helper-address 10.1.1.6
        l3-counters                                                
    vsx
        system-mac 02:01:00:00:01:00
        inter-switch-link lag 256
        role primary
        keepalive peer 192.168.0.1 source 192.168.0.0 vrf ka
        vsx-sync aaa bfd-global bgp dhcp-relay mclag-interfaces ospf qos-global route-map vsx-global
    ip dns server-address 10.251.1.21 vrf mgmt
    !
    !
    !
    !
    !
    https-server vrf default
    https-server vrf mgmt
    ICX-T1-Core-1# ​

    Access:
    ICX-T1-Access-1# show running-config                           
    Current configuration:
    !
    !Version ArubaOS-CX Virtual.10.06.0001
    !export-password: default
    hostname ICX-T1-Access-1
    user admin group administrators password ciphertext AQBapVP2a2ANo/d9wJvFUud2H0kW1I/MZb9b7sRdveQ1D1EAYgAAAKHS5OWKFDAqlM/T8Qs9HehJmSAed4LdgUkga1Rrn+s3aEQ6+ODO5RB8jgNCWpka6eq7oM1cUIiKa0MI
    /AxgPbvcG/pIFKzEUg0dIbI3YuY+dFXNFBlhnATph689FZQ+wGAU
    led locator on
    clock timezone us/eastern
    ntp server 10.253.1.15 iburst prefer
    ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
    ntp enable
    ntp vrf mgmt
    cli-session
        timeout 43200
    !
    !
    !
    !
    ssh server vrf mgmt
    vlan 1,11-13
    interface mgmt
        no shutdown
        ip static 10.251.1.4/24
        default-gateway 10.251.1.254                               
    interface lag 255
        no shutdown
        description core
        no routing
        vlan trunk native 1
        vlan trunk allowed all
        lacp mode active
        lacp rate fast
    interface 1/1/8
        no shutdown
        mtu 9100
        lag 255
    interface 1/1/9
        no shutdown
        mtu 9100
        lag 255
    ip dns server-address 10.251.1.21 vrf mgmt
    !
    !
    !
    !
    !
    https-server vrf mgmt                                          
    ICX-T1-Access-1# 
    ​

    Any help or ideas would be appreciated. :)

    ------------------------------
    Allyn Crowe
    ------------------------------


  • 2.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    MVP GURU
    Posted Mar 03, 2021 04:59 PM
    Hello Allyn - apparently (I see VSX node 1 configuration only, VSX node 2 configuration lacks) - it looks correct (don't understand why 9100 as MTU, I would have used 9198 instead in every interface and I don't understand why between the VSX LAG lag1 configured on VSX switch(es) and non-VSX LAG lag255 configured on Access switch there is a discrepancy about permitted VLANs...allowed "1, 11-13" VSX side versus allowed "All" Access side...but, after all, this discrepancy shouldn't be too important here).

    Could you show us the VSX node 2's configuration too?
    What's the outputs of show vsx status and show vsx brief commands executed on VSX node 1 and/or VSX node 2?
    What's the outputs of show lacp aggregates lag1 and show lacp aggregates lag1 vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp interfaces and show lacp interfaces vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp interfaces 1/1/1 and show lacp interfaces 1/1/1 vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp aggregates lag255 command executed on Access-1 Switch?

    It looks like there is something wrong in the way lag255 is connected to (VSX) lag1 on VSX node 1 and VSX node 2 (assuming VSX is OK).

    It's totally possible that - if this design is virtualized (OVA) as a Lab scenario - some feature could not work exactly as one expects.

    ------------------------------
    Davide Poletto
    ------------------------------



  • 3.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:24 PM

    > don't understand why 9100 as MTU, I would have used 9198 instead in every interface
    me either, just working down the study guide labs and they have you set those that way :p I agree the value is odd...

    > there is a discrepancy about permitted VLANs
    agreed :) I'd never use an allow all in production, but they are big on not pruning in the study guide ;)

    It is very possible that it's something in eve, it was just very odd that it manifested like this

    As to the requests:

    Core-2 Config:

    Current configuration:
    !
    !Version ArubaOS-CX Virtual.10.06.0001
    !export-password: default
    hostname ICX-T1-Core-2
    user admin group administrators password ciphertext AQBapSQBB2dn5FuH5eRuT0V2gECncjlnn
    8sN7cl7kY+1U0xcYgAAAPm7vxpZKQZqFiLNEP4GttzmQZbDhyxdKVGU4h50B23js6PYJwiKCyLUfV1UPofxC0
    /GpTSmwaRWMlqLcn7M/yRLfHmmN0Gy8H/d+n0Ys7ljyXiswHhDhFZ4SvKg2Riudw4m
    led locator on
    clock timezone us/eastern
    vrf ka
    ntp server 10.253.1.15 iburst prefer
    ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
    ntp enable
    ntp vrf mgmt
    cli-session
        timeout 43200
    !
    !
    !
    !
    ssh server vrf mgmt
    vlan 1
    vlan 11
        vsx-sync
    vlan 12                                                        
        vsx-sync
    vlan 13
        vsx-sync
    interface mgmt
        no shutdown
        ip static 10.251.1.3/24
        default-gateway 10.251.1.254
    interface lag 1 multi-chassis
        no shutdown
        description access-1
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,11-13
        lacp mode active
        lacp rate fast
    interface lag 2 multi-chassis
        no shutdown
        description access-2
        no routing
        vlan trunk native 1
        vlan trunk allowed 1,11-13
        lacp mode active
    interface lag 256                                              
        no shutdown
        no routing
        vlan trunk native 1 tag
        vlan trunk allowed all
        lacp mode active
    interface 1/1/1
        no shutdown
        mtu 9100
        description access-1
        lag 1
    interface 1/1/2
        no shutdown
        mtu 9100
        description access-2
        lag 2
    interface 1/1/7
        no shutdown
        mtu 9198
        lag 256
    interface 1/1/8
        no shutdown
        mtu 9198
        lag 256                                                    
    interface 1/1/9
        no shutdown
        vrf attach ka
        ip address 192.168.0.1/31
    interface vlan 1
        vsx-sync active-gateways policies
        ip address 10.1.1.3/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.1.1
    interface vlan 11
        vsx-sync active-gateways policies
        ip address 10.1.11.3/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.11.1
        ip helper-address 10.1.1.6
        l3-counters
    interface vlan 12
        vsx-sync active-gateways policies
        ip address 10.1.12.3/24
        active-gateway ip mac 02:02:00:00:01:00
        active-gateway ip 10.1.12.1
        ip helper-address 10.1.1.6
        l3-counters                                                
    vsx
        system-mac 02:01:00:00:01:00
        inter-switch-link lag 256
        role secondary
        keepalive peer 192.168.0.0 source 192.168.0.1 vrf ka
        vsx-sync aaa bfd-global bgp dhcp-relay mclag-interfaces ospf qos-global route-map
     vsx-global
    ip dns server-address 10.251.1.21 vrf mgmt
    !
    !
    !
    !
    !
    https-server vrf default
    https-server vrf mgmt
    ICX-T1-Core-2#


    What's the outputs of show vsx status and show vsx brief commands executed on VSX node 1 and/or VSX node 2?

    ICX-T1-Core-1# show vsx status
    VSX Operational State
    ---------------------
      ISL channel             : In-Sync
      ISL mgmt channel        : operational
      Config Sync Status      : In-Sync
      NAE                     : peer_reachable
      HTTPS Server            : peer_reachable
    
    Attribute           Local               Peer
    ------------        --------            --------
    ISL link            lag256              lag256
    ISL version         2                   2
    System MAC          02:01:00:00:01:00   02:01:00:00:01:00
    Platform            X86-64              X86-64
    Software Version    Virtual.10.06.0001  Virtual.10.06.0001
    Device Role         primary             secondary
    
    ICX-T1-Core-1# show vsx brief
    ISL State                              : In-Sync
    Device State                           : Peer-Established
    Keepalive State                        : Keepalive-Established
    Device Role                            : Primary
    Number of Multi-chassis LAG interfaces : 2
    ICX-T1-Core-1# 
    
    


    What's the outputs of show lacp aggregates lag1 and show lacp aggregates lag1 vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp aggregates lag1 
    
    Aggregate name   : lag1 (multi-chassis)              
    Interfaces       : 1/1/1                             
    Peer interfaces  : 1/1/1                             
    Heartbeat rate   : Fast                              
    Hash             : l3-src-dst                        
    Aggregate mode   : Active                            
    ICX-T1-Core-1# show lacp aggregates lag1 vsx-peer
    
    Aggregate name   : lag1 (multi-chassis)              
    Interfaces       : 1/1/1                             
    Peer interfaces  : 1/1/1                             
    Heartbeat rate   : Fast                              
    Hash             : l3-src-dst                        
    Aggregate mode   : Active                            
    ICX-T1-Core-1# 
    


    What's the outputs of show lacp interfaces and show lacp interfaces vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp interfaces
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ----------------------------------------------------------------------------------
    Intf       Aggr       Port  Port  State   System-ID         System Aggr Forwarding
               Name       Id    Pri                             Pri    Key  State     
    ----------------------------------------------------------------------------------
    1/1/1      lag1(mc)   1     1     ASFOE   02:01:00:00:01:00 65534  1    lacp-block
    1/1/2      lag2(mc)   2     1     ALFOE   02:01:00:00:01:00 65534  2    lacp-block
    1/1/7      lag256     8     1     ALFNCD  08:00:09:d0:51:08 65534  256  up        
    1/1/8      lag256     9     1     ALFNCD  08:00:09:d0:51:08 65534  256  up        
    
    
    Partner details of all interfaces:
    ----------------------------------------------------------------------------------
    Intf       Aggr       Port  Port  State   System-ID         System Aggr           
               Name       Id    Pri                             Pri    Key            
    ----------------------------------------------------------------------------------
    1/1/1      lag1(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
    1/1/2      lag2(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
    1/1/7      lag256     8     1     ALFNCD  08:00:09:6a:b9:73 65534  256            
    1/1/8      lag256     9     1     ALFNCD  08:00:09:6a:b9:73 65534  256            
    ICX-T1-Core-1# 
    
    ICX-T1-Core-1# show lacp interfaces vsx-peer 
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    Actor details of all interfaces:
    ----------------------------------------------------------------------------------
    Intf       Aggr       Port  Port  State   System-ID         System Aggr Forwarding
               Name       Id    Pri                             Pri    Key  State     
    ----------------------------------------------------------------------------------
    1/1/1      lag1(mc)   1001  1     ASFOE   02:01:00:00:01:00 65534  1    lacp-block
    1/1/2      lag2(mc)   1002  1     ALFOE   02:01:00:00:01:00 65534  2    lacp-block
    1/1/7      lag256     8     1     ALFNCD  08:00:09:6a:b9:73 65534  256  up        
    1/1/8      lag256     9     1     ALFNCD  08:00:09:6a:b9:73 65534  256  up        
    
    
    Partner details of all interfaces:
    ----------------------------------------------------------------------------------
    Intf       Aggr       Port  Port  State   System-ID         System Aggr           
               Name       Id    Pri                             Pri    Key            
    ----------------------------------------------------------------------------------
    1/1/1      lag1(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
    1/1/2      lag2(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
    1/1/7      lag256     8     1     ALFNCD  08:00:09:d0:51:08 65534  256            
    1/1/8      lag256     9     1     ALFNCD  08:00:09:d0:51:08 65534  256 
    
    


    What's the outputs of show lacp interfaces 1/1/1 and show lacp interfaces 1/1/1 vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp interfaces 1/1/1
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    
    Aggregate-name : lag1(multi-chassis)
    -------------------------------------------------
                           Actor             Partner
    -------------------------------------------------
    Port-id            | 1                  | 0                  
    Port-priority      | 1                  | 65534              
    Key                | 1                  | 0                  
    State              | ASFOE              | PLFOEX             
    System-ID          | 02:01:00:00:01:00  | 00:00:00:00:00:00  
    System-priority    | 65534              | 65534              
    
    ICX-T1-Core-1# show lacp interfaces 1/1/1 vsx-peer 
    
    State abbreviations :
    A - Active        P - Passive      F - Aggregable I - Individual
    S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
    C - Collecting    D - Distributing 
    X - State m/c expired              E - Default neighbor state
    
    
    Aggregate-name : lag1(multi-chassis)
    -------------------------------------------------
                           Actor             Partner
    -------------------------------------------------
    Port-id            | 1001               | 0                  
    Port-priority      | 1                  | 65534              
    Key                | 1                  | 0                  
    State              | ASFOE              | PLFOEX             
    System-ID          | 02:01:00:00:01:00  | 00:00:00:00:00:00  
    System-priority    | 65534              | 65534              
    
    ICX-T1-Core-1# 
    


    What's the outputs of show lacp aggregates lag255 command executed on Access-1 Switch?

    CX-T1-Access-1# show lacp aggregates lag255
    
    Aggregate name   : lag255                            
    Interfaces       : 1/1/9 1/1/8                       
    Heartbeat rate   : Fast                              
    Hash             : l3-src-dst                        
    Aggregate mode   : Active                            
    ICX-T1-Access-1# 
    





    ------------------------------
    Allyn Crowe
    ------------------------------



  • 4.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:52 PM
    Well with some additional tinkering I was able to get it to work...
    I realized in eve I had additional ports "set" that wasn't showing in the switch (since the OVA is limited to 10). So I shut the switch down and removed those.
    I then deleted the EVE interface connections
    Once the switch was back up I removed the MTU and shut the ports down on the Access Switch
    I reconnected the eve links and reenabled the ports.
    The links then came up.
    Set the MTU to 9100 again and all is still good.

    My *guess* at this point is something in the ova configs with the extra ports and the link references was broken. I had tried removing them and reading them before without luck. So now of course I'll spend more time seeing if I can break it again to narrow down the actual problem... Anything to distract me from the actual studying ;)

    Thanks for your help @parnassus

    ------------------------------
    Allyn Crowe
    ------------------------------



  • 5.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    EMPLOYEE
    Posted Mar 03, 2021 05:31 PM
    Allyn

    Seen some issues with 10.06.001 on the OVA. Please try 10.05.0020. see if this helps in the first instance


    ------------------------------
    Kamal Takodra
    If my post was useful accept solution and/or give kudos
    ------------------------------



  • 6.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:53 PM
    interesting... ok I'll give that a try at some point too. I was able to get it to work with some tinkering.

    ------------------------------
    Allyn Crowe
    ------------------------------



  • 7.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted 17 days ago

    Hi Acrowe, 

    what exactly did you do to resolve this issue? I am have exactly the same issue. 

    Lacp-blocked on some of the ports.  

    N:B: I don't have this same on the live switches. Its on EVE-ng.

    Kindly assist.




  • 8.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    EMPLOYEE
    Posted 17 days ago

    Hi Allyn,

    Take a look at the lab guide here (AOS-CX Simulator Lab - VSX (Part 2)):

    https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=565&MessageKey=dd0e5d49-8203-402f-bacf-d19139bc55b9&CommunityKey=aa40c287-728e-4827-b062-5eff4ed6410b&tab=digestviewer&ReturnUrl=%2fcommunity-home%2fdigestviewer%3fcommunitykey%3daa40c287-728e-4827-b062-5eff4ed6410b%26tab%3ddigestviewer

    For me it sounds similar to the MCLAG caveat described in the doc. Probably this limitation is already fixed in the newer versions, but you have to further check the same in the release notes for the newer OVA Versions. 



    ------------------------------
    Stanislav Naydenov
    ------------------------------