Wired Intelligent Edge

 View Only
Back to discussions
Expand all | Collapse all

Aruba CX in EVE-NG VSX LAG: lacp blocking

This thread has been viewed 84 times

  • 1.  Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 01:18 PM
    I'm working through the ACSP study guide labs using EVE-NG and running into what seems like a basic issue in the VSX lab.

    Here's the lab topology:

    VSX looks good and sync is working between Core-1 and Core-2. I've added the VSX LAG (multichassis) on the cores and all seems good there. Configured the LAG on the access switches and looks good there. However, the links are getting blocked by lacp:


    I see this in the logs (similar on core):
    2021-03-03T12:46:07.516897-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/9: Actor state: ALFOE, Partner state PLFOEX
2021-03-03T12:46:07.497691-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/9: Actor state: ALFO, Partner state PSFO
2021-03-03T12:46:07.469832-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/8: Actor state: ALFOE, Partner state PLFOEX
2021-03-03T12:46:07.422924-05:00 ICX-T1-Access-1 lacpd[755]: Event|1321|LOG_INFO|AMM|1/1|LAG 255 State change for interface 1/1/8: Actor state: ALFO, Partner state PSFO
2021-03-03T12:46:07.421809-05:00 ICX-T1-Access-1 lacpd[755]: Event|1311|LOG_WARN|AMM|1/1|Partner is lost (timed out) for interface 1/1/9 LAG sport: 1. State: Expired State -> Defaulted
 State 
2021-03-03T12:46:07.418302-05:00 ICX-T1-Access-1 lacpd[755]: Event|1311|LOG_WARN|AMM|1/1|Partner is lost (timed out) for interface 1/1/8 LAG sport: 1. State: Expired State -> Defaulted
 State ​
    Configs:
    Core Side:
    ICX-T1-Core-1# show running-config 
Current configuration:
!
!Version ArubaOS-CX Virtual.10.06.0001
!export-password: default
hostname ICX-T1-Core-1
user admin group administrators password ciphertext AQBapSQBB2dn5FuH5eRuT0V2gECncjlnn8sN7cl7kY+1U0xcYgAAAPm7vxpZKQZqFiLNEP4GttzmQZbDhyxdKVGU4h50B23js6PYJwiKCyLUfV1UPofxC0/GpTSmwaRWMlqLcn7M/yRLfHmmN0Gy
8H/d+n0Ys7ljyXiswHhDhFZ4SvKg2Riudw4m
led locator on
clock timezone us/eastern
vrf ka
ntp server 10.253.1.15 iburst prefer
ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
ntp enable
ntp vrf mgmt
cli-session
    timeout 43200
!
!
!
!
ssh server vrf mgmt
vlan 1
vlan 11
    vsx-sync
vlan 12                                                        
    vsx-sync
vlan 13
    vsx-sync
interface mgmt
    no shutdown
    ip static 10.251.1.2/24
    default-gateway 10.251.1.254
interface lag 1 multi-chassis
    no shutdown
    description access-1
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,11-13
    lacp mode active
    lacp rate fast
interface lag 2 multi-chassis
    no shutdown
    description access-2
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,11-13
    lacp mode active
interface lag 256                                              
    no shutdown
    no routing
    vlan trunk native 1 tag
    vlan trunk allowed all
    lacp mode active
interface 1/1/1
    no shutdown
    mtu 9100
    description access-1
    lag 1
interface 1/1/2
    no shutdown
    mtu 9100
    description access-2
    lag 2
interface 1/1/7
    no shutdown
    mtu 9198
    lag 256
interface 1/1/8
    no shutdown
    mtu 9198
    lag 256                                                    
interface 1/1/9
    no shutdown
    vrf attach ka
    ip address 192.168.0.0/31
interface vlan 1
    vsx-sync active-gateways policies
    ip address 10.1.1.2/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.1.1
interface vlan 11
    vsx-sync active-gateways policies
    ip address 10.1.11.2/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.11.1
    ip helper-address 10.1.1.6
    l3-counters
interface vlan 12
    vsx-sync active-gateways policies
    ip address 10.1.12.2/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.12.1
    ip helper-address 10.1.1.6
    l3-counters                                                
vsx
    system-mac 02:01:00:00:01:00
    inter-switch-link lag 256
    role primary
    keepalive peer 192.168.0.1 source 192.168.0.0 vrf ka
    vsx-sync aaa bfd-global bgp dhcp-relay mclag-interfaces ospf qos-global route-map vsx-global
ip dns server-address 10.251.1.21 vrf mgmt
!
!
!
!
!
https-server vrf default
https-server vrf mgmt
ICX-T1-Core-1# ​

    Access:
    ICX-T1-Access-1# show running-config                           
Current configuration:
!
!Version ArubaOS-CX Virtual.10.06.0001
!export-password: default
hostname ICX-T1-Access-1
user admin group administrators password ciphertext AQBapVP2a2ANo/d9wJvFUud2H0kW1I/MZb9b7sRdveQ1D1EAYgAAAKHS5OWKFDAqlM/T8Qs9HehJmSAed4LdgUkga1Rrn+s3aEQ6+ODO5RB8jgNCWpka6eq7oM1cUIiKa0MI
/AxgPbvcG/pIFKzEUg0dIbI3YuY+dFXNFBlhnATph689FZQ+wGAU
led locator on
clock timezone us/eastern
ntp server 10.253.1.15 iburst prefer
ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
ntp enable
ntp vrf mgmt
cli-session
    timeout 43200
!
!
!
!
ssh server vrf mgmt
vlan 1,11-13
interface mgmt
    no shutdown
    ip static 10.251.1.4/24
    default-gateway 10.251.1.254                               
interface lag 255
    no shutdown
    description core
    no routing
    vlan trunk native 1
    vlan trunk allowed all
    lacp mode active
    lacp rate fast
interface 1/1/8
    no shutdown
    mtu 9100
    lag 255
interface 1/1/9
    no shutdown
    mtu 9100
    lag 255
ip dns server-address 10.251.1.21 vrf mgmt
!
!
!
!
!
https-server vrf mgmt                                          
ICX-T1-Access-1# 
​

    Any help or ideas would be appreciated. :)

    ------------------------------
    Allyn Crowe
    ------------------------------


  • 2.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 04:59 PM
    Hello Allyn - apparently (I see VSX node 1 configuration only, VSX node 2 configuration lacks) - it looks correct (don't understand why 9100 as MTU, I would have used 9198 instead in every interface and I don't understand why between the VSX LAG lag1 configured on VSX switch(es) and non-VSX LAG lag255 configured on Access switch there is a discrepancy about permitted VLANs...allowed "1, 11-13" VSX side versus allowed "All" Access side...but, after all, this discrepancy shouldn't be too important here).

    Could you show us the VSX node 2's configuration too?
    What's the outputs of show vsx status and show vsx brief commands executed on VSX node 1 and/or VSX node 2?
    What's the outputs of show lacp aggregates lag1 and show lacp aggregates lag1 vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp interfaces and show lacp interfaces vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp interfaces 1/1/1 and show lacp interfaces 1/1/1 vsx-peer commands both executed on VSX node 1?
    What's the outputs of show lacp aggregates lag255 command executed on Access-1 Switch?

    It looks like there is something wrong in the way lag255 is connected to (VSX) lag1 on VSX node 1 and VSX node 2 (assuming VSX is OK).

    It's totally possible that - if this design is virtualized (OVA) as a Lab scenario - some feature could not work exactly as one expects.

    ------------------------------
    Davide Poletto
    ------------------------------

    Original Message


  • 3.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:24 PM

    > don't understand why 9100 as MTU, I would have used 9198 instead in every interface
    me either, just working down the study guide labs and they have you set those that way :p I agree the value is odd...

    > there is a discrepancy about permitted VLANs
    agreed :) I'd never use an allow all in production, but they are big on not pruning in the study guide ;)

    It is very possible that it's something in eve, it was just very odd that it manifested like this

    As to the requests:

    Core-2 Config:

    Current configuration:
!
!Version ArubaOS-CX Virtual.10.06.0001
!export-password: default
hostname ICX-T1-Core-2
user admin group administrators password ciphertext AQBapSQBB2dn5FuH5eRuT0V2gECncjlnn
8sN7cl7kY+1U0xcYgAAAPm7vxpZKQZqFiLNEP4GttzmQZbDhyxdKVGU4h50B23js6PYJwiKCyLUfV1UPofxC0
/GpTSmwaRWMlqLcn7M/yRLfHmmN0Gy8H/d+n0Ys7ljyXiswHhDhFZ4SvKg2Riudw4m
led locator on
clock timezone us/eastern
vrf ka
ntp server 10.253.1.15 iburst prefer
ntp server pool.ntp.org minpoll 4 maxpoll 4 iburst
ntp enable
ntp vrf mgmt
cli-session
    timeout 43200
!
!
!
!
ssh server vrf mgmt
vlan 1
vlan 11
    vsx-sync
vlan 12                                                        
    vsx-sync
vlan 13
    vsx-sync
interface mgmt
    no shutdown
    ip static 10.251.1.3/24
    default-gateway 10.251.1.254
interface lag 1 multi-chassis
    no shutdown
    description access-1
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,11-13
    lacp mode active
    lacp rate fast
interface lag 2 multi-chassis
    no shutdown
    description access-2
    no routing
    vlan trunk native 1
    vlan trunk allowed 1,11-13
    lacp mode active
interface lag 256                                              
    no shutdown
    no routing
    vlan trunk native 1 tag
    vlan trunk allowed all
    lacp mode active
interface 1/1/1
    no shutdown
    mtu 9100
    description access-1
    lag 1
interface 1/1/2
    no shutdown
    mtu 9100
    description access-2
    lag 2
interface 1/1/7
    no shutdown
    mtu 9198
    lag 256
interface 1/1/8
    no shutdown
    mtu 9198
    lag 256                                                    
interface 1/1/9
    no shutdown
    vrf attach ka
    ip address 192.168.0.1/31
interface vlan 1
    vsx-sync active-gateways policies
    ip address 10.1.1.3/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.1.1
interface vlan 11
    vsx-sync active-gateways policies
    ip address 10.1.11.3/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.11.1
    ip helper-address 10.1.1.6
    l3-counters
interface vlan 12
    vsx-sync active-gateways policies
    ip address 10.1.12.3/24
    active-gateway ip mac 02:02:00:00:01:00
    active-gateway ip 10.1.12.1
    ip helper-address 10.1.1.6
    l3-counters                                                
vsx
    system-mac 02:01:00:00:01:00
    inter-switch-link lag 256
    role secondary
    keepalive peer 192.168.0.0 source 192.168.0.1 vrf ka
    vsx-sync aaa bfd-global bgp dhcp-relay mclag-interfaces ospf qos-global route-map
 vsx-global
ip dns server-address 10.251.1.21 vrf mgmt
!
!
!
!
!
https-server vrf default
https-server vrf mgmt
ICX-T1-Core-2#


    What's the outputs of show vsx status and show vsx brief commands executed on VSX node 1 and/or VSX node 2?

    ICX-T1-Core-1# show vsx status
VSX Operational State
---------------------
  ISL channel             : In-Sync
  ISL mgmt channel        : operational
  Config Sync Status      : In-Sync
  NAE                     : peer_reachable
  HTTPS Server            : peer_reachable

Attribute           Local               Peer
------------        --------            --------
ISL link            lag256              lag256
ISL version         2                   2
System MAC          02:01:00:00:01:00   02:01:00:00:01:00
Platform            X86-64              X86-64
Software Version    Virtual.10.06.0001  Virtual.10.06.0001
Device Role         primary             secondary

ICX-T1-Core-1# show vsx brief
ISL State                              : In-Sync
Device State                           : Peer-Established
Keepalive State                        : Keepalive-Established
Device Role                            : Primary
Number of Multi-chassis LAG interfaces : 2
ICX-T1-Core-1#


    What's the outputs of show lacp aggregates lag1 and show lacp aggregates lag1 vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp aggregates lag1 

Aggregate name   : lag1 (multi-chassis)              
Interfaces       : 1/1/1                             
Peer interfaces  : 1/1/1                             
Heartbeat rate   : Fast                              
Hash             : l3-src-dst                        
Aggregate mode   : Active                            
ICX-T1-Core-1# show lacp aggregates lag1 vsx-peer

Aggregate name   : lag1 (multi-chassis)              
Interfaces       : 1/1/1                             
Peer interfaces  : 1/1/1                             
Heartbeat rate   : Fast                              
Hash             : l3-src-dst                        
Aggregate mode   : Active                            
ICX-T1-Core-1#


    What's the outputs of show lacp interfaces and show lacp interfaces vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp interfaces

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf       Aggr       Port  Port  State   System-ID         System Aggr Forwarding
           Name       Id    Pri                             Pri    Key  State     
----------------------------------------------------------------------------------
1/1/1      lag1(mc)   1     1     ASFOE   02:01:00:00:01:00 65534  1    lacp-block
1/1/2      lag2(mc)   2     1     ALFOE   02:01:00:00:01:00 65534  2    lacp-block
1/1/7      lag256     8     1     ALFNCD  08:00:09:d0:51:08 65534  256  up        
1/1/8      lag256     9     1     ALFNCD  08:00:09:d0:51:08 65534  256  up        


Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf       Aggr       Port  Port  State   System-ID         System Aggr           
           Name       Id    Pri                             Pri    Key            
----------------------------------------------------------------------------------
1/1/1      lag1(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
1/1/2      lag2(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
1/1/7      lag256     8     1     ALFNCD  08:00:09:6a:b9:73 65534  256            
1/1/8      lag256     9     1     ALFNCD  08:00:09:6a:b9:73 65534  256            
ICX-T1-Core-1# 

ICX-T1-Core-1# show lacp interfaces vsx-peer 

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state

Actor details of all interfaces:
----------------------------------------------------------------------------------
Intf       Aggr       Port  Port  State   System-ID         System Aggr Forwarding
           Name       Id    Pri                             Pri    Key  State     
----------------------------------------------------------------------------------
1/1/1      lag1(mc)   1001  1     ASFOE   02:01:00:00:01:00 65534  1    lacp-block
1/1/2      lag2(mc)   1002  1     ALFOE   02:01:00:00:01:00 65534  2    lacp-block
1/1/7      lag256     8     1     ALFNCD  08:00:09:6a:b9:73 65534  256  up        
1/1/8      lag256     9     1     ALFNCD  08:00:09:6a:b9:73 65534  256  up        


Partner details of all interfaces:
----------------------------------------------------------------------------------
Intf       Aggr       Port  Port  State   System-ID         System Aggr           
           Name       Id    Pri                             Pri    Key            
----------------------------------------------------------------------------------
1/1/1      lag1(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
1/1/2      lag2(mc)   0     65534 PLFOEX  00:00:00:00:00:00 65534  0              
1/1/7      lag256     8     1     ALFNCD  08:00:09:d0:51:08 65534  256            
1/1/8      lag256     9     1     ALFNCD  08:00:09:d0:51:08 65534  256


    What's the outputs of show lacp interfaces 1/1/1 and show lacp interfaces 1/1/1 vsx-peer commands both executed on VSX node 1?

    ICX-T1-Core-1# show lacp interfaces 1/1/1

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state


Aggregate-name : lag1(multi-chassis)
-------------------------------------------------
                       Actor             Partner
-------------------------------------------------
Port-id            | 1                  | 0                  
Port-priority      | 1                  | 65534              
Key                | 1                  | 0                  
State              | ASFOE              | PLFOEX             
System-ID          | 02:01:00:00:01:00  | 00:00:00:00:00:00  
System-priority    | 65534              | 65534              

ICX-T1-Core-1# show lacp interfaces 1/1/1 vsx-peer 

State abbreviations :
A - Active        P - Passive      F - Aggregable I - Individual
S - Short-timeout L - Long-timeout N - InSync     O - OutofSync
C - Collecting    D - Distributing 
X - State m/c expired              E - Default neighbor state


Aggregate-name : lag1(multi-chassis)
-------------------------------------------------
                       Actor             Partner
-------------------------------------------------
Port-id            | 1001               | 0                  
Port-priority      | 1                  | 65534              
Key                | 1                  | 0                  
State              | ASFOE              | PLFOEX             
System-ID          | 02:01:00:00:01:00  | 00:00:00:00:00:00  
System-priority    | 65534              | 65534              

ICX-T1-Core-1#


    What's the outputs of show lacp aggregates lag255 command executed on Access-1 Switch?

    CX-T1-Access-1# show lacp aggregates lag255

Aggregate name   : lag255                            
Interfaces       : 1/1/9 1/1/8                       
Heartbeat rate   : Fast                              
Hash             : l3-src-dst                        
Aggregate mode   : Active                            
ICX-T1-Access-1#





    ------------------------------
    Allyn Crowe
    ------------------------------

    Original Message


  • 4.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:52 PM
    Well with some additional tinkering I was able to get it to work...
    I realized in eve I had additional ports "set" that wasn't showing in the switch (since the OVA is limited to 10). So I shut the switch down and removed those.
    I then deleted the EVE interface connections
    Once the switch was back up I removed the MTU and shut the ports down on the Access Switch
    I reconnected the eve links and reenabled the ports.
    The links then came up.
    Set the MTU to 9100 again and all is still good.

    My *guess* at this point is something in the ova configs with the extra ports and the link references was broken. I had tried removing them and reading them before without luck. So now of course I'll spend more time seeing if I can break it again to narrow down the actual problem... Anything to distract me from the actual studying ;)

    Thanks for your help @parnassus​

    ------------------------------
    Allyn Crowe
    ------------------------------

    Original Message


  • 5.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:31 PM
    Allyn

    Seen some issues with 10.06.001 on the OVA. Please try 10.05.0020. see if this helps in the first instance

    ------------------------------
    Kamal Takodra
    If my post was useful accept solution and/or give kudos
    ------------------------------

    Original Message


  • 6.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Mar 03, 2021 05:53 PM
    interesting... ok I'll give that a try at some point too. I was able to get it to work with some tinkering.

    ------------------------------
    Allyn Crowe
    ------------------------------

    Original Message


  • 7.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Feb 05, 2024 09:18 AM

    Hi Acrowe, 

    what exactly did you do to resolve this issue? I am have exactly the same issue. 

    Lacp-blocked on some of the ports.  

    N:B: I don't have this same on the live switches. Its on EVE-ng.

    Kindly assist.


    Original Message


  • 8.  RE: Aruba CX in EVE-NG VSX LAG: lacp blocking

    Posted Feb 06, 2024 04:52 AM

    Hi Allyn,

    Take a look at the lab guide here (AOS-CX Simulator Lab - VSX (Part 2)):

    https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=565&MessageKey=dd0e5d49-8203-402f-bacf-d19139bc55b9&CommunityKey=aa40c287-728e-4827-b062-5eff4ed6410b&tab=digestviewer&ReturnUrl=%2fcommunity-home%2fdigestviewer%3fcommunitykey%3daa40c287-728e-4827-b062-5eff4ed6410b%26tab%3ddigestviewer

    For me it sounds similar to the MCLAG caveat described in the doc. Probably this limitation is already fixed in the newer versions, but you have to further check the same in the release notes for the newer OVA Versions. 



    ------------------------------
    Stanislav Naydenov
    ------------------------------

    Original Message


Related Content