Wired Intelligent Edge

 View Only

  • 1.  8325 VSX MC-Lag to 6300M

    Posted Nov 20, 2023 09:51 AM

    We have some strange behaviour in a fresh installed VSX-VSF  setup 

    see connections below:

    Note: ISL is atm only 1x 10Gbit will be changed to 2x100Gbit.

    On Core in Room1 everthing works: see config below:

    vrf Keepalive
system interface-group 4 speed 10G

interface lag 1 multi-chasssis
no shutdown
no routing
vlan trunk native 1
vlan trunk allowd all
lacp mode active
lacp fallback

interface lag 256
description ISL
no shutdown
no routing
vlan trunk native 1
vlan trunk allowed all
lacp mode active
lacp rate fast

interface 1/1/1
no shutdown 
lag1

interface 1/1/2
no shutdown 
lag1

interface 1/1/47
    description Keepalive
    no shutdown
    vrf attach Keepalive
    ip address 192.168.254.253/30

interface 1/1/48
    description ISL-Physical-Link
    no shutdown
    lag 256

vsx
    system-mac xx:xx:xx:xx:xx
    inter-switch-link lag 256
    role primary
    keepalive peer 192.168.254.254 source 192.168.254.253 vrf Keepalive

    On Core 1 in room 1 everthing works fine:

    Aggregate lag1 is up 
     Admin state is up 
     Description : MP-DS-1_Uplink
     MAC Address                 : bc:d7:a5:6a:24:00 
     Aggregated-interfaces       : 1/1/1 1/1/2 
     Aggregation-key             : 1
     Aggregate mode              : active 
     Speed                       : 50000 Mb/s 
     qos trust dscp
     VLAN Mode: native-untagged
     Native VLAN: 1
     Allowed VLAN List: all
     L3 Counters: Rx Disabled, Tx Disabled 

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                      20879493               835406             21714899
       Unicast                      482612               283469               766081
       Multicast                   4391919               547593              4939512
       Broadcast                  16004962                 4344             16009306
     Bytes                      2671621222            257910736           2929531958
     Jumbos                           2744                94434                97178
     Dropped                             0                    0                    0
     Filtered                        30184                    0                30184
     Pause Frames                        0                    0                    0
     Errors                              0                    0                    0
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                            0                  n/a                    0

    But on Core 2 we dont get the ports up:

    Aggregate lag1 is down 
     Admin state is up 
     State information : Disabled by aggregation 
     Description : 
     MAC Address                 : bc:d7:a5:6a:e8:00 
     Aggregated-interfaces       : 1/1/1 1/1/2 
     Aggregation-key             : 1
     Aggregate mode              : active 
     Speed                       : 0 Mb/s 
     qos trust dscp
     VLAN Mode: native-untagged
     Native VLAN: 1
     Allowed VLAN List: all
     L3 Counters: Rx Disabled, Tx Disabled 

     Statistic                          RX                   TX                Total
     ---------------- -------------------- -------------------- --------------------
     Packets                        145565                 8651               154216
       Unicast                          93                 2475                 2568
       Multicast                     25919                 6131                32050
       Broadcast                    119553                   45               119598
     Bytes                        16827093              2592048             19419141
     Jumbos                              3                  929                  932
     Dropped                             0                    0                    0
     Filtered                          203                    0                  203
     Pause Frames                        0                    0                    0
     Errors                              3                    0                    3
       CRC/FCS                           0                  n/a                    0
       Collision                       n/a                    0                    0
       Runts                             0                  n/a                    0
       Giants                            0                  n/a                    0

    We also found some strange log messages:

    LAG lag1 not sending LACPDUs through interface 1/1/1 because VSX information is not complete

    This was on friday

    since than we see random messages 

    2023-11-19T09:41:14.474185+01:00 MP-CR-2 intfd[1833]: Event|403|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is up
    2023-11-19T09:41:14.726845+01:00 MP-CR-2 intfd[1833]: Event|404|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is down
    2023-11-19T09:41:15.258492+01:00 MP-CR-2 intfd[1833]: Event|403|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is up
    2023-11-19T09:41:15.276894+01:00 MP-CR-2 lacpd[1855]: Event|1313|LOG_INFO|AMM|1/1|LAG 1 set as VSX
    2023-11-19T09:41:15.278340+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALIO, Partner state PLIO
    2023-11-19T09:41:15.279008+01:00 MP-CR-2 lacpd[1855]: Event|1310|LOG_WARN|AMM|1/1|Partner is out of sync for interface 1/1/1 LAG sport: 1. Actor state: ALFOX, partner state PSFO
    2023-11-19T09:41:15.282643+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state PLIO
    2023-11-19T09:41:15.284324+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state PLFO
    2023-11-19T09:41:15.285910+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOE, Partner state PLFO
    2023-11-19T09:41:15.287560+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOE, Partner state PSFO
    2023-11-19T09:41:15.289446+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOX, Partner state PSFO
    2023-11-19T09:41:16.092488+01:00 MP-CR-2 lacpd[1855]: Event|1309|LOG_INFO|AMM|1/1|Partner is detected for interface 1/1/1 LAG 1 : 65534,0c:97:5f:b8:b7:c0. Actor state: ALFOX, partner state ASFOE
    2023-11-19T09:41:16.092632+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOX, Partner state ASFOE
    2023-11-19T09:41:16.096170+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state ASFOE
    2023-11-19T09:41:16.098525+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state ASFOX
    2023-11-19T09:41:18.015712+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFN, Partner state ASFOX
    2023-11-19T09:41:18.062342+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFN, Partner state ASFN
    2023-11-19T09:41:18.064767+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFNC, Partner state ASFN
    2023-11-19T09:41:18.066970+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFNC, Partner state ASFNC
    2023-11-19T09:41:18.067962+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFNCD, Partner state ASFNC
    2023-11-19T09:41:18.069300+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFNCD, Partner state ASFNCD
    2023-11-19T09:41:18.092019+01:00 MP-CR-2 hpe-vsxd[1854]: Event|7013|LOG_INFO|AMM|1/1|VSX 1 state local up, remote up
    2023-11-19T09:41:19.097359+01:00 MP-CR-2 lldpd[1845]: Event|104|LOG_INFO|AMM|1/1|LLDP neighbor 0c:97:5f:b8:b7:c0 added on 1/1/1
    2023-11-19T09:41:26.555510+01:00 MP-CR-2 intfd[1833]: Event|404|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is down
    2023-11-19T09:41:26.579158+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state ASFNCD
    2023-11-19T09:41:26.581829+01:00 MP-CR-2 hpe-vsxd[1854]: Event|7012|LOG_INFO|AMM|1/1|VSX 1 state local down, remote up
    2023-11-19T09:41:27.117290+01:00 MP-CR-2 intfd[1833]: Event|403|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is up
    2023-11-19T09:41:28.135912+01:00 MP-CR-2 lacpd[1855]: Event|1313|LOG_INFO|AMM|1/1|LAG 1 set as VSX
    2023-11-19T09:41:28.136025+01:00 MP-CR-2 lacpd[1855]: Event|1310|LOG_WARN|AMM|1/1|Partner is out of sync for interface 1/1/1 LAG sport: 1. Actor state: ALFOX, partner state PSFO
    2023-11-19T09:41:28.137299+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALIO, Partner state PLIO
    2023-11-19T09:41:28.141863+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state PLIO
    2023-11-19T09:41:28.143614+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFO, Partner state PLFO
    2023-11-19T09:41:28.145421+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOE, Partner state PLFO
    2023-11-19T09:41:28.148575+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOE, Partner state PSFO
    2023-11-19T09:41:28.149136+01:00 MP-CR-2 intfd[1833]: Event|404|LOG_INFO|AMM|1/1|Link status for interface 1/1/1 is down
    2023-11-19T09:41:28.150376+01:00 MP-CR-2 lacpd[1855]: Event|1321|LOG_INFO|AMM|1/1|LAG 1 State change for interface 1/1/1: Actor state: ALFOX, Partner state PSFO



  • 2.  RE: 8325 VSX MC-Lag to 6300M

    Posted Nov 20, 2023 10:01 AM

    Hi

     

    Can you please confirm the 4 downstream switches connected to the 8325 VSX cluster are a stack (VSF) of 4 6300M and this stack is properly configured and working?

    Furthermore, please share the configuration of "the other side" (6300M) as well and the output of "show vsx status", "show lacp interface" and "show int brief".

     

    Regards,

    Thomas

     




    Original Message


  • 3.  RE: 8325 VSX MC-Lag to 6300M

    Posted Nov 20, 2023 10:50 AM

    6300 Stack

    show vsf 

    Force Autojoin             : Disabled
    Autojoin Eligibility Status: Not Eligible
    MAC Address                : 0c:97:5f:b8:b7:c0
    Secondary                  : 2
    Topology                   : Ring
    Status                     : No Split
    Split Detection Method     : None


    Mbr Mac Address         type           Status   
    ID
    --- ------------------- -------------- ---------------
    1   xxxxxxxxxxxxxxxx   JL658A         Conductor
    2   yyyyyyyyyyyyyyyyy   JL658A         Standby
    3  zzzzzzzzzzzzzzzzzz   JL661A         Member

    4   aaaaaaaaaaaaaa   JL661A         Member

     sh interface lag 2

    Aggregate lag2 is up 
     Admin state is up 
     Description : 
     MAC Address                 : 0c:97:5f:b8:b7:c0 
     Aggregated-interfaces       : 1/1/28 2/1/28 3/1/52 4/1/52 
     Aggregation-key             : 2
     Aggregate mode              : active 
     Speed                       : 50000 Mb/s 
     qos trust none
     VLAN Mode: native-untagged
     Native VLAN: 1
     Allowed VLAN List: all
     L3 Counters: Rx Disabled, Tx Disabled 

     sh interface lag 2

    Aggregate lag2 is up 
     Admin state is up 
     Description : 
     MAC Address                 : xxxxxxxxxxxxxxxxxx 
     Aggregated-interfaces       : 1/1/28 2/1/28 3/1/52 4/1/52 
     Aggregation-key             : 2
     Aggregate mode              : active 
     Speed                       : 50000 Mb/s 
     qos trust none
     VLAN Mode: native-untagged
     Native VLAN: 1
     Allowed VLAN List: all
     L3 Counters: Rx Disabled, Tx Disabled 

    MP-DS-01# sh interface transceiver 
    ------------------------------------------------------------------
    Port      Type           Product      Serial          Part        
                             Number       Number          Number      
    ------------------------------------------------------------------ 
    1/1/25    50G-DAC0.65    R0M46A       CN27L6W6KG      8121-1800   
    1/1/26    50G-DAC0.65    R0M46A       CN27L6W7NG      8121-1800   
    1/1/28    25G-eSR        JL485A       F7ALGAM         1990-4723   
    2/1/25    50G-DAC0.65    R0M46A       CN27L6W7NG      8121-1800   
    2/1/26    50G-DAC0.65    R0M46A       CN27L6W6KM      8121-1800   
    2/1/28    25G-eSR        JL485A       F7ALGAT         1990-4723   
    3/1/49    50G-DAC0.65    R0M46A       CN27L6W6KM      8121-1800   
    3/1/50    50G-DAC0.65    R0M46A       CN27L6W7MT      8121-1800   
    3/1/52    25G-eSR        JL485A       F7ALGAN         1990-4723   
    4/1/49    50G-DAC0.65    R0M46A       CN27L6W7MT      8121-1800   
    4/1/50    50G-DAC0.65    R0M46A       CN27L6W6KG      8121-1800   
    4/1/52    25G-eSR        JL485A       F7ALGAL         1990-4723 

    On 8325-1

    MP-CR-1# sh vsx status 
    VSX Operational State
    ---------------------
      ISL channel             : In-Sync
      ISL mgmt channel        : operational
      Config Sync Status      : In-Sync
      NAE                     : peer_reachable
      HTTPS Server            : peer_reachable

    Attribute           Local               Peer
    ------------        --------            --------
    ISL link            lag256              lag256
    ISL version         2                   2
    System MAC          xxxxxxxxx:16:00   xxxxxxxxxxxxx16:00
    Platform            8325                8325
    Software Version    GL.10.10.1090       GL.10.10.1090
    Device Role         primary             secondary

    MP-CR-1# sh interface transceiver 
    ------------------------------------------------------------------
    Port      Type           Product      Serial          Part        
                             Number       Number          Number      
    ------------------------------------------------------------------
    1/1/1     25G-eSR        JL485A       F7ALGAP         1990-4723   
    1/1/2     25G-eSR        JL485A       F7ALGAQ         1990-4723   
    1/1/4     25G-eSR        JL485A       F7ALGAS         1990-4723   
    1/1/5     25G-eSR        JL485A       F7ALGAR         1990-4723   
    1/1/47    10G-SR         J9150D       OS210602291     1990-3878   
    1/1/48    10G-SR         J9150D       CN88KBV479      1990-4391   
    1/1/55    100G-FR1       R9B63A       CN34L8L0C9      1990-4825   
    1/1/56    100G-FR1       R9B63A       CN34L8L0BV      1990-4825 

    On 8325-2

    VSX Operational State
    ---------------------
      ISL channel             : In-Sync
      ISL mgmt channel        : operational
      Config Sync Status      : In-Sync
      NAE                     : peer_reachable
      HTTPS Server            : peer_reachable

    Attribute           Local               Peer
    ------------        --------            --------
    ISL link            lag256              lag256
    ISL version         2                   2
    System MAC          xxxxxxxxx16:00   xxxxxxxxxxxx:16:00
    Platform            8325                8325
    Software Version    GL.10.10.1090       GL.10.10.1090
    Device Role         secondary           primary

    MP-CR-2# sh interface transceiver 
    ------------------------------------------------------------------
    Port      Type           Product      Serial          Part        
                             Number       Number          Number      
    ------------------------------------------------------------------
    1/1/1     25G-eSR        JL485A       F7ALGCM         1990-4723   
    1/1/2     25G-eSR        JL485A       F7ALGCP         1990-4723   
    1/1/4     25G-eSR        JL485A       F7ALGCL         1990-4723   
    1/1/5     25G-eSR        JL485A       F7ALGCN         1990-4723   
    1/1/47    10G-SR         J9150D       CN98KJW2GW      1990-4634   
    1/1/48    10G-SR         J9150D       OS210602293     1990-3878   
    1/1/55    100G-FR1       R9B63A       CN34L8L0C7      1990-4825   
    1/1/56    100G-FR1       R9B63A       CN34L8L0CT      1990-4825 


    Original Message


  • 4.  RE: 8325 VSX MC-Lag to 6300M

    Posted Nov 20, 2023 11:09 AM

    Thanks.

     

    That looks quite good apart from the fact that the lag "only" shows 50G (I'd expect 100G here, at least on the stack side).

    What about the output of "show lacp inter" and "show int brief" on the stack, and the two 8325?

     

    Regards,

    Thomas

     




    Original Message


  • 5.  RE: 8325 VSX MC-Lag to 6300M

    Posted Nov 22, 2023 04:46 AM
    Edited by Steinar Grande Nov 22, 2023 04:48 AM

    I agree, it looks correct, but the aggregated lag speed in your 6300 VSF stack should have been (25*4) 100 Mb/s,

    as for me indicate your members 2 & 4 is not recognizing the two interfaces, at the time of your output command.

    I assume you have to go through some old fashion investigating, starting with layer 0, and changing transceiver/ports/cables etc, in an orderly manner. 

    Good luck



    ------------------------------
    Steinar
    ------------------------------

    Original Message


Related Content