Wired

last person joined: 22 minutes ago 

Bring performance and reliability to your network with the Aruba Core, Aggregation, and Access layer switches. Discuss the latest features and functionality of the ArubaOS-Switch and ArubaOS-CX devices, and find ways to improve security across your network to bring together a mobile first solution.
Expand all | Collapse all

ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

This thread has been viewed 3 times
  • 1.  ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 08, 2018 06:33 AM

    Hello,

     

    What's the logic that is ruling the VSX ISL ports' sequence output provided by the show vsx lacp aggregates command (below an example on the VSX I've setup, command executed on primary Aruba 8320 node):

     

    Aruba-8320-1# show vsx lacp aggregates lag128
                        Local-peer                        Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate-name    : lag128                            lag128
    Interfaces        : 1/1/49 1/1/52 1/1/53 1/1/50       1/1/50 1/1/53 1/1/49 1/1/52 
    Heartbeat rate    : slow                              slow
    Hash              : l3-src-dst                        l3-src-dst
    Aggregate mode    : active                            active

    Note that show vsx lacp aggregates lag128 vsx-peer command shows the same sequence on each respective peer (clearly local/remote peer's references are swapped)...VSX is In-Sync, Keepalive is operational and DAC Cables used for VSL ISL were manually connected sequencially (first 1/1/49 to 1/1/49, few seconds after 1/1/50 to 1/1/50, then 1/1/52 to 1/1/52 and finally 1/1/53 to 1/1/53...1/1/51 and 1/1/54 were not used at all).

     

    Also note the (stripped) output of show interface brief:

     

    1/1/47    --      routed --          no      down    No XCVR installed      --    
    1/1/48    --      routed 1000SX      yes     up                             1000  
    1/1/49    1       trunk  QSFP+DA1    yes     up                             40000 
    1/1/50    1       trunk  QSFP+DA1    yes     up                             40000 
    1/1/51    --      routed --          no      down    No XCVR installed      --    
    1/1/52    1       trunk  QSFP+DA1    yes     up                             40000 
    1/1/53    1       trunk  QSFP+DA1    yes     up                             40000 
    1/1/54    --      routed --          no      down    No XCVR installed      --    
    lag128    1       trunk  --          yes     up      --                     160000  

    And this show lldp neighbor-info vsx-peer that shows that four ports I used in VSX ISL, as seen from the remote VSX peer, are all conneted sequencially (49 with 49, 50 with 50 and so on...) and aren't crossed:

     

    Aruba-8320-1# show lldp neighbor-info vsx-peer 
    
    LLDP Neighbor Information 
    =========================
    
    Total Neighbor Entries          : 5
    Total Neighbor Entries Deleted  : 0
    Total Neighbor Entries Dropped  : 0
    Total Neighbor Entries Aged-Out : 0
    
    LOCAL-PORT  CHASSIS-ID         PORT-ID       PORT-DESC     TTL      SYS-NAME    
    --------------------------------------------------------------------------------
    1/1/48      d0:67:26:xx:xx:xx  1/1/48        1/1/48        120      Aruba-8320-1
    1/1/49      d0:67:26:xx:xx:xx  1/1/49        1/1/49        120      Aruba-8320-1
    1/1/50      d0:67:26:xx:xx:xx  1/1/50        1/1/50        120      Aruba-8320-1
    1/1/52      d0:67:26:xx:xx:xx  1/1/52        1/1/52        120      Aruba-8320-1
    1/1/53      d0:67:26:xx:xx:xx  1/1/53        1/1/53        120      Aruba-8320-1

    (Port 1/1/48 on both ends is the Keepalive link).

     

    I'm really curious...

     

     



  • 2.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 08, 2018 12:14 PM

    Interesting question!

     

    I took a quick look at a VSX pair of 8320s in the TME lab and I am seeing the same behavior you observed.  What is more interesting is that the LAG we are using for ISL (lag 10) has three interfaces configured as members.  I've removed all but one cable to see if any changes occur and the output of 'show vsx lacp interface lag10' consistently shows three members in the LAG with the local peer listing the ports "out of sequence" and the remote-peer list the ports as I would expect:

     

    SWHQ-AGG1A# sh vsx lacp aggregates lag10
    Local-peer Remote-peer
    -------------------------------------------------------------------------------

    Aggregate-name : lag10 lag10
    Interfaces : 1/1/52 1/1/53 1/1/51 1/1/51 1/1/52 1/1/53

    Heartbeat rate : slow slow
    Hash : l3-src-dst l3-src-dst
    Aggregate mode : active active

     

     

    I'll ping some contacts in developmenet and share back what I find.

     

    To confirm, you didn't notice any problems/issues with packet forwarding just in the output of the 'show' commands?

     

    -t



  • 3.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 09, 2018 07:05 AM

    Hi, after a full reboot of the VSX by performing this sequence:

     

    1 - Power-down Aruba 8320-1 (Primary)

    2 - Power-down Aruba 8320-2 (Secondary)

    3 - Physically relocate both Aruba 8320 into definitive Rack position

    4 - Power-up Aruba 8320-1 (Primary)

    5 - Power-up Aruba 8320-2 (Secondary) about 20 seconds after powering up Aruba 8320-1 (so the first ready is the Primary).

     

    Now the VSX ISL LAG 128 (lag128) member ports' sequence on Secondary peer looks like this:

     

    Aruba-8320-2# show vsx lacp aggregates lag128
                        Local-peer                        Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate-name    : lag128                            lag128
    Interfaces        : 1/1/50 1/1/49 1/1/53 1/1/52       1/1/52 1/1/49 1/1/50 1/1/53 
    Heartbeat rate    : slow                              slow
    Hash              : l3-src-dst                        l3-src-dst
    Aggregate mode    : active                            active

    and, conversely, the Primary peer - queried by the Secondary - shows that:

     

    Aruba-8320-2# show vsx lacp aggregates lag128 vsx-peer
                        Local-peer                        Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate-name    : lag128                            lag128
    Interfaces        : 1/1/50 1/1/49 1/1/53 1/1/52       1/1/52 1/1/49 1/1/50 1/1/53
    Heartbeat rate    : slow                              slow
    Hash              : l3-src-dst                        l3-src-dst
    Aggregate mode    : active                            active

    So it looks like the sequence ordering changes after a reboot (DAC Cables are still connected sequencially: 49 with 49, 50 with 50, 52 with 52 and 53 with 53).

     

    Note how the running-config (JSON) sequence about lag128's member ports is represented on the Secondary:

     

     "lag128": {
                "admin": "up",
                "description": "VSX-ISL-LAG128-Link",
                "interfaces": [
                    "1%2F1%2F50",
                    "1%2F1%2F49",
                    "1%2F1%2F53",
                    "1%2F1%2F52"
                ],
                "lacp": "active",
                "name": "lag128",
                "other_config": {
                    "mclag_enabled": "false"
                },
                "vlan_mode": "native-tagged",
                "vlan_tag": "1",
                "vsx_sync": [
                    "^vlan.*"
                ]
            }

    I didn't notice any issues with packet forwarding BUT that's because simply the VSX I'm working with is still in its staging phase so no Multi-Chassis LAGs were defined (so there is no usage at Layer 2 level...), I'm going to start with deployment in a few days and then I hope the setup will be bullet-proof.



  • 4.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 09, 2018 11:27 AM

    Hi Davide,

     

    I'll share your additional observations with development and report back when I have an update for you.

     

    I don't forsee any problems with your configuration -- especially as the 'show lacp interface' shows that you have a LAG formed between the VSX peers.

     

    I have a question for you:  We generally see a pair of 40G links used for the LAG supporting ISL.  Did a bandwidth requirement drive you to use 4x40G interfaces?  What is your  solution for the keepalive configuration?  If you have free interfaces, our leading practice is to create a keepalive VRF with a pair of interface in a LAG.  You can also use any L3 path between the VSX peers if you don't have free interfaces.  Here is a config sample:

     

    vrf VSX_KEEPALIVE

     

    interface 1/1/47
    no shutdown
    lag 11
    exit

     

    interface 1/1/48
    no shutdown
    lag 11
    exit

     

    interface lag 11
    vrf attach VSX_KEEPALIVE
    ip address 192.168.1.1/24
    lacp mode active
    exit

     

    vsx
    <config omitted>

    keepalive peer 192.168.1.2 source 192.168.1.1 vrf VSX_KEEPALIVE

     

    -t



  • 5.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 09, 2018 11:46 AM

    Todd thanks for your support!


    Our configuration was extensively discussed in other threads (basically our scenario is a Layer 2 isolated network with a lot of VSX LAGs AKA MC-LAGs to downstream servers and all-flash storages).

     

    Our setup is 4x40Gbps lag128 for VSX ISL (this part will truly deserve another thread...since I've few questions about the potential usage - call it how traffic is going to be distributed - of the ISL LAGs during normal operations, not only during a node failure) and the Keepalive was implemented on a separate VRF (I called it "VSX-ISL-Keepalive") and uses the 1/1/48 ports on both peers equipped with SFP J4858C HPE X121 1G SFP LC SX Transceiver interconnected using 0,5 meter fiber optic cable&nbsp;(Aruba told us it is enough for the rate of traffic the Keepalive will see).

     

    I asked if Keepalive (which is a routed interface) could/should be done using a LAG (Layer 3 LAG) instead of using just a single link...in my vision that idea was to enhance resiliency of the heartbeat between both peers...but it was suggested me that it was too much and not worth the the pair of additional SFP+ slots lost for that (my initial idea was to use a LAG of two SFP transceivers - say port 1/1/47 and 1/1/48 - to form the Keepalive's LAG...see this thread posted few days ago about that).

     

    So...up to now, what I've done looks good enough...to me.


    VSX is already formed...all is in the staging area (we postponed the go-live for first week of September) so I've some time to explore the system and report what I find.


    Kind regards, Davide.



  • 6.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 10, 2018 11:05 AM

    Hi Davide,

     

    Vincent is one of my TME peers.  He is right -- there is not a bandwidth need.  If you have  free interfaces and cables, there is no harm in using a LAG.  This does provide minimal extra protection as the keepalive is "important" during an ISL failure.  I'd use a LAG and call it "cheap insurance" 

     

    -t



  • 7.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 11, 2018 02:34 AM

    I agree with you (it was my initial idea), if I'll find two additional SFP J4858C Transceiver spares I'll rearrange ISL Keepalive configuration by converting it from single link to a two links' LAG.



  • 8.  RE: ArubaOS-CX 10.01 VSX: aggregated ports' output sequence for VSX ISL LAG and VSX LAGs.

    Posted Aug 13, 2018 10:47 AM

    It looks like that also VSX LAGs (Multi-Chassis LAGs) are affected by an unusual output sequence for member ports:

     

    Aruba-8320-1# show vsx lacp aggregates lag2
                        Local-peer                Remote-peer
    -------------------------------------------------------------------------------
    
    Aggregate-name    : lag2 (multi-chassis)      lag2 (multi-chassis)
    Interfaces        : 1/1/3 1/1/4               1/1/4 1/1/3
    Remote-interfaces : 1/1/4 1/1/3               1/1/3 1/1/4
    Heartbeat rate    : slow                      slow
    Hash              : l3-src-dst                l3-src-dst
    Aggregate mode    : active                    active

    Notice how the sequence is 1/1/3, 1/1/4 on Local peer (VSX Primary) and 1/1/4, 1/1/3 on Remote peer (VSX Secondary).