Wired Intelligent Edge

Hi friends.
Is there any restriction about the number of member links of the ISL in a VSX deployment? ... I will use 40GE ports for ISL.
Regards

ISL has only one link, that is either a physical link or a LAG.

Then up to 8 links can configured part of the LAG.

Consequently up to 8 physical links can support ISL communication.

8x10 or 8x40 or 8x100.

The rule of thumb for ISL BW is to plan enough BW for a filure scenario

like supporting the uplink load over the ISL in case one of the uplink fails (this is one example among others). If you plan 1x40G uplink per VSX node to connect upstream, I would recommend to use 2x40G for ISL.

ISL protocol as such won't consumme a lot of BW.

Hi Vicent.

I have search de VSX configuration guide for the Aruba 8400 and have not found anything about ISL physical member links limit.

 

The only restiction I have found is about the VSX LAG:

"A VSX LAG supports a maximum of four member links per switch segment. A VSX LAG across a downstream switch can have at most a total of eight member links"

 

But it seems this talks about LAG between 2 core switches in VSX and a Distribution/Access Switch.

 

Regards

 

Please refer to p10 of LAG config guide that mentions a LAG can have up to eight physical interfaces.

Please note that to create ISL connectivity you use one regular LAG,

that can have up to 8 ports.

The limit of 4 you mentioned is the number of physical ports you can assign on VSX primary and on VSX secondary, for a total of 8, in a VSX LAG.

As @vincent.giles mentioned above, since the VSX ISL can be deployed also using a LAG approach (so the resulting LAG is just a classic non Multi-Chassis LAG just a little bit special due its VSX ISL assigned role) then classic LAG's restrictions/requirements will apply...those ones include a maximum of 8 member interfaces for each LAG (and, I add, the maximum LAG Id number that can be assigned will be 128, IIRC <- there is a non written best practice that suggest to deploy the VSX ISL as LAG using the Id 128...but this detail is just beyond your initial question).

Thank you Vincent and Parnassus.

Pretty clear.

As you already said, creating an ISL makes reference to an LAG group, so, the limit is imposed by the capacity of the LAG group, in this case: 8 physical links per LAG group, as the LAG configuration guide says.

 

Here you can see an example of configuration so that anybody could understand what Vincent and Parnassus talk about:

---------------------------------------------

! LAG for Inter-Switch-Link
  interface lag 128
    no shutdown
    no routing
    lacp mode active
    vlan trunk native 10 tag
    vlan trunk allowed 10,20
  interface 1/1/2
    no shutdown
    mtu 2068
    lag 128
  interface 1/1/3
    no shutdown
    mtu 2068
    lag 128
    exit

! Inter-Switch-Link setting
  vsx
    inter-switch-link lag 128
    keepalive peer 1.1.2.2 source 1.1.2.1

---------------------------------------------------------

Steps:

• Create a LAG (LAG 128 in the example above)
• Assign member ports to the LAG (interface 1/1/2 and interface 1/1/3)
• Finally, tell the VSX that the ISL will use LAG 128.

Regards

 

 

Mine VSX ISL configuration actually is this one:

 

interface lag 128
    vsx-sync vlans
    description VSX-ISL-LAG128-Link
    no shutdown
    no routing
    vlan trunk native 1 tag
    vlan trunk allowed all
    lacp mode active

VSX ISL Member Interfaces in my case are 4x40Gbps:

 

interface 1/1/49 
    description 8320-1-VSX-ISL-1-1-49
    no shutdown
    mtu 9198
    lag 128
interface 1/1/50 
    description 8320-1-VSX-ISL-1-1-50
    no shutdown
    mtu 9198
    lag 128
interface 1/1/52 
    description 8320-1-VSX-ISL-1-1-52
    no shutdown
    mtu 9198
    lag 128
interface 1/1/53 
    description 8320-1-VSX-ISL-1-1-53
    no shutdown
    mtu 9198
    lag 128

VSX configuration includes:

 

vsx
    inter-switch-link lag 128
    role primary
    keepalive peer 10.250.250.2 source 10.250.250.1 vrf VSX-ISL-Keepalive
    linkup-delay-timer 35

As you can see I lowered the default linkup-delay-timer to 35 seconds (pretty enough on my environment) and my dedicated VRF for VSX ISL Keepalive is called "VSX-ISL-Keepalive".

 

VSX Keepalive interface (actually just a single link) is:

 

interface 1/1/48 
    description ISL-Keepalive
    no shutdown
    vrf attach VSX-ISL-Keepalive
    ip address 10.250.250.1/30

even if I plan to use a 2 ports LAG.

Hi Parnassus,

Thanks for sharing your configuration with us, it has more details than mine and it will be very helpful.
Pals, finally the idea is the same, just with more details and suggestions from Parnassus.

Parnassus,

I would like to harden the keepalive link using a LAG group with 2 physical ports (using your configuration example), this would be my configuration for the primary vsx member, would you mind suggesting me any other add?:


interface lag 10
description ISL-Keepalive
no shutdown
vrf attach VSX-ISL-Keepalive
ip address 10.250.250.1/30
lacp mode active

interface 1/1/1
description 8320-1-ISL-Keepalive-1-1-1
no shutdown
mtu 9198
lag 10

interface 1/1/48
description 8320-1-ISL-Keepalive-1-1-48
no shutdown
mtu 9198
lag 10

vsx
inter-switch-link lag 128
role primary
keepalive peer 10.250.250.2 source 10.250.250.1 vrf VSX-ISL-Keepalive
linkup-delay-timer 35

Have a nice day pals

Hi!

 

I was discussing exactly that approach on the thread called ArubaOS-CX 10.01 VSX: Keep Alive deployed using a Layer 3 dynamic aggregation group? started during July 2018 (!)...time go by...and, finally, now I'm waiting four new J4858D SFP Transceivers to make that idea reality (actually my VSX ISL Keepalive single link is using two J4858C SFP Transceivers installed on both 1/1/48 interfaces).

Cool post.

Very similar configs.

My scenario is 02 buildings separated 350m.

Each building will deploy 01 Aruba 8400 in VSX configuration with the other one.

Customer used to have an OM4 fiber route between buildings (06 strands) but they had an accident and the 06 strands were broken. So,  implementing redundancy scheme requires 02 routes of fiber optics:

 

Route 01:

• Interface 1 on 8400-1 (10GE-SR) to Interface 1 on 8400-2 (10GE-SR) for ISL
• Interface 2 on 8400-1 (1GE-SX) to Interface 2 on 8400-2 (1GE-SX) for Keep-Alive

 

Route 02:

• Interface 3 on 8400-1 (10GE-SR) to Interface 3 on 8400-2 (10GE-SR) for ISL
• Interface 4 on 8400-1 (1GE-SX) to Interface 4 on 8400-2 (1GE-SX) for Keep-Alive

 

ISL: LAG 128 (Interfaces 1 and 3)

Keep-Alive: Lag 127 (Interfaces 2 and 4)

 

That way, if a similar accident happens and route 01 breaks, you always have route 02 working with both, a physical link for ISL and a physical link for keep-alive.

 

Some customers like banks are very exigent with redundancy for deployment.

 

Thanks a lot for you input.

 

Have a nice day.

 

Yeah, having two distinct well separated paths interconnecting two sites (you can extend the same approach inside a single DC!) is essential to have resiliency at every level (from the Power Supply modules, to Interface Modules, to Management Modules, Fans Modules, Copper/Fiber optic runs, Power Supply lines...and so on).

 

The point of having a VSX where the Primary is installed on one location and the VSX Secondary is installed on the other (even if well connected with dual redundant paths, each one following a different track) is OK...but you those amount of fiber optics cables can't be enough (for VSX ISL and for VSX ISL Keepalive), what's about all the VSX LAGs (aren't you planning them?) you're going to serve for Access/Distribution layers?

 

Maybe you have just left that detail hidden for sake of semplicity...in any case you will need a lot more fibers between your two sites.

Hi Parnassus,

 

Detail hidden for simplicity. VSX LAGs will be deployed towards 2930M stacks (02 different routes to each stack).

 

Thanks for your suggestions.

 

Have an excellent day.