Controllerless Networks

Hi!

Can anybody help me in this situation? After we deploy one IAP325 and one IAP215 ( each of them on different switch ) in cluster controllerless mode, our network switches is seem unstable:

• The uplink of the edge switch connect to the core switch is down/up in unexpect time ( random ), when trace the log of the two IAP it show it lose connect to each other and each of them is consider master and after two or three minutes the problem is resolved itself.

The problems cause the degrade of our infra network ( the uplink is down cause all of the switches of this branch can't accessable ). I am very new in ARUBA wifi networking so is there anyone can help me how to troubleshoot this situation?

Is the uplink to the core stable again when you shutdown both APs ?

Start troubleshooting from the lowerst ISO layer up.

• What is your ArubaOS instant release version
• What is your switch type and firmware version
• How is it connected to the coreswitch, LACP link aggregation or not
• Is spanning-tree enabled
• can you share the edgeswitch loggiing  "show logging -r".
• can you share the uplink interface statistics "show interface #.

First thing i think about is you probaly create some mesh point between both APs. Take also a look at this topic https://community.arubanetworks.com/t5/Controllerless-Networks/IAP-315-STP-Blocked/td-p/440040

If this is a urgent issue please open a TAC case by Aruba Support.

Hi mkk!

Here are the info for your requests:

Is the uplink to the core stable again when you shutdown both APs ?

• The problems are random ( We monitor the up/down by ping every 30s and see the problems auto resovled in ~2 minutes ). If we shutdown both APs , the users cannot connect to our site to work because most of them is using mobile devices. ( We maybe consider it if the situation is worse )

- Start troubleshooting from the lowerst ISO layer up.

What is your ArubaOS instant release version?

ARUBA83-c5:ba:9a# show version
Aruba Operating System Software.
ArubaOS (MODEL: 325), Version 8.4.0.5
Website: http://www.arubanetworks.com
(c) Copyright 2019 Hewlett Packard Enterprise Development LP.
Compiled on 2019-10-22 at 12:32:12 UTC (build 72830) by p4build
FIPS Mode :disabled

AP uptime is 6 days 18 hours 40 minutes 23 seconds
Reboot Time and Cause: AP rebooted Tue Dec 24 00:43:08 UTC 2019; System cmd at uptime 33D 15H 29M 33S: Image Upgrade Successful

ARUBA80-ce:13:e2# show version
Aruba Operating System Software.
ArubaOS (MODEL: 215), Version 8.4.0.5
Website: http://www.arubanetworks.com
(c) Copyright 2019 Hewlett Packard Enterprise Development LP.
Compiled on 2019-10-22 at 13:00:19 UTC (build 72830) by p4build
FIPS Mode :disabled

AP uptime is 6 hours 11 minutes 41 seconds
Reboot Time and Cause: AP rebooted Mon Dec 30 13:11:09 UTC 2019; System cmd at uptime 1D 1H 20M 44S: master transitioned to local (new master 10.10.10.83, uplink flaps 0, max beacon miss 21), uptime from being master 29 seconds.

What is your switch type and firmware version?

CORE SPINE ( Dell S6000-ON - Cumulus 3.3.2 ) <-> Leaf216 Switch ( N1524-Firmware 6.3.2.4 ) ( Port7-ARUBA80 )
<-> Leaf218 Switch ( N1548-Firmware 6.3.2.4 ) ( Port17-ARUBA83 )

How is it connected to the coreswitch, LACP link aggregation or not?

Leaf216 - No Aggregation
Leaf218 - LACP Link aggreation with two ten gigabitethernet uplink

Is spanning-tree enabled?

Yes the spanning tree is enabled and configure to RSTP mode.

can you share the edgeswitch loggiing "show logging -r".?

The log is very long so i upload as file for you investigate.

can you share the uplink interface statistics "show interface #.?

leaf216#show interfaces tengigabitethernet 1/0/3

Interface Name................................. Te1/0/3
SOC Hardware Info.............................. BCM56150_A0
Link Status.................................... Up /None
Keepalive Enabled.............................. FALSE
Err-disable Cause.............................. None
VLAN Membership Mode........................... Trunk Mode
VLAN Membership................................ (1),2-4096
MTU Size....................................... 1518
Port Mode [Duplex]............................. Full
Port Speed..................................... 10000
Link Debounce Flaps............................ 0
Auto-Negotation Status......................... Off
Burned MAC Address............................. E4F0.047A.D8BC
L3 MAC Address................................. E4F0.047A.D8BD
Sample Load Interval........................... 300
Received Input Rate Bits/Sec................... 438544
Received Input Rate Packets/Sec................ 483
Transmitted Input Rate Bits/Sec................ 1079648
Transmitted Input Rate Packets/Sec : .......... 226
Total Packets Received Without Errors.......... 10797877379
Unicast Packets Received....................... 6185770825
Multicast Packets Received..................... 657183560
Broadcast Packets Received..................... 3954922994
Total Packets Received with MAC Errors......... 0
Jabbers Received............................... 0
Fragments/Undersize Received................... 0
Alignment Errors............................... 0
FCS Errors..................................... 0
Overruns....................................... 0
Total Received Packets Not Forwarded........... 1756007234
Total Packets Transmitted Successfully......... 3781030834
Unicast Packets Transmitted.................... 3767233192
Multicast Packets Transmitted.................. 4521176
Broadcast Packets Transmitted.................. 9276466
Transmit Packets Discarded..................... 0
Total Transmit Errors.......................... 0
Total Transmit Packets Discarded............... 0
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0

leaf216#show interfaces gigabitethernet 1/0/7

Interface Name................................. Gi1/0/7
SOC Hardware Info.............................. BCM56150_A0
Link Status.................................... Up /None
Keepalive Enabled.............................. FALSE
Err-disable Cause.............................. None
VLAN Membership Mode........................... Trunk Mode
VLAN Membership................................ (10),1-9,11-4096
MTU Size....................................... 1518
Port Mode [Duplex]............................. Full
Port Speed..................................... 1000
Link Debounce Flaps............................ 0
Auto-Negotation Status......................... Auto
Burned MAC Address............................. E4F0.047A.D8BC
L3 MAC Address................................. E4F0.047A.D8BD
Sample Load Interval........................... 300
Received Input Rate Bits/Sec................... 1039136
Received Input Rate Packets/Sec................ 212
Transmitted Input Rate Bits/Sec................ 376544
Transmitted Input Rate Packets/Sec : .......... 379
Total Packets Received Without Errors.......... 2426485149
Unicast Packets Received....................... 2421299835
Multicast Packets Received..................... 2108437
Broadcast Packets Received..................... 3076877
Total Packets Received with MAC Errors......... 0
Jabbers Received............................... 0
Fragments/Undersize Received................... 0
Alignment Errors............................... 0
FCS Errors..................................... 0
Overruns....................................... 0
Total Received Packets Not Forwarded........... 91212
Total Packets Transmitted Successfully......... 3304052445
Unicast Packets Transmitted.................... 2538011770
Multicast Packets Transmitted.................. 43090035
Broadcast Packets Transmitted.................. 722950640
Transmit Packets Discarded..................... 345244
Total Transmit Errors.......................... 0
Total Transmit Packets Discarded............... 345244
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0

leaf218#show interfaces gigabitethernet 1/0/17

Interface Name................................. Gi1/0/17
SOC Hardware Info.............................. BCM56150_A0
Link Status.................................... Up /None
Keepalive Enabled.............................. FALSE
Err-disable Cause.............................. None
VLAN Membership Mode........................... Trunk Mode
VLAN Membership................................ (10),1-9,11-4096
MTU Size....................................... 1518
Port Mode [Duplex]............................. Full
Port Speed..................................... 100
Link Debounce Flaps............................ 0
Auto-Negotation Status......................... Auto
Burned MAC Address............................. E4F0.04D8.9314
L3 MAC Address................................. E4F0.04D8.9315
Sample Load Interval........................... 300
Received Input Rate Bits/Sec................... 2844280
Received Input Rate Packets/Sec................ 507
Transmitted Input Rate Bits/Sec................ 385720
Transmitted Input Rate Packets/Sec : .......... 519
Total Packets Received Without Errors.......... 1328243746
Unicast Packets Received....................... 1321948708
Multicast Packets Received..................... 995783
Broadcast Packets Received..................... 5299255
Total Packets Received with MAC Errors......... 109
Jabbers Received............................... 0
Fragments/Undersize Received................... 0
Alignment Errors............................... 0
FCS Errors..................................... 0
Overruns....................................... 0
Total Received Packets Not Forwarded........... 36336
Total Packets Transmitted Successfully......... 1714509842
Unicast Packets Transmitted.................... 1216567650
Multicast Packets Transmitted.................. 82898299
Broadcast Packets Transmitted.................. 415043893
Transmit Packets Discarded..................... 480562
Total Transmit Errors.......................... 0
Total Transmit Packets Discarded............... 480562
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0

First thing i think about is you probaly create some mesh point between both APs. Take also a look at this topic https://community.arubanetworks.com/t5/Controllerless-Networks/IAP-315-STP-Blocked/td-p/440040
Thank for your sharing info with me, i will look at it. Thanks!

Attachment(s)

20191230-leaf216-bufferlogs.txt   53 KB 1 version

20191230-leaf218-bufferlogs.txt   51 KB 1 version

Hi Anhdo,

Please upgrade to ArubaOS Instant 8.5.x.x or 8.6.x.x., your 8.4.x.x release is end-of-support on 31 december 2019, today ;). link

At the interface 1/0/3 of leaf216 the broadcast frames are 40% of all received frames, thats much to high.

Did you check the logging of the instant cluster?

• show log errorlog all 
• show log system all

I think your option is to open a TAC case on this.

Hi mkk!

Wish you a happy new year!

I cannot do the upgrade to version 8.5.x as you suggest because using the check new firmware version it show nothing ( please see the attach file ).

About the broadcast packet too much maybe the device is at production line and users test their device by plugin plugout. Anyway, i will note and monitor about this.

Hi Anhdo,

You can download and install the new firmware manualy.

Firmware can be download form https://asp.arubanetworks.com

I send you a PM with instructions.

Hi mkk!

Thank you for send me the link. I will look at it and do some test before applied into the production line. Hope the situation better. I will update when i have any news. Thanks!