Wireless Access

Reply
Occasional Contributor I
Posts: 6
Registered: ‎10-27-2015

Why would IAP's change master and reboot without prompting?

So, we have several access points in five different buildings throughout the school district that I work for. They are broken up into clusters with 8-14 access points in each cluster, being managed by virtual controllers. The middle school is giving us some serious problems, and now the elementary and the high school are starting to show the same symptoms. The access point that was designated as the master keeps randomly rebooting and forcing all of the access points to reboot or to just drop. This is getting rather crazy. We talked to TAC and they were sorta helpful, but we are still working on them. I have a bunch of logs and such, we are getting tons of PAPI errors

 

PAPI_Send failed: papi_send_status_callback, 2879: Connection timed out; from 8451"

 

I was hoping that someone has seen this before and could possibly offer some insight. Thanks!

Guru Elite
Posts: 20,801
Registered: ‎03-29-2007

Re: Why would IAP's change master and reboot without prompting?

That message is not very deterministic, in itself.  It could be cosmetic in some cases.  You need to look for the reboot reason in the IAP, by typing "show version" on the commandline or use the "show tech-support" in the diagnostics tab:

 

show version
Aruba Operating System Software.
ArubaOS (MODEL: 225), Version 6.4.2.6-4.1.1.8
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-07-25 at 04:49:45 PDT (build 50989) by p4build

AP uptime is 3 weeks 5 days 21 hours 14 minutes 25 seconds
Reboot Time and Cause: AP rebooted Thu Oct 1 15:23:16 CDT 2015; Image Upgrade Successful
end of show version


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor I
Posts: 6
Registered: ‎10-27-2015

Re: Why would IAP's change master and reboot without prompting?

Ok, I've done that for a few of the access points in the area where we are having the most issues, and the results are as follows:

 

MS-Lab15-205-C# sho ver
Aruba Operating System Software.
ArubaOS (MODEL: 205), Version 6.4.2.6-4.1.1.10
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-09-23 at 00:11:51 PDT (build 51810) by p4build

AP uptime is 5 minutes 8 seconds
Reboot Time and Cause: unknown

 


MS-CafeNorth-105# sho ver
Aruba Operating System Software.
ArubaOS (MODEL: 105), Version 6.4.2.6-4.1.1.10
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-09-23 at 00:58:16 PDT (build 51810) by p4build

AP uptime is 5 minutes 7 seconds
Reboot Time and Cause: AP rebooted Thu Oct 29 13:34:33 PDT 2015; Gateway unreachable

 

Aruba Operating System Software.
ArubaOS (MODEL: 205), Version 6.4.2.6-4.1.1.10
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-09-23 at 00:11:51 PDT (build 51810) by p4build

AP uptime is 7 minutes 7 seconds
Reboot Time and Cause: unknown

 

MS-Wrestling-105# sho ver
Aruba Operating System Software.
ArubaOS (MODEL: 105), Version 6.4.2.6-4.1.1.10
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-09-23 at 00:58:16 PDT (build 51810) by p4build

AP uptime is 3 hours 20 minutes 41 seconds
Reboot Time and Cause: AP rebooted Thu Oct 29 10:20:06 PDT 2015; Image Upgrade S uccessful

 

MS-Staff2-105# sho ver
Aruba Operating System Software.
ArubaOS (MODEL: 105), Version 6.4.2.6-4.1.1.10
Website: http://www.arubanetworks.com
Copyright (c) 2002-2015, Aruba Networks, Inc.
Compiled on 2015-09-23 at 00:58:16 PDT (build 51810) by p4build

AP uptime is 7 minutes 23 seconds
Reboot Time and Cause: AP rebooted Thu Oct 29 13:34:21 PDT 2015; Gateway unreachable

 

 

 

 

Guru Elite
Posts: 20,801
Registered: ‎03-29-2007

Re: Why would IAP's change master and reboot without prompting?

For the unknown, I would check whatever is powering the access points to see if the power is being interrupted.  For the gateway unreachable, I would check to make sure that there are no duplicate ip addresses in your network.

 

Those are two general things.  The specifics on how you would check that depends on how your network is setup and what you are using for DHCP.

 



Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor I
Posts: 6
Registered: ‎10-27-2015

Re: Why would IAP's change master and reboot without prompting?

Power is not being interrupted by anything other that us trying to reboot the AP's to get them to work. They are all on Cisco POE catalyst switches. I intentionally gave configs for access points that are all on different switches too, and the IP addresses are reserved and set out of the DHCP scope. 

Guru Elite
Posts: 20,801
Registered: ‎03-29-2007

Re: Why would IAP's change master and reboot without prompting?

Okay.  The unknown is probably you cutting the power to reboot.  That leaves the gateway unreachable.  How many APs do you have in each cluster?  Are there any different clusters that share a subnet? (two clusters to a subnet)



Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor I
Posts: 6
Registered: ‎10-27-2015

Re: Why would IAP's change master and reboot without prompting?

This particular cluster has 12 IAP's. One has 10, another has 9. No less than 8 and no more than 12. All of the different clusters are on different subnets. 

Guru Elite
Posts: 20,801
Registered: ‎03-29-2007

Re: Why would IAP's change master and reboot without prompting?

So how long has this been happening? Did you deploy clusters successfully and then this started happening?


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor I
Posts: 6
Registered: ‎10-27-2015

Re: Why would IAP's change master and reboot without prompting?

This has been going on for a week now and getting progressively worse. This system has been in place for about a year, and this is the first time we have experienced anything like this. This have been working great, and we have been extremely satisfied on the performance of the access points... until last Thursday when this started happening for no apparent reason. We have upgraded firmware, rolled back firmware, all kinds of different things to no avail. We took all of the access points down last night and brought them back up one by one. They held solidly for around 12 hours, and at 7:41 this morning, all of the symptoms started coming back. Without anyone touching anything, I was the only one in the office when it started happening. 

 

Around every half hour, (all on the same version of code) the master will reboot. Just the master, then all of the PAPI messages will start. We started syslogging for troubleshooting. The access points completely freak out, and all without prompting from anyone. 

Guru Elite
Posts: 20,801
Registered: ‎03-29-2007

Re: Why would IAP's change master and reboot without prompting?

What shares the management subnet with the access points at that location?  If the default gateway is being spoofed or taken over, you might want to take a look at other devices on that same subnet to see if they are having issues, as well.

 

Again, this is your infrastructure, so I can only guess what is happening.



Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Search Airheads
Showing results for 
Search instead for 
Did you mean: