Reply
Occasional Contributor II

APs reboot themselves

Aruba controller running 5.0.1 being monitored with AirWave 7.1.3. I've just fixed my trigger in AirWave to tell me when an access point goes offline so it actually works.

In the last couple of hours, I've had several AP down/up messages because uptime has changed - confirmed with 'show ap details ap-name ' on the controller. Is there any way to find out the cause of the reboots? The APs are all connected to different PoE switches in different parts of the network.

Cheers,
-Jeff
Occasional Contributor II

Re: APs reboot themselves

This is the command I use:

show ap debug system-status ap-name | include reboot

Steve
Occasional Contributor II

Re: APs reboot themselves

Thanks Steve,

Quite a few of my APs have:


Reboot Information
------------------
Reboot caused by kernel page fault at virtual address 0000000400000090, epc == c00000000064d9d4, ra == c0000000000eb6d8
-----------------------------------------------------------------------------------------------------------------------

Rebootstrap Information
-----------------------
Date Time Reason (Latest 10)
--------------------------------------
(none found)


Rebootstrap LMS
---------------
(none found)
------------

Crash Information
-----------------

Crash information is available; use 'show ap debug crash-info' to retrieve

and looking at the crash-info:

<6>vap aruba102 vlan is 513. not discovering tunnel vlan
<4>bogus non HT station count 0Break instruction in kernel code:
<4>Cpu 0
<4>$ 0 : 0000000000000000 ffffffff80830000 000000000000001f ffffffff80471b78
<4>$ 4 : ffffffff80471b78 a8000000007cfde8 0000000000000034 0000000000000001
<4>$ 8 : ffffffff80837708 0000000000000000 a8000000004cc980 ffffffff808384c4
<4>$12 : ffffffffffffffff 000000000000000a ffffffff808388a7 ffffffff80401e80


<4>$16 : a800000001dd0400 a800000002ae4000 a800000002814400 a800000001dc0400
<4>$20 : a800000002814400 a800000001dc0400 0000000000000001 a8000000018b2090
<4>$24 : 0000000000000000 0000000000000020
<4>$28 : ffffffff8042c000 ffffffff8042fa60 0000000000000000 c00000000065fcb8
<4>Hi : 0000000000000090
<4>Lo : 00000000000000ab
<4>epc : c00000000065fcb8 ieee80211_node_leave_cleanup+0x6f0/0x788 Tainted: P
<4>ra : c00000000065fcb8 ieee80211_node_leave_cleanup+0x6f0/0x788
<4>Status: 1010ffe2 KX SX UX KERNEL EXL
<4>Cause : 40808424
<4>PrId : 000d0601
<4>Modules linked in: ath_pktlog wlan_wep wlan_tkip wlan_ccmp ath_pci ath_dev dfs ath_rate_atheros ath_hal wlan asap_mod drvlog_mod bonding asap_switch
<4>Process swapper (pid: 0, threadinfo=ffffffff8042c000, task=ffffffff80430000)
<4>Stack : a800000002814400 a800000002ae4000 0000000000000000 a8000000018b2080
<4> a800000001dc0400 ffffffff8042fb80 a800000002814000 c00000000065fdd8
<4> a800000002814400 a8000000018a2880 a800000002ae4000 c00000000064f974
<4> ffffffff802f3ad0 000000000000000e a80000000284e680 ffffffff8034f8f8
<4> 0000000000000000 00000000000000b0 0000000000000000 0000000000000000
<4> ffffffff8042fb10 a8000000018a2a80 0000000000000000 ffffffff8042fbd0
<4> 0000000000000000 a8000000025acd98 ffffffff8042fca0 a800000002ae4000
<4> a8000000018a2880 a8000000018a2880 0000000000000007 ffffffff8042fca0
<4> a8000000018b2080 a8000000018b2084 0000000000000000 c0000000000eb6d8
<4> 0000000000000022 ffffffc300001770 000000080000096c 000000000074a440
<4> ...
<4>Raw Call Trace:
<4> ieee80211_node_leave+0x88/0x260
<4> ieee80211_input+0x2084/0x2d00
<4> strncmp+0x0/0x48
<4> skb_dequeue+0x0/0x50
<4> ath_net80211_input+0x238/0x2c0
<4> ath_net80211_rx+0x2ec/0x11b0
<4> __alloc_skb+0x98/0x168
<4> ath_rx_indicate+0x3c/0xf0
<4> am_dev_xmit+0x114/0x340
<4> ath_rx_tasklet+0x7c4/0x1680
<4> ath_rx_tasklet+0x6f4/0x1680
<4> ath_intr+0x424/0x4b8
<4> ath_intr+0x84/0x4b8
<4> ath_handle_intr+0x160/0x678
<4> tasklet_action+0xcc/0x188
<4> __do_softirq+0xc4/0x190
<4> do_softirq+0x70/0x88
<4> do_IRQ+0x24/0x38
<4> ret_from_irq+0x0/0x10
<4> r4k_wait+0x0/0x18
<4> udp_poll+0x0/0x198
<4> sched_clock+0x0/0x38
<4> cpu_idle+0x60/0x68
<4> r4k_wait+0xc/0x18
<4> start_kernel+0x288/0x2c8
<4> start_kernel+0x280/0x2c8
<4>
<4>Stack : a800000002814400 a800000002ae4000 0000000000000000 a8000000018b2080
<4> a800000001dc0400 ffffffff8042fb80 a800000002814000 c00000000065fdd8
<4> a800000002814400 a8000000018a2880 a800000002ae4000 c00000000064f974
<4> ffffffff802f3ad0 000000000000000e a80000000284e680 ffffffff8034f8f8
<4> 0000000000000000 00000000000000b0 0000000000000000 0000000000000000
<4> ffffffff8042fb10 a8000000018a2a80 0000000000000000 ffffffff8042fbd0
<4> 0000000000000000 a8000000025acd98 ffffffff8042fca0 a800000002ae4000
<4> a8000000018a2880 a8000000018a2880 0000000000000007 ffffffff8042fca0
<4> a8000000018b2080 a8000000018b2084 0000000000000000 c0000000000eb6d8
<4> 0000000000000022 ffffffc300001770 000000080000096c 000000000074a440
<4> ...
<4>Unwound Call Trace:
<4> ieee80211_node_leave_cleanup+0x6f0/0x788
<4> ieee80211_node_leave+0x88/0x260
<4> ieee80211_input+0x2084/0x2d00


<4> ath_net80211_input+0x238/0x2c0
<4> ath_net80211_rx+0x2ec/0x11b0
<4> ath_rx_indicate+0x3c/0xf0
<4> ath_rx_tasklet+0x7c4/0x1680
<4> ath_handle_intr+0x160/0x678
<4> tasklet_action+0xcc/0x188
<4> __do_softirq+0xc4/0x190
<4> do_softirq+0x70/0x88
<4> do_IRQ+0x24/0x38
<4> ret_from_irq+0x0/0x10
<4> r4k_wait+0xc/0x18
<4> cpu_idle+0x60/0x68
<4> start_kernel+0x288/0x2c8
<4>
<4>
<4>Code: 0041102d 0040f809 0000282d <0000800d> 08197dc4 9607b706 96260060 3c04c000 3c010069
<4>Flash erase @ 0xee0000


I can't find the time related to the crash to know if this has just happened. I'm going to open a support case.
New Contributor

any update?

We are seeing different random APs rebooting themselves during the day. We've opened a ticket and got nowhere for 2 weeks.
Guru Elite

Re: APs reboot themselves

At anytime during the support process you have the right to ask for an escalation if you are not getting anywhere.

Please exercise that right so you can get to the bottom of this.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II

Re: APs reboot themselves

Hi,

I too have seen some unusual rebooting behaviour of some of my APs. I noticed a small group of them had rebooted at around the same time, all within say 30 minutes of each other and getting the debug info from one of the APs I got the below response:

Reboot Information
------------------
AP rebooted Sat Feb 5 13:02:24 GMT 2011: SAPD: Reboot requested by controller
-----------------------------------------------------------------------------

Rebootstrap Information
-----------------------
Date Time Reason (Latest 10)
--------------------------------------
1999-12-31 16:00:49 Changing to LMS #0 (#.#.#.#)


Rebootstrap LMS
---------------
1999-12-31 16:00:49 Changing to LMS #0 (#.#.#.#)
----------------------------------------------------

Now.....I didn't reboot those APs from the controller and I'm the only one with access to it...so if it was the controller that told them to reboot, it wasn't because of human intervention.

It was about 20 APs that rebooted, not all in the same AP Group, but they do share some "default" profiles" however so do 200 others so I can't put my finger on any reason for this "mass" reboot!

p.s. (#.#.#.#) being the IP of my controller.

I would open a case but since the portal update I don't have the permissions to create a new case!
Guru Elite

Re: APs reboot themselves

You can still open a case by emailing support@arubanetworks.com

With that being said, to catch issues that do not show up consistently, it is best practice to setup a syslog server that your controllers point to so that support can go back in time and see what was happening. This gives them additional data so that troubleshooting can have a meaningful direction. This goes for all issues, real or perceived.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II

Re: APs reboot themselves

Hi,

I have logged a case, also seen some APs rebooting on their own with the following reason!

Reboot Information
------------------
AP rebooted Fri Dec 31 16:24:33 PST 1999; Process /usr/sbin/dnsmasq has too many open files (831

This was happening before I upgraded to 5.0.3.0 but the "mass" reboot I don't think happened while on 5.0.2.0.

I'll report back what support find in case anyone sees this
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: