Readme: Upgrading AP51x from 8.4.x to 8.5.x can cause APs to become Inactive after upgrade
4 weeks ago - last edited 4 weeks ago
This is a kind of mini (unofficial) advisory to help people avoid an outage, or better, resolve the issue quickly should they find themselves in it.
There is already a thread about this elsewhere in the forum (including most of the below info), but I plan to have this stickied at the top for a couple of weeks to help raise awareness and help avoid outages.
TL;DR; AP-51x requires 802.3at power or better (as noted in the datasheet) and should it not be present and Intelligent Power Management (IPM) is not enabled, the AP will be stuck in an inactive state post upgrade to 8.5.0.x (despite that it was working* in 8.4.0.x)
* When we say working in 8.4.0.x, well, that's kind of the problem, it wasn't really working "well". The radios had been shut down into 1x1 operation and the power LED would have been coloured amber and blinking once per 2 seconds the whole time to indicate something was amiss.
Read on below for the symptom, resolution and a workaround.
Post upgrade to AOS 8.5.0.x previously operational AP51x APs are now stuck in inactive (according to show ap database), with the flag "If" (upper case I, lower case f):
(7008-core) [mynode] #show ap database AP Database ----------- Name Group AP Type IP Address Status Flags Switch IP ---- ----- ------- ---------- ------ ----- --------- ---------- ap515 lab 515 192.168.1.61 Up 10h:19m:3s If 192.168.1.160
upon checking closer using the command show ap debug system-status ap-name <the ap> | begin "Power Status" we can find the reason:
(7008-core) [mynode] #show ap debug system-status ap-name ap515 | begin "Power Status" Power Status ------------ Item Value ---- ----- Power Supply : POE-AF LLDP Power : 0.0W Current Operational State : USB disabled, All radios disabled Eth0 HW POE status : POE AF, LLDP power: 0.0W
refer to the "Current Operational State" which is "All radios disabled".
The datasheet says:
* Operating the AP with an 802.3af (class 3 or lower) PoE source and IPM disabled is not supported
but in AOS 8.4 the AP was allowed to operate in a degraded manner as noted above. This has been rectified and enforced in AOS 8.5.
Unfortunately the only log message that is omitted is the following:
Jul 23 22:19:15 KERNEL(firstname.lastname@example.org): [ 2205.928006] Disabling USB power
which is both related (due to insufficient power) and unrelated (e.g. not causing any service impact) - this is clearly not informative enough and a bug has been raised internally to improve this going forward.
The correct resolution for this issue is to ensure that the AP is connected to an 802.3at (or better) capable switch. If the AP is already on such a switch it may be that LLDP needs to be enabled so that the AP can learn about extra power available via LLDP "power-management" TLVs.
Once the AP is operating on 802.3at or better power, the power status should show the following:
(7008-core) [mynode] #show ap debug system-status ap-name ap515 | begin "Power Status" Power Status ------------ Item Value ---- ----- Power Supply : POE-AT LLDP Power : Successfully negotiated at 25.0W Current Operational State : USB port enabled Eth0 HW POE status : POE AT, LLDP power: 25.0W
As noted in the datasheet, the AP can run on 802.3af only when IPM is enabled. If the powering switch does not support 802.3at or there is no way to administratively enable LLDP, then you can enable IPM in order to bring the AP into service.
"ipm-enable" needs to be enabled in the relevant ap-system profile of the affected APs. Below is a CLI based example for ap system profile "default"
(7008-core) [mynode] #configure terminal ap system-profile default ipm-enable (7008-core) ^[mynode] #write mem (7008-core) [mynode] #
<wait a moment>
(7008-core) [mynode] #show ap debug system-status ap-name ap515 | begin "Power Status" Power Status ------------ Item Value ---- ----- Power Supply : POE-AF LLDP Power : 0.0W Current Operational State : 2.4GHz radio: No restrictions, 5GHz radio: No restrictions Eth0 HW POE status : POE AF, LLDP power: 0.0W
please note the following comments about IPM from the datasheet:
* When powered by 802.3af (class 3) PoE and with the IPM feature enabled, the AP will start up in unrestricted mode, but it may apply restrictions depending on the PoE budget and actual power. What IPM restrictions to apply, and in what order, is programmable.
Re: Readme: Upgrading AP51x from 8.4.x to 8.5.x can cause APs to become Inactive after upgrade
Sunday - last edited Sunday
Wish I had seen this advisory last week. I managed to figure out on my own that the Inactive flag was power related. Tested my theory by enabling IPM and immediately the 515 sprang to life.
Did some reading on Cisco PoE...the switch will not give 30W unless LLDP is enabled.
I disabled IPM, enabled LLDP on the Cisco switch, the switch immediately gave 30W.
One additional thing to note is that the AP will reboot when the PSE changes from 15.4W to 30W.
Would be really great if the Inactive flag had a reason attached in future!