Wireless Access

Running a pair of 7210 controllers in a Master/Local setup and recently updated from 6.4.2.15 to 6.4.4.11 and 90% of my APs are fine and stay up and running but a handful of AP-115s will reboot like clock work at around 3 minutes and 40 seconds.  I have mixture of AP-105, AP-115, AP-205 and AP-215.  No issues seen with the AP-105, AP-205 and AP-215 and probably half or more of the AP-115 will work while the others will stay active and reboot over and over and will not stop since the update.  Wondered if anyone has seen this or not?  Are there any ramifications if I go back to 6.4.2.15 will it downgrade without issue and retain all the configuration settings such as how it upgraded and retains the configuration settings?

Type "show log system 50" to see why they are rebooting.

Took a look and this is what I am seeing.

 

Jan 18 20:44:07 :343005:  <INFO> |mdns|  mdns_generate_refresh_query_packet 2580 record expiry: send refresh query for server mac: 00:26:73:04:31:cd
Jan 18 20:44:07 :343005:  <INFO> |mdns|  ag_mdns_get_unique_token_list 621 Insert:Returning list of 4 tokens
Jan 18 20:44:08 :304057:  <INFO> |stm|  tunnel_timeout: Tunnel <IP of AP> 18:64:72:c5:04:3a last_hbt_seq_num_rcvd 120
Jan 18 20:44:08 :305003:  <INFO> |stm|  AP <AP Name> is down

 

this repeats for other APs that are impacted.  Odd part is it only seems to be a handful of AP-115s.  I even took a new AP-115 out of the box let it update and same results.  Then took an older AP-105 that had been good as well that need to be updated it completed and ran without issue.  All of the APs leverage the same AP Group.

When the AP comes back up, you should execute "show ap tech-support ap-name <name of AP>" and see if you can see the reboot reason.

Are these APs on the other side of a VPN tunnel?
How are these APs powered?

Came across this within the logs

 

Rebootstraps and Control Messages Log
-------------------------------------
Recent Messages            Time now: Thu Jan 19 06:36:58 2017
---------------            ----------------------------------
Time Offset                Message details
-----------                ---------------
-21                        RCVD: STATUS_REPORT len=24 peer=<AP IP> seq_num=2 rtt=0 result=OK
-26                        SENT: ESSID_LIST len=171 peer=<AP IP> seq_num=5 tries=1 rtt=0 result=OK
-26                        SENT: MONITORING_MSG_CONFIG len=112 peer=<AP IP> seq_num=4 tries=1 rtt=0 result=OK

Rebootstrap                Thu Jan 19 06:36:29 2017
Time Offset                Message details
-----------                ---------------
-0                         RCVD: HELLO len=1247 peer=<AP IP> seq_num=0 rtt=0 result=OK
-227                       RCVD: STATUS_REPORT len=24 peer=<AP IP> seq_num=2 rtt=0 result=OK
-232                       SENT: ESSID_LIST len=171 peer=<AP IP> seq_num=5 tries=1 rtt=0 result=OK

Rebootstrap                Thu Jan 19 06:32:35 2017
Time Offset                Message details
-----------                ---------------
-0                         RCVD: HELLO len=1247 peer=<AP IP> seq_num=0 rtt=0 result=OK
-174                       RCVD: STATUS_REPORT len=24 peer=<AP IP> seq_num=3 rtt=0 result=OK
-214                       SENT: ESSID_LIST len=171 peer=<AP IP> seq_num=5 tries=1 rtt=0 result=OK

Rebootstrap                Thu Jan 19 06:28:39 2017
Time Offset                Message details
-----------                ---------------
-0                         RCVD: HELLO len=1247 peer=<AP IP> seq_num=0 rtt=0 result=OK
-229                       RCVD: STATUS_REPORT len=24 peer=<AP IP> seq_num=2 rtt=0 result=OK
-234                       SENT: ESSID_LIST len=171 peer=<AP IP> seq_num=5 tries=1 rtt=0 result=OK

Info of last session       
Last heartbeat seq number  120
Reason for disconnect      Controller aged out
Timestamp                  2017-01-19 06:35:16

 

Which is happening around ever 3 and half to 4 minutes.  Still doesn't make sense that it happened after the update to only 10 or so AP-115 devices and will happen to any new ones I provision and no impact to AP-105 devices.  They are within our LAN they are in different buildings but over a 1 gig link.  There is a mix some are Aruba PoE Injectors and others are Cisco PoE Switches.  The log above is an AP connected to a Cisco PoE Switch.

Wondered if you had any other thoughts on the issue?  I did open a support case 5316508802 back on Tuesday did some initial trouble shooting and log gathering that day but not many updates since.

 

 Thanks,

   Evan