Wireless Access

Of my 32 Remote sites 29 have an heartbeat issue.  Last month we upgraded our Cisco Switch to latest IOS and since then we have issues.  On the Switch i am seeing large amounts of Giants, while the AP are showing large amounts of heartbeats missed. Please look at the attached file. 

Attachment(s)

AP heartbeats.txt   5 KB 1 version

The heartbeat failures started after the switch firmware upgrade? What version of IOS were the switches upgrade from/to?

These are 2960G and X  I went from c2960-lanbasek9-mz.150-2.SE10a to 11 and for the 2960X I wnet from 2960x-universalk9-mz.152-2.E6 to 152-2.E7.

Thank you for the answer, most AP's (215) work fine, some I have had to re-provision to get them to work.  Then the User are stating that the SSID is telling them the wrong passord is applied.  From the controller the password was changed just last week and several sites will attribute this to the IOS upgrade while I think it is the AP not getting GRE.  Yes when i look at the AP with bssid-table all SSID are broadcasting...?..

I don't have history with the versions of Cisco firmware you're using, but it does sound like there is a communications issue between the APs and the controller.

Can you check the port configuration on the Cisco switch side to validate that the AP's ports are still autonegotiating 1 Gig and full duplex? 

Checked all ports on the Cisco devices and yes they are still set to auto-negotition.  I have been manually (via an RDP session and console cable ) either factory reset or purging the environment and re-configuring the devices.  That seems to work, but i can also say it is a certain batch of AP 215's.  It is only involving device with this series of mac addresses AC:A3:1E:C5:Ex:xx.

What is the infrastructure between those access points and the controller?

There are several Firewalls between the AP and the controller.  I have already checked the ports to ensure that the correct ports are open for communications.  Especially 47-GRE

Is it a site to site VPN?

None of these sites are using s2s VPN.  All are using BGP routed connections.  Which was working until the IOS upgrade.  The only change I had to make to the Switches was removing VSTACK due to a Cisoc CVE about remote install software.   

Allright.

It suggests a connectivity issue.

You should type "show ap debug counters" to see what APs and to what extent have bootstrap issues.  It is typical to have 2 bootstraps by default.

That command showd alot of bootstraps and reboots.  Look at the test file.

Attachment(s)

Debugs.txt   11 KB 1 version