I am working on an issue at a customer site that is quite perplexing and I have not been unable to uncover the root cause;
Random IAPs (115s) disappear from the virtual controller running on campus. Airwave also shows that they are down which means they have not been separated from the VC and formed their own cluster. However, I am able to ping the IP that they were assigned and I can SSH to the IP and console access the "down" IAP.
The weirdness starts in that I cannot ping the VC IP address or the IP of the AP that is functioning as the VC from the console of the "down" IAP. However, I can ping other IAPs that are showing as "up" in the VC and I can ping the subnet gateway that the VC and the "down" IAP share from the console of the "down" IAP.
I also am unable to ping the "down" IAP IP address from the VC even though I can ping it from my desktop whether I am on a different or same VLAN as the VC and the "down" IAP. From the VC console I can ping the other IAPs that the "down" IAP can ping as well as the shared subnet gateway.
All IAPs are on a dedicated VLAN (100) across campus and there is a total of about 80 IAPs on campus. There are multiple buildings on campus and VLANs span buildings. All switching is HP Procurve, dedicated VLAN specifically for IAPs and wireless users have their own VLANs. IAP models are 115s mostly with about ten 225s mixed into high usage areas. Current code running on IAPs is 6.4.2.0-4.1.1.0.
Interesting note is that we are seeing this issue on the 115s and not once has it occurred on the 225s. I have looked at the client load on all IAPs and not one exceeds more than 30 users. In fact the ones that have been experiencing the issue have very low users counts >10.
If we reboot the IAP it comes back but experiences the issue again at some random later time. We have tried factory resetting the IAPs with the issue and all seemed fine for them over the weekend but at 9am this morning it began happening again.
The issue is not on every IAP. We are seeing this on about 20 IAPs across campus.
The VC AP has stayed consistent and the VC role has not roamed across IAPs.
Anyone else seeing anything similar? I contacted another of my customers with a similar deployment and they noted something similar with same code version and IAP model but did not do enough troubleshooting to state that it was exactly the same. They rebooted the IAPs and all was well.
#AP225