08-17-2015 06:03 AM
Since upgrading to 18.104.22.168 we've seen very odd behaviour from some of our AP125 units. Symptoms are that most users can't connect. We suspect the AP doesn't allow any new connections and any users still active just haven't left since the problem occurred.
We use 802.1x authentication and users are asked for their password, even though it's correct. The correct password isn't accepted.
A dead giveaway of this occurring is a low number of clients in the university library, which is always busy. As soon as a problem AP is rebooted, clients associate and authenticate successfully. A rebooted AP will be fine for a couple of days before it fails again.
Auth-tracebuf seems to show the client repeatedly disconnecting during the auth process however we can be confident this is not a client issue.
I've raised a case with TAC but wondered if anyone else has seen this behaviour.
Solved! Go to Solution.
08-17-2015 06:08 AM
You did the right thing opening a TAC case. Did you turn in user debugging to have a clue what could be going on? If it is intermittent and random, there could be quite a few reasons. Is this a 100% ap-125 deployment or are there other access points?
Aruba Customer Engineering
Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base
08-17-2015 06:28 AM
It's really weird. It isn't really intermittent in that once an AP stops working it stays in that state, so there's the opportunity to do stuff and see how it responds. I turned on user debugging but didn't see much there, not entirely what I'm looking for.
We have about 450 125s but we also have 134/5, 105, 205, 225, 93h, 103h.
08-17-2015 06:28 AM
I've just got off the phone with the good people of the TAC. Turns out it's a known bug in 22.214.171.124 with client match. It apparently affects all APs, we're probably just seeing it on our 125s because they're often in busy areas such as the library.
Workaround is to disable client match. A fix should be out by the weekend.
08-21-2015 04:08 AM
we 've hit this bug as well.
Aruba has added a remark on the software download page:
"If you have deployed AP-9x,10x,11x,12x,13x,17x, RAP-3,10x,15x and Client Match is enabled, please do not upgrade to this release."
If you have the above O/S and APs on your estate you can identify APs that have issues by running:
#show log system all | include "STM failed"
you will see messages like:
Aug 21 11:56:55 :311020: <ERRS> |xxx@A.B.C.D sapd| An internal system error has occurred at file sapd_msg.c function sapd_papi_snd_cb line 1410 error Message to 127.0.0.1:AP STM failed: err Connection timed out msgcode 0 arg (nil).
You could reboot APs if they are just a handfull, but contact TAC, as they can support further.
08-21-2015 04:24 AM
Thanks Gem, that's really helpful. We only had two APs left that were reported as problematic. I'd already been through every AP with 0 clients and rebooted those (about 400 APs) so nearly got them all ;)
We've scheduled an upgrade to the fixed version on Tuesday.
08-27-2015 07:28 AM
No... TAC haven't been very good on this. We've installed .12 but only by chance happened to notice that .11 breaks everything so, no thanks to an absence of updates from the TAC, we nearly installed another broken version of code. i've re-enabled client match and so far things are quiet so maybe all is well. However I'm out of the office, so I'll let my colleagues know the problem potentially continues.
Bit concerned about all this... Come on Aruba, don't you test releases any more?
08-27-2015 07:45 AM
Thanks, there has been a small issue with the updates this time indeed, but we discussed this with TAC and we have a clear view.
64.2.11 come out and went away very quickly as it had a major bug as you said. It's the only bug on the release notes of 126.96.36.199. To be honest I never even noticed the 188.8.131.52 being available for download.
However, looking at the archives, I found 184.108.40.206, which has a fix for our issue:
BUG ID: 122542, 121740, 122374, 122539,122914
Symptom: The station management process of the AP was busy, that resulted in clients failing to
associate with the AP. This issue is resolved by enhancing the WLAN driver.
Scenario: This issue was observed when client match was enabled and the controller moved the client
from one AP to another. This issue was observed on AP-110 Series, AP-120 Series, and AP-130 Series
access points associated with controllers running ArubaOS 6.3.1.x, 6.4.2.x, or 6.4.3.x.
Platform: AP-110 Series, AP-120 Series, and AP-130 Series access points.
Reported Version: ArubaOS 220.127.116.11
So as per standard practice, 18.104.22.168 contains all those fixes as well.
As the download page for current releace has the 22.214.171.124 and 126.96.36.199, I asked them to add a remark on the .12 that bug fixes compared to the .10 are on the archive folder, release .11
If that makes sence...
In any case it appears to be fixed.
08-27-2015 12:17 PM
188.8.131.52 was available quite quickly after I'd made contact with the TAC but it has a bug that makes everything crash all the time (or words to that effect) so was pulled.
Our change management procedure requires we at least attempt to plan software upgrades, so we very nearly installed 184.108.40.206 which would have been a disaster. When we have campus wide wifi problems it makes us look bad, I don't like looking bad... Aruba keeps making us look bad :(