Higher Education

We have been getting increasingly frequent complaints from the students of being disconnected regularly from the wireless network.  Aside from the usual difficulties of getting the students to provide further details it has also been difficult to track down when the students do give details.  The problem can happen in the middle of an active session like a Skype call, but is often just a pop up on Windows laptops notifiying them of the loss of connectivity or they see the spinning wheel on an OS X laptop.  As the reports go the commputers often reconnect without any action needed but sometimes the computer only connects if the student shutsdown wifi and turns it back on. In all cases the students are  Setting debugging on the clients has not shown any obvious patterns. Sometimes the logs shown a Client Match event, other times they show the client timed out and was deauth'd, and sometimes nothing obvious is in the logs.  I can not reproduce the problem, predict when it will happen next or find a clean way of identifying it has happened in the logs.  Any suggestions are greatly appreciated.

 

We are running 6.4.2.2, having upgraded from 6.4.1.0 two weeks ago.  We have a pair M3's as the Master and two 7220's as local controllers sharing the load between ~560 APs.

 

It could be something in 6.4 and I am considering going back to the latet 6.3.  We went to 6.4 for the improvements in Airgroup and the addition of getting the Chromecast to work. 

#7220

We are a small shop with one 7210 controller and about 170 APs and we are seeing exactly what you describe after moving to 6.4.2.2 from 6.4.1.0.  We are new to Aruba (about three months and counting) so we are not as familiar with the tools available to troubleshoot, but likewise we have not been able to pin it to any particular event.  We are also thinking about bumping back down to 6.4.1.0.  Our primary SSID is 802.1X, and we also provide an open SSID for students to register devices.  Most that we've heard from are on the 802.1X network but we are not sure that it is exclusive.  Would be interested to hear if 802.1X is also involved for you.

 

#7210

---

@jwhitaker@transy.edu wrote:

We are a small shop with one 7210 controller and about 170 APs and we are seeing exactly what you describe after moving to 6.4.2.2 from 6.4.1.0.  We are new to Aruba (about three months and counting) so we are not as familiar with the tools available to troubleshoot, but likewise we have not been able to pin it to any particular event.  We are also thinking about bumping back down to 6.4.1.0.  Our primary SSID is 802.1X, and we also provide an open SSID for students to register devices.  Most that we've heard from are on the 802.1X network but we are not sure that it is exclusive.  Would be interested to hear if 802.1X is also involved for you.

 

---

jwhitaker,

 

Welcome to Aruba.  Please look at the article here:  http://community.arubanetworks.com/t5/Technology-Blog/Removing-the-Bottleneck-in-Wireless/ba-p/77978 to see a list of things that you can check to determine what your problem is.

If you open a TAC case in parallel, it would help everyone on this thread if you report back on any progress you make.

 

#7210

We are seeing the same problems in our environment.  We are seeing complaints only from our 802.1x SSID.  We are also seeing some issues where students are not able to connect at all despite fantastic signal strength and almost zero interference.  

 

We too upgraded to 6.4 for improvements in Airgroup.  We first started noticing problems in 6.4.2.0 but we suspect that there were some problems in 6.4.1.0 as well.  We updated to 6.4.2.2 code today to try to mitigate some of the issues.  

 

We are running a single 3600 as a master with 4 M3s.  We have a total of 878 access points with a mixture of 105, 130 series, and 220 series with a few 175 and 275.  We see the issues across all models.

 

Did you first notice the problem after you updated to 6.4?

Have any TAC cases been opened to investigate these issues?

 

interesting to know as I am running a hotspot network on 6.4.2.2 and I haven't heard of any issues yet but obviously the configuration is much simpler and we are running an open SSID.

We started on 6.4.1.0 because we have AP-205s and needed 6.4 to support them, so we've not known anything different.  And, we have been in the process of rolling out Aruba across campus and replacing our previous system, so there was a lot of stuff in flux - if someone had an issue, we said "we're under construction" most times.  Eventually, we got it rolled out everywhere and things seemed OK.  From our view, 6.4.2.2 made it worse, or at least that's when we started hearing more complaints.

 

#AP205

Thanks for all of the replies!

 

I have just opened a case with TAC on this issue.

 

I am not sure if we had this problem before 6.4.  It has been trickling up for a while and I am not sure if it goes back to this summer or earlier.

802.1x is how are students connect.  We do have a PSK network for things that have difficulty with 1x.  The examples I have are all 1x.

 

Logs and user-debug are uploaded to the ticket.  I will let you know what I find.

 

Thanks!

What type of APs are you using ?

We have a mix of 105, 125 and 225 APs.  Within a building the model type is usually the same.

We are using primarily 134 and 135s.  We have a few buildings that are running 224s and a few 105s in select locations.  For our outdoor environment we have 50 AP175P and 4 275

The reports that have actually given any details came in for Windows 7 Pro SP1 and OS X (10.9.3).  The word of mouth includes every flavor of Windows or OS X laptop.

 

802.1x EAP-PEAP for us.  We worked through some problems with Bradford Network Sentry earlier in the semester but we are not seeing any correlation of the disconnects with RADIUS events.  Patching and then upgrading to the latest with Bradford has taken them out of the process.  

 

 

I'm no authority here, but I believe we started seeing something similar when we upgraded to 6.3.xx. I know we opened a case with Aruba Support and believe we had to tweak our client-match settings. Since then, things have been much more stable. We are now happily running all 220 local controllers(3200's, 3400's and 7210's) and approx. 6000 APs(AP125, AP135 or AP225's) on 6.3.1.5.
I'm sure the new Apple OS versions aren't helping matters any. We noticed the problem most significantly on our Macbooks, they tended to drop randomly and took a long time to reconnect. At the time, we had to upgrade to 6.3 to support the AP225's. We initially thought it was an issue with the 225's and the 802.11ac settings.
The client-match adjustment may have been necessary because of our implemented 802.1x (EAP-PEAP, MSCHAPv2) settings where we still use WPA-AES and not WPA2. There was some issue related to that, I just don't recall what it was.
I hope that's somewhat helpful and not misleading

#AP225
#7210

So a fair amount of people have mention "twerks" (and not the Miley Cyrus kind) to client match that seemed to stabilize the situation. What adjustments were made if you guys don't mind?

I am curious what you all have for your SSID/Advanced settings, specifically basic and transmit rates.  

 

We have been fairly stable minus some odd disconnect issues with some of our 10.10 OSx users.  We are running 802.1x PEAP - WPA2-aes.

 

 

before making the adjustments i would ask support.  I've heard that the 'defaults' for CM where changed in some revision of 6.3.  Having said that here are the commands for it.  

 

cm-report-interval

cm-unst-ageout-intvl

cm-band-a-min-signal

cm-sticky-check-inter

cm-sticky-snr

cm-sticky-snr-delta

cm-sticky-min-signal

cm-steer-timeout

cm-lb-thresh

cm-stale-age

cm-max-steer-fails

cm-lb-client-thresh

cm-lb-snr-thresh

cm-lb-signal-delta

 

other things to look at are. 

 

how often are power settings changing

how often are channels changing

default data rates in that area

default beacon rates. 

 

all these could cause disconnects for roaming clients. 

Aaron,

 

I had to double-check the post because I thought I wrote it.  We have roughly the same AP's, same complaints, almost the same controllers (was actually hoping the 7200's were better since we're just M3's still), the same reason for upgrading (chromecast) and running the same ArubaOS.  6.3 didn't seem any better for us and we also have some new AP-104's that we loan out for students in their dorms.

 

This year has been brutal for us.  The prior 7 were great. People want to know what's up with wifi. My best guess at this point:

 

 

I think there is only so much I can 'fix'.  

It looks like I hit a nerve with this topic!  Wish I started it weeks ago.

 

It looks like the problem predates 6.4 for us.  One of my reports from two months ago mentions the problem going back a year.

 

TAC is on the case, so I will let folk know how it goes.

The nearby Colleges are struggling with similar problems as well - and they are not Aruba customers.  I guess these devices are really being tweaked for home and/or open networks; compatible with anything you could ever plugin.

Are these all types of devices or only OSX? If the latter, what version?

For us it is mostly OSX.  MacBooks are also the most common device we have.  When I ask our student workers if they see it with Windows Laptops as well the response was 'oh, yeah, those are even worse'.  However the configuration is generally the problem with Windows (or virus, software firewall, wired port doesn't work either, etc...); at least from what I've been able to get my hands on.

I love the graph!

---

@pgemme wrote:

Aaron,

 

I had to double-check the post because I thought I wrote it.  We have roughly the same AP's, same complaints, almost the same controllers (was actually hoping the 7200's were better since we're just M3's still), the same reason for upgrading (chromecast) and running the same ArubaOS.  6.3 didn't seem any better for us and we also have some new AP-104's that we loan out for students in their dorms.

 

This year has been brutal for us.  The prior 7 were great. People want to know what's up with wifi. My best guess at this point:

 

 

I think there is only so much I can 'fix'.  

---

pgemme,

 

You are missing a slice for "the real problem" (excellent graph, by the way).  

 

For everyone on this thread who is having issues and it is mission critical; if you are over your head with dealing with a problem, and TAC cannot help you, please consider having a trained professional take a look at it.  Basic troubleshooting and even TAC can only go so far.  Old and incorrect assumptions often cloud and delay defining a problem and getting to a resolution.  Unhelpful user feedback can also sway a situation in the wrong direction.  A trained professional can look at all the facts and narrow things down to something of substance, so that your issues can be defined and systematically dealt with.

 

Every environment is different and many times specific deployments require specific approaches.  That advice normally can only be given by someone who is onsite and gets a full picture of what could be happening.  Basic principles and even best practices can only go so far; those principles and practices need to be modified to suit your specific deployment.

 

Your Aruba Sales team would normally be aware of a go-to professional in your region who would be able to give you the help you need.

 

Another thing to look at is not only 802.1x but client match, channel utilziation, and whether clients are 2.4 only with roaming issues. We have run into issues like this as well but have found that a lot of the devices having the issue have poor 2.4 only 1x1 and 2x2 NICs. We end up disabling roaming in the driver settings. 

 

In regards to 802.1x take a look at your VAP, and AAA profile. Also take one of the users or one of your test machines you have been able to replicate the issues on. Look at your asscioation times/timeouts and compare to RADIUS or what ever your using for auth. I would also look at what your using whether it be EAP-Peap, EAP-TLS, etc and verify your not having issues there. Also look at your load balencer if your using one for Radius, etc

I too have had several complaints about this...I've got 2 different AOS versions, a 6.4.1.0 and a 6.3.1.6.  So far, only have complaints in 1 area on the 6.4.1.0 code version... I went there myself, with several different machines (android, mountian lion, yosemite, and windows 7), and experienced issues with all but the android, all simultaneously. 

 

My initial complaint came from a student who had just upgraded to yosemite... I've got a ticket open with Dell, who in turn has one open with Aruba on this matter. 

 

My experience was not only a disconnect - reconnect, but a failure to connect. I have several 802.1x networks, hitting several different radius servers, as well as an open network...During this failure to connect time-period, not a single SSID worked for me on the laptops... My android phone however, no issues....

---

@lkfirestone wrote:

I too have had several complaints about this...I've got 2 different AOS versions, a 6.4.1.0 and a 6.3.1.6.  So far, only have complaints in 1 area on the 6.4.1.0 code version... I went there myself, with several different machines (android, mountian lion, yosemite, and windows 7), and experienced issues with all but the android, all simultaneously. 

 

My initial complaint came from a student who had just upgraded to yosemite... I've got a ticket open with Dell, who in turn has one open with Aruba on this matter. 

 

My experience was not only a disconnect - reconnect, but a failure to connect. I have several 802.1x networks, hitting several different radius servers, as well as an open network...During this failure to connect time-period, not a single SSID worked for me on the laptops... My android phone however, no issues....

---

lkfirestone,

 

If you can go to the dashboard and look at one or two access points in the area, you might want to see what the channel utilization is for them in the area at that time when you are having issues.  If there is a hidden source of interference or high utilization, dropped or corrupted packets during 802.1x could prevent a client from connecting period.  It could also be environmental as in a non-wifi device causing interference.  How many SSIDs are you broadcasting in that area and do you have "Drop Broadcast and Multicast" enabled on all of your SSIDs?  Check out the post here:  http://community.arubanetworks.com/t5/Technology-Blog/Removing-the-Bottleneck-in-Wireless/ba-p/77978 to see if there are small things that you can check for...

 

 

Also take into account OSX Yosemite has known issues.

http://community.arubanetworks.com/t5/Education/Yosemite-roaming-issues/gpm-p/211019#M1106

We have seen the same issue and are running 6.3.1.10 with three M3 controllers (Master & two local) and about 590 AP's (105, 125, 134/5, 175 and some 225).  We are having random disconnects and dificulties reconnecting after (need to disable adapter or reboot computer).  More OS X users than Windows but not enough data to draw any defenite conclusions.

We got rid of some interference in one of the buildings and that helped a little bit with the number of times this happens.  However, students are still disconnecting randomly and the debug logs don't give any more information. We started getting reports of this problem about 4-5 weeks ago but we ran on the 6.3.1.10 long before that.  Please post if you were able to figure something out with TAC.

Just wanted to toss my hat into the ring as I am seeing some of these same issues. Small shop here, 4 - 3000 series controllers, dedicated master (3200xm) with three locals (3200xm, 3400 & 3600), running 6.4.2.0 across the board. Currently about 200 AP's, mix of 93's, 105's and now 225's. I have been seeing it on Win7 machines, iPads, iPhones and rarely on an Android. I haven't seen it on win8 though... hmmm there is some food for thought.

 

Anyway, I was attributing a lot of it to needed updates to various wireless drivers and whatnot, there apparently a known issue with the latest iOS and there was an update released for "known WIFI related issues" (according to our web designers iPad anyway). Now I am seeing the similarities and am thinking this is the same issue as everyone else here. Ill be monitoring this thread to see how it progresses too.

Since everybody is throwing in their hat. 

We are seeing the issue here at Boston College.  Dedicated Master with 8 local's (6.3.1.8) 

Windows/Apple OS on various devices. 

 

6.3.1.13 here and we're hearing of similar issues.   i'm trying to corner a client so i can catch a debug and get some client match and roaming information to see if i can pinpoint it..

 

7200 controllers master - local setup with roughly 4000 access points.  

 

If anyone catches some good info in a debug log let us know.

 

Thanks,

 

Brodi3man

 

We're having the same issues mostly from OSX (random disconnects and dificulties reconnecting after, no red flags in logs). We're running 6.3.1.8. We have 10 M3s (inlcuding master) and one 7220.

 

Has anyone tried disabling client match? Are folks running 80MHz channels in 5GHz?

 

Mike

#7220

---

@mldickson wrote:

We're having the same issues mostly from OSX (random disconnects and dificulties reconnecting after, no red flags in logs). We're running 6.3.1.8. We have 10 M3s (inlcuding master) and one 7220.

 

Has anyone tried disabling client match? Are folks running 80MHz channels in 5GHz?

 

Mike

---

 

We had a ticket (or 3) open with TAC regarding this issue.  They tried turning off client-match but it did not resolve the issue.  We worked with them for several weeks on this issue and had no luck resolving the problem.  We then made a few changes with them regarding our NAC (not Clearpass) and it seemed to fix the problem.  Recently we found out that the problem still exists but users had stopped reportin the issue to IT.

 

In our TAC support case we worked with head engineers and were not able to find a problem in the Arubas system but at the time nobody at TAC or engineering had heard of the issue.

 

Glad it's not jsut us

 

#7220

Here is an update on the latest from TAC.  They looked at the user debug for one of the few students reporting the problem in detail and are attributing the disconnect to "User Idle Timeout".  This was already on our list of suspects and it was interesting to get the same from them.  It does explain some, but not all, of the disconnects.

 

Last week we changed the timeout from default 300 seconds to 600 seconds.  This morning I have doubled it again to 1200 seconds.  Really hoping it does not cause problems in other areas but very curious to see how it goes.

 

I just sent them a user debug of another student from last night that was definitely using his computer to visit different web sites (they included a screen shot of their browser log!) when it (Windows 7 Pro SP1) was disconnected.  The logs showed the reauth but there was nothing leading up to it.  Waiting to hear back.

Make certain that your new user-idle-timeout does not exceed your DHCP max lease time. Otherwise, you could run into issues when the next client attempts to claim that same IP address (assuming you have Prohibit IP spoofing enabled).

Hi All,

I just searched throuh my archive and discovered this .....

 

Description: Macbooks client intermittently getting disconnected

 

Latest Status changed to:
It was found that the client was changing channels (moving to a to g radio) due to radar detected on DFS channel. The AP did not change channels..

Next Action changed to:
Recommendation to mitigate this is to remove DFS channels from regulatory domain profile. TAC is investigating to see if this is a known Apple issue..

 

I also reviewed my Client Match settings and they don't look that different from the defaults.

 

I might not have remembered all the events that surrounded our upgrade to 6.3 correctly. Apologies if I conveyed misleading information.

 

 

---

@jkotch wrote:

Hi All,

I just searched throuh my archive and discovered this .....

 

Description: Macbooks client intermittently getting disconnected

 

Latest Status changed to:
It was found that the client was changing channels (moving to a to g radio) due to radar detected on DFS channel. The AP did not change channels..

Next Action changed to:
Recommendation to mitigate this is to remove DFS channels from regulatory domain profile. TAC is investigating to see if this is a known Apple issue..

 

I also reviewed my Client Match settings and they don't look that different from the defaults.

 

I might not have remembered all the events that surrounded our upgrade to 6.3 correctly. Apologies if I conveyed misleading information.

 

 

---

jkotch,

 

You bring up a good point.  Do not run DFS channels unless you have absolute control over your clients.  In some cases, entire manufacturers do not have DFS certified or enabled on their clients.  Other manufactuerers that have enabled DFS channels don't scan on them often, so you could end up with an access point that serves few if any clients.  You would be creating black holes and irregular coverage for those devices with no DFS support.  In some cases, Radar will geniuinely create issues with DFS channels, but there is a chance that there are false positives. Disabling DFS channels will sidestep these issues.

 

If you used DFS to obtain spectrum, you should have a plan to unbond channels so that you can regain channel separation when you disable DFS.

 

 

 

I now also remember we had to adjust our 802.1x High and low watermarks, but I don't remember why.  We were seeing a lot of Throttling occurring before the adjustments.

I previously stated we're happily running 6.3.1.5 but it has just come to light that our Field Technicians all received new Macbooks last month and many of them are complaining of getting disconnected from one of our two 802.1X networks. I have to gather much more information still, but I believe they are experiencing this on MacOS. I have not heard complaints of Windows dropping connections.

Jkotch,

Is there any way you can be more specific about what parameters you had to change and in response to what?

I'm not purposely being elusive. I wish I could remember all the details. I welcome someone to connect up with me and take a look at our configuration. I don't want to post any false information so I am reluctant to post much of anything.
I believe that we customized our high and low watermark settings on the 7210 controllers because of some sort of 802.1x authentication issue that we had while running 6.3.1.5. I can't say for certain it was related to the disconnects, but there is a good chance it was. We were seeing a lot of 'throttling' happen and we had to keep adjusting the settings until we found the correct ones. We may have adjusted some of our Client Match settings as well. Based on other posts in this thread, I'm guessing the issue has something to do with 802.1X authentication.

Our current values.
Dot1x high watermark..................120
Dot1x low watermark...................114
Dot1x stm throttling percent..........50

 

 

If there's any more information you would like, please let me know.

#7210

jkotch,

 

No worries.  Thanks for being honest.  Maybe it will jog someone's memory.

 

There are alot of different issues in this thread.  Maybe someone can find something useful in everything that is contributed.

Update from SUNY Oneonta: 

 

Last Thursday night we worked with TAC on this and we changed ARM to maintain mode. Now reports from the two buildings where we made report better but still seeing disconnects. This is a client report from Airwave of currently connected clients. The problem exist AP's on campus not just one group, RF profile etc. So far I no matter what we have thrown at it nothing seems to change. Now this is the very same profile for RF that we have used for the past two years but haven't seen this problem. There are some increased devices from the student's side but we have pretty much the same enviroment in the reshalls as last fall semester that worked fine without a problem.  I'm digging up on my backup drive for a compairson of connected users then to now, but it really stands out when looking at the Duration and Association Time how big the problem is. Hays and Grant  AP's are where we have made changes with TAC. Hulbert below has all New AP 115's intalled in a Dense deployment.

 

 

 

For some reason it won't allow me to attach a file but here is the information that I wanted to convey. This is just a shortend version but I can send you the full file if you want to review. At this time there were 2508 users connected. But here are samples of what I'm seeing. Now this is all at random intervals, with not all the same client's bumped at the same time or from the same AP, or RF Profile.

 

Duration	Association Time	AP/Device	Sig. Qual.	Speed	Goodput	AOS Device Type	Role	SSID	Connection Mode	Usage	Ch BW
1 day 0 hours 16 minutes	11/11/14, 9:41 AM	Wilber-7E:0A	41	65	18	iPad	Admin-3	Red Dragon WiFi	11n 5 GHz	1.23 Kbps	HT20
1 day 0 hours 19 minutes	11/11/14, 9:37 AM	Macduff-104	32	36	55	iPod	registration	Red Dragon WiFi	11g 2.4 GHz	420 bps	-
1 day 0 hours 9 minutes	11/11/14, 9:48 AM	Higgins-A4:24	34	227	30	OS X	Resnet-9	Red Dragon WiFi	11n 5 GHz	10 Kbps	HT40
1 day 10 hours 18 minutes	11/10/14, 11:39 PM	Ford-C6:EE	41	0	48	iPad	Resnet-5	RedDragonSecure	11a 5 GHz	2.35 Kbps	-
1 day 10 hours 20 minutes	11/10/14, 11:37 PM	Grant-E0:EA	23	52	48	Android	Resnet-15	RedDragonSecure	11n 2.4 GHz	1.97 Kbps	HT20
1 day 10 hours 27 minutes	11/10/14, 11:29 PM	Hulbert-C019.aruba.oneonta.edu	52	0	26	Win 8	Resnet-15	Red Dragon WiFi	11n 5 GHz	314 bps	HT40
1 day 10 hours 31 minutes	11/10/14, 11:26 PM	Tobey-E2:39	35	0	20	iPad	Resnet-7	Red Dragon WiFi	11n 5 GHz	84 bps	HT40
1 day 10 hours 39 minutes	11/10/14, 11:17 PM	Huntington-CC:40	33	0	53	Windows	Resnet-13	RedDragonSecure	11a 5 GHz	773 bps	-
1 day 10 hours 9 minutes	11/10/14, 11:48 PM	Grant-E2:19	41	65	30	iPod	registration	Red Dragon WiFi	11n 2.4 GHz	445 bps	HT20
1 day 11 hours 1 minute	11/10/14, 10:56 PM	Matteson-2B:3A	32	36	45	iPod	Resnet-16	Red Dragon WiFi	11g 2.4 GHz	20 bps	-
1 day 11 hours 11 minutes	11/10/14, 10:45 PM	Matteson-CD:B6	39	35	45	Win 8	Resnet-16	Red Dragon WiFi	11a 5 GHz	4.26 Kbps	-
1 day 11 hours 14 minutes	11/10/14, 10:43 PM	Tobey-26:A7	44	0	70	OS X	Resnet-14	Red Dragon WiFi	11n 2.4 GHz	3 bps	HT20
1 day 11 hours 14 minutes	11/10/14, 10:43 PM	Wilber-7A:7E	49	64	25	Win 7	Resnet-16	RedDragonSecure	11n 2.4 GHz	1.41 Kbps	HT20
1 day 11 hours 25 minutes	11/10/14, 10:32 PM	Wilber-E2:C1	36	65	22	iPod	Resnet-1	Red Dragon WiFi	11n 2.4 GHz	49 bps	HT20
1 day 11 hours 38 minutes	11/10/14, 10:19 PM	Hulbert-B001.aruba.oneonta.edu	69	71	35	registration	Red Dragon WiFi	11n 2.4 GHz	0 bps	HT20
1 day 11 hours 47 minutes	11/10/14, 10:10 PM	Golding-E2:24	31	3	0	Win 8	Resnet-9	RedDragonSecure	11n 2.4 GHz	7.34 Kbps	HT20
1 day 11 hours 58 minutes	11/10/14, 9:59 PM	Hulbert-E011.aruba.oneonta.edu	63	65	18	iPad	Resnet-3	Red Dragon WiFi	11n 5 GHz	77 bps	HT20
1 day 11 hours 58 minutes	11/10/14, 9:59 PM	Wilber-7D:F7	0	0	46	registration	Red Dragon WiFi	11n 2.4 GHz	96 bps	HT20
1 day 12 hours 23 minutes	11/10/14, 9:33 PM	Higgins-c1:a4:2e	36	0	19	iPhone	Resnet-1	Red Dragon WiFi	11n 5 GHz	120 bps	HT40
1 day 12 hours 30 minutes	11/10/14, 9:26 PM	Littell-437.aruba.oneonta.edu	69	130	31	Win 8	Resnet-16	Red Dragon WiFi	11n 2.4 GHz	21 bps	HT20
1 day 12 hours 54 minutes	11/10/14, 9:02 PM	Huntington-317	39	30	54	registration	Red Dragon WiFi	11g 2.4 GHz	203 bps	-
1 day 12 hours 58 minutes	11/10/14, 8:59 PM	Hulbert-B031.aruba.oneonta.edu	58	0	90	iPad	Resnet-8	Red Dragon WiFi	11n 5 GHz	17 bps	HT20
1 day 13 hours 28 minutes	11/10/14, 8:28 PM	Hulbert-C016.aruba.oneonta.edu	58	65	18	iPad	Resnet-9	Red Dragon WiFi	11n 5 GHz	20 bps	HT20
1 day 13 hours 3 minutes	11/10/14, 8:53 PM	Blodgett-A3:2F.aruba.oneonta.edu	34	300	36	OS X	Resnet-4	Red Dragon WiFi	11n 5 GHz	1.99 Kbps	HT40
1 day 13 hours 5 minutes	11/10/14, 8:52 PM	Macduff-BC:32	29	0	49	iPod	Resnet-2	Red Dragon WiFi	11g 2.4 GHz	62 bps	-
1 day 13 hours 58 minutes	11/10/14, 7:59 PM	Littell-A4:7A	29	150	29	AppleTV	registration	Red Dragon WiFi	11n 5 GHz	736 bps	HT40

 

18 minutes	11/12/14, 9:39 AM	Hulbert-D074.aruba.oneonta.edu	31	65	55	iPhone	Resnet-8	Red Dragon WiFi	11n 5 GHz	62.5 Kbps	HT40
18 minutes	11/12/14, 9:39 AM	Hulbert-D077.aruba.oneonta.edu	44	2	21	Android	Resnet-9	Red Dragon WiFi	11n 2.4 GHz	96.4 Kbps	HT20
18 minutes	11/12/14, 9:39 AM	Hulbert-E004.aruba.oneonta.edu	73	0	46	Win 8	Resnet-16	Red Dragon WiFi	11n 2.4 GHz	53 bps	HT20
18 minutes	11/12/14, 9:39 AM	Hulbert-E019.aruba.oneonta.edu	27	65	16	iPod	registration	Red Dragon WiFi	11n 2.4 GHz	3.36 Kbps	HT20
18 minutes	11/12/14, 9:39 AM	Hulbert-E029.aruba.oneonta.edu	20	68	13	registration	Red Dragon WiFi	11n 2.4 GHz	0 bps	HT20
18 minutes	11/12/14, 9:39 AM	Hulbert-E029.aruba.oneonta.edu	61	283	189	OS X	Resnet-1	RedDragonSecure	11n 5 GHz	0 bps	HT40
18 minutes	11/12/14, 9:39 AM	Hulbert-E029.aruba.oneonta.edu	56	0	21	iPhone	Resnet-1	RedDragonSecure	11n 2.4 GHz	163 Kbps	HT20
18 minutes	11/12/14, 9:39 AM	Hulbert-E029.aruba.oneonta.edu	40	150	24	iPhone	Resnet-5	RedDragonSecure	11n 5 GHz	273 Kbps	HT40
19 hours 10 minutes	11/11/14, 2:47 PM	Tobey-E2:9E	31	150	19	iPad	Resnet-16	Red Dragon WiFi	11n 5 GHz	2.06 Mbps	HT40
19 hours 20 minutes	11/11/14, 2:36 PM	Huntington-225	47	0	51	Android	Resnet-6	RedDragonSecure	11g 2.4 GHz	28 Kbps	-
19 hours 31 minutes	11/11/14, 2:26 PM	Grant-E2:15	26	65	9	iPad	Resnet-9	Red Dragon WiFi	11n 5 GHz	412 bps	HT20
19 hours 34 minutes	11/11/14, 2:23 PM	Hulbert-D050.aruba.oneonta.edu	38	65	18	iPod	Resnet-1	Red Dragon WiFi	11n 2.4 GHz	0 bps	HT20
19 hours 34 minutes	11/11/14, 2:23 PM	Hulbert-D055.aruba.oneonta.edu	54	150	19	iPad	Resnet-12	Red Dragon WiFi	11n 5 GHz	20 bps	HT40
19 hours 41 minutes	11/11/14, 2:15 PM	Blodgett-CD:A0	32	33	54	registration	Red Dragon WiFi	11g 2.4 GHz	203 bps	-
19 hours 41 minutes	11/11/14, 2:15 PM	Curtis-5D:4F.aruba.oneonta.edu	30	0	34	Android	Resnet-5	Red Dragon WiFi	11n 5 GHz	13 bps	HT20
19 hours 41 minutes	11/11/14, 2:15 PM	Littell-446.aruba.oneonta.edu	33	65	53	iPad	registration	Red Dragon WiFi	11n 5 GHz	1.83 Kbps	HT20
19 hours 42 minutes	11/11/14, 2:14 PM	Matteson-c9:5d:42	30	148	103	iPad	Resnet-12	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:9e	28	150	19	iPhone	Resnet-2	Red Dragon WiFi	11n 5 GHz	90.5 Kbps	HT40
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:9e	47	0	34	iPod	Resnet-4	Red Dragon WiFi	11g 2.4 GHz	56.8 Kbps	-
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:9e	29	150	19	iPod	Resnet-18	Red Dragon WiFi	11n 5 GHz	66 Kbps	HT40
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:a2	61	0	-	OS X	Resnet-10	RedDragonSecure	11n 5 GHz	9 bps	HT40
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:bc	23	33	6	Android	Admin-3	Red Dragon WiFi	11n 2.4 GHz	842 bps	HT20
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:db:bc	18	81	18	iPhone	Resnet-18	Red Dragon WiFi	11n 5 GHz	322 bps	HT40
19 minutes	11/12/14, 9:37 AM	24:de:c6:ca:de:f4	41	50	25	iPhone	Resnet-5	Red Dragon WiFi	11n 2.4 GHz	505 bps	HT20

8 minutes	11/12/14, 9:49 AM	Hays-CC:94	25	24	9	iPhone	Resnet-16	Red Dragon WiFi	11g 2.4 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hays-CC:94	24	21	38	iPod	Resnet-18	RedDragonSecure	11g 2.4 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hays-CC:B0	0	0	26	iPhone	Resnet-14	RedDragonSecure	11g 2.4 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hays-CC:B2	30	36	27	iPhone	Resnet-18	Red Dragon WiFi	11g 2.4 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hays-CC:B2	44	0	-	OS X	Resnet-2	Red Dragon WiFi	11a 5 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hays-CC:C6	18	36	21	iPhone	Resnet-14	Red Dragon WiFi	11a 5 GHz	0 bps	-
8 minutes	11/12/14, 9:49 AM	Hulbert-B005.aruba.oneonta.edu	21	0	-	iPhone	Resnet-2	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
8 minutes	11/12/14, 9:49 AM	Hulbert-B010.aruba.oneonta.edu	32	0	34	iPhone	Resnet-14	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
8 minutes	11/12/14, 9:49 AM	Hulbert-B012.aruba.oneonta.edu	40	147	25	iPhone	Resnet-17	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
8 minutes	11/12/14, 9:49 AM	Hulbert-B013.aruba.oneonta.edu	23	0	9	iPhone	Resnet-4	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
8 minutes	11/12/14, 9:49 AM	Hulbert-B016.aruba.oneonta.edu	49	0	23	Win 7	Resnet-14	Red Dragon WiFi	11n 2.4 GHz	0 bps	HT20
8 minutes	11/12/14, 9:49 AM	Hulbert-B017.aruba.oneonta.edu	55	0	19	iPod	Resnet-11	Red Dragon WiFi	11n 5 GHz	0 bps	HT40
8 minutes	11/12/14, 9:49 AM	Hulbert-B035.aruba.oneonta.edu	30	91	131	Android	Resnet-13	RedDragonSecure	11n 5 GHz	0 bps	HT40

gilmorr,

 

There is no attachment.

 

Taking Ryan's advice one step further, you should coordinate your idle time-out, DHCP lease time, and 802.1X reauth intervals so tha they don't occur at the same time very often.  For example, If your idle timeout is 5 minutes, your reath interval is 10 minutes, and your DHPC time is 20 minutes, two of those will coincide every 10 minutes and all three every 20 minutes.  If you set them to 6 minutes, 11 minutes and 19 minutes, it will be an hour before 2 of them coincide, and a very long time before all three do.

I am having the same issues here.  I am running a single 7210 with around 500 AP's.  The complaints started out as gaming consoles and Apple TV's being disconnected every 10 minutes after upgrading from 6.4.1.0 to 6.4.2.2 because of the security flaw. 

 

However, it is not apparent its much more than the gaming consoles and apple TV's.  I have windows machines and OSX devices. 

 

I have noticed on a couple clients through AirWave that they have been moved frequenty due to client match. 

 

Curious if anyone has had any tweaks (not twerks :-) ) that have made a difference.

 

Thanks,
Chris

 

Lee University

#7210

---

@cgolden07 wrote:

I am having the same issues here.  I am running a single 7210 with around 500 AP's.  The complaints started out as gaming consoles and Apple TV's being disconnected every 10 minutes after upgrading from 6.4.1.0 to 6.4.2.2 because of the security flaw. 

 

However, it is not apparent its much more than the gaming consoles and apple TV's.  I have windows machines and OSX devices. 

 

I have noticed on a couple clients through AirWave that they have been moved frequenty due to client match. 

 

Curious if anyone has had any tweaks (not twerks :-) ) that have made a difference.

 

Thanks,
Chris

 

Lee University

---

 

We had noticed the same thing on windows, OSx, and iOS devices.  We worked with TAC on some client match tweaks and even more with a third-party vendor but it did not make much difference at all unfortunately.

#7210

This thread just popped up this morning and it almost describes what we have been seeing now since the start of the semester. I have been working with the TAC & Sales Engineer on this issue and we have adjusted max-retries and max-tx-fail... no change from user perspective. We then removed higher data rates and first reports were things seemed better. Then after a weekend reports that nothing is different.  We are running 6.3.1.13  and we have two  controller setups. 

A Master (3600) with two Local's (6000's), the other is a Master on a 7220 running the same OS. Now we just re-installed all new AP's in two buildings one with AP-225's and the other with AP-115's. These are on the newer controller. Users in these two buildings are not reporting the problem but all the users on the other are and they have been in place for a long time without problems. We are running client match on setups. So far this has peaked my interest and I'm just adding my voice and environment to maybe help figure it out.

 

Ronald R. Gilmore,

 

SUNY

ONEONTA

ITS NETWORKING &

TELECOMMUNICATIONS

 

#7220

I'm seeing some similar issues to this.

 

One thing I've done different than what I've read here is look at 

 

show ap client trail-info <client MAC>

 

I've been seeing "Ptk Challenge Failed" without "STA has roamed to another AP", :Unspecified Failure", and "Internal Deauth".

 

From what I've seen, Internal Deauth is client match trying to kick the station to another AP, but some clients don't seem to get it (MBP running Yosemite), especially when the kicking AP is .11ac (225) and the next available AP is .11n (135). In this case, I usually see Internal Deauth like 5 or 6 times in a row, every 10 seconds.

 

Generally, you always get a "Ptk Challenge Failed" on a roam, so  you'll see that along with a "STA has roamed to antoher AP". I see it sometimes without, several times in a row. I'm not sure what this means.

 

And, of course, "Unspecified failure" could mean anything.

 

 

 

Hope this helps.

We have been seeing this issue with OSx since 10.10.  Something interesting though - we had some of our users turn off bluetooth and they have not had a single disconnect...

 

I am still leaning to a driver issue and not a configuration issue...

We've also been dealing with this issue on 10.9 and 10.10, and have not yet found a solution. We're working with Apple and Alcatel support to find a fix. Both seem to think it's a bunch of small things that are all combining to form a big problem.

 

But specifically to address the 10.10 issue and bluetooth testing, has anyone checked out this link and tested with the command contained within for disabling AWDL? https://medium.com/@mariociabarra/wifried-ios-8-wifi-performance-issues-3029a164ce94

 

I've disabled that on my personal machine and I rarely see disconnects, although once in a blue moon it happens. It'd be awesome if others could test this out and get a community consensus on if the AWDL bluetooth issue is a contibuting factor.

We have this issue too and I think this is very relevant information from the Apple user forum :

https://discussions.apple.com/thread/6825371

 

Best regards

Rovinguser,

 

The thread specifically talks about 802.11h, which is not enabled by default.  Users who have 802.11h enabled should take a look at the Apple thread that you posted.

 

Thank you for that link.

I'm wondering if you're also facing issues with channel-changes either due to radar-detection (DFS) or ARM doing channel switches due to interference, high-noise or high error-rate. Please note all of these ARM-events and radar-detection are *NOT* client-aware. Channel switches will happen even if clients are connected.

 

At the CLI at the relevant local/master controller:

 

show log wireless all | include Radar

 

show log wireless all | include Noise

 

show log wireless all | include Interference

 

To see channel-switch history for a specific AP: 

 

show ap arm history ap-name <AP>

 

 

Also, we have seen issues with (especially MacOS) clients that do not handle the (background) scanning feature of ARM too well. In a default configuration AP's will be performing background scanning even if clients are connected. In a default configuration scanning will not happen if a voice-call exists on that radio or if 10Mbit/s of bandwidth is being used on that radio.

 

You can either disable ARM-scanning on the ARM-profile, but this is not advised unless you have dedicated AirMonitors in place. You can make a firewall-policy which will disable scanning if certain traffic exists on the radio. For example:

 

ip access-list session allowall
any any any permit disable-scanning
ipv6 any any any permit disable-scanning
!

---

@davidbr wrote:

Since everybody is throwing in their hat. 

We are seeing the issue here at Boston College.  Dedicated Master with 8 local's (6.3.1.8) 

Windows/Apple OS on various devices. 

 

---

Brian are you seeing a few disconnects or many disconnects?  Is there a pattern about where they are located?  If not, you might want to observe the controller's dashboard to get a bird's eye view of your environment and if there is somewhere that might warrant investigating.

 

---

@americanmcneil wrote:

Just wanted to toss my hat into the ring as I am seeing some of these same issues. Small shop here, 4 - 3000 series controllers, dedicated master (3200xm) with three locals (3200xm, 3400 & 3600), running 6.4.2.0 across the board. Currently about 200 AP's, mix of 93's, 105's and now 225's. I have been seeing it on Win7 machines, iPads, iPhones and rarely on an Android. I haven't seen it on win8 though... hmmm there is some food for thought.

 

Anyway, I was attributing a lot of it to needed updates to various wireless drivers and whatnot, there apparently a known issue with the latest iOS and there was an update released for "known WIFI related issues" (according to our web designers iPad anyway). Now I am seeing the similarities and am thinking this is the same issue as everyone else here. Ill be monitoring this thread to see how it progresses too.

---

americanmcneill,

 

If it is a code issue, typically when many users open tac cases, we try to find a commonality to determine what is the problem.  In the course of it, we will do a number of things to ensure that the RF itself is what we expect it to be and if it has enough capacity in your environment to support your current client load.  Looking at the dashboard on the controller and seeing your channel utilization on all of your access points is a good start to determine if you have an RF issue or not.  RF utilization should be at no more than 20% at midnight or when a an access point is being lightly used.  Start there to determine if you have an RF issue, if you can...  You can also open a TAC case in parallel to determine if there is something that can be done in the short term, or if you have a knob or counter that has been misconfigured that could contribute to the issue you are seeing.  If you do open a case, please write back to the thread so that we can leverage what you find.

 

---

@fkenner wrote:

We have seen the same issue and are running 6.3.1.10 with three M3 controllers (Master & two local) and about 590 AP's (105, 125, 134/5, 175 and some 225).  We are having random disconnects and dificulties reconnecting after (need to disable adapter or reboot computer).  More OS X users than Windows but not enough data to draw any defenite conclusions.

We got rid of some interference in one of the buildings and that helped a little bit with the number of times this happens.  However, students are still disconnecting randomly and the debug logs don't give any more information. We started getting reports of this problem about 4-5 weeks ago but we ran on the 6.3.1.10 long before that.  Please post if you were able to figure something out with TAC.

---

fkenner,

 

You are right: the debug log only offers a view of issues from the controller's point of view.  If a client cannot connect because of an RF issue, the client debug logs would not say specifically.  The unwritten rule, is if it happens in a specific location, the problem is the location.  If it happens with a specific client, the problem is with the client.  If it happens everywhere, it is either a global setting or a deployment issue.  Please take a look at the article here:  http://community.arubanetworks.com/t5/Technology-Blog/Removing-the-Bottleneck-in-Wireless/ba-p/77978 to see if it can give you some ideas..

 

I am seeing the exact same thing happen on my network. We have redundant 3600 masters, 2 7240's in one location, and 6 M3's split up evenly between 3 other locations. We have about 1,800 APs mixed between 105's, 125's, 135's and some 225's. We just upgraded to 6.3.1.10 from 6.3.1.8 this week to fix a different problem and it did not solve this issue. It does not seem to be happening on our 802.1x SSID, only our open SSID.

 

Tom

 

Johnson & Wales University

#7240

---

@tparenti wrote:

I am seeing the exact same thing happen on my network. We have redundant 3600 masters, 2 7240's in one location, and 6 M3's split up evenly between 3 other locations. We have about 1,800 APs mixed between 105's, 125's, 135's and some 225's. We just upgraded to 6.3.1.10 from 6.3.1.8 this week to fix a different problem and it did not solve this issue. It does not seem to be happening on our 802.1x SSID, only our open SSID.

 

Tom

 

Johnson & Wales University

---

tparenti,

 

When you say "the exact same thing", can you be more specific?  Do you have mixed capability access points within earshot of one another?  What type of area(s) do you have this issue?  What clients have this issue.  RF problems do not particularly care about the version of code.  If you have a fundamental issue in a specific area, upgrades or knobs typically can only do so much..

 

#7240

---

@cjoseph wrote:

tparenti,

When you say "the exact same thing", can you be more specific?  Do you have mixed capability access points within earshot of one another?  What type of area(s) do you have this issue?  What clients have this issue.  RF problems do not particularly care about the version of code.  If you have a fundamental issue in a specific area, upgrades or knobs typically can only do so much..

---

Exact same thing just in terms of hearing complaints about drops. Moslty from students in dorms that have AP125's on an open network. I had one call from a staff member in a building with AP 135's. I beleive she was also on the open network.

How close do you consider "earshot"? Out of 4 buildings I have heard about these drops only one has a mix. The dorm side has 125's and the office has 225's. I dont think they are close enough to cause any issue but I am not certain. On my phone I have not been able to reproduce the issue. I am in a building with AP 105's. I have a debug running on the staff members phone and so far she has not had this happen again.

tparenti,

 

Thank you for answering.

 

Let's focus on the dorm specifically:

 

Are the access points mounted in the halls or in the room?

 

Another institution checking in with this problem.  We are migrating for a lone 3600 controller to master/local 7210's, with approximately 150 AP's.  Most are AP-125's, but around 20 of the 225's as well.  We have been on 6.4.0.3 for quite awhile now.  Haven't had any reports of issues until recently.  Unlike the previous poster, the only reports we are getting are on our 802.1X network, not the other open/captive portal networks.  

 

We have had issues with machines taking 5+ minutes to be able to login (machine not picking up the AP), machines dropping connections during usage, etc.  We have just begun troubleshooting in earnest, but if I find out anything I will post it here.

 

Patrick

Massasoit Community College

#7210

---

@pmauretti wrote:

Another institution checking in with this problem.  We are migrating for a lone 3600 controller to master/local 7210's, with approximately 150 AP's.  Most are AP-125's, but around 20 of the 225's as well.  We have been on 6.4.0.3 for quite awhile now.  Haven't had any reports of issues until recently.  Unlike the previous poster, the only reports we are getting are on our 802.1X network, not the other open/captive portal networks.  

 

We have had issues with machines taking 5+ minutes to be able to login (machine not picking up the AP), machines dropping connections during usage, etc.  We have just begun troubleshooting in earnest, but if I find out anything I will post it here.

 

Patrick

Massasoit Community College

---

pmauretti,

 

What is the operating system of the machine is this taking 5+ minutes to login?  Do you have machine authentication configured on that computer.  Is it new users or users who have their profiles cached that are taking a long time?  Is the computer connected to an access point that already has 30+ users connected to it?  Do you use roaming profiles so that users have to download megabytes of traffic when they login?  5 minutes + should not be happening, but if it is, there is a reason.

 

#7210

These are mainly Windows 7 machines.  Machine authentication is turned on.  Most times they are able to login fine.  The profiles are new, as the machine is wiped clean when the PC is restarted via Deep Freeze.  We have seen the issue crop up on highly utilized AP's as well as ones with low utilization.  

 

The approximately 5-10 minute timeframe is not them waiting for it to login.  It is the machine not being able to authenticate itself on the GPO configured wireless network, as the user receives a no logon server available message until it clears up.  Reboots do not seem to affect it.  Eventually they are able to login as normal.  However, there are also issues of disassociations during sessions.  The only observable behavior I have noticed on the client side, is that the machine has a DHCP lease, but the lease timer is way off.  A release and renew usually will get them connected again, with a correct lease and the same IP, but the other issue of logging in still exists.  

Pmauretti,

There are a few things at play in your setup. The most important thing is when does your computer even attempt to machine authenticate to the wireless network. Nothing can happen until then...

If the machine has a slow bootup or does not attempt to connect to the network, no logon servers will appear and there is nothing that we can do.

The question is, is this problem pervasive and when it happens, has the computer already logged into the wireless as a machine? If yes, how do you determine that?

Other than watching the monitoring in real time when a report comes in, I have been basing it on a machine successfully booting up and the person being able to login 95%+ of the time, so that I had always assumed all was well on the configuration portion of it.  I may be assuming too much, but the machines did not seem to have an issue until recently.  These are different models of PC, with different nic cards, in different buildings and all.  All running Windows 7, though.

 

I know machine authentication is enabled, as a computer will present itself in the console as host/machinename when it does, and I see the appropriate role.  I haven't been lucky enough to see whether it is in a proper state as the issue is happening, unfortunately, as the reports do not get to me quickly enough.  The one time I saw the issue myself, foolishly I did not check the console to see what it showed up as.  

 

I will try to recreate the problem to get more diagnostic information.

 

Thank you everyone that has chimed in on this ticket!  Knowing other schools are having the problem helps, as well as hearing suggestions and questions.

 

Not much to add.  We tried turning off Client Match but still had the problem.   Not being able to reproduce the problem at will is really frustrating the effort.  Most recently we have moved some of the students experiencing the problem to the eduroam SSID to see if they still have the problem.  The hope being that this would eliminate some variables since the eduroam SSID is still an dot1x SSID but has fewer changes from the default options and Bradford is not part of the equation at all.   Divide and conquer.

 

If anything changes I will let you know.

I have been going over what is going on here and making a time line of all changes upgrades and etc that have gone on. We upgraded our Aruba controller's from 6.2.1.3 to 6.3.1.5 back in April of this year. We wanted to start to take advantage of Airgroups. Well it wasn't but a week in to the semester when we started getting a few reports of drops and as the semester is going on we have gotten much more. Now we have upgraded a few more since then to the current code. I see that there are reports on the 6.4 code as well but has any one heard or seen this dropping issue occured in the previous 6.2 and back versions? I have put this to my Ticket and Sales engineer as well but I thought I'd also put this out there to see if it indeed is a pattern. Or is maybe something to do with Airgroups. I've speculated about disabling that as well. 

 

Ron Gilmore

IT - Network Operations

SUNY Oneonta

Hey there, 

 

So we have come accross similar issues here at liberty and wanted to see if anybody else is seeing this. We first reviewed our 802.1x (AAA) timers, idle timeout, Station ageout and updated our client match band steering thresholds. After further investigation we took three clients with the disconnect issue and monitored over a 24 period. During this time i found multiple channel changes were occuring. 

 

AoS 6.3.1.13

HA 7220 Masters and 4 local 7220s the 4th being N+1.

Issue has been occuring since semister startup.

We use 4 20k CPPM physical nodes behind an F5 for Radius Auth Airgroup, Guest, etc.

 

When running the show ap arm history command we saw a lot of channel changes with the reason "E: Error threshold exceeded". 

 

Interface :wifi0
ARM History
-----------
Time of Change Old Channel New Channel Old Power New Power Reason
-------------- ----------- ----------- --------- --------- ------
2014-11-06 14:48:09 149+ 149+ 21 24 P+
2014-11-06 14:42:04 149+ 149+ 18 21 P+
2014-11-06 10:51:53 149+ 149+ 24 18 P-
2014-11-06 02:00:13 161- 149+ 24 24 E
2014-11-05 19:40:10 153- 161- 24 24 E
2014-11-05 19:39:08 157+ 153- 24 24 E
2014-11-05 19:29:38 153- 157+ 24 24 E
2014-11-05 19:14:06 153- 153- 21 24 P+
2014-11-05 18:42:08 153- 153- 18 21 P+
2014-11-05 14:59:19 40- 153- 18 18 E
2014-11-05 14:58:13 40- 40- 24 18 P-
2014-11-05 14:51:44 157+ 40- 24 24 E
2014-11-05 13:12:03 157+ 157+ 21 24 P+
2014-11-05 12:49:47 157+ 157+ 18 21 P+
2014-11-05 12:36:43 157+ 157+ 24 18 P-
2014-11-05 11:47:33 40- 157+ 24 24 E
2014-11-05 10:06:35 40- 40- 21 24 P+
2014-11-05 10:02:18 40- 40- 18 21 P+
2014-11-05 09:50:39 40- 40- 24 18 P-
Interface :wifi1
ARM History
-----------
Time of Change Old Channel New Channel Old Power New Power Reason
-------------- ----------- ----------- --------- --------- ------
2014-11-05 06:43:09 11 11 15 12 P-
2014-11-05 06:38:51 11 11 12 15 P+
I: Interference, R: Radar detection, N: Noise exceeded, Q: Bad Channel Quality E: Error threshold exceeded, INV: Invalid Channel, G: Rogue AP Containment, M: Empty Channel, P+: Increase Power, P-: Decrease Power, 40INT: 40MHZ intol detected on 2.4G, NO40INT: 40MHz intol cleared on 2.4G, OFF: Turn off Radio, ON: Turn on Radio

(JFL-local-01) #show ap association ap-name RCH01-A827-AP135

The phy column shows client's operational capabilities for current association

Flags: A: Active, B: Band Steerable, H: Hotspot(802.11u) client, K: 802.11K client, R: 802.11R client, W: WMM client, w: 802.11w client

PHY Details: HT : High throughput; 20: 20MHz; 40: 40MHz
VHT : Very High throughput; 80: 80MHz; 160: 160MHz; 80p80: 80MHz + 80MHz
<n>ss: <n> spatial streams

 

Association Table
-----------------
Name bssid mac auth assoc aid l-int essid vlan-id tunnel-id phy assoc. time num assoc Flags Band steer moves (T/S)
---- ----- --- ---- ----- --- ----- ----- ------- --------- --- ----------- --------- ----- ----------------------
RCH01-A827-AP135 24:de:c6:0c:4c:91 7c:7a:91:cd:de:21 y y 1 250 Liberty-Secure 3801 0x104af a-HT-40sgi-2ss 2h:54m:52s 1 WAB 0/0
RCH01-A827-AP135 24:de:c6:0c:4c:82 50:1a:c5:c0:f3:0d y y 1 10 Liberty-Wireless 3920 0x104b4 g-HT-20-2ss 21h:29m:43s 1 WAB 0/0
RCH01-A827-AP135 24:de:c6:0c:4c:91 60:03:08:8f:05:5c y y 3 10 Liberty-Secure 3825 0x104af a-HT-40sgi-3ss 42m:59s 1 WAB 0/0
Num Clients:3
Total num of 5G capable clients:3
Total num of 5G capable clients in 2.4G band:1
Total num of 5G capable clients in 5G band:2
Total num of 2.4G only clients:0

 

When conducting a client trail we found the following:

 

Client Trail Info
-----------------
MAC BSSID ESSID AP-name VLAN Deauth Reason Alert
--- ----- ----- ------- ---- ------------- -----
50:1a:c5:c0:f3:0d 24:de:c6:0c:4c:82 Liberty-Wireless RCH01-A827-AP135 3920 Unspecified Failure Unspecified Failure

Deauth Reason
-------------
Reason Timestamp
------ ---------
Unspecified Failure Nov 5 19:39:07
Unspecified Failure Nov 5 19:29:37
Unspecified Failure Nov 5 14:59:19
Unspecified Failure Nov 5 14:51:43
Unspecified Failure Nov 5 11:47:32
STA has left and is deauthenticated Nov 5 11:34:02
STA has left and is deauthenticated Nov 5 11:33:08
STA has left and is deauthenticated Nov 5 11:30:33
Num Deauths:8

Alerts
------
Reason Timestamp
------ ---------
Unspecified Failure Nov 5 19:39:07
Unspecified Failure Nov 5 19:29:37
Unspecified Failure Nov 5 14:59:19
Unspecified Failure Nov 5 14:51:43
Unspecified Failure Nov 5 11:47:32
STA has left and is deauthenticated Nov 5 11:34:02
STA has left and is deauthenticated Nov 5 11:33:08
STA has left and is deauthenticated Nov 5 11:30:33
Num Alerts:8

Mobility Trail
--------------
BSSID ESSID AP-name Timestamp
----- ----- ------- ---------
24:de:c6:0c:4c:82 Liberty-Wireless RCH01-A827-AP135 Nov 5 19:39:10
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 19:39:07
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 19:29:41
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 19:29:37
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 14:59:24
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 14:59:19
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 14:51:48
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 14:51:43
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 13:29:07
24:de:c6:0c:4c:92 Liberty-Wireless RCH01-A827-AP135 Nov 5 11:47:32
Num Mobility Trails:10

 

In most clients when we see "Unspecified Failure" it matches up with the channel changes that reside with the threshold errors. Currently we have a known issue with our internet pipes being over subscibed and are in the process of upgrading to 10gb uplings. I think some of what were is airtime being crowded due retrys however when we look at the controller the channels look healthy for this ap on the 5ghz.

 

Airwave RF capacity statisics below for the follow A radio. The WAP is an AP135 that averages 8 clinets per radio. 

 

 Last Min Max Avg 

Busy	16.93 %	1.57 %	33.46 %	11.42 %
Interference	1.57 %	0 %	24.41 %	1 %
Receiving	9.84 %	0.79 %	30.71 %	8.27 %
Transmitting	6.69 %	0 %	16.93 %	2.76 %

 

 

AoS 6.3.1.13

HA 7220 Masters and 4 local 7220s the 4th being N+1.

Issue has been occuring since semister startup.

We use 4 20k CPPM physical nodes behind an F5 for Radius Auth Airgroup, Guest, etc.

#7220

Tnorton7,

Why is your access point at power levels 21 and 24? It should probably be no more than 18 or you will observe interference like you have seen.

in addition, since you have 40 megahertz channelsyour access points only have 4 channels to go to avoid interference. You should use 20 megahertz channels and it will give youmore room.

Hey Colin, That makes sense to me as of right now we are still using the default ARM profiles on a lot of our AP groups. In terms of 20 meg channels I agree with you there. We have been working to move away from 40 mhz channels in our dense enviorments but have yet to make the change. Ron is planning on coming out the 19th and that is one of the areas i wanted to get his input on. We will do some reading up on what changes need to be made in this area and coordinate with you guys. Thanks for the input! 

---

@tnorton7@liberty.edu wrote:

Hey Colin, That makes sense to me as of right now we are still using the default ARM profiles on a lot of our AP groups. In terms of 20 meg channels I agree with you there. We have been working to move away from 40 mhz channels in our dense enviorments but have yet to make the change. Ron is planning on coming out the 19th and that is one of the areas i wanted to get his input on. We will do some reading up on what changes need to be made in this area and coordinate with you guys. Thanks for the input! 

---

tnorton7,

 

You can go to 20mhz channels without any issues, because all clients support 20mhz channels.  You also gain 5 channels in the process and reduce your interference footprint.  The power of your access points is also crucial because many, many performance and disconnection issues have too high or too lower power at the root of those issues.  

 

Everyone on this thread should first check the range of their power on their access points (12 to 18 range) to ensure that they are not creating or contributing to their issues.

 

---

@gilmorrr wrote:

I have been going over what is going on here and making a time line of all changes upgrades and etc that have gone on. We upgraded our Aruba controller's from 6.2.1.3 to 6.3.1.5 back in April of this year. We wanted to start to take advantage of Airgroups. Well it wasn't but a week in to the semester when we started getting a few reports of drops and as the semester is going on we have gotten much more. Now we have upgraded a few more since then to the current code. I see that there are reports on the 6.4 code as well but has any one heard or seen this dropping issue occured in the previous 6.2 and back versions? I have put this to my Ticket and Sales engineer as well but I thought I'd also put this out there to see if it indeed is a pattern. Or is maybe something to do with Airgroups. I've speculated about disabling that as well. 

 

Ron Gilmore

IT - Network Operations

SUNY Oneonta

---

gilmorr,

 

Airgroup is not directly related to client connectivity.  The only one I can think of is if you are using Airgroup with ClearPass and the radius server is overloaded, so no 802.1x authentication can take place.  Airgroup "not working" would result in clients not seeing airplay and airprint devices, but not specifically client connectivity.  Like you mentioned, you can turn Airgroup off just to double-check...

Hi Aaron,

 

Like everyone else has mentioned on the forums, you are not alone.  We've also noted isolated instances where this appears to be occuring @ Drexel.  

 

We were first alerted to the situation w/ a campus in Sacramento, CA.  The report focused on the specific issue of iPads being disconnected & searching for wifi & then re-connecting; never explicitly needing to re-authenticate, but interrupting flows while it scans & re-associates w/ the wifi network.  We disabled Client Match for the profile in use there and have not heard of any major complaints since then; which reminds me that I need to follow up.  

 

I've also heard scattered reports from various individuals around campus; not a lot, less than 5 - 7.  However, for those that occur in a specific location / building, we can also disable Client Match on the AP Group Profile, should the need arrise.  

 

We're running AOS 6.3.5 & the release notes for 6.3.7 state that they've added an Client Match config parameter to help lessen its aggressiveness.  Hopefully, upgrading to the latest stable release will help alleviate any issues.  

 

Since not that many users complain, we aren't turning off Client Match campus wide just yet.  For those that do, I've been disabling Client Match on individual machines w/ a command Gallagher provided me.

 

add ap arm client-match unsupported <mac>

 After doing so I've tried to maintain an eye on the client in Airwave & have ask them to follow up if they notice any significant change.  

 

Possibly related to this whole mess is wifi issues w/ Mac OS X Yosemite; not sure how much of your client base is Mac OS X & what percentage of them have upgraded.  See this PC World article.   I just added a 10.10 user to the client-match unsupported list and did not immediately observe any noticable change in their Association History via Airwave.  I've asked the user to comment & they did note that the issue appeared to still occur; in which case, I'm assuming its OS based.  

 

BTW, after disabling Client Match in Sacramento, we DID observe a substantial decrease in the number of association history entires; which makes sense being that they're no longer being steered.  More importantly, the complains have seemed to have subsided.  

 

I'm using the client-match unsupported to try and handle those that bring up issues about constant disconnections.   If they're running Mac OS X 10.10 & they constantly get disconnected, then disabling Client Match will hopefully improve the situation; but if it doesn't, then the issue might be w/ the OS.  If the complaining machine is a non Mac OS computer, then if disconnections lessen, it only lets us know that Client Match has been set too aggressively.  

  

 

 

---

Raf,

 

I believe a full post-mortem of what is going in those specific portions of your network is warranted.  Adding devices to the unsupported list is not scalable and is not encouraged.  

 

 

This is a good thread. Thanks for contributing. We received lots of complaints when moving from 6.2.x to 6.3.1.5. It looked like clients were being pushed to APs that made no sense. We(TAC) disabled client match for a while and things improved. We enabled it again when we upgraded to 6.3.12 and we are seeing some complaints. My gut feeling is there are several problems, client match, inadequate 5 Ghz coverage, and client drivers. I'm looking forward to trying to disable client match on a per client basis to see if that helps those that are most vocal. :-) Thanks for the tip.

 

 

---

@rwilsonblue wrote:

This is a good thread. Thanks for contributing. We received lots of complaints when moving from 6.2.x to 6.3.1.5. It looked like clients were being pushed to APs that made no sense. We(TAC) disabled client match for a while and things improved. We enabled it again when we upgraded to 6.3.12 and we are seeing some complaints. My gut feeling is there are several problems, client match, inadequate 5 Ghz coverage, and client drivers. I'm looking forward to trying to disable client match on a per client basis to see if that helps those that are most vocal. :-) Thanks for the tip.

 

 

---

rwilsonblue,

 

The difficulty with troubleshooting something that is random is that you do not have any way of measuring whether or not you made progress.  Was it clientmatch?  Was it the user not reporting issues?  Was it the fact that all of the circumstances that create poor wifi did not line up to create a problem after you did what you think is the solution?

 

The real solution is to look at the network wholistically and see if you are serving up good RF and are giving all of your clients to work in good situations and are you giving them enough room to still operate when things get challenging.  Please read the article here:  http://community.arubanetworks.com/t5/Technology-Blog/Removing-the-Bottleneck-in-Wireless/ba-p/77978 to go through a list of things to check to make sure your wifi is at least configured correctly to start.  It is not an all inclusive list, but it is a start....

 

I'd be curious if anyone else has tried turning off client-match.  @Raf let me know if you do go campus-wide.  We're still working on this from all angles and I've got some really good tech support with TAC at the moment.  I'll keep the thread in the loop with any progress we make.

 

I'd be curious what percent other campuses are seeing for clients on the 2.4G band vs. the 5G band day-to-day on average too.  We tend to hover around 60% of clients are on 2.4 no matter what our arm settings are.  It may be those old thick walls.

 

Colin I did miss a few slices to that pie; I've since added 'coverage' as well.  We did add close to 100 APs since last year but I think the amount of clients and total spatial streams is up quite a bit.  That and maybe the clients desire better, stronger signals.  Seth (@sethfiermonti) posted a great apple link (http://support.apple.com/en-us/HT6463).  

     

if that same user is not in a call, or transmitting or receiving a series of data packets, then iOS 8 only considers BSSIDs with an RSSI of -63 dBm or better.

I know throughout our dorms we're not -63 or better in all rooms.  We're working on that.  We provision and loan out AP-105's to students and we've seen that help.

 

pgemme,

 

I will tell you that the biggest two issues are (1) defining your specific problem in a meaningful way and (2) measuring what you changed is indeed making a difference.

 

The most important thing to understand is that whatever wireless you have is built on the RF, so you need a way to evaluate whether the RF is solid to start.  When I say solid, it should be able to work when there is nobody in the environment, but also work if we suddenly double the number of users on that access point.  The best way is to look at at area where you have the most issues reported, because it will be easier to know if the problem is fixed thereafter.  Dealing with that specific area would present the best opportunity for you to notice other things about your environment that could be contributing to your problem.  To get accurate and timely reporting you might have to walk the area and solicit feedback from users about their experience or engage a user who seems to have more issues than any other.  Accurate and timely reporting are a key to understanding the client experience that user debugs and RF telemetry do not provide by themselves.

 

 Many things that are problems in wifi contribute to other existing issues, but do not by themselves create an issue.  TAC will remove clientmatch as a way to take it out of the equation to make contributory issues more obvious.  Here is a list of things you can check:

 

- Is my RF utilization on that access point under 20% on both bands when few if any users are on it?  During production hours does my utilization show more receiving or transmitting?

- Is my RF noise in the -80 range at any time?

- Remove things that influence client behavior like Local Probe Response Threshold and Cell Size Reduction

- Make sure your access point is min and max TX are at 12 and 18 to match the power of clients that would be connecting to it.

- How many access points can I see on the same channel as my access point with an RSSI of over 20 when I run the command "show ap arm state ap-name <name of ap>"?  What is their transmit power?

- Is drop broadcast and multicast enabled on ALL Virtual APs in your AP group so that stray broadcasts and multicasts do not add additional traffic to your network?

- On my client, how many access points can see on the same ESSID and can I roam to them easily?

 

 

 

---

@cjoseph wrote:

- Make sure your access point is min and max TX are at 12 and 18 to match the power of clients that would be connecting to it.

 

 

---

Hi Colin,

 

Just to clarify, are you referring to 'min-tx-power' and 'max-tx-power' commands within the ARM profile? Thismight help those who have the default value (127).

 

From the 6.3 UG:

 

max-tx-power

Maximum effective isotropic radiated power (EIRP) from 3 to 33 dBm in 3 dBm increments. You may also specify a special value of 127 dBm for regulatory maximum to disable power adjustments for environments such as outdoor mesh links. This value takes into account both radio transmit power and antenna gain. Higher power level settings may be constrained by local regulatory requirements and AP capabilities.

 

Range:  3, 6, 9, 12, 15,18, 21, 24,27, 30, 33,127

 

Default: 127 dBm

mldickson,

 

Yes.  Max-TX-Power in the ARM profile for the 802.11a band 18.  Min-TX-Power in the ARM profile on the 802.11a band 12.

 

There is nothing wrong with starting off the 802.11g ARM profile with these same settings;  users with significant density might want to change the 802.11g profile to MAX 15 and MIN 9.

 

 

 

@pgemme,

 

At the moment, THERE ARE NO PLANS TO DISABLE CLIENT MATCH CAMPUS WIDE.  I agree with @cjoseph in that disabling Client Match on individual clients isn't scalable; not that we've reached any of those limits, but the handeling of those individuals is a PITA. 

 

There isn't a large percentage of clients that are reporting a constant disconnect issue & we're taking those that do on an individual case by case basis.  As we all know, clients rarely take the time to report an issue unless it is massively affecting their connection.  

 

If a client does open a ticket, I first ask if they've upgraded to OS X 10.10 & if they recall having issue prior to upgrading; we can always use the info Airwave reports, but its helpful to underscore basic troubleshooting proceedures.  

 

I then evaluate the number of Client Match events & the number of assoications reported in Airwave & make a judgement as to whether or not place the client in the unsupported list.  I try to work with each individual that opens a ticket so that they can give us constant feed back, once we make changes.   

 

I also place them in the user debug list & run a couple of Air Recorder script to gather additional troubleshooting information so that I can open a TAC case with substancial info as needed.  

 

AFAIK, RF has not been an issue.  We didn't make any drastic changes to the RF config in any of the buildings, & if we did, we did so by carefully taking a look at the previous configs & re-surveying once we made those changes.  The major difference in our environment is the introduction of Client Match & 6.3.1.x code.  Additionally, both the RF Capacity & RF Performance, across campus, has been better with the config changes we've made & the upgrade to AOS 6.3.1.x.   The only major thing I can think of would be some kind of rougue or external interferrence, but nothing has jumped out at me to begin suspecting such a thing; especially given the frequency of associations. 

 

Since our AOS upgrade, I've had the opporutnity to work with RonZ & our local SE to review our environment & some useful Air Recorder scripting (I believe there's a solution in the ASE ; I highly recommend checking that out, beats having to write expect scripts).  Static configs aren't exactly new, but they do require some additional thought on implementation.  Now that we've got the AP density needed in most places, I am starting to look down that path & I'm very excited.

 

Of the 10 or so issues reported, only 2 of them have been non-Mac OS X OS'es.  I believe 3 of them have been issues directly related to their OS X 10.10 upgrade, the rest I'm still actively troublshooting with.  I've really only added 2 clients to the client-match unsupported list; it seems as if only one of them has experienced a positive change where as the other one still shows a great number of associations & continues to report constant 'disconnects'.  

 

As far as the location where we've disabled Client Match...  Offically, we have yet to hear back.  When we look at those clients that were complaining, we definately don't see as many associations in their history as when Client Match was enabled (we're talking going from 25-30 assoications, down to 4-5 throughout the day).    

 

To follow up on the 2.4 vs 5 GHz...

 

During our migration we turned on 802.11n in 2.4 GHz & 5 GHz (this was a decision that was made back in the pre-N days) & made additional changes to the rates we support.  In most places, we increased the number of users in 5 GHz, however overall, we're seeing the same 60%, maybe even as much as 50/50 on a given day.  Besides ARM, I would suggest you take a look @ the rates you're supporing in your SSID. 

 

 

We have just upgraded one of our local controllers as a stand alone master running 6.4.2.2. We disabled the 80MHz channel and lowered the EIRP to max 18 min 12.

We`re not certain how they got increased to the values they were set to. We have noticed a large improvement at this site and are going to roll out similar changes to a few other sites to see.

 

Were you using defaul arm/radio profiles or your own?

One thing to check is your ARM profiles and your APs for channels changes as well look for the Reason if you see alot of "E"s or "I"s that is a problem. The way around this is to look at the areas, density of the walls, and clients.

 

Complete "show ap arm history ap-name" and select clients with known issues and look for "unspecified error" when running the "show ap client trail-info" command. First if your seeing alot of "E" flags I would recommend updating your "Error Rate Threshold" to 90 % and "Error Rate Wait Time" to 60 seconds. As always this depends on your enviroment.

 

If your also seeing a decent amount of "I" flags i recommend adjusting EIRP as needed. However, rember each APs EIRP is regualted so also verify using "show ap allowed-max-EIRP ap-name "name" country-code us.

 

Below is a good example of one of our updated dorm configs and as always all of this depends on your enviroment.

 

rf ht-radio-profile "ht-radio-2.4Ghz-dorm-room-sparse-hard"
!

rf arm-profile "arm-2.4Ghz-dorm-room-sparse-hard"
min-tx-power 12
max-tx-power 12
client-aware
no rogue-ap-aware
no active-scan
ota-updates
scanning
multi-band-scan
voip-aware-scan
no ps-aware-scan
video-aware-scan
ideal-coverage-index 2
acceptable-coverage-index 2
free-channel-index 40
backoff-time 1800
error-rate-threshold 90
error-rate-wait-time 300
load-aware-scan-threshold 1250000
no mode-aware
client-match
cm-report-interval 30
cm-unst-ageout-intvl days 2 hours 0
cm-band-a-min-signal 75
cm-sticky-check-inter 3
cm-sticky-snr 18
cm-sticky-snr-delta 10
cm-sticky-min-signal 65
cm-steer-timeout 3
cm-lb-thresh 20
cm-stale-age 120
cm-max-steer-fails 2
cm-lb-client-thresh 30
cm-lb-snr-thresh 30
cm-lb-signal-delta 5
cm-band-g-max-signal 45
cm-steer-backoff 300
!

rf dot11g-radio-profile "radio-2.4Ghz-dorm-room-sparse-hard"
high-throughput-enable
arm-profile "arm-2.4Ghz-dorm-room-sparse-hard"
ht-radio-profile "ht-radio-2.4Ghz-dorm-room-sparse-hard"
!

rf arm-profile "arm-5Ghz-dorm-room-sparse-hard"
max-tx-power 18
min-tx-power 18
client-aware
no rogue-ap-aware
no active-scan
ota-updates
scanning
multi-band-scan
voip-aware-scan
no ps-aware-scan
video-aware-scan
ideal-coverage-index 10
acceptable-coverage-index 4
free-channel-index 25
backoff-time 240
error-rate-threshold 90
error-rate-wait-time 300
load-aware-scan-threshold 1250000
no mode-aware
client-match
cm-report-interval 30
cm-unst-ageout-intvl days 2 hours 0
cm-band-a-min-signal 75
cm-sticky-check-inter 3
cm-sticky-snr 18
cm-sticky-snr-delta 10
cm-sticky-min-signal 65
cm-steer-timeout 3
cm-lb-thresh 20
cm-stale-age 120
cm-max-steer-fails 2
cm-lb-client-thresh 30
cm-lb-snr-thresh 30
cm-lb-signal-delta 5
cm-band-g-max-signal 45
cm-steer-backoff 300
!
rf ht-radio-profile "ht-radio-5Ghz-dorm-room-sparse-hard"
!

rf dot11a-radio-profile "radio-5Ghz-dorm-room-sparse-hard"
high-throughput-enable
very-high-throughput-enable
arm-profile "arm-5Ghz-dorm-room-sparse-hard"
ht-radio-profile "ht-radio-5Ghz-dorm-room-sparse-hard"
!

rf arm-profile "arm-5Ghz-dorm-room-sparse-hard"
no 80MHz-support
!

rf arm-profile "arm-2.4Ghz-dorm-room-sparse-hard"
40MHz-allowed-bands a-only
!
rf arm-profile "arm-5Ghz-dorm-room-sparse-hard"
40MHz-allowed-bands a-only
!

wlan ht-ssid-profile "ht-ssid-RHM-17"
!
wlan ssid-profile "Liberty-Secure-RHM-17"
clone "Liberty-Secure"
a-basic-rates 12
a-tx-rates 12 18 24 36 48 54
g-basic-rates 11 12
g-tx-rates 11 12 18 24 36 48 54
mcast-rate-opt
ht-ssid-profile "ht-ssid-RHM-17"
g-beacon-rate 11
a-beacon-rate 12
!
wlan virtual-ap "Liberty-Secure-RHM-17"
clone "Liberty-secure"
ssid-profile "Liberty-Secure-RHM-17"
broadcast-filter arp
broadcast-filter all
!

Like Raf stated above another thing to check is your data rates, however i would consult aruba always before making any changes. 

 

Air recorder and visual config can be good tools to use when making changes or looking for issues. We have also found ase.arubanetworks.com to be helpful .

 

tnorton7,

 

Thanks for that information.

 

In the vast majority of installations the power just needs to be adjusted to achieve markedly improved results.  It is entirely possible that your specific installation had a requirement that necessitated other things to be adjusted or fine-tuned.  Since you were in the hands of a professional, his assessment of your individual deployment probably gave him enough information to make the changes you stated.

 

I would not change the noise parameters or most of those settings unless under the direction of a professional or under instructions from TAC.

Colin, I agree, Always consult colegues and Aruba folks for validation before making changes.

 

In our enviroment this is actually something new we are trying in some of our dorms. Mind you the above config peice was only meant to be an example and is specific per ap-group.  

 

Most of our ARM profiles power levels were and still are set to min 9min/18max, or 12 min/18max based on the enviroment. 

 

 In terms of error thresholds no matter what enviroment "E" and "I" flags are never good when seen in consistant intervals. Espeicially when the time stamps match up with client trail-info.

I wanted to give an update on how things turned out.  We ended up closing the ticket due to a lack of data.  Sadly, we could never recreate the problem or capture enough data with debugs to identify an actionable problem.  So the habit of turning off/on their wifi adapter is still the quickest fix for the problem.

 

Thanks for all of the great suggestions these last few months and good luck to those still working through their own tickets.

 

Aaron

I'll tack on to that.  We are a relatively new Aruba shop and are still working things out as we upgrade from our older Cisco wireless.  Similar to others, we had reports of disconnects and issues and went through some troubleshooting and tickets.  We were watching this thread with interest thinking it sounded like there was some global Aruba issue that was going to be sussed out.

 

In a number of cases, we updated client software to fix the issue (some known Yosemite updates, some very old Ubuntu digital signage installations, and so on).  In other cases, we are still in the process of tweaking our AP placements and power settings.  Coming from Cisco, our first pass was to put an Aruba AP wherever we had a Cisco one.  While the Cisco APs were mostly omni-directional antennas, the Arubas are designed to hang on the ceiling grid and aim down.  In some spots, this caused us to rethink and shift our AP placements.  In others, we simply did not have enough APs for coverage and ended up placing more.  Indeed, AirWave showed us that some clients were being moved due to "load balancing" in places where we were getting reports from.  So, we ended up solving a lot of our own "disconnect" issues, and we continue to revist AP placements, etc. across campus.

 

The part of this thread that I can't comment on (and can't get out of my head) is the reports by some that "we've had Aruba a while and this just started happening on the newer OS versions".  We started on 6.4.x.x so we've known nothing else.

 

At any rate, I would say it is certainly worth revisiting "the basics" of wireless deployments to see if things like that might be in play.

For areas where ceiling mount APs are not desired, we have been using APs with onmidirectional external antennas (AP-224 instead of AP-225).

 

As Aruba keeps adding more features and enhancing existing ones, sometimes issus are created and wireless strategies are changed to ultimately make things work better.

 

You should expect a few more issues on ArubaOS 6.4 because the OS is a Technical Preview used to find and resolve more bugs. Some companies that have a specific need for the new features run 6.4 too. We are running the lates 6.3 General Availability release because, currently, reliability is one of our top concerns and we have not yet seen a 6.4 feature that says we need to switch yet. We also have some older APs that will not work on 6.4 and must be replaced.

 

This Aruba Solution should give you some good wireless design ideas. It is primarily written bu one of the advanced Aruba Support Engineers. https://ase.arubanetworks.com/solutions/id/75

---

aaron.smith@swarthmore.edu wrote:

I wanted to give an update on how things turned out.  We ended up closing the ticket due to a lack of data.  Sadly, we could never recreate the problem or capture enough data with debugs to identify an actionable problem.  So the habit of turning off/on their wifi adapter is still the quickest fix for the problem.

 

Thanks for all of the great suggestions these last few months and good luck to those still working through their own tickets.

 

Aaron

---

We had the same exact issue with our ticket.  The issue was non-reproducable and even when we could collect data with an engineer on the line the data was incomplete or inconclusive.

I also want to provide an update. While troubleshooting reports of dropped connections and other ephemeral anomalies reported in this thread, we did a comprehensive analysis of all ancillary services supporting wireless. This included DHCP, DNS, and the authentication backend infrastructure.  We discovered that during high mobility events (class changes) delays increased from some point downstream of our radius servers, causing "back pressure" on the radius servers, but no auth dead events.

 

To the device this manifested as dropped connections on 802.1X, particularly in dense environments (high roaming events). To the user this is perceived as degradation and dropped connections.

 

After some digging it was determined that saslauthd (provides ldap -> kerberos lookups) was slogging during these peak times. We resolved that issue and quickly found that the radius servers were now insufficient to handle to newly-unconstrained throughput. We doubled the amount of radius servers serving .1X . We also physically moved them closer to the controllers, direct-connected them to the same router hosting our controllers. This reduced packet hop count and redundant traverasl of established authentication network links. This eliminated the timeouts, even during high mobility events, and reduced radius auths-per-second to a more reasonable number.

 

Having resolved the backend auth delays we are now getting reports of happier clients and can verify faster roaming events and handoffs. We cleared out our t-ticket queue and are now looking at new trouble reports with a fresh perspective.

 

While this certainly won't apply to many (most?) who are experincing issues the take away for us was to look deeper into *all* components that make up what we call "wireless".

 

Mike

 

Great post. Highlights that the scale of networks and large mobility events need tolooked at holistically.

Great write-up Mike, thanks!

 

This also helps to highlight why EAP-TLS is preferred over EAP-PEAP and EAP-TTLS when feasible. LDAP/AD add additional latency to an authentication request.

---

@mldickson wrote:

I also want to provide an update. While troubleshooting reports of dropped connections and other ephemeral anomalies reported in this thread, we did a comprehensive analysis of all ancillary services supporting wireless. This included DHCP, DNS, and the authentication backend infrastructure.  We discovered that during high mobility events (class changes) delays increased from some point downstream of our radius servers, causing "back pressure" on the radius servers, but no auth dead events.

 

To the device this manifested as dropped connections on 802.1X, particularly in dense environments (high roaming events). To the user this is perceived as degradation and dropped connections.

 

After some digging it was determined that saslauthd (provides ldap -> kerberos lookups) was slogging during these peak times. We resolved that issue and quickly found that the radius servers were now insufficient to handle to newly-unconstrained throughput. We doubled the amount of radius servers serving .1X . We also physically moved them closer to the controllers, direct-connected them to the same router hosting our controllers. This reduced packet hop count and redundant traverasl of established authentication network links. This eliminated the timeouts, even during high mobility events, and reduced radius auths-per-second to a more reasonable number.

 

Having resolved the backend auth delays we are now getting reports of happier clients and can verify faster roaming events and handoffs. We cleared out our t-ticket queue and are now looking at new trouble reports with a fresh perspective.

 

While this certainly won't apply to many (most?) who are experincing issues the take away for us was to look deeper into *all* components that make up what we call "wireless".

 

Mike

 

---

Mike,

 

Excellent work!

 

Do you now have a ratio of users, access points or authentications/second calculation formula to know when to add an additional radius server or are you just monitoring saslauthd?  Others are probably looking for guidance on how to monitor their radius infrastructure....

 

Great insight on the radius issue.  As we are noticing similar issues (as far as the disconnects) I thought I might add a question to the mix.  Should it be the case that every time a device hits a different AP (say between classes when 2000 devices are moving around) it has to re-auth?  I thought there was some kind of token system within the controller that kept you authenticated.

If the client OS supports is, the controller uses Opportunistic Key Caching for quicker roaming. If this is turned on in the 802.1X authentication profile, be sure to turn on "Validate PMKID" to validate the cashed value & discard it for clients that do not support OCC. 

mnaylor,

 

It depends on the device.  If it supports OKC, it should not have to re-auth.  OKC is enabled by default in the 802.1x profile.

 

 

Colin,

 

Is "Validate PMKID" enabled by default now? It wasn't in the past and caused poor user roaming experience.

It is not:  http://www.arubanetworks.com/techdocs/ArubaOS_64_Web_Help/Web_Help_Index.htm#ArubaFrameStyles/1CommandList/aaa_authentication_dot1x.htm

 

 

Colin,

I think that is a bug. Under what condistions would it make sense to enable OCC without "Validate PKMID"? That is a recipe for disaster.

The default values should be sane (make sense logically) and perferably valid for a majority of customers, if porrible.

It does appear to be a bug, as the GUI suggests that it is enabled by default.  

pmauretti,

 

Thank you.  I will get the doc corrected.  When I created a new 802.1x profile, it indeed had Validate PMKID enabled.

Is there any way to know if the device supports or is using OKC?

mnaylor,

 

The manufacturer would know that.  Validate PMKID is a way to check to see if those devices support OKC.

I'm comfortable leaving "Advertise 802.11d and 802.11h Capabilities" disabled, but would like to enable Channel Switch Announcements (CSA).   I was under the impression that CSA was an 802.11h feature.  Does anybody have any experience enabling CSA without the other 802.11h features?  If so, does it seem to have any adverse effects on the clients?

 

Thanks,

 

Chuck Enfield

Penn State University