Controllerless Networks

Hi,

 

We have recently upgraded out network and Added x6 new AP 225 Access points, having upgraded the firmware to version 6.3.1.2-4.0.0.2_41506 i am still seeing alot of DHCP Timeouts?

 

Can anyone advise? The new versions of firmware seem to be causing this, The network seems alot more stable with previous versions however i was told to upgrade for compatability for the new access points.

 

Any advice on how to resolve this would be much appreciated.

 

Thanks

Nisha

#AP225

>From the time out alerts, does it indicate which APs generating them? Does it show if the alerts are primarily coming from AP225 or is it from any type of AP?

The alerts are from any 225's, we have 6 so it could be 3 from AP03 and 4 from AP05, it varies?

 

Just wanted to chime in I have a test deployment of 6 Ap225 and am seeing a lot of DHCP errors as well. They come from all clients and all Ap's. I have not seen any problems with clients getting address's. Its hard to tell exatly how many im getting as instant auto clears them every few minutes but if I sit and watch they role through every 4-6min. This is with only 3 or 4 clients as well so Im not exactly hitting them hard. I am having a lot of problems with reliablity as well, I was assuming it was because of the brand new AP's having some bugs but it may have more to do with the new firmware.

 

Hi,

i do believe its to do with the firmware as we have IAP 135's in some of our colleges and i also see alot of DHCP timeouts on the latest version, if i downgrade to a previous version it seems to be ok you see a few alerts but not 30-40 which is what i am seeing now with these new IAP-225's.

I have experienced this with versions of 3.x code.  I have so far had good luck with 4.x code but am wondering if this is a recurring issue.

 

I have raised a TAC Case for this now, Aruba have removed the above firmware version from their site as most likely they have bugs and have recommened i downgrade to version 4.0.0.1 to see if this resolves this issue.

I will reply back once i have done this.

 

Also Aruba say for large number of clients we should not use Virtual Controller assigned DHCP we should have a dedicated server?

 

Thanks

Nisha

We are not using the VC for DHCP and are getting those DHCP errors. I also have under 10 clients...

 

Alex

I am still seeing alot of DHCP Timeout even after downgrading the firmware as per the attachement.

 

Any suggestions?

 

I think we have the same or a very similar problem. We have a new Instant installation with 9 IAP-225 access points. The APs arrived with the 4.0.0.2 firmware. The first day was terrible with lot of connection issues. Then an attempted downgrade bricked the whole system, so it was restored to 4.0.0.2. After that trying various options like disabling client match, 802.11d/h, etc. we got to state where it is not perfect, but can be used with Macbooks. Today we upgraded to 6.3.1.2-4.0.0.3_41660, but it seems the same. Linux and Windows machines has serious issues. Sometimes they cannot reconnect only after using the hardware wifi switch on the laptop or rebooting the AP. We can observe higher than usual latency on the LAN (to the default gateway and the virtual controller) and sometimes high packet loss (over 10%). In the alert sections we can find the same DHCP timeout and Integrity check failure messages. We also have a TAC case opened by the distributor. If we won't find the solution very soon we will try to convert the system to a controller based one. 

We had the debugging session with Aruba TAC today. After changing back to the Instant setup and reenabling HT modes we save high latency and packet loss. Disabling "Client match" somewhat stabilized the situation, but I can still measure somewhere between 1-5% packet loss. There are times when it perfect, but if you measure it long enough you will get packet loss.

 

I think disabling client match is just a workaround and it relieves the AP of some tasks so it can perform better. (Though the CPU load on the AP was never high, so it's just my theory). The reason I think there is a performance issue with packet processing is that we could see >1000ms latency while client match was enabled.

 

On my MacBookPro I could see the following entries in the system.log while I was on the Instant setup (we use WPA2 PSK):

Feb 11 14:58:28 Marks-MacBook-Pro kernel[0]: MacAuthEvent en1 Auth result for: 18:64:72:e3:ab:d1 Auth timed out
Feb 11 14:58:30 Marks-MacBook-Pro kernel[0]: MacAuthEvent en1 Auth result for: 18:64:72:e3:98:11 Auth timed out
Feb 11 14:58:31 Marks-MacBook-Pro kernel[0]: MacAuthEvent en1 Auth result for: 18:64:72:e3:98:01 Auth timed out
Feb 11 14:58:31 Marks-MacBook-Pro kernel[0]: MacAuthEvent en1 Auth result for: 18:64:72:e3:9f:81 Auth timed out
Feb 11 14:58:37 Marks-MacBook-Pro kernel[0]: MacAuthEvent en1 Auth result for: 18:64:72:e3:98:11 Auth timed out

...

 

I never had these on the controller based setup.

 

So if you have problems with Instant and MacBooks I recommend you to:

1. disable client match

2. open an ticket with Aruba TAC and let them know that others has this problem as well.

 

I have had a similar experience as mszabo, in order to get things working well enough to deply we had to disable client match and 80mhz channels still getting tons of DHCP errors in the logs.

For the Macs I had the problems you are describing when I had 802.11r enabled, try turning that off?

 

Alex

Wondering if anyone has found a solution to this. It seems to be getting worse as I go on. I spent the whole day today trying to figure this out,, I have a case open with TAC but am hoping im not the only one. Today I found a AP in an area that uses were complaing a lot about. I mirrored the port that AP is on so I could run a capture and then attempted to connect with my laptop. The laptop connected just fine but got a 169 address. No traffic from my laptop made it past the AP on its wierd port. For some reason the AP is dropping all traffic from some clients not allowing them to get an address from the DHCP server. If I turn the adapter on and off a few times it will eventually connect and I see the appropriate DHCP traffic on the port im monitoring.

 

 

Alex

I am having the same issue. I have IAP225s 17 of them. I have opened a tac case too. With other symtoms.

We are an all Mac OSx environment with about 160 cleints. They all have issues with the Wifi and can't stay connected more than a few minutes up to  two hours. The issue comes and goes with various clients and I have difficulty nailing it down to any specific situation.
To Troubleshoot, I ask cleints to start ICMP ping sessions that just fail. I then have them open their Wifi radio disgnostics to see the MCS and transmit rates. MCS is usually 0 or 1-2 and transmit rates are below 15. Issue continues for hours till client turns there Wifi radio off and Turns it On.

Still no REAL resolution, but TAC keeps having me try hair-ball ideas that never bring any results. However, I am in a extremly saturated environment with lots of rouge APs, Clients, and SSIDs.

I think we are going to have to try going to a Real controller vs. the Virtual.  But to be honest I don't see that as a resolution, just an Upsell.

 

Edit: I attached my Config and did some editing to my reply.

 

We just went live last week with 43 Iap225's. I have tons and tons of DHCP timeout erros now. I have not seen the same problems as you with the Mac's can you post your config up and we can compare?

 

Alex

We also had/have issues like this with a full IAP-225 deployment with 9 access points. Disabling client match, 802.11r, 802.11d/h, and after that HT modes (so now it's running in a/bg mode) somewhat helped. I feel like it is trial and error and we are being lab rats. This is a new deployment, so a TAC case was eventually opened by the local reseller. We got a controllered based setup until they figure it out with the Instant.  Tomorrow we will have an Aruba engineer available to continue troubleshooting. They said the issue might be somewhere around MPDU aggregation.  I will post it here if there is any positive outcome.

Did anyone ever get a resolution on this issue?  I'm seeing something very similar.  22 iAPs in a controller-less configuration, 250 OS X clients + mobile.  Firmware 6.4.2.0-4.1.1.1.  Occasionally laptops connect to an iAP, but the iAP doesn't pass traffic.  Other devices connected to the iAP continue to operate just fine, and toggling WiFi off and on resolves the issue temporarily (sometimes the client has problems again, sometimes not).  I've got a TAC case open but so far they've been unable to definitively resolve.

 

thanks -

Was this ever resolved? I have multiple IAP225's (6.4.2.6-4.1.1.10_51810) deployed. 10-20 clients most of the time. DHCP, DNS, and ICMP timeouts all over the place. Have tried all the adjustments in this thread. Still see between 10% and 20% packet loss as reported by long-running ping sessions. 

 

Curiously I only see the packet loss on traffic initiated by the client. When checking against the clients from an upstream source there is no packet loss reported. 

 

It has been so problematic that I am currently using an off the shelf Asus RT-AC66 to support all the clients because it is faster, and works without problems. 0% packet loss when connected to Asus. It's kind of embarrassing actually.