Controllerless Networks

Dear community,

I hope I can post problem here, although devices are Alcatel-Lucent branded (FW 6.4.2.0-4.1.1.1_46936 on each device).

I have in control Instant AP on client premises which consists of three IAP-105 and one IAP-205. They cover medium-high density environment (up to 180 wifi devices on 400 sq mt), as public wifi hotpos with autorization (ticketing system controlled on router).

They are configured in a way that they serve network as access layer, and forward all request to router (on same VLAN) which servers DHCP, DNS and authorizes clients which try to access public internet.

Same configuration whole last year worked fine, but this year we have occasional huge problems with:
- clients being disconnected or unable to connect to wireless (no wifi key used)
- the ones which stay connected are unable to reach central router nor public internet
- or the ones which are able have extremly slow access to internet (pipe of 100/100Mbit is used by avg 150kb in those moments).
- sometimes in that period VC on master is not reachable, and ping latency to AP's is about 1500ms from router
- if AP's in those moments are reachable, I ssh to them, and I can without problems ping from them client IP which reports me problems (slow traffic, router not reachable etc)

Those moments don' have any rule when or why they happen (at least which I could discover). AP's are powered by standalone PoE injector, and they are directly connected to router (in period of problems interface is always up but with low traffic per interface, cca 150kb throughput.

Can someone please assist me in troubleshooting and point me which data/configuration to gather and post here?

Thank you in advance,
regards

Start by upgrading the firmware to the latest GA, there´s tons of fixes from the version you´re currently at that have fixed major issues like the ones you´re describing.

Cheers,

Thanks Christoffer!

I've updated all AP's to 6.4.4.3-4.2.2 (Taurus and Orion, latest firmware by today, but delivered by Alcatel, not Aruba) and I can see from first day of production significant performance improvement of AP's. Much less CPU and memory consumption on all AP's for same amount of traffic and number of clients active (around 130, consuming alltogether about 90-100Mbits in download).

No issues with clients but I will be careful with conclusions and I will wait at least for 7 days to accept this action as solution.

Two AP's are not strictly used in indoor environment (slight exposure to outdoor environment, wind and salt in air) so I'm bit suspicious in their reliability.

Is there anything in logs of AP's that can warn on such problems? Some kind of diagnostics?

Thanks!

Cheers

Unfortunately I didn't need to test too long, first day same major issues are still present. The only difference in comparison to former problems is that VC is accessible, and each of the AP is visible in network (ping latency <10ms). I'm able to access AP's via SSH and https.

So major problem is that without any rules clients (except bit higher load of clients on cluster, about 70-80 of them) connected to cluster cannot communicate with main router (which servers DNS). Some of them cannot connect to AP's, and the ones which connect to AP get served with DHCP (ip,gw,mask,dns), but unable to resolve any local or public ip/host.

In those moments, when I cannot to AP, I can ping those clients, and they are visible on VC as part of clients list.

When number of clients is reduced to cca 35-40 problem dissapears, althoug in very rare moments we have encountered same problem with such small number of clients connected.

I have "show tech" and "show tech supplemental" dump, but I would like to provide it only via PM to anyone interested in helping in this matter.

Any kind of help is appreciated because we are without any kind of official support in our country, thanks!

Does clients get IP from the same VLAN as the AP? Does that VLAN also contain wired clients?

If you have wired clients on the VLAN, do they also experience performance issues when the issue happens or is it wireless clients only?

Do you use both 2,4ghz and 5ghz for the network? Does both experience issues?

What channel widths are you using?

Cheers,

1. Clients get same IP from VLAN as the AP, but the IP addresses of AP's are "pass-throughed" through the hotspot captive portal so they can communicate between each other with no proxy-arp filtering or anything else.

2. I don' have wired clients in same VLAN, only AP's.

3. I use both 2.4 and 5 GHz, with 5Ghz as preffered in ARM for access. When I payed attention to currently connected wifi clients which have problems, some of them were connected with GN protocol, but haven't examined every of them...

4. I don't use 80Mhz, only narrowed 20Mhz (because of previous bad experience with iPhone devices on wide channels but the wifi equipment was from different manufacturer)

Do you want me to send you tech dump via PM?

Thank you once again for your appreciated effort!

Do you have "broadcast filter ARP" on all of your SSIDS?

---

@cjoseph wrote:

Do you have "broadcast filter ARP" on all of your SSIDS?

---

Hi Colin, thanks for your reply!

No - it's turned off (disabled).

Try to turn on broadcast filtering (filter all). Also, whenever possible separate the networks (VLANs) where the virtual controllers live from the wireless client VLANs.

See more best practices for Aruba Instant in the Aruba Instant VRD: http://community.arubanetworks.com/t5/Validated-Reference-Design/Aruba-Instant-Validated-Reference-Design/ta-p/258782

---

@Herman Robers wrote:

Try to turn on broadcast filtering (filter all). Also, whenever possible separate the networks (VLANs) where the virtual controllers live from the wireless client VLANs.

---

Thanks, I've already tried few months ago Broadcast filtering (filter all) but same results.

I didn't tried separation of VLAN's since last year with this same configuration and same equipment and much higher load we didn't have single problem like this one.

I will try that to, but I have to examine first how to make configuration. Thanks!

This hotspot captive portal is enforced on the router?

Do you do anything specific to "solve" the issue or it goes away by itself?

One idea, if you still want to provide connectivity on 2,4ghz, is to create a new SSID with "-5ghz" on the end or something that only allows access on 5ghz spectrum to see if the issue might be RF related on 2,4ghz.

Do you have Airwave?

I´m afraid I won´t have time to dig through a tech-dump right now, but I´ll answer in this thread when I find the time.

Cheers,

---

@Christoffer wrote:

This hotspot captive portal is enforced on the router?

 

Do you do anything specific to "solve" the issue or it goes away by itself?

 

One idea, if you still want to provide connectivity on 2,4ghz, is to create a new SSID with "-5ghz" on the end or something that only allows access on 5ghz spectrum to see if the issue might be RF related on 2,4ghz.

 

Do you have Airwave?

---

1. Captive portal is enforced on router (RouterOS)

2. Nothing specificaly solves the problem, just the users which have problems (all at once) start logging out (disconnecting from wireless) while others work normally. When total number of clients is reduced, new clients are able to connect, but haven't tried with the former clients whch had problems (guest network, people just walk away when they experience problems).

3. It's but tricky to advertise two SSID's, but I'll check with owner of system is he willing to the testing that way. Maybe there is womething related to 5GHz, I'll try in future situations to pay more attention to it...

4. There is no Airwave, we just use it as "Virtual controller", one of the AP is master.

Thanks for you effort, cheers