Wireless Access

Hi all,

I need some help with Aruba 7220 controller and APs which connected to this controller. We have got an Aruba 7220 controller. We are using AP-105, AP-93 and IAP-225(With controller mode) in production environment. The majority of Access Points are AP-105. As you can see below screenshots we have a throughput problem. The throughput cuts out occasionally.  When the throughput dropped suddenly, the clients continue to the controller but the clients can't access to internet. 

Any help or guidance would be greatly appreciated.

Here are some relevant details for your info:

#show version

ROM: System Bootstrap, Version CPBoot 1.2.3.0 (build 43722)

Built: 2014-05-13 10:03:12

Built by: p4build@re_client_43722

Switch uptime is 14 days 10 hours 30 minutes 50 seconds

Reboot Cause: User reboot (Intent:cause:register 78:86:50:2)

Supervisor Card

Processor (XLP432 Rev B1 (Secure Boot) , 1000 MHz) with 7370M bytes of memory.

32K bytes of non-volatile configuration memory.

7744M bytes of Supervisor Card system flash.

#show ap essid

ESSID Summary

-------------

ESSID     APs  Clients  VLAN(s)                                         Encryption

-----     ---  -------  -------                                                                                                             ----------

SDU_WiFi  166  189      xxxx                                             Open

eduroam   166  36         xxxx                                             WPA 8021X AES,WPA2 8021X AES

Num ESSID:2

Thanks!

Thank you for the reply.

I uploaded a piece of system log file. Could you tell me if there is something wrong in log file. 

On second thought, do you have a screenshot from the "Performance" Tab?

Yes, i have ps from performance tab. But now, number of clients in 200s. 

We probably needed that performance creenshot when you had issues.  The system log does not look like anything bad is going on from a system perspective.  I did see that you have access points that have over 40 users on them at a time.  Please PM me your email address and I will send you a link to send me your logs.tar with tech support so I can take a look.

Thank you for your answer cjoseph. I sent my adress. As you said, i should take performance screenshoot when problem occured. Is there anything else you want? Any log or printscreen?

Let me take a look at your logs.tar first, and we can take it from there.

Colin, 

We had just same problem again. I took print screen both "Performance" tab and "Usage" tabs. I sent you  "log.tar" file via "Secure File Downloads"

Also i sent the printscreens here.

Thank you.

Hakan

Hakan,

I got your logs and emailed you a few suggestions for a few crucial changes you should make.  Please let me know if you have any questions.

Colin,

I received your mail. I will check your valuable suggestions.
Thank you again for your feedback.

Hakan Orcan

We solved the Throughput problem with Colin's valuable helps and analysis. 

Firstly, we reduced the RF utilization. We have too high AP power on 2.4ghz band. We set it using the following configurations: 

Configuration> Wireless> AP Configuration> click on “Kablosuz-1”. Expand RF Management, Expand 802.11g Radio.  Click on Adaptive Radio Management.  Click on Advanced.  Change “Allowed Bands for 40mhz Channels” to “none”.    Change the Max-TX-EIRP to 18, and change the Min-TX-Eirp to 12.  Uncheck 80 mhz support.    Click on Apply. 

We haven't Broadcast Filtering on our Virtual Aps.  

You can fix this by going to configuration: 

Wireless> AP Configuration> click on “Kablosuz-1”.  Expand Wireless LAN.  Expand Virtual AP.  Click on Kablosuz-1-vAPp.  In the right pane, make sure that Drop Broadcast and Unknown Multicast has a checkbox next to it.  Click on Apply.  Click on SDU-eduroam-vAPp.  In the right pane, make sure that Drop Broadcast and Unknown Multicast has a checkbox next to it.  Click on Apply.   Go to configuration> Wireless> AP Configuration> click on “SDU-Aps”.  Expand Wireless LAN.  Expand Virtual AP.  Click on SDU-WiFi-vAPp.  In the right pane, make sure that Drop Broadcast and Unknown Multicast has a checkbox next to it.  Click on Apply. 

Also we have too much logging enabled on the controller. The controller generates a lot of logs. 

We use these commands to reduce log stress on our contoller.  

config t 
no logging level debugging arm 
no logging level debugging arm subcat all 
no logging level debugging arm subcat client-match 
no logging level debugging arm subcat radio-mgmt 
no logging level debugging network 
no logging level debugging network subcat all 
no logging level debugging network subcat dhcp 
no logging level debugging network subcat mobility 
no logging level debugging network subcat packet-dump 
no logging level debugging security 
no logging level debugging security subcat aaa 
no logging level debugging security subcat all 
no logging level debugging security subcat certmgr 
no logging level debugging security subcat dot1x 
no logging level debugging security subcat firewall 
no logging level debugging security subcat ids 
no logging level debugging security subcat ids-ap 
no logging level debugging security subcat ike 
no logging level debugging security subcat kerberos 
no logging level debugging security subcat mobility 
no logging level debugging security subcat ntlm 
no logging level debugging security subcat packet-trace 
no logging level debugging security subcat vpn 
no logging level debugging security subcat webserver 
no logging level debugging security subcat wl-sync 
no logging level debugging system 
no logging level debugging system subcat all 
no logging level debugging system subcat amon 
no logging level debugging system subcat amon-ale 
no logging level debugging system subcat amon-amp 
no logging level debugging system subcat ap 
no logging level debugging system subcat configuration 
no logging level debugging system subcat ha 
no logging level debugging system subcat mapc 
no logging level debugging system subcat messages 
no logging level debugging system subcat pan 
no logging level debugging system subcat reg-tbl 
no logging level debugging system subcat snmp 
no logging level debugging system subcat webserver 
no logging level debugging user 
no logging level debugging user subcat all 
no logging level debugging user subcat captive-portal 
no logging level debugging user subcat client-match 
no logging level debugging user subcat dot1x 
no logging level debugging user subcat mapc 
no logging level debugging user subcat pan 
no logging level debugging user subcat radius 
no logging level debugging user subcat voice 
no logging level debugging user subcat vpn 
no logging level debugging wireless 
no logging level debugging wireless subcat all 
logging level warning wireless 
logging level warning system 
logging level warning network 
logging level warning user 
logging level warning arm 
exit 
 
 

After entering commands, the log situations were as follows: 

(SduAruba7220) #show logging level verbose 

 LOGGING LEVELS 

-------------- 

Facility  Level     Sub Category  Process 

--------  -----     ------------  ------- 

arm       warnings  N/A           N/A 

network   warnings  N/A           N/A 

security  warnings  N/A           N/A 

system    warnings  N/A           N/A 

user      warnings  N/A           N/A 

wireless  warnings  N/A           N/A 

Another problem is access points have a lot of bootstraps.  We saw them using this command: 

#show ap debug counters  

According to Colin's theory, the cause of bootstraps is we use almost 200 APs and we have just one 1 gig port on controller.  According to Colin's, a single gigabit Ethernet port for every 100 access points. That is true. This is cause our users and throughput to drop randomly. We need to add secong 1 GE links to controller. And we need to have a port channel.  

Port Status 

----------- 

Slot-Port  PortType  AdminState  OperState  PoE  Trusted  SpanningTree  PortMode  Speed   Duplex 

---------  --------  ----------  ---------  ---  -------  ------------  --------  -----   ------ 

0/0/0      GE        Disabled    Down       N/A  Yes      Disabled      Access    Auto    Auto 

0/0/1      GE        Disabled    Down       N/A  Yes      Disabled      Trunk     Auto    Auto 

0/0/2      GE        Enabled     Up         N/A  Yes      Disabled      Trunk     1 Gbps  Full 

0/0/3      GE        Enabled     Down       N/A  Yes      Disabled      Access    Auto    Auto 

0/0/4      GE        Enabled     Down       N/A  Yes      Disabled      Access    Auto    Auto 

0/0/5      GE        Enabled     Down       N/A  Yes      Disabled      Access    Auto    Auto 

There is an article about to set it up. 

http://community.arubanetworks.com/t5/ArubaOS-and-Controllers/EtherChannel-between-Aruba-and-Cisco-switch/m-p/438/highlight/true#M45  

The system works very vell. Thank you Colin Joseph. 

Best regards, 

Hakan Orcan 
Suleyman Demirel University 
Information Technologies Department 
Network and Systems Management Group 

Hi!

Thanks for letting us know what solved your problem. I do have a question regarding the "1Gbit port per 100 APs" recommendation.

cjoseph, could you please elaborate this? I havn´t seen or heard of this recommendation in my years working with Aruba and never experienced any issues because of it. Is the recommendation based on that if the APs combined push traffic congesting that link so the AP-> Controller communication gets interupted, causing rebootstraps?

Cheers,

Hi

What about spanning-tree on the upstream switches?

Looks like the system stops transferring data, might look like this if there is something wrong with the topology and spanning-tree re-calculates for whatever reason.

During the recalculation and learning, the upstream device will stop switching.

Roar

Mosher,

Thank you for answer. As you said, we have probably a STP problem. You said true. But, before doing STP configuration contol on upstream switched i wanted to find out whether problem on Aruba Controller and APs. 

Certainly, you are right.