Wireless Access

last person joined: yesterday 

Access network design for branch, remote, outdoor, and campus locations with HPE Aruba Networking access points and mobility controllers.
Back to discussions
Expand all | Collapse all

Remote AP dropped clients, tunnel IP change

This thread has been viewed 1 times

  • 1.  Remote AP dropped clients, tunnel IP change

    Posted Nov 06, 2013 12:52 PM

    We have been monitoring an Aruba 135 we have set up in the cafeteria of one of our high schools. All of the APs in this building are set up as remote APs as opposed to campus. We wanted to monitor AP and client health during the AP's peak usage time, the students' lunch hour. We have seen upwards of 100 clients on the AP during peak usage. We were unsure of the amount of stress the AP could handle, and we were using the students as a test. While monitoring the AP, I noticed all of the clients dropped for roughly 30 seconds. Most of the clients were able to re-associate fine, but a few seemed to have issues re-associating. We noticed on our controller that the AP also had it's IP tunnel changed during this brief outage.

     

     

     

    Here is info I pulled from the debug log for the AP on the controller. At 12:00 is exactly when the outage occurred, and you can even see the AP's tunnel IP changes from 74.122 to 74.35. I believe the font that is bolded could mean something, but I am unsure how to interpret it.

     

    Nov 6 11:58:44 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.122 stm| ^[msg WPA2 Key Message 2] [mac c8:d1:5e:33:95:c6] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]
    Nov 6 11:58:46 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.122 stm| ^[msg WPA2 Key Message 2] [mac c8:d1:5e:33:95:c6] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]
    Nov 6 11:58:47 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.122 stm| ^[msg WPA2 Key Message 2] [mac c8:d1:5e:33:95:c6] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]
    Nov 6 11:59:44 stm[1413]: <132093> <ERRS> |AP RHS_CAFE_330041@192.168.74.122 stm| ^[msg WPA2 Key message 2] [mac 04:f7:e4:b1:8f:55] [bssid 24:de:c6:f1:f5:50] [apname RHS_CAFE_330041] [stcnt1 0] [stcnt2 1] [apcnt1 0] [apcnt2 2]
    Nov 6 11:59:59 sapd[1392]: sapd_redun_config_dnsmasq, rewrite dnsmasq config file
    Nov 6 12:00:25 sapd[1392]: PAPI_Send: sendto RAPPER_PORT2 failed: No such file or directory Message Code 9 Sequence Num is 27853
    Nov 6 12:00:27 sapd[1392]: PAPI_Send: To: 7f000001:8424 Type:0x3 Timed out.
    Nov 6 12:00:30 KERNEL(RHS_CAFE_330041@172.16.12.220): asap_station_add: WARNING: !
    Nov 6 12:00:41 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.35 stm| ^[msg WPA2 Key Message 2] [mac b0:9f:ba:35:97:b0] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]
    Nov 6 12:00:42 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.35 stm| ^[msg WPA2 Key Message 2] [mac b0:9f:ba:35:97:b0] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]
    Nov 6 12:00:44 stm[1413]: <132094> <WARN> |AP RHS_CAFE_330041@192.168.74.35 stm| ^[msg WPA2 Key Message 2] [mac b0:9f:ba:35:97:b0] [bssid 24:de:c6:f1:f5:40] [apname RHS_CAFE_330041]

     

     

     

     

    I ran a show ap remote debug association-failure ap-name on the AP right after the outage, and it returned a few clients that did not seem to reassociate properly:

     

    b0:9f:ba:35:97:b0 RHS_CAFE_330041 24:de:c6:f1:f5:50 OpenVVSDWiFi auth 802.11a 1m:54s Did not attempt to associate
    b0:9f:ba:35:97:b0 RHS_CAFE_330041 24:de:c6:f1:f5:40 OpenVVSDWiFi 802.11g 1m:54s Sapcp Ageout (internal ageout)
    84:38:35:eb:92:a7 RHS_CAFE_330041 24:de:c6:f1:f5:50 OpenVVSDWiFi auth 802.11a 2m:14s Did not attempt to associate
    c0:63:94:4e:b5:6e RHS_CAFE_330041 24:de:c6:f1:f5:50 OpenVVSDWiFi auth 802.11a 2m:14s Did not attempt to associate
    c0:63:94:4e:b5:6e RHS_CAFE_330041 24:de:c6:f1:f5:40 OpenVVSDWiFi auth 802.11g 2m:14s Did not attempt to associate
    8c:58:77:1c:39:c6 RHS_CAFE_330041 24:de:c6:f1:f5:40 OpenVVSDWiFi auth 802.11g 2m:14s Did not attempt to associate
    Num Association Failures:6

     

     

    I finally did a show ap debug system-status ap-name on the AP and saw that the CPU and Memory peaked right at the same time:

     

    Peak CPU Util in the last one hour
    ----------------------------------
    Timestamp CPU Util(%) Memory Util(%)
    --------- ----------- --------------
    2013-11-06 12:00:31 48 28

     

     

     

    My original thought was maybe the AP had too many clients on it, as it had I believe 102 cilents right before the outage. If anyone has any insight or suggestions, please share. Thanks.


    #AP135


  • 2.  RE: Remote AP dropped clients, tunnel IP change

    Posted Nov 06, 2013 01:16 PM
    How many APs are terminating IPSEC tunnels to this controller? This creates unnecessary overhead. Why no local controller?

    Also, with RAPs all terminating to your controller, ARM is not working as designed, the RAPs see each other as neighbours/interferers unless you specifically marked them all as valid on the controller.

    Have you confirmed that you did not suffer an internet outage or link issue? Have you check the switch logs as well?

    What version of controller are you using?



  • 3.  RE: Remote AP dropped clients, tunnel IP change

    Posted Nov 06, 2013 01:45 PM

     

    Re: Remote AP dropped clients, tunnel IP change
     

    13m ago

    How many APs are terminating IPSEC tunnels to this controller? Close to 100 APs, all of which are 135s
     
     
    This creates unnecessary overhead. Why no local controller? We have over 20 buidlings located on our campus. All of our buildings are connected to our administration center through our own WAN. Our administration center houses the controller which is a W-7240. We were told that in our scenario, the single controller would be sufficient.

    Also, with RAPs all terminating to your controller, ARM is not working as designed, the RAPs see each other as neighbours/interferers unless you specifically marked them all as valid on the controller. We were having issues in a few of our buildings where clients would randomly disconnect throughout the day. After countless hours of troubleshooting with Aruba phone support, we were told to try provisioning the APs as remote AP as opposed to campus. We were told that campus APs actually create two tunnels back to the controller (CPsec and IPsec) where as remote AP only creates the one IPsec. We then converted the APs to Remote AP in the buildings where we were having issues and have had great results.
     
    We were not told that converting the APs to RAP would not have any issues with ARM, and from what I see, ARM is working properly.

    Have you confirmed that you did not suffer an internet outage or link issue? We have a network monitoring system and did not see any issues with outages. No pings were lost to any of the switches or the building's router.
     
     Have you check the switch logs as well? We have not looked at switch logs yet, but we do have a few other APs connected to that switch that did not experience the same outage.

    What version of controller are you using? Our controller is a W-7240 and is running OS 6.2.1.3.
     
    Thank you for taking the time to respond and look forward to hearing back.