Wireless Access

Any help with interpreting the following output would be appreciated, how disastrous/distruptive are these actions, and what is the relationship between what looks like a pair of entries, on some RAPs they are close together and others are up to 40mins apart -

Rebootstrap Information
-----------------------
Date       Time     Reason (Latest 10)
--------------------------------------
2013-01-26 05:31:25 Missed heartbeats: Last Sequence Generated=28371 Last Rcvd=28340
2013-01-26 05:31:28 Changing to LMS #0 (XXX.XXX.XXX.XXX)
2013-02-04 11:45:52 Missed heartbeats: Last Sequence Generated=41456 Last Rcvd=41425
2013-02-04 11:45:55 Changing to LMS #0 (XXX.XXX.XXX.XXX)
2013-02-05 08:10:08 Missed heartbeats: Last Sequence Generated=49341 Last Rcvd=49310
2013-02-05 08:10:22 Changing to LMS #0 (XXX.XXX.XXX.XXX)
2013-02-05 08:14:05 Missed heartbeats: Last Sequence Generated=49577 Last Rcvd=49547
2013-02-05 08:14:13 Changing to LMS #0 (XXX.XXX.XXX.XXX)
2013-02-05 10:18:03 Missed heartbeats: Last Sequence Generated=56980 Last Rcvd=56950
2013-02-05 10:18:45 Changing to LMS #0 (XXX.XXX.XXX.XXX)

I would also like to understand exactly what happens during a "Re-bootstrap" compared with powering down the AP and powering it up again.

Thanks in advance.

M

Please make sure that the access point negotiated the proper duplex/speed with the switch it is plugged into.  If it is a RAP, it could be the quality of the internet connection.  If it happens to ALL raps, it might be the headend that is creating a delay or drops.

Also check the quality of the cable run.

A rebootstrap happens when an access point misses a certain number of heartbeats.  It resets its connection to the controller and tries to reach it and obtain its configuration again.  A reboot requires a new ip address and is pretty much the same as a full reset.

The AP sends a heartbeat - which is actually a WiFi frame over the GRE tunnel to the controller (which as this is a RAP is inside the IPSec tunnel - so tunnel in a tunnel).

When the controller gets this frame it checks to see if the tunnel is valid - if not it send back an ICMP unreachable message. The AP will then immediately rebootstrap or connect to the defined backup LMS.

If the tunnel is valid it replies straight back to the AP.

So how do you get the messages below;

Depending on the AP type the default packet loss is significantly different, for a RAP it is 30 consecutive heartbeat losses, and for campus AP this is 8 heartbeat losses (30 seconds and 8 seconds respectively).

This can be over ridden by;

(Aruba650) (config) #ap system-profile test

(Aruba650) (AP system profile "test") #boot?
bootstrap-threshold     The number of missed heartbeats that will cause the
                        AP to rebootstrap. Range: 1-65535. Default: 8.

2013-01-26 05:31:25 Missed heartbeats: Last Sequence Generated=28371 Last Rcvd=28340

we can see here you have lost 30 heartbeats.

In 'newer' releases we do make it slightly easier to debug what's going on.

On the controller you can issue the following command;

(Aruba) #show datapath tunnel heartbeat

Datapath Tunnel Table Entries
-----------------------------

Flags: E - Ether encap,  I - Wi-Fi encap,  R - Wired tunnel,  F - IP fragment OK
       W - WEP,  K - TKIP,  A - AESCCM,  G - AESGCM,  M - no mcast src filtering
       S - Single encrypt,  U - Untagged,  X - Tunneled node,  1(cert-id) - 802.1X Term-PEAP
       2(cert-id) - 802.1X Term-TLS,  T - Trusted,  L - No looping, d - Drop Bcast/Mcast,
       D - Decrypt tunnel,  a - Reduce ARP packets in the air, e - EAPOL only
       C - Prohibit new calls, P - Permanent, m - Convert multicast
       n - Convert RAs to unicast(VLAN Pooling enabled), s - Split tunnel
       V - enforce user vlan(open clients only)

 #          Source       Destination    Prt  MTU         Acls                BSSID          Decaps     Encaps   Heartbeats  Seqnum  Missed  Outoforder Cpu QSz Flags
------  --------------  --------------  ---  ----  -------------------  ----------------- ---------- ---------- ----------  ------  ------  ---------- --- --- -----
41      192.168.254.9   192.168.254.247 47   1500  0    0    0    0     D8:C7:C8:C0:CC:48     12785          0      12783    12898      107         0    6   0 TES
21      192.168.254.9   192.168.254.252 47   1500  0    0    0    0     D8:C7:C8:C0:CC:50    328758          0     328756     3848     2764         0    5   0 TES
56      192.168.254.9   192.168.254.249 47   1500  0    0    0    0     D8:C7:C8:C0:CC:42     12775          0      12773    12889      107         0    7   0 TES
13      192.168.254.9   192.168.254.251 47   1500  0    0    0    0     D8:C7:C8:C0:CC:2E   1009988          0    1009986    35441     8487         0    5   0 TES
47      192.168.254.9   192.168.254.236 47   1500  0    0    0    0     D8:C7:C8:C0:CC:36   1009263          0    1009261    34711     8482         0    6   0 TES
23      192.168.254.9   192.168.254.41  47   1500  0    0    0    0     00:24:6C:CE:99:32     28280          0      27816    27824        0         0    7   0 TES

(Aruba650) #

To debug the AP;

(Aruba650) #show ap debug gre-tun-stats ap-name office

GRE HBT Tunnel Stats
--------------------
AP IP            Controller IP  Sent Count  HBT Tx Seqnum  Idle (secs)  Rcvd Count  HBT Rx Seqnum  Idle (secs)
-----            -------------  ----------  -------------  -----------  ----------  -------------  -----------
192.168.254.252  192.168.254.9  331558      3884           0            328794      3884           0
GRE Tunnel Packet Stats
-----------------------
MAC                BSSID     Tun Input  In IP Frags  To WLAN  Idle (secs)  Rate pps  From WLAN  Tun Output  Out IP Frags  Idle (secs)  Rate pps
---                -----     ---------  -----------  -------  -----------  --------  ---------  ----------  ------------  -----------  --------
00:26:B0:C0:AF:4B  8C:C5:00  5208       0            5208     60           0         4796       4796        0             60           0
00:27:10:2C:2A:A4  8C:C5:00  539461     2            539461   0            6         409290     409290      2             0            6
01:80:C2:00:00:00  8C:C5:00  157497     0            157879   0            0         0          0           0             0            0
FF:FF:FF:FF:FF:FF  8C:C5:00  174278     0            174278   0            0         0          0           0             0            0
01:00:5E:7F:FF:FA  8C:C5:00  256402     0            256402   0            3         0          0           0             0            0
01:00:5E:00:00:FB  8C:C5:00  5636       0            5636     120          0         0          0           0             0            0

(Aruba650) #

To check speed /duplex of the ethernet port of the AP you can issue the following command;

(ArubaTelford) #show ap debug port status ap-name gscott-rap5

Tue Feb 12 10:12:11 2013

AP "gscott-rap5" Port Status
----------------------------
Port  MAC                Type  Forward Mode  Admin     Oper  Speed   Duplex  802.3az  PoE  TX-Packets  TX-Bytes    RX-Packets  RX-Bytes
----  ---                ----  ------------  -----     ----  -----   ------  -------  ---  ----------  --------    ----------  --------
0     00:0b:86:66:0d:a3  GE    N/A           enabled   up    1 Gb/s  full    N/A      N/A  12453993    6146002748  23991004    9685567011
1     00:0b:86:66:0d:a4  FE    tunnel        enabled   down  N/A     N/A     N/A      N/A  0           0           0           0
2     00:0b:86:66:0d:a5  FE    N/A           disabled  down  N/A     N/A     N/A      N/A  0           0           0           0
3     00:0b:86:66:0d:a6  FE    N/A           disabled  down  N/A     N/A     N/A      N/A  0           0           0           0
4     00:0b:86:66:0d:a7  FE    tunnel        enabled   down  N/A     N/A     N/A      N/A  0           0           0           0

(ArubaTelford) #

a full reboot, means the AP will do a full power cycle so it will re-run the POST and then have to reload the AOS image from flash, and then start its connection to the controller (this POST + image load take at least 30 seconds)

Where as a rebootstrap, will just relearn the controller address and resetup an IPSec connection etc.

Thanks Giles,

    The devices are RAPs with the AP system profile bootsrrap threshold set to 8. I take it this means that the bootstrap threshold is overidden if they are RAPs and fixed to 30?

Matt

Yes thats correct the system detects the AP is a RAP and automaticaly changes the rebootstrap threshold from 8 to 30.