D flag troubleshooting

Question: Why is my AP stuck with "D" flag after upgrade?

 

Environment Information: AP deployed as Campus AP at a remote location (example :: connected via site-to-site VPN tunnel)

 

Symptoms


"D" flag denote Dirty configuration. It would mean AP has incomplete / inconsistent configuration.


Confirm following is true ::

AP was UP in older code but comes with "D" flag after upgrade.

Other APs on same AP-Group would come up fine (i.e. no config issues).

Same AP would come fine on a different location (i.e. no AP hardware issue).


Above checklist would isolate the issue to the network path which the AP takes while on the site which has issues.


Use "show ap details advanced ap-name <name of AP>" to further narrow down the issue to narrow down the issue to direction / specific message.

The command output covers AP to Controller Control Message and Controller to AP Controller Messages. This would help in identify which direction we have and the message which isn't being delivered.

In below example; AP to Controller communication seem to be fine. Total, New & Ack have same value i.e. message was transmitted on 1st attempt and was acknowledged by Controller.


AP "TAC-Test-AP1" AP to Switch Message Counts

---------------------------------------------
Message Total New Acknowledged
------- ----- --- ------------
HELLO 1 1 1
AP_READY 1 1 1
KEEPALIVE 6 6 6
CHAN_PWR_CHANGE 0 0 0
PROV_RESULT 0 0 0
FLASHING 0 0 0
REBOOTING 0 0 0
BW_REPORT 0 0 0
OVRTEMP 0 0 0
STATUS_REPORT 0 0 0
LED_RESPONSE 0 0 0
PORT_CLEAR_RESPONSE 0 0 0
CSR 0 0 0
PRELOAD_COMPLETE 0 0 0
GET_CERT 0 0 0

Under Controller to AP communication; it can be noticed that Config message New is 1 while Total is increasing. This would mean Config message hasn't been received by AP (Ack is 0 as well). Log_Config seem to be unaffected.

 

AP "TAC-Test-AP1" Switch to AP Message Counts

---------------------------------------------
Message Total New Acknowledged
------- ----- --- ------------
CONFIG 123 1 0
PROV 0 0 0
FLASH 0 0 0
REBOOT 0 0 0
DISCONNECT 0 0 0
LOG_CONFIG 1 1 1
CLEAR_FW_CONFIG 0 0 0
FW_CONFIG 0 0 0
ALG_CONFIG 0 0 0
DNS_ID_MAP_CONFIG 0 0 0
ROLE_CONFIG 0 0 0
ROLE_BWM_CONFIG 0 0 0
ACL_CONFIG 0 0 0
ACL_DOWNLOADED 0 0 0
MONITORING_MSG_CONFIG 0 0 0
LED_ACTION 0 0 0
AM_FILTER_CONFIG 0 0 0
IDLE_USER_TIMEOUT 0 0 0
PORT_CLEAR_ACTION 0 0 0
REQ_CSR 0 0 0
INSTALL_CERT 0 0 0
NAT_POOL_CONFIG 0 0 0
SETENV 0 0 0
CP_FQDN_CONFIG 0 0 0
ESSID_LIST 0 0 0
REQ_PRELOAD 0 0 0
CERT_DATA 0 0 0


"Rebootstraps and Control Messages Log" section can be used to get insight of the message size.


Rebootstraps and Control Messages Log

-------------------------------------

Recent Messages Time now: Tue Jul 30 12:56:34 2013

--------------- ----------------------------------

Time Offset Message details

----------- ---------------

-17 SENT: CONFIG len=3412 peer=10.51.204.7 seq_num=1 tries=123 rtt=-1

-35 RCVD: KEEPALIVE len=20 peer=10.51.204.7 seq_num=7 rtt=0 result=MSG_IN_PROGRESS

-635 RCVD: KEEPALIVE len=20 peer=10.51.204.7 seq_num=6 rtt=0 result=MSG_IN_PROGRESS


Additionally, we can use "show ap debug config-msg-history ap-name <name>" command to get messages sent / received by the AP. This command is collected from the AP itself i.e. AP's perspective of things. In below example; we would notice there is no "RCVD REQ type=CONFIG" message while we can see LOG_CONFIG (which corresponds to "Switch to AP Message" section of show ap details advanced ap-name "name of AP")

show ap debug config-msg-history ap-name "TAC-Test-AP1"

 

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Fri Dec 31 16:00:42 1999(428532955 secs ago): SENT REQ type=HELLO len=1032 peer=10.50.60.99 seq_num=0 num_attempts=1 rtt=0 secs 04000004030400000000050A33CC0704386D43A50400000000050A323C630400000015040000000004000001330400000001040000014204000000170400000

Fri Dec 31 16:00:42 1999(428532955 secs ago): SENT REQ type=HELLO len=1034 peer=10.51.195.250 seq_num=0 num_attempts=1 rtt=0 secs 04000004050400000000050A33CC0704386D43AA0400000000050A33C3FA04000000150400000000040000013304000000010400000142040000001704000

Tue Jul 30 11:55:59 2013(3638 secs ago): SENT REQ type=AP_READY len=25 peer=10.51.195.250 seq_num=1 num_attempts=1 rtt=0 secs 04000000140400000001050A33CC0704386D43AA0400000001

Tue Jul 30 11:55:59 2013(3638 secs ago): RCVD REQ type=LOG_CONFIG len=131 peer=10.51.195.248 seq_num=0 resps_sent=1 040000007E0400000014050A33C3F80451F81A4E0400000000070201070200040000000504000001900400000004040000000004000001980400000004040000

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Wed Dec 31 16:00:00 1969(1375217797 secs ago): RCVD RESP type=HELLO len=0 peer=0.0.0.0 seq_num=0

Tue Jul 30 12:05:59 2013(3038 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=2 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000020451F81CA705FFFFFF80050A33CC7E0000000000

Tue Jul 30 12:15:59 2013(2438 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=3 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000030451F81EFF05FFFFFF80050A33CC7E0000000000

Tue Jul 30 12:25:59 2013(1838 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=4 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000040451F8215705FFFFFF80050A33CC7E0000000000

Tue Jul 30 12:35:59 2013(1238 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=5 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000050451F823AF05FFFFFF80050A33CC7E0000000000

Tue Jul 30 12:45:59 2013(638 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=6 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000060451F8260705FFFFFF80050A33CC7E0000000000

Tue Jul 30 12:55:59 2013(38 secs ago): SENT REQ type=KEEPALIVE len=45 peer=10.51.195.250 seq_num=7 num_attempts=1 rtt=0 secs 04000000280400000002050A33CC0704386D43AA04000000070451F8285F05FFFFFF80050A33CC7E0000000000

 

 

Cause: Config message size may have increased after upgrade and the network path isn't be able to deliver it to the AP.

 

Resolution: The issue is triggered due to inability of network to handle Campus AP control messages.  For remote deployments; Aruba's recommandation is to use Remote AP. This issue wouldn't be encountered for Remote AP as they form an IPSEC tunnel and config message would be delivered within the tunnel. Being transmitted within AP--> Controller IPSEC tunnel; the network's MTU size wouldn't affect the config message. Alternately; CPSEC can be enabled as it would also use IPSEC tunnel for Control-Plane Communication.

Version History
Revision #:
1 of 1
Last update:
‎09-10-2014 01:43 PM
Updated by:
 
Contributors
Search Airheads
Showing results for 
Search instead for 
Did you mean: 
Is this a frequent problem?

Request an official Aruba knowledge base article to be written by our experts.