Controller Based WLANs

APs struck in Upgrading and strange ports used by the AP

Aruba Employee

Question :

 

After upgrading the controller software, the AP can't come up. We can see PAPI getting setup on the controller datapath. What could be the reason for which the APs stuck at upgrading state on the Controller?


Why do we see some strange ports for traffic between AP and Controller after code upgrade of Controller?

 

Environment Information :

 

Any Aruba Controller 
Any Access Points
Any Aruba OS

 

 

Symptoms : We can see there is PAPI established from AP to the Controller, however the AP never completes upgrade.  We can see from the switch logs that the AP is trying to access ports like 1025, 1026, 1039 etc.

 

Cause : File Transfer Protocol (FTP) dynamically uses high ports for transferring data between client and server.  If the intermediate device between the FTP client and server blocks the ephemeral ports, then FTP fails.

 

Resolution :

After upgrading the controller software, the AP can't come up. We can see PAPI getting setup on the controller datapath. What could be the reason for which the APs stuck at upgrading state on the Controller?
Why do we see some strange ports for traffic between AP and Controller after code upgrade of Controller?
 
We must first ensure the ap-role on the Controller allows FTP and TFTP, as mentioned in other articles.  In this article, we will assume this has been verified.  The use-case in this article applies to a network in which we are trying to use an ACL to apply at the switch port (say Cisco) where the AP connects to allow necessary traffic for the respective AP to Controller Communication. We will also assume Controller was upgraded from 5.x to 6.x.
 
The ACLs allows an AP to register without an issue and we see PAPI established fine.   The following are sample ACLs used on the switch:
 
ip access-list extended aruba-ap_permit
 permit udp any any eq bootps
 permit udp host 172.250.1.2 eq 8211 host 172.16.0.1 eq 8211
 permit gre host 172.250.1.2 host 172.16.0.1
 permit udp host 172.250.1.2 host 172.16.0.1 eq syslog
 permit tcp host 172.250.1.2 host 172.16.0.1 eq ftp
 permit tcp host 172.250.1.2 host 172.16.0.1 eq 22
 permit udp host 172.250.1.2 host 172.16.0.1 eq tftp
 deny  ip any any log
 
AP registers to the Controller with PAPI and we can see the AP is Upgrading as per "show ap database".  
 
(aruba_6000_2) #show ap database | include ap124-09
Aruba-AP1  AP-Group1  135      172.250.1.2    Upgrading  UI     172.16.0.10
 
(aruba_6000_2) #show datapath session | include 172.250.1.2
172.250.1.2     172.16.0.1      17   8211  8211   0/0     0 0   1   local       b    0      0      FY
172.16.0.1      172.1250.1.2     17   8211  8211   0/0     0 0   1   local       b    4      4      FC
172.16.0.1    172.250.1.2     17   514   49152  0/0     0 0   1   pc1         b    5      5      FY
172.250.1.2     172.16.0.1    17   49152 514    0/0     0 0   0   pc1         b    0      0      FC
172.16.0.1    172.250.1.2     17   8211  8211   0/0     0 0   0   pc1         b    5      5      FY
172.250.1.2     172.16.0.1    17   8211  8211   0/0     0 0   0   pc1         b    0      0      FC
 
We can see there is PAPI established from AP to the Controller, however the AP never completes upgrade.  We can see from the switch logs that the AP is trying to access ports like 1025, 1026, 1039 etc. For example:
 
Aug  2 21:32:42.586: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(62962) -> 172.16.0.1(1039), 1 packet
Aug  2 21:32:45.632: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1025), 1 packet
Aug  2 21:36:24.364: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(52294) -> 172.16.0.1(1041), 1 packet
Aug  2 21:36:33.433: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1026), 1 packet
 
AP connects to the controller with PAPI and see it's Upgrading in "show ap database".
 
(aruba_6000_2) #show ap database | include ap124-09
ap124-09 mimo01 124 10.250.0.32 Upgrading UI 172.19.208.5
 
(aruba_6000_2) #show datapath session | include 10.250.0.32
10.250.0.32 10.250.0.2 17 8211 8211 0/0 0 0 1 local b 0 0 FY
10.250.0.2 10.250.0.32 17 8211 8211 0/0 0 0 1 local b 4 4 FC
172.19.208.1 10.250.0.32 17 514 49152 0/0 0 0 1 pc1 b 5 5 FY
10.250.0.32 172.19.208.1 17 49152 514 0/0 0 0 0 pc1 b 0 0 FC
172.19.208.1 10.250.0.32 17 8211 8211 0/0 0 0 0 pc1 b 5 5 FY
10.250.0.32 172.19.208.1 17 8211 8211 0/0 0 0 0 pc1 b 0 0 FC
 
This behavior of APs being unable to download (FTP) the Aruba OS code during upgrade when connected to Ethernet ports with static ACLs, is an expected outcome.  File Transfer Protocol (FTP) dynamically uses high ports for transferring data between client and server.  If the intermediate device between the FTP client and server blocks the ephemeral ports, then FTP fails.  Below are some suggestions that would help address this requirement – 
 
1. Use dynamic ACL on the uplink switches (say Cisco) to allow high ports for FTP communication between the APs and controllers 
2. Configure the AP as a 802.1x supplicant to authenticate the AP to the switch port on which it is connected 
3. Configure CPSEC on the controller and ensure to allow UDP ports 4500 and 514, and Protocol 47 pass through (any ACL on) the switch port.   This will help, since upgrade traffic will all be forwarded inside the IPSec tunnel.

 

Related Links : https://kb.arubanetworks.com/app/answers/detail/a_id/63

 

Answer :

After upgrading the controller software, the AP can't come up. We can see PAPI getting setup on the controller datapath. What could be the reason for which the APs stuck at upgrading state on the Controller?
Why do we see some strange ports for traffic between AP and Controller after code upgrade of Controller?
 
We must first ensure the ap-role on the Controller allows FTP and TFTP, as mentioned in other articles.  In this article, we will assume this has been verified.  The use-case in this article applies to a network in which we are trying to use an ACL to apply at the switch port (say Cisco) where the AP connects to allow necessary traffic for the respective AP to Controller Communication. We will also assume Controller was upgraded from 5.x to 6.x.
 
The ACLs allows an AP to register without an issue and we see PAPI established fine.   The following are sample ACLs used on the switch:
 
ip access-list extended aruba-ap_permit
 permit udp any any eq bootps
 permit udp host 172.250.1.2 eq 8211 host 172.16.0.1 eq 8211
 permit gre host 172.250.1.2 host 172.16.0.1
 permit udp host 172.250.1.2 host 172.16.0.1 eq syslog
 permit tcp host 172.250.1.2 host 172.16.0.1 eq ftp
 permit tcp host 172.250.1.2 host 172.16.0.1 eq 22
 permit udp host 172.250.1.2 host 172.16.0.1 eq tftp
 deny  ip any any log
 
AP registers to the Controller with PAPI and we can see the AP is Upgrading as per "show ap database".  
 
(aruba_6000_2) #show ap database | include ap124-09
Aruba-AP1  AP-Group1  135      172.250.1.2    Upgrading  UI     172.16.0.10
 
(aruba_6000_2) #show datapath session | include 172.250.1.2
172.250.1.2     172.16.0.1      17   8211  8211   0/0     0 0   1   local       b    0      0      FY
172.16.0.1      172.1250.1.2     17   8211  8211   0/0     0 0   1   local       b    4      4      FC
172.16.0.1    172.250.1.2     17   514   49152  0/0     0 0   1   pc1         b    5      5      FY
172.250.1.2     172.16.0.1    17   49152 514    0/0     0 0   0   pc1         b    0      0      FC
172.16.0.1    172.250.1.2     17   8211  8211   0/0     0 0   0   pc1         b    5      5      FY
172.250.1.2     172.16.0.1    17   8211  8211   0/0     0 0   0   pc1         b    0      0      FC
 
We can see there is PAPI established from AP to the Controller, however the AP never completes upgrade.  We can see from the switch logs that the AP is trying to access ports like 1025, 1026, 1039 etc. For example:
 
Aug  2 21:32:42.586: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(62962) -> 172.16.0.1(1039), 1 packet
Aug  2 21:32:45.632: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1025), 1 packet
Aug  2 21:36:24.364: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(52294) -> 172.16.0.1(1041), 1 packet
Aug  2 21:36:33.433: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1026), 1 packet
 
AP connects to the controller with PAPI and see it's Upgrading in "show ap database".
 
(aruba_6000_2) #show ap database | include ap124-09
ap124-09 mimo01 124 10.250.0.32 Upgrading UI 172.19.208.5
 
(aruba_6000_2) #show datapath session | include 10.250.0.32
10.250.0.32 10.250.0.2 17 8211 8211 0/0 0 0 1 local b 0 0 FY
10.250.0.2 10.250.0.32 17 8211 8211 0/0 0 0 1 local b 4 4 FC
172.19.208.1 10.250.0.32 17 514 49152 0/0 0 0 1 pc1 b 5 5 FY
10.250.0.32 172.19.208.1 17 49152 514 0/0 0 0 0 pc1 b 0 0 FC
172.19.208.1 10.250.0.32 17 8211 8211 0/0 0 0 0 pc1 b 5 5 FY
10.250.0.32 172.19.208.1 17 8211 8211 0/0 0 0 0 pc1 b 0 0 FC
 
This behavior of APs being unable to download (FTP) the Aruba OS code during upgrade when connected to Ethernet ports with static ACLs, is an expected outcome.  File Transfer Protocol (FTP) dynamically uses high ports for transferring data between client and server.  If the intermediate device between the FTP client and server blocks the ephemeral ports, then FTP fails.  Below are some suggestions that would help address this requirement – 
 
1. Use dynamic ACL on the uplink switches (say Cisco) to allow high ports for FTP communication between the APs and controllers 
2. Configure the AP as a 802.1x supplicant to authenticate the AP to the switch port on which it is connected 
3. Configure CPSEC on the controller and ensure to allow UDP ports 4500 and 514, and Protocol 47 pass through (any ACL on) the switch port.   This will help, since upgrade traffic will all be forwarded inside the IPSec tunnel.

 

Version history
Revision #:
1 of 1
Last update:
‎07-17-2014 05:58 AM
Updated by:
 
Labels (1)
Contributors
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: 
Is this a frequent problem?

Request an official Aruba knowledge base article to be written by our experts.