Airheads Community

Jul 17, 2014 08:58 AM

AnandKumar Sukumar

Question :

After upgrading the controller software, the AP can't come up. We can see PAPI getting setup on the controller datapath. What could be the reason for which the APs stuck at upgrading state on the Controller?

Why do we see some strange ports for traffic between AP and Controller after code upgrade of Controller?

Environment Information :

Any Aruba Controller
Any Access Points
Any Aruba OS

Symptoms : We can see there is PAPI established from AP to the Controller, however the AP never completes upgrade. We can see from the switch logs that the AP is trying to access ports like 1025, 1026, 1039 etc.

Cause : File Transfer Protocol (FTP) dynamically uses high ports for transferring data between client and server. If the intermediate device between the FTP client and server blocks the ephemeral ports, then FTP fails.

Resolution :

After upgrading the controller software, the AP can't come up. We can see PAPI getting setup on the controller datapath. What could be the reason for which the APs stuck at upgrading state on the Controller?

Why do we see some strange ports for traffic between AP and Controller after code upgrade of Controller?

We must first ensure the ap-role on the Controller allows FTP and TFTP, as mentioned in other articles. In this article, we will assume this has been verified. The use-case in this article applies to a network in which we are trying to use an ACL to apply at the switch port (say Cisco) where the AP connects to allow necessary traffic for the respective AP to Controller Communication. We will also assume Controller was upgraded from 5.x to 6.x.

The ACLs allows an AP to register without an issue and we see PAPI established fine. The following are sample ACLs used on the switch:

ip access-list extended aruba-ap_permit

permit udp any any eq bootps

permit udp host 172.250.1.2 eq 8211 host 172.16.0.1 eq 8211

permit gre host 172.250.1.2 host 172.16.0.1

permit udp host 172.250.1.2 host 172.16.0.1 eq syslog

permit tcp host 172.250.1.2 host 172.16.0.1 eq ftp

permit tcp host 172.250.1.2 host 172.16.0.1 eq 22

permit udp host 172.250.1.2 host 172.16.0.1 eq tftp

deny ip any any log

AP registers to the Controller with PAPI and we can see the AP is Upgrading as per "show ap database".

(aruba_6000_2) #show ap database | include ap124-09

Aruba-AP1 AP-Group1 135 172.250.1.2 Upgrading UI 172.16.0.10

(aruba_6000_2) #show datapath session | include 172.250.1.2

172.250.1.2 172.16.0.1 17 8211 8211 0/0 0 0 1 local b 0 0 FY

172.16.0.1 172.1250.1.2 17 8211 8211 0/0 0 0 1 local b 4 4 FC

172.16.0.1 172.250.1.2 17 514 49152 0/0 0 0 1 pc1 b 5 5 FY

172.250.1.2 172.16.0.1 17 49152 514 0/0 0 0 0 pc1 b 0 0 FC

172.16.0.1 172.250.1.2 17 8211 8211 0/0 0 0 0 pc1 b 5 5 FY

172.250.1.2 172.16.0.1 17 8211 8211 0/0 0 0 0 pc1 b 0 0 FC

We can see there is PAPI established from AP to the Controller, however the AP never completes upgrade. We can see from the switch logs that the AP is trying to access ports like 1025, 1026, 1039 etc. For example:

Aug 2 21:32:42.586: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(62962) -> 172.16.0.1(1039), 1 packet

Aug 2 21:32:45.632: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1025), 1 packet

Aug 2 21:36:24.364: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied tcp 172.250.1.2(52294) -> 172.16.0.1(1041), 1 packet

Aug 2 21:36:33.433: %SEC-6-IPACCESSLOGP: list aruba-ap_permit denied udp 172.250.1.2(49153) -> 172.16.0.1(1026), 1 packet

AP connects to the controller with PAPI and see it's Upgrading in "show ap database".

(aruba_6000_2) #show ap database | include ap124-09

ap124-09 mimo01 124 10.250.0.32 Upgrading UI 172.19.208.5

(aruba_6000_2) #show datapath session | include 10.250.0.32

10.250.0.32 10.250.0.2 17 8211 8211 0/0 0 0 1 local b 0 0 FY

10.250.0.2 10.250.0.32 17 8211 8211 0/0 0 0 1 local b 4 4 FC

172.19.208.1 10.250.0.32 17 514 49152 0/0 0 0 1 pc1 b 5 5 FY

10.250.0.32 172.19.208.1 17 49152 514 0/0 0 0 0 pc1 b 0 0 FC

172.19.208.1 10.250.0.32 17 8211 8211 0/0 0 0 0 pc1 b 5 5 FY

10.250.0.32 172.19.208.1 17 8211 8211 0/0 0 0 0 pc1 b 0 0 FC

This behavior of APs being unable to download (FTP) the Aruba OS code during upgrade when connected to Ethernet ports with static ACLs, is an expected outcome. File Transfer Protocol (FTP) dynamically uses high ports for transferring data between client and server. If the intermediate device between the FTP client and server blocks the ephemeral ports, then FTP fails. Below are some suggestions that would help address this requirement –

1. Use dynamic ACL on the uplink switches (say Cisco) to allow high ports for FTP communication between the APs and controllers

2. Configure the AP as a 802.1x supplicant to authenticate the AP to the switch port on which it is connected

3. Configure CPSEC on the controller and ensure to allow UDP ports 4500 and 514, and Protocol 47 pass through (any ACL on) the switch port. This will help, since upgrade traffic will all be forwarded inside the IPSec tunnel.

Answer :