Wireless Access

last person joined: 22 hours ago 

Access network design for branch, remote, outdoor, and campus locations with HPE Aruba Networking access points and mobility controllers.
Back to discussions
Expand all | Collapse all

RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

This thread has been viewed 3 times

  • 1.  RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 11:30 AM

    Ever since I upgraded our controller master standby pair to 6.3.1.16 none of our RAPs have worked.  Some details:


    --RAPs definitely talking to controller I see 4500 traffic.  The RAPs show up on 'show crypto ipsec sa' and isakmp sa
    --I can see their L2TP internal IP slowly increasing every few minutes (flapping?)
    --In the security ike logs i see them establish the tunnel then immediately tear it down

    May 26 08:25:45  isakmpd[1657]: <103076> <INFO> |ike|  IKEv2 IPSEC Tunnel created for peer <external IP>:54267
May 26 08:25:45  isakmpd[1657]: <103077> <INFO> |ike|  IKEv2 IKE_SA succeeded for peer <external IP>:54267
May 26 08:25:45  isakmpd[1657]: <103078> <INFO> |ike|  IKEv2 CHILD_SA successful for peer <external IP>:54267
May 26 08:25:45  isakmpd[1657]: <103082> <INFO> |ike|  IKEv2 Client-Authentication succeeded for 10.50.43.179 (External 73.196.151.108) for default-vpn-role
May 26 08:25:45  isakmpd[1657]: <103101> <INFO> |ike|  IPSEC SA deleted for peer <external IP>
May 26 08:25:45  isakmpd[1657]: <103102> <INFO> |ike|  IKE SA deleted for peer <external IP>
    •  They don't make it as far as getting to the AP table
    • Show datapath session table on the IP of the RAP shows some traffic flagged as FY or FYDC but not apparent why.
    • This is happening on RAP2 and RAP5 devices -- none are working.  Aruba OS 5.0.  I tried factory reset and I tried provisioning the RAP within our network to eliminate firewall issues.
    • I diff'd the configs before and after the upgrade and I see no big differences.

    I have a case open but support has been slow and unhelpful so far.  It took them an hour of CLI to even see the RAP traffic and then they wanted me to downgrade the controller or check the port channel to the controller (no reason whatsoever to suspect this).  I would be grateful if you could let me know anything else I can try or look into. 



  • 2.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 11:34 AM

    - Why are you upgrading (so that we can understand if you were trying to avoid an issue or not)?

    - Does the public ip address poing to the master or the standby?



  • 3.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 11:36 AM
    @cjoseph wrote:

    - Why are you upgrading (so that we can understand if you were trying to avoid an issue or not)?

    - Does the public ip address poing to the master or the standby?

    -Upgrading because of some SSL bugs that were fixed after .5 -- our controller was showing up in some internal security audits.

    -Master

     

    Thanks



  • 4.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 11:37 AM

    Does your public ip address NAT to your Master controller or your standby?



  • 5.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 11:40 AM
    @cjoseph wrote:

    Does your public ip address NAT to your Master controller or your standby?

    Public IP NATs to the master controller.  However I also factory reset one of the RAPs and plugged it into the internal network and tried to provision to the internal Master IP (to eliminate firewall issues) and it behaved the same way.



  • 6.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 11:47 AM
    Are you using zero touch provisioning for your raps?


  • 7.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 11:52 AM
    @cjoseph wrote:
    Are you using zero touch provisioning for your raps?

    Not 100% sure what that entails.  For a new RAP I would add its MAC to the RAP whitelist on the controller.  Then it would need to be plugged into home internet router (or similar) in e0, with laptop in e1.  Then launch browser, rapconsole.arubanetworks.com, and type in the controller's external IP/DNS.  RAP would then provision.



  • 8.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 12:12 PM

    Yes, that is what zero touch provisioning means.

     

    What is the output of "show rights default-vpn-role"?

     

    Also, do you have an LMS-IP in the AP System Profile of the ap group that your RAPs are in?

     



  • 9.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 12:14 PM 
    #show rights default-vpn-role

Derived Role = 'default-vpn-role'
 Up BW:No Limit   Down BW:No Limit
 L2TP Pool = default-l2tp-pool
 PPTP Pool = default-pptp-pool
 Periodic reauthentication: Disabled
 ACL Number = 69/0
 Max Sessions = 65535

 Check CP Profile for Accounting = TRUE

access-list List
----------------
Position  Name         Type     Location
--------  ----         ----     --------
1         ra-guard     session
2         allowall     session
3         v6-allowall  session

ra-guard
--------
Priority  Source  Destination  Service           Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------           ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         user    any          icmpv6 rtr-adv    deny                             Low                                                           6
allowall
--------
Priority  Source  Destination  Service  Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------  ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          any      permit                           Low                                                           4
v6-allowall
-----------
Priority  Source  Destination  Service  Action  TimeRange  Log  Expired  Queue  TOS  8021P  Blacklist  Mirror  DisScan  ClassifyMedia  IPv4/6
--------  ------  -----------  -------  ------  ---------  ---  -------  -----  ---  -----  ---------  ------  -------  -------------  ------
1         any     any          any      permit                           Low                                                           6

Expired Policies (due to time constraints) = 0


  • 10.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 12:17 PM

    Also, do you have an LMS-IP in the AP System Profile of the ap group that your RAPs are in?



  • 11.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 12:18 PM
    @cjoseph wrote:

    Also, do you have an LMS-IP in the AP System Profile of the ap group that your RAPs are in?

    Sorry missed that question -- No, doesn't look like it.  I don't see one when i do

     

     #show ap system-profile Remote-APs

AP system profile "Remote-APs"
------------------------------
Parameter                               Value
---------                               -----
RF Band                                 g
RF Band for AM mode scanning            all
Native VLAN ID                          1
Tunnel Heartbeat Interval               1
Session ACL                             allowall
Corporate DNS Domain                    N/A
SNMP sysContact                         N/A
LED operating mode (11n/11ac APs only)  normal
SAP MTU                                 N/A
RAP MTU                                 1200 bytes
LMS IP                                  N/A
Backup LMS IP                           N/A
LMS IPv6                                N/A
Backup LMS IPv6                         N/A
LMS Preemption                          Disabled
LMS Hold-down Period                    600 sec
LMS ping interval                       20
GRE Striping IP                         N/A
Remote-AP DHCP Server VLAN              N/A
Remote-AP DHCP Server Id                192.168.11.1
Remote-AP DHCP Default Router           192.168.11.1
Remote-AP DHCP DNS Server               N/A
Remote-AP DHCP Pool Start               192.168.11.2
Remote-AP DHCP Pool End                 192.168.11.254
Remote-AP DHCP Pool Netmask             255.255.255.0
Remote-AP DHCP Lease Time               0 days
Remote-AP uplink total bandwidth        0 kbps
Remote-AP bw reservation 1              N/A
Remote-AP bw reservation 2              N/A
Remote-AP bw reservation 3              N/A
Remote-AP Local Network Access          Disabled
Bootstrap threshold                     8
Double Encrypt                          Enabled
Dump Server                             N/A
Heartbeat DSCP                          0
Maintenance Mode                        Disabled
Maximum Request Retries                 10
Request Retry Interval                  10 sec
Number of IPSEC retries                 85
AeroScout RTLS Server                   N/A
RTLS Server configuration               N/A
RTLS Server Compatibility Mode          Enabled
Telnet                                  Disabled
Spanning Tree                           Disabled


  • 12.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 12:22 PM
      |   view attached

    That being said, during provisioning of the RAP it does get to the LMS Connectivity part and list the external IP of the controller - see attached (it switches between the same error for Master and LMS connectivity every couple minutes)

     

     



  • 13.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted May 26, 2015 12:24 PM

    Unfortunately, everything looks okay in General.  Since you have a support case open, you should keep working with them for a possible resolution or downgrade to see if the only issue is the code.  The RAP5 requires a special console cable that few people have otherwise we would be able to see things from the access point's perspective.  I want to ask if you can provision any other APs as RAPs to see if the issue is specific to the RAP5 and RAP2.

     

     



  • 14.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted May 26, 2015 01:01 PM
    @cjoseph wrote:

    Unfortunately, everything looks okay in General.  Since you have a support case open, you should keep working with them for a possible resolution or downgrade to see if the only issue is the code.  The RAP5 requires a special console cable that few people have otherwise we would be able to see things from the access point's perspective.  I want to ask if you can provision any other APs as RAPs to see if the issue is specific to the RAP5 and RAP2.

     

     

    Yes I will try a spare AP-as-RAP tomorrow.  Thanks for your time.



  • 15.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade
    Best Answer

    Posted May 27, 2015 07:02 PM

    Just a followup if anyone has a similar issue -- those FDYC flags i saw in the RAP traffic were the firewall denying the RAPs.  Turns out the user role "ap-role" (the default the RAPs get) were missing the built in ACLs which allow the RAPs to provision.  The tech copied the standard ACL off his test controller onto mine and everything began working again.  He couldnt explain how for almost a year back (we save our configs every night) the ap-role was empty but the RAPs continued to work. But adding it back in fixed it.



  • 16.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    Posted Mar 19, 2020 09:07 AM

    Hi,

     

    I'm expecting strange issue with RAPs fllaping - below log:

    Mar 19 06:02:07 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:15 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:15 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:23 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:23 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:31 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:31 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:38 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:38 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:46 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:46 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:02:54 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:02:54 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:02 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:02 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:09 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:09 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:17 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:17 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:25 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:25 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:33 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:33 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:41 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:41 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:48 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:48 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:03:56 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.
    Mar 19 06:03:56 stm[3735]: <305027> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: No valid instances of required profile "ids signature-matching-profile"
    Mar 19 06:04:04 stm[3735]: <305004> <3735> <ERRS> |stm| AP 9c:1c:12:c9:5e:95: ids signature-matching-profile "default" is invalid.

     

    I was using mig tool to mirate I did not change any default IDS, roles, policy etc...

     

    Does anybody have the same issue?

     



  • 17.  RE: RAPs broken, tunnel flapping after 6.3.1.5 -> 6.3.1.16 upgrade

    EMPLOYEE
    Posted Mar 19, 2020 09:11 AM

    Closing, because this thread is  FIVE years old.  Please open another thread.