Wireless Access

last person joined: yesterday 

Access network design for branch, remote, outdoor, and campus locations with HPE Aruba Networking access points and mobility controllers.
Expand all | Collapse all

AP not registering at controller after topology change

This thread has been viewed 21 times
  • 1.  AP not registering at controller after topology change

    Posted Mar 13, 2017 06:15 PM
      |   view attached

    Hi,

     

    After doing a topology change at a customer with master/local config meaning we changed the vlan for an ap group. The APs of that group are not shown in the controller anymore. The APs get an DHCP IP Adress (from the controller they should connect to), can ping the controller's ip, the controller can also ping the AP.

     

    A console trace of the AP shows the correct IP of the master, and ADP also delivering the correct values (txt output attached)

    What we don't have in that subnet is a DNS Server (as it only serves for the APs) and we can't set a DHCP Value 43 because another AP group is also usind that AP VLAN with a different local controller to connect to,

     

    We also updated the controller to a newer firmware (6.4.4.12) and the AP also pulled the firmware and was updated. 

     

    What else could we try to get that group of AP 105 and 103 connected?

    Any ideas?

    BR

    Peter

    Attachment(s)

    txt
    boot_ap.txt   5 KB 1 version


  • 2.  RE: AP not registering at controller after topology change

    Posted Mar 13, 2017 08:00 PM
    What do you mean that you changed the VLAN for the AP group ?

    Can you run the following command in the controller
    Show log system 50 and see if it shows any logs related to those APs

    Any other APs able to join ?

    Get Outlook for iOS


  • 3.  RE: AP not registering at controller after topology change

    Posted Mar 14, 2017 01:21 AM

    Hi,

     

    the APs got an IP Adress out of the internal customers network before. As the customer ran out of availiable IP Adresses as his WLAN growed, we decided to set up a VLAN for the APs and wanted to move the APs into that VLAN. We also setup the VLAN at the Controller, entered an IP Adress and tagged that vlan on the uplink.

     

    There are lots of other APs connecting to that controller and working fine - and what is so strange is that we already have used that vlan at another customers site (that is directly linked via fiber) for another AP group and the APs there are connecting to the same controller without probs.

     

    I'll go and check the log as soon as I am onsite again and let you know what it says

     

    Peter



  • 4.  RE: AP not registering at controller after topology change

    Posted Mar 14, 2017 03:02 AM

    Hi Peter,

     

    Please enable the following debug & paste the logs:

     

    config# logging level debugging ap-debug <name of AP>

     

    #show  datapath session table <ip-address of AP>

    # show log system all | include <ap-name>

     

    Please try pinging the AP's default gateway from the controller with different MTU sizes

     

    # ping <ip> packet-size <>  count 100

     

    if possible, please attach the console output from one of the AP's in question.

     



  • 5.  RE: AP not registering at controller after topology change

    Posted Mar 14, 2017 07:13 AM
      |   view attached

    Hello, My name is Stefan and i am the Left Hand (often not that good as the Right) of Peter.

     

    -----------------------------------------------

    show log system 50

    Mar 14 07:59:12 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 08:44:03 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 08:54:58 KERNEL(f0:5c:19:c4:2b:98@192.168.98.28): RESTARTING ALL TX
    Mar 14 09:28:55 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 10:13:16 :303022: <WARN> |AP f0:5c:19:c4:2d:fa@192.168.98.37 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:07:34 PST 1999; DHCP timed out
    Mar 14 10:13:47 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 10:14:28 :311002: <WARN> |AP f0:5c:19:c4:2d:fa@192.168.98.37 sapd| Rebooting: SAPD: Reboot after successful image upgrade
    Mar 14 10:14:28 :303086: <ERRS> |AP f0:5c:19:c4:2d:fa@192.168.98.37 nanny| Process Manager (nanny) shutting down - AP will reboot!
    Mar 14 10:16:31 :311004: <WARN> |AP f0:5c:19:c4:2a:c2@192.168.98.17 sapd| Missed 8 heartbeats; rebootstrapping
    Mar 14 10:26:34 KERNEL(f0:5c:19:c4:2a:c2@192.168.98.17): RESTARTING ALL TX
    Mar 14 10:34:00 :311004: <WARN> |AP f0:5c:19:c4:2d:76@192.168.98.29 sapd| Missed 8 heartbeats; rebootstrapping
    Mar 14 10:35:03 KERNEL(f0:5c:19:c4:2d:76@192.168.98.29): RESTARTING ALL TX
    Mar 14 10:42:11 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: No reboot message found.
    Mar 14 10:54:18 KERNEL(EG-VT Kche@192.168.2.42): RESTARTING ALL TX
    Mar 14 10:54:54 KERNEL(f0:5c:19:c4:2e:e0@192.168.98.24): RESTARTING ALL TX
    Mar 14 10:55:17 KERNEL(f0:5c:19:c4:2c:b2@192.168.98.5): RESTARTING ALL TX
    Mar 14 10:55:46 KERNEL(f0:5c:19:c4:2e:94@192.168.98.8): RESTARTING ALL TX
    Mar 14 10:55:54 KERNEL(f0:5c:19:c4:2d:4c@192.168.98.18): RESTARTING ALL TX
    Mar 14 10:57:36 KERNEL(84:d4:7e:cd:d9:36@192.168.98.11): RESTARTING ALL TX
    Mar 14 10:58:01 KERNEL(Gemeinde_DG@192.168.98.54): RESTARTING ALL TX
    Mar 14 11:02:06 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: No reboot message found.

    --------------------------------------------

    At the Moment i use AP "6c:f3:7f:cb:a7:4c@192.168.98.51" for my tryings.

    With this Log i was able to find out that there is a problem with the Gateway.

     

    Mar 14 10:13:47 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0

     

    I think i was able to fix this. I am able to ping the AP(192.168.98.51), the Gateway(192.168.98.191, the Primary Master(192.168.98.190) and backwards.

    But i think ther is a problem left. I was able to Ping the local Master on the 2nd location (192.168.2.180) but i was not able to Ping backwards.

     

    ---------------------------------------

    (Aruba7210) (config) #show log system all | include 6c:f3:7f:cb:a7:4c

    Mar 14 07:59:12 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 08:44:03 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 09:28:55 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 10:13:47 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:14 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0
    Mar 14 10:42:11 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: No reboot message found.
    Mar 14 11:02:06 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: No reboot message found.
    Mar 14 11:46:59 :303022: <WARN> |AP 6c:f3:7f:cb:a7:4c@192.168.98.51 nanny| Reboot Reason: AP rebooted Fri Dec 31 16:44:16 PST 1999; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last rebootstrap reason: HELLO-TIMEOUT, 228 sec before: Last Ctrl msg: HELLO len=943 dest=192.168.2.180 tries=10 seq=0

    -------------------------------------

    Shows nearly the Same. Why does he try 192.168.2.180!?!?

     

    ------------------------------------- 

    Ping results: 

     

    (Aruba7210) (config) # ping 192.168.98.51 packet-size 2000 count 100
    Press 'q' to abort.
    Sending 100, 2000-byte ICMP Echos to 192.168.98.51, timeout is 2 seconds:
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    Success rate is 100 percent (100/100), round-trip min/avg/max = 0.325/0.37367/0.886 ms

    (Aruba7210) (config) # ping 192.168.98.51 packet-size 1600 count 100
    Press 'q' to abort.
    Sending 100, 1600-byte ICMP Echos to 192.168.98.51, timeout is 2 seconds:
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    Success rate is 100 percent (100/100), round-trip min/avg/max = 0.304/0.35784/0.519 ms

    (Aruba7210) (config) # ping 192.168.98.51 packet-size 1500 count 100
    Press 'q' to abort.
    Sending 100, 1500-byte ICMP Echos to 192.168.98.51, timeout is 2 seconds:
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    Success rate is 100 percent (100/100), round-trip min/avg/max = 0.317/0.50528/1.809 ms

    (Aruba7210) (config) # ping 192.168.98.51 packet-size 10 count 100
    Press 'q' to abort.
    Sending 100, 10-byte ICMP Echos to 192.168.98.51, timeout is 2 seconds:
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    Success rate is 100 percent (100/100), round-trip min/avg/max = 0.15/0.1761/0.305 ms

    -----------------------------------------

     

    #show datapath session table 192.168.98.51 

    no results

     

    Console Output of the AP was attached.

     

    thanks for Help

    Stefan

     

     

    Attachment(s)

    txt
    boot_ap_2.txt   5 KB 1 version


  • 6.  RE: AP not registering at controller after topology change
    Best Answer

    Posted Mar 14, 2017 01:31 PM

    Hi Airheads,

    thanks a lot for some hints and Tips.

    I was able to solve the problem.

     

    The Problem:

    New Accesspoints are connect to the Master and load settings from AP-Group default. In AP-system LMS was set to 2nd Controller on different Location. Vlans between the Locations are not fully Configured.

     

    The Solution:

    Cleaning up the AP-Group default. So only new AP get this profile with full default Settings (Reset of the AP was nessesary).

    All operational W-Lans get own Groups. So be carefull when you design a Network.

    Cleanig up the mess in the Network is in Progress.

     

    best regards

    Stefan



  • 7.  RE: AP not registering at controller after topology change

    Posted Jul 04, 2017 09:28 AM

    Hello,

     

    I am getting same issue. I have three new access points and two old access points. when I register new access point in master controller then I am getting below alerts from Master controller. New AP will be take IP but not showing in controller rather than I plugin the old AP in same switch port then old AP was working.

    It's reaily amazing. I have not found before this issue.

    Please help on this. I have tried many times.

    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

    (HTAINNOI20XXXWC01) #show log system all | include 70:3a:0e:c8:5d:22
    Jul 4 16:43:52 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 16:49:40 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:04:47 PST 1969; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last Ctrl msg: HELLO len=1038 dest=10.156.3.2 tries=10 seq=0
    Jul 4 16:58:28 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 17:13:30 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:04:34 PST 1969; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last Ctrl msg: HELLO len=1038 dest=10.156.3.2 tries=10 seq=0
    Jul 4 17:17:47 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 17:23:01 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 17:28:36 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:04:34 PST 1969; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last Ctrl msg: HELLO len=1038 dest=10.156.3.2 tries=10 seq=0
    Jul 4 17:34:16 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:04:32 PST 1969; SAPD: Unable to contact switch: HELLO-TIMEOUT. Last Ctrl msg: HELLO len=1038 dest=10.156.3.2 tries=10 seq=0
    Jul 4 17:48:49 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:04:29 PST 1969; DHCP timed out
    Jul 4 17:51:05 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 17:58:36 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 18:03:29 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: No reboot message found.
    Jul 4 18:19:55 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.143.3.36 nanny|  Reboot Reason: AP rebooted Wed Dec 31 16:01:19 PST 1969; failed to get master-ip
    Jul 4 18:41:50 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.33.72.11 nanny|  Reboot Reason: No reboot message found.
    Jul 4 18:44:56 :303022:  <WARN> |AP 70:3a:0e:c8:5d:22@10.33.72.11 nanny|  Reboot Reason: No reboot message found.

     

    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<



  • 8.  RE: AP not registering at controller after topology change

    Posted Jul 04, 2017 09:44 PM

    Hi Deepesh,

     

    As per the logs:

     

    Reboot Reason: AP rebooted Wed Dec 31 16:01:19 PST 1969; failed to get master-ip

     

    How are your AP's finding the controller ?

     

    Is it via DHCP/ADP or DNS mechanism ?

     

    There is a chance that your old AP's might be provisioned with static Master settings but the new ones do not have the same.

     

    Please try one of the below mentioned menthods:

     

    1. You can configure option 43/60 on the DHCP server (on dhcp scope used for AP's) to return the IP of master controller.

     

    2. You can add a entry on DNS-server for aruba-master & point it to IP address of controller. This DNS server is the one which AP's receives during the DHCP process.

     

    3. If you have console access to the AP, interupt the boot sequence & set the following environment variables.

     

    apboot>setenv master <ip-address of controller>

    apboot>save

    apboot>boot



  • 9.  RE: AP not registering at controller after topology change

    Posted Jul 05, 2017 04:56 AM

    Hello,

     

    We are using DHCP option43. we are getting IP.

    Issue is not related to this. My question was that when we are connecting old AP in the switch then working fine but when we are conneting new access point in the same switch (same port) then not working.

    I have already tried with below command but not working.



  • 10.  RE: AP not registering at controller after topology change

    MVP EXPERT
    Posted Jul 05, 2017 05:06 AM

    Have you also specified Option 60 as well? 

     

    http://www.arubanetworks.com/techdocs/ArubaOS_60/UserGuide/DHCP_Option_43.php

     

    There is a chance that the old AP was configured different but still has retained the master IP. Then your new AP has no config and also no Option 60 defined in the DHCP. 



  • 11.  RE: AP not registering at controller after topology change

    Posted Jul 05, 2017 05:08 AM

    Hi Deepesh,

     

    Please open a TAC case so that this can be debugged live.

     

    If possible, attach the AP console logs to the post .