Wireless Access

last person joined: yesterday 

Access network design for branch, remote, outdoor, and campus locations with HPE Aruba Networking access points and mobility controllers.
Expand all | Collapse all

Master unreachable in standby controller after failover happens

This thread has been viewed 8 times
  • 1.  Master unreachable in standby controller after failover happens

    Posted Dec 16, 2014 11:16 AM

    Hi

     

    Last week I configured 2 controllers in a master-standby configuration. It took a lot of time because the standby kept saying master is unreachable, i decided to delete redundancy, kept VRRP, reboot the cotrollers and check ping between them after every step until finally worked.

     

    Redundancy works and the APs switched to the other controller, the problem is that when the failed controller comes back online the "master unreachable" error appears again. I did tests and failover still works even tough they don't see each other but the problem is that they are not syncing configuration during that time.

     

    To make the active master see the standby controller I had to delete redundancy in the stanby (but keeping VRRP), reboot the controller and create redundancy again. I don't know if this has something to do with the fact that I deactivated preemption in the VRRP and that I didn't cretead an HA group (i don't even know what's that for).

     

    I appreciate your help with this because I have to solve that issue as quickly as possible.



  • 2.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 16, 2014 12:13 PM

    Hi,

     

    Surely I can help you on this.

     

    What is the AOS version and model of those controllers ?

     

    Please feel free to come back on this



  • 3.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 16, 2014 12:18 PM

    Hi, thanks for your answer

     

    We have a 3600 and a 7210 both running AOS 6.3.1.5.

     

    Thanks for your help.


    #7210


  • 4.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 16, 2014 01:15 PM

    Hi,

     

    Please share the output of "Show log security 30", "Show arp" of both the controllers after replicating the issue. it will tell us why the standby is not able to reach the master.

     

    Mean time I will replicate your setup and see whether it is any known bug or not.

     

     

     



  • 5.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 16, 2014 03:26 PM

    Hi

     

    The problem is the solution is installed at a hospital where wifi is used for their clinical information systems so I can't get permission to try the failover if there's risk it won´t work properly. If i pull out those logs from the backup controller which hasn't been touched since the redundancy was created, will it be useful?

     

    I appreciate your help trying my setup. I used the following parameters for the redundancy configuration:

     

    3600 CONTROLLER

    virtual router id: 10
    description: primary-master
    ip address: 192.168.254.178
    enable router pre-emption: no
    priority: 110
    admin state: up
    vlan: 1
    tracking master up time: 30
    tracking master up time priority: 20

    ------------------------------------

    Enable periodic database synchronization: yes
    Database synchronization period in minutes: 60
    Master redundancy
    Master VRRP: 10
    Peer's ip address: 192.168.254.179

     

    7210 CONTROLLER (active master at the moment after trying the failover)

    virtual router id: 10
    description: backup-master
    ip address: 192.168.254.178
    enable router pre-emption: no
    priority: 100
    admin state: up
    vlan: 1
    tracking master up time: 30
    tracking master up time priority: 20

    ------------------------------------

    Enable periodic database synchronization: yes
    Database synchronization period in minutes: 60
    Master redundancy
    Master VRRP: 10
    Peer's ip address: 192.168.254.180

     

    Aditionally centralized licensing is enabled.

     

    Thanks for your help.


    #7210


  • 6.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 17, 2014 04:30 AM

    Hi Friend,

     

    Your issue is looking like a bug found in 6.3.1.3 with bug #98005.

     

    can you confirm couple of things here,

     

    1. any of those controllers having controller based licenses( permanent)

    2. can you tell what isthe msg you will get when you use "Show Switches" command ,

    "update required" or "update terminated" in master and "master unreachable"

     

    Any way let me replicate the issue and come back to you.

     

    Please feel free for any further assistance on this.



  • 7.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 18, 2014 09:30 AM
      |   view attached

    Hi dhanraj

     

    Today the backup controller showed an alert saying that licenses received from centralized licensing will expire in 26 days (image attached). Does that mean that redundancy stopped working? Additionally they supposedly say that they lose administration of the controller because of that but I believe that's not related

     

    The 7210 doesn't have permanent licenses, i'm not completely sure about the 3600 but i'll check it as soon as possible because with this new issue i have to go visit the client.

     

     

    jgoff

     

    We don't have loopback IPs configured, they area seeing each other through the IPs in VLAN 1 and now that you mention it, I think nat is configured in that VLAN; i'll have to check that and disable it.

     

    Thanks both for your help.


    #7210


  • 8.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 18, 2014 12:34 PM

    Hi,

     

    It looks like there is no communication between Master and Standby. if possible share the output of "Show Switches", " show master-configpending", "show database synchronize" so that I can understand few points about your issue.

     

    I believe this issue will be fixed by code upgrading. If possible please open a ticket with Aruba TAC.

     

    Please feel free for any further help on this.



  • 9.  RE: Master unreachable in standby controller after failover happens

    EMPLOYEE
    Posted Dec 17, 2014 11:07 AM

    Darthjp,

    some other things to ensure for happy master/local or master/master-redundant operation.

     

    Don't mix interface IPs and switch IPs - if your controllers have loopback IPs configured, use them for all switch IPs, don't mix interface on one side and loopback on the other side.

     

    Ensure ip nat inside is _not_ enabled in vlan1, even if you are not using vlan 1 for mgmt.

     

    Take a step back and check if the ipsec tunnel is up between the two. Usual symptom of busted ipsec is (assuming loopback IPs are in use) that you cannot ping the other controllers loopback IP - but you can ping its interface IP. This is due to the ispec route-map  which is a sink if the ipsec is not up.

     

    Common causes of ipsec not up.... incorrect shared key, accidentally duplicated IP on network. ip nat inside and occasionally quirky stuff if you mix and match interface IPs with loopback IPs.

     

    regards

    -jeff



  • 10.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 18, 2014 05:21 AM

    Hmmm... is even master redundancy with different controller models supported?



  • 11.  RE: Master unreachable in standby controller after failover happens

    Posted Dec 18, 2014 05:40 AM

    Hi,

     

    Yes it is possible to configure Master redundancy with different models even with different image ( AOS) versions but it is not recomended to configure with different AOS.

     

    Please feel free to comeback for any further help on this.