Wireless Access

Hi.

Currently I have 2 controllers set for Master Redundancy (active/standby) and Fast Failover (both set as dual) at the same time. In AP profile LMS IP has IP address of active controller and backup LMS IP has IP address of standby controller.

The problem is when I break communication between APs and both controllers for a moment (controllers notice that APs are down) and then restore communication, APs will not reconnect to any controller. I have to reboot APs.

Similar issue is when active controller is down and standby becomes new active then when APs are booted, they will not connect to new active controller.

Is it normal behavior in this setup?

Also can I move some APs (make them active) on standby controller? I think the answer is no but I want to make sure.

If you are not using locals and all of your APs are terminating on masters, you should not use these two features together.

Couple of weeks ago I upgraded controllers to 6.4 to have this option supported and it works well except these issues.

I haven't seen any document stating it's not recommended. Can you provide any details? What setup would you recommend?

Hi,

 

You can find the below note @ http://www.arubanetworks.com/techdocs/ArubaOS_63_Web_Help/Content/ArubaFrameStyles/VRRP/HighAvFastFailover.htm

 

===

High Availability:Fast Failover provides redundancy for APs, but not for controllers. Deployments that require master controller redundancy should continue to use an existing VRRP redundancy solution.

===

 

Rajaguru Vincent

I know that note but for me it means that Fast Failover can be used for AP redundancy and VRRP can be used for controller redundancy. For me these are two seperate solutions but one doesn't exclude the second one.

Anyway, what solution would you recommend?

The only issue I see is if APs build a second stand by tunnel but because the controller is part of the stand by master redundancy it won't build it since the standby master by default can't build a any tunnels with APs until
It becomes the primary

I think the tunnels are established because 'show ap standby' on standby controller displays all APs and during failover APs become active very quickly (without it I think it would take much longer).

I thought you mentioned that APs were not able to reconnect when standby became active

Hi,

 

Remove backup lms IP from AP system profile. Just have LMS IP pointed to Master controller's physical IP address.

 

Make sure the ip-addresses you are using in HA config are controller/switch IPs and not just any other interface IPs. 

 

 

Here is my config for Master-Standby Master HA setup:

 

• Master (10.10.1.1)

vlan 10
!
interface vlan 10
  ip address 10.10.1.1 255.255.255.0
!
controller-ip vlan 10
!
vrrp 10
  ip address 10.10.1.3
  vlan 10
  no shutdown
  priority 110
 !
master-redundancy
  master-vrrp 10
  peer-ip-address 10.10.1.2 ipsec <key>
!
Ha group-profile Building-A							
  controller 10.10.1.1 role dual
  controller 10.10.1.2 role dual
  state-sync
  pre-shared-key <key>
!
Ha group-membership Building-A
!
ap system-profile Building-A
  lms-ip 10.10.1.1
!

• Standby Master (10.10.1.2)

vlan 10
!
interface vlan 10
  Ip address 10.10.1.2 255.255.255.0
!
controller-ip vlan 10
!
vrrp 10
  ip address 10.10.1.3
  vlan 10
  no shutdown
  priority 110
 !
master-redundancy
  master-vrrp 10
  peer-ip-address 10.10.1.1 ipsec <key>
!
Ha group-membership Building-A
!

Thanks. I'll try that configuration.

Did this suggestion work? We're trying this exact same configuration and are not able to get it to work. It seems as though the HA Heartbeats aren't making it between the Master Redundancy pair or something.

 

On the primary master the heartbeat counters are all zero

 

(LCPS-7220-1) #show ha heartbeat counters

Heartbeat stats
---------------
Controller IP Active Reference Count Total Heartbeat Sent Total Heartbeat Received Last Missed Heartbeat (Count) Time
------------- ---------------------- -------------------- ------------------------ ----------------------------------
10.32.31.247 0 0 0 0

 

On my backup master the heartbeat counters are all zero.

 

(LCPS-7220-2) #show ha heartbeat counters

Heartbeat stats
---------------
Controller IP Active Reference Count Total Heartbeat Sent Total Heartbeat Received Last Missed Heartbeat (Count) Time
------------- ---------------------- -------------------- ------------------------ ----------------------------------
10.32.31.246 0 0 0 0

 

I've also removed the backup LMS IP address from the AP system profile and only have the LMS IP address defined as suggest in this article and still no go.

 

Not sure what we're missing???

 

What version of ArubaOS and what controller platforms are you using?

 

We are running:

 

(LCPS-7220-2) #show image ver
----------------------------------
Partition : 0:0 (/dev/usb/flash1) **Default boot**
Software Version : ArubaOS 6.4.2.8 (Digitally Signed - Production Build)
Build number : 50314
Label : 50314
Built on : Wed Jun 3 12:21:36 PDT 2015

 

I kept digging and saw another post that AP Fast Failover is NOT possible when also configuring redundant masters. It was suggested if you want/need redundant masters we should go back to using VRRP and directing aruba-master to the VRRP VIP address.

It (master/backup master) is supported on 6.4 on onwards.  If you have redundant masters, just pointing APs at the VRRP is more straightforward.

 

If you want to do it (HA/fast failover on master/backup master), this is how you would do it.  In this example, 10.10.1.1 is the master and 10.10.1.2 is the backup master.  These are the controller-ips, and not VRRP or loopback addresses.

 

On the Master:

ap system-profile "primary"
 lms-ip 10.10.1.1
 bkup-lms 10.10.1.2
!
ha group-profile "Cluster-A"
 preshared key presharedkey
 state-sync
 controller 10.10.1.1 role dual
 controller 10.10.1.2 role dual
!
ap-group "Cluster-A"
 ap system-profile "primary"
!
ha group-membership Cluster-A

On the Backup Master:

ha group-membership Cluster-A

backup-lms is required in case of controller failure and AP reboot.

 

NOTE:  Inter-Controller heartbeat is not needed/supported in master/backup master deployment.  The APs should be pointed at the VRRP for master discovery (aruba-master, DHCP option)

 

If you are terminating APs on a master/backup master pair, it is much more straightforward just to point the APs at the VRRP using your discovery method and in the LMS-IP.

 

 

Hi Colin;

 

>If you are terminating APs on a master/backup master pair, it is much more straightforward

>just to point the APs at the VRRP using your discovery method and in the LMS-IP.

 

My understanding is that pointing the APs lms-ip address at the VRRP would not give fast failover.  Am I missing something?  

 

I'm just getting ready to make the same migration from VRRP master/standby to a dual/dual HA group with a pair of 7040 controllers.  Currenly we do AP discovery with the VRRP address and the lms-ip is unset.

 

Andrew

Andrew,

 

You can either do VRRP-based failover or use the instructions above to do Fast Failover with Master/Backup Master.  It is your choice.

I noticed your configuration put the controllers in the dual role.  Will the APs still load balance (as in the "active/active deployment mode" in the docs) with the controllers in a master/backup master configuration?  Or does setting the backup lms-ip override this feature? 

 

 

The APs do not load balance.  The only reason for making both controllers DUAL is so that they can assume any role.  Again, the real strength of HA is realized in a master/local situation or an N+1 oversubscription situation.  If you have a master/backup master it is easier to configure vrrp-based redundancy which has never been too shabby in the failover department.

 

With regards to the fast failover configuration, the backup-lms is only to guard against if the master fails and the APs also reboot in a master/backup master scenario.  Again, it is more straightforward to configure VRRP-based redundancy in a master/backup master scenario.

Thanks for claifying that Colin. That darn salesguy told me FFO was the best thing since sliced bread though. :)

 

So the choices are;

1. master redundancy and VRRP AP failover (simplest)
2. master redundancy and N+1 AP fast failover ("hitless" failover)
3. master/local and N:1 AP fast failover with load balancing (most complex)

Option 3 appeals to me because I've been burnt before by standby systems that weren't connected correctly and then didn't work during failover. With load balancing you always know if both units are operational. But to get that I'd have to give up all of the other benefits of controller redundancy.

 

I think I'll go check with the magic 8-ball.

 

 

Andrew Bell,

 

You should only do #1, because that is what the manual says to do if you are using master redundancy deployment model here:  http://www.arubanetworks.com/techdocs/ArubaOS_64x_WebHelp/Web_Help_Index.htm#ArubaFrameStyles/VRRP/HighAvOverView.htm

 

Fast Failover/HA is designed for more complex deployments than the one you are deploying.  There are some people that need to deploy master/ local and don't have the option for master/redundancy either because they cannot co-locate two controllers in the same datacenter or other reasons.  Those people should do Fast Failover/HA.

 

VRRP based failover has existed for a number of years and it works very well, if configured properly.  Fortunately, it is also easy to configure.

I hope its ok that I reply in this old thread?

 

I've tried the setup with master/standby configured in one HA group. This works without the HA heartbeats (those are not there as already mentioned).

 

However, there is one thing that doesn't work in my setup:

The aruba-master DNS entry points to the VRRP address. LMS 1 points to the master physical address, LMS 2 to the standby.

When an AP boots and the master is up, all works fine. When the AP boots while the master is down I would expect the AP to try the 2nd LMS (which is now the master) after 3 failed attempts to reach LMS 1. What happens is that the AP builds its GRE tunnels to the VRRP address, but the IPSEC tunnel to LMS 2. The SSIDs are not coming up at that point.

Rebooting the AP with a reachable master (LMS1) is the only way that I can make it work. Any ideas?

 

Thx

Peter

What version of ArubaOS is this, a 6.3 ot 6.4 variant?

 

To be honest, if you have just two controllers, a master and backup master, just VRRP redundancy (instead of FF) might be enough.

Hi Colin

 

Yes I know for the VRRP solution. Although I must say that the fast-failover does work well in this setup.

I'm testing this in my lab with two controllers (preparing for ACMX). I wouldn't suggest this setup in 'real life'. I have however seen this setup in production environments, with active/standby HA roles for the controllers.

The OS version on the controllers is 6.4.4.6.

 

Best regards

Peter

I faced the same issue in the past but I believe I used the same configuration that you stated.

 

I believe I resolved it by adding the bklms-ip of the secondary master (which was missing prior). You have already have that configured.

 

I will lab this up tonight and test because now I am curious.

I would check all of the points that HLavender recommended.  Not having the Backup LMS pointing to controller ip of the backup could create that issue.

Hi

Yes, both LMS IPs are configured. I've found the culprit however. It was the HA role that was set to standby for the standby master. A booting AP could never terminate on this controller.

Changing the roles to 'dual' solved the issue.

Just to be complete: here is what I saw with those active / standby roles:

As you can see, after booting the AP build it's IPSec tunnel with the standby (now active) master .202 but the GRE tunnel is set to the vrrp address address (because standby won't accept the AP).

 

Br

Peter

 

This is possible since 6.4. I seemed to have heard a lot, even from TAC, about FF not being supported. But The documents say this works, even a VRD for it. Aps say they are in standby on the controller, with 0 heartbeart counters. When I rebooted, the APs did not rebootstrap like they did with traditional VRRP failover. At this point VRRP is only useful for management and DNS discovery.

 

https://ase.arubanetworks.com/solutions/id/53

With fast failover you should not be using VRRP for your APs - the "local" part of being a master-local. Use LMS of the primary controller and backup LMS of the secondary (although the backup setting may not even be necessary - but it doesn't hurt).

 

You can still use it for the "master" part.

I don't think the thread was about Master/Local, it was about Master/Standby using Master Redundnacy with no locals. What your saying makes sense, but for a master/local deployment. This is how I understood it.