Question- What happens if the standby license server reboots when the license server is down?
Environment Information- This article applies to controller running 6.3.1.3 and 6.4.0.1 above with centralized licensing feature enabled
There was a limitation using centralized licensing feature when the primary server went down. If standby license server rebooted and comes back while the license server is down, all license contributed by the license server will be removed on standby license server and license clients
Symptoms- Symptom: The licenses on a standby master controller causing the configuration on the local controller to be lost. Caching the master controller's license limits on the standby controller for a maximum of 30 days resolved this issue.
This issue occurred when the standby comes up before the master after a reboot. This may also occur in an all master-local topology when running ArubaOS 6.3 or later.
Cause- Scenario
=======
Bring Master controller down. Standby master will take over the licensing part.
Reboot standby master as well. After standby master comes up license count on it will be zero.
Check the config on local controller after standby master is backup. Because of No licenses on standby master,
config on the local controller /ACLs are erased.
This can happen at customer sites, during the upgrade process. All the controller will be rebooted, but
in any case Master doesn't come up, then licenses on the standby master are gone, which can cause outage.
Resolution- Workarounds:
===========
1. Add just 1 PEF and AP and RFP(if applicable) license on standby controller. This will avoid config from getting erased.
However as a downside only one AP can be up on the network after the standby reboots after a failover.
2. Add eval licenses on the backup controller.
Solution:
========
The standby license server will cache the limits installed on the master controller on flash. These limits will be used when the standby license server becomes the license server (when the master is unreachable/down). The cache will be available for 30 days after the standby license server becomes the license server. After 30 days, the cache will be removed from flash.
The standby controller will then advertise 0 license limits but the features will not be disabled. If an AP is UP it will not be brought down. If the standby controller reboots after the 30 days are up it will come up with 0 limits after reboot. The features will not be disabled after a reboot.
During these 30 days if the standby license server reboots, it will still use the cached limits after reboot. If a local controller reboots before the 30 days are up, the standby license server will be able to send licenses to it.
Benefits
========
If the master controller is not restored within 30 days
1.The config will NOT be lost since the features will be persistent.
2.After 30 days no new APs will be able to come up.
3.Any AP that is UP on the local controllers will continue to stay UP (with the acls enabled).
4.The features will be persistent forever. This mainly applies to PEFNG and will ensure that config is not lost.
Related Links- https://arubanetworkskb.secure.force.com/pkb/articles/FAQ/How-does-license-server-failover-work-in-AOS-version-6-3
https://arubanetworkskb.secure.force.com/pkb/articles/HowTo/What-is-License-Server-Redundancy-and-how-to-configure-it-on-Aruba-controller
Answer- Test Topology:
============
master-standby-locals, with all licenses only installed on master controller/license server
3400 controller set up as Master-Standby
Code: 6.3.1.13
- Installed 10 AP and PEF license on Master controller and no license on Standby
- Also created a user role TEST
Commands to debug
=================
(Aruba) #show license aggregate
(Aruba) #show license debug
(Aruba) #show license server-table
(Aruba) #show license client-table
We no longer have this constraint. So while the master licensing server is down and if the standby license server happens to reboot, we see the licenses on the standby controller. This enhancement was part of bug fix 87424 which is fixed in 6.3 and 6.4 code versions.
Symptom: The licenses on a standby master controller causing the configuration on the local controller to be lost. Caching the master controller's license limits on the standby controller for a maximum of 30 days resolved this issue.
Scenario: This issue occurred when the standby comes up before the master after a reboot. This may also occur in an all master-local topology when running ArubaOS 6.3 or later.
On Master controller
================
master) (config) #show switches
All Switches
------------
IP Address Name Location Type Model Version Status Configuration State Config Sync Time (sec) Config ID
---------- ---- -------- ---- ----- ------- ------ ------------------- ---------------------- ---------
172.16.0.254 master Building1.floor1 master Aruba3400 6.3.1.9_44832 up UPDATE SUCCESSFUL 0 3
172.16.0.30 standby Building1.floor1 standby Aruba3600 6.3.1.13_46575 up UPDATE SUCCESSFUL 17 3
(master) (config) #show license aggregate
Aggregate License Table
-----------------------
Hostname IP Address AP PEF RF Protect xSec Module ACR Last update (secs. ago)
-------- ---------- --- --- ---------- ----------- --- -----------------------
master 172.16.0.254 10 10 0 0 0 26
standby 172.16.0.30 0 0 0 0 0 4
Total AP License Count :10
Total PEF License Count :10
Total RF Protect License Count :0
Total XSEC License Count :0
Total ACR License Count :0
On Standby controller:
=================
show license verbose
License Table
-------------
Key Installed Expires Flags Service Type
--- --------- ------- ----- ------------
License Entries: 0
Standby Controller after reboot
========================
show switches
All Switches
------------
IP Address Name Location Type Model Version Status Configuration State Config Sync Time (sec) Config ID
---------- ---- -------- ---- ----- ------- ------ ------------------- ---------------------- ---------
172.16.0.30 standby Building1.floor1 master Aruba3600 6.3.1.13_46575 up UPDATE SUCCESSFUL 0 1
(standby) #show license aggregate
Aggregate License Table
-----------------------
Hostname IP Address AP PEF RF Protect xSec Module ACR Last update (secs. ago)
-------- ---------- --- --- ---------- ----------- --- -----------------------
172.16.0.254 10 10 0 0 0 11
standby 172.16.0.30 0 0 0 0 0 9
Total AP License Count :10
Total PEF License Count :10
Total RF Protect License Count :0
Total XSEC License Count :0
Total ACR License Count :0
(standby) #show license server-table
License Server Table
--------------------
Service Type Aggregate Lic. Used Lic. Remaining Lic.
------------ -------------- --------- --------------
Access Points 10 0 10
Next Generation Policy Enforcement Firewall Module 10 0 10
RF Protect 0 0 0
xSec Module 0 0 0
Advanced Cryptography 0 0 0