Wireless Access

Hi Alls.

I need your help. One of my Controller in the Cluster is having some issues and we are going to replace it with the new controller. We got an RMA from Aruba. I would like to know the Process and Step by step procedure on how to replace the controller in the cluster without any user impact. I have been looking everywhere and could not able to get the proper info.

Please assist.

Regards

KK

How I would do it? 

Add the new controller to the existing cluster.

Access Points will take some time to rebalance.

Remove old controller.

There WILL be user impact (slowness, etc) due to the fact that access points and users will be re-balanced between controllers, so I would declare an outage of an hour just to be 100%

Thanks for your quick reply.

We don't want to add a new controller in the exiting cluster as we have to allocate new ports and IPs to the new controller. instead of that, We want to replace the faulty controller with the new one.

There is not a way to replace one controller with another with zero user impact.  At minimum there will always be some slowness due to some churn, due to removing and adding a controller and load balancing of users and access points.

With that being said, replacing a controller in an RMA situation has not been documented, but is very involved.  If you have not installed a controller before and you are not handy with the commandline, I would get Technical Support on the phone to assist.

If you still want the information, I can paste it in here, but the complexity could be higher depending on how things are designed...

Thanks for your update.

We are doing it for first time.

Could you please share your rough plan.

Regards

KK

@cjoseph 

I am also curious for a rough overview of any pitfalls.

Not had to replace a cluster controller yet, but I figured it would be relatively simple as all the configuration is kept with the MM controller.

I was thinking a restore of the faulty controller onto its replacement wouldn't even be needed anymore. 

The way I envisioned this:

1) shut down faulty controller if it is still running

2) remove the controller from the cluster

3) add the new controller with identical ip and cluster settings

4) the end

Am I missing some complexity here? 

We do use the Controller MAC address to add a Controller to MM. So if we just change the IP and add it to the MM, It won't work due to MAC mismatch. So I believe we have to remove the Faulty device from the MM first (no configuration device XX.XX.XX.XX.XX.XX) and then add a new Device with the same IP & New MAC(configuration device XX.XX.XX.XX.XX.XX) under appropriate group.

How do we remove the controller from the cluster?

"no lc-cluster group-membership" Does this command works or we have to remove the "no lc-cluster group-profile" as well? 

How do we take the config back of the Faulty device alone from the MM?

I know there is an option in MM to take configuration backup or flash backup, But this will take the backup of all the MD in the MM.

- First, take a backup of the device-specific information from the MD (controller that you are replacing); on the MM: CD to the name or the full path of that controller.  Type "show configuration committed" to show the device-specific config.  Copy that to a text file.  In the text file you just copied, separate out the MD-specific cluster configuration, because we will be pasting that in separately from the config:

lc-cluster group-membership <cluster-name>
lc-cluster exluded vlans <list>

- Unplug the old controller

- While still in the old controller's folder on the MD, remove the cluster-specific info by typing:

no lc-cluster group-membership
write mem

- Move up to level of the cluster profile by typing cd .. and then remove the old controller's cluster information from the cluster by typing:

config t
lc-cluster group-profile <cluster-name>
no controller <old MD’s ip address>
write mem

 - Delete the old controller either through the CLI or the GUI:

no configuration device <mac address of old device>

- Add the replacement MD using the full console startup script with PSK and IP.  Let the MD reboot.

- Create the device:  On the MM, keep typing "show switches debug".  Eventually the new MD with model number and mac address will show up with a "UNK" designation, because we have not created a device for it yet.  Take that opportunity to create a new device  at the correct level using the mac address and the model number from "show switches debug":

configuration device <mac address> device-model <device model> /md/<md-config-folder>

- CD to the cluster level and add the new device to the cluster group profile:

cd <full path of the cluster level>
lc-cluster group-profile <cluster-name>
controller priority <lower Priority than current Leader> <MD IP> 
wr mem

***2023-12-26 --- I moved the section above below, because I had the order wrong.  You need to browse to the MD's folder on the GUI and add it to the cluster, first, OR CD to the full path of the MD and add it to the cluster, before adding the MD to the cluster level******
 
- CD to the full path of the new MD and paste the configuration saved in the text file (except for the  cluster definition we separated out) into the new device:

cd <full path of new device>

config t

<paste in text file>

- The MD will reboot

- While it is rebooting, paste in the MD-specific cluster info:

lc-cluster group-membership <cluster-name>
lc-cluster exluded vlans <list>

- CD to the cluster level and add the new device to the cluster group profile:

cd <full path of the cluster level>
lc-cluster group-profile <cluster-name>
controller priority <lower Priority than current Leader> <MD IP> 
wr mem

 

That should be it.  There are some gotchas and quirkiness but if you are uncomfortable, you should put in a call to TAC to have them lead you through it.

Thanks for your detailed plan. It's very useful 

I have one query, When do we have to unplug the faulty controller?

After we remove the Cluster config's and Push it to the controller or before it?

Regards

AK

 If you don't need it for active APS, you can unplug it at any time.

EDIT: If you are replacing it with the new controller, it must be removed/deleted before you provision the new controller.

One gotcha is that you need to be able to reconcile any errors that you might have when pasting in the configuration for the new controller.  If anything errors out, you need to find out why and resolve it.  There are some commands that are sensitive to order and need to be pasted in a specific way (e.g. VLAN 1 needs to be created before assigning it to a port).  Ideally, your controller-specific commands are minimal and you will not have to do much reconciliation. 

...That is exactly why I am saying it is much easier to introduce another controller with a new ip address live and then remove the other controller at your convenience....  Introducing an additional controller also allows you to be able to back out at any time and go back to what you were doing before...

---

@koen wrote:

@cjoseph 

I am also curious for a rough overview of any pitfalls.

 

Not had to replace a cluster controller yet, but I figured it would be relatively simple as all the configuration is kept with the MM controller.

I was thinking a restore of the faulty controller onto its replacement wouldn't even be needed anymore. 

 

The way I envisioned this:

1) shut down faulty controller if it is still running

2) remove the controller from the cluster

3) add the new controller with identical ip and cluster settings

4) the end

 

Am I missing some complexity here? 

---

Yes, there is some nuanced complexity.