Wireless Access

Have a multi controller environment, master with 3 locals running v3.3.2.14 and trying to upgrade to v5.0.4.9.  500 APs are split over the 4 controllers, 

 

Was trying to avoid Aruba's big bang approach to the upgrade, and was hoping to be able to control the upgrade by moving all APs from one of the controllers, upgrade it, then move them back, then repeat.  I first attempted to move AP groups by changing their LMS address so APs would move to another controller,   I then upgraded the controller they moved from,  changed the LMS to move them back, but they seemed to just sit in limbo.  I then attempted to use VRRP to failover the APs to the upgraded controller, but this dint work either, again, the APs just seemd to be in limbo.  

 

Prior to upgrading a controller, I had used both above methods to simply move the APs between controllers, proving that the method should work, at least when all controllers where on same version of code.  The problem seemed to be simply when one of the controllers was on v5, that I couldt get an AP to terminate.  I tried both methods leaving the controller as local, and also breaking the pairing with the Master, neither of which were successful.

 

Im now trying to find out whether either of the above should actually work, although I know Aruba guidelines dont mention using either method, but I had been advised by our support company to use the vrrp approach, and change the role of the controller being upgraded to Master.  

 

The only way we could actually get this to work would be to either reprovision all APs and use the upgraded controller's IP as the master IP address, or to change aruba-master dns entry to point to the new controller and hope that we didnt lose power to any other AP's we werent upgrading!  However, both of these mehtods seem rather crude.

 

It may be that we simply end up doing the big bang approach, I was just curious to know whether either of the above should actually work.

The reason to upgrade everything at once is because, unless you are using a DHCP option to ensure that cold-booted access points will go to their lms-ip and stay there, they will hit the master upon reboot and end up in a upgrade/downgrade loop.  In addition, you cannot make changes to code when the master and the locals are significantly different versions.  A partial upgrade simply opens you up to too many gotchas.

 

 

If you move access points to a controller via lms-ip, if they are ONLY using DNS to discover the master, all access points need to hit the master  upon reboot and will need to be on that same version of code as the master when they cold boot.  Long story short:  upgrade everything at the same time and keep score on the master controller.

 

Remember that this is general information.  You should do your own research to determine what is the best path for you.

 

Here is an upgrade tip:

 

*If you go from one major version of code to another, if the access point messaging has changed between those two versions you will not see the access points on the master or locals indicate that they are upgrading, because the messaging has changed between versions.  You can do "show datapath session table <ip address of access point>" and see the port 21 or FTP sessions that will indicate an individual access point is upgrading.  Looking for port 21 is the most definitive way to see that access points are upgrading.  After that, type "show log system 50" to see the messages that access points send when they are upgraded.*

 

The upgrade/downgrade lopp you mention makes sense, but it did make me wonder why I hadnt seen the AP "upgrading" on either controller, but youve just answered that!

 

Thanks for the advice! 

 

 

Maybe this is along the same path:

 

I've had my master at 5.0.3.3 and approx 70% of my locals at 6.1.3.6.  I was used to seeing the number of controllers on the master being reported as the smaller number being up and larger number being down (different code, so I understand not being able to properly communicate.  Honestly, these are separate locations, and we don't make many changes, so the need to contact the master was only for initial deployment and ease of management).

 

I recently started my push to 6.1.3.8, and  I upgraded my master last night.

 

Out of the 183 controllers that I have, they are at this level:

 

5.0.3.3 = 7

6.1.2.1 = 1 (not sure how this version got out there)

6.1.3.6 = 120

6.1.3.8 = 55

 

My master only "sees" my local controllers that are still on 5.0.3.3.  

 

Is there something I need to do to have the master see the rest of the controllers?  I was under the assumption that a master running an older code would not be able to see controllers with newer code, but that newer code shouldn't have any problem see older versions.

 

 

Again,

The messaging between controllers on different major versions of code is not the same so you will not see that. You also cannot configure your controllers that are not the same version of code with a master that does not recognize it.

Once all controllers are at the same version, should they not begin to communicate and recognize each other?

 

At this point I have 55 locals (620) at the same code as my master, yet my master doesn't see/recognize them.

In practice, they should, but you could have an issue. Based on the state of your network you probably need a plan to move forward/manage as well as a TAC case to determine why they are not talking.

It's been less than 24 hours, and no complaints yet, but I'll keep an eye on them.....maybe it will start working here shortly. 

 

I want to get these last 7-8 controllers up to 6.1.3.8, that way everything will be at 6.1.3.6 or 6.1.3.8. 

 

BUT, in theory they should start talking or syncing back up.  That's all I need to warrant opening a case.

 

No real explanation as to why they aren't talking.  I guess the local is supposed to initiate the IPSec tunnel back to the master, but for some reason they aren't doing it.  The info is still in the config on the controllers, they just aren't syncing back up with each other.

 

The fix is as "simple" as manually re-entering the IPSec key on the local and rebooting.   Everything is still up and functional, so it isn't anything critical.  Given that I used to manually configure and manage 350+ Cisco APs, I'll take having to touch a few controllers any day of the week.