03-10-2013 09:26 PM
4xM3 (2 each in 2 6000 chassis) each chassis is in a separate data center.
1 pair is active as MASTER-BACKUP
1 pair of local controllers are acting as MASTER-BACKUP as well.
This gave us one chassis as a primary and a second chassis acting as our dr backup.
Given our architecture:
Here's a quick note about what I experienced when upgrading from 126.96.36.199 to 188.8.131.52 in our environment.
1) On the first attempt, I upgraded partition 0 on all 4 controllers and rebooted the two backups immediately.
2) Once they came back online, I upgraded the two primary controllers for failover.
3) Once all four controllers came back online, a small subset of APs were offline and never came back after about 40 minutes.
4) I rolled back all four controllers to 184.108.40.206 and all of my APs came back.
5) Since I read the "Known Issues and Limitations" of the release notes and saw that if there is too much traffic on the VRRP VLAN that this could happen.
6) On the second attempt, I set the boot partition back to partition 0 and reloaded three of the four controllers immediately. Once they came back online, I reloaded the remaining controller.
7) This time, all of my APs came back almost immediately.
My lesson learned is that, given the nature of our architecture, by reloading three of the four controllers and waiting for them to come back online I was able to reduce the traffic on the VRRP VLAN so that the APs could get their configs when I reloaded the final controller.
Granted this worked, it was less than seamless and is not a recommended method of upgrade during production hours.
04-10-2013 05:36 AM
Ouch. I wonder, did you remove the backup lms in the AP profile before doing the backups? We are not running VRRP but when I upgrade our 10 M3Ks (9 locals, 1 Master, N+1) I remove the backup lms in all AP profiles then restore when finished. This has worked for us so far..
04-10-2013 11:36 AM
I didn't need to remove the backup LMS as we don't utilize the backup LMS address in our architecture. The LMS address for each AP profile points to one of the two (one being the master pair and the other being the local pair) VRRP address so that no matter which controller is active, it owns that address. You do bring up an interesting idea. Since, using VRRP as the LMS address is essentially the same thing as using a primary and secondary LMS address, I wonder if there wasn't some issue between the controllers about who really "owned" the VRRP address. I'll have to look into that more. Thanks.