ArubaOS and Controllers

Reply
Occasional Contributor II

Potential VRRP Dropouts

Lately we're noticing dropouts on our network related to VRRP. We have three controllers (diagram attached) - one master and two locals. Each local controller looks after half the APs in the campus with the master acting as VRRP backup for the LMS ip should the local fail.

When the problem occurs our test spectralink phones dropout and reboot. Checking the APs I notice the following event:




The odd thing however is that the local controller for that LMS isn't dropping off or experiencing network issues. Its VRRP stats show it only became vrrp master once:

Virtual Router 1:

Admin State UP, VR State MASTER

Advertisements:
Sent: 1004426 Received: 88
Zero priority sent: 0 Zero priority received: 0
Lower IP address received 88 Lower Priority received 88
Tracking priority overflow: 0
Advertisements received errors:
Interval mismatch 0 Invalid TTL 0
Invalid packet type 0 Authentication failure 0
Invalid auth type 0 Mismatch auth type 0
Invalid VRRP IP address 0 Invalid packet length 0
VRRP Up timestamp: Thu May 14 17:55:26 2009
Master Up timestamp: Thu May 14 17:55:29 2009
Last advertisement sent timestamp: Fri May 29 10:07:42 2009
Last advertisement received timestamp: never
Current time: Fri May 29 10:07:42 2009
Number times became VRRP Master: 1


But the master controller which is acting as backup believes it is becoming vrrp master each time these blips occur:

Virtual Router 1:

Admin State UP, VR State BACKUP

Advertisements:
Sent: 43 Received: 341283
Zero priority sent: 0 Zero priority received: 0
Lower IP address received 0 Lower Priority received 0
Tracking priority overflow: 0
Advertisements received errors:
Interval mismatch 0 Invalid TTL 0
Invalid packet type 0 Authentication failure 0
Invalid auth type 0 Mismatch auth type 0
Invalid VRRP IP address 0 Invalid packet length 0
VRRP Up timestamp: Sun May 24 10:18:45 2009
Master Up timestamp: N/A, not MASTER
Last advertisement sent timestamp: Fri May 29 09:55:52 2009
Last advertisement received timestamp: Fri May 29 09:56:32 2009
Current time: Fri May 29 09:56:33 2009
Number times became VRRP Master: 17


I've looked through all system logs and find no correlation to any events logged. The LAN switches that the controllers are connected to are showing no port events or spanning tree issues. Does anyone have any ideas on how to further troubleshoot this please?

Guru Elite

Logging

Jason,

The first thing you might want to do is configure logging for VRRP. You can do that by doing this on both controllers:

config t
logging level debugging network process vrrpd

You would show the results of the log by doing "show log network x" where x is the last number of lines you want to see. This will tell you why each is flapping.

Question: Has this ever worked? Did it just start happening, or has it been for awhile?

Either way, if you use "show audit-trail" on each controller, you can see the changes you have made and what possibly you can do to roll them back. Also, are you using tracking to increase or decrease the priority of the VRRP on each side? Are you using preemption? If you are, disable these options temporarily so that the VRRP can be more deterministic.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II

Re: Potential VRRP Dropouts

Thanks Colin.

We've got preemption enabled but not tracking. I'm going to configure the logging now (really useful as I didn't know we could log down to processes).

Sorry, forgot to say its only been happening recently but we don't have a definite date. I believe it started occuring around 1 week following an upgrade to v3.3.2.13. It happens, on average, around twice per day. Times don't correlate.

Jason
Aruba Employee

Potential VRRP Dropouts

Do you have AP-125s? If so, there is a bug that was fixed in 3.3.2.15
that would cause this type of behavior. I don't have the bug ID in
front of me, but if you have AP-125s, I bet that is the issue.
Occasional Contributor II

Re: Potential VRRP Dropouts

We have all AP 65s but thanks anyway.
Guru Elite

Maybe....

Jason,

This is out there, but I was speaking to an esteemed colleague this week about another customer who was having an issue like this. He was running VRRP on a different wired network as well, but a WIRELESS client was bridging his wireless AND wired interfaces together, and the VRRP advertisements were being bridged between the Aruba management network and the wired network that the laptop was bridging through. Since the customer was using the same VRRP instance for both networks, the network on the wired laptop advertised its increased priority and took control of the VRRP....blackholing traffic. You can avoid this two ways:

- Use Authentication your VRRP instances, so nothing can take control of it
- put the ACL "any any 112 deny" in your user role to drop VRRP traffic so that it does not get bridged (protocol 112 is VRRP)

Good Luck.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Aruba Employee

Potential VRRP Dropouts

Ok, there was an issue just identified with VRRP at one of my customers.
Development has fixed the issue and will include the fix in an upcoming
code release. A temporary work-around is to set the LMS and backup LMS
IPs to the non-VIP addresses of the primary and backup controllers.
Search Airheads
cancel
Showing results for 
Search instead for 
Did you mean: