Wireless Access

 

Has anyone had a VRRP failover between their master and masterbackup 72xx controllers that was initiated with the following error?

 

"fpapps[3347]: <399816> <ERRS> |fpapps|  hapiPortLinkStatus: Failed to read phy status on port 0/0/0"

 

Thanks guys,

Fred

 

 

Nope, but quickest way to get an answer would be to open a support case

 

 

Thank you for the null answer Pasguela,

 

Aruba recommends you check with the community first. 

 

If anyone has a productive answer please post.

 

Thanks,

Fred

fhollowa,

 

I suggest you open a support case with our support organization.

We do suggest you check our community first but if this is impacting your service to your "customers" going dirrectly to TAC would be the best route.

 

For gathering more information on certain topics or non impacting issues community is a good route to go.

 

Regards,

Hannes

 

It's not service impacting. I was trying to gather more information.

---

@fhollowa wrote:

 

 

Thank you for the null answer Pasguela,

 

Aruba recommends you check with the community first. 

 

If anyone has a productive answer please post.

 

Thanks,

Fred

---

Here from the syslog message guide:

 

[error:%s]
Description: This is a transient error. This typically occurs if communication between
master/local is lost and a new session established, the controller is awaiting the next
frame in the sequence and receives an unexpected message, such as messages
belonging to a new sequence. This will only happen if the communication failure occurs
as heartbeat message is exchanged, and is usually the result of frame loss between the
involved controllers.

 

At least get my name right ;).

Thanks Pasquale,

 

It's between the master and the masterbackup. It's not between the master and a local. Also connectivity was not lost between the master/masterbackup controllers. The main thing I was looking for was if anyone knew of any related Bug IDs.

 

Fred

 

 

Therefore like I mentioned, opening a support case will give you the best viewpoint on this because if it is a bug they will tell you or if they identified a new bug from your case, they will get it fixed.

 

EDIT: The syslog message guide shows this as the reason and it only mentions master/local but it can apply to master/master

 

 

Okay, First of all did you ever consider that we both opened a TAC case and queried the community? Are you telling me that I am wrong in posting this question on Aruba's discussion forums? Does Aruba not want us as a customer to post questions to this forum or did I in some manner errantly present the question?

 

Secondly, all evidence points to this being an issue with the API failing to receive a response to a status query against the interface.

 

Now please allow us to move forward in trying to gain a better understanding of the error message and the unexpected VRRP failover.

I lay down the law around here… Just Kidding. :-)

 

Hi Fred (fhollowa), you are right, I recommend going to the community first unless the problem is currently impacting users like others have mentioned. Also not everyone has support services… in your defense. In this case, the community is the only place to go.

 

This is the first time I have ever seen this error message and probably so for others. We will try to help you the best we can. If it is an already reported bug, support would be able to alert you faster. At the beginning of the convo users were trying to get you help as quickly as possible because of an expecting outage. This is why pmonardo (one of or MVPs) quickly stated call support. Since he did not know the answer then he figured most of the people on here would not know as well and you would be helped faster with support.

 

Then…. it seems like it escalated from there :-)

 

Let’s try this again. Yo, Airheads Squad! (  @cjoseph @cappalli @SethFiermonti @Michael_Clarke  @tarnold @Anonymous @clembo @Marcus @cdelarosa @Victor Fabian   ) You guys ever seen anything like this?

Sorry Sean/Fred,

 

I have not come across the error before, nor do I see it cross referenced anywhere.   If you do get a response/resolution from TAC, please update the post for the community's sake.

 

Thanks

Not sure what version you are running but hopefull this is easily resolved or is a bug you discovered and can be remedied. You have evidently discovered a unicorn being that there are some impressive minds with loads of expirence in this community and they have never encountered this.

 

Like Clembo said im interested to know the cause of the error, please report back.

how are the two connected? If by a switch does the interface counters show anything bad?

Does this error only occur once or periodically? Is it only happening on one of the two master controllers? Like others have said, I have not seen this before. Are there any other leading or tailing indicators around this error message in the show log all output?

Fred, or those after some details...

 

fpapps[3347]: <399816> <ERRS> |fpapps|  hapiPortLinkStatus: Failed to read phy status on port 0/0/0"

 

this is a generic error, despite what the docs say. Nothing specific to do with the master/master-R or master/local, mostly this error seems to exist for 72xx alarms and a few other minor (non related to this discussion) cases.

 

In this case, hapi refers to the hardware API, and we can infer from the name that some bad juju has happened trying to interogate the port 0/0/0 for its PHY status. As it turns out, this error lives within the port polling section and if this error is seen the port is actually marked as down (I suspect the full logs would show the port going down just after this) - which would cause VRRP to flap

 

The question from Seth is quite relevant - is this happening frequently ?

 

Plus I would wonder does this error appear for other ports ? are there other ports in use that do not have the error (and if not, can a dummy port be added) ? did the port come back up immediately ? What did the device on the far end report ?

 

may turn out to be hardware, might be that the 72xx just incorrectly complained just as the port was transitioning state for an external reason etc.

 

regards

-jeff

 

 

 

Thanks guys,

 

Jgoff, that is great information. I suspected it was the API and was looking to confirm.

 

So far it has only happened one time. The interface stats on the controllers and the Nexus 7010 between the controllers are all clean. The port also has a number of ipsec VPN tunnels which never took a hit.

 

I believe this is all of the relevent logs. I don't think the interface was ever down because we didn't lose any of the ipsec tunnels to the local controllers. We also did not use the ipsec tunnel between master and masterbackup.

 

 

May 28 14:29:36 fpapps[3347]: <399816> <ERRS> |fpapps| hapiPortLinkStatus: Failed to read phy status on port 0/0/0
May 28 14:29:37 fpapps[3347]: <399838> <WARN> |fpapps| Sending to licensemgr vrrid : 102, cur State : INIT
May 28 14:29:37 fpapps[3347]: <399838> <WARN> |fpapps| publish_license_vrrp_state : VRRP State change vrrid : 102, cur State : INIT, licensevrid : 0 , mvrid : 102
May 28 14:29:41 fpapps[3347]: <313328> <WARN> |fpapps| vrrp: vrid "102" - VRRP state transitioned from INIT to BACKUP
May 28 14:29:41 fpapps[3347]: <399838> <WARN> |fpapps| Sending to licensemgr vrrid : 102, cur State : BACKUP
May 28 14:29:41 fpapps[3347]: <399838> <WARN> |fpapps| publish_license_vrrp_state : VRRP State change vrrid : 102, cur State : BACKUP, licensevrid : 0 , mvrid : 102
May 28 14:29:42 KERNEL: 0:<6>ADDRCONF(NETDEV_CHANGE): eth1.2099: link becomes ready
May 28 14:30:11 fpapps[3347]: <313328> <WARN> |fpapps| vrrp: vrid "102" - VRRP state transitioned from BACKUP to MASTER
May 28 14:30:11 fpapps[3347]: <399838> <WARN> |fpapps| Sending to licensemgr vrrid : 102, cur State : MASTER
May 28 14:30:11 fpapps[3347]: <399838> <WARN> |fpapps| publish_license_vrrp_state : VRRP State change vrrid : 102, cur State : MASTER, licensevrid : 0 , mvrid : 102

 

Thanks for the help guys!

 

Correction:

 

We also did not lose the ipsec tunnel between master and masterbackup.