ArubaOS and Controllers

Reply
Occasional Contributor II
Posts: 10
Registered: ‎08-13-2009

Layer 2 Up, Layer 3 down on our Controllers

This morning, we came in and both our Primary and Secondary controllers appeared to be up and running, but when we tried to log into them, we couldn't connect with the web interface or via ssh. We pinged each controller individually and the vrrp address and got no response from any of them. The APs were all up and broadcasting our SSIDs, and we could connect to them, but they weren't passing any traffic to the router. We looked at the interface on the controller through the console and it looked like everything was up, no errors, everything was clean, but we couldn't ping anything. Same thing when we logged into the router, the port was clean and it looked as if it was linked but we couldn't ping it (see console output below). The weird thing was ARP requests were working, so we had communication at layer 2, just not layer 3. It was as if the TCP stack got munged up. We initially hadn't gotten any reports that wireless was down, so we were about to call aruba support, but then started getting calls, so we ended up just rebooting the controllers and everything came back up and is running fine now. I know from experience that support has a much more difficult time helping you once you reboot the controller and lose the logs, and it's working now, so I figured I'd just ask if this is something anyone has seen before and if so, if they know what might have caused it.

Thanks in advance.

Here's what we saw on the Cisco side:


cc_core#sh ip arp 172.30.161.1
Protocol Address Age (min) Hardware Addr Type Interface
Internet 172.30.161.1 0 000b.860b.af80 ARPA Vlan602
cc_core#


cc_core#clear arp-cache
cc_core#

cc_core#ping 172.30.161.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.30.161.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
cc_core#sh ip arp 172.30.161.1
Protocol Address Age (min) Hardware Addr Type Interface
Internet 172.30.161.1 0 000b.860b.af80 ARPA Vlan602
cc_core#


172.30.161.1 is the address of our primary controller, our vrrp address is .3.

Stephie
Aruba Employee
Posts: 455
Registered: ‎04-02-2007

Re: Layer 2 Up, Layer 3 down on our Controllers

It's hard to do once you reboot, did you happen to run a show tech support prior to the reboot?

-awl
Andy Logan, ACDX
Director, Strategic Account Solutions
Aruba Networks
Occasional Contributor II
Posts: 10
Registered: ‎08-13-2009

Re: Layer 2 Up, Layer 3 down on our Controllers

No, I unfortunately didn't get a chance to do that before someone else in the group rebooted the controllers, which has happened before and I know it's almost impossible to trouble shoot after that, so I'm pretty much shooting in the dark here.
Aruba Employee
Posts: 455
Registered: ‎04-02-2007

Re: Layer 2 Up, Layer 3 down on our Controllers

it would be good if folks started running that be default when they hit a snag, any chance you can make that part of the policy? I know it's easier just to reboot everything and get things going again, but as you said it makes it very difficult to find out why it happened.

thanks,
-awl
Andy Logan, ACDX
Director, Strategic Account Solutions
Aruba Networks
Occasional Contributor II
Posts: 10
Registered: ‎08-13-2009

Re: Layer 2 Up, Layer 3 down on our Controllers

Hehe, it was actually my boss that did it and when he told me, the first thing I said was "did you save the logs"? And I lectured him on it again this morning, that's all I can do, he outranks me :)
New Contributor
Posts: 3
Registered: ‎02-01-2010

Re: Layer 2 Up, Layer 3 down on our Controllers

Did you have default gateway setting on your controller?
New Contributor
Posts: 2
Registered: ‎02-08-2011

Re: Layer 2 Up, Layer 3 down on our Controllers

All, this is happening almost every second night on our master controller, now that i know what to grab i will grab these logs and send them off. We are running ArubaOS 6.0.0.1 on a 3200
Guru Elite
Posts: 20,559
Registered: ‎03-29-2007

Re: Layer 2 Up, Layer 3 down on our Controllers

Please describe your deployment and how it is connected to your infrastructure. Please also mention what happens when you have your issue.

If you want a quicker resolution, please open up a case with technical support.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

New Contributor
Posts: 2
Registered: ‎02-08-2011

Re: Layer 2 Up, Layer 3 down on our Controllers


Please describe your deployment and how it is connected to your infrastructure. Please also mention what happens when you have your issue.

If you want a quicker resolution, please open up a case with technical support.




So the issue for us was the 3200 controller was found to have faulty power internals. once we replaced the controller this issue no longer occured
Search Airheads
Showing results for 
Search instead for 
Did you mean: