Remote Networking

Reply
Occasional Contributor II
Posts: 61
Registered: ‎08-12-2009

Remote office AP-65 resets wireless every couple seconds

Good afternoon all

We have a 3600 Controller and AP-65s through out our district in 8 different location. We don't have any branch controllers in place at this point. This morning one of one of our locations starting having an issue we the APs A and B/G light started resetting themselves. The power and ethernet lights stayed on and I confirmed through the console by going to Monitoring - Access Point - select one of the effected APs - selecting AP Activity that the wireless connection was resetting every 10 seconds or so. When it was connected it gave stats but when not available it said AP not available and the bar graph would drop to zero. I checked that the APs had a valid IP address by looking my DHCP scope finding the ip address for one of the APs and doing a continuous ping and it never dropped. I also went through the console and to make sure the AP was getting a configuration by going to Diagnostics - Received Configuration - giving the APs IP address and it gave me the config info it had. I would make a little change somewhere then check it again and the AP had teh change so I know it was getting the config. Now that everyone except for a hand full of teacher has left the APs are staying up and teh coupel that are there are connected. When we are having an issue NO ONE can connect.

Has anyone seen this issue in the past and can share any insight?

Thanks
Ed
Guru Elite
Posts: 20,775
Registered: ‎03-29-2007

Not Specifically

If you type "show log system 100" on the commandline, it will tell you why the APs are losing connections.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II
Posts: 61
Registered: ‎08-12-2009

Remote office AP-65 resets wireless every couple seconds

Here is what my log is telling me:

Sep 9 12:46:43 :311004: |AP 00:1a:1e:c5:a2:42@10.52.68.125 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:44 :311004: |AP 00:1a:1e:c5:a0:22@10.52.76.115 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:45 :311004: |AP 00:1a:1e:c5:ce:d4@10.52.68.124 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP 00:1a:1e:c5:a1:ec@10.52.68.77 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP Maintenance Closet@10.52.76.69 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP 00:1a:1e:c5:a3:3a@10.52.68.240 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:47 :311004: |AP 00:1a:1e:c5:a2:84@10.52.76.92 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:47 :311004: |AP QE Library@10.52.76.54 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:48 :311004: |AP 00:1a:1e:c5:a2:a8@10.52.68.136 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:49 :311004: |AP 00:1a:1e:c5:a1:d4@10.52.76.113 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:50 :311004: |AP 00:1a:1e:c7:0a:32@10.52.68.166 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:51 :311004: |AP 00:1a:1e:c5:a2:34@10.52.68.126 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:51 :311004: |AP 00:1a:1e:c5:a0:32@10.52.69.2 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:52 :311004: |AP 00:1a:1e:c5:a2:9e@10.52.68.78 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:53 :311004: |AP 00:1a:1e:c5:a3:68@10.52.68.84 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:54 :311004: |AP 00:1a:1e:c7:0a:02@10.52.68.150 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:57 :311004: |AP 00:1a:1e:c5:a2:42@10.52.68.125 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:57 :311004: |AP 00:1a:1e:c5:a0:22@10.52.76.115 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:59 :311004: |AP 00:1a:1e:c5:ce:d4@10.52.68.124 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:59 :311004: |AP 00:1a:1e:c5:a1:ec@10.52.68.77 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP Maintenance Closet@10.52.76.69 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP 00:1a:1e:c5:a3:3a@10.52.68.240 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP 00:1a:1e:c5:a2:84@10.52.76.92 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:01 :311004: |AP QE Library@10.52.76.54 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:02 :311004: |AP 00:1a:1e:c5:a2:a8@10.52.68.136 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:03 :311004: |AP 00:1a:1e:c5:a1:d4@10.52.76.113 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:03 :311004: |AP 00:1a:1e:c7:0a:32@10.52.68.166 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping

(Solanco-Wireless) #
Guru Elite
Posts: 20,775
Registered: ‎03-29-2007

Looks like


Here is what my log is telling me:

Sep 9 12:46:43 :311004: |AP 00:1a:1e:c5:a2:42@10.52.68.125 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:44 :311004: |AP 00:1a:1e:c5:a0:22@10.52.76.115 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:45 :311004: |AP 00:1a:1e:c5:ce:d4@10.52.68.124 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP 00:1a:1e:c5:a1:ec@10.52.68.77 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP Maintenance Closet@10.52.76.69 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:46 :311004: |AP 00:1a:1e:c5:a3:3a@10.52.68.240 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:47 :311004: |AP 00:1a:1e:c5:a2:84@10.52.76.92 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:47 :311004: |AP QE Library@10.52.76.54 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:48 :311004: |AP 00:1a:1e:c5:a2:a8@10.52.68.136 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:49 :311004: |AP 00:1a:1e:c5:a1:d4@10.52.76.113 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:50 :311004: |AP 00:1a:1e:c7:0a:32@10.52.68.166 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:51 :311004: |AP 00:1a:1e:c5:a2:34@10.52.68.126 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:51 :311004: |AP 00:1a:1e:c5:a0:32@10.52.69.2 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:52 :311004: |AP 00:1a:1e:c5:a2:9e@10.52.68.78 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:53 :311004: |AP 00:1a:1e:c5:a3:68@10.52.68.84 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:54 :311004: |AP 00:1a:1e:c7:0a:02@10.52.68.150 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:57 :311004: |AP 00:1a:1e:c5:a2:42@10.52.68.125 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:57 :311004: |AP 00:1a:1e:c5:a0:22@10.52.76.115 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:59 :311004: |AP 00:1a:1e:c5:ce:d4@10.52.68.124 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:46:59 :311004: |AP 00:1a:1e:c5:a1:ec@10.52.68.77 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP Maintenance Closet@10.52.76.69 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP 00:1a:1e:c5:a3:3a@10.52.68.240 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:00 :311004: |AP 00:1a:1e:c5:a2:84@10.52.76.92 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:01 :311004: |AP QE Library@10.52.76.54 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:02 :311004: |AP 00:1a:1e:c5:a2:a8@10.52.68.136 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:03 :311004: |AP 00:1a:1e:c5:a1:d4@10.52.76.113 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping
Sep 9 12:47:03 :311004: |AP 00:1a:1e:c7:0a:32@10.52.68.166 sapd| Missed 8 heartbeats on radio 0 VAP 0; rebootstrapping

(Solanco-Wireless) #




That looks like all of your APS are rebootstrapping because of conectivity issues. You should ensure that the speed and duplex of the uplink controller interface is correct in your infrastructure. If it was only one access point, I would say check your infrastructure closer to it. Since it is multiple access points, check to find out what that controller interface has negotiated to:

(Aruba3600) #show interface gigabitethernet 1/0

GE 1/0 is up, line protocol is up
Hardware is Gigabit Ethernet, address is 00:0B:86:64:1E:61 (bia 00:0B:86:64:1E:61)
Description: GE1/0 (RJ45 Connector)
Encapsulation ARPA, loopback not set
Configured: Duplex ( AUTO ), speed ( AUTO )
Negotiated: Duplex (Full), speed (100 Mbps)
MTU 1500 bytes, BW is 100 Mbit
Last clearing of "show interface" counters 2 day 16 hr 22 min 36 sec
link status last changed 2 day 16 hr 18 min 15 sec
934868 packets input, 354061862 bytes
Received 175695 broadcasts, 0 runts, 7 giants, 0 throttles
117 input error bytes, 7 CRC, 0 frame
121611 multicast, 759173 unicast
729475 packets output, 144971823 bytes
0 output errors bytes, 0 deferred
0 collisions, 0 late collisions, 0 throttles
This port is TRUSTED
POE Status of the port is OFF


If you see any input errors, CRC errors that will indicate an issue, as well.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II
Posts: 61
Registered: ‎08-12-2009

Remote office AP-65 resets wireless every couple seconds

I have no CRC errors on my ethernet port on my Aruba controller and none on my uplink to the WAN from the buildings Cisco 3560. I moved one AP to a port on our 3560 and hard coded the port to 100/Full and still having the issue on that AP. Also the controller is in a different location the where I am having the issue. Is there a setting in the AP config that will hard code port speed and duplex?
Guru Elite
Posts: 20,775
Registered: ‎03-29-2007

Re: Remote office AP-65 resets wireless every couple seconds



There is a setting, but it will apply to all access points in a group. To find out how widespread your AP rebooting problem is use the "show ap debug counters" command. If your APs are plugged into ports that have speed and duplex that are not auto, auto, they will misnegotiate and have errors. To check to see what duplex your APs are negotiated to, check this post here: http://airheads.arubanetworks.com/vBulletin/showthread.php?t=761



Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II
Posts: 61
Registered: ‎08-12-2009

Wireshark to analyze traffic

Still trying to find a root cause to this issue. We have relived the issue at one location by borrowing a 200 controller from our consultant but still have the issue in 2 other location. Was on phone with support for over 2 hours last week and they said the master controller config is fine and that it must be something on the network side. So I want to mirror our uplink port to our remote locations with Wireshark which I have done successfully but not sure what to look for when searching for the heartbeats coming back to the master. If anyone can give some guidance it would be much appreciated.

Thanks
Ed
Guru Elite
Posts: 20,775
Registered: ‎03-29-2007

Path Back to the Master


Still trying to find a root cause to this issue. We have relived the issue at one location by borrowing a 200 controller from our consultant but still have the issue in 2 other location. Was on phone with support for over 2 hours last week and they said the master controller config is fine and that it must be something on the network side. So I want to mirror our uplink port to our remote locations with Wireshark which I have done successfully but not sure what to look for when searching for the heartbeats coming back to the master. If anyone can give some guidance it would be much appreciated.

Thanks
Ed




If you have a local controller and it disappears, then the path is between the remote site and the master controller. Is are the remote sites connected via VPN? Could there be a MTU issue?


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Occasional Contributor II
Posts: 61
Registered: ‎08-12-2009

Path Back to the Master

It is a fiber 100mb connection through Comcast. Support tried changing it from the default 1500 MTU to like 1200 and still nothing. I would like to see if the packet are actually coming across the WAN link to rule out one side or the other or even Comcast.
Guru Elite
Posts: 20,775
Registered: ‎03-29-2007

Packet Captures

If you wanted to do packet captures on the wire, you would have to do it, (1) at the switchport that the access point is plugged into and (2) at the switchport that the controller is plugged into and compare. Not easy, to say the least.


Colin Joseph
Aruba Customer Engineering

Looking for an Answer? Search the Community Knowledge Base Here: Community Knowledge Base

Search Airheads
Showing results for 
Search instead for 
Did you mean: