Wireless Access

I have an annoying problem on my wired port access setup and need some advice on how to troubleshoot.

The setup

Client 1---VLAN 600—AP---VLAN 30-- controller ---VLAN 600 --- switch ---VLAN 600-- client 2

This is an Alcatel 6000 with M3 blade running code 5.0.4.6. The controller and APs were rebooted. the radios are turned off on the APs.

Problem – Hi latency spikes on clients connected to the second port of AP. When sending ICMP requests from a client ( 1 )  directly connect to the second port of the AP to a client ( 2 ) on the same VLAN connected to a port of a switch I see high latency spikes. Spikes up to 50ms at times. When client 1 sends a ICMP request to the controller’s IP interface on VLAN 600 I get the same latency spikes. We see this behavior also when client 2 sends ICMP to client 1.

Example:

64 bytes from 169.232.231.190: icmp_req=173 ttl=61 time=0.887 ms

64 bytes from 169.232.231.190: icmp_req=174 ttl=61 time=0.913 ms

64 bytes from 169.232.231.190: icmp_req=175 ttl=61 time=0.837 ms

64 bytes from 169.232.231.190: icmp_req=176 ttl=61 time=0.998 ms

64 bytes from 169.232.231.190: icmp_req=177 ttl=61 time=0.830 ms

64 bytes from 169.232.231.190: icmp_req=178 ttl=61 time=1.93 ms

64 bytes from 169.232.231.190: icmp_req=179 ttl=61 time=0.816 ms

64 bytes from 169.232.231.190: icmp_req=180 ttl=61 time=26.0 ms

64 bytes from 169.232.231.190: icmp_req=335 ttl=61 time=0.749 ms

64 bytes from 169.232.231.190: icmp_req=336 ttl=61 time=0.806 ms

64 bytes from 169.232.231.190: icmp_req=337 ttl=61 time=0.844 ms

64 bytes from 169.232.231.190: icmp_req=338 ttl=61 time=0.852 ms

64 bytes from 169.232.231.190: icmp_req=339 ttl=61 time=0.933 ms

64 bytes from 169.232.231.190: icmp_req=340 ttl=61 time=0.901 ms

64 bytes from 169.232.231.190: icmp_req=341 ttl=61 time=0.891 ms

64 bytes from 169.232.231.190: icmp_req=342 ttl=61 time=30.5 ms

64 bytes from 169.232.231.190: icmp_req=343 ttl=61 time=0.862 ms

Iperf shows that same latency spikes using UDP tests.

To me this shows that the AP or the controller is buffering the traffic at times. What I don’t get is why it does this at all, even when there is little to no traffic going to / from the controller ( middle of the night ).  I do not see any latency spikes when the clients are connected to a different parts of my LAN, only when connected to the AP.

Any advice,  tips, or some magic on how to troubleshoot this would be greatly appreciated.

Q.

Anyone?

In order to narrow down where the latency is getting introduced, I would run simultaneous ping from client 1 to 

1. controller 

2. switch 

3. client 2 

I just ran 2 more tes.

client 1 ---- contoller ---- client 2 ( high latency spikes and some packet drops )

client 1 --- switch ---- client 2 ( no loss or high latency spikes )

So to me this points at the controller, but when I look at the interface info I see no dropps / errors. Is there any more info I can get off this interface? like a debug command or somehting?

Thanks

Q.

GE 0/7 is up, line protocol is up
Hardware is Gigabit Ethernet, address is 00:0B:86:0B:7C:08 (bia 00:0B:86:0B:7C:08)
Description: Engineering_testing
Encapsulation ARPA, loopback not set
Configured: Duplex ( AUTO ), speed ( AUTO )
Negotiated: Duplex (Full), speed (1000 Mbps)
MTU 1500 bytes, BW is 1000 Mbit
Last clearing of "show interface" counters 25 day 1 hr 55 min 8 sec
link status last changed 0 day 0 hr 30 min 35 sec
    2741 packets input, 294249 bytes
    Received 79 broadcasts, 0 runts, 0 giants, 0 throttles
    0 input error bytes, 0 CRC, 0 frame
    54 multicast, 2662 unicast
    9120 packets output, 802159 bytes
    0 output errors bytes, 0 deferred
    0 collisions, 0 late collisions, 0 throttles
This port is TRUSTED

QLD,

Please open a TAC case for this issue as it will be time consuming to resolve it using the forums. 

Thanks,

--

HT 

A TAC case has been opened with Alcatel support.

Thank you for your help!

Q.

What is the latency when wired client connected to controller and you ping the IP address on the controller in the same VLAN?

How many APs and users are up on the controller?

Can you please take multiple outputs of "show datapath maintanace counter | include "Buffer"" and paste it here?

Here is the output..

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #
(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610

(wc01-anderson) #show datapath maintenance counters | include "Buffer"
Buffer Underruns       0
BWM Buffer Exhaustion  0
Buffer Alloc Failure   9848610