Wireless Access

Hello

I'm facing ramdomize drops for 802.1x clients connected on the wired ports of my RAP 155P.

I have a AAA profile for MAC and 802.1x authenitcations and for all the hosts connected via mac autheitcation they work without any problem. Connectivity is recovered automatically after 2 seconds but all connections from the 802.1x clients during the hit are dropped which is causing alot issues with any flows from the host.

Endpoints are Win7 with latest LAN driver, ArubaOS is 6.4.3.5 and ClearPass is on 6.5.3. 

I've also verify my Radius Cert which have been issued from my internal PKI, I also tried to deactivate validate Cert on the 802.1x configuration for the Windows NIC and I don't see events on the Radius server for the hits in most ofthe cases and on the controller, debugging the endpoints I don't see anything which can indicate a problem.

I've been trying to get a resolution from the TAC for more than a week but I don't receive any direction and I don't think this looks like they will solve the issue so I plan to scalate the case next week anyway if someone coulld help me to stablize this will be great.

 

Thanks in advance

Why are you doing mac authentication on top of 802.1x? You should eliminate that to get to the bottom of your problem. You should execute "show auth-tracebuf" to find out why it is happening.

Thanks for answering. I need both authentication methods to support devices which are not performing 8021.x.  I will look into the output for the command auth-tracebuf right after a hit happens to see if I can narrow down the root of the problem.

Regards,

 

I haven't triggered yet the vent but so far I see something.

Although my client is set to 802.1x seems to send mac authentication requests

 

Dec  6 11:01:22  m-auth req             *  34:e6:d7:2f:87:be  01:80:c2:00:00:03        -    -    
Dec  6 11:01:22  m-auth resp            *  34:e6:d7:2f:87:be  01:80:c2:00:00:03        -    -     failed
Dec  6 11:01:59  rad-acct-int-update   ->  34:e6:d7:2f:87:be  01:80:c2:00:00:03/cppm1  -    -    
Dec  6 11:12:12  rad-acct-int-update   ->  34:e6:d7:2f:87:be  01:80:c2:00:00:03/cppm1  -    -    

If you enable mac authentication AND  802.1x in the AAA profile, both are required.  You should enable "L2 Authentication Fail Through" in the AAA profile so that only one is required.

 

In your output above, mac authentication is failing and then interim radius accounting is being triggered...

I have L2 Authentication Fail Trough enabled and under AAA profile for MAC Authentication I have a the following options enabled: Reauthentication, Reauthentication interval to 60sec and use Server provided provided reauthentication interval which I sent from the CPPM to 3600

Why do you have reauthentication enabled?

 

Just to be clear, you are having "drops" on 802.1x.  Does that mean your 802.1x device is losing connectivity?

 

Reauthentication is enabled to allow the endpoints which are using MAC authenticatio to try again automatically after the set interval to avoid end users to disconnect from the RAP the cable to force an authentication.

802.1x clients are losing connectoivity for 2/3 seconds, MAC endpoints work fine

Ok.

 

How often does the disconnect happen?  Find a single wired 802.1x client and do this:

 

config t
logging level debugging user-debug <mac address of client>

Observe the client until it disconnects and then do this:

show log user-debug 100
show auth-tracebuf mac <mac address of client>

 

 

It is very variable, there are days where this is become more than annoying and of course you get disconnted from your session so very distrusctive. I will have a checks logs for my mac address after gets disconnected

To be clear, when you enable user debugging, you will only be able to see the auth-tracebuf for that individual user that you turn on debugging for.  You would have to add other mac addresses to the debug to see their auth-tracebuf, as well.  The auth-tracebuf also does not keep a long historical record of radius traffic, that is why you try to limit it to single users that you are focusing on.

 

The big question is, how long does the initial 802.1x authentication take from when you plug in, to when it is successful?  If it takes 2 to 3 seconds, then it is possible that the "drops" you are seeing is  the regular radius reauthentication interval which is 86400 seconds (24 hours) by default.  Does this occur for devices that are plugged in full-time?  If that is the case that could be your problem.  In that case, you might consider what it would take to get that down from 2 to 3 seconds.  Having reauthentication in your mac authentication profile could put additional load on your radius server and tie up cycles needed to authenticate your other client; I would look to see if that is happening:

 

show dot1x watermark history

What kind of controller is this?

 

I've added another computer connected on the same RAP.  In order to simulate a reauthentication, from the controller I use the command aaa user delete and the user takes onle 1 ping out and looking at the network connection I don't even see the interface marked as diconnected, it seems to happen very quick.

Controller model is  7210 and the number of RAP connected at this time is about 10.

The output for show dot1x watermark history is as you can see below

 

High WaterMark                  :550

Low WaterMark                   :522

Threshhold for STM throttling   :275

STM throttling percent          :50

 

The load on the controller and CPPM is low specially right now.  This issue happends with computers connected all the time. Now I have two endpoints using 802.1x connected on the same RAP with a ssh session to the same controller ( logging session is set to 0 to keep it up all the time).  I'm also debugging both MAC on the controller

Okay.  it is quite possible that you have WAN congestion (I am assuming those are RAPS), OR you need to enable broadcast controls on the wired VLAN that your devices are on:

 

config t

interface vlan x

bcmc optimization

 

I have plenty of capacity at both ends. Devices are RAP155P, I also have bcmc-optimization of the VLAN for all clients

 

interface vlan 610
        ip address 172.20.x.x 255.255.X.X
        ip helper-address 172.20.X.X
        ip helper-address 172.20.X.Y
        ip local-proxy-arp
        operstate up
        bcmc-optimization

You should open a tac case. I am just guessing based on limited information.

Thank you very much for all your suggestions. I have a case open with TAC but enigneer seems to be very unresponsive, no idea why he wasn't already escalating the issue to someone else

 

Customers have to ask for a case to be escalated. TAC will only escalate if it is severe like if a system is completely down or degraded.

TAC couldn't identify the root the problem.  RAP where I used to have the drops was using the same egress connection for an IAP cluster which was sending 3/4 attempt to be converted from IAP to RAP due to a default covert rule on activate. Since I removed the rules on activate everything is much more stable and I haven't seen drops anymore and my CPPM doesn't have to proccess the tons on request.

 

I will say that successful 802.1x authentication is less about bandwidth and more about latency.  Any latency over 100 milliseconds could negatively affect a 802.1x transaction.

I will say that successful 802.1x authentication is less about bandwidth and more about latency.  Any latency over 100 milliseconds could negatively affect a 802.1x transaction.