Wired Intelligent Edge

I've run into what I think is a timing issue after a general campus power failure.  We have access switches that are coming online and supplying PoE before the aggregation switch is online and forwarding traffic, causing RADIUS MAC auth to fail on some devices (because the RADIUS server is unreachable).  We're going to try extending the RADIUS timeouts and retries in attempt to add more wiggle room for the slow aggregation switch, but I was curious if anyone here has run into that problem and knows of a different way to solve it.  FYI, not all of the devices we need to MAC auth do LLDP; I've considered LLDP groups as an alternative.

------------------------------
Daniel Waites
Sabyr Consulting

------------------------------

Maybe you have Quick PoE turned on. You didn't say which CX platform.  This expedites the process for end points. To give you wiggle room maybe look int this tuned off if your cores /upstream keep on losing power. But a site failure like this need to look into more holistic issues

------------------------------
Kamal Takodra
If my post was useful accept solution and/or give kudos
------------------------------

Original Message:
Sent: Jul 21, 2021 03:30 PM
From: Daniel Waites
Subject: Any ideas for delaying port-access auth on AOS-CX after power failure?

I've run into what I think is a timing issue after a general campus power failure.  We have access switches that are coming online and supplying PoE before the aggregation switch is online and forwarding traffic, causing RADIUS MAC auth to fail on some devices (because the RADIUS server is unreachable).  We're going to try extending the RADIUS timeouts and retries in attempt to add more wiggle room for the slow aggregation switch, but I was curious if anyone here has run into that problem and knows of a different way to solve it.  FYI, not all of the devices we need to MAC auth do LLDP; I've considered LLDP groups as an alternative.

------------------------------
Daniel Waites
Sabyr Consulting

------------------------------

Sure, this is a 6300.  Quick PoE is not turned on, however we are using the concurrent onboarding feature of the newer software images that does MAC authentication concurrently with EAP, which causes headless devices to come up much faster than would ordinarily be the case.  Because the feature is new, it's likely that most others have not run into this issue yet.  Our test with increased RADIUS retries/timeouts was successful, but this is a potential downside to faster headless onboarding.

------------------------------
Daniel Waites
------------------------------

Original Message:
Sent: Jul 22, 2021 08:26 AM
From: Kamal Takodra
Subject: Any ideas for delaying port-access auth on AOS-CX after power failure?

Maybe you have Quick PoE turned on. You didn't say which CX platform.  This expedites the process for end points. To give you wiggle room maybe look int this tuned off if your cores /upstream keep on losing power. But a site failure like this need to look into more holistic issues

------------------------------
Kamal Takodra
If my post was useful accept solution and/or give kudos

Original Message:
Sent: Jul 21, 2021 03:30 PM
From: Daniel Waites
Subject: Any ideas for delaying port-access auth on AOS-CX after power failure?

I've run into what I think is a timing issue after a general campus power failure.  We have access switches that are coming online and supplying PoE before the aggregation switch is online and forwarding traffic, causing RADIUS MAC auth to fail on some devices (because the RADIUS server is unreachable).  We're going to try extending the RADIUS timeouts and retries in attempt to add more wiggle room for the slow aggregation switch, but I was curious if anyone here has run into that problem and knows of a different way to solve it.  FYI, not all of the devices we need to MAC auth do LLDP; I've considered LLDP groups as an alternative.

------------------------------
Daniel Waites
Sabyr Consulting

------------------------------

Hello, 
I think maybe I had a similar issue with a customer when using MAC authentication. The access port was coming online much faster than the RADIUS server was becoming reachable. For this reason the MAC authentication was failing and the switch was actually doing by default only 1 attempt to validate the MAC address with the RADIUS server. If this first attempt failed, the switch was not trying again even when the RADIUS server was reachable again, the port remaining in status "failed authentication".
I concentrated on how to make the switch do a new authentication attempt after a while. This can be done using reauthentication with a short reauthentication period. Configuring reauthentication in the mac-auth context like this

switch(config-if)#aaa authentication port-access mac-auth

switch(config-if-macauth)#reauth

switch(config-if-macauth)#reauth-period 65

is not feasible because it will happen continuously after successful authentication and overwhelm the RADIUS server with requests.
It is better to use the critical-role feature. The critical-role is applied when the authentication fails because RADIUS server is not reachable. You can configure a short reauthentication period in the critical role and maybe some void VLAN. When the MAC authentication fails the switch will not block the client completely but place it in the critical-role where it has no network access and reauthentication is enforced in some short intervals. Eventually the RADIUS server will come online and one of the subsequent reauthentication attempts will succeed which will place the user in the right role with the correct settings and possible no reauthentication or reauthentication with a more reasonable period.
This is the config I used in the test.

interface 1/1/1

    shutdown

    downshift-enable

    qos trust dscp

    no routing

    vlan access 1

    spanning-tree port-type admin-edge

    aaa authentication port-access allow-cdp-bpdu

    aaa authentication port-access client-limit 10

    aaa authentication port-access critical-role TEST

    port-access allow-flood-traffic enable

    aaa authentication port-access dot1x authenticator

        eapol-timeout 2

        max-eapol-requests 2

        max-retries 1

        enable

    aaa authentication port-access mac-auth

        enable

port-access role TEST

    reauth-period 40

This is what worked for this customer, not sure how helpful it can be for you.

------------------------------
Emil Gogushev
------------------------------

Original Message:
Sent: Jul 21, 2021 03:30 PM
From: Daniel Waites
Subject: Any ideas for delaying port-access auth on AOS-CX after power failure?

I've run into what I think is a timing issue after a general campus power failure.  We have access switches that are coming online and supplying PoE before the aggregation switch is online and forwarding traffic, causing RADIUS MAC auth to fail on some devices (because the RADIUS server is unreachable).  We're going to try extending the RADIUS timeouts and retries in attempt to add more wiggle room for the slow aggregation switch, but I was curious if anyone here has run into that problem and knows of a different way to solve it.  FYI, not all of the devices we need to MAC auth do LLDP; I've considered LLDP groups as an alternative.

------------------------------
Daniel Waites
Sabyr Consulting

------------------------------

Original Message:
Sent: 7/29/2021 3:53:00 AM
From: Emil_G
Subject: RE: Any ideas for delaying port-access auth on AOS-CX after power failure?

Hello, 
I think maybe I had a similar issue with a customer when using MAC authentication. The access port was coming online much faster than the RADIUS server was becoming reachable. For this reason the MAC authentication was failing and the switch was actually doing by default only 1 attempt to validate the MAC address with the RADIUS server. If this first attempt failed, the switch was not trying again even when the RADIUS server was reachable again, the port remaining in status "failed authentication".
I concentrated on how to make the switch do a new authentication attempt after a while. This can be done using reauthentication with a short reauthentication period. Configuring reauthentication in the mac-auth context like this

switch(config-if)#aaa authentication port-access mac-auth

switch(config-if-macauth)#reauth

switch(config-if-macauth)#reauth-period 65

is not feasible because it will happen continuously after successful authentication and overwhelm the RADIUS server with requests.
It is better to use the critical-role feature. The critical-role is applied when the authentication fails because RADIUS server is not reachable. You can configure a short reauthentication period in the critical role and maybe some void VLAN. When the MAC authentication fails the switch will not block the client completely but place it in the critical-role where it has no network access and reauthentication is enforced in some short intervals. Eventually the RADIUS server will come online and one of the subsequent reauthentication attempts will succeed which will place the user in the right role with the correct settings and possible no reauthentication or reauthentication with a more reasonable period.
This is the config I used in the test.

interface 1/1/1

    shutdown

    downshift-enable

    qos trust dscp

    no routing

    vlan access 1

    spanning-tree port-type admin-edge

    aaa authentication port-access allow-cdp-bpdu

    aaa authentication port-access client-limit 10

    aaa authentication port-access critical-role TEST

    port-access allow-flood-traffic enable

    aaa authentication port-access dot1x authenticator

        eapol-timeout 2

        max-eapol-requests 2

        max-retries 1

        enable

    aaa authentication port-access mac-auth

        enable

port-access role TEST

    reauth-period 40

This is what worked for this customer, not sure how helpful it can be for you.

------------------------------
Emil Gogushev
------------------------------

Original Message:
Sent: Jul 21, 2021 03:30 PM
From: Daniel Waites
Subject: Any ideas for delaying port-access auth on AOS-CX after power failure?

I've run into what I think is a timing issue after a general campus power failure.  We have access switches that are coming online and supplying PoE before the aggregation switch is online and forwarding traffic, causing RADIUS MAC auth to fail on some devices (because the RADIUS server is unreachable).  We're going to try extending the RADIUS timeouts and retries in attempt to add more wiggle room for the slow aggregation switch, but I was curious if anyone here has run into that problem and knows of a different way to solve it.  FYI, not all of the devices we need to MAC auth do LLDP; I've considered LLDP groups as an alternative.

------------------------------
Daniel Waites
Sabyr Consulting

------------------------------