Security

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive? 

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break): Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)

I see the issue that with a RADIUS proxy it may not be useful to disable the upstream RADIUS server if one backend service does not respond. On the other hand, you would like ClearPass to respond if the server itself would go down.

Have not tried, but would running your RADIUS proxy over RadSec help?

If not, please work/report with Aruba Support so they can see if there is an option to find a better balance between error detection and not shutting down the service too fast. If these remote domains are (always) the same and persistently down, you may create a service before your eduroam service to 'catch' those domains and authenticate locally/deny access to avoid them to go to your eduroam.us servers.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
------------------------------

Original Message:
Sent: Dec 29, 2022 03:27 PM
From: Steve Bohrer
Subject: eduroam proxy timeouts from downstream failures

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive?

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break):

And the details of one:

Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)

Thanks! I do have a case for this issue, and you're right, it is a short coming of the original RADIUS protocol. I guess I can try RadSec and see if that helps
Original Message:
Sent: Jan 02, 2023 07:57 AM
From: Herman Robers
Subject: eduroam proxy timeouts from downstream failures

I see the issue that with a RADIUS proxy it may not be useful to disable the upstream RADIUS server if one backend service does not respond. On the other hand, you would like ClearPass to respond if the server itself would go down.

Have not tried, but would running your RADIUS proxy over RadSec help?

If not, please work/report with Aruba Support so they can see if there is an option to find a better balance between error detection and not shutting down the service too fast. If these remote domains are (always) the same and persistently down, you may create a service before your eduroam service to 'catch' those domains and authenticate locally/deny access to avoid them to go to your eduroam.us servers.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.

Original Message:
Sent: Dec 29, 2022 03:27 PM
From: Steve Bohrer
Subject: eduroam proxy timeouts from downstream failures

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive?

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break):

And the details of one:

Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)

Did you get further with this?

We are looking at a similar situation with an eduroam like setup. Marking the servers down seems not what we want, but the cause makes sense.

Original Message:
Sent: Jan 14, 2023 09:34 AM
From: skbohrer
Subject: eduroam proxy timeouts from downstream failures

Thanks! I do have a case for this issue, and you're right, it is a short coming of the original RADIUS protocol. I guess I can try RadSec and see if that helps
Original Message:
Sent: Jan 02, 2023 07:57 AM
From: Herman Robers
Subject: eduroam proxy timeouts from downstream failures

I see the issue that with a RADIUS proxy it may not be useful to disable the upstream RADIUS server if one backend service does not respond. On the other hand, you would like ClearPass to respond if the server itself would go down.

Have not tried, but would running your RADIUS proxy over RadSec help?

If not, please work/report with Aruba Support so they can see if there is an option to find a better balance between error detection and not shutting down the service too fast. If these remote domains are (always) the same and persistently down, you may create a service before your eduroam service to 'catch' those domains and authenticate locally/deny access to avoid them to go to your eduroam.us servers.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.

Original Message:
Sent: Dec 29, 2022 03:27 PM
From: Steve Bohrer
Subject: eduroam proxy timeouts from downstream failures

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive?

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break):

And the details of one:

Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)

Not really. I had a case, and it was suggested that RADSEC might maybe resolve the issue, though they weren't sure, and I haven't tried that out yet. They did confirm that I could not reduce the timeout after a no-response lower than 1 minute.  I also read some of the RADIUS RFCs, and found that this is indeed a known issue. The FreeRadius site has a handy list of RADIUS RFCs ( https://freeradius.org/rfc/ ), and RFC 5997 about server status packets talks about this issue.

In the past, back when we were still on Aruba 6, we this same issue between our local Mobility Controllers and our two clearpass boxes. That time, the issue was that the MC would time out based on no reply from clearpass, and mark the local clearpass box as down. Our workaround then was to split our eduroam auths from our local ones, with a longer timeout on the eduroam ones. Eduroam says they should reply in 10 seconds, so our local controllers allow 12 seconds on Radius requests to clearpass. (I don't know where the clearpass timeout setting for RADIUS proxies is, though!) 

------------------------------
Steve Bohrer
IT Infrastructure, Emerson College
------------------------------

Original Message:
Sent: Mar 26, 2023 01:09 PM
From: Unknown User
Subject: eduroam proxy timeouts from downstream failures

Did you get further with this?

We are looking at a similar situation with an eduroam like setup. Marking the servers down seems not what we want, but the cause makes sense.

Original Message:
Sent: Jan 14, 2023 09:34 AM
From: skbohrer
Subject: eduroam proxy timeouts from downstream failures

Thanks! I do have a case for this issue, and you're right, it is a short coming of the original RADIUS protocol. I guess I can try RadSec and see if that helps
Original Message:
Sent: Jan 02, 2023 07:57 AM
From: Herman Robers
Subject: eduroam proxy timeouts from downstream failures

I see the issue that with a RADIUS proxy it may not be useful to disable the upstream RADIUS server if one backend service does not respond. On the other hand, you would like ClearPass to respond if the server itself would go down.

Have not tried, but would running your RADIUS proxy over RadSec help?

If not, please work/report with Aruba Support so they can see if there is an option to find a better balance between error detection and not shutting down the service too fast. If these remote domains are (always) the same and persistently down, you may create a service before your eduroam service to 'catch' those domains and authenticate locally/deny access to avoid them to go to your eduroam.us servers.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.

Original Message:
Sent: Dec 29, 2022 03:27 PM
From: Steve Bohrer
Subject: eduroam proxy timeouts from downstream failures

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive?

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break):

And the details of one:

Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)

At the risk of necro-ing this again, did you come to a solution you ended up liking? I'm running into this myself at my org. 

Original Message:
Sent: Mar 29, 2023 11:35 AM
From: skbohrer
Subject: eduroam proxy timeouts from downstream failures

Not really. I had a case, and it was suggested that RADSEC might maybe resolve the issue, though they weren't sure, and I haven't tried that out yet. They did confirm that I could not reduce the timeout after a no-response lower than 1 minute.  I also read some of the RADIUS RFCs, and found that this is indeed a known issue. The FreeRadius site has a handy list of RADIUS RFCs ( https://freeradius.org/rfc/ ), and RFC 5997 about server status packets talks about this issue.

In the past, back when we were still on Aruba 6, we this same issue between our local Mobility Controllers and our two clearpass boxes. That time, the issue was that the MC would time out based on no reply from clearpass, and mark the local clearpass box as down. Our workaround then was to split our eduroam auths from our local ones, with a longer timeout on the eduroam ones. Eduroam says they should reply in 10 seconds, so our local controllers allow 12 seconds on Radius requests to clearpass. (I don't know where the clearpass timeout setting for RADIUS proxies is, though!) 

------------------------------
Steve Bohrer
IT Infrastructure, Emerson College

Original Message:
Sent: Mar 26, 2023 01:09 PM
From: Unknown User
Subject: eduroam proxy timeouts from downstream failures

Did you get further with this?

We are looking at a similar situation with an eduroam like setup. Marking the servers down seems not what we want, but the cause makes sense.

Original Message:
Sent: Jan 14, 2023 09:34 AM
From: skbohrer
Subject: eduroam proxy timeouts from downstream failures

Thanks! I do have a case for this issue, and you're right, it is a short coming of the original RADIUS protocol. I guess I can try RadSec and see if that helps
Original Message:
Sent: Jan 02, 2023 07:57 AM
From: Herman Robers
Subject: eduroam proxy timeouts from downstream failures

I see the issue that with a RADIUS proxy it may not be useful to disable the upstream RADIUS server if one backend service does not respond. On the other hand, you would like ClearPass to respond if the server itself would go down.

Have not tried, but would running your RADIUS proxy over RadSec help?

If not, please work/report with Aruba Support so they can see if there is an option to find a better balance between error detection and not shutting down the service too fast. If these remote domains are (always) the same and persistently down, you may create a service before your eduroam service to 'catch' those domains and authenticate locally/deny access to avoid them to go to your eduroam.us servers.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.

Original Message:
Sent: Dec 29, 2022 03:27 PM
From: Steve Bohrer
Subject: eduroam proxy timeouts from downstream failures

Our clearpass Events logs shows fairly frequent timeouts from both eduroam.us proxy servers, and it responds by marking the proxy as down for a minimum of 60 seconds. The cause is when some random school somewhere doesn't reply, and clearpass can't tell the difference between no response from the proxy (which is up, and is handling _lots_ of downstream schools) and the failure of a single down stream. Mostly one or the other proxy is up, but I think that is just luck -- is there a better way?

Maybe I just have too short a timeout on these proxies? What is a recommended setting?

If I establish a radsec connection to eduroam.us rather than our existing bare RADIUS, would that let clearpass tell that the proxy is up, even if a particular remote .edu is non-responsive?

Or, is there any way to have a shorter "down time" after a failed proxy than the 60 sec minimum the GUI allows?

Here's the events I'm concerned about, which hit pretty frequently, (though not so many now with all the students away for break):

And the details of one:

Following through with the event viewer at the specified time shows a "no response from home server", and that is enough for CPPM to shut down half of our connection to Eduroam.us !

(Our campus is on the edge of Boston Common, so we get lots of walk-by eduroam traffic)