Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.


Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"


Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed
​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!


Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.


Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.


The details here don't matter; it is just a temporary workaround to remove the server certs that have a link to the root cert that needs to be removed.


This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.





Now it can be deleted.


Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List


Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.


However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provider, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----​

And it imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:


For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C
------------------------------

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

On 2/12/21 9:20 am, Tim C via Airheads Community wrote:
> You should never use LetsEncrypt for EAP. In fact, you should never use
> a WebPKI CA for EAP in general. ------------------------------

And yet Android is basically requiring that now.


--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877


Original Message:
Sent: 12/1/2021 8:20:00 PM
From: timms
Subject: RE: Howto: ClearPass and Expired Root Certificate

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C
------------------------------

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

James, can you share the source of that information? I may have missed something.

The information that I have is that Android 11 and up removed the option to 'don't validate the RADIUS server certificate' as required for WPA3 certification. That means that it is even harder for end-users to manually configure their device, and you are strongly encouraged to use an automated method for provisioning your client devices in a secure way, which includes installation of your private Root CA for the EAP certificate. MDM, ClearPass Onboard, or similar tools facilitate that.

Public trusted certificates for EAP have some important drawbacks, like the low maximum lifetime, the lack of control over the CA, and it's nearly impossible to switch to another Root without end-user device control, once deployed. Even in the eduroam community, which has been mainly on public certificates till the moment that public certificates got reduced lifetime to 2 years (currently even 1 year), I see a movement to private CAs; though they have their own provisioning tooling (CAT / geteduroam) to fix the root trust and enforce simple and correct configuration of the client devices.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
------------------------------

Original Message:
Sent: Dec 02, 2021 03:29 AM
From: James Andrewartha
Subject: Howto: ClearPass and Expired Root Certificate

On 2/12/21 9:20 am, Tim C via Airheads Community wrote:
> You should never use LetsEncrypt for EAP. In fact, you should never use
> a WebPKI CA for EAP in general. ------------------------------

And yet Android is basically requiring that now.


--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877


Original Message:
Sent: 12/1/2021 8:20:00 PM
From: timms
Subject: RE: Howto: ClearPass and Expired Root Certificate

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

Original Message:
Sent: 12/2/2021 4:18:00 AM
From: Herman Robers
Subject: RE: Howto: ClearPass and Expired Root Certificate

James, can you share the source of that information? I may have missed something.

The information that I have is that Android 11 and up removed the option to 'don't validate the RADIUS server certificate' as required for WPA3 certification. That means that it is even harder for end-users to manually configure their device, and you are strongly encouraged to use an automated method for provisioning your client devices in a secure way, which includes installation of your private Root CA for the EAP certificate. MDM, ClearPass Onboard, or similar tools facilitate that.

Public trusted certificates for EAP have some important drawbacks, like the low maximum lifetime, the lack of control over the CA, and it's nearly impossible to switch to another Root without end-user device control, once deployed. Even in the eduroam community, which has been mainly on public certificates till the moment that public certificates got reduced lifetime to 2 years (currently even 1 year), I see a movement to private CAs; though they have their own provisioning tooling (CAT / geteduroam) to fix the root trust and enforce simple and correct configuration of the client devices.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.
------------------------------

Original Message:
Sent: Dec 02, 2021 03:29 AM
From: James Andrewartha
Subject: Howto: ClearPass and Expired Root Certificate

On 2/12/21 9:20 am, Tim C via Airheads Community wrote:
> You should never use LetsEncrypt for EAP. In fact, you should never use
> a WebPKI CA for EAP in general. ------------------------------

And yet Android is basically requiring that now.


--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877


Original Message:
Sent: 12/1/2021 8:20:00 PM
From: timms
Subject: RE: Howto: ClearPass and Expired Root Certificate

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

They're protecting users from following instructions that tell them to compromise their security and privacy.

------------------------------
Tim C
------------------------------

Original Message:
Sent: Dec 02, 2021 11:36 AM
From: James Andrewartha
Subject: Howto: ClearPass and Expired Root Certificate

Well that's it, because provisioning a private root CA is extremely hard to do manually for the regular user. Not every organisation has the ability to use CP Onboard or an MDM. Don't get me wrong, I agree that a private CA should be used, but Google clearly doesn't care and would rather encourage the use of public CAs due to the horrible UX they've engineered.

 

Original Message:
Sent: 12/2/2021 4:18:00 AM
From: Herman Robers
Subject: RE: Howto: ClearPass and Expired Root Certificate

James, can you share the source of that information? I may have missed something.

The information that I have is that Android 11 and up removed the option to 'don't validate the RADIUS server certificate' as required for WPA3 certification. That means that it is even harder for end-users to manually configure their device, and you are strongly encouraged to use an automated method for provisioning your client devices in a secure way, which includes installation of your private Root CA for the EAP certificate. MDM, ClearPass Onboard, or similar tools facilitate that.

Public trusted certificates for EAP have some important drawbacks, like the low maximum lifetime, the lack of control over the CA, and it's nearly impossible to switch to another Root without end-user device control, once deployed. Even in the eduroam community, which has been mainly on public certificates till the moment that public certificates got reduced lifetime to 2 years (currently even 1 year), I see a movement to private CAs; though they have their own provisioning tooling (CAT / geteduroam) to fix the root trust and enforce simple and correct configuration of the client devices.

------------------------------
Herman Robers
------------------------
If you have urgent issues, always contact your Aruba partner, distributor, or Aruba TAC Support. Check https://www.arubanetworks.com/support-services/contact-support/ for how to contact Aruba TAC. Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba Networks.

In case your problem is solved, please invest the time to post a follow-up with the information on how you solved it. Others can benefit from that.

Original Message:
Sent: Dec 02, 2021 03:29 AM
From: James Andrewartha
Subject: Howto: ClearPass and Expired Root Certificate

On 2/12/21 9:20 am, Tim C via Airheads Community wrote:
> You should never use LetsEncrypt for EAP. In fact, you should never use
> a WebPKI CA for EAP in general. ------------------------------

And yet Android is basically requiring that now.


--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877


Original Message:
Sent: 12/1/2021 8:20:00 PM
From: timms
Subject: RE: Howto: ClearPass and Expired Root Certificate

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------

Original Message:
Sent: 12/2/2021 3:29:00 AM
From: TRS-80
Subject: RE: Howto: ClearPass and Expired Root Certificate

On 2/12/21 9:20 am, Tim C via Airheads Community wrote:
> You should never use LetsEncrypt for EAP. In fact, you should never use
> a WebPKI CA for EAP in general. ------------------------------

And yet Android is basically requiring that now.


--
James Andrewartha
Network & Projects Engineer
Christ Church Grammar School
Claremont, Western Australia
Ph. (08) 9442 1757
Mob. 0424 160 877


Original Message:
Sent: 12/1/2021 8:20:00 PM
From: timms
Subject: RE: Howto: ClearPass and Expired Root Certificate

You should never use LetsEncrypt for EAP. In fact, you should never use a WebPKI CA for EAP in general.

------------------------------
Tim C
------------------------------

Original Message:
Sent: Dec 01, 2021 08:25 AM
From: Richard Litchfield
Subject: Howto: ClearPass and Expired Root Certificate

Howto: ClearPass and Expired Root Certificate

Let's Encrypt

The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to https://letsencrypt.org/certificates/.

The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.

Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .

Tale of Two Pixels^*

^* If you are exclusively an Apple environment, these are both Android phones.

Google Pixel 6

The new phone - a Pixel 6 - had two issues:

1. It was unable to connect to the .1x wireless SSID "LW1X"
2. It also mandated the use of the "Domain" field that was previously allowed to be blank.

Google Pixel 3

The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work even after the root CA cert had expired. There were no errors or warnings that anything was amiss.

However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:

1. It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
2. It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.

Other Devices

Every other tested device connected to LW1X without error. That included:

• Windows notebooks
• Android tablets
• Android mobiles
• ereader
• UXI Cape Sensor

Symptoms

Mobile

From the mobile, this is the mandatory domain requirement.

Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials

ClearPass

This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"

Delving into the logs might show something similar to this:

2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed​

Changing Certificates in ClearPass

Challenges

Cannot Delete Certificate
This will be easy - just delete the expired root certificate!

Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.

Cannot Remove the Server Certs
My next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.

Self-Signed Certificate

Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.

The details here don't matter; it is just a temporary workaround to remove the server certs that have a ling to the root cert that needs to be removed.

This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.

Delete Expired Certificate

Now it is possible to disable the expired certificate.

Now it can be deleted.

Gone!

Adding a New Certificate

Initial Attempt
I tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List

Ensure self-signed ISRC Root X1 cert is installed
I had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.

However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.

Install R3 Cert
The next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.

Recreate Server Certificate and Import
Depending on the certificate provide, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.

You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:

-----BEGIN CERTIFICATE----------END CERTIFICATE-----​

imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.

Android - Domain cannot be blank

The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.

This is the ClearPass error seen when the domain is incorrectly specified:

For my network, the following entries worked in the domain field:

• net (this later stopped working, or was unreliable)
• litchwan2.net <-- this is the one you would normally use
• bv.litchwan2.net
• clearpass.bv.litchwan2.net

These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...

These entries did not work in the domain field:

• litchwan2
• test
• litch
• clearpass

References

https://letsencrypt.org/certificates/
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewer
https://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update

------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------