Howto: ClearPass and Expired Root Certificate
Let's Encrypt
The challenge with the expiration of the Let's Encrypt Root CA certificate has been a discussion point for some time, and there is plenty of material covering this from at least a year ago. Refer to
https://letsencrypt.org/certificates/.
The key problem is that the root at the start of the chain has now expired:
Certificate CA: CN=DST Root CA X3,O=Digital Signature Trust Co.
Note that free certificates like the ones provided via Let's Encrypt are useful for labs and testing. However, for everything else, paid certificates with more detailed validation processes (Organization Validated - OV, or Extended Validation - EV) and longer expiration dates (free ones are typically a maximum of 90 days) are recommended .
Tale of Two Pixels*
* If you are exclusively an Apple environment, these are both Android phones.
Google Pixel 6
The new phone - a Pixel 6 - had two issues:
- It was unable to connect to the .1x wireless SSID "LW1X"
- It also mandated the use of the "Domain" field that was previously allowed to be blank.
Google Pixel 3
The old phone - a Pixel 3 - had been connecting to the LW1X wireless SSID with .1x for about 3 years. It continued to work
even after the root CA cert had expired. There were no errors or warnings that anything was amiss.
However, as I was testing with the Pixel 3, I deleted the LW1X profile on the phone and recreated it...and it stopped working. It now had the same two issues as the Pixel 6:
- It was unable to connect to the .1x wireless SSID "LW1X" because of the expired certificate.
- It also mandated the use of the "Domain" field that was previously allowed to be blank. One of the Android OS updates obviously included these security updates, but only enforced them on NEW profiles.
Other Devices
Every other tested device connected to LW1X without error. That included:
- Windows notebooks
- Android tablets
- Android mobiles
- ereader
- UXI Cape Sensor
Symptoms
Mobile
From the mobile, this is the mandatory domain requirement.
Checking the phones show that the required ISRG root CA is already in place.
Settings | Certificate management|Encryption and credentials | Trusted credentials
ClearPass
This is one of the ways that the expired certificate presents in ClearPass. The key text is "fatal alert by client - certificate_expired" and "alert certificate expired"
Delving into the logs might show something similar to this:
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS Alert read:fatal:certificate expired
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - TLS_accept:failed in error
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: SSL_read failed inside of TLS (-1), TLS session fails. error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
2021-11-18 00:07:03,449 [Th 41 Req 2014 SessId R000000f4-01-6194fe77] ERROR RadiusServer.Radius - rlm_eap_tls: TLS Handshake failed
Changing Certificates in ClearPass
Challenges
Cannot Delete CertificateThis will be easy - just delete the expired root certificate!
Unfortunately not; there is no way to disable or remove any certificate from a chain that is in use.
Cannot Remove the Server CertsMy next thought was to remove/disable the server certs so that the root cert could be removed. There is also no way to disable or remove a server cert - you can only replace it.
Self-Signed Certificate
Luckily I didn't have to go through the process of obtaining a certificate with a different chain of trust - ClearPass is able to generate a self-signed certificate. This is a really easy process - just click the Create Self-Signed Certificate in Administration » Certificates » Certificate Store, and follow the instructions.
The details here don't matter; it is just a temporary workaround to remove the server certs that have a link to the root cert that needs to be removed.
This needs to be done for each server type that had a certificate assigned previously. In my case, this was 3 times, for RADIUS, HTTPS, RADsec.
Delete Expired Certificate
Now it is possible to disable the expired certificate.
Now it can be deleted.
Gone!
Adding a New Certificate
Initial AttemptI tried to re-import the existing server certificate, but this was unsuccessful with the following error:
Certificate CA "CN=DST Root CA X3,O=Digital Signature Trust Co." with appropriate Subject Key Identifier must be added and enabled in Certificate Trust List
Ensure self-signed ISRC Root X1 cert is installedI had previously loaded both the self-signed and cross-signed ISRC Root X1 CA certificates (cross-signed by the expired DST Root CA X3).
I deleted the cross-signed cert, leaving just self-signed X1.
However, I was still getting the same error when attempting to import the server cert. Several other actions including rebooting ClearPass server, and disabling most certs in ClearPass made no difference.
Install R3 CertThe next step was to download and import the intermediate R3 cert. The certificate chain of root + intermediate is now installed in ClearPass.
Recreate Server Certificate and ImportDepending on the certificate provider, you might be able to specify what is included in the certificate being generated - whole chain or just the server cert. I had no choice, so I modified the existing certificate PEM file to remove the root and intermediate certificates.
You can modify the certificate file with your favourite text editor, or use a proper certificate management tool-set like OpenSSL to extract the server cert. From my original PEM file, I deleted the last two certificate blocks - the intermediate and root - leaving just the server cert. The certificate blocks are designated by:
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
And it imported OK! Those pesky Pixel phones can now connect successfully to the .1x wireless SSID.
Android - Domain cannot be blank
The Pixel mobiles (and maybe eventually all Android devices) need to specify the domain. You cannot proceed without entering data in this field.
This is the ClearPass error seen when the domain is incorrectly specified:
For my network, the following entries worked in the domain field:
- net (this later stopped working, or was unreliable)
- litchwan2.net <-- this is the one you would normally use
- bv.litchwan2.net
- clearpass.bv.litchwan2.net
These are all variations of the server certificate imported into ClearPass. Different versions and devices may behave differently...
These entries did not work in the domain field:
- litchwan2
- test
- litch
- clearpass
References
https://letsencrypt.org/certificates/https://letsencrypt.org/2020/12/21/extending-android-compatibility.html Excellent description of the problem and the workaround
https://community.arubanetworks.com/community-home/digestviewer/viewthread?GroupId=7&MessageKey=8aacb717-781f-4794-a2b0-e103b94e473c&CommunityKey=867d34a4-1e6a-4b9a-9349-df7b4c84b3cb&tab=digestviewerhttps://www.reddit.com/r/GooglePixel/comments/mngy9k/pixel_phone_not_connecting_to_enterprise_wifi/ Pixel Phone not connecting to Enterprise WiFi after Android OS Dec 2020 update
------------------------------
Richard Litchfield
Airheads MVP 2020, 2021
------------------------------