Security

Hi All, 

 

Hoping the community can assist me with a weird problem i have found. 

 

I have a CPPM 6.4.2 setup performign EAP-TLS for Wired /Wireless clients (Windows 7 ) using certificates. 

 

The problem i have is that the initial connection attempt fails to complete across multiple controller / cisco switch platforms. This shows up as a timeout on clearpass (client did not complete EAP transaction) and an error on the client. 

 

I've narrowed it down to the fact that the custom EAP-TLS method we're using in ClearPass did not have session resumption enabled. I've spent days going over packet captures and have found that every one of the fail cases was where the client was sending the EAP-TLS Client-Hello message with a 32 bit session ID. 

 

The response form ClearPass appears to be normal in that it responds with a nulll session ID. 

 

From here i ran RAS logs on windows and found that at the end of the handshake the following error occurs:

 

pTlsMakeMessage(host/computer.domain.com)
[11328] 02-10 12:15:07:355: >> Received Request (Code: 1) packet: Id: 10, Length: 13, Type: 13, TLS blob length: 0. Flags:
[11328] 02-10 12:15:07:355: EapTlsCMakeMessage, state(2) flags (0x3410)
[11328] 02-10 12:15:07:355: MakeReplyMessage
[11328] 02-10 12:15:07:355: SecurityContextFunction
[11328] 02-10 12:15:07:355: InitializeSecurityContext returned 0x80090326
[11328] 02-10 12:15:07:355: Returning error -2146893018
[11328] 02-10 12:15:07:355: State change to RecdFinished. Error: 0x80090326
[11328] 02-10 12:15:07:355: BuildPacket
[11328] 02-10 12:15:07:355: << Sending Response (Code: 2) packet: Id: 10, Length: 6, Type: 13, TLS blob length: 0. Flags:
[16496] 02-10 12:15:07:371:
[16496] 02-10 12:15:07:371: EapTlsMakeMessage(host/computer.domain.com)
[16496] 02-10 12:15:07:371: >> Received Request (Code: 1) packet: Id: 11, Length: 10, Type: 13, TLS blob length: 0. Flags: L
[16496] 02-10 12:15:07:371: EapTlsCMakeMessage, state(4) flags (0x3400)
[16496] 02-10 12:15:07:371: Unexpected code: 1 in state RecdFinished

 

At the same time as this the windows CAPI2 event logs show a repeating error similar to this:

 

- UserData

- CryptCATAdminEnumCatalogFromHash

- CATQueryInfo

[ nextEnum] true
[ hash] 387E2E76C44D6BA72B38C14C4E69A2CDC8718842

- EventAuxInfo

[ ProcessName] wmpnetwk.exe

- CorrelationAuxInfo

[ TaskId] {D042E902-4537-4A2A-B11B-07E49382D159}
[ SeqNumber] 1

- Result Element not found.

[ value] 490

 

 

The client then fails to respond to the TLS Certificate request and doesn't send it's user certificate hence the session timeout occurs. 

 

After this the client restarts authentication attempt with Client Hello showing TLS session ID of 0. The process the repeates and authentication is successful. 

 

When i enable session resumption the problem seems to go away. 

 

This leaves me with 2 main questions:

 

1) Is this normal client behaviour when session resumption is not enabled (i'm thinking no) 

2) Is there any practical / valid reason not to use session resumption for EAP-TLS. 

 

Scott

 

 

 

 

We're going to have to go back to basics on this one:

 

- What CA issued the EAP-TLS certificate?

- Does ClearPass have the CA that issued the EAP-TLS certificate in its trusted CA's?

- Did this ever work?

- Are you doing anything like OCSP or authorization in your EAP-TLS method?

- Did you ever get this to work in a "bare" EP-TLS method?

 

If it never worked "bare" then you need to go back and check all of your steps.  Session resumption only works to resume an existing session and has nothing to do with the initial connection.

 

Hi Colin,

 

- What CA issued the EAP-TLS certificate? - Enterprise trusted windows PKI. Group policy publishes all intermediate and root CA's. 

- Does ClearPass have the CA that issued the EAP-TLS certificate in its trusted CA's? Yes full chain installed and enabled. 

- Did this ever work? - This was "tested" a few months ago but we're only now starting to actually use it. i suspect this has been present all along with the custom authentication method in use. 

- Are you doing anything like OCSP or authorization in your EAP-TLS method? No OCSP or Authorisation. 

- Did you ever get this to work in a "bare" EP-TLS method? Yes if we use defaulth EAP-TLS method this issue doesn't happen. 

 

Authentication is successful for all but the first authentication attempt. The retry is always successful. 

 

Initial testing has shown that by enabling the session resumption feature the windows client is happy and doesn't timeout on the first request. 

 

i'm heading to my lab tomorrow morning to capture the working wireshark trace to confirm the differences.

 

at a packet level, the only difference between working and failed EAP handshake is the 32 bit session ID in the client hello.

 

I have had this with the TAC for 3 weeks and haven't had any progress until i found this yesterday. 

 

Well, how is your eap-tls method different from the default? That is key here. You are omitting that and that is the most important part. Why hide that?

Colin, our custom EAP TLS Method has OCSP / authorisation disabled. 

 

 

 

Can I assume you have "fast reconnect" enabled on your Windows 7 systems?    When enabled on the client, the client will send its sessionid to see if it is present/cached on the authenticator.  This is typically seen on roams.   If CPPM is not set for session resumption, this attempt will fail.   The client will then retry authentication.

 

If you have fast reconnect on, you'll want the session resumption feature enabled within your EAP-TLS source.    I recommend you enable both.

Hi Clembo, i don't have a fast reconnect option for EAP-TLS, only for EAP-PEAP-MSCHAP

 

What i am seeing does fit that theory though. The client seems to detect that the server doesn't support resumption (server returns null for session id length in hello message).

From the RFC: 

 

"

3.4.  Client Behavior: Initial Handshake

Note that this section and Section 3.5 apply to both full handshakes
   and session resumption handshakes.

   o  The client MUST include either an empty "renegotiation_info"
      extension, or the TLS_EMPTY_RENEGOTIATION_INFO_SCSV signaling
      cipher suite value in the ClientHello.  Including both is NOT
      RECOMMENDED.

   o  When a ServerHello is received, the client MUST check if it
      includes the "renegotiation_info" extension:

      *  If the extension is not present, the server does not support
         secure renegotiation; set secure_renegotiation flag to FALSE.
         In this case, some clients may want to terminate the handshake
         instead of continuing; see Section 4.1 for discussion."

 

The way the session drops seems messy from the windows side. particularly as i don't seem to have a client side option to turn of session caching (that i can see) .

 

This happens each time the client disconnects and reconnects to WLAN / Switch port. 

You are correct, there is no option to enable/disable this for EAP-TLS.   The following is from RFC 5216 (The EAP-TLS Authentication Protocol).   As part of the protocol, the client will sends its sessionID.    Was there a specific reason you disabled it on your EAP-TLS authentication method (it is enabled in the default EAP-TLS methods)?

 

2.1.2. Session Resumption

   The purpose of the sessionId within the TLS protocol is to allow for improved efficiency in the case where a peer repeatedly attempts to authenticate to an EAP server within a short period of time.  While this model was developed for use with HTTP authentication, it also can be used to provide "fast reconnect" functionality as defined in Section 7.2.1 of [RFC3748].

   It is left up to the peer whether to attempt to continue a previous session, thus shortening the TLS conversation.  Typically, the peer's decision will be made based on the time elapsed since the previous authentication attempt to that EAP server.  Based on the sessionId chosen by the peer, and the time elapsed since the previous authentication, the EAP server will decide whether to allow the continuation or to choose a new session.

   In the case where the EAP server and authenticator reside on the same device, the peer will only be able to continue sessions when connecting to the same authenticator.  Should the authenticators be set up in a rotary or round-robin, then it may not be possible for the peer to know in advance the authenticator to which it will be connecting, and therefore which sessionId to attempt to reuse.  As a result, it is likely that the continuation attempt will fail.  In the case where the EAP authentication is remoted, then continuation is much more likely to be successful, since multiple authenticators will utilize the same backend authentication server.

   If the EAP server is resuming a previously established session, then it MUST include only a TLS change_cipher_spec message and a TLS finished handshake message after the server_hello message.  The finished message contains the EAP server's authentication response to the peer.

 

Hi Clembo,

 

As to why it was disabled, thats a mystery i'm trying to resolve. This particular customer used Aruba professional services to do the configuration so this was a change made by the engineer who built the system originally. I am waiting for the TAC to advise on whether there is a legitimate reason to disable this feature. Otherwise the custom service was built to allow us to change OCSP settings in the method. 

 

It seems like WIndows is just being difficult for the sake of it then (really?)

 

I just did some more testing and here is what i found:

 

Windows 7 - Client always attempts to resume SSL connection and fails first attempt if session resumption not enabled in custom EAP-TLS Method. Disconnect / Reconnect of client triggers same behaviour.  If session resumption is enabled all attempts succeed. 

 

Mac OSX 10.9.5 - Client never attempts to resume TLS session, success with session resumption enabled / disabled

 

Cisco 7965 IP Phone - client never attempts to resume TLS session, success with session resumption enabled / disabled. 

 

Scott

After lots of testing, it seems there is something buggy about windows 7 EAP supplicant. In all of my test cases, the Windows client always starts an EAP-TLS session by sending a Client Hello TLS Message containing a session ID. When the server replies without a session ID, the RFC states that the client is supposed to establish a new session. There are multiple extensions to the EAP-TLS RFC's that suggest that a client can choose to terminate the session if a server is not capable of session resumption. This seems to be happening however Windows 7 doesn't generate the necessary SSL alerts in this case. I'm still at a loss as to why Windows 7 always tries to resume a session during 802.1x authentication. MAC OSX and Cisco IP Phone do no exhibit this behavior. In the end we have decided to enable Session Resumption on ClearPass. We are still awaiting testing from Aruba engineering to determine if there is anything wrong with the EAP exchanges with this feature disabled.

Aruba TAC has confirmed this behaviour. Windows 7 for some reason will always send a session ID in an EAP-TLS request and in the event that the server does not support resumption, will terminate the session and restart a new request with no session ID. This shows us as a "Client did not complete EAP" log on access tracker and will be recorded as a timeout. This can be overcome by enabling Session Resumption in the EAP-TLS method.

Hi, I am also facing this issue, but even i set EAP-TLS Session Resumption enabled, but still after the NAD(switch timeout 5minutes), then the session cannot resumpt.

My case is in front of PC there is one LLDP-MED voip-phone connected to switch.

if without voip-phone, I do not face this problem. 

i  did one more test scenorio.

remove the voip phone, and insert one switch, then wait for switch timeout 5minutes, do a plugin cable again, it shows no EAP-TLS authentication timeout, straightly the PC can get IP and authenticated.

So I suspect the voip phone cause the TLSv1.2 handshake from PC(client hello) not successfully send out.

the packet capture shows PC not send out client hello, but root cause could be related with voip phone.

Guo Gang - Please open up a TAC case.