I had an integration previously with Cisco ASA for Anconnect and OnGuard posture health check.
I had a OU Group in Active Directory for VPN.
This is what i configured and it works:
1. The Service Overview
Role.
Enforcement.
You can test it directly from ASA using CLI with the following command:
test aaa-server authentication <SERVER-Name> host <IPAddress of the Server> username <username> password <Password>
Also, enable the debug in ASA firewall to check if CoA is working.
Also, try Changing the Type of Service "RADIUS Enforcement (Generic)" to "Cisco Web Authentication Proxy".
For Rules, Apply:
Type: RADIUS:Cisco-ASA
Name: ASA-TunnelGroupName
Operator: CONTAINS
Value: ANYCONNECT