I've done something similar with Endpoint attributes to add timestamps and blacklist/whitelist values. You can also use something like the "Unique Device Count" attribute that is created for Guests to create an incrimental value of failed authentications.
Create an Enforcement Profile that increases the failed number of authentications.
Create an Enforcement Profile that includes an Entity Update for "Blacklisted = True", which is assigned under the condition the authentication fails x number of times.
Create an Enforcement Profile that has a time entry for Blacklist time or Reset time, and use it as needed.
You will need a combination of role mapping, enforcement policy, and enforcement profiles to make this happen.