I know how you feel about leaving remnants, I'm the exact same. My method is rolling back to an earlier snapshot, but that's less possible when it's a production setup instead of a test lab setup.
================
I'll expand more on the 2nd part.
AMP Setup -> General -> Historical Data Retention -> Archived Device Configs = count of configs kept (clean up happens at nightly maintenance)
AMP Setup -> General -> Device Config Audit Interval -> this governs the periodity of Audits, each audit has the potential to create a new backup config if there are changes
*ignore backup interval since that's the complete AMP backup interval
Group -> select group -> Basic -> Aruba -> Archive Controller/Switch Backups = yes/no
This is the flag that enables keeping backups for the devices in that group tree. When this flag is set to yes, the backups kept are only if there's been a detected config change at audit time. And they get pruned to the config count that's in AMP Setup
There's a feature request out there to take the backup no matter if there's config changes or not, but it'd need more votes to make it happen.