Please read these instructions before doing this.
What you want to do is turn on Control Plane Security and Auto Cert Provisioning under Configuration> Network> Controller. This will cause all of your APs to reboot for 8 to 10 minutes and get certificates. After you do that, make sure that all of your APs came back up. After they come back up, uncheck Auto Cert provisioning.
After you do that, on the same page will be a list of your "authorized" APs. You can delete any AP that you do not want to come back up and they will not come back until you put a checkbox back into Auto Cert provisioning.
You will want to do the initial procedure after hours, because it will involve 10 to 15 minutes of downtime.