What you are trying to do cannot be done in the method you are describing. The controller discovered during boot will be the point of configuraiton for the AP; the AP Group (through the AP System profile) will then instruct it where to terminate. It is possible to setup your enviornment in this redundant setup, however it will require the creation of multiple AP Groups and AP System profiles to support the primary and backup LMS addresses; even though it sounds as though that is what you are trying to avoid.
To do what you want, I think you'd need at least 12 AP Groups.
Group 1 - 16 APs - Primary LMS = Controller 1; Backup LMS = Controller 2
Group 2 - 16 APs - Primary LMS = Controller 1; Backup LMS = Controller 3
Group 3 - 16 APs - Primary LMS = Controller 1; Backup LMS = Controller 4
Group 4 - 16 APs - Primary LMS = Controller 2; Backup LMS = Controller 1
Group 5 - 16 APs - Primary LMS = Controller 2; Backup LMS = Controller 3
Group 6 - 16 APs - Primary LMS = Controller 2; Backup LMS = Controller 4
Group 7 - 16 APs - Primary LMS = Controller 3; Backup LMS = Controller 1
Group 8 - 16 APs - Primary LMS = Controller 3; Backup LMS = Controller 2
Group 9 - 16 APs - Primary LMS = Controller 3; Backup LMS = Controller 4
Group 10- 16 APs - Primary LMS = Controller 4; Backup LMS = Controller 1
Group 11 - 16 APs - Primary LMS = Controller 4; Backup LMS = Controller 2
Group 12 - 16 APs - Primary LMS = Controller 4; Backup LMS = Controller 3