OK let's do a bit of verification that you have the same issue.
Please go to each controller and check "show ha hearbeat counters" and see if any were missed
while you had the feature turned on. The stats should still be there as long as you have not reloaded.
On one of the APs, do a "show ap debug system-status ap-name XXX | begin "HA Failover Information"
See if you have a line like this there matching the same time as the APs moved:
2014-09-24 15:10:03 Failover request from standby: fail-over to 10.5.5.81
Do you have entries in the syslog from about that that time like this and also at other
times?
sbHeartbeat: PAPI RxPacketFromSibyte: ACK to invalid packet SN = 0x0000a36e opcode=0x6
We also have to rule out that you had actual packet loss. I know you check the latency, but
did you also check for queueing drops?