If you're doing ping testing between hosts belonging to the same VLAN...no IP routing is involved since hosts communication happens at Layer 2...so, if I were you - considering you see Ping's high latencies (and also strange variations) when CPU is lightly loaded - I would check hosts sides (did it happen if you change hosts? are there OS's Firewall apps that can interfere? Which OS your two hosts are running on? and so on...).
P.S.
Is your Aruba 2930M currently updated?