This is strange.. the MAC should not be moving around a lot.
With standard settings (TLB) NIC teaming sends packet from all team ports, but with the original MAC-addresses. So you should see separate MACs on each port, but with the same source IP.
Did you change the default TLB settings? Which particular settings are you using?
NFT will move the mac, but only when there is a failover, which hopefully doesn't happen that often.
You should, as you say, not need any special config for TLB.