Do you see your VLAN going up and down? If it is not trunked to another switch, there are no active local ports on the controller in that VLAN and no users using it, the VLAN may go down/down. That may be causing the decrement in the priority and VRRP switch (if you have preempt enabled).
You can force a VLAN to stay up using the "operstate up" command under the "interface vlan x" config.