Do you have QoS classifiers for prioritizing outbound packets set on the 5300? If so, read below; if not, your best bet would be to open a support call with hp serivices to address the problem. This is not something I've heard of outside of QoS classifiers..
There are 7 QoS classifiers (packet criteria) for traffic prioritization evaluated in the following order of precedence:
1) UDP/TCP Application Type (port)
2) Device Priority (destination or source IP address)
3) IP Type of Service (ToS) field (IP packets only)
4) Protocol Priority (IP, IPX, ARP, DEC LAT, AppleTalk, SNA, and NetBeui)
5) VLAN Priority
6) Incoming source-port on the switch
7) Incoming 802.1p Priority (present in tagged VLAN environments)
If the switch is configured with multiple classifiers that address the same packet,
the switch uses only the QoS configuration for the QoS classifier that has the
highest precedence.
Please review Knowledge Base doc id KBAN00001146 or page 8-9 8-10 of the 5300XL advanced traffic management guide:
ftp://ftp.hp.com/pub/networking/software/Advanced_Traffic-5300xl-3400cl.pdf