I apologize if you are seeing multiple posts by me on this topic. My two previous attemps were about an hour ago and I still do not see them in here.
I have about 50 ProCurve switches (2520,2530, and 2610) that I have spread across our branches throughout North America. The branches are connected by a WAN utilizing BGP. We have an Avaya IP Office VoIP phone system that has had problems since it was implemented over a year ago. In the past I had issues with dropped calls, callers not being able to hear us, and garbled speech. I seem to have mostly resolved that by implementing QoS statements to mark all our traffic on our voice VLAN (200) as well as specifying the voice VLAN with the "voice" command. That was implemented about a month ago but some problems persist. Our problems now all seem to have to do with Signaling and result in the following:
Slow to get dial-tone resulting in needing to wait several secondsPhone continues to ringing for several seconds even after handset is picked upOur staff occasionally hearing echoes of their own voicesOur staff occasionally report delayed audio that results in our staff's voices being delayed 5-10 seconds in being heard at the other end of the callOur Network/WAN provider determines what to prioritize for Voice based on packets are marked EF (46) and packet captures indicate that the Signaling packets coming from our phones are not being "tagged" as EF. RTP and UDP appear to be tagged properly. Since the implementation of the QoS commands and defining the voice VLAN with the "voice" option the RTP and UDP traffic is being marked properly but not the TCP signaling packets. I am at loss here. Any help is appreciated.
In order to reduce costs our branches do not have routers. Our WAN/Network provider implemented bridges at all locations aside from the Corporate and DR NOCs. I will paste a typical config below as well as attach some screenshots from WireShark, but first:
Yes, I am familiar with the document Interoperability between Avaya IP phones and ProCurve switches We do not have a spanning-tree network and our network seems to be simpler than the one described in that document. I focused on the statements in the section for the ProCurve 2610 at the end of the document.VLAN 200 = voice VLANVLAN 50 = data VLAN (anything non-voice)172.20.2.120 = DR Avaya IP Office Server (branch offices utilize this as primary)10.1.200.2 = Corporate Avaya IP Office Server (primary for corporate office and secondary for branches)INT 26 is where the switch uplinks to the bridge ("MIB", as our provider calls it)I have enabled QoS type-of-service as diff-servicesI use a bunch a QOS DEVICE-PRIORITY statements the first two of which are for our Avaya IP Office servers. I am not sure whether the remiaining statements make any difference for prioritizing non-voice traffic but implementing these statements (and subsequently removing them so as to be sure) resulted in no difference in Signaling packets being "tagged" for EF
hostname "OLB-ProCurve2610B"time timezone -360time daylight-time-rule Continental-US-and-Canadamirror-port 1no web-managementweb-management sslno telnet-serverinterface 2name "SWOBFILE"exitinterface 23name "OLB-ProCurve2610C Uplink"exitinterface 25name "OLB-ProCurve2610A Uplink"exitinterface 26name "MIB Uplink"speed-duplex 100-fullexitip default-gateway 10.16.200.1timesync sntpsnmp-server community xxxxxxxxsnmp-server community xxxxxxxxvlan 1name "DEFAULT_VLAN"untagged 25,27-28no ip addresstagged 1no untagged 2-24,26exitvlan 50name "Data"untagged 1-22,24ip address 10.16.50.3 255.255.255.0qos priority 5tagged 23,25exitvlan 200name "Voice"untagged 26ip address 10.16.200.3 255.255.255.0qos dscp 101110tagged 1-14,16-25voiceexitinterface 18monitorexitqos device-priority 172.20.2.120 dscp 101110qos device-priority 10.1.200.2 dscp 101110qos device-priority 10.1.10.2 priority 5qos device-priority 10.1.10.3 priority 5qos device-priority 172.20.2.102 priority 3qos device-priority 10.1.10.102 priority 5qos device-priority 10.1.20.161 priority 5qos device-priority 10.1.10.170 priority 5qos device-priority 172.20.2.180 priority 5qos device-priority 10.1.10.5 priority 3qos device-priority 10.16.50.10 priority 3qos device-priority 10.1.10.17 priority 3qos type-of-service diff-servicessntp unicastsntp 30sntp server 10.1.10.2ip ssh
Can the Avaya PBX and its phones be configured, to send out signaling packets (TCP 1720 and maybe other H323 ports like 1719) with DSCP value set to standard AF31 or CS3?If this isn't the case, switches should be configured to remark DSCP values and assign proper COS in VLAN tag.
The Avaya server is already configured to mark both Signaling (TCP) and Voice (UDP/RTP) packets as EF. Maybe the Avaya server and/or phones are not marking data properly but shouldn't the ProCurve be marking all VLAN 200 traffic as EF, regardless? Is there a flaw in my ProCurve config?
Voice signalling is not generally tagged as EF. The breakdown on the LAN should be:
EF (46) = voice packets = IP precedence 5AF41 (34) = video packets = IP precedence 4AF31 (26) = signalling = IP Precedence 30 = everything else = 0
Your WAN provider should offer at least 4 classes of service.
Your task is to(a) ensure your packets are tagged correctly(b) negotiate with the WAN provider to prioritise as per your tags into the 4 classes of service, so that Voice packets are not competing with anything else, and signalling is given priority over any non-realtime packets (voice & video).
By the sound of things, it does sound like your signalling isn't getting prioritised.
The qos device-priority statements you have are setting the layer-2 QoS on packets. I don't think this is useful, as the IP address doesn't tell you what the traffic/application is - if you need to re-classify your frames (because the voice devices aren't tagging properly) then you need to classify them based on TCP/UDP ports. *Then* you also have to check your 802.1p-DSCP mappings to make sure that whatever layer-2 QoS you have on packets are being mutated to the correct Layer-3 QoS values. (eg, qos dscp-map 101110 priority 7) (That's EF--> 7)
I don't know what the qos dscp ... commands you have on your VLAN interfaces are going to do for you. Probably nothing?
Your command qos type-of-service diff-services probably doesn't do anything either - I think this command is used to enable you to go on to re-mark packets, eg, qos type-of-service diff-services 001010 dscp 011010. (That's AF11 --> AF31).
On the whole, it is much easier to ensure the endpoints are correctly tagging, and configure the network switches to trust the endpoint tagging. The phone config should include two separate QoS values, 46 for RTP and 26 for signalling.
If you can't get your endpoints to mark QoS properly, then you should use a QoS classifier, something like:class ipv4 VOICE 10 match udp 10.1.1.0 0.0.0.255 range 61000 64000 0.0.0.0 255.255.255.255 20 match udp 10.1.1.0 0.0.0.255 0.0.0.0 255.255.255.255 eq 6123policy qos tag-VOICE class ipv4 VOICE action dscp efvlan 200 service-policy tag-VOICE in
Then create another classifer to ensure all your signalling gets AF31.
Then create another classifier to ensure everything else is set to 0. This could be important.
I have been asked by customers to classify all packets on ingress like this before. It's a pain, because everytime they change something, the policy needs editing, and they have trouble finding anybody that understands it well enough to make the changes.
I can see what you are saying about my marking packets for Layer-2 and not for Layer-3. I do know that many of the voice-quality issues were resolved by implementing the VLAN 200 VOICE command, though, because those issues do appear to have gone away and packet captures confirm that UDP/RTP traffic is properly marked across the WAN.
I am not sure but the last set of commands for setting a QoS policy appear to be for Cisco switches. I need to figure out how to do that on my ProCurves 2520, 25230, and 2610 switches.
The statement qos dscp-map 101110 priority 7 is not shown in my config but telling the switch to show the existing dscp mappings confirm this is mapped properly. Documentation on these ProCurves confirm that is a default.
Thanks so much for the help.
Looking at my packet captures from one of my phones I can see the TCP destination port for Signaling is 1720 and UDP destination port for Voice is in a range of 49152 to 53246, so I have added the following statements:
qos tcp-port 1720 dscp 101110qos udp-port range 49152 53246 dscp 101110
Now, it makes sense that my previous QoS statements within my VLANs only applied to Layer 2. However, I have a concern: my phones allow PCs to piggyback through them and we use that extensively. PCs are on VLAN 50, so won't anything on the ports I defined be marked as dscp 101110? If so then this will result in some non-voice related data being inappropriately prioritized. How do I limit the the TCP and UDP ports I listed above to only apply for traffic to/from 172.20.2.120 and 10.1.200.2? I already had the following statements in my config:
qos device-priority 172.20.2.120 dscp 101110qos device-priority 10.1.200.2 dscp 101110
If there is no way to do the above then how about I remove the statements about the TCP and UDP ports, leave the qos device-priority statements in, and change the qos type-of-service to to ip-precedence? At this point I am fine with getting Signaling and Voice prioritized at the same level simply because our MPLS provider says this is how they do it for their customers as well as their own hosted phone services.
qos device-priority 172.20.2.120 dscp 101110qos device-priority 10.1.200.2 dscp 101110qos type-of-service ip-precedence
As your problems with the voice packets went away as soon as you applied the "voice" command to the voice VLAN (trusting the tag set by the endpoints) it would seem the obvious answer is that your WAN provider is looking for "46" but not looking for the signalling (presumably 26).
So you can see how very easy it all is if you just configure the endpoints to tag, and the network to trust.
In your position, I'd be unwilling to allow my WAN provider to lump in all my signalling with the voice traffic, just on the grounds of professionalism.If you want the path of least resistance - can you not just change the phone config (probably being served up via a DHCP option? Simple as editing the DHCP vendor option to change "26" to "46".)? Much easier than reclassifying everything on the switch.Here is what Avaya says about this:
(NOTE: There are many other values that can be entered under your option 176 and option 242 such as L2QAUD and L2QSIG but those can just as easily be set in your 46xxsettings.txt file)
With hindsight, it would be nice if your vlan 200 qos dscp 101110 was working. This operates on *outbound* packets, so I am guessing it will only work on a switch where the traffic is being routed out of the VLAN.
I'm sorry my brain went into Cisco-mode yesterday. I know how that happened: I've never had to actually think about QoS on HP networks (or Nortel/Avaya) - it's always been a no-brainer and just worked for me, it's only on Cisco networks where I've had to spend weeks and weeks fiddling with things to get it working.
My DHCP scope option 242 on the Voice VLAN was already as follows:
Based on what you said I changed it to this:
I told the phone to clear itself which caused it to reboot and pull all settings down again but the following
Maybe the config txt file takes precedence of anything assigned within the DHCP scope?
Within the Avaya IP Office Manage (the administration console for the entire Avaya system) the Diffserv settings are:
The settings here appear to be right for both Hex and Decimal. The only thing I am not sure about is the DSCP Mask.
OK. Some important new info:
When I said I didn't see the packets marked properly as EF I was monitoring the switchport that her phone was plugged into. I guess that reflects "raw" data coming from the phone prior to any massaging by the switch. When I monitor the uplink port to the bridge and filter for her phone's IP address it appears that the traffic is marked correctly. Thus, the problem here is that the TCP Signaling packets are being stripped stripped of EF outbound on the uplink while RP Voice is fine. I know this because of packet captures provided by my MPLS provider.
This may have something to do with whether the uplink port on the switch is tagged or untagged. The bridge requires the port be untagged. Setting the port to tagged stops traffic on the port, at least in one of the directions.
A document I have found on ProCurves says the following. It is a little difficult for me to follow but the No Tagged VLANs not being able to carry the 802.1p Priority Assignment on downstream devices has me concerned. I say this because I am assuming that outgoing packets are heading downstream to the bridge.
Outbound Packet Options Tagged VLANs No Tagged VLANs
Control Queue Priority for Packet Types Yes Yes
Carry the 802.1p Priority Assignment to Next Downstream Device Yes No
If an outbound packet is in an 802.1Q tagged VLAN environment (that is, if the packet is assigned to a tagged VLAN on the outbound port), the packet carries an 802.1p priority setting that was configured in the switch. This priority can be used by downstream devices having up to eight outbound port queues. Thus, while packets within the switch move at the four priority levels, they still can carry an 802.1p priority that can be used by downstream devices having more or less than four priority levels. Also, if the packet enters the switch with an 802.1p priority setting, QoS can override this setting if configured to do so.
If a packet is not in an 802.1Q tagged VLAN environment, the QoS settings control only to which outbound queue the packet goes, and no 802.1p priority is added to the packet for use by downstream devices. But if the packet is in an 802.1Q tagged VLAN environment, the above setting is also added to the packet as an 802.1p priority that can be used by downstream devices and applications, as indicated in the next table. In either case, an IP packet can also carry a prioritization policy to downstream devices by using codepoint-marking in the ToS byte.
Looking on the bright side, this simplifies the problem!You don't have to care about L2 Qos (CoS) settings since provider edge simply doesn't carry them. All you have to do is focus on proper L3 QoS (DSCP) markings.
Yes, 802.1p is not useful for this issue.
Just to clarify: 802.1p tells the *switch* which switch buffer to put each *frame* in when it is *switching* them.
As long as you don't have any switch interfaces running at 80%+, and consequently dropping frames, then .1p isn't doing anything for you.
Your concern is Layer3 QoS, which is how the *packets* are *routed*. And this is only important on the contended infrastructure belonging to the WAN provider.
Your packet captures seem to be indicating you have everything tagged correctly. Sound like something somewhere is (correctly) not agreeing with your DSCP marking "46" being applied to signalling packets.
Ultimately, the cause of your issues is the failure of your WAN provider to offer the standard class-of-service split for traffic that I would expect them to.
© Copyright 2023 Hewlett Packard Enterprise Development LPAll Rights Reserved.