Hi Folks,
I may have the same issue(s), so I've been lurking! :D I've got 5 x 515s and 2 x 505Hs at 8.8.0.1, and there are complaints about coverage and low signal strength from close-by APs.
One thing I'd like to point out is there are two aspects to consider when troubleshooting WiFi issues. The signal quality at each client from any/every AP they could access, and the APs' ability to connect to the Internet.
If your VC is just "in-house", then never mind, but it we're talking Google, Zoom, Teams, Webex, etc. you need consistent, predictable Internet speeds.
If your client has a good connection - say a 1 Gbps Comcast Business link, you need to make sure you (or the customer) knows what they've really got provisioned with the ISP. A hardwired PC connected to the same switch / router as your APs will help you check your speed (speedtest.com, and each ISP has their own speed test if your connected to their networks...so try this too!). Check it before others get to the site for the day, and check it just after people get there on a Monday, at noon, etc.
This is VERY important, as most vendors, especially ISPs that are using DOCSIS-based cable data connections, are going to NOT give you synchronous up and down speeds. You may have a 1Gbps (1,000 Mbps) download speed, but you may have signed up for only 20 Mbps upload speed! (Hello Comcast, Hello Cox, Hello RCN, etc.) Video conferencing is quite picky when it comes to upload speed. Both the upload speed and the "jitter" (the variations in speed over time) are very important for VC!
Some VC clients, like Google Meet or Zoom, have real-time data on a call so if you're hosting a meeting, get these real-time data to get reports of bandwidth and jitter as the call is underway.
So if you're trying to connect, say 5 or 10 users to a Google or Zoom VC session, you may need 5 or 10 times some minimum upload speed, so if you can get people to connect from conference rooms that would help reduce issues at people's desk - plus it would give you control of the conferencing hardware in a central location, vs "all around the campus" on local PCs/iPads.
Additionally, some kind of SNMP manger could be used, whether pricey like SolarWinds, or cheap like PRTG or What'sUpGold. Plot the bandwidth of the internet by having SNMP check the "closest" path to the Internet. This won't the your ISP's router, but it may be the edge router of your company, or the cable-modem / router that connects you to the Internet.
Then, you can see over a period of days, weeks, or months what that site's total inbound and outbound bandwidth is. This will NOT be the maximum your connection will take. It will be the day-to-day amount that is actually getting consumed. If it seems you're using 70% of the available bandwidth for long periods of time - especially during office hours (backups running at night will consume a lot anyway), you way want to get a quote for more speed from the ISP.
I must say, even if you have no ongoing issues with tickets for VC, it will benefit you to have a history of the network's performance. For that one time that you're boss's boss asks you in the elevator - "Why could I remote access into the Citrix cluster last night? Was it OK?" Or even for piece of mind when you come back from vacation! Get an SNMP manager!
Oh, and check if your hardware supports Quality of Service - QoS. Sometimes called Class of Service. Any good router will support some kind of prioritization for traffic and help you drop stuff that's less important, or more tolerant of a few dropped packets.
Please let us know how you get on with your issues!
Regards,
-Ambi
------------------------------
Lawrence Brandt
------------------------------
Original Message:
Sent: Sep 29, 2021 07:28 AM
From: Ville Heikkinen
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi,
Yea, noticed that when going through release notes and decided to use 8.6.0.12.
We haven't had any issues so far, hopefully this problem will be fixed in other major versions (8.7.x.x & 8.8.x.x) as well.
------------------------------
Ville Heikkinen
Original Message:
Sent: Sep 02, 2021 08:34 PM
From: Ariya Parsamanesh
Subject: Troubleshooting intermittent poor throughput on AP-515
thats great. also note that the latest firmware in this sw train is 8.6.0.12
Check the release notes for details.
------------------------------
Any opinions expressed here are solely my own and not necessarily that of Hewlett Packard Enterprise or Aruba.
Original Message:
Sent: Sep 02, 2021 10:04 AM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
I'm happy to say that our absence of WiFi tickets regarding dropped videoconferences continues since rolling back to 8.6.0.9. My office is lightly populated of late, but I think I have enough hours/week of VC going on that if this was still an issue, it would have been impossible to never get a report of it recurring.
cathyf (sorry, the @ failed me) the idea of filling the space with a similar hospitality model is solid. I might just have to do that.
------------------------------
Steve Yuroff
Original Message:
Sent: Sep 02, 2021 05:12 AM
From: Ville Heikkinen
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi all,
We have seen exactly the same behaviour with one IAP-515 and 8.8.0.0 firmware. Luckily we got some extra 315s in our stock and were able to replace the problematic one with 315. I will downgrade the firmware to 8.6.0.9 today and if it works, I'll owe you beers.
------------------------------
Ville Heikkinen
Original Message:
Sent: Sep 01, 2021 11:35 AM
From: Cathy Fasano
Subject: Troubleshooting intermittent poor throughput on AP-515
This may be a crazy idea, but have you considered something like buying a 305H off ebay or another reseller? It looks like "open box specials" can be had for <$200, and used for <$100. This has the advantage that you can just drop it in the spot that you want the 505H to go, and that might tell you if something else is going on. And it's not bad to have a spare or two around...
------------------------------
Cathy Fasano
Original Message:
Sent: Jul 19, 2021 09:59 AM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi John,
I am pleased to say that since moving to 8.6.0.9, we've ended the inflow of "my videoconference suddenly dropped" tickets.
Only bummer is that it renders my 505H standalone, and that device can only run a firmware where we'll still have this experience. I've repositioned APs to adapt to removing it from the floorplan, and am pinning my hope that someday it will get a fully stable firmware, exempt from all drama.
------------------------------
Steve Yuroff
Original Message:
Sent: Jul 19, 2021 02:03 AM
From: John Cirocco
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi SWY, how has performance been on 8609, has it resolved the intermittent slowness?
------------------------------
John Cirocco
Original Message:
Sent: Jul 09, 2021 10:14 AM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi Jochem,
I appreciate knowing this is an option, thanks for pointing it out.
My reluctance to follow through on this option is that I don't trust 8.7.1.1, because that's where I started this cluster at, and it's had problems since deployment. I'm cautiously optimistic that 8.6.0.[9|10] is a happy place, and feel like that's an uncomfortable risk to have one standalone on a firmware that has proven itself to lead to Ticket Generating Events.
I'm rather surprised that Aruba firmware has so many troubling behaviors, and product that I bought over 6 months ago can't actually be used on proven, stable "Conservative" firmware.
------------------------------
Steve Yuroff
Original Message:
Sent: Jul 09, 2021 04:34 AM
From: Jochem Knoben
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi,
you could force that 505H to run as "standalone" AP utilizing a 8.7.x firmware version (AP can run then even in same mgmt-vlan as the "cluster AP's")
Of course - you would need to manually copy the config to that device and keep the configs manually in sync.
And as soon as a stable working 8.7.x or higher firmware will be available join that AP to the cluster again.
Groetjes
Jochem
------------------------------
Jochem Knoben
Original Message:
Sent: Jul 08, 2021 09:13 PM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Good news: we've gone 3 days with only 1 user complaint about wifi connectivity, and even received kudos when feedback was solicited.
Bad news: the one user complaint was about coverage. That happened because one can't run a 505H on firmware 8.6.0.9, the minimum system requirements for a 505H is 8.7.0.0, leading to a predictable coverage gap.
The Aruba TAC did not advise that we'd loose an AP from our cluster by making this change... not impressed. TAC should note the incompatibility of the plan vs hardware.
So I think I have a stable system... and a paperweight. That feels a little short of success.
------------------------------
Steve Yuroff
Original Message:
Sent: Jul 02, 2021 10:11 PM
From: John Cirocco
Subject: Troubleshooting intermittent poor throughput on AP-515
I understand your frustration, please let me know how 8609 works for you.
JC
------------------------------
John Cirocco
Original Message:
Sent: Jul 02, 2021 12:23 PM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Was on with TAC early this AM, agreed to downgrade to 8.6.0.9. Really hoping I get the same success out of that as sram (Stuart) got above.
------------------------------
Steve Yuroff
Original Message:
Sent: Jul 02, 2021 08:27 AM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi JC, Oh yes: multiple minutes of data flow pause, completely breaking Zoom/Teams calls is EXACTLY what's causing me stress right now. I AM NOT ALONE.
I've documented this behavior under both 8.7.1.1_78245, and on 6/20 I upgraded to 8.8.0.1_80393, and the problems persist.
All I want is drama-free, boring as paint firmware.
Original Message:
Sent: 7/2/2021 1:47:00 AM
From: JC19
Subject: RE: Troubleshooting intermittent poor throughput on AP-515
Hi Steve, what firmware are you running?
On 8711 I experienced regular random pauses of data flow, some as long as 2 minutes. Zoom/Teams calls would drop completely. Mac's would not resume data flow after sleep. I proved this in the lab back and forth, 8713 resolved both the sleep and data pause issue, essentially they were the same symptom. Device stays connected to Wifi yet has no data flow. 8400 did not have these problems. I remedied this by going to 8713, which had transient speed issues, remedied by going to 8801.
JC
------------------------------
John Cirocco
Original Message:
Sent: Jul 01, 2021 10:29 AM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
Funny, I have a ticket/day on the Wifi running 8.8.x, with only a skeleton crew in the office.
I have an open ticket with TAC, and am getting frustrated with the latency of getting assistance from them.
------------------------------
Steve Yuroff
Original Message:
Sent: Jul 01, 2021 01:21 AM
From: John Cirocco
Subject: Troubleshooting intermittent poor throughput on AP-515
Performance & stability has vastly improved on 8.8.0.1, even on AP's with higher user counts. So far so good.
------------------------------
John Cirocco
Original Message:
Sent: Jun 29, 2021 11:01 PM
From: John Cirocco
Subject: Troubleshooting intermittent poor throughput on AP-515
Updating to 8.7.1.4 caused the AP's to go into a reboot loop. The AP's appeared to crash randomly (not all at once) so impact to client was not great, after a couple of hours all AP's became stable and I pushed 8801.
8801 has been OK for a few hours this morning, will provide a further update over the coming days.
TAC have advised that all feature updates and fixes in 87 are also in 88.
------------------------------
John Cirocco
Original Message:
Sent: Jun 29, 2021 04:27 PM
From: Stuart Ramdeen
Subject: Troubleshooting intermittent poor throughput on AP-515
Apologies for the late update. I forgot that I was due to take a week off. While I was away one of my colleagues looked into the issue. He tried various things that were recommended but nothing helped. In the end he bit the bullet and downgraded from 8.8 to 8.6.0.9 and it has been absolutely rock solid with not a single complaint in over two weeks. I would definitely try a downgrade if you have not given this version of firmware a try.
Fingers crossed that this may help you.
Stuart
Original Message:
Sent: Jun 29, 2021 03:56 PM
From: Steve Yuroff
Subject: Troubleshooting intermittent poor throughput on AP-515
I think I might be seeing the same... in troubleshooting unreliable videoconferencing experiences through a relatively new Instant deployment, I've been observing the behavior of a consistent 12Mbps iperf stream. It will typically hold a consistent 1.5MB/sec... until it sometimes just pauses, then unleashes a backlog of queued data. This is the kind of thing that might feel like a slight glitch in your webpage load, and is a giant smack in the face for videoconferencing.
[ 5] 61.00-62.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 5] 62.00-63.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 5] 63.00-64.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 5] 64.00-65.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 5] 65.00-66.00 sec 1.14 MBytes 9.54 Mbits/sec
[ 5] 66.00-67.00 sec 0.00 Bytes 0.00 Mbits/sec
[ 5] 67.00-68.00 sec 0.00 Bytes 0.00 Mbits/sec
[ 5] 68.00-69.00 sec 0.00 Bytes 0.00 Mbits/sec
[ 5] 69.00-70.00 sec 0.00 Bytes 0.00 Mbits/sec
[ 5] 70.00-71.00 sec 0.00 Bytes 0.00 Mbits/sec
[ 5] 71.00-72.00 sec 802 KBytes 6.57 Mbits/sec
[ 5] 72.00-73.00 sec 4.17 MBytes 35.0 Mbits/sec
[ 5] 73.00-74.00 sec 4.15 MBytes 34.8 Mbits/sec
[ 5] 74.00-75.00 sec 4.76 MBytes 39.9 Mbits/sec
[ 5] 75.00-76.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 5] 76.00-77.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 5] 77.00-78.00 sec 1.50 MBytes 12.6 Mbits/sec
[ 5] 78.00-79.00 sec 1.50 MBytes 12.6 Mbits/sec
------------------------------
Steve Yuroff
Original Message:
Sent: Jun 28, 2021 08:36 AM
From: John Cirocco
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi, I'm also experiencing a similar issue on the 515's. Seems transient yet consistent over the course of a day, not specific to an individual AP or channel. I am experiencing this on 8.7.1.3, tonight have upgraded to 8.7.1.4, not sure if we should upgrade to 8.8 yet.
We're running 40Mhz bands, HE disabled.
JC
------------------------------
John Cirocco
Original Message:
Sent: May 27, 2021 07:26 AM
From: Stuart Ramdeen
Subject: Troubleshooting intermittent poor throughput on AP-515
Hi all
This is my first post, so be gentle with me :-)
Please feel free to point me in the direction of a troubleshooting guide so that I can try and nail the basics before troubling the rest of you with my issues.
We've just started to plan the roll out of 30 AP-515s and I have installed one unit in a small office as a trial. At first, performance was excellent and everyone was happy. However, as soon as I left for the day (of course) the users started to report performance issues. I have now experienced this for myself and I would like to ask for some advice from those more experienced with the Instant system. For background, I have a reasonable amount of wired and wireless knowledge, but I would not necessarily consider myself an expert. We were using Meru/Fortinet for many years and I also have a lot of experience with Ubiquiti equipment, but we are moving away from those two systems to standardise on Aruba.
The basic symptom here is that everything is fine one moment, clients are working happily and a speed test reveals almost 300Mbps to the Internet. Then suddenly and without any warning everything will grind to a halt and a speed test will reveal about 1Mbps. After reading some of the manual and searching on here I have tried looking at `show clients` which reports signal and speed as 'good'. From a client (Macs and Chromebooks mainly) I am seeing RSSI figures of about -45 - -40 (the AP is only 3-10 metres from the clients in this office). So raw signal strength and transmit power should hopefully not be the issue here.
The AP was initially powered by an 802.3af PoE switch and I had IPM enabled. I thought that may have been the cause of this poor behaviour and so I switched it to an 802.3at switch and disabled IPM, but that has not made any difference.
I've tried disabling 80MHz channels in config -> RF -> ARM -> Access point control.
I've tried enabling Client Match (all settings left at default values)
I have the output of `show tech-support` at various times when the problem is occurring, just let me know what portions will be helpful to post. I've also attached some screenshots from the gui.
Also, is it advisable to strip out MAC addresses and serial numbers of APs?
Thanks for your assistance.