The 3 GHz portion of our mesh in Hillsboro, OR seems to have some spotty degradation where I used to see 100% LQ and NLQ. It seems to be assymmetric, like one node is having trouble hearing other nodes, but the peer nodes are receiving at 100% link quality, so I'm not convinced it just the annual tree growth. It seemed to coincide with the deployment of 5G. I have not dismissed that the node has degraded, and will swap in a spare identical node at some point.
Has anyone else observed this on their 3 GHz network?
73's,
Collier
NM7B
You are here
Degradation In 3 GHz performance with 5G roll out?
Sun, 05/29/2022 - 15:07
#1
Degradation In 3 GHz performance with 5G roll out?
Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer
Have not see it here in Irvine, CA. We have three 3GHz links about 4 to 7 miles in distance. Our signals are 100% LQ and NLQ with 30+ SNR and a low of 39 Mbx on one link and 50+ Mbs on the other two. All our links use Rocket M3's with Rocket dishes pointing to each other. No sector antennas.73
Bob
KE6GYD
what channels are you using?
We are using channels 3400, 3410, 3430, and 3440
Hi, Collier:
What channels/frequencies are you using?
73, Chuck
Make sure that you are not going out of band into 3450. Depending on your channel width, you might be.
We did migrate out of the reallocated frequencies and decided to move to 3380 MHz as the return loss with the antennas being used is the highest on that end of the band, so we're well clear of the lost frequencies. That's what doesn't seem to make sense: I would expect the new 5G service to be far enough away to not impact impact reception at this one node. We'll see: I'm going to swap in a different Rocket/sector when we reconfigure this site.
A separate path that is much shorter at about 3/4 mile also has seen degradation, but it's symmetrical. That link is at 3400 MHz, and may be impacted by season tree growth in the path.
Thank you, and 73's,
Collier
Hi, Collier:
I am thinking that LQ/NLQ is not a good metric to use to quantify 'degradation'.
SNR might be a better tool. Did you save SNR values over time?
I am unclear what 'spotty' degradation is as compared to other types of degradation.
Since AREDN firmware automaticly seeks highest throughput, perhaps measureing the chosen MCS might be a good tool.
I am wondering if your 3 GHz links are point-to-point, point-to-multi-point, or (mesh) multipoint-to-multipoint.
Are the degraded nodes co-located on cellular towers?
Where might the 5G interference be from?
I have used a point-to-point link and tested on all the channels.
I don't know why, but some channels were better than others.
Have you tried other channels?
I hope this helps,
Chuck
Hi, Chuck,
The SNR values haven't changed more than a few dB, with the most challenging path around 14 dB. What I have noticed is that iperf runs have degraded too. Instead of multiple samples with a small variation in throughput values, I'm getting 3 to 5 test samples that are 0 byte transferred. I've also noticed some degradation in the VoIP telephony service.
The main node that I monitor is in the middle of a square mile campus with the nearest cellsite about 3/4 mile away. I don't know if that site supports 3 GHz, but there is an airport only a few blocks away, so they may not be using it yet.
The system is currently configured as point to multipoint, with a wide variety of distances from a node on an adjacent building to a dish about 3.5 miles away.
Trying different channels is a good idea. I will give that a try after logging the current system readings. I also will be scheduling some antenna work and will swap in a different but identical 3 GHz node to see if that changes anything.
Thank you again for brainstorming my observations.
Have you tried using the new Link Quality Manager feature of the latest firmware? If you want to use LQM you should probably load the latest Nightly Build on both ends of a link and it should self adjust to get the best throughput. Do you have access to a spectrum analyzer? Depending on the 5G implementation in your area, they definitely may be overlapping with your frequencies. The FCC is sunsetting Amateur Radio use in the entire band, so this may be an opportunity for you to consider swapping equipment to use a different band.
I did update all of the nodes I admin to 3.22.6, and have enabled LQM. I'll let it run for a while and see what it reports.
I am in the process of phasing out our 3 GHz nodes as I saw the FCC's cautionary note that amateurs will have to vacate the remaining 3 GHz privileges probably on short notice. The county EOCs that we connect to are currently on 3 GHz as well, so I need to maintain 3 GHz for that connection path at least in the near future. The county EM has requested funding to add five 5 GHz AREDN nodes to some of their key facilities in the area.
Thank you,
Collier
I attempted to move my 3 GHz nodes up from 3380 MHz to 3410 MHz, but got stuck by this: https://www.arednmesh.org/content/rocket-m5hp-xw-hardware
I have not seen this.
Nanostation M3 (XM x4).
Stables 2018 through 3.22.6.0.
Nightlies through 1449.
I have a mix of 3 GHz NanoStations and Rockets. The NanoStations QSYed no issues. It's the one Rocket that coughed up the Bad Gateway issue. I even tried power-cycling the Rocket twice but that didn't resolve the issue.
Can you post a support data dump of the offfending node right after it exhibits the "Bad Gateway" message?
You can always get a support dump using the URL directly: http://your-nodename:8080/cgi-bin/supporttool
Per instructions, I reproduce the issue by selecting a new channel, then clicking on "Save Changes". When the "Bad Gateway" message appeared, I immediately manually changed to the ./supporttool URL and saved the attached dump.
As a point of reference, I tried the same channel change and save on the other Rocket M3 I have, and it did not exhibit this "Bad Gateway" issue. That Rocket is also running 3.22.6
Thank you for delving into this!
Collier
NM7B
Here is the error, and I'll pass it along to the coders. THX.
This seems to indicate that a ping to/from your node is unable to connect to that IP subnet.
Thank you for the quick parsing of the dump. I'm kind of baffled why one Rocket M3 threw such a fit, while the other one worked perfectly. I'll have to see if there's any hardware differences. I purchased two in the first round, then the third at a later date.
73's,
Collier
NM7B
An update was just merged and should appear in tomorrow's Nightly Build firmware that will hopefully fix your "Bad Gateway" message. Let us know how it goes when you have a chance to install that NB firmware. THX
Thank you for the heads-up on the nightly build. I'll give it a try.
Another anomalous behavior that might or might not be related: For the Rocket M3 that is throwing the Bad Gateway message, all of the Optional settings, specifically everything below the upper webpage screen where the channel, power output, DHCP parameters are set, everything below there is missing. I'm not able to enter any location information because all of those entry fields are completely missing. I checked the other Rocket M3 that doesn't throw the Bad Gateway message, and the location information setting fields are visible and available to be entered/modified. I tried re-flashing 3.22.6 to see if that would alleviate the issue, but the missing fields persisted after the reflash.
I was preparing the spare Rocket M3/sector antenna to be swapped in to see if that makes a difference for my asymmetric path performance, and was attempting to copy the location data when I realized that I couldn't find the Optional Settings field. This antenna work will also involve adding a 5 GHz node which is the long-term replacement for 3 GHz. Antenna work at this site is very involved, so physical access to the node is extremely limited.
Thank you,
Collier
NM7B
Has anyone used airview on your local spectrum. I don't use the AREDN fw at this site.
I know I have a very crowded rf inviorment!
https://youtu.be/YsdjY-4IfGo
https://youtu.be/RRMajsGiVt4
I'm a little limited by NSM3 its got a narrower usable bandwidth compared to the airfiber.
Spectral view available in Nightly Build.