You are here

crawler and visualizations for AREDN mesh

5 posts / 0 new
Last post
KK7LZM's picture
crawler and visualizations for AREDN mesh
Our club is building out an AREDN mesh in Seattle, WA and to help us plan our deployment I put together a little crawler and network/map visualization for our mesh in python.

It's a quick and simple tool. I'm still learning AREDN. There's probably bugs/issues.

The tool can produce two visuals (right now): 

A network graph (using graphviz):

A map (this is post processed into KML, but it natively produces geojson):

It uses sysinfo.json to get link information and saves the responses as it crawls. I'm happy to answer questions, consider feature requests, collaborate or support another more mature effort.

I was inspired by which is in node.js. But I'm not so good with node.js and, sadly for me, this was quicker to bang out in python in a night than to figure out how npm works :P

nc8q's picture
gelmce@HP-Laptop:~/Downloads$ echo "" |python3 craw
gelmce@HP-Laptop:~/Downloads$ echo "" |python3

2023-09-16 14:28:23,645 root         INFO     crawling
2023-09-16 14:28:23,645 root         WARNING failed to get sysinfo Missing dependencies for SOCKS support.

KK7LZM's picture
Ah, yes, I have it set up to
Ah, yes, I have it set up to run through an ssh tunnel socks proxy to a node connected to my home network.

The issue is that the crawler needs to be able to route to all the nodes. If you're using your AREDN node as your gateway, that's easy. If you're not (as in my topology), you need some way to route to all the other nodes. I do that with an ssh tunnel SOCKS proxy.

Sorry I didn't include that info in the documentation. I will update it and make it configurable shortly.

And thanks for taking a look!
nc8q's picture
Is this appropriate for python developers or python users?
gelmce@HP-Laptop:~/Downloads$ python -mvenv .venv
Command 'python' not found, did you mean:
  command 'python3' from deb python3
  command 'python' from deb python-is-python3
gelmce@HP-Laptop:~/Downloads$ python3 -mvenv .venv
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: /home/gelmce/Downloads/.venv/bin/python3


73, Chuck

KK7LZM's picture
I updated documentation
I updated documentation regarding proxies and clarifying prerequisites.

Using this tool right now definitely requires comfort with the command line and python tooling, e.g. pip.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer