I am looking to get a port forward from WAN IP to a remote node LAN client.
The WAN is coming from a tunnel server, such that the internet traffic from the mesh DOES appear to come from the public IP of the tunnel server. I can control this IP address and firewall. The tunnel client I cannot control the WAN port forwarding, so I did not chose to share that WAN over the mesh. The tunnel client device DOES have RF to the mesh. I have full access to tunnel server, tunnel client, remote node.
I was able to get SSH forwarded to the remote node itself with this example inside of /etc/config/firewall on the node that is supplying WAN (the tunnel server mentioned above):
config redirect
option src 'wan'
option src-dport '22222'
option dest-ip 'REMOTE_NODE_IP'
option dest_port '2222'
I am able to ssh to the remote node by the public IP address of the node with WAN.
I tried adding a second stanza like this to the same tunnel server with WAN, for UDP 54504 to a remote LAN IP at this same REMOTE_NODE:
config redirect
option src 'wan'
option src_dport '54504'
option dest_ip 'REMOTE_LAN_IP'
option dest_port '54504'
This doesn't seem to work.
Is there another consideration I need to take for firewall/forwading, with this topology in mind?
Matt, KB9OIV
The WAN is coming from a tunnel server, such that the internet traffic from the mesh DOES appear to come from the public IP of the tunnel server. I can control this IP address and firewall. The tunnel client I cannot control the WAN port forwarding, so I did not chose to share that WAN over the mesh. The tunnel client device DOES have RF to the mesh. I have full access to tunnel server, tunnel client, remote node.
I was able to get SSH forwarded to the remote node itself with this example inside of /etc/config/firewall on the node that is supplying WAN (the tunnel server mentioned above):
config redirect
option src 'wan'
option src-dport '22222'
option dest-ip 'REMOTE_NODE_IP'
option dest_port '2222'
I am able to ssh to the remote node by the public IP address of the node with WAN.
I tried adding a second stanza like this to the same tunnel server with WAN, for UDP 54504 to a remote LAN IP at this same REMOTE_NODE:
config redirect
option src 'wan'
option src_dport '54504'
option dest_ip 'REMOTE_LAN_IP'
option dest_port '54504'
This doesn't seem to work.
Is there another consideration I need to take for firewall/forwading, with this topology in mind?
Matt, KB9OIV
I hope I am guessing what you intend.
config redirect
option src 'wan'
option src_dport '54504'
option dest_ip 'REMOTE_LAN_IP'
option dest_port '54504'
>
config redirect
option src 'wan'
option src_dport '54504'
option dest_ip 'REMOTE_WAN_IP'
option dest_port '54504'
Then use the remote's 'Port Forwardng' to get to its LAN port.
The GUI forwarding option seems to only take into consideration WAN<>local LAN, and not other interconnected nodes.
Ooopps,
option dest_ip 'REMOTE_WiFi_IP'
All the nodes (and LAN attached devices) in the network have access to a nodes 'Node Name' and 'WiFi-address'.
I hope this helps,
Chuck
https://www.arednmesh.org/comment/3683#comment-3683
To make sure that the device at port 54504 is not the problem, I am going to change this setup a bit, so that the final device I have complete control over. I have created a test scenario.
The 'WAN' device is behind a private network, such that is like a double NAT of my home network. On my regular home network, I will be attempting to access a 'remote node' web interface of a LAN connected rpi.
The 'WAN' device is a Mikrotik hAP. The 'wan' IP is 172.16.2.177/24.
The 'remote' device is a Nanostation, connected to this hAP via DtD link. The 'remote' LAN rpi is connected to this Nanostation. I can access the web gui of the rpi that is connected to the Nanostation, from any 'internal' device to this mesh as expected.
So in this test setup, let us assume that I would like to connect to this web gui of the pi, using a WAN device at say, 172.16.2.103/24 (effectively, something 'on the internet' as far as the Mikrotik is concerned).
In this test, I would like the 'WAN' to forward a connection from WAN:8888 to this remote web gui at RPI:80.
--------------------------------------------------
I ran:
tcpdump -i any
on each of those 2 hosts, while trying to load 172.16.2.177:8888, from 172.16.2.103
The web gui of the pi did not load, despite seeing 'some' relevant IP traffic back and forth in these tcpdumps.
I have attached those files to this post.
-------------------------------------------------------
The rpi IP address 'on the mesh' is 10.246.189.246.
On the Mikrotik I have added a single '/etc/config/firewall' stanza at the bottom, like this (granted, I have no idea if this is correct, or complete):
config redirect
option src 'wan'
option src_dport '8888'
option dest_ip '10.246.189.246'
option dest_port '80'
-----------------------------------------------------------------------------
ifconfig of the hAP:
root@KB9OIV-1:~# ifconfig
br-lan Link encap:Ethernet HWaddr 2C:C8:1B:5F:45:9F
inet addr:10.200.179.225 Bcast:10.200.179.255 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:216 errors:0 dropped:0 overruns:0 frame:0
TX packets:157 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:42953 (41.9 KiB) TX bytes:36524 (35.6 KiB)
eth0 Link encap:Ethernet HWaddr 2C:C8:1B:5E:45:99
inet addr:172.16.2.177 Bcast:172.16.2.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1362 errors:0 dropped:24 overruns:0 frame:0
TX packets:678 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:420864 (411.0 KiB) TX bytes:112563 (109.9 KiB)
Interrupt:4
eth1 Link encap:Ethernet HWaddr 2C:C8:1B:5F:45:9F
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:297 errors:0 dropped:0 overruns:0 frame:0
TX packets:570 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:95526 (93.2 KiB) TX bytes:446374 (435.9 KiB)
Interrupt:5
eth1.0 Link encap:Ethernet HWaddr 2C:C8:1B:5F:45:9F
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:75 errors:0 dropped:5 overruns:0 frame:0
TX packets:103 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:31670 (30.9 KiB) TX bytes:34876 (34.0 KiB)
eth1.2 Link encap:Ethernet HWaddr 2C:C8:1B:5F:45:9F
inet addr:10.95.69.159 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:169 errors:0 dropped:0 overruns:0 frame:0
TX packets:294 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41126 (40.1 KiB) TX bytes:223176 (217.9 KiB)
eth1.3975 Link encap:Ethernet HWaddr 2C:C8:1B:5F:45:9F
inet addr:10.94.69.159 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:173 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:186042 (181.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:298 errors:0 dropped:0 overruns:0 frame:0
TX packets:298 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20390 (19.9 KiB) TX bytes:20390 (19.9 KiB)
tun60 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:172.31.44.129 P-t-P:172.31.44.130 Mask:255.255.255.252
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1450 Metric:1
RX packets:317 errors:0 dropped:0 overruns:0 frame:0
TX packets:215 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:263669 (257.4 KiB) TX bytes:63760 (62.2 KiB)
wlan1 Link encap:Ethernet HWaddr 2C:C8:1B:5E:45:9F
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:145 errors:0 dropped:0 overruns:0 frame:0
TX packets:149 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15232 (14.8 KiB) TX bytes:19395 (18.9 KiB)
------------------------------------------------------------------------------------------------------------------
ifconfig of the 'remote' Nanostation:
root@KB9OIV-Nanostation-M5-1:~# ifconfig
br-lan Link encap:Ethernet HWaddr B4:FB:E4:3F:D7:BE
inet addr:10.246.189.241 Bcast:10.246.189.247 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4758 errors:0 dropped:17 overruns:0 frame:0
TX packets:4072 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1167909 (1.1 MiB) TX bytes:346265 (338.1 KiB)
eth0 Link encap:Ethernet HWaddr B4:FB:E4:3F:D7:BE
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:62086 errors:0 dropped:0 overruns:0 frame:0
TX packets:42437 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:68096347 (64.9 MiB) TX bytes:8950045 (8.5 MiB)
Interrupt:4
eth0.1 Link encap:Ethernet HWaddr B4:FB:E4:3F:D7:BE
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:13438 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:4595796 (4.3 MiB)
eth0.2 Link encap:Ethernet HWaddr B4:FB:E4:3F:D7:BE
inet addr:10.63.215.190 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:57328 errors:0 dropped:0 overruns:0 frame:0
TX packets:24927 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:65829922 (62.7 MiB) TX bytes:3854524 (3.6 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:61826 errors:0 dropped:0 overruns:0 frame:0
TX packets:61826 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3958407 (3.7 MiB) TX bytes:3958407 (3.7 MiB)
wlan0 Link encap:Ethernet HWaddr B4:FB:E4:3E:D7:BE
inet addr:10.62.215.190 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:22883 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:2863300 (2.7 MiB)
wlan0-1 Link encap:UNSPEC HWaddr B6-FB-E4-3E-D7-BE-00-44-00-00-00-00-00-00-00-00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
------------------------------------------------------------------------------------------------------------
iptables from the hAP, with that additional rule above
iptables -L -t nat
root@KB9OIV-1:~# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
prerouting_rule all -- anywhere anywhere /* !fw3: Custom prerouting rule chain */
zone_lan_prerouting all -- anywhere anywhere /* !fw3 */
zone_wan_prerouting all -- anywhere anywhere /* !fw3 */
zone_wifi_prerouting all -- anywhere anywhere /* !fw3 */
zone_dtdlink_prerouting all -- anywhere anywhere /* !fw3 */
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
postrouting_rule all -- anywhere anywhere /* !fw3: Custom postrouting rule chain */
zone_lan_postrouting all -- anywhere anywhere /* !fw3 */
zone_wan_postrouting all -- anywhere anywhere /* !fw3 */
zone_wifi_postrouting all -- anywhere anywhere /* !fw3 */
zone_dtdlink_postrouting all -- anywhere anywhere /* !fw3 */
Chain postrouting_dtdlink_rule (1 references)
target prot opt source destination
Chain postrouting_lan_rule (1 references)
target prot opt source destination
Chain postrouting_rule (1 references)
target prot opt source destination
Chain postrouting_wan_rule (1 references)
target prot opt source destination
Chain postrouting_wifi_rule (1 references)
target prot opt source destination
Chain prerouting_dtdlink_rule (1 references)
target prot opt source destination
Chain prerouting_lan_rule (1 references)
target prot opt source destination
Chain prerouting_rule (1 references)
target prot opt source destination
Chain prerouting_wan_rule (1 references)
target prot opt source destination
Chain prerouting_wifi_rule (1 references)
target prot opt source destination
Chain zone_dtdlink_postrouting (1 references)
target prot opt source destination
postrouting_dtdlink_rule all -- anywhere anywhere /* !fw3: Custom dtdlink postrouting rule chain */
Chain zone_dtdlink_prerouting (1 references)
target prot opt source destination
prerouting_dtdlink_rule all -- anywhere anywhere /* !fw3: Custom dtdlink prerouting rule chain */
Chain zone_lan_postrouting (1 references)
target prot opt source destination
postrouting_lan_rule all -- anywhere anywhere /* !fw3: Custom lan postrouting rule chain */
Chain zone_lan_prerouting (1 references)
target prot opt source destination
prerouting_lan_rule all -- anywhere anywhere /* !fw3: Custom lan prerouting rule chain */
Chain zone_wan_postrouting (1 references)
target prot opt source destination
postrouting_wan_rule all -- anywhere anywhere /* !fw3: Custom wan postrouting rule chain */
MASQUERADE all -- anywhere anywhere /* !fw3 */
Chain zone_wan_prerouting (1 references)
target prot opt source destination
prerouting_wan_rule all -- anywhere anywhere /* !fw3: Custom wan prerouting rule chain */
DNAT tcp -- anywhere anywhere tcp dpt:8888 /* !fw3: @redirect[0] */ to:10.246.189.246:80
DNAT udp -- anywhere anywhere udp dpt:8888 /* !fw3: @redirect[0] */ to:10.246.189.246:80
Chain zone_wifi_postrouting (1 references)
target prot opt source destination
postrouting_wifi_rule all -- anywhere anywhere /* !fw3: Custom wifi postrouting rule chain */
SNAT tcp -- 10.0.0.0/8 10.246.189.246 tcp dpt:80 /* !fw3: @redirect[0] (reflection) */ to:10.94.69.159
SNAT udp -- 10.0.0.0/8 10.246.189.246 udp dpt:80 /* !fw3: @redirect[0] (reflection) */ to:10.94.69.159
Chain zone_wifi_prerouting (1 references)
target prot opt source destination
prerouting_wifi_rule all -- anywhere anywhere /* !fw3: Custom wifi prerouting rule chain */
DNAT tcp -- 10.0.0.0/8 172.16.2.177 tcp dpt:8888 /* !fw3: @redirect[0] (reflection) */ to:10.246.189.246:80
DNAT udp -- 10.0.0.0/8 172.16.2.177 udp dpt:8888 /* !fw3: @redirect[0] (reflection) */ to:10.246.189.246:80
----------------------------------------------------------------------------------------------------------------------------------------------------
The 'WAN' device is a Mikrotik hAP. The 'wan' IP is 172.16.2.177/24.
The 'remote' device is a Nanostation, connected to this hAP via DtD link.
The 'remote' LAN rpi is connected to this Nanostation.
All devices on the WAN can reach the Mikrotik hAP.
You need to 'config redirect' port 54504 from the hAP to the Nanostation.
At the Nanostation the standard AREDN GUI 'Port Forwarding' may be configured to portforward 54504 to the remote-device-rpi.
I think.
I hope this helps,
Chuck
Chain zone_wifi_postrouting (1 references)
target prot opt source destination
postrouting_wifi_rule all -- anywhere anywhere /* !fw3: Custom wifi postrouting rule chain */
SNAT tcp -- 10.0.0.0/8 10.246.189.246 tcp dpt:80 /* !fw3: @redirect[0] (reflection) */ to:10.94.69.159
SNAT udp -- 10.0.0.0/8 10.246.189.246 udp dpt:80 /* !fw3: @redirect[0] (reflection) */ to:10.94.69.159
If I understand correctly, the packets ared being routed out the dtdlink interface to get to the LAN device. If so, we probably need the same rule above but for the "zone_dtdlink_postrouting" chain. It's been a while since I've looked over the rules :) .
In my setup, I can see some ACK's come back that ACK the WAN requests, on the Mikrotik, as far as eth1.2, using tcpdump.
This is the custom rule I used in this case:
iptables -t nat -A zone_wan_prerouting -p tcp -m tcp --dport 8888 -j DNAT --to 10.246.189.246:80
Edit this template script on the gateway node to do this: https://github.com/aredn/aredn/blob/main/files/etc/local/mesh-firewall/59-custom-rules
update: if the incoming wan traffic is on the node that also has the tunnel connection, then an addition entry would be needed in this custom rule script.
Joe AE6XE
Thank you for your suggestions.
Joe:
This is what I have tried now, the above doesn't seem to work for device on remote node.
What I did now for further troubleshooting:
Then, I removed (on this direct WAN access node) the regular GUI WAN<>Local LAN settings (save), and then added this to the "59-..." file (10.200.179.240 is the local LAN IP address of the target device, on the local device with the WAN):
iptables -t nat -A zone_wifi_postrouting -p tcp -m tcp -d 10.246.189.246 -j SNAT --dport 80 --to-source 10.94.69.159 -m comment --comment "web GUI test"
iptables -t nat -A zone_dtdlink_postrouting -p tcp -m tcp -d 10.246.189.246 -j SNAT --dport 80 --to-source 10.95.69.159 -m comment --comment "web GUI test"
I then executed firewall reload. This also DID work for WAN<>Local LAN, as I had hoped.
However, having the device on remote node, with this in that same "59-..." file (on the node with WAN) -- 10.246.189.246 is the remote node LAN IP that was assigned to that device, did NOT work as I had hoped.
I tried this with firewall reload, and a reboot:
iptables -t nat -A zone_wifi_postrouting -p tcp -m tcp -d 10.246.189.246 -j SNAT --dport 80 --to-source 10.94.69.159 -m comment --comment "web GUI test"
iptables -t nat -A zone_dtdlink_postrouting -p tcp -m tcp -d 10.246.189.246 -j SNAT --dport 80 --to-source 10.95.69.159 -m comment --comment "web GUI test
Lets rule out a possible option that could explain why it might be failing... Some services may open another socket or stream from server back to the internet side client, which doesn't work well through a NAT on the firewall.
In this situation with the 59_custom_rule template, the traffic would dead-end at the gateway node, because the address to go back to the client on the internet does not exist. The SNAT in the rule is replacing the 'from' internet address with the address of the gateway node, telling the mesh LAN device to route packets back to this specific gateway, as opposed to another default gateway to route to an internet address. If the packet is a reply back, the mapping to the internet address still exists on the gateway. But new sockets opened up from the LAN device do not have knowledge of the internet address to reach it. We'd have to drop the SNAT option, which would work in your case, but not all cases (when there is a different default gateway path in use from the node where the LAN Device is).
Joe AE6XE
I have been spending a lot of time pouring over this, you know, stare at it long enough and maybe it will make sense? Anyway, hopefully I can turn this into a learning experience.
We have been asked to support an amateur radio repeater with a local network for a VOIP voter, as well as an added connection to a server on the internet. The repeater requires a specific UDP port to be open from the internet. Unfortunately not an MMDVM host that punches through easily.
I understand that forwarding ports from the internet to mesh is quite unsupported out of the box, so it may not happen.
My hope was to try to get this to work in a test setup. I have chosen to use as my test device, an rpi with raspbx, and if I could get that remote web interface to 'open' from the WAN port side of my test network at home, that this could be extended to the repeater scenario.
I have done some closer inspection of tcpdumps of this attempt using what you have suggested, and here is what I see upon closer inspection:
Web request from 172.16.2.103>172.16.2.177:8888 is ending up like this on my WAN Mikrotik hAP ac lite:
No SNAT:
[Interface:eth0] 172.16.2.103.51674 > 172.16.2.177.8888
[Interface:eth1.2] 172.16.2.103.51674 > 10.246.189.246.80
[Interface:eth1.2] 10.246.189.246.80 > 172.16.2.103.51674
[Interface:eth1] 10.246.189.246.80 > 172.16.2.103.51674
With SNAT:
[Interface:eth0] 172.16.2.103.51699 > 172.16.2.177.8888
[Interface:eth1.2] 10.95.69.159.51699 > 10.246.189.246.80
[Interface:eth1.2] 10.246.189.246.80 > 10.95.69.159.51699
[Interface:eth1] 10.246.189.246.80 > 10.95.69.159.51699
The traffic seems to get placed on eth1, and that is where it stops. I am not sure what eth1 is. There is no IP interface that I can see on the Mikrotik hAP that has a layer 3 for IP for eth1. I thought it should be placed on eth0, the way it came in! Lol.
There is no route entry that uses eth1:
root@KB9OIV-1:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 172.16.2.1 0.0.0.0 UG 0 0 0 eth0
10.0.0.0 * 255.0.0.0 U 0 0 0 eth1.2
10.0.0.0 * 255.0.0.0 U 0 0 0 wlan1
10.200.179.224 * 255.255.255.224 U 0 0 0 br-lan
172.16.2.0 * 255.255.255.0 U 0 0 0 eth0
172.31.44.128 mid11.KB9OIV-2 255.255.255.252 UG 0 0 0 tun60
172.31.44.128 * 255.255.255.252 U 0 0 0 tun60
root@KB9OIV-1:~#
Edit:
ip route show table all
This shows a ton of routes, but I am not sure I see a route that would result with eth1.
the wan gateway is advertised for use across the mesh? This basic setup option is required for the remote LAN device to access the internet. This option is NOT required for a local LAN device on the gateway node to access the internet.
Joe AE6XE
Oh My, that was it!
Turned on the 'Allow Others to use my WAN', and the port forward worked!
duh
Joe mentioned this might require more NAT rules, not sure yet.
Joe AE6XE
Thanks for your help everyone.
I applied Joe's sample rule to the remote tunnel node, and I was able to access the remote node by the public IP address of the remote tunnel node.
It seems that I have confirmed that this 'should' work for what we have been asked to do.
Andre, K6AH