You are here

Home » Forums » Development » Nightly Builds

Tunnels Disappeared b02

9 posts / 0 new
Log in or register to post comments
Last post
Sat, 04/23/2016 - 07:00
#1
kc8rgo
Tunnels Disappeared b02

Operating two tunnels last night and playing with IP phone into K7OPA Asterisk, both tunnels disappeared within three minutes of each other.  Rebooted NSM2 this morning and both tunnels are back.

** Now I am  not seeing the option for Support Data for this entry.

Tunnels disappeared means that the blue cloud and associated MESH STATUS for the tunnels went away.  My two nodes were still visible on the Status.

 

Top
Sat, 04/23/2016 - 07:05
kc8rgo
Under "Add a new Comment" the
Under "Add a new Comment" the Support File Attachments" is back and uploaded.
Support File Attachments: 
File supportdata-KC8RGO-NSM2-1-201604231038.tgz
Top
Sat, 04/23/2016 - 07:15
(Reply to #2)
kc8rgo
Same Tunnels disappear again
Same Tunnels disappear again shortly after Reboot.

 
Support File Attachments: 
File supportdata-KC8RGO-NSM2-1-201604231113.tgz
Top
Sat, 04/23/2016 - 12:01
(Reply to #3)
AE6XE
AE6XE's picture
In the log file, the WAN
In the log file, the WAN interface connecting to the internet (eth0.1) went down/up 4 times over ~14 min period.  The tunnel rides over the WAN interface.   Figure out why the connection to the internet is bouncing like a yo-yo....   Cable problem?    Is this over a 500' cat5 cable?   Supply voltage level dropping too low to mesh node?    Probably not related, but wondering why there is an "invalid argument" message...

Sat Apr 23 10:57:29 2016 daemon.info vtund[1396]: Invalid argument (22)
Sat Apr 23 10:57:29 2016 daemon.info olsrd[10446]: Removing interface tun61
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Network device 'tun61' link is down
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Interface 'tun61' has link connectivity loss
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Interface 'tun61' is now down
Sat Apr 23 10:57:29 2016 daemon.info vtund[1396]: Session KC8RGO-NSM2-1-172-31-229-176[k7opa.ddns.net] closed
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Interface 'tun61' is disabled
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Interface 'tun61' is enabled
Sat Apr 23 10:57:29 2016 daemon.notice netifd: Interface 'tun61' is disabled
Sat Apr 23 10:57:34 2016 daemon.info vtund[1396]: Connecting to k7opa.ddns.net
Sat Apr 23 10:57:34 2016 daemon.debug vtund[1396]: Remote Server sends <TuL9K>
Sat Apr 23 10:57:34 2016 daemon.info vtund[1396]: Session KC8RGO-NSM2-1-172-31-229-176[k7opa.ddns.net] opened
Sat Apr 23 10:57:34 2016 daemon.info vtund[1396]: LZO compression[level 9] initialized
Sat Apr 23 10:57:34 2016 daemon.notice netifd: Interface 'tun61' is enabled
Sat Apr 23 10:57:34 2016 daemon.notice netifd: Network device 'tun61' link is up
Sat Apr 23 10:57:34 2016 daemon.notice netifd: Interface 'tun61' has link connectivity 
Sat Apr 23 10:57:34 2016 daemon.notice netifd: Interface 'tun61' is setting up now
Sat Apr 23 10:57:34 2016 daemon.info olsrd[10446]: Writing '0' (was 1) to /proc/sys/net/ipv4/conf/tun61/send_redirects
Sat Apr 23 10:57:34 2016 daemon.info olsrd[10446]: Writing '0' (was 0) to /proc/sys/net/ipv4/conf/tun61/rp_filter
Sat Apr 23 10:57:34 2016 daemon.info olsrd[10446]: Adding interface tun61
Sat Apr 23 10:57:34 2016 daemon.notice netifd: Interface 'tun61' is now up
Sat Apr 23 10:57:37 2016 daemon.info olsrd_hotplug: [OK] ifup: 'tun61' => 'tun61'
Sat Apr 23 10:57:37 2016 daemon.debug olsrd_hotplug: [OK] interface 'tun61' => 'tun61' not used for olsrd
Sat Apr 23 10:59:07 2016 daemon.notice netifd: Network device 'eth0' link is down
Sat Apr 23 10:59:07 2016 daemon.notice netifd: Interface 'lan' has link connectivity loss
Sat Apr 23 10:59:07 2016 daemon.notice netifd: VLAN 'eth0.1' link is down
Sat Apr 23 10:59:07 2016 daemon.notice netifd: Interface 'wan' has link connectivity loss
Sat Apr 23 10:59:07 2016 kern.info kernel: [ 1601.120000] eth0: link down
Sat Apr 23 10:59:07 2016 daemon.notice netifd: VLAN 'eth0.2' link is down
Sat Apr 23 10:59:07 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity loss
Sat Apr 23 10:59:07 2016 daemon.notice netifd: wan (13579): Received SIGTERM
Sat Apr 23 10:59:09 2016 daemon.notice netifd: Network device 'eth0' link is up
Sat Apr 23 10:59:09 2016 daemon.notice netifd: Interface 'lan' has link connectivity 
Sat Apr 23 10:59:09 2016 daemon.notice netifd: VLAN 'eth0.1' link is up
Sat Apr 23 10:59:09 2016 daemon.notice netifd: Interface 'wan' has link connectivity 
Sat Apr 23 10:59:09 2016 daemon.notice netifd: Interface 'wan' is setting up now
Sat Apr 23 10:59:09 2016 kern.info kernel: [ 1603.120000] eth0: link up (100Mbps/Full duplex)
Sat Apr 23 10:59:09 2016 daemon.notice netifd: VLAN 'eth0.2' link is up
Sat Apr 23 10:59:09 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity 
Sat Apr 23 10:59:09 2016 daemon.notice netifd: wan (16310): udhcpc (v1.22.1) started
Sat Apr 23 10:59:09 2016 daemon.notice netifd: wan (16310): Sending discover...
Sat Apr 23 10:59:10 2016 daemon.notice netifd: wan (16310): Sending select for 192.168.1.105...
Sat Apr 23 10:59:10 2016 daemon.notice netifd: wan (16310): Lease of 192.168.1.105 obtained, lease time 86400
Sat Apr 23 10:59:10 2016 daemon.notice netifd: Interface 'wan' is now up
Sat Apr 23 10:59:10 2016 user.notice firewall: Reloading firewall due to ifup of wan (eth0.1)
Sat Apr 23 10:59:11 2016 daemon.err olsrd[10446]: OLSR: sendto IPv4 Operation not permitted
Sat Apr 23 10:59:12 2016 daemon.info olsrd_hotplug: [OK] ifup: 'wan' => 'eth0.1'
Sat Apr 23 10:59:12 2016 daemon.debug olsrd_hotplug: [OK] interface 'wan' => 'eth0.1' not used for olsrd
Sat Apr 23 11:01:37 2016 daemon.notice netifd: Network device 'eth0' link is down
Sat Apr 23 11:01:37 2016 daemon.notice netifd: Interface 'lan' has link connectivity loss
Sat Apr 23 11:01:37 2016 daemon.notice netifd: VLAN 'eth0.1' link is down
Sat Apr 23 11:01:37 2016 daemon.notice netifd: Interface 'wan' has link connectivity loss
Sat Apr 23 11:01:37 2016 kern.info kernel: [ 1751.430000] eth0: link down
Sat Apr 23 11:01:37 2016 daemon.notice netifd: VLAN 'eth0.2' link is down
Sat Apr 23 11:01:37 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity loss
Sat Apr 23 11:01:37 2016 daemon.notice netifd: wan (16310): Received SIGTERM
Sat Apr 23 11:01:39 2016 daemon.notice netifd: Network device 'eth0' link is up
Sat Apr 23 11:01:39 2016 daemon.notice netifd: Interface 'lan' has link connectivity 
Sat Apr 23 11:01:39 2016 daemon.notice netifd: VLAN 'eth0.1' link is up
Sat Apr 23 11:01:39 2016 daemon.notice netifd: Interface 'wan' has link connectivity 
Sat Apr 23 11:01:39 2016 daemon.notice netifd: Interface 'wan' is setting up now
Sat Apr 23 11:01:39 2016 daemon.notice netifd: VLAN 'eth0.2' link is up
Sat Apr 23 11:01:39 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity 
Sat Apr 23 11:01:39 2016 kern.info kernel: [ 1753.430000] eth0: link up (100Mbps/Full duplex)
Sat Apr 23 11:01:39 2016 daemon.notice netifd: wan (17703): udhcpc (v1.22.1) started
Sat Apr 23 11:01:39 2016 daemon.notice netifd: wan (17703): Sending discover...
Sat Apr 23 11:01:40 2016 daemon.notice netifd: wan (17703): Sending select for 192.168.1.105...
Sat Apr 23 11:01:40 2016 daemon.notice netifd: wan (17703): Lease of 192.168.1.105 obtained, lease time 86400
Sat Apr 23 11:01:41 2016 daemon.notice netifd: Interface 'wan' is now up
Sat Apr 23 11:01:41 2016 user.notice firewall: Reloading firewall due to ifup of wan (eth0.1)
Sat Apr 23 11:01:42 2016 daemon.info dnsmasq[1505]: reading /tmp/resolv.conf.auto
Sat Apr 23 11:01:42 2016 daemon.info dnsmasq[1505]: using nameserver 8.8.8.8#53
Sat Apr 23 11:01:42 2016 daemon.info dnsmasq[1505]: using nameserver 8.8.4.4#53
Sat Apr 23 11:01:42 2016 daemon.info dnsmasq[1505]: using nameserver 192.168.20.1#53
Sat Apr 23 11:01:42 2016 daemon.info olsrd_hotplug: [OK] ifup: 'wan' => 'eth0.1'
Sat Apr 23 11:01:43 2016 daemon.debug olsrd_hotplug: [OK] interface 'wan' => 'eth0.1' not used for olsrd
Sat Apr 23 11:01:47 2016 daemon.notice netifd: Network device 'eth0' link is down
Sat Apr 23 11:01:47 2016 daemon.notice netifd: Interface 'lan' has link connectivity loss
Sat Apr 23 11:01:47 2016 daemon.notice netifd: VLAN 'eth0.1' link is down
Sat Apr 23 11:01:47 2016 daemon.notice netifd: Interface 'wan' has link connectivity loss
Sat Apr 23 11:01:47 2016 kern.info kernel: [ 1761.430000] eth0: link down
Sat Apr 23 11:01:47 2016 daemon.notice netifd: VLAN 'eth0.2' link is down
Sat Apr 23 11:01:47 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity loss
Sat Apr 23 11:01:47 2016 daemon.notice netifd: wan (17703): Received SIGTERM
Sat Apr 23 11:01:48 2016 daemon.notice netifd: Network device 'eth0' link is up
Sat Apr 23 11:01:48 2016 daemon.notice netifd: Interface 'lan' has link connectivity 
Sat Apr 23 11:01:48 2016 daemon.notice netifd: VLAN 'eth0.1' link is up
Sat Apr 23 11:01:48 2016 daemon.notice netifd: Interface 'wan' has link connectivity 
Sat Apr 23 11:01:48 2016 daemon.notice netifd: Interface 'wan' is setting up now
Sat Apr 23 11:01:48 2016 daemon.notice netifd: VLAN 'eth0.2' link is up
Sat Apr 23 11:01:48 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity 
Sat Apr 23 11:01:48 2016 kern.info kernel: [ 1762.430000] eth0: link up (100Mbps/Full duplex)
Sat Apr 23 11:01:48 2016 daemon.notice netifd: wan (17911): udhcpc (v1.22.1) started
Sat Apr 23 11:01:48 2016 daemon.notice netifd: wan (17911): Sending discover...
Sat Apr 23 11:01:49 2016 daemon.notice netifd: wan (17911): Sending select for 192.168.1.105...
Sat Apr 23 11:01:49 2016 daemon.notice netifd: wan (17911): Lease of 192.168.1.105 obtained, lease time 86400
Sat Apr 23 11:01:49 2016 daemon.info dnsmasq[1505]: reading /tmp/resolv.conf.auto
Sat Apr 23 11:01:49 2016 daemon.info dnsmasq[1505]: using nameserver 8.8.8.8#53
Sat Apr 23 11:01:49 2016 daemon.info dnsmasq[1505]: using nameserver 8.8.4.4#53
Sat Apr 23 11:01:49 2016 daemon.notice netifd: Interface 'wan' is now up
Sat Apr 23 11:01:50 2016 user.notice firewall: Reloading firewall due to ifup of wan (eth0.1)
Sat Apr 23 11:01:51 2016 daemon.info olsrd_hotplug: [OK] ifup: 'wan' => 'eth0.1'
Sat Apr 23 11:01:51 2016 daemon.debug olsrd_hotplug: [OK] interface 'wan' => 'eth0.1' not used for olsrd
Sat Apr 23 11:06:08 2016 kern.info kernel: [ 2020.220000] eth0: link down
Sat Apr 23 11:06:08 2016 kern.info kernel: [ 2021.240000] eth0: link up (100Mbps/Full duplex)
Sat Apr 23 11:06:08 2016 daemon.notice netifd: Network device 'eth0' link is down
Sat Apr 23 11:06:08 2016 daemon.notice netifd: Interface 'lan' has link connectivity loss
Sat Apr 23 11:06:08 2016 daemon.notice netifd: VLAN 'eth0.1' link is down
Sat Apr 23 11:06:08 2016 daemon.notice netifd: Interface 'wan' has link connectivity loss
Sat Apr 23 11:06:14 2016 daemon.notice netifd: VLAN 'eth0.2' link is down
Sat Apr 23 11:06:14 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity loss
Sat Apr 23 11:06:14 2016 daemon.notice netifd: Network device 'eth0' link is up
Sat Apr 23 11:06:14 2016 daemon.notice netifd: Interface 'lan' has link connectivity 
Sat Apr 23 11:06:14 2016 daemon.notice netifd: VLAN 'eth0.1' link is up
Sat Apr 23 11:06:14 2016 daemon.notice netifd: Interface 'wan' has link connectivity 
Sat Apr 23 11:06:14 2016 daemon.notice netifd: Interface 'wan' is setting up now
Sat Apr 23 11:06:15 2016 daemon.notice netifd: VLAN 'eth0.2' link is up
Sat Apr 23 11:06:15 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity 
Sat Apr 23 11:09:40 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:09:40 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 321 addresses
Sat Apr 23 11:09:40 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:10:17 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:10:17 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 318 addresses
Sat Apr 23 11:10:17 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:12 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:11:12 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 276 addresses
Sat Apr 23 11:11:12 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:17 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:11:17 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 224 addresses
Sat Apr 23 11:11:17 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:22 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:11:22 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 117 addresses
Sat Apr 23 11:11:22 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:27 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:11:27 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 52 addresses
Sat Apr 23 11:11:27 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:32 2016 daemon.info dnsmasq[1505]: read /etc/hosts - 5 addresses
Sat Apr 23 11:11:32 2016 daemon.info dnsmasq[1505]: read /var/run/hosts_olsr - 9 addresses
Sat Apr 23 11:11:32 2016 daemon.info dnsmasq-dhcp[1505]: read /etc/ethers - 1 addresses
Sat Apr 23 11:11:48 2016 kern.info kernel: [ 2362.690000] eth0: link down
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Network device 'eth0' link is down
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'lan' has link connectivity loss
Sat Apr 23 11:11:49 2016 daemon.notice netifd: VLAN 'eth0.1' link is down
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'wan' has link connectivity loss
Sat Apr 23 11:11:49 2016 daemon.notice netifd: VLAN 'eth0.2' link is down
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity loss
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Network device 'eth0' link is up
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'lan' has link connectivity 
Sat Apr 23 11:11:49 2016 daemon.notice netifd: VLAN 'eth0.1' link is up
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'wan' has link connectivity 
Sat Apr 23 11:11:49 2016 daemon.notice netifd: VLAN 'eth0.2' link is up
Sat Apr 23 11:11:49 2016 daemon.notice netifd: Interface 'dtdlink' has link connectivity 
Sat Apr 23 11:11:49 2016 kern.info kernel: [ 2363.690000] eth0: link up (100Mbps/Full duplex)
 
Top
Sat, 04/23/2016 - 12:04
(Reply to #4)
KG6JEI
I'm seeing a number of
I'm seeing a number of network port link loss reports for the wired port.  When the port looses its link the tunnels would be expected to fail. You may want to check the network cable and possibly replace the cable to rule out a problem there.

It could also be the switch (less likely) or the hardware (especially if you have had a lighting storm recently) but the cable would look to be the most likely culprit to start with.

Also at the time of the dump no WAN port IP address was shown on the node but I would expect this to be related to the port issues and that DHCP may have a problem with all the ups/downs or that it just hadn't fired off another request yet.
  
[  745.580000] eth0: link down
[  747.620000] eth0: link up (100Mbps/Full duplex)
[  939.160000] eth0: link down
[  940.160000] eth0: link up (100Mbps/Full duplex)
[ 1021.020000] eth0: link down
[ 1022.020000] eth0: link up (100Mbps/Full duplex)
[ 1312.340000] eth0: link down
[ 1314.510000] eth0: link up (100Mbps/Full duplex)
[ 1601.120000] eth0: link down
[ 1603.120000] eth0: link up (100Mbps/Full duplex)
[ 1751.430000] eth0: link down
[ 1753.430000] eth0: link up (100Mbps/Full duplex)
[ 1761.430000] eth0: link down
[ 1762.430000] eth0: link up (100Mbps/Full duplex)
[ 2020.220000] eth0: link down
[ 2021.240000] eth0: link up (100Mbps/Full duplex)
[ 2362.690000] eth0: link down
[ 2363.690000] eth0: link up (100Mbps/Full
Top
Sun, 04/24/2016 - 06:56
(Reply to #5)
kc8rgo
I will checkout the cables
I will checkout the cables etc.  I will be getting 2.5 WIDI in the next week or so.  My cat5 links are about 50 feet.

I have a triangle Tunnel arrangement as a client with K7OPA and KE2N.  K7OPA had an Tunnel with KE2N and is a server to me.  The weird part is that ALL the blue clouds for K7OPA and me KC8RGO went away at about the same time.  
Top
Sun, 04/24/2016 - 10:05
(Reply to #6)
K5DLQ
K5DLQ's picture
if they all went away at the

In my opinion, if they all went away at the same time, that would indicate an "upstream" internet outage/glitch.
 

Top
Sun, 04/24/2016 - 09:51
AE6XE
AE6XE's picture
Based on the log file, all
Based on the log file, all the mesh nodes with K7OPA and KE2N would have started showing for 1 or 2 minutes in mesh status, then disappeared for 1 or 2 minutes--and did this sequence 4 times.   This is the outcome when the mesh node loses connectivity with the internet.  

"eth0" is the physical network port and traffic has stopped on that port when it goes 'down'.  eth0.1 is vlan 1 traffic (WAN) related traffic.  eth0.2 is  vlan 2 related  traffic (dtdlink).     The node would have only been accessible over RF through another mesh node when eth0 is down.  

Something failed when the traffic started to ramp up on the network port.  One possible option is that the network port itself has a hardware failure.  But hopefully, replacing a cable will do the trick.
 
Top
Mon, 04/25/2016 - 08:04
kc8rgo
My original config was:

My original config was:
laptop - GS105E w/ IP phone, to NSM2, to Linksys WRT150N (standard Router), to Ubiquiti M900 (WISP).

Over the weekend, changed config to eliminate or change out cables etc:
laptop - ARHP (AREDN) w/ IP phone - new cable to WRT150N - Ubiquiti M900 (WISP).
NSM2 RF remote node w/ GS105E w/ IP phone.

ARHP is Tun Client to K7OPA, NSM2 is Tun Client to KE2N

While working on tests with the IP phones, ARHP lost visual (i.e. the nodes supported by the K7OPA Tun Server ) tunnel access to K7OPA, while the NSM2 (Remote node to the ARHP) maintained visual w/ KE2N (i.e. the nodes supported by the KE2N Tun Server) through multiple REFRESHes of the NSM2.

I believe that with the changes I have at least changed out a good portion of my original Mesh config without removing the problem.  The unique thing here is K7OPA lost his Tun Server connection to KC8RGO as well as his connect to Ke2N at about the same timeframe as I lost my Tun Client connection to K7OPA.  However if the problem is an internet outage upstream or a cable problem connecting my nodes to the i-net, I would have expected that my Tun Client on the remote node NSM2 would have dropped out as well since it passes through the ARHP.  A reboot of the ARHP later restores normal ops.

The above added for FYI purposes.

 

Top

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer