Mysterious Internet Outages

[Piyono]

My internet's been going out daily since I configured the network as in the image below, and I'm darn near my wits' end trying to figure out what's going on.

The outages come once or twice a day, without out apparent rhyme or reason or correlation to any of my activities.

When an outage occurs the router remains responsive and the OpenWrt overview shows that the WAN connections are up and running. In fact, almost nothing appears out of the ordinary in the GUI.

The only way I've been able to restore service is with a reboot, either from within LuCI or at the power switch.

I've tried factory-resetting and re-flashing OpenWRT (currently at v 19.07.5) but that doesn't help.

I've been logging everything I can using luci-app-statistics. Below is a screencap of the RRDTOOL report.

The only erratic behaviors I can identify are
a) the sudden surge in activity on the loopback adaptor which begins immediately at the start of the outage and ceases only when the router is reset. Loopback is quiet otherwise.

and b) the spike in CPU load at the point of outage. The router is definitely doing something, just not what I want.


I'm also capturing the system log to a file but I see nothing in it that would suggest that an outage is about to occur.
For example, here's a snippet of the system log corresponding to the screenshots above:

Sun Dec 13 14:56:49 2020 daemon.info dnsmasq-dhcp[1783]: DHCPDISCOVER(br-lan) xx:xx:xx:xx:xx:xx
Sun Dec 13 14:56:49 2020 daemon.info dnsmasq-dhcp[1783]: DHCPOFFER(br-lan) 192.168.1.109 xx:xx:xx:xx:xx:xx
Sun Dec 13 14:56:49 2020 daemon.info dnsmasq-dhcp[1783]: DHCPREQUEST(br-lan) 192.168.1.109 xx:xx:xx:xx:xx:xx
Sun Dec 13 14:56:49 2020 daemon.info dnsmasq-dhcp[1783]: DHCPACK(br-lan) 192.168.1.109 xx:xx:xx:xx:xx:xx mercku
Sun Dec 13 16:16:24 2020 daemon.err uhttpd[1280]: luci: accepted login on / for root from 192.168.1.222

192.168.1.222 is me logging in to reboot the router.

What should be my next troubleshooting step? I sit here all day watching graphs and system log and kernel logs but I'm not sure that these are giving me what I need.

I'm going to try swapping the TP-Link for the D-Link router in the diagram that's currently being used as a switch and see if that changes anything.

The paranoid delusional in me is wondering whether this is the effect of malicious software operating on a device within or without my network.

Any tips?

 

[Piyono]

I suppose there could also be a hardware issue with the TP-Link. I swapped it for the D-Link running DD-WRT. So far (9 hours) no dropouts.

 

[fb]

I almost don't know anything about networking. But at my previous job I had the same problem when an "extra" switch in an office gave up with age and gave the same symptoms as you mention.
The troubleshooting process was the same as you are doing now, disconnect equipment, one at a time, and see if the problems go away.

I hope you find the problem. It looks promising.

 

[sechs]

What was the configuration before?

 

[Piyono]

The previous configuration involved a DSL modem in the garage and there was no cable running between the office and garage so it's hard to compare. Anyway, right now we're using the Mercku as the primary router, which kind of sucks because the firmware is very basic but it also hasn't dropped the connection.