Stacks becoming unresponsive

[Superp]

Hello,

I am having some trouble with two stacks, both becoming unresponsive when connected via Wifi.

Last week I

1. received two Master Brick 3.1's and two WIFI Master Extension 2.0's
2. plugged the extensions each on top of a Master Brick
3. connected each stack via USB, and in Brickv:
   • configured the extension
   • configured and saved Wifi
   • reset
4. unplugged the stacks and powered them up with separate power supplies

Immediately, both stacks had the same problem (when I boot up just 1 of the stacks):

• They boot up and connect to WLAN
• At this point, they behave normally:
  • They respond to pings (response <15ms)
  • They can be discovered and used in Brickv
  • They can be discovered and used via Bindings
• After some time (between 5 seconds and 10 minutes, approx), the stacks stop responding: each of the three methods above just produces timeouts
• To get back online, I have to reset the Master Bricks or disconnect/reconnect power.

The most extreme case is where a stack booted, I ping'ed it, I received 3 responses and then only timeouts.

Other times, a stack works normally for up to 10 minutes, then hangs.

I have also tried postponing first connection for 10 minutes after booting. First connection after 10 minutes also results in timeouts, so the stack "hangs" by itself (?).

The WLAN is otherwise stable and fast; the router is an ASUS RT-AX86U.

Things I have tried:

• Fixed IP vs DHCP assigned IP
• Different power supplies
• Reposition stack in relation to router
• Configure different Wifi standards (b/g/n)
• No Bricklets attached vs Bricklets attached
• Connect from a different host (MacOS vs Pi OS)
• Reboot Wifi router
• Repeat init/config of stack from start

While trying to solve the problem, there were distractions:

• I discovered the Wifi status LED behaves oddly. This issue is thought to be unrelated. Discussed here.
• I discovered the Master Bricks return unexpected values for chip temperature. Also thought to be unrelated. Discussed here.
• One of the Master Bricks died; it appears stuck in bootloader mode. Solution still pending. Discussed here.

This is the dialogue Brickv produces when a stack times out:

 

 

This is what happens when I ping a stack right after it boots, and again 15 minutes later. No connection attempts in between:

m1:~ ping 192.168.1.41
PING 192.168.1.41 (192.168.1.41): 56 data bytes
64 bytes from 192.168.1.41: icmp_seq=0 ttl=128 time=92.117 ms
64 bytes from 192.168.1.41: icmp_seq=1 ttl=128 time=9.247 ms
64 bytes from 192.168.1.41: icmp_seq=2 ttl=128 time=6.411 ms
64 bytes from 192.168.1.41: icmp_seq=3 ttl=128 time=4.392 ms
64 bytes from 192.168.1.41: icmp_seq=4 ttl=128 time=5.195 ms
64 bytes from 192.168.1.41: icmp_seq=5 ttl=128 time=6.888 ms
^C
--- 192.168.1.41 ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 4.392/20.708/92.117/31.971 ms
m1:~ ping 192.168.1.41
PING 192.168.1.41 (192.168.1.41): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
^C
--- 192.168.1.41 ping statistics ---
6 packets transmitted, 0 packets received, 100.0% packet loss

Let me know if you need more info.

 

[Superp]

[rtrbt]

I'm currently trying to reproduce this. So far the WiFi connection still works.

Is this just some dirt or a bit of solder? Can you scrape it away? (Just to make sure this does not short-circuit anything)

[Superp]

That is why I uploaded the image.

The four rightmost contacts look like they have been hand-soldered, post assembly.

There are also traces of heating visible on the board.

The speck looks metallic. It is firmly stuck on the 5th pad from the right.

[...]

I scrapped it off, it is a tiny bit of metal.

[...]

Flashing the extension to 2.1.5

[...]

Flashing the extension to 2.1.6 (!!)

More to follow

[Superp]

Same problem as originally described.

[rtrbt]

Hi,

The new firmwares only fix unrelated problems, for example 2.1.5 fixes the LED issue you've found.

On 12/17/2021 at 2:43 PM, Superp said:

The four rightmost contacts look like they have been hand-soldered, post assembly.

Yeah, this does not look perfect. Are the WiFi problems also happening with the other extension? If yes, I would assume that they are not caused by bad soldering.

Some questions to find out on which level this is broken:

• If you configure an extension to the Client + AP mode the status LED should be blinking continously. When the stack becomes unresponsive, does the LED still blink?
• When a stack becomes unresponsive, and you plug in a USB cable, can you still reach the stack on localhost via USB?

On 12/15/2021 at 2:47 PM, Superp said:

I have also tried postponing first connection for 10 minutes after booting. First connection after 10 minutes also results in timeouts, so the stack "hangs" by itself (?).

Did you only postpone TFP connections or also pings etc?

Another idea: Does this also happen if you don't connect the stack to your network at all, but instead only use its access point?

[Superp]

33 minutes ago, rtrbt said:

Are the WiFi problems also happening with the other extension?

Yes, as I said, both stacks have the same problem. Both extensions were part of the same order.

35 minutes ago, rtrbt said:

When a stack becomes unresponsive, and you plug in a USB cable, can you still reach the stack on localhost via USB?

The stacks are USB-powered, so I can't plug in USB after they become unresponsive.

But when I connect (& power) via USB, the Wifi will become unresponsive as described, but the stack keeps responding via USB/localhost.

51 minutes ago, rtrbt said:

Did you only postpone TFP connections or also pings etc?

I postponed any and all connections, including pings.

If I have the time today, I will try to configure a stack: a) AP only, b) client+AP, and see what happens.

[Superp]

Some new observations.

• First, I configured a stack as AP only, letting it create its own network, USB power-only. After 30 minutes, the stack was still responding normally via that dedicated network.
• Second, I re-configured the same stack as client only and hooked it up via USB to a Mac. It soon stopped responding on its IP address. It is then still possible to connect via USB/localhost, as I said. When I do, the Wifi status in Brickv still shows changes in Signal Strength. So the extension seems to still be running, but it no longer responds on the network.
• Third, I re-configured the same stack as client and AP, USB power-only. This has now been running for an hour. When I connect to the AP (dedicated SSID), the stack still responds. When I connect to the client (regular Wifi network), it times out.

The problem seems to be specific for the Wifi client function.

Let me know i you have any ideas.

[rtrbt]

To be honest, I am out of ideas. As a last resort: Can you test the WiFi client mode with another access point? (If you don't have one available you could use one WiFi extension in AP mode and connect the other one to it) Maybe there is a firmware update available for your AP?

If nothing of this helps, we can offer you to either exchange the extensions for new ones (but as the issue occurs with both extensions, I would not expect this to magically solve it) or you can of course return the extensions.

[Superp]

We'll setup an alternate AP to test this. I will get back to you after the Christmas weekend.

[Superp]

Since a few days we have a GL.iNET Mango setup as a second AP with a separate SSID serving only Tinkerforge stacks on 2.4GHz.

The main router powers the Mango from one of its USB ports.

Connections for both stacks have been stable for a few days with this setup.

So it is indeed the Wifi extensions not getting along with the big ASUS router.

[Superp]

Unfortunately I have to retract what I wrote above. The Wifi extensions do not maintain a stable connection to the Mango router either. The longest time the connection survived was about 2.5 days; usually a reset is required daily. This is not workable.

Unless a magic solution comes up, I will remove the extensions and Master Bricks from the project soon.

[batti]

Hi Superp,

it's really weird. What we can try is to send you a new set of hardware to make sure that it is not an hardware issue. In that case please contact us via email.

Best regards,

Bastian