[W600-PICO] Issues with MicroPython v1.20.0

[AlexYeryomin]

Hi,

I have updated Wemos W600-PICO to the latest available version from https://github.com/robert-hh/Shared-Stuff/blob/master/w600_firmware/wm_w600_WEMOS_W600.fls built by @robert-hh (thank you, again, for this!)

I have performed the updating routine as usual, erased and then flashed the module using w600tool.py. The firmware reports on the startup:

../../lib/littlefs/lfs2.c:1223:error: Corrupted dir pair at {0x0, 0x1}
MicroPython v1.20.0-384-gaf40c48a3 on 2023-08-14; Wemos W600 Pico with W600-B8
Type "help()" for more information.

I tried to erase again, re-flash, it didn't help. I tried on a couple of W600-PICO, the same behavior. Interesting, I can upload & download files without any problem, and test programs run properly as well, except the network, though.

Not sure if it relates to this issue or not, but the module can't connect to my home network anymore. There is a simple test that doesn't work, connect() never returns:

import easyw600
easyw600.connect(ssid, password)

It never returns because it is waiting for the connections status to be connected, but it doesn't happen. To get more information what is going on, I wrote this code:

    sta_if = network.WLAN(network.STA_IF)
    sta_if.active(True)
    sta_if.connect(ssid, password)
    while True:
        s = sta_if.status()
        print(s) # See network.py for status explanation.        
        if s != network.STAT_CONNECTING:
            break
        utime.sleep(1)
    return sta_if if sta_if.isconnected() else None 

When this code is running, it is printing the status of the connection. As expected, it is a series of code 1 ("connecting in progress"), but then suddenly it returns code 4 ("other problems"). See codes here: https://docs.micropython.org/en/latest/library/network.WLAN.html#network.WLAN.status

connecting in progress (code 1)
. . .
connecting in progress (code 1)
other problems (code 4)

It takes 15-25 seconds to get this error. What is weird: as soon as I run this code, I can see a new device connected to my router (in its admin console), and the router assigns a proper IP, and it shows an excellent signal. After the code 4 is printed, the device is marked as disconnected on the router's list.

I tried on a couple of W600-PICO, the same behavior. Also, I have tried the same code on ESP32, ESP8266 (S2, D1 mini, etc.), Micropython v1.20, everything works as expected.

If I use AP on W600-PICO, it works just fine. I can configure it, I see it in available WiFi networks, I can connect to it, no problem whatsoever. So, it seems the network does work on the module, but for some reason when it connects as STA to the router, it doesn't realize that the connection has been already established (and waiting for something else from the router?).

MP v1.19_ doesn't report any issues on the same hardware.

The question is: has anyone else faced these issues?

[robert-hh]

I can confirm the issue. Reason: Due to recent updates of the firmware base the code size increased and now file system and code overlap. I will push new versions during the day with smaller code (by removing the frozen python script pye.py).

[robert-hh]

The firmware files are updated. In my test here there error is gone, and connecting WiFi works well.

[AlexYeryomin]

Oh, that was the reason!

The new firmware is much larger than the previous version, compare 701,724 bytes (current wm_w600_WEMOS_W600.fls) vs. 616,796 bytes (the previous wm_w600_lfs.fls). I have updated the firmware, but the 1st issue is still there:

../../lib/littlefs/lfs2.c:1223:error: Corrupted dir pair at {0x0, 0x1}
MicroPython v1.20.0-481-g2a733133d-dirty on 2023-09-05; Wemos W600 Pico with W600-B8
Type "help()" for more information.

But! The network works now!!! The log shows:

connecting in progress (code 1)
connecting in progress (code 1)
connecting in progress (code 1)
connecting in progress (code 1)
got IP (code 5)
network config: ('192.168.2.29', '255.255.255.0', '192.168.2.1', '192.168.2.1')

I tested this on a couple of modules, both work. Puh! What a relief. I use W600-PICO as a network adapter for PI PICO, and I would like to continue using it, for time being. Well, it is a small piece of code, but it works, so "don't touch it".

Dear Robert, thank you so much!!!

[robert-hh]

This error message about the file system should not appear. I do not see it here on my Wemos W600 board with the same firmware than your's.
`MicroPython v1.20.0-481-g2a733133d-dirty on 2023-09-05; Wemos W600 Pico with W600-B8

You could try to re-create the file system with:

import os
import w600

bdev = w600.Flash()
os.VfsLfs2.mkfs(bdev, progsize=256)
vfs = os.VfsLfs2(bdev, progsize=256)
os.mount(vfs, "/")

This code is usually executed on boot when no file system is found or cannot be mounted.
There were many smaller and larger changes since v1.19.1, each of them may contribute to the increased code size. But most of it should be caused by re-enabling SSL support.

[AlexYeryomin]

I didn't know this trick. I recreated the fs, and this resolved the "corrupted dir" message at startup. Just in case, to make sure this wasn't a glitch, I reuploaded the latest firmware; the error message is here again. Well, not big issue as now I know how to fix this.

Everything else I tried so far works properly.

[robert-hh]

I tried to replicate youir observation. Took a WEMOS W600 Pico, uploaded the file wm_w600_WEMOS_W600.fls from the web site using w600tool.py with the Linux command line:
./w600tool.py -p /dev/ttyUSB0 -b 115200 --upload-baud 115200 -u wm_w600_WEMOS_W600.fls
The device boots and shows the prompt:
MicroPython v1.20.0-482-gc7c45a36a on 2023-09-07; Wemos W600 Pico with W600-B8

Running import os;os.statvfs("") I get:
(4096, 4096, 68, 66, 66, 0, 0, 0, 0, 255)
So 4096 * 68 is the total file system size of 272k. Could you try to run this import os;os.statvfs("") and report the result?

Some figures:
The flash area reserved for code has a size of 0xa7F00. The code starts at 0x80010100. There is another area of 0x4000 reserved for static data. So the file system starts at 0x80010100 + 0xa7f00 + 4000 = 0x800bc000 and ends at0x800fffff for a WEMOS W600 Pico That's a size of 0x44000 = 272k.
The actual code size is 0x9b5d4, which is definitely smaller than 0xa7f00. So everything should fit, with 50k extra space available.

[AlexYeryomin]

I have 2 modules Wemos' W600 at hands that I can test (other ones are identical). I erased and re-uploaded the latest firmware, this is what I got:

>>> import os;os.statvfs("")
(4096, 4096, 68, 65, 65, 0, 0, 0, 0, 255)

So, 278528 bytes total, 266240 bytes free.

Just out of curiosity, I recreated the fs as you suggested above, I got:

>>> import os;os.statvfs("")
(4096, 4096, 68, 66, 66, 0, 0, 0, 0, 255)

A bit more free space, 266240 bytes. I believe this is because after recreating the fs, 2 files were deleted (boot.py, main.py).

As soon as the fs has been created, there is no error message on corrupted dir anymore. Maybe my modules are somehow different / counterfeit? When buying something on AliExpress, you never know what will come.

Well.. Why I wanted to update to MP v1.20 is "mip". I just wanted to see how this work. But, it doesn't work at all:

>>> import mip
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mip/__init__.py", line 1, in <module>
  File "urequests.py", line 1, in <module>
ImportError: no module named 'usocket'

I will try to add missing Python files.

Anyway, it seems the new firmware works just fine with the network; that's the only critical part for me. Other things like "mip" I can test on ESP8266 or ESP32 modules.

[robert-hh]

As part of the recent firmware updates, must of the 'u'-prefixed modules lost their 'u'- So usocket is now socket. You can tell which modules are built-in to the firmware by calling:
help("modules").
But using usocket should still be possible, like the urequsts module does. Some setting is missing. I have to find the respective commit.

[robert-hh]

Found the respective commit and the change needed to make import usocket and mip work. It was one of Jimmo's round-kick commits, which change a lot at many place, and it's sometimes hard to see what has been affected.
The w60x firmware files are updated.

Edit. Thanks for using, trying and testing. That reveals open topics and is very helpful.

[AlexYeryomin]

I have tested the latest firmware. mip works as expected. I had to add "/lib" to the system path before installing packages. I tried to install the standard module and a custom one; both methods work:

import mip
import network_utils as nu # A module to work with Wi-Fi.

sys.path.append("/lib")
net = nu.connect(...)
if net:
    mip.install("ssd1306")
    mip.install("https://raw.githubusercontent.com/robert-hh/SH1106/master/sh1106.py") 

MicroPython v1.20.0-483-g688bb2449 on 2023-09-09; Wemos W600 Pico with W600-B8
Type "help()" for more information.
connecting in progress (code 1)
connecting in progress (code 1)
got IP (code 5)
Installing ssd1306 (latest) from https://micropython.org/pi/v2 to /lib
Copying: /lib/ssd1306.mpy
Done
Downloading https://raw.githubusercontent.com/robert-hh/SH1106/master/sh1106.py to /lib
Copying: /lib/sh1106.py
Done

Great. I don't know yet where I would need mip, just didn't think if it would be useful for me, but I am glad that it works (and yet another test for the network subsystem, too).

Robert, thank you for these fixes! I am always ready to test and try new firmware you prepared.

(I can't rebuild the firmware on my own at the moment. I have tons of tool chains for various MCU installed on Ubuntu 18.04, and it seems there was some conflict when I added gcc you recommended for WM_SDK. I don't want to touch this till I complete my project for STM32F103 MCU related to hacks for hoverboard firmware. I should use a VM for experiments, but I am too lazy to deal with this...)

[rkompass]

Well.. Why I wanted to update to MP v1.20 is "mip".

Have you noted that there is much more memory available now? That might be another reason to change to 1.20. (with the W600).

[AlexYeryomin]

Oh, yes, the available memory is twice as much with v1.20:

MicroPython v1.19.1-1019-g84040f230-dirty on 2023-04-20; WinnerMicro module with W600
Memory: 54912 bytes total, 53584 bytes free (97.6%)
Storage: 180224 bytes total, 178688 bytes free (99.1%)

MicroPython v1.20.0-483-g688bb2449 on 2023-09-09; Wemos W600 Pico with W600-B8
Memory: 105280 bytes total, 101872 bytes free (96.8%)
Storage: 278528 bytes total, 270336 bytes free (97.1%)

This is nice. For my needs, it means a longer time to keep collected data before transmission.

system_info.zip

[robert-hh]

Thanks for the hint. I extended the firmware to include /lib into sys.path and to create /lib with a new file system. That's done in scripts/_boot.py, which runs at every boot event.

Edit: b.t.w., the command line for creating the firmware is:

make V=s SECBOOT=1 CODESIZE=0xa8000 BOARD=WEMOS_W600

You could try to reduce CODESIZE a little bit, if you need more space in the file system. 0xa4000 should work as well, gaining 16k. And you can set SECBOOT=0, which adds another 56k to the file system.

[robert-hh]

The uypsh module from micrpython-lib is embedded as frozen module. That add a few useful IX like commands like ls, cat, mkdir, rmdir, rm ... Start it with from upysh import *.