Increasing Code Storage Space on STM32F405 Board

[daleka]

Hi everyone,

I am working with a Pyboard STM32F405, and after flashing MicroPython, I am left with only about 90 kilobytes of free memory for storing code. This has proven to be quite limiting for my project, and I am looking for ways to increase this space by at least 2–3 times.

Here’s what I’ve considered so far:

Using external memory (e.g., an SD card or SPI Flash). However, for my project, it’s important that the code is stored and executed directly in the internal memory of the microcontroller.
Optimizing the MicroPython firmware. Is it possible to reduce the firmware size to free up more memory for user code?
Other methods specific to the STM32F405.
For my project, I only require UART support, and all 5 available UART interfaces must be active. Other functionalities (e.g., Ethernet, SPI, I2C, ADC, etc.) are not needed and can be disabled to free up additional space.

I’d greatly appreciate any advice or links to resources that could help me solve this issue!

Thank you for your help!

[rkompass]

Hi daleka,

the STM32F405 has 192 kB of RAM in total but only 128 kB are usable for Micropython (fully accessible by the CPU including for heap, stack, code execution). The other RAM (64 kB) is used for buffers (IO). There were successful attempts to use the other RAM for heap IIRC.
Anyway, you cannot increase this space by at least 2–3 times which would be 90 x 3 = 270 kB. It seems you need a different board.

[daleka]

Hi daleka,

the STM32F405 has 192 kB of RAM in total but only 128 kB are usable for Micropython (fully accessible by the CPU including for heap, stack, code execution). The other RAM (64 kB) is used for buffers (IO). There were successful attempts to use the other RAM for heap IIRC. Anyway, you cannot increase this space by at least 2–3 times which would be 90 x 3 = 270 kB. It seems you need a different board.

Hi,

Thank you for your response. However, I believe there might be a misunderstanding. The issue I’m referring to is with the flash memory, not the RAM.

The STM32F405 has 1 MB of flash memory, which is where MicroPython stores both the firmware and user code. After flashing MicroPython, I am left with only about 90 KB of flash memory available for my code. This is the limitation I'm trying to address.

I’m not experiencing any issues with RAM — the 128 KB available for MicroPython is sufficient for heap, stack, and runtime execution in my project.

The question remains: how can I free up more space in the flash memory for user code? Whether through firmware optimization, disabling unnecessary modules, or any other method.

If you have any advice or suggestions specific to increasing available flash memory, I’d be grateful to hear them.

[shariltumin]

The easiest solution would be to use SD card. You will probably need "sdcard.py" or something similar. Now your main.py can look like this:

import pyb, sdcard, os, sys
sd = sdcard.SDCard(pyb.SPI(1), pyb.Pin.board.X5)
pyb.mount(sd, '/sd')
os.chdir('/sd') # os.getcwd() == '/sd'

# import your app (myapp.py or myapp.mpy) from SDcard
from myapp import prog

prog() # and run it

I hope this can be of some help.

[daleka]

Thank you so much for your suggestion and for taking the time to help! I really appreciate your input.

However, I wanted to point out a potential issue with using an SD card for storing the code. While it does provide more storage capacity, it significantly increases the consumption of RAM, which is already quite limited on the STM32F405.

Here’s why:

When the code is stored on an SD card, it cannot be executed directly from the card. Instead, the code must first be loaded into RAM before it can be executed. This is because SD cards do not support "execute in place" (XIP) functionality, unlike the internal flash memory of the STM32F405.

On the other hand, when the code is stored in the internal flash memory, it can be executed directly from there, without consuming additional RAM for loading the code. This is especially important for microcontrollers like the STM32F405, where the available RAM (128 KB for MicroPython) is already used for the heap, stack, and runtime operations.

Given these constraints, while an SD card is an excellent option for storing large data files or additional resources, using it to store and execute code can quickly exhaust the available RAM, which is a critical resource for my project.

Thank you again for your help, and I truly appreciate your insights!

[shariltumin]

While not XIP, making you whole application script and all the needed libraries into Frozens modules can help reduce RAM usage. However, the bytecode must be copied to RAM for execution (i.e. not XIP). You can use "boards/manifest.py" to include your specific script files.

You may also want to modify ports/stm32/mpconfigport.h to disable modules that you do not need, for example:

#define MICROPY_PY_BUILTINS_HELP    (0)
#define MICROPY_PY_BUILTINS_HELP_MODULES (0)
#define MICROPY_PY_HASHLIB_MD5      (0)
#define MICROPY_PY_HASHLIB_SHA1     (0)
#define MICROPY_PY_CRYPTOLIB        (0)
#define MICROPY_PY_LWIP_PPP         (0)
#define MICROPY_PY_LWIP_SOCK_RAW    (0)
#define MICROPY_PY_WEBSOCKET        (0)
#define MICROPY_PY_WEBREPL          (0)
#define MICROPY_PY_SOCKET           (0)
#define MICROPY_PY_NETWORK          (0)

[rkompass]

Hi daleka,

now I see. Two relevant discussions are here and here.

Extending the approach by @saraverbeecke you might go with:

MEMORY
{
    FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 1024K /* entire flash */
    FLASH_ISR (rx)  : ORIGIN = 0x08000000, LENGTH = 16K   /* sector 0 */
    FLASH_FS (rx)   : ORIGIN = 0x08004000, LENGTH = 176K  /* sectors 1,2,3 are 16K, 4 is 64K, 5 is 128K (64K used) for filesystem */
    FLASH_FS2 (rx)  : ORIGIN = 0x08040000, LENGTH = 64K   /* sector 6 is 128K (64K used) for filesystem */
    FLASH_FS3 (rx)  : ORIGIN = 0x08060000, LENGTH = 64K   /* sector 7 is 128K (64K used) for filesystem */
    FLASH_TEXT (rx) : ORIGIN = 0x08080000, LENGTH = 512K  /* sectors 8,9,10,11 are 128K each */
    CCMRAM (xrw)    : ORIGIN = 0x10000000, LENGTH = 64K   /* used for filesystem cache */
    RAM (xrw)       : ORIGIN = 0x20000000, LENGTH = 128K
}

to have 176 + 64 + 64 = 304 kB of Flash for filesystem.
This would be possible even without reducing the MP image, as in it's standard configuration it is not larger than ~ 400 kB.
(citing @jimmo :)

The PYBV11 firmware only needs about ~400k, so in theory there's a fair bit that can be reallocated to the filesystem.

So even another 64 kB of flash (or more) should be possible with modest (or graver) restrictions on MP functionality and a dedicated build.

[rkompass]

I see now that also flashbdev.c has to be adapted which I just did :-), file is attached. It uses the definitions

/* Filesystem cache in RAM, and storage in flash */
_micropy_hw_internal_flash_storage_ram_cache_start = ORIGIN(CCMRAM);
_micropy_hw_internal_flash_storage_ram_cache_end = ORIGIN(CCMRAM) + LENGTH(CCMRAM);
_micropy_hw_internal_flash_storage_start = ORIGIN(FLASH_FS);
_micropy_hw_internal_flash_storage_end = ORIGIN(FLASH_FS) + LENGTH(FLASH_FS);
_micropy_hw_internal_flash_storage2_start = ORIGIN(FLASH_FS2);
_micropy_hw_internal_flash_storage2_end = ORIGIN(FLASH_FS2) + LENGTH(FLASH_FS2);
_micropy_hw_internal_flash_storage3_start = ORIGIN(FLASH_FS3);
_micropy_hw_internal_flash_storage3_end = ORIGIN(FLASH_FS3) + LENGTH(FLASH_FS3);

given in an updated linker script.

In flashbdev.c for usage of the 2nd and 3rd flash segments the definitions

MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE_SEGMENT2
MICROPY_HW_ENABLE_INTERNAL_FLASH_STORAGE_SEGMENT3

are evaluated, which again have to be set in ports/stm32/mpconfigport.h, I suppose.

I wonder if the maximum number of possible flash storage segments should once and for all be set to 4 or even 5.
Similar changes as I just did.
And the code in flashbdev.c shows that there is no negative impact on performance (or code size) if these are not activated by the corresponding definitions in mpconfigport.h.

flashbdev.zip
stm32f405.zip

[rkompass]

Forgot the change to TEXT1_ADDR = 0x08080000 in boards/PYBV11/mpconfigboard.mk.
So the dfu-util placed the binary at 0x08020000 instead of 0x08080000.
As a consequence the board is unaccessible. Even reset or factory reset do not work anymore.
I cannot even verify yet that this last change is sufficient to get a working board (although I think so).
I will have to erase and flash the board using ST Link.

[rkompass]

After this change the Pyboard works and with:

import os, pyb
os.umount('/flash')
os.VfsLfs2.mkfs(pyb.Flash(start=0))
os.mount(pyb.Flash(start=0), '/flash')
os.chdir('/flash')

(from the docs) the flash filesystem displays a size of 304 (or 368) kB.

I did tests of the filesystem which passed. The test is in attached fs_test.py.

After being successful with this I compiled a version with 368 kB of flash filesystem, that also passed the tests.
Test run and result:

>>> run(n=40, flen=9000)
File system free space: 367.0 KB
create file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 
FS free: 276.0 KB. Bytes written: 90000 B. FS usage per Bytes: 1.04
create file10 file11 file12 file13 file14 file15 file16 file17 file18 file19 
FS free: 184.0 KB. Bytes written: 180000 B. FS usage per Bytes: 1.04
create file20 file21 file22 file23 file24 file25 file26 file27 file28 file29 
FS free: 92.0 KB. Bytes written: 270000 B. FS usage per Bytes: 1.04
create file30 file31 file32 file33 file34 file35 file36 file37 file38 file39 

FS free: 0.0 KB. Bytes written: 360000 B. FS usage per Bytes: 1.04
-------------------------------------
checking file content... all correct
-------------------------------------
delete file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 
FS free: 91.0 KB. Bytes written: 270000 B. FS usage per Bytes: 1.05
delete file10 file11 file12 file13 file14 file15 file16 file17 file18 file19 
FS free: 182.0 KB. Bytes written: 180000 B. FS usage per Bytes: 1.05
delete file20 file21 file22 file23 file24 file25 file26 file27 file28 file29 
FS free: 275.0 KB. Bytes written: 90000 B. FS usage per Bytes: 1.05
delete file30 file31 file32 file33 file34 file35 file36 file37 file38 file39 

File system free space: 367.0 KB
files: 40,  len: (9000,), mism: 0, fsusg: 375808, usgmxrat: 1.043911
(0, 375808, 1.043911)

Both binaries are attached as dfu files.
You may flash the Pyboard with dfu-util the same way like the official images.
Subjectively the filesystem with 368 kB has gotten slower (did not measure it, perhaps you like to do it).

fs_test.zip
Pyb11_MP1_25Prv129_304kFlash.zip
Pyb11_MP1_25Prv129_368kFlash.zip

[daleka]

Hi rkompass!

I just wanted to say a big thank you for all the help you’ve provided. Your advice on optimizing flash memory and the file system was a real lifesaver for me! I honestly didn’t expect someone to spend so much time explaining everything in detail and even testing the solution.

Your explanations were incredibly helpful, and I’ve already started implementing the changes. Without your support, I’d still be struggling with this problem. I really appreciate you sharing your expertise and all the technical insights.

Thanks again, and if you ever need help with something, I’d be happy to return the favor!

Best of luck with your projects! 🙌

[rkompass]

Hi daleka,

many thanks for the feedback.
This was an interesting problem and the sort of that I like: Could learn something I didn't know before,
and start from existing solutions and improve. Of course your request was a big motivation.
I experience that even programming is a very social thing (like everything else obviously).
And I have the impression that we (MP community) do not use the potential of this for motivation too much yet.
I appreciate the explanations others give in this forum a lot. Actually it is much fun to follow most of the discussions here.
So it is nice to hear that my findings are relevant to you and perhaps others (despite my very limited knowledge).

Now having solved the flash increasing problem for the pyboard in a first run, a few new ideas appeared:

1. Shouldn't the flash layout be changed so that the TEXT (i.e. program code) always is at TEXT1_ADDR = 0x08020000?
   This would make changes to boards/PYBV11/mpconfigboard.mk unnecessary.
   A second part of flash could be placed above the TEXT. This might allow for more flexibility, given that the binary depending on compilation settings may vary in size.

2. Going further: Is there a way to automatically define a 128K-aligned flash start constant, depending on the size of the binary?
   Or is there a way for the MP code to determine the binary size so it can be available in flashbdev.c? Then the routines in flashbdev.c could be made to automatically use the maximum remaining flash space.
   No need to adjust anything in stm32f405.ld.
   The maximal flash space would be available after a new formatting of the flash filesystem.

3. Thinking about the problem that in the higher flash segments only one half of the 128K segment size is usable because the cache size of 64K limit: One might try to at least partially benefit from this waste in the following way:
   When the segment is deleted all 128K are empty. Now the cache could write alternately to the lower and higher half of that segment, keeping in mind which half is the recent one holding the data. This would save half of the flash deletions and thus reduce flash wear to a half too. Perhaps it would even make the flash access faster.
   Assuming this would be done with all the flash filesystem, a corresponding memory area definition might look like:

MEMORY
{
    FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 1024K /* entire flash */
    FLASH_ISR (rx)  : ORIGIN = 0x08000000, LENGTH = 16K   /* sector 0 */
    FLASH_TEXT (rx) : ORIGIN = 0x08004000, LENGTH = 368K  /* sectors 1,2,3: 16K, 4: 64K, 5,6: 128K each */
    FLASH_FS (rx)   : ORIGIN = 0x08060000, LENGTH = 640K  /* sector 7-11 are 128K (64K used, interchangedly) for filesystem */
    CCMRAM (xrw)    : ORIGIN = 0x10000000, LENGTH = 64K   /* used for filesystem cache */
    RAM (xrw)       : ORIGIN = 0x20000000, LENGTH = 128K
}

From the 5*128 = 640K FLASH_FS the half would be available.
A problem to be solved would be keeping the info which (higher or lower) part of the flash segments is the recent one through resets or restarts.

What do you think?