Memory Exhausted Error

[Noah]

Hello,

I have an intermittent issue that occurs within a VisualGDB CMake project. I often (~80% of the time) get the following error during the linking phase:

c:/sysgcc/arm-eabi/bin/../lib/gcc/arm-none-eabi/12.3.1/../../../../arm-none-eabi/bin/ld.bfd.exe: final link failed: memory exhausted
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
-------------------------------------------------------------
Command exited with code 1
Executable: C:\PROGRA~2\Sysprogs\VISUAL~1/ninja.exe
Arguments:
Directory: D:\[...]\..\..\../build/VisualGDB/Debug
Command-line action failed

[See MemoryExhausted.png]

The GCC documentation says this error occurs when the linker runs out of memory, however I have 64GB of RAM in my system and it’s nowhere near full.
Additionally, if I build this via CMake on the command line (outside of VisualGDB), then this error goes away and it builds more quickly and links MUCH more quickly. I have to rebuild 4 or 5 times within VisualGDB in order to get the image to successfully link.

I have tried switching from the Sysprogs CMake to the “vanilla” CMake within the project settings, but that didn’t make a difference.

I am able to build without error using the “Developer Powershell” that is built into Visual Studio as well.

I am wondering if there is some memory limitation imposed on the VisualGDB plugin by Visual Studio which might be causing this issue?

I appreciate the help,

Noah

• This topic was modified 3 weeks ago by Noah.

Attachments:

You must be logged in to view attached files.

[Noah]

I should note that I am using VisualGDB Custom Edition, Version 6.0 (build 5147) with Visual Studio Professional 2022 (64-Bit) Version 17.8.5

[support]

Hi,

The linker itself is a 32-bit executable, so it has a 2GB memory limit rather than 64GB.

VisualGDB itself does not do anything special that would affect the memory usage by the linker. It merely launches Ninja, that in turn launches GCC/ld. If it works from developer power shell and doesn’t work with VisualGDB, you can troubleshoot it as follows:

1. Run both builds in verbose mode to obtain the exact linker command lines.
2. Try manually running both command lines from a separate command prompt.
3. If you can reproduce the behavior (one command line triggers the problem, while the other one doesn’t), try comparing the command lines and editing them to gradually remove the differences, until you narrow down a specific option that triggers the problem.

 

[bflannery]

You could try building the toolchain yourself with the large address aware flag.

I am curious how big your binary is when your build doesn’t fail. I ran into this issue when I had external memory sections set to load when they shouldn’t have been.

[support]

Hi,

We merely repackage the toolchain that comes from ARM. If you would like us to do a custom build for you, feel free to contact our sales for a quote.

[Noah]

It looks like VisualGDB has some special CMake stuff going on. I compared my build.ninja files from the VisualGDB build and the command line build. I found that the VisualGDB build somehow builds with the -g3 -O0 flags while my command line build does not. I tracked this down to a line in C:\Program Files (x86)\Sysprogs\VisualGDB\CMake\embedded\root.cmake that is shown in “AddedFlags.png” where these flags are unconditionally added if the underlying compiler is GCC. I’m not sure if this is the root issue, but it is definitely a problem. I am intentionally leaving my CMAKE_TOOLCHAIN_FILE unset because the ZephyrOS build system takes care of the toolchain file at a later stage. I see that in this case, VisualGDB detects this unset variable and gives CMAKE_TOOLCHAIN_FILE a value (see “vgdb_toolchain_file.png”).

I cannot find a setting to tell VisualGDB not to assign its own value to the CMAKE_TOOLCHAIN_FILE variable.

Thank you again for your time,

Noah

Attachments:

You must be logged in to view attached files.

[support]

Hi,

If this is an nRFConnect project, you can reset optimization options via the first page of VisualGDB Project Properties.

[Noah]

Thank you for the quick responses.

It’s not an nRF project, it’s an NXP i.MXRT running Zephyr. The VisualGDB project itself is a custom CMake project.

Please see “project_settings.png”

Attachments:

You must be logged in to view attached files.

[support]

Hi,

Then VisualGDB should not change any flags unless you have overridden it somewhere.

You can export the exact build/configuration commands used by VisualGDB as described here and compare them against the manual build commands.

[Noah]

Ok thank you, I followed the guide to export the build command line.

I agree that VisualGDB shouldn’t change any flags, however it really does seem like it is. This specific CMAKE_TOOLCHAIN_FILE issue has been a problem for me in the past and I have found ways to work around it or I would supply my own toolchain file in the project settings.

I exported the build command that VisualGDB is using and it is adding the following flags to the CMake configuration step (note that I redacted some full paths for privacy):

-DCMAKE_BUILD_TYPE=DEBUG -DTOOLCHAIN_ROOT=c:/sysgcc/arm-eabi -DCMAKE_MAKE_PROGRAM="C:/Program Files (x86)/Sysprogs/VisualGDB/ninja.exe" -DCMAKE_TOOLCHAIN_FILE=x:\[...]\..\..\../build/VisualGDB/Debug/toolchain.cmake -DPLATFORM=VisualGDB -DLOCALAPPDATA=C:/Users/[...]/AppData/Local

I have marked the options that I believe could cause problems. You can see from my “project_settings.png” image in the post above that I am not adding any of these options via the VisualGDB project settings, so I really don’t know where they are coming from.

 

Thanks,

Noah

[support]

Hi,

It really depends on how you imported the project and toolchain.

If you need help locating the relevant setting, please make sure you can reproduce the issue on a new project created from scratch. Then, please share the exact steps needed to reproduce it, along with the relevant screenshots, and we will try to point out the setting that is causing this.

[Noah]

Hello,

No problem, It is very possible this is user error.

I have reproduced this on an extremely simple new project created from scratch. The step by step screenshots are shared in the attached .pdf.

This project consists of 3 files:

• CMakeLists.txt
• toolchain.cmake
• main.cpp

Attachments:

You must be logged in to view attached files.

[support]

Hi,

Thanks for the repro steps. If you do not want VisualGDB to generate a toolchain.cmake file, please try using the “Use existing CMAKE_TOOLCHAIN_FILE” option on the import page of the wizard (step 5 in the PDF). You can also change it via VisualGDB Project Properties (Step 6, second setting from the bottom in the first group).

[Noah]

Thanks for taking a look. The problem is that if I check “use existing CMAKE_TOOLCHAIN_FILE” and put a null value in (empty string) then VisualGDB still generates its own toolchain.cmake file.

I’ll try adding an empty toolchain.cmake file to see if that works.

[support]

If you already have an existing toolchain.cmake file, we would advise simply pointing VisualGDB to it instead of creating a new empty one.

Edit: you can also try manually setting DisableToolchainFile in the .vgdbcmake file to false to completely disable the VisualGDB-level handling of toolchain files.

• This reply was modified 2 weeks, 6 days ago by support. Reason: Mentioned DisableToolchainFile

[Author]

Posts