Memory Exhausted Error

Sysprogs forums Forums VisualGDB Memory Exhausted Error

Viewing 15 posts - 1 through 15 (of 20 total)
  • Author
    Posts
  • #35481
    Noah
    Participant

    Hello,

    I have an intermittent issue that occurs within a VisualGDB CMake project. I often (~80% of the time) get the following error during the linking phase:

    c:/sysgcc/arm-eabi/bin/../lib/gcc/arm-none-eabi/12.3.1/../../../../arm-none-eabi/bin/ld.bfd.exe: final link failed: memory exhausted
    collect2.exe: error: ld returned 1 exit status
    ninja: build stopped: subcommand failed.
    -------------------------------------------------------------
    Command exited with code 1
    Executable: C:\PROGRA~2\Sysprogs\VISUAL~1/ninja.exe
    Arguments:
    Directory: D:\[...]\..\..\../build/VisualGDB/Debug
    Command-line action failed

    [See MemoryExhausted.png]

    The GCC documentation says this error occurs when the linker runs out of memory, however I have 64GB of RAM in my system and it’s nowhere near full.
    Additionally, if I build this via CMake on the command line (outside of VisualGDB), then this error goes away and it builds more quickly and links MUCH more quickly. I have to rebuild 4 or 5 times within VisualGDB in order to get the image to successfully link.

    I have tried switching from the Sysprogs CMake to the “vanilla” CMake within the project settings, but that didn’t make a difference.

    I am able to build without error using the “Developer Powershell” that is built into Visual Studio as well.

    I am wondering if there is some memory limitation imposed on the VisualGDB plugin by Visual Studio which might be causing this issue?

    I appreciate the help,

    Noah

    • This topic was modified 7 months ago by Noah.
    Attachments:
    You must be logged in to view attached files.
    #35483
    Noah
    Participant

    I should note that I am using VisualGDB Custom Edition, Version 6.0 (build 5147) with Visual Studio Professional 2022 (64-Bit) Version 17.8.5

    #35486
    support
    Keymaster

    Hi,

    The linker itself is a 32-bit executable, so it has a 2GB memory limit rather than 64GB.

    VisualGDB itself does not do anything special that would affect the memory usage by the linker. It merely launches Ninja, that in turn launches GCC/ld. If it works from developer power shell and doesn’t work with VisualGDB, you can troubleshoot it as follows:

    1. Run both builds in verbose mode to obtain the exact linker command lines.
    2. Try manually running both command lines from a separate command prompt.
    3. If you can reproduce the behavior (one command line triggers the problem, while the other one doesn’t), try comparing the command lines and editing them to gradually remove the differences, until you narrow down a specific option that triggers the problem.

     

    #35487
    bflannery
    Participant

    You could try building the toolchain yourself with the large address aware flag.

    I am curious how big your binary is when your build doesn’t fail. I ran into this issue when I had external memory sections set to load when they shouldn’t have been.

    #35488
    support
    Keymaster

    Hi,

    We merely repackage the toolchain that comes from ARM. If you would like us to do a custom build for you, feel free to contact our sales for a quote.

    #35493
    Noah
    Participant

    It looks like VisualGDB has some special CMake stuff going on. I compared my build.ninja files from the VisualGDB build and the command line build. I found that the VisualGDB build somehow builds with the -g3 -O0 flags while my command line build does not. I tracked this down to a line in C:\Program Files (x86)\Sysprogs\VisualGDB\CMake\embedded\root.cmake that is shown in “AddedFlags.png” where these flags are unconditionally added if the underlying compiler is GCC. I’m not sure if this is the root issue, but it is definitely a problem. I am intentionally leaving my CMAKE_TOOLCHAIN_FILE unset because the ZephyrOS build system takes care of the toolchain file at a later stage. I see that in this case, VisualGDB detects this unset variable and gives CMAKE_TOOLCHAIN_FILE a value (see “vgdb_toolchain_file.png”).

    I cannot find a setting to tell VisualGDB not to assign its own value to the CMAKE_TOOLCHAIN_FILE variable.

    Thank you again for your time,

    Noah

    Attachments:
    You must be logged in to view attached files.
    #35496
    support
    Keymaster

    Hi,

    If this is an nRFConnect project, you can reset optimization options via the first page of VisualGDB Project Properties.

    #35497
    Noah
    Participant

    Thank you for the quick responses.

    It’s not an nRF project, it’s an NXP i.MXRT running Zephyr. The VisualGDB project itself is a custom CMake project.

    Please see “project_settings.png”

    Attachments:
    You must be logged in to view attached files.
    #35499
    support
    Keymaster

    Hi,

    Then VisualGDB should not change any flags unless you have overridden it somewhere.

    You can export the exact build/configuration commands used by VisualGDB as described here and compare them against the manual build commands.

    #35500
    Noah
    Participant

    Ok thank you, I followed the guide to export the build command line.

    I agree that VisualGDB shouldn’t change any flags, however it really does seem like it is. This specific CMAKE_TOOLCHAIN_FILE issue has been a problem for me in the past and I have found ways to work around it or I would supply my own toolchain file in the project settings.

    I exported the build command that VisualGDB is using and it is adding the following flags to the CMake configuration step (note that I redacted some full paths for privacy):

    -DCMAKE_BUILD_TYPE=DEBUG -DTOOLCHAIN_ROOT=c:/sysgcc/arm-eabi -DCMAKE_MAKE_PROGRAM="C:/Program Files (x86)/Sysprogs/VisualGDB/ninja.exe" -DCMAKE_TOOLCHAIN_FILE=x:\[...]\..\..\../build/VisualGDB/Debug/toolchain.cmake -DPLATFORM=VisualGDB -DLOCALAPPDATA=C:/Users/[...]/AppData/Local

    I have marked the options that I believe could cause problems. You can see from my “project_settings.png” image in the post above that I am not adding any of these options via the VisualGDB project settings, so I really don’t know where they are coming from.

     

    Thanks,

    Noah

    #35501
    support
    Keymaster

    Hi,

    It really depends on how you imported the project and toolchain.

    If you need help locating the relevant setting, please make sure you can reproduce the issue on a new project created from scratch. Then, please share the exact steps needed to reproduce it, along with the relevant screenshots, and we will try to point out the setting that is causing this.

    #35502
    Noah
    Participant

    Hello,

    No problem, It is very possible this is user error.

    I have reproduced this on an extremely simple new project created from scratch. The step by step screenshots are shared in the attached .pdf.

    This project consists of 3 files:

    • CMakeLists.txt
    • toolchain.cmake
    • main.cpp
    Attachments:
    You must be logged in to view attached files.
    #35505
    support
    Keymaster

    Hi,

    Thanks for the repro steps. If you do not want VisualGDB to generate a toolchain.cmake file, please try using the “Use existing CMAKE_TOOLCHAIN_FILE” option on the import page of the wizard (step 5 in the PDF). You can also change it via VisualGDB Project Properties (Step 6, second setting from the bottom in the first group).

    #35506
    Noah
    Participant

    Thanks for taking a look. The problem is that if I check “use existing CMAKE_TOOLCHAIN_FILE” and put a null value in (empty string) then VisualGDB still generates its own toolchain.cmake file.

    I’ll try adding an empty toolchain.cmake file to see if that works.

    #35507
    support
    Keymaster

    If you already have an existing toolchain.cmake file, we would advise simply pointing VisualGDB to it instead of creating a new empty one.

    Edit: you can also try manually setting DisableToolchainFile in the .vgdbcmake file to false to completely disable the VisualGDB-level handling of toolchain files.

    • This reply was modified 6 months, 4 weeks ago by support. Reason: Mentioned DisableToolchainFile
Viewing 15 posts - 1 through 15 (of 20 total)
  • You must be logged in to reply to this topic.