Making VisualGDB work with ESP-IDF Master branch

[dedvalson]

Hi,

I have an ESP-IDF project that builds fine on Windows (MINGW32, standard ESP-IDF configuration) with the Release/3.1 branch of ESP-IDF. However I was interested in moving to the Master branch to utilize the new HTTP Server that is part of the 3.2 release.

When changing to the Master Branch I get the following errors:

The following Python requirements are not satisfied:
future>=0.16.0
cryptography
Please run "/usr/bin/python -m pip install --user -r /esp-idf/master/requirements.txt" for resolving the issue.
make: *** [/esp-idf/master/make/project.mk:454: check_python_dependencies] Error 1

I tried running the python command given and it failed as follows:

Don@Viking MINGW32 ~/esp/toolchain/esp-idf/master
$ python -m pip install --user -r requirements.txt
Requirement already satisfied: setuptools in c:/msys32/mingw32/lib/python2.7/site-packages (from -r requirements.txt (line 4)) (40.4.3)
Requirement already satisfied: pyserial>=3.0 in c:/msys32/mingw32/lib/python2.7/site-packages (from -r requirements.txt (line 8)) (3.4)
Requirement already satisfied: future>=0.16.0 in c:/msys32/mingw32/lib/python2.7/site-packages (from -r requirements.txt (line 9)) (0.16.0)
Collecting cryptography (from -r requirements.txt (line 10))
  Using cached https://files.pythonhosted.org/packages/22/21/233e38f74188db94e8451ef6385754a98f3cad9b59bedf3a8e8b14988be4/cryptography-2.3.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:/msys32/mingw32/lib/python2.7/site-packages/setuptools/__init__.py", line 191, in <module>
        monkey.patch_all()
      File "C:/msys32/mingw32/lib/python2.7/site-packages/setuptools/monkey.py", line 101, in patch_all
        patch_for_msvc_specialized_compiler()
      File "C:/msys32/mingw32/lib/python2.7/site-packages/setuptools/monkey.py", line 164, in patch_for_msvc_specialized_compiler
        patch_func(*msvc9('find_vcvarsall'))
      File "C:/msys32/mingw32/lib/python2.7/site-packages/setuptools/monkey.py", line 151, in patch_params
        mod = import_module(mod_name)
      File "C:/msys32/mingw32/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "C:/msys32/mingw32/lib/python2.7/distutils/msvc9compiler.py", line 306, in <module>
        raise DistutilsPlatformError("VC %0.1f is not supported by this module" % VERSION)
    distutils.errors.DistutilsPlatformError: VC 6.0 is not supported by this module

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:/users/don/appdata/local/temp/pip-install-qndipa/cryptography/

I have attempted numerous fixes to get around this with no success. Am I missing something obvious?

Don

[support]

Hi,

It looks like an incompatibility between a particular Python module and the MinGW32 environment used by the Espressif’s toolchain. You could try using our Cygwin-based toolchain that is slower than the original MSYS2-based one, but should have less compatibility problems between various components.

It could be also worthwhile to report this to Espressif, as it looks like a bug that would trigger for everyone trying to use the new ESP-IDF with their own toolchain.

[dedvalson]

Thanks for the help. It turns out this was answered just today over on the espressif forum. I should have posted this there in the first place.

https://www.esp32.com/viewtopic.php?f=13&t=7419

Thanks,
Don

[support]

Hi,

No worries. Given the complexity of the ESP32 build tools it’s often not clear what component is causing trouble, so it’s always worthwhile to check with us if you suspect any of our tools might be involved. Although we are not able to solve most problems that are outside our control, we can often help you narrow them down and suggest where to seek further help.

BTW, if you are using the Espressif’s msys2 environment, please consider using the VisualGDB build and a toolchain from this post. The new toolchain is a repackaged version of the original Espressif’s toolchain with the necessary XML definitions for VisualGDB to fully support all ESP32-related features out-of-the-box (you would still need to apply the fix from the Espressif’s forum, but you should be able to use VisualGDB’s GUI to manage ESP-IDF checkouts now).

[dedvalson]

Hi,

I have clearly gotten myself confused. I thought I was running the toolchain you provided, not the msys2 environment. The toolchain in my .vdbproj file is pointed to a toolchain I downloaded using visualgdb (5.2.0/7.10/r12). I ran the fixes suggested by Espressif in a minw32 environment, so apparently I was fixing their toolchain, not the one I downloaded. Which makes sense because I still got the failure on compile.

What environment should I have tried to run the python updates in?

Now I really got myself into a bind though because I installed the latest version of VisualGDB while I was at it (5.3R8) and now my visualgdb project won’t open anymore. When I try it returns the not very useful error message:

C:\Users\Don\source\CoralVue\Atlas\VisualGDB\Atlas.vgdbproj : error : There is an error in XML document (4, 4).

I have attached the vgdbproj file. Can you give me some hint what is wrong with it.

Thanks.

[support]

No problem, we can clarify.

It looks like you have installed the stable version of VisualGDB (5.3) that doesn’t support advanced ESP-IDF projects at all. Please install VisualGDB 5.4 Preview 5 instead.

With the toolchains, there are 2 options:

• You can use our Cygwin-based toolchain. It’s slower than the MSYS2-based toolchain, but provides a more consistent set of Linux-world tools (e.g. python).
• Alternatively you can either use the MSYS2-based toolchain and setup the toolchain XML files manually, or download our experimental R13 toolchain (see the link above) and the corresponding experimental VisualGDB build. This toolchain comes straight from Espressif (with some configuration files from us), is faster than our Cygwin environment, but is prone to common MSYS drawbacks (e.g. the Python issue you discovered). This toolchain only works with the experimental VisualGDB build linked above, as the regular Preview 5 doesn’t contain a few workarounds for other MSYS issues.

Hope this explains. Let us know if you need further help.

[dedvalson]

Hi,

Thanks for the tip on the Version 5.3 install, stupid mistake.

I still can’t compile anything against the master branch using the toolchain from visualgdb.

I wanted to make everything as clean as possible so I did the following:

1. Deleted all VisualGDB toolchains.
2. Downloaded a new version of the VisualGDB ESP32 toolchain (5.20/7.10/r12 using the Tools | VisualGDB | ManageGDBPackages menu entry in Visual Studio.
3. Created a new ESP32 IDF project based on the “Hello World sample” and using release/3.1 esp-idf version.
4. Built the project. It builds fine.
5. Within the “VisualGDB Properties” dialog I selected “Clone an ESP_IDF release from Github and downloaded the master branch.
6. Changed the project to use this version of ESP-IDF.
7. The project will now not load. The following errors are reported:

Loading cached code model from C:\Users\Don\source\repos\EmbeddedProject2\VisualGDBCache\EmbeddedProject2-Debug-VisualGDB\BuildCommandLines.txt…
Failed to parse code model from cached file: System.Exception: ESP-IDF build log did not report any built executables
at y2.c.f_2(IEnumerable1 a)
at a22.b_2()
Checking if any source files need uploading…
C:\SysGCC\esp32\bin\bash.exe –login -c “export IDF_PATH=/esp-idf/master && export BATCH_BUILD=1 && export -n PYTHONHOME && export LANG=en_US && cd /cygdrive/c/Users/Don/source/repos/EmbeddedProject2 && make partition_table BUILD_DIR_BASE=’/cygdrive/c/Users/Don/source/repos/EmbeddedProject2/VisualGDBCache/EmbeddedProject2-Debug-VisualGDB/CodeSenseDir’ SDKCONFIG=’/cygdrive/c/Users/Don/source/repos/EmbeddedProject2/VisualGDBCache/EmbeddedProject2-Debug-VisualGDB/sdkconfig’ V=0 && make -n -k BUILD_DIR_BASE=’/cygdrive/c/Users/Don/source/repos/EmbeddedProject2/VisualGDBCache/EmbeddedProject2-Debug-VisualGDB/CodeSenseDir’ SDKCONFIG=’/cygdrive/c/Users/Don/source/repos/EmbeddedProject2/VisualGDBCache/EmbeddedProject2-Debug-VisualGDB/sdkconfig’ V=0”
Saved the code model to C:\Users\Don\source\repos\EmbeddedProject2\VisualGDBCache\EmbeddedProject2-Debug-VisualGDB\BuildCommandLines.txt
System.Exception: ESP-IDF build log did not report any built executables
at y2.c.f_2(IEnumerable`1 a)
at a22.b_2()
at rr1.h()

8. I did a rebuild all but that failed also. The error reported was:

The following Python requirements are not satisfied:
cryptography
Please run “/usr/bin/python -m pip install –user -r /esp-idf/master/requirements.txt” for resolving the issue.
make: *** [/esp-idf/master/make/project.mk:454: check_python_dependencies] Error 1

9. Tried to select “Open Cygwin Terminal here” using the right click menu option in the Project Explorer. This fails with the error “Failed to run “bash” : no such file or directory”
10. Manually opened a cygwin terminal using the command “C:\sysgcc\esp32\bin\mintty.exe -i /Cygwin-Terminal.ico -”
11 Ran /usr/bin/python -m pip install –user -r /esp-idf/master/requirements.txt as suggested. This failed because it tried to build the package instead of just install (see https://www.esp32.com/viewtopic.php?f=13&t=7419)
12 Tried using the work around suggested by espressif by installing pac-cyg and running ./pac-cyg -S mingw-w64-i686-python2-cryptography this failed.

Tried several other methods to get the cryptography package installed into the cygwin environment, but failed in every case.
In any case, it appears that the current toolchain will not work with the master branch without some modification that currently escapes me.

Don

• This reply was modified 5 years, 7 months ago by dedvalson.
• This reply was modified 5 years, 7 months ago by dedvalson.

[support]

Hi,

Strange. Our toolchain is a repackaged copy of this archive with the following extra files/directories:

• toolchain.xml
• esp32-bsp
• esp-idf

None of those should be related to any Python issues, so most likely there is some other difference between the toolchain that worked and the current one. Could you please double-check which MSYS2 environment you used originally? If you cannot find it anymore, please try downloading it from scratch from the Espressif site, applying the fix and check if the new ESP-IDF works there. If not, there might be some other step that you performed with the previous checkout.

The “Open Cygwin Terminal here” command was renamed to “Open ESP-IDF Terminal here” in the build that fixed compatibility with the MSYS2 toolchains, so you are likely still using an old build (the officially released Preview 5 doesn’t include MSYS2 toolchain support as we haven’t tested it sufficiently yet). Please try this one: http://sysprogs.com/files/tmp/VisualGDB-5.4.5.2448.msi

[dedvalson]

I believe that the primary issue is that I am unable to carryout the work around steps described by espressif at https://www.esp32.com/viewtopic.php?f=13&t=7419 in a cygwin environment. The instruction to work around the problem is:

pacman -S mingw-w64-i686-python2-cryptography

This solves the problem in msys2. However, pacman does not exist in Cygwin. I have not found a suitable replacement that works. I believe that if I could run that command it would fix the problem. pac-syg does not do it, or at least I have not found the command line arguments that will make it work.

Don

[dedvalson]

I installed the newer version of VisualGDB as you referenced above. I was able to see no differences other than the drop down menu name change you mentioned above.

However, the command issued by that drop down menu still doesn’t work because it can’t find bash.

I was able to solve the compile problem however.

I created a new msys2 toolchain, ran the commands mentioned in https://www.esp32.com/viewtopic.php?f=13&t=7419 in the msys2 toolchain.
Then I copied the following directories:

cffi*
cryptography*
future*

from the /lib/python2.7/site-packages folder of the msys2 toolchain to the same folder in the cygwin based toolchain.

Don

[dedvalson]

I left out that after you copy those files you need to run:

/usr/bin/python -m pip install –user -r /esp-idf/master/requirements.txt

Don

[support]

Hi,

Thanks for sharing this and good to know it works. Looks like there might have been another confusion though.

Have you tried using our latest experimental toolchain (http://sysprogs.com/files/gnutoolchains/esp32/esp32-gcc5.2.0-r13.exe, not listed on the download page yet, linked in the post above), or did you download the R12 toolchain listed on the download page?

[Author]

Posts