Where to find ESP8266 xtensa-lx 106-elf for GDB stub?

[gojimmypi]

Greetings.

I am following the tutorial for Debugging ESP8266 firmware with the UART GDB Stub here:

http://visualgdb.com/tutorials/esp8266/gdbstub/

On step 3 there’s a pic indicating selecting the “ESP8266 in C:\SysGCC\xtensa-lx 106-elf” toolchain. I see only the “ESP8266 in C:\SysGCC\esp8266”

Thus, there are no options for me – as otherwise shown in the tutorial – for things like “Enable UART GDB Stub”

Per step 3, I downloaded the latest toolchain (esp8266-gcc5.2.0-r9.exe) from  http://gnutoolchains.com/esp8266/

(note that even here, the screen here says C:\SysGCC\esp8266 and not  C:\SysGCC\xtensa-lx 106-elf)

Is this feature still supported? Where can I find the xtensa-lx toolchain to do UART GDB stub debugging on the ESP8266?

Thanks.

 

[support]

Hi,

The link you mentioned is the correct one. Perhaps the download was corrupt? Please let us know which VisualGDB version you are using and attach a screenshot of the toolchain selection page so that we could help you understand what’s wrong.

[gojimmypi]

Hi.

Thanks for your reply. I have rebooted and still no difference.

I have been unable to upload a picture on the forum here. No errors, but no actual picture appears at end of process – so I tweeted a pic of the toolchains I see here:

https://twitter.com/gojimmypi/status/820986248969981953

I’m using Visual Studio 2015 Version 14.0.25431.01 Update 3 with VisualGDB 5.2

Upon trying to re-run the installation, Norton Anti-virus intervened, claimed it was a virus, and deleted my download file (not sure why it would not have done that the first time around – obviously I was able to download it, and I’m fairly sure it ran successfully without incident).

http://sysprogs.com/files/gnutoolchains/esp8266/esp8266-gcc5.2.0-r9.exe

I tried to download again, and this time Norton immediately deletes the file.

I did not have a problem with the similar process for the MSP430 a few days ago. I still have that file, and Norton Insight says that less than 5 users have used the file (sometimes Norton will think something is bad if there’s no user history). However this MSP430 file is still there, Norton is happy with it, and I was able to successfully install the toolchain to find my specific MSP430FR6989

It would not be the first time that Norton gives a false positive. But before I force the file onto my system, can you confirm the ESP8266 toolchain here is indeed virus free?

Thanks

[ket]

Hi,

Thanks, it looks indeed like your antivirus is interfering with the installation and deleting some files.

This is a known issue. Unfortunately modern antivirus software works by simply trying to locate known sequences inside exe files, so a large installer file with a compressed block of data will almost certain trigger a false positive. We submit those cases to antivirus vendors from time to time and get our files excluded, but it looks like this one is new.

Our best advice would be to disable your antivirus during installation.

[gojimmypi]

Upon disabling Norton, I was able to install esp8266-gcc5.2.0-r9.exe from

http://gnutoolchains.com/esp8266/

(I had previously also installed esp8266-gcc5.2.0-r8.exe )

I left the defaults, and it appears to have installed to C:\SysGCC\ESP8266 as shown here:

https://twitter.com/gojimmypi/status/820986248969981953

although there’s a C:\SysGCC\esp8266\xtensa-lx106-elf directory. (but not C:\SysGCC\xtensa-lx106-elf as shown in the tutorial)

I’m following the tutorial for OpenOCD on the ESP8266 here:

http://visualgdb.com/tutorials/esp8266/openocd/

And I have a Segger J-Link that I am trying to use, that seems mostly happy (see second pic with pins used):

https://twitter.com/gojimmypi/status/822541791027200002

However when attempting to debug, I see this in the OpenOCD Window:

Open On-Chip Debugger 0.9.0 (2015-11-04-20:38)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
trst_and_srst separate srst_gates_jtag trst_push_pull srst_open_drain connect_deassert_srst
adapter speed: 1000 kHz
stop_wdt
Info : J-Link V10 compiled Jan  9 2017 17:48:51
Info : J-Link caps 0xb9ff7bbf
Info : J-Link hw version 101000
Info : J-Link hw type J-Link
Info : J-Link max mem block 22664
Info : J-Link configuration
Info : USB-Address: 0x0
Info : Kickstart power on JTAG-pin 19: 0xffffffff
Info : Vref = 3.335 TCK = 1 TDI = 0 TDO = 0 TMS = 0 SRST = 1 TRST = 1
Info : J-Link JTAG Interface ready
Info : clock speed 1000 kHz
Info : TAP esp8266.cpu does not have IDCODE
Info : halted: PC: 0x40003b53
Info : debug cause: 0x20
Info : accepting ‘gdb’ connection on tcp/3333
Error: JTAG scan chain interrogation failed: all zeroes
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway…
Error: esp8266.cpu: IR capture error; saw 0x00 not 0x01
Warn : Bypassing JTAG setup events due to errors
Warn : xtensa_deassert_reset: ‘reset halt’ is not supported for Xtensa. Have halted some time after resetting (not the same thin
g!)
target state: halted
Info : halted: PC: 0x4000dd38
Info : debug cause: 0x20
Info : halted: PC: 0x4000dd3b
Info : debug cause: 0x1
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Interrupt suppression during single-stepping is now enabled
Watchdog feeding during stops is now enabled
Info : halted: PC: 0x40100004
Info : debug cause: 0x2

And this in this GDB Session Window:

Your VisualGDB trial expires in 23 days!
C:\SysGCC\esp8266\bin\xtensa-lx106-elf-gdb.exe –interpreter mi C:\workspace\Segger3\VisualGDB\Debug\Segger3
Warning: could not set a breakpoint on main. ‘Step into new instance’ will not work.
Loaded image in 17142 ms
Cannot resolve the address of _estack. Skipping stack pointer validity check.

And sure enough, I am unable to stop at breakpoints.

It appears the main problem is that I still am not using the xtensa-lx toolchain, eh?

Any suggestions?

Thanks.

[support]

Hi,

The “Error: JTAG scan chain interrogation failed” error usually indicates wiring problems. Please double-check your connections. If this does not help, please try using exactly the same hardware and layout as shown in one of our tutorials.

[gojimmypi]

thanks for your reply. What about the issue of  <span style=”font-family: Thread-0000079c-Id-00000082;”>C:\SysGCC\esp8266</span>  vs <span style=”font-family: Thread-0000079c-Id-00000082;”>C:\SysGCC\xtensa-lx 106-elf ? </span>How can I determine if I’m using the correct toolchain?  (I don’t have the xtensa-lx 106-elf  in the dropdown, even after manually installing the esp8266-gcc5.2.0-r9.exe toolchain download)

Unfortunately I don’t have an Olimex ESP8266-EVB board – however I’ve used the references for that ESP8266 to connect to my NodeMCU equivalent pins.  See:

https://twitter.com/gojimmypi/status/822571682544185344

btw – given that warning  “if you are using OpenOCD instead of xt-ocd, connect the reset signal to JTAG20 pin 3, not pin 15” – please note I am using pin 3.

thanks again for your help 🙂

[support]

Hi,

You are using the correct toolchain. xtensa-lx-106 is the CPU architecture for ESP8266 (like ARM would be for STM32). The ESP8266 toolchain can be installed either in esp8266 folder or the xtensa-lx-106-elf folder depending on which installer you use.

Regarding the wiring, from our experience with ESP8266 and ESP32 the chip has great features, but it’s extremely flimsy. Same setup may work on one board and randomly break on another one; sometimes a board works for some time and then randomly dies; sometimes doing a full erase helps, sometimes it does not.

It also looks like the Olimex boards are more reliable than the original ESP-XX modules and NodeMCU. So our advice would be to get it working on the Olimex one to rule out problems with JTAG cable, JTAG debugger and software bugs and then try the same setup on NodeMCU. We understand this is a big inconvenience, but unfortunately the ESP8266/ESP32 devices are still very fresh and not very reliable.

[gojimmypi]

Hello.

Well I ordered an Olimex ARM-USB-OCD-H to try instead of the Segger, and while at it I also took your advice and ordered an Olimex ESP8266-EVB. I’ve spent all day trying to get the JTAG debugging working, with no success. I am able to get GDB debugging working via the serial pins, but not JTAG.

VisualGDB insists on installing only its driver when pressing the “Test Settings” button. However I get this result:

C:\SysGCC\esp8266\esp8266-bsp\OpenOCD\bin\openocd.exe -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f target/esp8266.cfg
Open On-Chip Debugger 0.9.0 (2015-11-04-20:38)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
trst_and_srst separate srst_gates_jtag trst_push_pull srst_open_drain connect_deassert_srst
adapter speed: 1000 kHz
stop_wdt
Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED
Info : clock speed 1000 kHz
Info : TAP esp8266.cpu does not have IDCODE
Warn : Warning: Target not halted, breakpoint/watchpoint state may be unpredictable.
Info : accepting ‘telnet’ connection on tcp/4444
shutdown command invoked
Info : dropped ‘telnet’ connection

Although VisualGDB pops up a dialog box saying that “your settings appear to be valid”… well, given the error, no surprise that I am unable to program or debug via JTAG.

I’ve double, triple, quadruple checked my pin-by-pin connections:

http://visualgdb.com/tutorials/esp8266/

and following this tutorial:

http://visualgdb.com/tutorials/esp8266/openocd/

If I ignore the message and try to debug anyhow, I get this result:

Open On-Chip Debugger 0.9.0 (2015-11-04-20:38)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
trst_and_srst separate srst_gates_jtag trst_push_pull srst_open_dra
in connect_deassert_srst
adapter speed: 1000 kHz
stop_wdt
Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED
Info : clock speed 1000 kHz
Info : TAP esp8266.cpu does not have IDCODE
Warn : Warning: Target not halted, breakpoint/watchpoint state may
be unpredictable.
Info : accepting ‘gdb’ connection on tcp/3333
undefined debug reason 7 – target needs reset
Warn : target not halted
Info : TAP esp8266.cpu does not have IDCODE
Warn : xtensa_deassert_reset: ‘reset halt’ is not supported for Xte
nsa. Have halted some time after resetting (not the same thing!)
target state: halted
Info : halted: PC: 0x40244a26
Info : debug cause: 0x20
Error: xtensa_step: Timed out waiting for target to finish stepping
.
Warn : target esp8266.cpu is not halted
Info : halted: PC: 0x40244a05
Info : debug cause: 0x20
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Info : halted: PC: 0x4010013c
Info : debug cause: 0x8
Interrupt suppression during single-stepping is now enabled
Watchdog feeding during stops is now enabled
Info : halted: PC: 0x40100004
Info : debug cause: 0x2
Info : halted: PC: 0x4010017d
Info : debug cause: 0x10

It actually does stop on my breakpoint!  But it is not actually debugging C code, as I drop into the “Frame not in module” message. Sometimes thought, it seems that I can actually step though the assembly language. And although that might be interesting at other times – I’m trying to debug C code.

If I try to force any other drivers with zadig or UsbDriverTool – VisualGDB wants to force its own version when selecting a programmer.

I’ve burned through another week of my eval, and I’d really like this to work… any suggestions would be greatly appreciated.

Thanks

 

 

 

 

 

 

[support]

Hi,

First of all, please note that ESP8266 and ESP32 debugging is extremely unreliable and random errors are very common.

We will try to clarify what is going on based on your description. The LIBUSB_ERROR_NOT_SUPPORTED error can actually be safely ignored. OpenOCD shows it as it first tries to connect to the ‘Virtual COM port’ part of the Olimex JTAG probe. If you get your breakpoint to hit, the problem is most likely not related to drivers or JTAG connections, but is caused by one of the tool limitations. We will try to outline them below.

The ESP8266 chip has undocumented watchdog mechanisms that can break debugging if the chip is stopped at a breakpoint for too long. Our OpenOCD port disables the known watchdog, but sometimes the chip still goes to an undebuggable state.

The chip also has only one hardware breakpoint, so you cannot set more than 1 breakpoint in FLASH code. Furthermore, when you step through the code with F10 or F11, gdb internally uses an extra breakpoint, so in order for that to work reliably, you need to remove all other FLASH breakpoints. We usually recommend moving the functions you want to debug to RAM.

Another problem is that the ESP8266 chip sometimes enters some undocumented state where it does not respond to debug requests via JTAG. This happens when using the AT command firmware, but may happen with other functions as well.

Our general recommendation would be to understand what debugging functionality works (e.g. setting breakpoints only in RAM) and rely on it to debug.

[gojimmypi]

Hello & thanks for your reply. I’m not entirely convinced that all of the blame is on the ESP8266 (although ya, it is hard to argue with issues like “undocumented”, eh?)

I created a video on my ESP8266 debugging experience (based on the Olimex per your suggestion. I’ve not yet had a chance to target the NodeMCU) Here you’ll see what I consider “unexpected” operations when single stepping.

https://youtu.be/9Hid7ixEigM

You’ll see in the video, VisualGDB seems to do best when “running to breakpoint”, and not so much when single-stepping. This leads me to wonder if when single-stepping, VisualGDB could (in theory) programmatically place similar, pseudo-breakpoints?

If nothing else, perhaps you should tell users about that. So many times when I’d see the “Frame not in module” error – I thought: “game over, not working”.

And ya – I know there are more robust target platforms out there, however <$3 for a complete IoT device is pretty compelling. 🙂

<edit>fwiw: Here’s a detailed experience I wrote on the Olimex forum, when I thought it might by a problem with my new JTAG hardware: https://www.olimex.com/forum/index.php?topic=5676.msg23470#msg23470 </edit>

• This reply was modified 7 years, 2 months ago by gojimmypi. Reason: added link to more detailed description of issues left on Olimex forum :)

[support]

Hi,

Thanks for the video. This is totally expected. In order to do reliable stepping, gdb needs 2 mechanisms:

1. To be able to step one instruction at a time
2. Once it steps into a function, it needs to set a breakpoint on either the first code line of that function (for stepping into) or on the first line after the calling line (for stepping over).

On ESP8266 both mechanisms don’t work reliably:

1. Stepping instruction-by-instruction often triggers interrupts and gdb does not know how to step out of them
2. ESP8266 supports only 1 hardware breakpoint at a time, so if you already have one breakpoint in your code, gdb cannot set the temporary breakpoint as described above.

Additionally to that, gdb does not know how to reliably unwind the call stack on ESP8266, so if the target starts running some pre-built library code from Espressif, you see the ‘frame not in module’ message and don’t get a meaningful call stack.

These issues don’t happen with ARM devices because the ARM platform is very old and these problems were solved long ago. So for ARM devices, VisualGDB relies on the free tools like gdb to provide basic functionality (like stepping) and builds advanced features (like real-time watch) on top of them. For ESP8266/ESP32 there are no reliable low-level debugging tools and making them on our side would drive the VisualGDB price sky high (you would probably not want to pay $5K to debug a $2 chip), so unfortunately VisualGDB has to depend on the unreliable open-source tools that may be eventually fixed by the chip vendor. Hence <3$ for an IoT device is indeed compelling, however unfortunately you end up paying with spending several times more time to accomplish the same basic results.

We may eventually consider creating a list of the low-level tool issues and launching a crowdfunding campaign to fix them in the free tools, but it is hard to say how successful this would turn out.

We hope this explains it and sorry for the inconvenience caused by this.

[gojimmypi]

hm. interesting. thanks for your detailed response. 🙂

Do you think if I have an app that is not using interrupts, that I’d get more reliability if I explicitly disabled them?

But I suppose there are many more hidden, unstoppable system interrupts, eh? Although it is my understanding that system background stuff only runs during yield & delay functions… so I’m not sure I’d really call them “interrupts”.

[support]

Hi,

Based on our experience, most likely this won’t work. As we don’t have any insight in the closed-source parts of the ESP8266 libraries, unfortunately we cannot offer much help here. Please consider asking on the Espressif forums.

[Author]

Posts