Compiler warnings from Keil toolchain incorrectly formatted

[mmoore]

I encountered this issue after upgrading from VS 2015 to VS 2017 (VGDB 5.2).  Under 2015, I followed the tutorial at https://visualgdb.com/tutorials/arm/keil/ to configure a VS/VGDB project based on an existing uVision project.  In the tutorial, there is a section on compiler error formatting that indicates the need to download BuildMessageTemplates.xml and put it in the VGDB directory.  I never had to follow this step under VS 2015 and never had any error message formatting issues.  After upgrading to 2017, I started getting error message formatting problems.  I followed the steps in the tutorial without success.  I also attempted to modify the RegularExpressions.xml file in the VGDB directory.  I gather that the way formatting messages is handled has changed between the version the tutorial was written against and version 5.2.  Nothing that I did had any effect on the output.  The problem stems from the fact that VS 2017 adds a “n>” prefix to all build output log entries.  This is due to the parallel build support, where n is the build number.  This throws off the whatever RegEx VGDB is using, but it doesn’t appear to be using any that I modified.

Is there any RegEx that I can modify under version 5.2 to remedy this issue?

[support]

Hi,

Sorry, looks like we forgot to update the tutorial. Starting from VisualGDB 5.2, the build message templates have been moved to the <VisualGDB directory>\RegularExpressions.xml file along with other regular expressions used by VisualGDB so that you can tweak them more easily.

You can now also override the message templates for each project individually. Please see the updated Keil tutorial for details.

[mmoore]

Thanks for the update!  I downloaded the file and added the changes to the .vgdgsettings file.  It appears that VS now formats and displays compiler errors correctly, but it won’t build.  I get this error when attempting to build:

1>Inline build failed – ReadFile failed, win32 error code 109. Doing a normal build…

1>Build/launch failed: Object reference not set to an instance of an object.

The source of the error is the Microsoft.MakeFile.Targets file, in this section:

  <Target Name=”Build” DependsOnTargets=”PrepareForNMakeBuild;ResolveReferences;GetTargetPath;$(PreNMakeBuildTarget)” Returns=”$(NMakeManagedOutput)”>

<VCMessage Code=”MSB8005″ Type=”Warning” Arguments=”NMakeBuildCommandLine” Condition=”‘$(NMakeBuildCommandLine)’==””/>

<Exec Command=”$(NMakeBuildCommandLine)”  Condition=”‘$(NMakeBuildCommandLine)’!=””/>

</Target>

Specifically, the Exec Command line fails.  This occurs when this line exists in the .vgdbsettings file (added per the updated tutorial):

  <BuildMessageTemplateFile>BuildMessageRegexes.xml</BuildMessageTemplateFile>

I have tried moving this line within the .vgdbsettings file, but it has the same effect regardless of location within the file.

[mmoore]

It would seem I spoke to soon about the messages formatting correctly.  The message I saw that was correctly formatted was generated by VS, not the Keil toolchain.  I tinkered with the BuildMessageRegexes.xml file a bit.  If I remove the <GCCDetailedMessage> line, it will build successfully.  However, it still does not format the compiler messages correctly.  It does not seem to matter how the RegEx is written, it always formats the same.  It appears to be ignoring the RegEx defined in the BuildMessageRegexes.xml file.  I validated the modified RegEx against the build output in RegExBuddy and it matches.  What am I missing?

• This reply was modified 7 years, 1 month ago by mmoore.

[support]

Hi,

Thanks for reporting this, looks like the updated BuildMessageRegexes.xml file contained a small error. We have updated the version linked from our tutorial. Let us know if it works.

[mmoore]

Thanks for the update.  This has resolved the issue where the project won’t build with   <BuildMessageTemplateFile>BuildMessageRegexes.xml</BuildMessageTemplateFile> defined in the .vgdbsettings file.  However, I am still getting incorrectly formatted compiler errors and the changes I make to the RegEx in BuildMessageRegexes.xml appear to make no difference.  I have tested my RegEx against the build output in RegexBuddy and it matches as expected.  Here is the RegEx for compiler messages:

<GCCMessage>^1>”(.*)”, line ([0-9]+) : (error|warning) : (.*)$</GCCMessage>

I thought perhaps XML did not like the angle bracket, so I have also tried:

<GCCMessage>^1<![CDATA[>]]>”(.*)”, line ([0-9]+) : (error|warning) : (.*)$</GCCMessage>

and

<GCCMessage>”(.*)”, line ([0-9]+) : (error|warning) : (.*)$</GCCMessage>

as well as several other permutations.  I am attempting to match compiler messages in build output like this:

1>”../application/some_file.c”, line 424 : warning : #177-D: function “naughty_function”  was declared but never referenced

The error list is always formatted as “#177-D: function “naughty_function”  was declared but never referenced” with blank file and line 1.  Please let me know what I need to do differently to make this work.

[support]

Hi,

OK, we’ve tried reproducing this with the following extra line in Makefile:

ifeq ($(TARGETTYPE),APP)
$(BINARYDIR)/$(TARGETNAME): $(all_objs) $(EXTERNAL_LIBS)
    $(LD) -o $@ $(LDFLAGS) $(START_GROUP) $(all_objs) $(LIBRARY_LDFLAGS) $(END_GROUP)
    @echo "LEDBlink.cpp", line 10 : warning : #177-D: function "naughty_function"  was declared but never referenced
endif

The vgdbsettings file contains the following element:

  <BuildMessageTemplateFile>BuildMessageRegexes.xml</BuildMessageTemplateFile>

The BuildMessageRegexes.xml defines GCCMessage as follows:

  <GCCMessage>"(.*)", line ([0-9]+) : (error|warning) : (.*)$</GCCMessage>

This does actually produce the expected results (the message is converted to VS format in the Build Output window and VS recognizes it):

1>e:\projects\temp\EmbeddedProject19\LEDBlink.cpp(10): warning : #177-D: function "naughty_function"  was declared but never referenced

Please try checking your code against the examples shown here (note that this forum engine converts the ” character to a slightly different one and it won’t work if you simply copy-paste it). If it does not work, please try reducing the problem to a GCC project with an ‘echo’ line injected into the Makefile and send us the entire repro project so that we could help you troubleshoot this.

[mmoore]

Thanks for your response.  I have ensured that the .vgdbsettings and BuildMessageRegexes.xml files are configured as you indicated.  I modified the Makefile like so:

ifeq ($(TARGETTYPE),APP)
ROM_SECTION_NAME := ER_IROM1
$(BINARYDIR)/$(TARGETNAME): $(all_objs) $(EXTERNAL_LIBS)
$(LD) -o $(@:.elf=.axf) $(LDFLAGS) $(START_GROUP) $(all_objs) $(LIBRARY_LDFLAGS) $(END_GROUP)
@echo "../../LEDBlink.cpp", line 10 : warning : #177-D: function "naughty_function" was declared but never referenced
@echo "../application/source_file.c", line 71 : warning : #177-D: variable "_naughty_variable" was declared but never referenced
@echo "../application/source_file.c", line 71 : warn : #177-D: variable "_naughty_variable" was declared but never referenced
$(FROMELF) --bin --output $(@:.elf=.bin) $(@:.elf=.axf)
$(OBJCOPY) --remove-section $(ROM_SECTION_NAME) \
--add-section $(ROM_SECTION_NAME)=$(@:.elf=.bin) \
--set-section-flags \
$(ROM_SECTION_NAME)=CONTENTS,ALLOC,LOAD,READONLY,CODE \
--change-section-address $(ROM_SECTION_NAME)=0x00029000 \
$(@:.elf=.axf) $@
endif

I added three echo lines to the Makefile.  The first is per your listing above.  The second is a verbatim copy of the build output.  The third is a copy of the build output with ‘warn’ instead of ‘warning’ to test the RegEx match.  These all format in the build output as expected.  However, the compiler output is still not formatted.  Here is a relevant section of the build output:

1>C:/Keil_v5/ARM/ARMCC/bin/armcc.exe -g -O0 --c99 --gnu ... -o Debug/app_nvm_settings.o
1>"../application/source_file.c", line 71 : warning : #177-D: variable "_naughty_variable" was declared but never referenced
1> static some_type_t _naughty_variable;
1> ^
1>"../application/source_file.c", line 424 : warning : #177-D: function "naughty_function" was declared but never referenced
1> static void naughty_function(void)
1> ^
1>../application/source_file.c: 2 warnings, 0 errors
1>C:/Keil_v5/ARM/ARMCC/bin/armcc.exe -g -O0 --c99 --gnu ... -o Debug/source_file_2.o
1>C:/Keil_v5/ARM/ARMCC/bin/armcc.exe -g -O0 --c99 --gnu ... -o Debug/source_file.o
1>Program Size: Code=39920 RO-data=1780 RW-data=436 ZI-data=8420
1>C:/SysGCC/arm-eabi/bin/arm-eabi-objcopy.exe: Debug/target.axf: warning: Empty loadable segment detected, is this intentional ?
1>C:/SysGCC/arm-eabi/bin/arm-eabi-objcopy.exe: Debug/target.elf: warning: allocated section 'RW_IRAM1' not in segment
1>C:/SysGCC/arm-eabi/bin/arm-eabi-objcopy.exe: Debug/target.elf: warning: allocated section 'ER_IROM1' not in segment
1>C:/Keil_v5/ARM/ARMCC/bin/armcc.exe -g -O0 --c99 --gnu ... -o Debug/source_file_3.o
1>C:/Keil_v5/ARM/ARMCC/bin/armlink.exe -o Debug/target.axf ...
1>C:\development\the\path\to\the\project\visual_gdb\..\..\LEDBlink.cpp(10): warning : #177-D: function "naughty_function" was declared but never referenced
1>C:\development\the\path\to\the\project\visual_gdb\..\application\ source_file.c(71): warning : #177-D: variable "_naughty_variable" was declared but never referenced
1>"../application/source_file.c", line 71 : warn : #177-D: variable "_naughty_variable" was declared but never referenced

I also tried removing the end-of-line anchor from the RegEx, but the results were the same.  Have you tested this with your demo project from the Keil toolchain tutorial under VS 2017?  Is the Keil toolchain output somehow bypassing the RegEx filtering? Please let me know what I can do or if there is anything I can provide to get this resolved.  Thanks!

• This reply was modified 7 years, 1 month ago by mmoore.
• This reply was modified 7 years, 1 month ago by mmoore.

[support]

Hi,

The Keil toolchain output should not bypass the regex filters, but it could use some special unicode characters that could interfere with the regexes.

Please try running “make > build.log” from the command line and then double-check using a hex editor that the lines generated by Keil look exactly the same as the lines printed from the Makefile. You can also send the log file to our support email so that we could check it for you (please attach it instead of pasting to avoid automatic character transformation).

We have not tested it with Keil as we don’t officially support it out-of-the-box yet, sorry.

[mmoore]

I finally got it figured out. Apparently the output was already being re-written by the time that it showed up incorrectly formatted in the build output. So the line that I have been trying to match (per the previous post):

"../application/source_file.c", line 71 : warning : #177-D: variable "_naughty_variable" was declared but never referenced

actually comes out of the compiler looking like this:

"../application/source_file.c", line 71: Warning: #177-D: variable "_naughty_variable" was declared but never referenced

So apparently the capitalization of warning/error was different, as well as the spacing before the colon. I modified the RegEx and the compiler messages are now formatted correctly. Thanks for all your help.

• This reply was modified 7 years, 1 month ago by mmoore.

[chadgra]

I have VisualGDB version 5.3R8 and am running into these same issues. The BuildMessageRegexes.xml I grabbed from the Keil tutorial gives this warning:

Warning: cannot perform inline build: Object reference not set to an instance of an object

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3073

and then never builds.

If I remove the <GCCDetailedMessage> tag from BuildMessageRegexes.xml then it will build, but it won’t display Keil compiler warnings/error correctly. I’ve tried changing the <GCCMessage> tag to some of the things suggested above but none of them change the output.

I’m using VisualStudio 2017 Community edition.

Can anyone point me to a BuildMessagesRegexes that will work for this configuration?

[support]

Hi,

Most likely some parts of the BuildMessageRegexes.xml are invalid and this is causing an internal error in VisualGDB. Please feel free to attach your BuildMessageRegexes.xml file here and we will investigate it and improve the error message.

[mmoore]

Hi @chadgra,

I updated VisualGDB and encountered the same issue but was able to get it resolved. If a future update breaks the compiler message parsing again, here’s how I resolved it:

1. Open the C:\Program Files (x86)\Sysprogs\VisualGDB\RegularExpressions.xml file
2. Copy the contents of the BuildMessages element
3. Paste to replace the contents of the BuildMessageRegexes element of your BuildMessageTemplates.xml file
4. Replace the GCCMessage contents with correct regex (for Keil ArmCC it would be ^"(.*)", line ([0-9]+): (Error|Warning): (.*)$)
5. Save the BuildMessageTemplates.xml file and build – compiler messages should now be working.

Edit:
The XML file attachment failed for security. I’m posting the contents, but formatting should be checked as WordPress may modify the text.

<?xml version="1.0" encoding="UTF-8"?>
<BuildMessageRegexes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <GCCMessage>^"(.*)", line ([0-9]+): (Error|Warning): (.*)$</GCCMessage>
    <GCCInclusionMessage>^(In file included |[ \t]+)from ([^:]+|[a-zA-Z]:[^:]+|):([0-9]+)(:[0-9]+|)[,:]$</GCCInclusionMessage>
    <UndefinedReferenceMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (undefined reference to .*)$</UndefinedReferenceMessage>
    <UndefinedReferenceMessage2>^(.*):\(.*\): (undefined reference to .*)$</UndefinedReferenceMessage2>
    <UndefinedReferenceMessage3>^(.*): (undefined reference to .*)$</UndefinedReferenceMessage3>
    <MultipleDefinitionMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (multiple definition of .*)$</MultipleDefinitionMessage>
    <GCCDetailedMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (error|fatal error|warning|note):(.*)$</GCCDetailedMessage>
    <GCCOtherMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (.* required from.*)$</GCCOtherMessage>
    <RecursiveMakeMessage>make\[([0-9]+)\]: (Entering|Leaving) directory (.*)$</RecursiveMakeMessage>
    <MakeMessage>([^:]+):([0-9]+): \*\*\* (.*)$</MakeMessage>
    <CMakeMessage>CMake (Error) at (.*):([0-9]+)(| \(.*\)):$</CMakeMessage>
</BuildMessageRegexes>

• This reply was modified 6 years, 2 months ago by mmoore.

[support]

Hi,

Thanks for the update. This should indeed work, however the project-specific templates should work as well. If anyone can reproduce the problem with the broken per-project regexes and attach a repro, we should be able to investigate and fix it.

[mmoore]

Hi @support,

I did reproduce the situation, and the fix above is my solution. I updated from 5.2.15 to 5.3.18. The BuildMessageTemplates.xml (project-specific) from before is:

<?xml version="1.0" encoding="UTF-8"?>
<BuildMessageRegexes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <GCCMessage>"(.*)", line ([0-9]+): (Error|Warning): (.*)$</GCCMessage>
  <GCCInclusionMessage>^(In file included |[ \t]+)from ([^:]+|[a-zA-Z]:[^:]+|):([0-9]+)(:[0-9]+|)[,:]$</GCCInclusionMessage>
  <UndefinedReferenceMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (undefined reference to .*)$</UndefinedReferenceMessage>
  <UndefinedReferenceMessage2>^(.*):\(.*\): (undefined reference to .*)$</UndefinedReferenceMessage2>
  <MultipleDefinitionMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (multiple definition of .*)$</MultipleDefinitionMessage>
  <GCCDetailedMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (error|fatal error|warning|note):(.*)$</GCCDetailedMessage>
  <GCCOtherMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (.* required from.*)$</GCCOtherMessage>
  <RecursiveMakeMessage>make\[([0-9]+)\]: (Entering|Leaving) directory (.*)$</RecursiveMakeMessage>
  <MakeMessage>([^:]+):([0-9]+): \*\*\* (.*)$</MakeMessage>
</BuildMessageRegexes>

The fixed BuildMessageTemplates.xml is this:

<?xml version="1.0" encoding="UTF-8"?>
<BuildMessageRegexes xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <GCCMessage>^"(.*)", line ([0-9]+): (Error|Warning): (.*)$</GCCMessage>
    <GCCInclusionMessage>^(In file included |[ \t]+)from ([^:]+|[a-zA-Z]:[^:]+|):([0-9]+)(:[0-9]+|)[,:]$</GCCInclusionMessage>
    <UndefinedReferenceMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (undefined reference to .*)$</UndefinedReferenceMessage>
    <UndefinedReferenceMessage2>^(.*):\(.*\): (undefined reference to .*)$</UndefinedReferenceMessage2>
    <UndefinedReferenceMessage3>^(.*): (undefined reference to .*)$</UndefinedReferenceMessage3>
    <MultipleDefinitionMessage>^([^:]+|[a-zA-Z]:[^:]+):([0-9]+): (multiple definition of .*)$</MultipleDefinitionMessage>
    <GCCDetailedMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (error|fatal error|warning|note):(.*)$</GCCDetailedMessage>
    <GCCOtherMessage>^([^:]+|[a-zA-Z]:[^:]+|[a-zA-Z]:/[^:]+):([0-9]+):([0-9]+): (.* required from.*)$</GCCOtherMessage>
    <RecursiveMakeMessage>make\[([0-9]+)\]: (Entering|Leaving) directory (.*)$</RecursiveMakeMessage>
    <MakeMessage>([^:]+):([0-9]+): \*\*\* (.*)$</MakeMessage>
    <CMakeMessage>CMake (Error) at (.*):([0-9]+)(| \(.*\)):$</CMakeMessage>
</BuildMessageRegexes>

[Author]

Posts