Cannot run less than all tests in release build

[dabramson]

I have a project that build with Keil. There are unit tests using tinyembedded framework. When built for debug, the integration with VS test explorer seems to work fine; I can run all tests, or right click a group/single test and run them. But when built for release the only way to get tests to run is to run ALL of the tests. Right-clicking a group of/single test(s) and selecting run, or filtering test explorer to a group of tests and clicking “Run all tests in view” both result in the follow exception in the tests pane of the VS Output Window

========== Starting test run ==========
Running tests from C:\pxc_apl\APL\ADM\TestADM1\.visualgdb\VisualGDBCache\TestADM1-Release\TestADM1.vgdbtestcontainer...
Failed to select tests to run: Cannot query scheduled test count
Test container loaded (lookup: 2 msec, fetch: 1 msec, symbol list: 61 msec)

Started unexpected test: FlashAPIGroup::eraseSector_whenNumSectors1_Erases1Sector
System.Exception: Started unexpected test: FlashAPIGroup::eraseSector_whenNumSectors1_Erases1Sector
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.ReportTestResults(String testContainer, nf3 ctx, IBasicStream stream, Dictionary2 selectedTests, List1 tests, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.RunTestsInSource(String source, Dictionary`2 selectedTests, IRunContext runContext, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.DoRunTests(IEnumerable`1 sets, IRunContext runContext, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle)
An exception occurred while invoking executor 'executor://visualgdb/TestRunner': Started unexpected test: FlashAPIGroup::eraseSector_whenNumSectors1_Erases1Sector
Stack trace:
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.ReportTestResults(String testContainer, nf3 ctx, IBasicStream stream, Dictionary2 selectedTests, List1 tests, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.RunTestsInSource(String source, Dictionary`2 selectedTests, IRunContext runContext, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.DoRunTests(IEnumerable`1 sets, IRunContext runContext, IFrameworkHandle frameworkHandle)
at VisualGDBTestExtensionShared.VisualGDBTestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithTests.InvokeExecutor(LazyExtension2 executor, Tuple2 executorUri, RunContext runContext, IFrameworkHandle frameworkHandle)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.<>c__DisplayClass48_0.<RunTestInternalWithExecutors>b__0()
at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.<>c__DisplayClass0_0.<Run>b__0()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.Run(Action action, PlatformApartmentState apartmentState, Boolean waitForCompletion)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.TryToRunInSTAThread(Action action, Boolean waitForCompletion)
at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)

========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in 8.6 sec ==========

Any ideas?

Thanks,

Dave

 

• This topic was modified 2 years, 8 months ago by dabramson.

[support]

Hi,

In order to select the exact set of tests to run, VisualGDB sets a breakpoint in the SysprogsTestHook_SelectTests() function and reads/writes the variables pointing to the test that are about to run. Most likely, the optimizer is somehow interfering with this variable in your case.

We would advise enabling the GDB logging via VisualGDB Project Properties, checking the exact gdb commands sent by VisualGDB when the breakpoint in SysprogsTestHook_SelectTests() triggers and double-checking that they succeed (i.e. the actual memory gets modified). If not, you may need to disable optimization for the corresponding source file, or reorganize the code (e.g. promote some local variables to global ones to prevent the optimizer from removing them).

[Author]

Posts