TinyEmbeddedTest problems in CMake project using FreeRTOS

[elivoe]

Hi,

 

we are using TinyEmbeddedTest to verify our systems performance and correctness. Our project structure looks like the following:

• Advanced CMake project on Infineon XMC4500 microcontroller
• Custom BSP for the XMC4500 (small adaptions and addons to your BSP)
• Using semihosting and profiler framework
• FreeRTOS as underlying real-time-operating-system
• TinyEmbeddedTest for unit test execution

 

The tests are started from a separate FreeRTOS task after successful power up procedure. This worked well under MSBuild projects a few month ago and I now ported it to advanced CMake project.

 

I now have the following problem: when I simply run the tests, all tests get executed, no matter how many I select. If I want to debug selected tests, it always starts from the first test, again no depending on which one I selected.

What we could observe:

In the GDB Session window, the message “No symbol “testCount” in current context.” occurs right after the function “SysprogsTestHook_SelectTests(testCount, (void **)pAllInstances);” gets called (from TinyEmbeddedTest.cpp, gets called in RunAllTests()).  Those instances that should not be tested are not set to zero like it should be done.

In the test output window it states that an unexpected test has been started, as the first one gets started instead of the ones I selected.

 

Of course I have tried to reconstruct the error on a new project but until know I cannot reconstruct it reliably. Therefore I cannot provide you a project to test it on your own. I will further work on this. But on a very simple new project it works, so the combination of cmake projects, TinyEmbeddedTest started from RTOS and my custom BSP don’t seem to be the problem.

 

It is a bit hard go get deeper into the rabbit hole without detailed documentation of the TinyEmbeddedTest framwork. Do you have any further ideas what the problem could be in my case? Or what I could try out to find the out what is going on?

 

Thank you very much.

 

Yours,

Elias

[support]

Hi,

No problem, we can help you understand what is going on.

When using the TinyEmbeddedTest framework, VisualGDB selects the tests to run as follows:

1. Sets a breakpoint in SysprogsTestHook_SelectTests()
2. Once it hits, evaluates testCount and pTests to get all available tests
3. Edits the array pointed by pTests (and updates testCount) to reflect the tests that were actually selected

These rules are stored in the <TestSelection> element in the TestFramework.xml file.

Most likely, something about your project structure (e.g. optimization) interferes with this. In order to get it working, we would advise trying to run the debugging normally and doing the test selection steps manually via the Watch window. If you run into issues (e.g. missing testCount variable), you may need to change the structure of your program to resolve them (e.g. declaring some variables static volatile if the compiler ends up optimizing them away).

Feel free to share the details about a workaround that worked and we will consider adding it on our side, unless it’s something specific to a particular project.

[elivoe]

Hi,

 

thank you for the background information. It helped me a lot understanding what is going on. I was able to fix the error, it really seems to be an optimization problem.

 

What helped for me was to make the testCount and the pTests variable global and volatile instead of passing it with the function. In this case GDB could find the variables.  Maybe GCC optimized the unused arguments although I use -O0 optimization?

 

 

I changed the code of RunAllTests() to the following:

 

volatile int testCount = 0;
volatile void **pTests = nullptr;

void RunAllTests()
{
for (TestGroup *pGroup = TestGroup::s_pFirstTestGroup; pGroup; pGroup = pGroup->m_pNextGroup)
for (TestInstance *pTest = pGroup->m_pFirstTest; pTest; pTest = pTest->m_pNextTestInGroup)
testCount++;

#ifdef __ICCARM__
TestInstance **pAllInstances = (TestInstance **)malloc(testCount * sizeof(TestInstance *)); //WARNING: this will cause a one-time memory leak
#else
TestInstance **pAllInstances = (TestInstance **)alloca(testCount * sizeof(TestInstance *));
#endif
int index = 0;

for (TestGroup *pGroup = TestGroup::s_pFirstTestGroup; pGroup; pGroup = pGroup->m_pNextGroup)
for (TestInstance *pTest = pGroup->m_pFirstTest; pTest; pTest = pTest->m_pNextTestInGroup)
pAllInstances[index++] = pTest;

#ifdef SIMULATION
TinyEmbeddedTest::InitializeSimulation(testCount, pAllInstances);
#endif

pTests = (volatile void **)pAllInstances;
SysprogsTestHook_SelectTests(testCount, (void **)pAllInstances);

TestGroup *pGroup = 0;

for (index = testCount - 1; index >= 0; index--)
{

…

 

 

Maybe you could change that in your code too?

 

Yours,

Elias

 

[support]

Hi,

No problem. Normally, GCC should keep the arguments of SysprogsTestHook_SelectTests() visible even with optimization turned on. Either way, could you please try checking if declaring the function arguments volatile instead of creating extra global variables works as well?

void __attribute__((noinline)) SysprogsTestHook_SelectTests(volatile int testCount, volatile void **pTests)
{
    asm("nop");
}

If not, please try adding the following line after asm(“nop”):

volatile int unused = testCount + (int)pTests;

Does this fix the evaluation of the function arguments?

• This reply was modified 2 years, 8 months ago by support.

[elivoe]

Hi,

 

I’ve tried both of your suggestions and both ended in the same problems as I described in my first post. The ‘testCount’ variable couldn’t be found by GDB and so all tests are executed.

 

Yours,

Elias

[support]

Hi,

No problem and thanks for checking this. We can add the changes on our side if we can confirm that this affects multiple devices and is not a side effect of some very rare condition (e.g. a bug in gcc/gdb).

If you could reproduce the issue on a simple project created from scratch, please feel free to attach it, and we will try to run a few experiments on it to understand whether it can affect other scenarios. If not, please consider simply patching the framework on your side. You can fork the framework internally (change its ID in the EFP.xml file and move it to a different subdirectory) if you would like to avoid having your changes overwritten by an update.

[elivoe]

Hi,

 

I have already tried to reproduce it on a clean project created from scratch before I started this topic. I wasn’t able to reproduce it in this small project, although it occurred in two different of our larger projects. I will put some further effort into this and hope to reproduce it in future to understand what exactly is going on here.

[Author]

Posts