{"id":12125,"date":"2017-08-22T16:06:59","date_gmt":"2017-08-22T15:06:59","guid":{"rendered":"https:\/\/sysprogs.com\/w\/?p=12125"},"modified":"2017-08-22T16:24:35","modified_gmt":"2017-08-22T15:24:35","slug":"introducing-embedded-integration-tests","status":"publish","type":"post","link":"https:\/\/sysprogs.com\/w\/introducing-embedded-integration-tests\/","title":{"rendered":"Introducing Embedded Integration Tests"},"content":{"rendered":"<p>VisualGDB\u00a0started supporting unit tests in version 5.2.\u00a0Having\u00a0out-of-the-box support for CppUTest, GoogleTest and TinyEmbeddedTest,\u00a0VisualGDB made it easy to check\u00a0whether every separable component of your system works. However\u00a0the size overhead and the necessity to create a separate test project makes it hard to use them for testing the\u00a0behavior of the entire\u00a0embedded system.<\/p>\n<p>So in VisualGDB 5.3 Preview 7 we introduced a new mechanism called Embedded Integration Tests that does exactly that &#8211; lets you conveniently verify the behavior of your\u00a0firmware without\u00a0any significant overhead.<\/p>\n<p><a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-large wp-image-12126\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1024x668.png\" alt=\"progress\" width=\"474\" height=\"309\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1024x668.png 1024w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress.png 1141w\" sizes=\"(max-width: 474px) 100vw, 474px\" \/><\/a><\/p>\n<p><!--more--><\/p>\n<h2>Overview<\/h2>\n<p>Embedded Integration Tests\u00a0let you modify the\u00a0flow of the program using the debugger functionality (via commands like &#8216;Set Next Statement&#8217; and &#8216;Set variable value&#8217;), record the program response and allow adding\u00a0extra checks (e.g. verifying that a variable value falls within a certain range). You can create Embedded Integration Tests inside any embedded project by simply selecting Test-&gt;Record Embedded Integration Test:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12127 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest.png\" alt=\"newtest\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/newtest-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>VisualGDB will automatically start a new debug session\u00a0and will stop in the <strong>main()<\/strong> function. Now you can use the normal debugging functionality to make your program follow a certain path and check its consistency along the way.\u00a0For example, you can:<\/p>\n<ul>\n<li>Ensure that the critical interrupt handlers get invoked and don&#8217;t get stuck.<\/li>\n<li>Set a breakpoint inside a critical function and ensure that the stack pointer\u00a0does not exceed the\u00a0expected limit.<\/li>\n<li>Use ARM Cortex performance counter to validate timing of the critical functions.<\/li>\n<li>Set a breakpoint in the function that checks\u00a0for user input, imitate various input events by forcing the function to exit with different status codes and check that the program responds as expected.<\/li>\n<li>Step through some critical functions, simulate common error conditions by manipulating hardware\u00a0registers, check that the errors are handled correctly.<\/li>\n<\/ul>\n<h2>Recorded Actions<\/h2>\n<p>VisualGDB will record (and replay when testing) the following actions:<\/p>\n<ul>\n<li>Setting, enabling\/disabling and deleting breakpoints<\/li>\n<li>Resuming execution\/stepping (and checking whether a specific breakpoint triggers)<\/li>\n<li>Running custom GDB commands (you can\u00a0check their output with regular expressions)<\/li>\n<li>Modifying\u00a0values of variables or hardware registers<\/li>\n<li>Setting next statement<\/li>\n<\/ul>\n<p>Each time you create a breakpoint or use &#8220;Set next statement&#8221;, VisualGDB will suggest tagging the corresponding source line by adding a specially formatted comment:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12128 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag.png\" alt=\"tag\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tag-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>This comment will help VisualGDB locate this line when running tests even if the code in the line changes:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12129 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone.png\" alt=\"tagdone\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/tagdone-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>VisualGDB will show the recorded actions in the &#8220;Integration Tests&#8221; window and will let you edit them:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12130 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set.png\" alt=\"set\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/set-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>If you have just checked a variable value by hovering your mouse over it, you can tell VisualGDB to always check for the same value by adding a check. VisualGDB will automatically suggest recently evaluated expressions:<\/p>\n<p><a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12131 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size.png\" alt=\"size\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/size-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a>You can also\u00a0check the\u00a0consistency of your program by running complex gdb commands and letting VisualGDB validate their output via regular expressions:<br \/>\n<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12132 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command.png\" alt=\"command\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/command-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<h2>Editing and Running Tests<\/h2>\n<p>Embedded Integration Tests are stored in test set files (.vgdbtest) that are added to the project via Solution Explorer. You can view and edit the tests by simply opening the test set file in Visual Studio:<\/p>\n<p><a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-12133 size-full\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest.png\" alt=\"edittest\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/edittest-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>Tests from all sets included in a project will automatically appear in the Test Explorer letting you run them as if they were the regular unit tests:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12134\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest.png\" alt=\"runtest\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/runtest-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<p>VisualGDB will conveniently display the progress of the currently running test\u00a0and give the control back to you if it stumbles upon any inconsistency:<a href=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-12135\" src=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1.png\" alt=\"progress\" width=\"1141\" height=\"744\" srcset=\"https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1.png 1141w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1-300x196.png 300w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1-768x501.png 768w, https:\/\/sysprogs.com\/w\/wp-content\/uploads\/2017\/08\/progress-1-1024x668.png 1024w\" sizes=\"(max-width: 1141px) 100vw, 1141px\" \/><\/a><\/p>\n<h3>Give It a Try<\/h3>\n<p>You can download VisualGDB 5.3 Preview 7 <a href=\"https:\/\/visualgdb.com\/download\/\">here<\/a>. As always, we appreciate your feedback, suggestions for more recordable\u00a0actions and\u00a0general ideas, so don&#8217;t hesitate to drop us a line via our <a href=\"https:\/\/sysprogs.com\/support\/\">support form<\/a> or the <a href=\"https:\/\/sysprogs.com\/w\/forums\/forum\/visualgdb\/\">VisualGDB forum<\/a>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>VisualGDB\u00a0started supporting unit tests in version 5.2.\u00a0Having\u00a0out-of-the-box support for CppUTest, GoogleTest and TinyEmbeddedTest,\u00a0VisualGDB made it easy to check\u00a0whether every separable component of your system works. However\u00a0the size overhead and the necessity to create a separate test project makes it hard to use them for testing the\u00a0behavior of the entire\u00a0embedded system. So in VisualGDB 5.3 Preview &hellip; <a href=\"https:\/\/sysprogs.com\/w\/introducing-embedded-integration-tests\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Introducing Embedded Integration Tests<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[1],"tags":[422],"_links":{"self":[{"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/posts\/12125"}],"collection":[{"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/comments?post=12125"}],"version-history":[{"count":3,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/posts\/12125\/revisions"}],"predecessor-version":[{"id":12141,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/posts\/12125\/revisions\/12141"}],"wp:attachment":[{"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/media?parent=12125"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/categories?post=12125"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sysprogs.com\/w\/wp-json\/wp\/v2\/tags?post=12125"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}