{"id":396,"date":"2019-09-26T22:31:57","date_gmt":"2019-09-27T05:31:57","guid":{"rendered":"https:\/\/sysprogs.com\/tutorials\/?p=396"},"modified":"2019-09-26T22:31:57","modified_gmt":"2019-09-27T05:31:57","slug":"debugging-linux-kernel-modules-on-stm32mp1-devices","status":"publish","type":"post","link":"https:\/\/sysprogs.com\/VisualKernel\/tutorials\/stm32mp1\/","title":{"rendered":"Debugging Linux Kernel Modules on STM32MP1 Devices"},"content":{"rendered":"<p>This tutorial shows how to build and debug a basic kernel module on the STM32MP1 device running the OpenSTLinux distribution with Visual Studio and VisualKernel. Before you begin, follow <a href=\"https:\/\/visualgdb.com\/tutorials\/linux\/stm32mp1\/sdk\">this tutorial<\/a> to build an SDK and the SD card images for your STM32MP1 board using the OpenSTLinux build system. Once the build is complete, take a note of the following paths:<\/p>\n<ul>\n<li>The path where you have installed the SDK (by running the .sh file generated by the <strong>populate_sdk<\/strong> target) and the full path to its environment setup file (e.g. <strong>\/opt\/st\/stm32mp1\/2.6-snapshot\/environment-setup-cortexa7t2hf-neon-vfpv4-openstlinux_weston-linux-gnueabi<\/strong>).<\/li>\n<li>The full path of the Linux kernel built by the OpenSTLinux scripts. You can find it by searching for &#8216;vmlinux&#8217; inside the build subdirectory of your OpenSTLinux checkout. If you followed our <a href=\"https:\/\/visualgdb.com\/tutorials\/linux\/stm32mp1\/sdk\">SDK building tutorial<\/a>, it will be located in <strong>\/opt\/stlinux-weston\/build-openstlinuxweston-stm32mp1\/tmp-glibc\/work\/stm32mp1-openstlinux_weston-linux-gnueabi\/linux-stm32mp\/4.19-r0\/linux-stm32mp1-standard-build\/vmlinux<\/strong>.<\/li>\n<li>The full path of the gdb binary included in the OpenSTLinux SDK. You can find it out by applying the environment file from the SDK and then running the &#8220;which arm-openstlinux_weston-linux-gnueabi-gdb&#8221; command in the same terminal:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/cmd.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-408\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/cmd.png\" alt=\"\" width=\"1051\" height=\"411\" \/><\/a><\/li>\n<\/ul>\n<p>Then ensure you have VisualKernel 3.1 or later installed and follow the steps below to create, build and debug a simple kernel module project.<\/p>\n<ol>\n<li>Start Visual Studio and open the VisualKernel Project Wizard:<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/01-newprj.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-397\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/01-newprj.png\" alt=\"\" width=\"1024\" height=\"710\" \/><\/a><\/li>\n<li>Enter the name and the location of the kernel module that will be created by VisualKernel: <a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/02-prjname.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-398\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/02-prjname.png\" alt=\"\" width=\"1024\" height=\"710\" \/><\/a><\/li>\n<li>On the first page of the VisualKernel project wizard proceed with creating the default &#8220;Hello, World&#8221; example&#8221; <a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/03-hello.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-399\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/03-hello.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a><\/li>\n<li>On the next page specify the following settings:<br \/>\n<table border=\"1\">\n<tbody>\n<tr>\n<td>Setting<\/td>\n<td>Value<\/td>\n<td>Example<\/td>\n<\/tr>\n<tr>\n<td>Targeted Linux computer<\/td>\n<td>Your STM32MP1 board<\/td>\n<td>root@stm32mp1<\/td>\n<\/tr>\n<tr>\n<td>Build the module on<\/td>\n<td>The Linux machine containing the OpenSTLinux build<\/td>\n<td>testuser@stbuildvm<\/td>\n<\/tr>\n<tr>\n<td>Compatible GDB binary<\/td>\n<td>Select Manually =&gt; Full path to the GDB binary from the SDK<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/sysroots\/x86_64-openstlinux_weston_sdk-linux\/usr\/bin\/arm-openstlinux_weston-linux-gnueabi\/arm-openstlinux_weston-linux-gnueabi-gdb<\/td>\n<\/tr>\n<tr>\n<td>Kernel Build Environment<\/td>\n<td>Configure Manually<\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/04-gdbbin.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-400\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/04-gdbbin.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a><\/li>\n<li>On the next page you will need to enter the location of the kernel headers (that should be the kernel build directory since OpenSTLinux doesn&#8217;t have a separate header directory) and the cross-compiler parameters so that VisualKernel can configure the build tools. Enter the following settings:<br \/>\n<table border=\"1\">\n<tbody>\n<tr>\n<td>Setting<\/td>\n<td>Value<\/td>\n<td>Example<\/td>\n<\/tr>\n<tr>\n<td>Kernel Header Directory<\/td>\n<td>Kernel build directory inside the OpenSTLinux checkout<\/td>\n<td>\/opt\/stlinux-weston\/build-openstlinuxweston-stm32mp1\/tmp-glibc\/work\/stm32mp1-openstlinux_weston-linux-gnueabi\/linux-stm32mp\/4.19-r0\/linux-stm32mp1-standard-build<\/td>\n<\/tr>\n<tr>\n<td>Build the project using<\/td>\n<td>Custom cross-compiler<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Target architecture<\/td>\n<td>arm<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Cross-compiler prefix<\/td>\n<td>Full path to the gdb executable inside the OpenSTLinux SDK except the last 3 characters<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/sysroots\/x86_64-openstlinux_weston_sdk-linux\/usr\/bin\/arm-openstlinux_weston-linux-gnueabi\/arm-openstlinux_weston-linux-gnueabi-<\/td>\n<\/tr>\n<tr>\n<td>Environment setup file<\/td>\n<td>The environment setup file inside the SDK<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/environment-setup-cortexa7t2hf-neon-vfpv4-openstlinux_weston-linux-gnueabi<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/05-build.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-401\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/05-build.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a><\/li>\n<li>Now we will import the kernel debugging symbols for the built kernel into VisualKernel. Open the &#8220;Kernel symbols&#8221; dropdown and click &#8220;Manage cached Linux Kernel Symbols&#8221;:<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/06-getsym.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-403\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/06-getsym.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a>o<\/li>\n<li>If you have not configured any kernel symbols on the machine used to build OpenSTLinux yet, click &#8220;Configure&#8221; to choose a directory where VisualKernel will cache various kernel parameters:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/06-configure.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-402\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/06-configure.png\" alt=\"\" width=\"1175\" height=\"753\" \/><\/a><\/li>\n<li>We recommend proceeding with the default <strong>\/opt\/KernelCache<\/strong> directory:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/07-test.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-404\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/07-test.png\" alt=\"\" width=\"486\" height=\"262\" \/><\/a><\/li>\n<li>Click &#8220;Import symbols and sources for another kernel&#8221; to start importing the kernel symbols into VisualKernel: <a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/08-import.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-405\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/08-import.png\" alt=\"\" width=\"1175\" height=\"753\" \/><\/a><\/li>\n<li>On the first page of the kernel import wizard select &#8220;Link to an existing kernel build&#8221;:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/09-link.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-406\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/09-link.png\" alt=\"\" width=\"786\" height=\"593\" \/><\/a><\/li>\n<li>On the second page specify the following settings:<br \/>\n<table border=\"1\">\n<tbody>\n<tr>\n<td>Setting<\/td>\n<td>Value<\/td>\n<td>Example<\/td>\n<\/tr>\n<tr>\n<td>Kernel file with symbols<\/td>\n<td>Full path to the vmlinux file built by the OpenSTLinux scripts<\/td>\n<td>\/opt\/stlinux-weston\/build-openstlinuxweston-stm32mp1\/tmp-glibc\/work\/stm32mp1-openstlinux_weston-linux-gnueabi\/linux-stm32mp\/4.19-r0\/linux-stm32mp1-standard-build\/vmlinux<\/td>\n<\/tr>\n<tr>\n<td>Source directory<\/td>\n<td>The directory with the kernel sources (containing the KConfig file and .c files in subdirectories)<\/td>\n<td>\/opt\/stlinux-weston\/build-openstlinuxweston-stm32mp1\/tmp-glibc\/work\/stm32mp1-openstlinux_weston-linux-gnueabi\/linux-stm32mp\/4.19-r0\/linux-4.19.9<\/td>\n<\/tr>\n<tr>\n<td>Custom GDB binary<\/td>\n<td>GDB binary from the built SDK<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/sysroots\/x86_64-openstlinux_weston_sdk-linux\/usr\/bin\/arm-openstlinux_weston-linux-gnueabi\/arm-openstlinux_weston-linux-gnueabi-gdb<\/td>\n<\/tr>\n<tr>\n<td>Target architecture<\/td>\n<td>arm<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Cross-compiler prefix<\/td>\n<td>GDB binary path without the last 3 characters<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/sysroots\/x86_64-openstlinux_weston_sdk-linux\/usr\/bin\/arm-openstlinux_weston-linux-gnueabi\/arm-openstlinux_weston-linux-gnueabi-<\/td>\n<\/tr>\n<tr>\n<td>Environment setup file<\/td>\n<td>Environment setup file from the SDK<\/td>\n<td>\/opt\/st\/stm32mp1\/2.6-snapshot\/environment-setup-cortexa7t2hf-neon-vfpv4-openstlinux_weston-linux-gnueabi<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/10-settings.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-407\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/10-settings.png\" alt=\"\" width=\"786\" height=\"593\" \/><\/a><\/li>\n<li>VisualKernel will begin indexing the kernel sources and symbols. Once this process completes, the results will be saved in the kernel cache directory so that they can be easily reused for further projects:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/11-index.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-411\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/11-index.png\" alt=\"\" width=\"470\" height=\"284\" \/><\/a><\/li>\n<li>Verify that the newly imported kernel appears in the Linux Kernel Symbol Manager window and click &#8220;Close&#8221; to get back to the kernel module project wizard:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/12-indexed.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-412\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/12-indexed.png\" alt=\"\" width=\"840\" height=\"429\" \/><\/a><\/li>\n<li>Select the imported symbols in the &#8220;Kernel Symbols&#8221; field and ensure VisualKernel confirms that they match the currently running kernel:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/13-matched.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-413\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/13-matched.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a><\/li>\n<li>The next page of the wizard allows choosing between keeping the source files on Windows (and uploading them on each build) or keeping them on the machine with the OpenSTLinux SDK and accessing them via SSH. In this tutorial we will show the second option:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/14-store.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-414\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/14-store.png\" alt=\"\" width=\"886\" height=\"693\" \/><\/a><\/li>\n<li>Finally, the Debug Settings page of the wizard allows specifying the method that VisualKernel will use to debug the target. Most of the popular boards and devices are supported by the mainline OpenOCD and hence can be simply selected via the &#8220;Debug embedded board via JTAG&#8221; option, however as of July 2019, STM32MP1 boards require a special fork of OpenOCD that needs to be downloaded and launched explicitly. Get the latest STM32MP1 OpenOCD release from our <a href=\"https:\/\/github.com\/sysprogs\/openocd\/releases\/tag\/stm32mp1-20190709\">Github page<\/a>, unpack it and specify the following settings on the Debug Settings page:<br \/>\n<table border=\"1\">\n<tbody>\n<tr>\n<td>Setting<\/td>\n<td>Value<\/td>\n<td>Example<\/td>\n<\/tr>\n<tr>\n<td>Connect to<\/td>\n<td>Host\/Port<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Host<\/td>\n<td>[Host name or IP address of the Windows machine]:3334<\/td>\n<td>TVM:3334<\/td>\n<\/tr>\n<tr>\n<td>Command<\/td>\n<td>Full path to OpenOCD<\/td>\n<td>C:\\OpenOCD-STM32MP1\\bin\\openocd.exe<\/td>\n<\/tr>\n<tr>\n<td>Arguments<\/td>\n<td>-c &#8220;bindto 0.0.0.0&#8221; -f interface\/stlink-dap.cfg -f target\/stm32mp15x.cfg<\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/15-debug-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-422\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/15-debug-1.png\" alt=\"\" width=\"886\" height=\"694\" \/><\/a><\/p>\n<p>Note that OpenOCD will listen for GDB on 2 ports: 3333 for debugging the Cortex-M4 core and 3334 for debugging the Cortex-A7 core running Linux. Also ensure that your firewall allows connecting to OpenOCD running on the Windows machine from the Linux build machine and that the Linux machine can find the Windows machine using its host name (unless you are specifying an IP address instead).<\/li>\n<li>Press &#8220;Finish&#8221; to create the project. Once it is created, open VisualKernel Project Properties and go to the Debug Settings page. Then change 2 settings:\n<ol>\n<li>Because the default STM32MP1 kernel is built without tracing support, VisualKernel&#8217;s module that allows tracing loading\/unloading of other modules without parsing the full kernel symbols won&#8217;t work out-of-the-box. Hence select <strong>Obtain module information via: Parsing kernel structures<\/strong>.<\/li>\n<li>Reading the target memory via JTAG involves many separate low-level operations, hence capturing the <strong>printk()<\/strong> output would take considerable time (several seconds per line). Hence we recommend disabling it to avoid long wait if the kernel produces verbose debugging output via <strong>printk()<\/strong>.<\/li>\n<\/ol>\n<p><a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/16-dbgsettings.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-416\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/16-dbgsettings.png\" alt=\"\" width=\"1039\" height=\"693\" \/><\/a><\/li>\n<li>Finally, we recommend adding &#8220;monitor cortex_a smp_on&#8221; command via <strong>Additional GDB Commands =&gt; after selecting a target<\/strong>. It will show the multiple Cortex-A7 cores as separate threads in the Threads window:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/17-smp.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-417\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/17-smp.png\" alt=\"\" width=\"1039\" height=\"693\" \/><\/a><\/li>\n<li>Set a breakpoint in the init() function and press F5 to build the module and start debugging it. Note that due to the OpenSTLinux symbol layout, VisualKernel may report an incompatible kernel version when you start debugging. If this happens, simply press &#8220;Yes&#8221; to continue debugging:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/18-ignore.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-418\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/18-ignore.png\" alt=\"\" width=\"1239\" height=\"799\" \/><\/a><\/li>\n<li>VisualKernel will automatically load the kernel module and the breakpoint in the init() function will trigger:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/19-bkpt.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-419\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/19-bkpt.png\" alt=\"\" width=\"1239\" height=\"799\" \/><\/a><\/li>\n<li>You can step into the printk() function by pressing F11. VisualKernel will automatically locate the relevant sources:<br \/>\n<a href=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/20-stepped.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-420\" src=\"https:\/\/sysprogs.com\/tutorials\/wp-content\/uploads\/2019\/07\/20-stepped.png\" alt=\"\" width=\"1239\" height=\"799\" \/><\/a><\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial shows how to build and debug a basic kernel module on the STM32MP1 device running the OpenSTLinux distribution<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19],"tags":[38,52],"_links":{"self":[{"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/posts\/396"}],"collection":[{"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/comments?post=396"}],"version-history":[{"count":4,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/posts\/396\/revisions"}],"predecessor-version":[{"id":424,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/posts\/396\/revisions\/424"}],"wp:attachment":[{"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/media?parent=396"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/categories?post=396"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sysprogs.com\/tutorials\/wp-json\/wp\/v2\/tags?post=396"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}