This tutorial shows how to debug a Linux kernel module using Visual Studio and the internal VirtualBox debugger. This method of debugging is faster than using KGDB with a virtual COM port, however it has several limitations:
- Only 32-bit virtual machines are supported
- Hardware virtualization must be disabled for the virtual machine
Please also note that the VirtualBox debugger is currently extremely unreliable. In most cases it locks up after 5-10 minutes of debugging, so only use it for short debug sessions. Otherwise please use KGDB that is slower but much more stable.
Before you begin, please install Visual Studio, VirtualBox and the latest VisualKernel. Then power off your virtual machine, go to its settings, disable the “Enable VT-x/AMD-v” option and restart the VM:
Then follow the steps below:
- Launch Visual Studio. Begin creating a new project with VisualKernel kernel module wizard:
- On the first wizard page select “Create a new kernel module” and continue by clicking “Next”:
- On the next page select your VM in the “Targeted Linux computer” field:
- VisualKernel will download the kernel symbols and sources for your Linux kernel version. This may take a few minutes:
- Proceed with the default settings on the next wizard page:
- On the last wizard page select “Debug a Virtual Machine”, then select “VirtualBox” and specify the path to your .vbox file. Use the “find” button to browse the list of VMs for the current user:
- Press “Finish” to generate your project. Now you can build it using the “Build All” command in Visual Studio:
- If you try debugging your project now, VisualKernel will complain that the internal VirtualBox debugger has not been enabled yet:
- To enable it run <Program Files>\Sysprogs\VisualKernel\KernelTools\Common\VBoxDebugMgr.exe without UAC Elevation, select your VM and enable debugging for it:
- Go back to Visual Studio, set a breakpoint in the init function and start debugging. Your breakpoint will be triggered:Warning: if your debugging does not start, try restarting the virtual machine.
- You can now use the normal debugging techniques to debug your kernel module. Note that VirtualBox debugger is very unstable so your session can freeze at any time. Use KGDB over a virtual COM port for a more stable solution.