KD extension DLLs & KDCOM protocol

WinDbg debugger allows you to debug all modern versions of Windows using a built-in kernel debugger and either COM or IEEE1394 port. Let's see how is it implemented. To start windows in Kernel Debugging mode, you specify additional parameters in boot.ini file that look like this:

Let's now see what actually happens when NTOSKRNL detects that it was started with /DEBUG parameter. First of all, it analyzes the /DEBUGPORT parameter from boot.ini and determines what packet-level plugin (KD extension DLL in Microsoft terminology) to load. For COM-based debugging the plugin DLL is called KDCOM.DLL, for IEEE1394-based debugging it is called KD1394.DLL. Fortunately, when you specify something like /DEBUGPORT=FOO, NTOSKRNL will try to load KDFOO.DLL and use it as a KD extension DLL. As we are providing our own DLL making a fast interface to WinDbg, that expects a named pipe from a virtual COM port, we need to solve two problems here:

• The problem of creating a valid KD extension DLL, i.e. providing the same set of exported functions working in an expected way.
• The problem of understanding and reimplementing KDCOM protocol, i.e. providing the same data at the end of our fast pipe, as the original KDCOM.DLL provides.

Note that in Windows Vista the kernel debugging flags are specified using bcdedit.exe utility and cannot specify a non-standard KD extension DLL. The only way to load KDVMWare DLL to kernel is to replace a standard one, for example, KD1394.DLL.

KD extension DLLs

Let's explore the structure of a typical KD extension DLL. A quick analysis of KDCOM.DLL shows that it exports the following functions:

Let's analyze how this functions work.

Initialization

As it is evident from their names, two functions are used to initialize a KD extension DLL: KdDebuggerInitialize0() and KdDebuggerInitialize1(). Fortunately, Microsoft provides PDB file for WinXP version of KDCOM.DLL. Additionally, some of the functions are described by Ken Johnson (http://www.nynaeve.net/?p=169). Let's use the PDB to recover declarations for the initialization functions:

The first function performs initial initialization of a KD extension DLL. For example, it can read the parameters specified in BOOT.INI using the LOADER_PARAMETER_BLOCK::LoadOptions field. For example, KDCOM.DLL can get determine the COM port number to use and its baud rate. Both initialization functions return a NTSTATUS value with STATUS_SUCCESS corresponding to successful completion. Note that if an initialization function such as KdDebuggerInitialize0() returns an unsuccessful status, kernel is started without debugging support and the DLL is not actually used any more.

Sending and receiving packets

All communication between kernel and a kernel debugger is packet-based. The following rules describe the packet behavior:

• All KD extension DLL calls are called synchronously, i.e. should not leave any executing code after they return.
• A KD extension DLL guarantees successful packet delivery, i.e. retries sending a packet when KdSendPacket() was called until debugger acknowledges it.
• Kernel uses polling model to check whether new packets are available. For that purpose it calls KdReceivePacket() with a special parameter.
• When kernel wants to receive a packet, it knows the type of the packet to receive. It passes this type to KdReceivePacket() that drops all packets of other types (and sends corresponding resend requests).
• Each send/receive operation uses 2 buffers for data transfer. Typically, the first buffer contains some fixed-size message header and the second one contains variable-sized message body. However, this is a typical use case, not the only one. Basically, in a send operation the two buffers are simply sent one after another as a single data block with no additional indication where the first one ends and the second one starts. When a receive function is called with first buffer having a size of N bytes, the first N bytes of the message are put to it, while the rest is put to the second buffer.

Each buffer is represented by a STRING structure defined in NTDEF.H (on x64 systems the data pointer is aligned at 8-byte boundary):

In KDVMWare this structure is redefined as KD_BUFFER.

A special structure called KD_CONTEXT maintains the global state for KD packet layer:

The RetryCount member is set before a call to KdSendPacket() and specifies the number of retries for a droppable packet to set. A droppable packet is a packet that can be simply dropped if no acknowledgment comes from WinDbg after some number of retries (KdSendPacket() will just return). The BreakInRequested is set to TRUE by KdReceivePacket() if WinDbg has requested a kernel breakpoint (ctrl+break was pressed, or WinDbg was just started). The break-in request is not a part of a packet and is transferred separately (see KDCOM protocol description below).

Here are the definitions for packet sending and receiving functions:

The KdReceivePacket() return value can be defined as a following enumeration:

The PacketType parameter specifies the type of the packet being sent or being received (all packets with other types should be ignored), however there is one exception. When PacketType is set to 8 in a KdReceivePacket() call, the function checks whether there is any data available (for example, whether the COM port buffer is non-empty), and returns immediately either KD_RECV_CODE_OK or KD_RECV_CODE_TIMEOUT.

Additional support functions

A KD extension DLL exports some additional functions that are not directly involved in packet sending/receiving and can simply return STATUS_SUCCESS in most of implementations:

The information about these functions was taken from RectOS documentation pages. Although Microsoft implementation can be different from ReactOS one, just returning STATUS_SUCCESS from these functions should work.

KDCOM protocol

Another problem to be solved in order to connect kernel and WinDbg using a custom KD extension DLL is the protocol that KDCOM.DLL uses to transfer packets over a COM port. As WinDbg receives and sends KDCOM packets when connected to a kernel using a named pipe, our tool should be able to produce and to parse such packets. In KDVMWare these packets  are processed in KDCLIENT.DLL on host side, however, in Microsoft implementation, all packet processing logic is implemented inside KDCOM.DLL. Let's see, how it works.

First of all, there are two kinds of packets: control packets and data packets. Data packets directly transfer KdSendPacket()/KdReceivePacket() data, while control packets signalize receive acknowledgment, retry requests, resync requests, etc. Each control packet consists of a packet header, a data block, and a terminating byte (0xAA). A data block contains contents of two buffers, one after another, with no indication of where one ends and another starts. Moreover, sender and receiver can use split the packet data in different ways:

Let's define a C structure describing the packet header:

Packet signature is either 0x30303030 ('0000') for data packets, or 0x69696969 for control packets ('iiii'). Packet type specifies the exact type of the packet. Types for control and data packets are members of the same enumeration:

As it was described before, packet type 8 is not used as a packet type. Instead, when KdReceivePacket() is called with that value, it checks whether any data can be received from WinDbg and returns immediately.

Packet ID is used to detect if a single packet was missed, as the least significant bit of a packet ID toggles with every new packet sent. The initial packet ID is 0x80800800, however, resync command sets it to 0x80800000. Checksum is just an arithmetic sum of all bytes from the data section of the packet.

As I have discovered after development of KDVMWare, a file named windbgkd.h was included in Windows 2000 DDK and contained information about KDCOM protocol internals. The ReactOS version containing most of the information from it can be found here. According to that file, the following packet types are actually used:

Let's discuss the types of control packets and their roles in KDCOM protocol:

• Acknowledgment packets are sent by both Kernel and WinDbg when a data packet was successfully received.
• Resend packet is sent when Kernel or WinDbg has received a damaged packet, a packet with wrong ID, or a packet with unexpected type.
• Resync packet is sent by WinDbg when it is initially connected to kernel. The kernel acknowledges resync operation by sending back another resync packet.

To illustrate, how KDCOM packet layer works, let's check out some examples:

1. Normal operation. Kernel continiously checks for new packets. When a packet is found, kernel receives it (assuming it knows the type for the packet).


2. Normal packet sending. Kernel sends a packet to WinDbg. KdSendPacket() waits for acknowledgment packet from WinDbg.


3. Packet sending with retry. Kernel sends a packet to WinDbg, however the latter does not receive it. KDCOM then resends the packet after timeout.


4. Packet collision. Kernel sends a packet to WinDbg when WinDbg sends a packet to kernel. Kernel sends a resend request to WinDbg. The latter gets the data packet instead of acknowledgment and buffers it. Then acknowledges it, receives a resend request for the first packet and sends it once again. KDCOM receives acknowledgement and returns control to kernel, that calls KdReceivePacket() to get packet from WinDbg (if expects one).
   
5. Resynchronization. Kernel receives a packet and encounters a resync request from WinDbg.
   

Droppable packets

Some packets are "droppable". It means that KdSendPacket() may return control when such a packet was not acknowledged by WinDbg after some number of retries. KDCOM.DLL treats the following packets as droppable:

• Type 3, subtype 0x3230
• Type 7, subtype 0x3031
• Type 11, subtype 0x3430

Packet subtype (ApiNumber) is the first DWORD in the packet data block. The types referenced here is defined in the following way in windbgkd.h:

Resync bounce problem

There is one significant detail in original KDCOM implementation. When the KDCOM.DLL initializes, it reinitializes the COM port and resets its buffer. A named pipe implementation should do the same. In other case, the following scenario is possible:

1. WinDbg connects to a named pipe with no kernel listening and sends a few resync packets.
2. Kernel loads our KD DLL, it receives first resync packet and replies with a resync.
3. WinDbg resynchronizes and sends some data.
4. Kernel receives another resync from buffer and resyncs again, replying with a resync.
5. WinDbg receives an unexpected resync, resynchronizes and sends another resync packet to acknowledge resynchronization.
6. The WinDbg/kernel couple will continue producing resync packets till the end of time and will never synchronize normally.

To avoid this problem, KDVMWare simply clears the named pipe receive buffer when it receives a resync packet.

Implementation in KDVMWare

All KDCOM-related functionality is implemented in the KdComDispatcher class. Feel free to explore its documentation using the link above.