VMWare GuestRPC mechanism

As you have probably noticed, a virtual machine is more than just a CPU emulator running as a process on a host machine. Modern virtual machines, such as VMWare also provide support for virtual various hardware as well as some means of interaction between host and guest machine. For example, VMWare provides clipboard synchronization, mouse synchronization and other similar functions. It is evident that such a mechanism can be adapted to exchange arbitrary information between guest and host.

Let's make a little digression and see, in which ways can such communication between guest and host be possible:

The shared memory approach should be the most fast, however it has a very significant disadvantage: synchronization. To ensure maximum performance, some thread should continuously check some status bit to determine whether new requests are issued from the other side. This will mean 100% CPU load for all the time, that is not very good. Moreover, Shared memory API was an experimental feature in VMWare Workstation 6.0.x and is deprecated since 6.5. The VMCI Sockets API that appeared in 6.5 is reported to be very slow and unreliable. Another problem is that VMCI API works in user mode, however, to implement a kernel debugging extension, the communication interface should work directly from kernel avoiding any library and OS calls.

The host-to-guest requests can be theoretically implemented in an interrupt-like way: a host calls a function forcing guest machine to execute some kind of an interrupt handler receiving request parameters and returning some data. However, no such interface is documented and no evidence of it actually existing was found. Moreover, it would not be very convenient to use such model, as Windows Kernel calls KD extension functions to send or receive a packet and does not specify any calbacks.

The guest-to-host requests can be most useful, as such an interface could be used for marshalling KdSendPacket()/KdReceivePacket() calls to make them being executed at host side. VMWare contains such an interface called VMWare backdoor. Let's see how it works in detail.

VMWare backdoor interface

When you install VMWare tools, some usability-related features are added to the VM. For example, mouse cursor synchronization. Let's see how it is implemented:

To get some information from host, VMWare tools package performs a so-called backdoor operation. A partial description of backdoor operation types can be found on this page. Another way to find information about VMWare backdoor interface is to look at open-vm-tools project source code. The following function declared in backdoor.h actually perform a backdoor operation:

void Backdoor_InOut(Backdoor_proto *myBp) // IN/OUT

The following definitions from backdoor_def.h show the possible backdoor request types:

#define BDOOR_MAGIC 0x564D5868
#define BDOOR_PORT 0x5658
#define BDOOR_CMD_GETMHZ 1
#define BDOOR_CMD_GETDISKGEO 3
#define BDOOR_CMD_GETPTRLOCATION 4
#define BDOOR_CMD_SETPTRLOCATION 5
//Continued in backdoor_def.h

Let's now make a simple program querying the current mouse cursor location using VMWare backdoor interface:

#include <stdio.h>
unsigned __declspec(naked) GetMousePos()
{
    __asm
    {
        mov eax, 564D5868h
        mov ecx, 4
        mov edx, 5658h
        in eax, dx
        ret
    }
}

void main()
{
    unsigned mousepos = GetMousePos();
    printf("Mouse cursor pos: x=%d,y=%d\n", mousepos >> 16, mousepos & 0xFFFF);
}
 

If this program is executed on a real machine, the in instruction will cause a "privileged instruction" exception, as user-mode code runs in Ring 3. However, when this program is executed on the virtual machine, it will print the correct mouse cursor position.

GuestRPC

One of the methods of executing guest-to-host requests, that was described above, is using Backdoor interface. However, patching VMWare code to support some additional backdoor commands could be tricky. For that purpose VMWare provides more flexible interface called GuestRPC. Basically, a single GuestRPC call consists of a sequence of requests:

  1. Open a GuestRPC channel
  2. Send command length
  3. Send command data
  4. Receive reply size
  5. Receive reply data
  6. Signalize end-of-receive
  7. Close channel

The following functions are used in open-vm-tools to perform GuestRPC calls:

RpcOut *RpcOut_Construct(void);
void RpcOut_Destruct(RpcOut *out);
Bool RpcOut_start(RpcOut *out);
Bool RpcOut_send(RpcOut *out, char const *request, size_t reqLen,
char const **reply, size_t *repLen);
Bool RpcOut_stop(RpcOut *out);

GuestRPC commands can be executed using the vmware-service.exe tool from the VMWare Tools package. For example, the following command prints the guest IP address:

vmware-service.exe -cmd "info-get guestinfo.ip"

KDVMWare provides a convenient object-oriented way for executing GuestRPC commands: VMWareRPCChannel and BufferedRPCChannel classes. Here is an example of a simple C++ program querying guest machine IP:

#include <stdio.h>
#include <assert.h>
#include <string.h>
#define ASSERT assert
#include "vmwrpc.h"

void main()
{
    VMWareRPCChannel chn;
    char sz[256];
    if (chn.ExecuteCommand("info-get guestinfo.ip", sz, sizeof(sz)) && (sz[0] == '1'))
        printf("Guest computer IP: %s\n", &sz[2]);
}
 

The vmwrpc.h file is a part of KDVMWare source code. As GuestRPC works both in ring0 and ring3, the VMWareRPCChannel class can be used both in kernel and user mode.

Managing handler table

By using VMWareRPCChannel class it is possible to execute arbitrary GuestRPC requests, that VMWare supports. However, the question of adding our own request types is still open. Let's examine the VMWARE-VMX.EXE internals. When a GuestRPC is being issued by guest, code inside the VMWARE-VMX.EXE searches the so-called GuestRPC handler table for a handler corresponding the the issued request. A GuestRPC handler entry format can be defined by the following structure:

typedef bool (*GRPCHANDLER)(void *pContext,
                            int ChannelNumber,
                            char *pCommandBody,
                            unsigned CommandBodyLength,
                            char **ppReply,
                            unsigned *pReplyLen);

struct RPCHandlerRecord
{
    char *pCommandPrefix;
    unsigned CommandPrefixLength;
    GRPCHANDLER pHandler;
    void *pHandlerContext;
    bool Enabled;
};

The code for fetching and executing a handler from a table can look like this:

RPCHandlerRecord g_HandlerTable[GUESTRPC_HANDLER_COUNT];
static const char gszUnknownCommand = "Unknown command";

void OnGuestRpcRequest(int ChannelNumber, char *pRequestBody, unsigned RequestSize)
{
    char *pReply = gszUnknownCommand;
    unsigned replyLen = sizeof(gszUnknownCommand) - 1;
    unsigned replyCode = 0;
    for (unsigned i = 0; i < GUESTRPC_HANDLER_COUNT; i++)
    {
        if (g_HandlerTable[i].Enabled &&
            !memcmp(pRequestBody, g_HandlerTable[i].pCommandPrefix, g_HandlerTable[i].CommandPrefixLength)
        {
            replyCode = g_HandlerTable[i].pHandler(g_HandlerTable[i].pHandlerContext,
                                                   ChannelNumber,
                                                   pRequestBody + g_HanderTable[i].CommandPrefixLength + 1,
                                                   RequestSize - (g_HanderTable[i].CommandPrefixLength + 1),
                                                   &pReply,
                                                   &pReplyLen);
        }
    }
    char sz[10];
    sprintf(sz, "%d ", replyCode);
    SendReplyPart(sz, strlen(sz));
    SendReplyPart(pReply, ReplyLen);
}

The GuestRPC table is contained in a data section of the VMWARE-VMX.EXE process, i.e. it is not allocated from heap. However, initially the table is filled with zeroes. All entries are being initialized on VMWare startup. By setting an unused entry from the table to our custom handler, it is possible to execute our code on a request from guest OS, passing some parameters to it and getting some data back. The last problem on the way to adding our own GuestRPC handler to VMWare is about finding the table in memory.

Finding the GuestRPC handler table

As the table is not exported from the EXE file, it should be manually located using some sort of an algorithm. The algorithm used in KDVMWare is relatively simple:

  1. Search all data sections of the EXE file for strings looking like GuestRPC command names. During the first attempt, strings starting with "tools." are found. If first attempt is unsuccessful, all strings between 10 and 100 characters containing at least one dot, are found.
  2. Search all data sections of the EXE file for 32-bit values equal to addresses of each found string (locate pointers to strings).
  3. Group found pointers using a simple rule: the distance between two pointers from a single group should be a multiple of RPCHandlerRecord structure size and should not exceed 1024 times sizeof(RPCHandlerRecord).
  4. Assume that each found group can be a GuestRPC handler table (stretching between handlers with the lowest and the highest addresses). For each group, verify each enabled entry between first and last one according to GuestRPC handler entry consistency check rules:
  5. Select a group with maximum number of elements and assume that it is the GuestRPC handler table.

In order to speed up table patching, KDVMWare maintains a database of VMWARE-VMX.EXE versions. For each version (identified by file size, time stamp from PE header, and first PE section size) the offset and size of GuestRPC handler table is stored. When KDCLIENT.DLL is loaded inside VMWARE-VMX.EXE, it tries to find a corresponding entry in the database, and in case of failure runs the detection algorithm.

Implementation

To see, how the GuestRPC mechanism is used by KDVMWare, check out this page.