diff --git a/_posts/2016-12-02-xen-a-backend-frontend-driver-example.md b/_posts/2016-12-02-xen-a-backend-frontend-driver-example.md new file mode 100644 index 0000000..5ae791a --- /dev/null +++ b/_posts/2016-12-02-xen-a-backend-frontend-driver-example.md @@ -0,0 +1,209 @@ +--- +layout: post +title: "Xen - a backend/frontend driver example" +date: 02.12.2016 10:10 +--- + +Recently I began working on my master thesis. For this I have to get familiar with the [Xen hypervisor][xen] and its implementation of drivers. +As the documentation on its implementation is quite sparse I want to write down some of my findings, so others don't have to re-read and re-learn everything. +In this post I'll focus on how to get a minimal *driver* in a paravirtualized VM running. Following posts will then focus on how to do communication through event channels and shared memory +These are all things I need for the project I am working on, so I need to figure out how this works anyway. + +### Background + +The Xen hypervisor is only a minimal hypervisor implementation, which is booted and then boots a special Linux machine, the so-called **dom0**. +This **dom0** is most often just a regular Linux distribution such as Ubuntu. +Using Xen-specific tools it is then possible to launch additional virtual machines (VMs). These are called **domU**. +In the default case, **dom0** is responsible to acutally talk to the hardware attached to a machine, such as hard disks and the network card. +However, VMs of course also need some way to store data or generate network traffic. +In Xen this is handled by virtual devices attached to the **domU**. +Generic drivers then proxy data that should be written to disk or network packets to send out through the **dom0** to the actual device. + +These drivers follow a *split-driver* model, where one part of the driver, the backend, resides in the **dom0** and the other half, the frontend, +is a module in the **domU** machine. +Both parts can be implemented as kernel modules and be loaded dynamically. + +What's not documented as clearly as it should be: +Activation of the *virtual device* and thus invoking the right methods of the kernel module is done by writing data to the [XenStore][xenstore]. +For actual hardware this is already handled automatically. For your own custom *virtual device* this can be done manually. + +[xen]: https://www.xenproject.org/ +[xenstore]: https://wiki.xen.org/wiki/XenStore + +### A minimal driver + +Our driver won't do anything useful besides saying "Hello" and showing a message when it is activated. +The boilderplate for this example is quite huge, the full code can also be found in [the `xen-split-driver-example` repository][xen-split-driver-example]. + +I assume you already have a Xen host, you are connected to the **dom0** and have at least one **domU** running. + +The frontend driver resides in `mydevicefront.c`: + +~~~c +#include /* Needed by all modules */ +#include /* Needed for KERN_ALERT */ + +#include /* We are doing something with Xen */ +#include + +// The function is called on activation of the device +static int mydevicefront_probe(struct xenbus_device *dev, + const struct xenbus_device_id *id) +{ + printk(KERN_NOTICE "Probe called. We are good to go.\n"); + return 0; +} + +// This defines the name of the devices the driver reacts to +static const struct xenbus_device_id mydevicefront_ids[] = { + { "mydevice" }, + { "" } +}; + +// We set up the callback functions +static struct xenbus_driver mydevicefront_driver = { + .ids = mydevicefront_ids, + .probe = mydevicefront_probe, +}; + +// On loading this kernel module, we register as a frontend driver +static int __init mydevice_init(void) +{ + printk(KERN_NOTICE "Hello World!\n"); + + return xenbus_register_frontend(&mydevicefront_driver); +} +module_init(mydevice_init); + +// ...and on unload we unregister +static void __exit mydevice_exit(void) +{ + xenbus_unregister_driver(&mydevicefront_driver); + printk(KERN_ALERT "Goodbye world.\n"); +} +module_exit(mydevice_exit); + +MODULE_LICENSE("GPL"); +MODULE_ALIAS("xen:mydevice"); +~~~ + +The backend driver is very similar and resides in `mydeviceback.c`: + +~~~c +#include /* Needed by all modules */ +#include /* Needed for KERN_ALERT */ + +#include /* We are doing something with Xen */ +#include + +// The function is called on activation of the device +static int mydeviceback_probe(struct xenbus_device *dev, + const struct xenbus_device_id *id) +{ + printk(KERN_NOTICE "Probe called. We are good to go.\n"); + return 0; +} + +// This defines the name of the devices the driver reacts to +static const struct xenbus_device_id mydeviceback_ids[] = { + { "mydevice" }, + { "" } +}; + +// We set up the callback functions +static struct xenbus_driver mydeviceback_driver = { + .ids = mydeviceback_ids, + .probe = mydeviceback_probe, +}; + +// On loading this kernel module, we register as a frontend driver +static int __init mydeviceback_init(void) +{ + printk(KERN_NOTICE "Hello World!\n"); + + return xenbus_register_backend(&mydeviceback_driver); +} +module_init(mydeviceback_init); + +// ...and on unload we unregister +static void __exit mydeviceback_exit(void) +{ + xenbus_unregister_driver(&mydeviceback_driver); + printk(KERN_ALERT "Goodbye world.\n"); +} +module_exit(mydeviceback_exit); + +MODULE_LICENSE("GPL"); +MODULE_ALIAS("xen-backend:mydevice"); +~~~ + +To compile each module indivudally, put them in their own directory and add a `Makefile` per module: + +~~~make +obj-m += mydevicefront.o + +all: + make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules + +clean: + make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean +~~~ + +Change the first line to `obj-m += mydeviceback.o` for the backend driver. +You can then compile each module on their host and will get a `mydeviceback.ko` and `mydevicefront.ko`. + +Next, you need to load the modules. +In the **dom0**: + +~~~ +insmod mydeviceback.ko +~~~ + +In the **domU**: + +~~~ +insmod mydevicefront.ko +~~~ + +Check with `dmesg` that on both sides you get the "Hello World". + +Activation of the driver requires to add a virtual device to the Xenstore. I wrote a small script, `activate.sh` to do that. + +~~~bash +#!/bin/bash + +DOMU_ID=$1 + +if [ -z "$DOMU_ID" ]; then + echo "Usage: $0 [domU ID]]" + echo + echo "Connects the new device, with dom0 as backend, domU as frontend" + exit 1 +fi + +DEVICE=mydevice +DOMU_KEY=/local/domain/$DOMU_ID/device/$DEVICE/0 +DOM0_KEY=/local/domain/0/backend/$DEVICE/$DOMU_ID/0 + +# Tell the domU about the new device and its backend +xenstore-write $DOMU_KEY/backend-id 0 +xenstore-write $DOMU_KEY/backend "/local/domain/0/backend/$DEVICE/$DOMU_ID/0" + +# Tell the dom0 about the new device and its frontend +xenstore-write $DOM0_KEY/frontend-id $DOMU_ID +xenstore-write $DOM0_KEY/frontend "/local/domain/$DOMU_ID/device/$DEVICE/0" + +# Make sure the domU can read the dom0 data +xenstore-chmod $DOM0_KEY r + +# Activate the device, dom0 needs to be activated last +xenstore-write $DOMU_KEY/state 1 +xenstore-write $DOM0_KEY/state 1 +~~~ + +This adds 3 paths per domain, setting up the virtual device and thus activating the driver. +Once you executed that, you again check `dmesg`. You should now see the `Probe called` message. + +The full code can be found in [the example repository][xen-split-driver-example]. + +[xen-split-driver-example]: https://github.com/badboy/xen-split-driver-example