Part 3 of 6 · 2 min read
Virtqueues
Think of a restaurant order spike. A waiter hangs order slips on a rail in the kitchen window, and rings the bell only if the cook is idle. If the cook is busy, the cook picks up new slips when finished, without any bell. The analogy stops in one place: a rail holds paper, and a virtqueue holds descriptors that point to buffers in guest memory.
In plain words: A virtqueue is a ring of descriptors in memory that the guest and the hypervisor both can read and write. The guest puts a request, a pointer to a buffer and its length, on the available ring and then kicks the host: one write to a notification register, one VM exit. The host reads all new requests, does the I/O, writes the answers on the used ring and interrupts the guest (a virtual interrupt). Either side can turn off the notification when it is already polling, so a busy device runs with few or no exits. The request data stay in guest memory and are not copied by the exit.
A worked example. An application does 100,000 small writes per second. Under emulation, each write needs 4 exits: 400,000 exits per second, and at a made-up 5 microseconds each that is 2 s of exit time per second. This is more than one core can give, so the guest cannot reach 100,000 writes per second at all. Under virtio with a batch of 32 requests per kick: 100,000 / 32 = 3,125 kicks per second, which is 3,125 x 5 us = 15.6 ms per second, 1.6 per cent of one core. Same work, 128 times fewer exits (400,000 / 3,125).
The key lines. Virtio gains speed by moving the cost from per-request traps to shared memory, which costs nothing per entry. The price is some delay, since a request may wait for the batch.
Virtqueue
A virtqueue is the data structure of virtio that carries requests between guest and host: a ring of descriptors in shared memory with an available part (guest to host) and a used part (host to guest), plus optional notifications in each direction.
Example: the guest adds a disk request to the available ring and kicks once for a whole batch.
Check yourself: 1. What does a "kick" do? 2. Why can a busy virtio device run with almost no exits? 3. What is the price of batching?