Part 5 of 6 · 4 min read
Ballooning
Taking memory back: ballooning
Think of a guest at a hotel who is asked to give a room back because the hotel is full. The guest cannot be thrown out, so the hotel asks the guest to pack a bag and hand over the key. The analogy stops in one place: the guest knows which room is least used, and the hotel does not, so asking the guest is the better choice.
In plain words: The hypervisor may promise guests more memory in total than the host has, which is memory overcommit. When the host runs short, it has two ways to get memory back. It can swap guest pages to its own disk, which is slow and blind, because the host does not know which guest pages are useful. Or it can use ballooning: a driver inside the guest (the balloon driver) asks the guest kernel for pages, as if a program needed them. The guest kernel frees its least useful pages first, cache and old data (Level 7), which is the right choice. The balloon driver then tells the host that these guest pages are unused, and the host reuses the real memory behind them. To give memory back, the host asks the balloon to deflate, and the guest frees the pages.
A worked example (made up to show the idea). A host has 64 GiB and has promised 80 GiB to its guests, which is 80 / 64 = 1.25 times the real memory. Guests use less than they were promised, but one afternoon total use reaches 68 GiB. The host needs 4 GiB back. It inflates the balloon of a guest that has 10 GiB of cache by 4 GiB. That guest drops 4 GiB of cache, nothing it needs is lost, and the host's total use falls to 64 GiB. On the test machine the balloon is a real device:
$ cat /sys/bus/virtio/devices/virtio0/device
0x0005
$ basename $(readlink /sys/bus/virtio/devices/virtio0/driver)
virtio_balloon
$ grep virtio0 /proc/interrupts (lines named)
virtio0-config virtio0-inflate virtio0-deflate virtio0-stats virtio0-reporting_vq
Device type 5 is the balloon in the virtio specification, and the queues named inflate, deflate, stats and reporting show the messages that pass between guest and host. The stats queue lets the host see how much memory the guest really needs, and reporting lets the guest say which free pages the host may take.
The key lines. Ballooning cooperates with the guest, and host swapping does not. A host that must swap has the worse choice. Memory overcommit is safe only if guests rarely use all they were promised, and it needs monitoring.
Memory ballooning
Memory ballooning is a technique in which a driver in the guest allocates pages from the guest kernel on request of the hypervisor and tells the hypervisor that they are free, so that the hypervisor can reuse the real memory behind them for other guests.
Example: the virtio_balloon driver inflates by 4 GiB, and the guest drops 4 GiB of cache.
Common mistake
Treating free inside a guest as the memory of the machine. A guest sees only its own pages, and a balloon, or the host's overcommit, can make the real memory smaller than the guest thinks.
Check yourself: 1. What are the two ways that a host can reclaim memory from a guest? 2. Why is ballooning a better choice than host swapping? 3. What does an overcommit of 1.25 mean?
Words you will meet
| Word | Meaning |
|---|---|
| vCPU | The processor a guest sees, run as a host thread. |
| Overcommit | Promising guests more than the host has. |
| GVA / GPA / HPA | Guest virtual / guest physical / host physical address. |
| Shadow page table | A combined table kept by the hypervisor. |
| Nested paging | Hardware walks the guest and host tables. |
| Ballooning | A guest driver gives pages back to the host. |
Quick notes
- A vCPU is a host thread. Overcommit is safe while guests are mostly idle.
- Three addresses: GVA to GPA (guest table) to HPA (host table).
- Shadow tables: short walks, slow updates. Nested paging: fast updates, longer walks.
- Nested walk references = (g + 1) x (h + 1) - 1, so 24 for 4 by 4.
- Ballooning takes memory back with the guest's help. Host swapping is blind.
- The test machine has a
virtio_balloondevice (type 5).