Repository navigation
xen/dom0: refactor dom0 memory growth to avoid splitting based on max - #63
Conversation
This reverts commit 0a2ac17. Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
This reverts commit 8788269. We no longer want to split the unpopulated boot-time space between the dom0 balloon and grant and foreign mappings. Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
This reverts commit 7498978. Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
Consolidate existing duplicated code. No functional change. Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
The boot-time unbacked address space reaches the unpopulated allocator (page_list) one 4 KiB page at a time, with a lock round trip and a list push for each page. On a large dom0 that is millions of round trips at boot. page_list also is unaware of page contiguity, limiting dom0 balloon growth to 4 KiB updates. Hand each extra memory region over whole, and keep its aligned 2 MiB blocks on a list per node, linked through the first page of each block as page_list links pages. The pages around the blocks, and a block that a node boundary cuts, go on page_list. Grants and foreign mappings still take pages from page_list only. When page_list of a node is empty, one block moves onto it, so taking a page stays a list pop. Pages given back go on page_list and do not form a block again.
With no pages parked on the balloon, dom0 grows by hotplugging a memory section at the first free physical address. That address space can belong to a PCI device that has not claimed it yet, and Xen does not stop dom0 from growing when dom0_mem sets no max:. dom0's balloon needs room to grow into, and grows faster when Xen populates a whole 2 MiB extent at a time. The unpopulated allocator keeps that room, the boot-time unbacked address space. Grant and foreign mappings and the balloon have to share it. Feed the dom0 balloon from the allocator's 2 MiB blocks when it is instructed to grow.
Add `info/room_kb` to the sysfs of the xen balloon driver, so user tools can discover the amount of memory dom0 can grow by. Reported are only complete 2MiB blocks in the unpopulated allocator.
max
| for (node = 0; node < MAX_NUMNODES; node++) | ||
| while (nr < nr_blocks && spare_blocks(node)) | ||
| heads[nr++] = pop_block(node); |
There was a problem hiding this comment.
@bleggett apropos #59 (comment)
AFAIK this fills all of NUMA node 0 before it touches node 1, so I don't think (could be wrong) that this plays particularly nicely with NUMA?
xen_alloc_unpopulated_blocks is not given any information from the caller to instruct it which node to grow from. We could allocate RR?
a8bc870 to
f3450aa
Compare
|
@bleggett apropos #59 (comment)
I haven't dug into this much, but on ARM what would be different is where the "available" memory would come from at boot. ARM uses a device tree to discover the memory, not an e820 map. However, the xen changes that use the logarithm to decide on the amount of dom0 memory still apply. Those changes were meant to make it less likely to need to adjust memory size at startup. |
|
Also, PV has no NUMA support, so it remains unaffected. |
|
back port is building here: https://github.com/edera-dev/linux/actions/runs/40818328735 |
Description
Part of COR-141
Replacement of proposal in #59 where we use separate page list arrays for small and super pages, rather than a range allocator.
Prior work in ENG-99 split unbacked boot-time memory using the
max:parameter indom0_memfrom Xen between the dom0 balloon driver and grant / foreign mappings. This makes it hard to remain flexible across varied deployments.Remove the implied function of
max:from splitting this memory, and add 2mb ranges to an additional list alongside the existing page list. Balloon and mappings share this unified memory range: the balloon consumes super pages found in the block list, and mappings take from the page list.Testing