Skip to content

fix(role): keep a Colima VM's set memory within what its host can spare - #213

Merged
Mearman merged 1 commit into
mainfrom
fix/colima-memory-budget
Nov 4, 2026
Merged

Mearman merged 1 commit into
mainfrom
fix/colima-memory-budget

Conversation

@Mearman

@Mearman Mearman commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Refs #208

Follow-up to #212.

Budget as a safety limit. With github_runner_cluster_colima_memory_reserved_gib set, an explicit github_runner_cluster_colima_memory_gib must fit within host memory minus the reserve minus github_runner_cluster_colima_memory_host_reserve_gib. Above that the run fails before any colima call or cordon, and the message names the host's memory and both reserves. Without a reserve there is no budget check. A fleet that sets an explicit memory (the CI need) together with the reserve of a model that shares the host therefore can never end up with a VM the host cannot afford if the model grows.

Host reserve default 10 GiB, measured read-only on the two node hosts as macOS wired memory plus the resident memory of every process except the node VM: 3.68 + 6.28 = 9.96 GiB on the 64 GiB host and 2.52 + 7.85 = 10.4 GiB on the 32 GiB host. The compressor is excluded because it follows from memory pressure, not need. The previous default of 4 was measured on a machine that runs no node. The guest baseline's comment now records its measurement: free -m inside the node VM's k3s container showed 3396 MiB used.

Non-Colima hosts still apply nothing, and now print one line under -v naming the skipped values.

First run on the 64 GiB node host: its node VM was resized to 12 GiB by hand. With github_runner_cluster_colima_memory_gib: 12, the first role run there must report no change for the VM size.

Tests were written first and failed on main (7 failures), then passed. Each rule was then broken on purpose and the test guarding it failed:

Mutation Test that failed
explicit above the budget accepted test_an_explicit_memory_above_the_budget_is_refused_naming_the_figures, test_an_explicit_memory_above_the_budget_fails_before_anything_is_read_from_the_vm
a memory equal to the budget refused test_an_explicit_memory_within_the_budget_is_kept
budget checked with no reserve test_without_a_reserve_an_explicit_memory_has_no_budget
budget demands the runner limit test_the_budget_needs_neither_the_runner_limit_nor_the_baseline
host memory gathered only for a derivation test_the_host_memory_is_gathered_whenever_a_reserve_is_set
non-Colima line not printed test_a_host_whose_runtime_is_not_colima_is_left_alone
host reserve default back to 4 test_the_measured_reserves_are_the_defaults

With github_runner_cluster_colima_memory_reserved_gib set, a memory set
explicitly in github_runner_cluster_colima_memory_gib must now fit the
same budget a derivation uses: the host's memory less the reserve and
github_runner_cluster_colima_memory_host_reserve_gib. A memory above it
fails the run, naming the host's memory and both reserves, before the VM
is read or the node cordoned, so a workload whose reserve grows can never
leave the VM larger than the host affords. Without a reserve there is no
budget, and the check needs neither the runner limit nor the baseline.
The host's memory is gathered whenever a reserve is set.

The host reserve defaults to 10 GiB, measured on two node hosts as
macOS's wired memory plus the resident memory of every process but the
node VM's, the compressor left out; the earlier default was read on a
machine that runs no node. The guest baseline's comment records its
measurement inside a node VM.

A host whose runtime is not Colima still applies nothing, and now says
so under -v, naming the values it skips, so a size set for the wrong
host is visible.

Refs #208
@Mearman
Mearman marked this pull request as ready for review November 4, 2026 05:08
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ⚠️ Failed 2026-11-04T05:08:17.338022Z
We couldn’t complete this review. Comment @codex security review to try again.
d35e9a0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.

@Mearman
Mearman merged commit 03eff7e into main Nov 4, 2026
33 checks passed
@Mearman
Mearman deleted the fix/colima-memory-budget branch November 4, 2026 05:08
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 2.35.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant