Skip to content

Add heartbeat seq tracking - #68

Merged
j0sh merged 4 commits into
mainfrom
ja/seq-cancel
Nov 10, 2026
Merged

j0sh merged 4 commits into
mainfrom
ja/seq-cancel

Conversation

@j0sh

@j0sh j0sh commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

Stale heartbeats can arrive after an app is removed, making the app available for use again.

Heartbeats now carry a strictly increasing seq number; invalid seq numbers are rejected and the heartbeat is not applied.

Stale heartbeats can arrive after an app is removed, making the app
available for use again.

Heartbeats now carry a strictly increasing seq number; invalid seq
numbers are rejected and the heartbeat is not applied.
Copilot AI balanced review requested due to automatic review settings November 10, 2026 05:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The stale-snapshot rejection path lacks regression coverage for sequence ordering, state preservation, app resurrection prevention, and invalid heartbeats not advancing the counter.

1 open issue
What changed in this PR

This PR adds heartbeat sequence tracking to prevent stale runner snapshots from making removed apps available again after sequence tracking is enabled.

Changes:

  • Track accepted sequences and reject non-increasing or missing values after opt-in.
  • Return the accepted sequence in successful responses and sequence-conflict errors.
  • Document heartbeat sequence behavior.
File Description
orchestrator/​registry.go Tracks and validates heartbeat sequences before applying state changes.
orchestrator/​http.go Includes the last accepted sequence in conflict responses.
docs/​http-api.md Documents sequence ordering, response echoes, and sequence resets.

Review effort: Balanced

Comment thread orchestrator/registry.go
Comment on lines +455 to +456
if current.Seq != nil && (req.Seq == nil || *req.Seq <= *current.Seq) {
return heartbeatResponse{}, http.StatusConflict, seqConflict(*current.Seq)
Copilot AI balanced review requested due to automatic review settings November 10, 2026 05:48
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 05:50 — with GitHub Actions Inactive
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 05:50 — with GitHub Actions Inactive
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 05:50 — with GitHub Actions Inactive

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The API documentation conflates local restarts with a new credential scope and must explain that runners reusing credentials need to persist the sequence counter or recover it from an HTTP 409 response.

2 open issues

Review effort: Balanced

Comment thread docs/http-api.md Resolved
Copilot AI balanced review requested due to automatic review settings November 10, 2026 06:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the opt-in heartbeat sequence tracking change.

1 open issue
1 resolved since last review

Review effort: Balanced

@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 06:23 — with GitHub Actions Inactive
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 06:24 — with GitHub Actions Inactive
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 06:24 — with GitHub Actions Inactive
@j0sh
j0sh temporarily deployed to docker-publish November 10, 2026 06:24 — with GitHub Actions Inactive
@j0sh
j0sh merged commit 17713df into main Nov 10, 2026
14 checks passed
@j0sh
j0sh deleted the ja/seq-cancel branch November 10, 2026 06:25

This branch was previously deployed

1 inactive deployment
docker-publish — b099ad60 Deployed Nov 10, 2026 by j0sh via docker-pr / promote #136
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants