Skip to content

Automate Dependabot merging and Copilot triage - #4634

Open
cklin wants to merge 8 commits into
mainfrom
cklin/dependabot-automation
Open

cklin wants to merge 8 commits into
mainfrom
cklin/dependabot-automation

Conversation

@cklin

@cklin cklin commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds automation for open, non-draft Dependabot PRs that originate in this repository and target main. The automation discovers eligible PRs after configured CI workflows finish, on an hourly schedule, or through manual dispatch.

When CI passes, the automation approves and merges the PR. When CI fails, it retries the failed jobs once. If the rerun succeeds, the automation approves and merges the PR.

If the rerun fails, the automation invokes an isolated Copilot worker to investigate. Copilot either prepares a small migration or explains why a maintainer must intervene. The automation publishes the repair or explanation and requires a human to review and merge the PR, even if subsequent CI passes. It also retains the Copilot transcript and session events for seven days.

The first 5 commits form the main implementation. The remaining commits include additional documentation and code change to address security concerns.

Define the restricted repair result contract and collect structured PR,
patch, and failed-job evidence with explicit UTF-8 log truncation.

Add focused artifact tests and run the automation suite in CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/dependabot-automation/worker.cjs Dismissed
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/dependabot-automation/worker.cjs Dismissed
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/dependabot-automation/prompt.txt Dismissed
cklin and others added 7 commits October 30, 2026 13:36
Run Copilot with scoped local tools and curated documentation access.
Collect and validate actual checkout changes, while discarding edits
when the worker hands the problem to a maintainer. Export a Markdown
session transcript separately from the structured repair response.

Cover unattended execution, repair collection, and failure handling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Merge only pristine updates after configured CI passes. Retry eligible
failures once before a permanent Copilot handoff to human review.
Include path-filtered CLI workflow runs when present at the PR head.

Revalidate PR identity and head before mutations, publish restricted
repairs with an atomic head guard, and log merge refusals without
blocking later reconciliation. Cover lifecycle and race guards.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Connect discovery, retry, merge, isolated investigation, publication,
and failure reporting with scoped credentials and per-PR concurrency.
Serialize main-branch merge jobs with a shared pending queue.
Reconcile standard and path-filtered PR workflow completions.

Load trusted automation from main, transfer evidence and repair
artifacts across jobs, and document the lifecycle and trust boundaries.
Preserve session transcripts and event logs as diagnostic artifacts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Select up to 32 eligible PRs per run with a partial Fisher-Yates shuffle.

Skip held PRs unless native auto-merge needs disarming. Document
probabilistic fairness and cover filtering, empty sets, and sampling
without replacement.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Explain how trusted discovery selects PR heads independently of
workflow_run events, and document the restricted worker and separate
publisher boundaries.

Add foreign-repository eligibility and lifecycle regression tests,
including failed CI with an exhausted retry budget.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Save Git metadata outside the inference container and restore it only
after the container exits. Finalize repairs from the trusted checkout on
the runner instead of launching Git in the credential-bearing worker.

Keep the raw response separate from the final artifact and reject
nonregular response files before reading them. Preserve the existing
Git commands, publication checks, and human-review boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
Use the job token instead of an App token for publication, suppressing
push workflows and requiring approval for resulting PR workflows.

Tell maintainers to review repairs before approving CI. Retain the
permanent review hold and expected-head publication check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d63bcb34-3969-4d14-b422-33728176ec03
@cklin
cklin force-pushed the cklin/dependabot-automation branch from 4bde1b4 to a89166f Compare October 30, 2026 20:45
Comment thread .github/dependabot-automation/worker.cjs Dismissed
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/workflows/dependabot-pr.yml Dismissed
Comment thread .github/dependabot-automation/prompt.txt Dismissed
@cklin
cklin marked this pull request as ready for review October 30, 2026 21:16
Copilot AI auto review requested due to automatic review settings October 30, 2026 21:16
@cklin
cklin requested a review from a team as a code owner October 30, 2026 21:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical vulnerabilities allow dependency scripts to steal the GitHub token and expose runner-only files through symlinked diagnostic uploads.

5 open issues
What changed in this PR

Adds a Dependabot maintenance pipeline that merges pristine updates after CI passes and sends persistent failures to a containerized Copilot investigation. Once investigation begins, the PR requires human review and merging.

Changes:

  • Discover eligible PRs after CI completion, hourly, or manually; retry failed jobs once before investigation.
  • Collect failure evidence, prepare repairs or explanations, and publish validated results while retaining a review hold.
  • Add automation tests, lifecycle documentation, and seven-day retention for Copilot session artifacts.
File Description
.github/​workflows/​main.yml Runs the automation’s test suite in CI.
.github/​workflows/​dependabot-pr.yml Coordinates per-PR evaluation, retries, merging, investigation, and publication.
.github/​workflows/​dependabot-automation.yml Defines discovery triggers and dispatches eligible PRs.
.github/​dependabot-automation/​worker.cjs Invokes Copilot and collects checkout changes for validation.
.github/​dependabot-automation/​repair-protocol.cjs Validates repair outcomes, allowed paths, and size limits.
.github/​dependabot-automation/​README.md Documents lifecycle, trust boundaries, artifacts, and local checks.
.github/​dependabot-automation/​prompt.txt Defines investigation, migration, and escalation instructions.
.github/​dependabot-automation/​evidence.cjs Collects PR patches, failed-job logs, and an evidence index.
.github/​dependabot-automation/​controller.test.cjs Tests lifecycle decisions, provenance checks, and repair safeguards.
.github/​dependabot-automation/​controller.cjs Implements discovery, CI decisions, GitHub writes, and review holds.

Review effort: Max (auto)

docker run --rm --init --cpus=2 --memory=6g \
--user "$(id -u):$(id -g)" \
--cap-drop=ALL --security-opt=no-new-privileges \
-e GITHUB_TOKEN -e HOME=/tmp -e COPILOT_HOME=/diagnostics/copilot \
Comment on lines +212 to +213
diagnostics/transcript.md
diagnostics/copilot/session-state/**/events.jsonl
}

async function retry() {
const pr = await requireExpectedPr();
Comment on lines +70 to +73
if (!fs.existsSync(filename)) return { path: name, content: null };
if (!fs.lstatSync(filename).isFile()) {
throw new Error(`Non-regular repair file rejected: ${name}`);
}
Comment on lines +124 to +126
permission-contents: write
permission-pull-requests: read
permission-actions: read

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants