[simonchrz]

Retention evicts every file that hasn't been served for the configured time, including the newest version of a package. On a small cache with a short retention, that means the proxy drops the one version it will most likely be asked for next.

storage.retention.keep_latest: N keeps the files of the N highest cached versions of each package from eviction by age:

storage:
  retention:
    default: "30d"
    keep_latest: 2

• Versions are ordered by the ecosystem's version rules (vers.CompareWithScheme), not by fetch time. Prereleases count like any other version.
• Only cached versions count, so no upstream request is needed.
• All files of a kept version stay (e.g. sdist and wheels).
• storage.max_size still evicts as before.
• Default is 0, which changes nothing.

It applies to cargo, golang, npm and pypi. An ecosystem opts in by setting CompareVersions in its retention.Spec; the others ignore the setting. On startup the proxy logs the covered ecosystems as keep_latest_covered. More ecosystems can follow one at a time. OCI needs a different approach, since its cache is keyed by digest; I'll send that as a separate PR.

The sweep runs one indexed query per package that has an expired file, cached for the sweep.

Summary by CodeRabbit

• New Features
  • Added storage.retention.keep_latest to retain a configurable number of the newest cached versions of each package from age-based eviction. Supported for Cargo, Go, npm, and PyPI; defaults to 0 (no versions protected).
  • Version ordering follows each ecosystem’s version rules, and prereleases count toward the limit.
• Documentation
  • Documented the setting, its supported ecosystems, and its interaction with the storage size limit.
• Validation
  • Negative keep_latest values are rejected.

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

The change adds storage.retention.keep_latest to protect the newest cached package versions from age-based eviction. It adds ecosystem-specific version ordering and cached-version lookup, then applies the selection during retention sweeps.

Changes: Keep Latest Retention

Layer / File(s)	Summary
Configuration and validation config.example.yaml, docs/configuration.md, internal/config/config.go, internal/config/retention.go, internal/config/retention_test.go	Configuration and documentation add keep_latest, with a default of 0. Validation rejects negative values.
Version ordering and cached-version lookup internal/retention/retention.go, internal/retention/keep_latest_test.go, internal/handler/{cargo,go,npm,pypi}.go, internal/handler/go_test.go, internal/database/retention.go, internal/database/retention_test.go	Retention specs add version comparators, including Go module version decoding. NewestVersions selects the highest distinct versions, and the database returns cached upstream version PURLs for a package.
Age-based retention sweep internal/server/retention.go, internal/server/retention_keep_latest_test.go, internal/server/retention_test.go	The sweep keeps selected cached versions when they would otherwise be removed by age-based retention. Tests cover selection, package and ecosystem scope, and cases without keep-latest support.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant RetentionSweep
  participant ClearIfExpired
  participant LatestKeeper
  participant Database
  participant NewestVersions
  RetentionSweep->>ClearIfExpired: check expired artifact
  ClearIfExpired->>LatestKeeper: check whether artifact is retained
  LatestKeeper->>Database: list cached version PURLs
  Database-->>LatestKeeper: cached version PURLs
  LatestKeeper->>NewestVersions: rank distinct versions
  NewestVersions-->>LatestKeeper: selected versions
  LatestKeeper-->>ClearIfExpired: retain or allow clearing

Loading

Possibly related PRs

• git-pkgs/proxy#645: Adds background refreshes that place newer artifacts in the cache, which can be affected by this retention behavior.

Suggested labels: enhancement

Suggested reviewers: pinguinfuss, andrew

Merge Risk: Minimal · up to b7316

This adds an opt-in setting, defaulting to 0, that protects the newest cached package versions from age-based eviction. Existing behavior is unchanged unless the setting is enabled, and no concrete merge-blocking risk was identified.

Security architecture risk: Low · up to b7316

This opt-in policy changes how long cached files remain without introducing a demonstrated access-control or privilege change. Existing conditional deletion safeguards remain. Large-cache behavior and deployment-specific exposure were not fully established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The demonstrated effect is longer residence of selected upstream artifacts in the configured cache. Package/version inputs can influence rank, but the examined path does not use that rank to grant access or privileges. Tenant, environment, and shared-database exposure cannot be determined from the available deployment evidence.

Trust Boundaries and Controls

• observed — Configuration supplies the exemption count, while static ecosystem registration supplies the comparator. Database ecosystem mapping and exact package/version PURLs constrain selection; a version label is not interpreted as a cross-package selector or executable callback.

Resilience and Maintainability Implications

• observed — Existing eviction safeguards remain: retention uses its advisory job lock, checks cancellation between candidates, and conditionally clears only the matching still-expired artifact and storage path. Clearing and pending-delete insertion occur in one transaction. Lookup failure retains rather than deletes uncertain candidates, and interruption leaves no persistent keeper state.

Hardening Proposals

• proposed — Consider a measured work budget or cancellation-aware lookup for unusually large cached version histories. The current full-history query and sort justify evaluating resource containment, but the available evidence does not establish an exploitable resource-exhaustion condition.

Pre-merge checks: 5

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly summarizes the main change: adding storage.retention.keep_latest to retain the newest versions.
Docstring Coverage		Docstring coverage is 85.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 85.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 20 functions across 15 files. (2 skipped: 2 unsupported.)

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: ea7a98d2-6340-4b5b-a7d9-8fe3edae5815

Commits

Reviewing files that changed from the base of the PR and between 12246c7 and b731648.

Files selected for processing: 17

• config.example.yaml
• docs/configuration.md
• internal/config/config.go
• internal/config/retention.go
• internal/config/retention_test.go
• internal/database/retention.go
• internal/database/retention_test.go
• internal/handler/cargo.go
• internal/handler/go.go
• internal/handler/go_test.go
• internal/handler/npm.go
• internal/handler/pypi.go
• internal/retention/keep_latest_test.go
• internal/retention/retention.go
• internal/server/retention.go
• internal/server/retention_keep_latest_test.go
• internal/server/retention_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.