From 8427d2b77721c4c21185f80a4f429dcad6842d1d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 14:47:52 +0000 Subject: [PATCH 01/11] feat(assail): route languages through a tested table; retire the vacuous PA1 PA1 was an Idris2 model of the analyzer dispatch whose theorem could not fail: `GenericAnalyzer : HasAnalyzer lang` inhabited its witness for every language. It was also stale (50 of 52 languages) and partly false (it said cross-language checks always run; they are skipped for configuration). The routing is now real code with claims that can fail: - src/assail/routing.rs: `route(Language) -> Route`, `Route::analyzer()`, and `runs_cross_language(is_configuration)`. `analyze_inner` matches on `route(file_lang)` and calls `runs_cross_language`. Only the arm heads changed; every arm body is byte-identical. `route` and the dispatch are exhaustive with no wildcard (Chapel, the one language without a dedicated analyzer, is routed to Generic explicitly). - src/types.rs: `Language::ALL` (52 variants), kept complete by an exhaustive `ordinal` match test, following WeakPointCategory::ALL. - Tests, exhaustive over Language::ALL: (a) every language analyzer in analyzer.rs is reached; (b) no language falls back to analyze_generic while a dedicated analyzer for it exists; (c) cross-language checks run exactly off configuration, and on scripts embedded in it (Justfile + Lua scan, end to end); (d) README's language counts equal the supported count. Each was seen to fail on a planted fault. - README: 49 -> 51 languages (badge, capabilities, summary); (d) caught the drift. C and Php were added after the count was written. - PatternCompleteness.idr removed from src/abi and the ipkg; PROOF-NEEDS, EXPLAINME, PROOF-PROGRAMME and the 0-AI-MANIFEST entry updated. PA1's status is TESTED; the Creusot/Pearlite and Verus proof is OPEN. Found while testing (c), recorded as an open obligation, not fixed here: the cross-language check strips `//` comments before matching, which also cuts unquoted `http://` URLs (shell, Justfile recipes). Checked: cargo fmt; clippy -D warnings with all and default features; cargo test --all-features (789 passed, 0 failed); the Idris2 package type-checks (4 modules); panic-attack's self-scan reports nothing new. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Jma5ixtpeZyJRmfoSsC98u --- 0-AI-MANIFEST.a2ml | 2 - EXPLAINME.adoc | 2 +- PROOF-NEEDS.adoc | 58 +-- PROOF-PROGRAMME.adoc | 11 +- README.adoc | 6 +- .../PanicAttack/ABI/PatternCompleteness.idr | 259 ------------- src/abi/panic-attack-abi.ipkg | 1 - src/assail/analyzer.rs | 87 ++--- src/assail/mod.rs | 1 + src/assail/routing.rs | 350 ++++++++++++++++++ src/types.rs | 134 +++++++ 11 files changed, 570 insertions(+), 341 deletions(-) delete mode 100644 src/abi/PanicAttack/ABI/PatternCompleteness.idr create mode 100644 src/assail/routing.rs diff --git a/0-AI-MANIFEST.a2ml b/0-AI-MANIFEST.a2ml index 116c3de0..b4225e63 100644 --- a/0-AI-MANIFEST.a2ml +++ b/0-AI-MANIFEST.a2ml @@ -37,8 +37,6 @@ (jit-context "src/jit_context.rs" "JIT-framework classifier: Cranelift / Llvm / Wasm / Javascript / None")) (proof-modules - (pa1 "src/abi/PatternCompleteness.idr" - "Pattern detection completeness — every Lang / WPCategory has a detector") (pa2 "src/abi/ClassificationSoundness.idr" "Classification soundness — Severity total order + maxSeverity commutativity") (layer-1-0 "src/abi/Stripping.idr" diff --git a/EXPLAINME.adoc b/EXPLAINME.adoc index f2ea5532..904514d5 100644 --- a/EXPLAINME.adoc +++ b/EXPLAINME.adoc @@ -52,7 +52,7 @@ The README makes claims. This file backs them up. | `src/comment_marker.rs` | Inline `// panic-attack: accepted [- reason]` suppression marker — `//` / `#` / `--` / `;` / `%` leaders, string-literal aware | `src/ffi_kind.rs` | UnsafeFFI (PA013) subtyping: BuildSystem (`build.zig` / `build.rs` — audit-accepted), RuntimeAbi (`bindings/` / `ffi/` / `sys/` / `cdef.zig`), TestMock (`tests/mocks/` / `_stub`), Unknown | `src/jit_context.rs` | JIT-framework classifier: Cranelift / Llvm / Wasm / Javascript / None; `transmute_targets_fn_ptr` recogniser for fn-ptr type-punning -| `src/abi/PanicAttack/ABI/PatternCompleteness.idr` | PA1 Idris2 model of analyzer dispatch; vacuous as stated (see `PROOF-NEEDS.adoc`) +| `src/assail/routing.rs` | PA1 routing table: which analyzer each language reaches and when cross-language checks run; its claims are tested exhaustively (see `PROOF-NEEDS.adoc`) | `src/abi/PanicAttack/ABI/ClassificationSoundness.idr` | PA2 Idris2 proof — severity total order, `sevMax` upper bounds, monotone encoding | `src/abi/PanicAttack/ABI/Stripping.idr` | Layer 1.0 — Idris2 model of line-comment stripping, proved idempotent (`stripLineCommentsIdempotent`) | `src/kanren/` | Logic engine (unification, fact database, taint, cross-lang) diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index 25b024ec..4d74d288 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -92,21 +92,29 @@ or signature scheme. So these are theorems of a symbolic (Dolev-Yao style) model, not a computational security reduction. Model to `src/attestation/`: `ASSUMED`. A computational proof is not claimed. -|Every language reaches an analyzer; every weak-point category has a -detector; cross-language checks run on every file (PA1) -|`src/abi/PanicAttack/ABI/PatternCompleteness.idr` -|`PROVEN` (model); supports no claim about the Rust code -|The theorem is vacuous: `GenericAnalyzer : HasAnalyzer lang` inhabits -the witness type for every language, so it holds whatever the dispatch -does. The guarantee it was meant to give already holds in Rust: -`match file_lang` in `src/assail/analyzer.rs` ends in -`_ => analyze_generic`, and `rustc` checks the match is exhaustive. - -The model has also drifted. Its `Lang` has 50 constructors and -`src/types.rs` has 52; `C` and `Php` are missing. `crossLangAlwaysApplied` -says cross-language checks run on every file, but the Rust code skips them -when `configuration.is_some()`. That model claim is false of the -implementation. +|Analyzer routing (PA1). (a) Every language analyzer in `analyzer.rs` is +reached by some language. (b) No language falls back to the generic +analyzer while a dedicated one exists for it. (c) Cross-language checks run +exactly on non-configuration content, and on scripts embedded in +configuration. (d) The README's language counts equal the languages the +table supports. +|`src/assail/routing.rs`: `route`, `Route::analyzer`, `runs_cross_language` +and their tests +|`TESTED` +|The dispatch in `analyze_inner` matches on `route(file_lang)`, so the +table is the code that runs, not a model of it. `route` and the dispatch +are exhaustive matches with no wildcard, so `rustc` makes every language +and every route explicit. The tests run over all 52 `Language` variants +(`Language::ALL`, kept complete by an exhaustive `ordinal` match). (a) and +(b) compare the table with the analyzer methods found in the source. (c) +scans a Justfile and a Lua file end to end. Each test was seen to fail on +a planted fault: an unrouted analyzer, Java sent to the generic analyzer, +cross-language checks forced on, and the README's stale count of 49. + +This replaces the Idris2 `PatternCompleteness.idr`, which was vacuous +(`GenericAnalyzer` inhabited its witness for every language), stale (50 of +52 languages) and partly false (it said cross-language checks always +run). It is retired. |Declarations for the ABI-facing types and their C integer encodings |`src/abi/PanicAttack/ABI/Types.idr` @@ -152,10 +160,18 @@ gap. fixed point) is theory and belongs in Agda with Rocq and Lean rechecks. Its realisation in `src/kanren/` is a Rust obligation. -|Replace or retire PA1 -|Creusot/Pearlite and Verus, or retire -|A non-vacuous statement would be, for example, that each language with -a dedicated analyzer is routed to it. If that is not wanted, retire the -module, because `rustc`'s exhaustiveness check already gives the current -guarantee. +|Prove PA1 (a) to (c) on `route` and `runs_cross_language` +|Creusot/Pearlite and Verus +|Pearlite contracts on the two functions, proved by both verifiers. The +tests above are exhaustive over a finite enum, but they are tests, and (a) +reads the source with a pattern match. (d) is a documentation check and +stays a test. + +|Unquoted `http://` URLs are missed by the cross-language check +|Fix first, then test +|`analyze_cross_language` strips C-style `//` comments from every file +before matching, which also cuts `curl http://host` in shell. A quoted +URL is found; an unquoted one in a `.sh` file or a Justfile recipe is not. +This is a recall gap in the InsecureProtocol detector, found while testing +PA1 (c). |=== diff --git a/PROOF-PROGRAMME.adoc b/PROOF-PROGRAMME.adoc index 9bcb2ce8..e1dbe672 100644 --- a/PROOF-PROGRAMME.adoc +++ b/PROOF-PROGRAMME.adoc @@ -14,10 +14,11 @@ ____ [width="100%",cols="34%,33%,33%",options="header",] |=== |ID |File |Mechanises -|*PA1* |`+src/abi/PanicAttack/ABI/PatternCompleteness.idr+` |Every `+Lang+` constructor -has at least one analyzer; every `+WPCategory+` has at least one -detector; cross-language checks apply uniformly. Top theorem: -`+completeScanForAll+`. +|*PA1* |`+src/assail/routing.rs+` |The routing table the dispatch uses: every +language analyzer is reached, no language falls back to the generic +analyzer while a dedicated one exists, cross-language checks run exactly +off configuration, and the README language counts match. Tested +exhaustively; the earlier Idris2 model was vacuous and is retired. |*PA2* |`+src/abi/PanicAttack/ABI/ClassificationSoundness.idr+` |`+Severity+` is totally ordered (`+LTE+`); `+maxSeverity+` is commutative + idempotent; numeric @@ -316,7 +317,7 @@ docs/proof artefacts that don’t touch the hot path. * `+PROOF-NEEDS.md+` — current proof-debt ledger (this document is the long-term plan; that one tracks the immediate next step). -* `+src/abi/PanicAttack/ABI/PatternCompleteness.idr+` (PA1) +* `+src/assail/routing.rs+` (PA1) * `+src/abi/PanicAttack/ABI/ClassificationSoundness.idr+` (PA2) * `+hyperpolymath/proven+` — leaf validator library; cross-fit candidates listed above. diff --git a/README.adoc b/README.adoc index 2bcedffc..eaa139c2 100644 --- a/README.adoc +++ b/README.adoc @@ -12,7 +12,7 @@ image:https://api.thegreenwebfoundation.org/greencheckimage/github.com[Green Web image:https://img.shields.io/badge/status-active-green[Status] image:https://img.shields.io/badge/domain-security--analysis-blue[Domain] image:https://img.shields.io/badge/tests-897%20passing-brightgreen[Tests] -image:https://img.shields.io/badge/languages-49-blue[Languages] +image:https://img.shields.io/badge/languages-51-blue[Languages] **panic-attack** is a multi-language security analysis and stress-testing system. @@ -69,7 +69,7 @@ panic-attack exists to address this by combining: panic-attack provides: -* **49-language static analysis** across multiple families +* **51-language static analysis** across multiple families * **Weak point detection** (30 canonical categories; see the PA rule-ID table below) * **Attack simulation (6 axes)**: CPU, memory, disk, network, concurrency, time * **miniKanren logic engine** for taint analysis and cross-language reasoning @@ -235,7 +235,7 @@ Current state: **v2.5.0** * v2.5.5 cohort landed 2026-06-02: context-aware FP suppression via `test_context` / `comment_marker` / `ffi_kind` / `jit_context` modules — `// panic-attack: accepted` inline markers, automatic PanicPath suppression in test-only code, build.zig/build.rs auto-accept, Cranelift/LLVM/Wasm/JS JIT detection. See `PROOF-PROGRAMME.md` for the formal soundness landscape * 0 warnings * 38 CLI subcommands -* 49 supported languages (30 canonical weak-point categories PA001–PA029; `PA001b` is a distinct stable rule ID grouped with PA001 under one allocation semantic class for cross-tool de-duplication) +* 51 supported languages (30 canonical weak-point categories PA001–PA029; `PA001b` is a distinct stable rule ID grouped with PA001 under one allocation semantic class for cross-tool de-duplication) See link:ROADMAP.adoc[ROADMAP.adoc] for full capability breakdown and milestones. diff --git a/src/abi/PanicAttack/ABI/PatternCompleteness.idr b/src/abi/PanicAttack/ABI/PatternCompleteness.idr deleted file mode 100644 index 52955752..00000000 --- a/src/abi/PanicAttack/ABI/PatternCompleteness.idr +++ /dev/null @@ -1,259 +0,0 @@ --- SPDX-License-Identifier: MPL-2.0 --- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell --- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) - -||| Pattern Matching Completeness Proofs -||| -||| Proves that every language in the scanner's catalogue has an -||| associated analyzer, and every weak point category in the catalogue -||| has at least one detection rule. This eliminates false negatives -||| caused by missing dispatch arms. -||| -||| Design: We model the Language and WeakPointCategory enumerations -||| as inductive types and prove that the dispatch function (modelled -||| as `analyzerFor` / `detectorsFor`) covers every constructor. -||| The proof is structural induction on the enumeration -- Idris2's -||| totality checker verifies that every case is handled. -module PanicAttack.ABI.PatternCompleteness - -%default total - --- ═══════════════════════════════════════════════════════════════════════ --- Language enumeration (mirrors src/types.rs Language enum, 49 variants) --- ═══════════════════════════════════════════════════════════════════════ - -||| All 49 programming languages supported by the scanner. -||| This MUST stay in sync with src/types.rs Language enum. -||| Note: Cpp is the C/C++ unified analyzer (analyze_c_cpp dispatches -||| on .c / .cpp / .h / .hpp / .cc / .hh — there is no separate C -||| constructor in the Rust enum). -public export -data Lang - = Rust | Cpp | Go | Java | Python | JavaScript | Ruby - -- BEAM family - | Elixir | Erlang | Gleam - -- ML family - | ReScript | OCaml | StandardML - -- Lisp family - | Scheme | Racket - -- Functional - | Haskell | PureScript - -- Proof assistants - | Idris | Lean | Agda | Isabelle | Coq - -- Logic programming - | Prolog | Logtalk | Datalog - -- Systems languages - | Zig | Ada | Odin | Nim | Pony | DLang - -- Config languages - | Nickel | Nix - -- Scripting / data - | Shell | Julia | Lua - -- HPC / parallel - | Chapel - -- Nextgen custom DSLs - | WokeLang | Eclexia | MyLang | JuliaTheViper | Oblibeny - | Anvomidav | AffineScript | Ephapax | BetLang | ErrorLang - | VCL | FBQL - -- Catch-all - | Unknown - --- ═══════════════════════════════════════════════════════════════════════ --- Analyzer dispatch witness --- ═══════════════════════════════════════════════════════════════════════ - -||| Witness that an analyzer function exists for a given language. -||| Each constructor corresponds to a dispatch arm in analyzer.rs. -public export -data HasAnalyzer : Lang -> Type where - ||| Language-specific analyzer (e.g., analyze_rust, analyze_c_cpp) - SpecificAnalyzer : HasAnalyzer lang - ||| Generic fallback analyzer (analyze_generic) - GenericAnalyzer : HasAnalyzer lang - -||| Every language in the enumeration has an analyzer. -||| This function is total: Idris2 verifies exhaustive coverage. -||| The dispatch mirrors src/assail/analyzer.rs analyze_inner(). -public export -analyzerFor : (lang : Lang) -> HasAnalyzer lang -analyzerFor Rust = SpecificAnalyzer -analyzerFor Cpp = SpecificAnalyzer -analyzerFor Go = SpecificAnalyzer -analyzerFor Java = SpecificAnalyzer -analyzerFor Python = SpecificAnalyzer -analyzerFor JavaScript = SpecificAnalyzer -analyzerFor Ruby = SpecificAnalyzer -analyzerFor Elixir = SpecificAnalyzer -analyzerFor Erlang = SpecificAnalyzer -analyzerFor Gleam = SpecificAnalyzer -analyzerFor ReScript = SpecificAnalyzer -analyzerFor OCaml = SpecificAnalyzer -analyzerFor StandardML = SpecificAnalyzer -analyzerFor Scheme = SpecificAnalyzer -analyzerFor Racket = SpecificAnalyzer -analyzerFor Haskell = SpecificAnalyzer -analyzerFor PureScript = SpecificAnalyzer -analyzerFor Idris = SpecificAnalyzer -analyzerFor Lean = SpecificAnalyzer -analyzerFor Agda = SpecificAnalyzer -analyzerFor Isabelle = SpecificAnalyzer -analyzerFor Coq = SpecificAnalyzer -analyzerFor Prolog = SpecificAnalyzer -analyzerFor Logtalk = SpecificAnalyzer -analyzerFor Datalog = SpecificAnalyzer -analyzerFor Zig = SpecificAnalyzer -analyzerFor Ada = SpecificAnalyzer -analyzerFor Odin = SpecificAnalyzer -analyzerFor Nim = SpecificAnalyzer -analyzerFor Pony = SpecificAnalyzer -analyzerFor DLang = SpecificAnalyzer -analyzerFor Nickel = SpecificAnalyzer -analyzerFor Nix = SpecificAnalyzer -analyzerFor Shell = SpecificAnalyzer -analyzerFor Julia = SpecificAnalyzer -analyzerFor Lua = SpecificAnalyzer -analyzerFor Chapel = SpecificAnalyzer -analyzerFor WokeLang = SpecificAnalyzer -analyzerFor Eclexia = SpecificAnalyzer -analyzerFor MyLang = SpecificAnalyzer -analyzerFor JuliaTheViper = SpecificAnalyzer -analyzerFor Oblibeny = SpecificAnalyzer -analyzerFor Anvomidav = SpecificAnalyzer -analyzerFor AffineScript = SpecificAnalyzer -analyzerFor Ephapax = SpecificAnalyzer -analyzerFor BetLang = SpecificAnalyzer -analyzerFor ErrorLang = SpecificAnalyzer -analyzerFor VCL = SpecificAnalyzer -analyzerFor FBQL = SpecificAnalyzer -analyzerFor Unknown = GenericAnalyzer - -||| Proof: for ALL languages, an analyzer exists. -||| This is the top-level completeness theorem. -public export -allLanguagesHaveAnalyzers : (lang : Lang) -> HasAnalyzer lang -allLanguagesHaveAnalyzers = analyzerFor - --- ═══════════════════════════════════════════════════════════════════════ --- Cross-language security check proof --- ═══════════════════════════════════════════════════════════════════════ - -||| Witness that cross-language checks are always applied. -||| In analyzer.rs, analyze_cross_language() runs on ALL files -||| regardless of language. This type encodes that invariant. -public export -data CrossLangChecked : Lang -> Type where - MkCrossLangChecked : CrossLangChecked lang - -||| Every language receives cross-language security checks. -||| This is trivially total because the check is unconditional. -public export -crossLangAlwaysApplied : (lang : Lang) -> CrossLangChecked lang -crossLangAlwaysApplied _ = MkCrossLangChecked - --- ═══════════════════════════════════════════════════════════════════════ --- WeakPointCategory enumeration (mirrors src/types.rs, 25 categories) --- ═══════════════════════════════════════════════════════════════════════ - -||| All 25 weak point categories detectable by the scanner. -public export -data WPCategory - = UncheckedAllocation - | UnboundedLoop - | BlockingIO - | UnsafeCode - | PanicPath - | RaceCondition - | DeadlockPotential - | ResourceLeak - | CommandInjection - | UnsafeDeserialization - | DynamicCodeExecution - | UnsafeFFI - | AtomExhaustion - | InsecureProtocol - | ExcessivePermissions - | PathTraversal - | HardcodedSecret - | UncheckedError - | InfiniteRecursion - | UnsafeTypeCoercion - -- PA021 ProofDrift: banned proof escape hatches or mirror files - -- substituting assertions for proofs. Detected in .idr/.lean/.agda/.thy/.v - -- and Julia mirror files. - | ProofDrift - -- PA022 CryptoMisuse: weak hash (MD5/SHA-1) in security context, - -- constant-time comparison violations, key/nonce reuse. - -- Detected in Rust, Python, JavaScript, Go, Elixir. - | CryptoMisuse - -- PA023 SupplyChain: unpinned deps, absent lock files, unverified - -- manifests. Cargo git deps without rev=, absent Cargo.lock, Julia - -- Manifest.toml without hash, flake.nix without narHash, deno.json - -- unpinned specifiers. Detected in Rust, Julia, Nix, JavaScript. - | SupplyChain - -- PA024 InputBoundary: unchecked CBOR/MessagePack deserialization - -- (serde_cbor, ciborium, rmp_serde in Rust), JSON.parse without - -- try/catch (JavaScript), JSON3.read without error handling (Julia). - | InputBoundary - -- PA025 MutationGap: test suites with no mutation-test tooling - -- (no cargo-mutants in Rust), no property-based testing in Elixir - -- (ExUnitProperties/StreamData absent), or Julia @testset blocks - -- with only type-check assertions and no value diversity. - | MutationGap - -||| Witness that a detection rule exists for a weak point category. -||| Each variant names the language(s) whose analyzer detects it. -public export -data HasDetector : WPCategory -> Type where - ||| Detected by one or more language-specific analyzers - DetectedBy : (langs : List Lang) -> HasDetector cat - -||| Every weak point category has at least one detector. -||| Total: Idris2 verifies all 25 constructors are covered. -||| The list of detecting languages mirrors the actual pattern -||| matching code in analyzer.rs. -public export -detectorsFor : (cat : WPCategory) -> HasDetector cat -detectorsFor UncheckedAllocation = DetectedBy [Cpp] -detectorsFor UnboundedLoop = DetectedBy [Rust, Cpp, Go, Python, JavaScript] -detectorsFor BlockingIO = DetectedBy [Rust, Go, Python, JavaScript] -detectorsFor UnsafeCode = DetectedBy [Rust, Cpp, Zig, Nim, DLang] -detectorsFor PanicPath = DetectedBy [Rust, Go, Haskell] -detectorsFor RaceCondition = DetectedBy [Rust, Cpp, Go] -detectorsFor DeadlockPotential = DetectedBy [Rust, Cpp, Go] -detectorsFor ResourceLeak = DetectedBy [Rust, Cpp] -detectorsFor CommandInjection = DetectedBy [Python, Ruby, JavaScript, Shell, Elixir, Erlang] -detectorsFor UnsafeDeserialization = DetectedBy [Python, Ruby, JavaScript, Java] -detectorsFor DynamicCodeExecution = DetectedBy [Python, JavaScript, Ruby, Elixir, Shell] -detectorsFor UnsafeFFI = DetectedBy [Elixir, Erlang, Haskell, Pony, OCaml, Zig] -detectorsFor AtomExhaustion = DetectedBy [Elixir, Erlang] -detectorsFor InsecureProtocol = DetectedBy [Rust, Cpp, Go, Python, JavaScript, Ruby] -detectorsFor ExcessivePermissions = DetectedBy [Shell] -detectorsFor PathTraversal = DetectedBy [Python, JavaScript, Ruby, Go, Java] -detectorsFor HardcodedSecret = DetectedBy [Rust, Cpp, Go, Python, JavaScript, Ruby] -detectorsFor UncheckedError = DetectedBy [Go, Rust, Cpp] -detectorsFor InfiniteRecursion = DetectedBy [Haskell, PureScript, Scheme, Racket] -detectorsFor UnsafeTypeCoercion = DetectedBy [OCaml, Haskell, DLang, Nim] -detectorsFor ProofDrift = DetectedBy [Idris, Lean, Agda, Isabelle, Coq, Julia] -detectorsFor CryptoMisuse = DetectedBy [Rust, Python, JavaScript, Go, Elixir] -detectorsFor SupplyChain = DetectedBy [Rust, Julia, Nix, JavaScript] -detectorsFor InputBoundary = DetectedBy [Rust, JavaScript, Julia] -detectorsFor MutationGap = DetectedBy [Rust, Julia, Elixir] - -||| Proof: every weak point category has at least one detector. -public export -allCategoriesDetected : (cat : WPCategory) -> HasDetector cat -allCategoriesDetected = detectorsFor - --- ═══════════════════════════════════════════════════════════════════════ --- Combined completeness theorem --- ═══════════════════════════════════════════════════════════════════════ - -||| A complete scan covers both language-specific analysis AND -||| cross-language security checks for any given language. -public export -data CompleteScan : Lang -> Type where - MkCompleteScan : HasAnalyzer lang -> CrossLangChecked lang -> CompleteScan lang - -||| Every language receives a complete scan. -public export -completeScanForAll : (lang : Lang) -> CompleteScan lang -completeScanForAll lang = MkCompleteScan (analyzerFor lang) (crossLangAlwaysApplied lang) diff --git a/src/abi/panic-attack-abi.ipkg b/src/abi/panic-attack-abi.ipkg index 4d874b80..1b3ab731 100644 --- a/src/abi/panic-attack-abi.ipkg +++ b/src/abi/panic-attack-abi.ipkg @@ -9,7 +9,6 @@ package panic-attack-abi sourcedir = "." modules = PanicAttack.ABI.Types - , PanicAttack.ABI.PatternCompleteness , PanicAttack.ABI.ClassificationSoundness , PanicAttack.ABI.Stripping , PanicAttack.ABI.AttestationUnforgeability diff --git a/src/assail/analyzer.rs b/src/assail/analyzer.rs index fb0af3a8..2dd13410 100644 --- a/src/assail/analyzer.rs +++ b/src/assail/analyzer.rs @@ -523,8 +523,8 @@ impl Analyzer { acc.record_file(&rel_path, &raw_bytes, &format!("{:?}", file_lang)); } - match file_lang { - Language::Unknown => { + match super::routing::route(file_lang) { + super::routing::Route::InputAdapter => { let Some(format) = configuration else { coverage.mark( file, @@ -589,7 +589,7 @@ impl Analyzer { } } } - Language::Php => match super::php::analyze(&content, &rel_path) { + super::routing::Route::Php => match super::php::analyze(&content, &rel_path) { Ok(findings) => file_weak_points.extend(findings), Err(error) => { coverage.mark( @@ -601,10 +601,10 @@ impl Analyzer { continue; } }, - Language::Rust => { + super::routing::Route::Rust => { self.analyze_rust(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::C | Language::Cpp => { + super::routing::Route::CCpp => { self.analyze_c_cpp( &content, &mut file_stats, @@ -612,10 +612,10 @@ impl Analyzer { &rel_path, )?; } - Language::Go => { + super::routing::Route::Go => { self.analyze_go(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Python => { + super::routing::Route::Python => { self.analyze_python( &content, &mut file_stats, @@ -623,7 +623,7 @@ impl Analyzer { &rel_path, )?; } - Language::JavaScript => { + super::routing::Route::JavaScript => { self.analyze_javascript( &content, &mut file_stats, @@ -631,11 +631,11 @@ impl Analyzer { &rel_path, )?; } - Language::Ruby => { + super::routing::Route::Ruby => { self.analyze_ruby(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } // BEAM family - Language::Elixir => { + super::routing::Route::Elixir => { self.analyze_elixir( &content, &mut file_stats, @@ -643,7 +643,7 @@ impl Analyzer { &rel_path, )?; } - Language::Erlang => { + super::routing::Route::Erlang => { self.analyze_erlang( &content, &mut file_stats, @@ -651,7 +651,7 @@ impl Analyzer { &rel_path, )?; } - Language::Gleam => { + super::routing::Route::Gleam => { self.analyze_gleam( &content, &mut file_stats, @@ -660,7 +660,7 @@ impl Analyzer { )?; } // ML family - Language::ReScript => { + super::routing::Route::ReScript => { record_migration_file(file_stats.total_lines); self.analyze_rescript( &content, @@ -669,7 +669,7 @@ impl Analyzer { &rel_path, )?; } - Language::OCaml => { + super::routing::Route::OCaml => { self.analyze_ocaml( &content, &mut file_stats, @@ -677,15 +677,15 @@ impl Analyzer { &rel_path, )?; } - Language::StandardML => { + super::routing::Route::StandardML => { self.analyze_sml(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } // Lisp family - Language::Scheme | Language::Racket => { + super::routing::Route::Lisp => { self.analyze_lisp(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } // Functional - Language::Haskell => { + super::routing::Route::Haskell => { self.analyze_haskell( &content, &mut file_stats, @@ -693,7 +693,7 @@ impl Analyzer { &rel_path, )?; } - Language::PureScript => { + super::routing::Route::PureScript => { self.analyze_purescript( &content, &mut file_stats, @@ -702,7 +702,7 @@ impl Analyzer { )?; } // Proof assistants - Language::Idris => { + super::routing::Route::Idris => { self.analyze_idris( &content, &mut file_stats, @@ -710,13 +710,13 @@ impl Analyzer { &rel_path, )?; } - Language::Lean => { + super::routing::Route::Lean => { self.analyze_lean(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Agda => { + super::routing::Route::Agda => { self.analyze_agda(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Isabelle => { + super::routing::Route::Isabelle => { self.analyze_isabelle( &content, &mut file_stats, @@ -724,11 +724,11 @@ impl Analyzer { &rel_path, )?; } - Language::Coq => { + super::routing::Route::Coq => { self.analyze_coq(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } // Logic programming - Language::Prolog | Language::Logtalk | Language::Datalog => { + super::routing::Route::Logic => { self.analyze_logic( &content, &mut file_stats, @@ -737,22 +737,22 @@ impl Analyzer { )?; } // Systems languages - Language::Zig => { + super::routing::Route::Zig => { self.analyze_zig(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Ada => { + super::routing::Route::Ada => { self.analyze_ada(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Odin => { + super::routing::Route::Odin => { self.analyze_odin(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Nim => { + super::routing::Route::Nim => { self.analyze_nim(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::Pony => { + super::routing::Route::Pony => { self.analyze_pony(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - Language::DLang => { + super::routing::Route::DLang => { self.analyze_dlang( &content, &mut file_stats, @@ -761,7 +761,7 @@ impl Analyzer { )?; } // Config languages - Language::Nickel | Language::Nix => { + super::routing::Route::Config => { self.analyze_config( &content, &mut file_stats, @@ -770,7 +770,7 @@ impl Analyzer { )?; } // Scripting - Language::Shell => { + super::routing::Route::Shell => { self.analyze_shell( &content, &mut file_stats, @@ -778,7 +778,7 @@ impl Analyzer { &rel_path, )?; } - Language::Julia => { + super::routing::Route::Julia => { self.analyze_julia( &content, &mut file_stats, @@ -786,22 +786,11 @@ impl Analyzer { &rel_path, )?; } - Language::Lua => { + super::routing::Route::Lua => { self.analyze_lua(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } // Nextgen DSLs - shared analyzer - Language::WokeLang - | Language::Eclexia - | Language::MyLang - | Language::JuliaTheViper - | Language::Oblibeny - | Language::Anvomidav - | Language::AffineScript - | Language::Ephapax - | Language::BetLang - | Language::ErrorLang - | Language::VCL - | Language::FBQL => { + super::routing::Route::NextgenDsl => { self.analyze_nextgen_dsl( &content, &mut file_stats, @@ -809,10 +798,10 @@ impl Analyzer { &rel_path, )?; } - Language::Java => { + super::routing::Route::Java => { self.analyze_java(&content, &mut file_stats, &mut file_weak_points, &rel_path)?; } - _ => { + super::routing::Route::Generic => { self.analyze_generic(&content, &mut file_stats, &rel_path)?; } } @@ -820,7 +809,7 @@ impl Analyzer { // Structured configuration uses parsed keys and values; treating // comments or schema identifiers as executable source creates false // findings. Source-language adapters retain their existing checks. - if configuration.is_none() { + if super::routing::runs_cross_language(configuration.is_some()) { self.analyze_cross_language(&content, &mut file_weak_points, &rel_path)?; } diff --git a/src/assail/mod.rs b/src/assail/mod.rs index 3a628465..3d21ebb3 100644 --- a/src/assail/mod.rs +++ b/src/assail/mod.rs @@ -14,6 +14,7 @@ mod forth; mod input_adapters; pub mod patterns; pub mod php; +mod routing; mod templates; use crate::kanren::core::{FactDB, LogicEngine}; diff --git a/src/assail/routing.rs b/src/assail/routing.rs new file mode 100644 index 00000000..aa19d80d --- /dev/null +++ b/src/assail/routing.rs @@ -0,0 +1,350 @@ +// SPDX-License-Identifier: MPL-2.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell + +//! Which analyzer each language is sent to, and when cross-language checks +//! run (PA1). +//! +//! `analyze_inner` dispatches on [`route`] instead of matching on the +//! language directly, so the table here is the dispatch, not a copy of it. +//! Both `match`es below are exhaustive with no wildcard: a new `Language` +//! fails to compile until it is given a route, and a new `Route` fails to +//! compile until it is given an analyzer. +//! +//! The claims about the table are tested exhaustively in this module's +//! tests (`TESTED` in `PROOF-NEEDS.adoc`; the Creusot/Pearlite and Verus +//! proofs remain open). + +use crate::types::Language; + +/// The analyzer a file of a given language is handed to. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub(crate) enum Route { + /// No source language: a registered configuration or build-input + /// adapter handles the file, if one matches. + InputAdapter, + Php, + Rust, + CCpp, + Go, + Python, + JavaScript, + Ruby, + Elixir, + Erlang, + Gleam, + ReScript, + OCaml, + StandardML, + Lisp, + Haskell, + PureScript, + Idris, + Lean, + Agda, + Isabelle, + Coq, + Logic, + Zig, + Ada, + Odin, + Nim, + Pony, + DLang, + Config, + Shell, + Julia, + Lua, + NextgenDsl, + Java, + /// The language-neutral fallback, for languages with no dedicated + /// analyzer. + Generic, +} + +/// The analyzer for `lang`. +pub(crate) fn route(lang: Language) -> Route { + use Language as L; + match lang { + L::Unknown => Route::InputAdapter, + L::Php => Route::Php, + L::Rust => Route::Rust, + L::C | L::Cpp => Route::CCpp, + L::Go => Route::Go, + L::Python => Route::Python, + L::JavaScript => Route::JavaScript, + L::Ruby => Route::Ruby, + L::Elixir => Route::Elixir, + L::Erlang => Route::Erlang, + L::Gleam => Route::Gleam, + L::ReScript => Route::ReScript, + L::OCaml => Route::OCaml, + L::StandardML => Route::StandardML, + L::Scheme | L::Racket => Route::Lisp, + L::Haskell => Route::Haskell, + L::PureScript => Route::PureScript, + L::Idris => Route::Idris, + L::Lean => Route::Lean, + L::Agda => Route::Agda, + L::Isabelle => Route::Isabelle, + L::Coq => Route::Coq, + L::Prolog | L::Logtalk | L::Datalog => Route::Logic, + L::Zig => Route::Zig, + L::Ada => Route::Ada, + L::Odin => Route::Odin, + L::Nim => Route::Nim, + L::Pony => Route::Pony, + L::DLang => Route::DLang, + L::Nickel | L::Nix => Route::Config, + L::Shell => Route::Shell, + L::Julia => Route::Julia, + L::Lua => Route::Lua, + L::WokeLang + | L::Eclexia + | L::MyLang + | L::JuliaTheViper + | L::Oblibeny + | L::Anvomidav + | L::AffineScript + | L::Ephapax + | L::BetLang + | L::ErrorLang + | L::VCL + | L::FBQL => Route::NextgenDsl, + L::Java => Route::Java, + L::Chapel => Route::Generic, + } +} + +impl Route { + /// The function that `analyze_inner` calls for this route. + #[cfg_attr(not(test), allow(dead_code))] + pub(crate) fn analyzer(self) -> &'static str { + match self { + Route::InputAdapter => "input_adapters::Adapter::analyze", + Route::Php => "php::analyze", + Route::Rust => "analyze_rust", + Route::CCpp => "analyze_c_cpp", + Route::Go => "analyze_go", + Route::Python => "analyze_python", + Route::JavaScript => "analyze_javascript", + Route::Ruby => "analyze_ruby", + Route::Elixir => "analyze_elixir", + Route::Erlang => "analyze_erlang", + Route::Gleam => "analyze_gleam", + Route::ReScript => "analyze_rescript", + Route::OCaml => "analyze_ocaml", + Route::StandardML => "analyze_sml", + Route::Lisp => "analyze_lisp", + Route::Haskell => "analyze_haskell", + Route::PureScript => "analyze_purescript", + Route::Idris => "analyze_idris", + Route::Lean => "analyze_lean", + Route::Agda => "analyze_agda", + Route::Isabelle => "analyze_isabelle", + Route::Coq => "analyze_coq", + Route::Logic => "analyze_logic", + Route::Zig => "analyze_zig", + Route::Ada => "analyze_ada", + Route::Odin => "analyze_odin", + Route::Nim => "analyze_nim", + Route::Pony => "analyze_pony", + Route::DLang => "analyze_dlang", + Route::Config => "analyze_config", + Route::Shell => "analyze_shell", + Route::Julia => "analyze_julia", + Route::Lua => "analyze_lua", + Route::NextgenDsl => "analyze_nextgen_dsl", + Route::Java => "analyze_java", + Route::Generic => "analyze_generic", + } + } +} + +/// Whether the cross-language checks run over a file's own content. +/// +/// They run on every source file and on no structured configuration file: +/// configuration is analysed as parsed keys and values, and treating its +/// comments or schema identifiers as executable source creates false +/// findings. Scripts embedded in configuration are still checked, each on +/// its own, by the input-adapter route. +pub(crate) fn runs_cross_language(is_configuration: bool) -> bool { + !is_configuration +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::assail::analyzer::Analyzer; + use crate::types::WeakPointCategory; + use std::collections::BTreeSet; + use std::fs; + use tempfile::TempDir; + + /// Language-analyzer methods defined in analyzer.rs: every + /// `fn analyze_(&self, content: &str, ...)`. Read from the source, + /// so a new analyzer is seen without anyone listing it here. + fn analyzer_methods_in_source() -> BTreeSet { + let source = include_str!("analyzer.rs"); + let compact: String = source.split_whitespace().collect::>().join(" "); + let mut found = BTreeSet::new(); + for piece in compact.split("fn analyze_").skip(1) { + let name: String = piece + .chars() + .take_while(|c| c.is_ascii_alphanumeric() || *c == '_') + .collect(); + let rest = &piece[name.len()..]; + if rest.starts_with("( &self, content: &str,") + || rest.starts_with("(&self, content: &str,") + { + found.insert(format!("analyze_{name}")); + } + } + found + } + + /// Methods with the analyzer signature that no language routes to by + /// design: helpers run on every file or called from other analyzers. + const NOT_LANGUAGE_ANALYZERS: [&str; 1] = ["analyze_cross_language"]; + + fn snake(lang: Language) -> String { + let name = format!("{lang:?}"); + let mut out = String::new(); + for (i, c) in name.chars().enumerate() { + if c.is_ascii_uppercase() && i > 0 { + out.push('_'); + } + out.push(c.to_ascii_lowercase()); + } + out + } + + /// Claim 1a: every language analyzer in the source is reached by some + /// language. An analyzer nothing routes to is dead code. + #[test] + fn every_language_analyzer_is_reached() { + let routed: BTreeSet<&str> = Language::ALL.iter().map(|l| route(*l).analyzer()).collect(); + let methods = analyzer_methods_in_source(); + assert!( + methods.len() > 30, + "source scan found only {methods:?}: the signature pattern no longer matches" + ); + for method in &methods { + if NOT_LANGUAGE_ANALYZERS.contains(&method.as_str()) { + continue; + } + assert!( + routed.contains(method.as_str()), + "{method} exists in analyzer.rs but no language routes to it" + ); + } + } + + /// Claim 1b: every routed analyzer exists, and no language goes to the + /// generic fallback while a dedicated analyzer for it exists. + #[test] + fn no_language_falls_back_while_a_dedicated_analyzer_exists() { + let methods = analyzer_methods_in_source(); + for lang in Language::ALL { + let r = route(lang); + let name = r.analyzer(); + if name.starts_with("analyze_") { + assert!( + methods.contains(name), + "{lang:?} routes to {name}, which analyzer.rs does not define" + ); + } + if r == Route::Generic { + let own = format!("analyze_{}", snake(lang)); + assert!( + !methods.contains(&own), + "{lang:?} falls back to analyze_generic, but {own} exists" + ); + } + } + } + + /// Claim 2: cross-language checks run exactly on non-configuration + /// content, and on scripts embedded in configuration. + #[test] + fn cross_language_checks_run_exactly_off_configuration() { + assert!(runs_cross_language(false)); + assert!(!runs_cross_language(true)); + + // Built at run time so this source file holds no insecure URL of its + // own for panic-attack's self-scan to report. + let url = |path: &str| format!("{}://example.org/{path}", "http"); + let tmp = TempDir::new().unwrap(); + // A Justfile is configuration. Its variable is configuration content; + // its recipe body is an embedded shell script. The recipe URL is + // quoted: the cross-language detector strips `//` comments first, + // which also cuts an unquoted URL (a separate recall gap). + fs::write( + tmp.path().join("Justfile"), + format!( + "mirror := \"{}\"\n\nfetch:\n curl \"{}\"\n", + url("variable"), + url("recipe") + ), + ) + .unwrap(); + fs::write( + tmp.path().join("main.lua"), + format!("local url = \"{}\"\nprint(url)\n", url("source")), + ) + .unwrap(); + + let report = Analyzer::new(tmp.path()).unwrap().analyze().unwrap(); + let insecure: Vec<_> = report + .weak_points + .iter() + .filter(|w| w.category == WeakPointCategory::InsecureProtocol) + .collect(); + + assert!( + insecure.iter().any(|w| w.description.contains("main.lua")), + "source file was not checked: {insecure:?}" + ); + let justfile: Vec<_> = insecure + .iter() + .filter(|w| w.description.contains("Justfile")) + .collect(); + assert!( + justfile.iter().any(|w| w.description.contains("embedded")), + "the recipe's embedded script was not checked: {insecure:?}" + ); + assert!( + justfile.iter().all(|w| w.description.contains("embedded")), + "configuration content was checked as source: {insecure:?}" + ); + } + + /// Claim 3: the language counts the README states are the count the + /// table supports (every language except the `Unknown` sentinel). + #[test] + fn readme_language_counts_match_the_table() { + let supported = Language::ALL + .iter() + .filter(|l| **l != Language::Unknown) + .count(); + let readme = include_str!("../../README.adoc"); + let claims = [ + ("badge", "languages-", "-blue"), + ("capabilities", "**", "-language static analysis"), + ("summary", "* ", " supported languages"), + ]; + for (label, before, after) in claims { + let stated = readme + .lines() + .find_map(|line| { + let end = line.find(after)?; + let start = line[..end].rfind(before)? + before.len(); + line[start..end].parse::().ok() + }) + .unwrap_or_else(|| panic!("README no longer states the {label} count")); + assert_eq!( + stated, supported, + "README {label} says {stated} languages; the routing table supports {supported}" + ); + } + } +} diff --git a/src/types.rs b/src/types.rs index 1c7b7eae..fad80e6b 100644 --- a/src/types.rs +++ b/src/types.rs @@ -97,6 +97,67 @@ pub enum Language { } impl Language { + /// Every language, in declaration order, `Unknown` last. Tests iterate + /// over it, so a property checked "for every language" covers the whole + /// enum; `language_tests::all_lists_every_variant_once_in_order` keeps it + /// complete. + // The binary re-declares this module and never reads ALL itself. + #[allow(dead_code)] + pub const ALL: [Language; 52] = [ + Language::Rust, + Language::C, + Language::Cpp, + Language::Go, + Language::Java, + Language::Python, + Language::JavaScript, + Language::Ruby, + Language::Php, + Language::Elixir, + Language::Erlang, + Language::Gleam, + Language::ReScript, + Language::OCaml, + Language::StandardML, + Language::Scheme, + Language::Racket, + Language::Haskell, + Language::PureScript, + Language::Idris, + Language::Lean, + Language::Agda, + Language::Isabelle, + Language::Coq, + Language::Prolog, + Language::Logtalk, + Language::Datalog, + Language::Zig, + Language::Ada, + Language::Odin, + Language::Nim, + Language::Pony, + Language::DLang, + Language::Nickel, + Language::Nix, + Language::Shell, + Language::Julia, + Language::Lua, + Language::Chapel, + Language::WokeLang, + Language::Eclexia, + Language::MyLang, + Language::JuliaTheViper, + Language::Oblibeny, + Language::Anvomidav, + Language::AffineScript, + Language::Ephapax, + Language::BetLang, + Language::ErrorLang, + Language::VCL, + Language::FBQL, + Language::Unknown, + ]; + /// Infer the language from the final, case-sensitive extension without reading /// the file. Missing or unrecognised extensions yield `Unknown`. /// @@ -1154,6 +1215,79 @@ mod rule_id_tests { } } +#[cfg(test)] +mod language_tests { + use super::Language; + + // Exhaustive, with no wildcard: a new variant fails to compile here + // until it is given a place, and the test below then fails until ALL + // lists it at that place. + fn ordinal(l: Language) -> usize { + use Language::*; + match l { + Rust => 0, + C => 1, + Cpp => 2, + Go => 3, + Java => 4, + Python => 5, + JavaScript => 6, + Ruby => 7, + Php => 8, + Elixir => 9, + Erlang => 10, + Gleam => 11, + ReScript => 12, + OCaml => 13, + StandardML => 14, + Scheme => 15, + Racket => 16, + Haskell => 17, + PureScript => 18, + Idris => 19, + Lean => 20, + Agda => 21, + Isabelle => 22, + Coq => 23, + Prolog => 24, + Logtalk => 25, + Datalog => 26, + Zig => 27, + Ada => 28, + Odin => 29, + Nim => 30, + Pony => 31, + DLang => 32, + Nickel => 33, + Nix => 34, + Shell => 35, + Julia => 36, + Lua => 37, + Chapel => 38, + WokeLang => 39, + Eclexia => 40, + MyLang => 41, + JuliaTheViper => 42, + Oblibeny => 43, + Anvomidav => 44, + AffineScript => 45, + Ephapax => 46, + BetLang => 47, + ErrorLang => 48, + VCL => 49, + FBQL => 50, + Unknown => 51, + } + } + + #[test] + fn all_lists_every_variant_once_in_order() { + for (i, l) in Language::ALL.iter().enumerate() { + assert_eq!(ordinal(*l), i, "ALL[{i}] is {l:?}, out of place"); + } + } +} + #[cfg(test)] mod language_detect_tests { use super::Language; From 39db4e30658e6e767cfc8f705c962566a518ccf4 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 9 Nov 2026 22:30:23 +0000 Subject: [PATCH 02/11] fix(deps): move futures-util to 0.3.34, dropping pin-utils (#273) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RUSTSEC-2026-0441 marks `pin-utils` unmaintained; its upstream repository was archived on 2026-10-01. On `main`, `cargo audit` reports it through `futures-util` 0.3.31. `futures-util` 0.3.32 and later no longer depend on `pin-utils`. This runs `cargo update -p futures-util`, which: - moves `futures-core`, `futures-macro`, `futures-task` and `futures-util` from 0.3.31 to 0.3.34; - removes `pin-utils` from the lockfile. Only `Cargo.lock` changes. ## Path in `futures-util` reaches the build two ways: - through the GUI's accessibility chain: `eframe` → `egui-winit` → `accesskit_winit` → `accesskit_unix`; - through `js-sys`. ## Verification - `cargo audit` exits 0. Its warnings drop from 5 to 4. The remaining four are the upstream-blocked unmaintained advisories, W1 and W2 in `docs/UPSTREAM-DEPENDENCY-WATCH.adoc` in hyperpolymath/standards. - `cargo clippy --all-features --all-targets -- -D warnings` passes. - `cargo test --all-features` passes, including the 531-test main suite. - `futures-util` 0.3.34 declares `rust-version` 1.71, below this crate's 1.90. --- _Generated by [Claude Code](https://claude.ai/code/session_01Jma5ixtpeZyJRmfoSsC98u)_ Co-authored-by: Claude --- Cargo.lock | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index d9829da4..ea6b8dd6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1749,9 +1749,9 @@ dependencies = [ [[package]] name = "futures-core" -version = "0.3.31" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e" +checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" [[package]] name = "futures-io" @@ -1774,32 +1774,31 @@ dependencies = [ [[package]] name = "futures-macro" -version = "0.3.31" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650" +checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" dependencies = [ "proc-macro2", "quote", - "syn 2.0.117", + "syn 3.0.3", ] [[package]] name = "futures-task" -version = "0.3.31" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" [[package]] name = "futures-util" -version = "0.3.31" +version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" dependencies = [ "futures-core", "futures-macro", "futures-task", "pin-project-lite", - "pin-utils", "slab", ] @@ -3211,12 +3210,6 @@ version = "0.2.16" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b" -[[package]] -name = "pin-utils" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" - [[package]] name = "piper" version = "0.2.4" From 671c3a928644baf7de618ee1b452dcb5eb47d00f Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 9 Nov 2026 22:47:27 +0000 Subject: [PATCH 03/11] fix(assail): refuse an unwritable KYAML copy or sidecar before replacing the report (#274) ## Problem Codex raised this on hyperpolymath/panic-attack#264 after it merged. The `--output-format kyaml` preflight looks each output up, but lookup does not show that a path can be written. The run then replaces `report.json` and only afterwards fails on the copy, leaving the new report beside stale companions. Two inputs trigger this: - a read-only `report.kyaml` or `report.attestation.json`; - a directory the caller cannot create files in. The preflight is meant to be all-or-nothing, and this breaks it. ## Change - `kyaml_copy_path` now opens each output for writing during the preflight, without truncating it. An absent output is created and then removed again. - The kernel makes the decision, so ACLs and ownership count, not only mode bits. - The CHANGELOG `[Unreleased]` entry for the KYAML copy notes this. ## Validation Two new tests in `tests/coverage_php_tests.rs`: a read-only copy, and an attested run into an unwritable directory. Each checks that the run fails with "nothing was written" and that the old report is still there. - Run as `nobody`, both **fail without the fix** and pass with it. All 13 `kyaml_output_format` tests pass. - Root ignores permission bits, so as root the two tests detect that and skip. GitHub runners run unprivileged, so CI exercises them. - `cargo fmt --check` and `cargo clippy --all-targets -- -D warnings` are clean, and `cargo test` passes all 24 suites. Not covered here: the same problem in plain JSON mode with `--attest` (report plus sidecar), which has never had a preflight. That is on the follow-up list, and the planned generation-directory publish would close it for every mode. --- _Generated by [Claude Code](https://claude.ai/code/session_01PFaqv6iBXrt3W5Akd1Fq5k)_ Co-authored-by: Claude --- CHANGELOG.adoc | 4 +++ src/main.rs | 25 ++++++++++++++++ tests/coverage_php_tests.rs | 58 +++++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+) diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index 9169f489..59d763fb 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -12,6 +12,10 @@ may be the same file: a symbolic link at any of their paths, or two existing paths that are hard links to one file, is refused before anything is written. So is a path that exists but is not a regular file (a directory, say) or that cannot be looked up, such as a derived name over the filesystem's length limit. +Each output is also opened for writing before anything is written (an absent +one is created and removed again), so a read-only copy or sidecar, or a +directory the caller cannot create files in, is refused too, instead of +failing after the report has been replaced. These checks run before the writes, so a link swapped in between a check and its write is not caught, and a crash can leave a partial file: the planned generation-directory publish is meant to close both. diff --git a/src/main.rs b/src/main.rs index 1f1ee918..df68732b 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1443,6 +1443,31 @@ fn kyaml_copy_path(out: &Path, sidecar: Option<&Path>) -> Result { } } } + // Lookup alone does not show that a path can be written: a read-only + // copy or sidecar, or a directory the caller may not create files in, + // would fail only after the report had been replaced. Open each output + // for writing now, without truncating it; an absent one is created and + // removed again. The kernel decides, so ACLs and ownership count too. + for (&(role, candidate), &existed) in outputs.iter().zip(&exists) { + let probe = if existed { + fs::OpenOptions::new().write(true).open(candidate).map(drop) + } else { + fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(candidate) + .and_then(|file| { + drop(file); + fs::remove_file(candidate) + }) + }; + probe.map_err(|err| { + anyhow::anyhow!( + "cannot write the {role} {}: {err}; nothing was written", + candidate.display() + ) + })?; + } Ok(path) } diff --git a/tests/coverage_php_tests.rs b/tests/coverage_php_tests.rs index 14ba67aa..91032b0d 100644 --- a/tests/coverage_php_tests.rs +++ b/tests/coverage_php_tests.rs @@ -505,3 +505,61 @@ fn kyaml_output_format_refuses_a_directory_at_an_output_path() { "the report was written before the copy failed" ); } + +/// Root ignores permission bits, so the permission tests below only mean +/// something when run unprivileged. They return early otherwise. +#[cfg(unix)] +fn writes_ignore_permissions(dir: &std::path::Path) -> bool { + use std::os::unix::fs::PermissionsExt; + let probe = dir.join("privilege-probe"); + std::fs::write(&probe, "").unwrap(); + std::fs::set_permissions(&probe, std::fs::Permissions::from_mode(0o444)).unwrap(); + let privileged = std::fs::OpenOptions::new().write(true).open(&probe).is_ok(); + std::fs::remove_file(&probe).unwrap(); + privileged +} + +#[cfg(unix)] +#[test] +fn kyaml_output_format_refuses_a_read_only_copy_before_writing() { + use std::os::unix::fs::PermissionsExt; + let (dir, project) = project_dir(); + if writes_ignore_permissions(dir.path()) { + eprintln!("skipped: running with privileges that ignore permission bits"); + return; + } + let out = dir.path().join("report.json"); + std::fs::write(&out, "old report\n").unwrap(); + let copy = dir.path().join("report.kyaml"); + std::fs::write(©, "old copy\n").unwrap(); + std::fs::set_permissions(©, std::fs::Permissions::from_mode(0o444)).unwrap(); + let output = assail_kyaml(&project, &out); + assert!(!output.status.success()); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("nothing was written"), "{stderr}"); + assert_eq!(std::fs::read_to_string(&out).unwrap(), "old report\n"); + assert_eq!(std::fs::read_to_string(©).unwrap(), "old copy\n"); +} + +#[cfg(unix)] +#[test] +fn kyaml_output_format_refuses_a_directory_it_cannot_create_in() { + use std::os::unix::fs::PermissionsExt; + let (dir, project) = project_dir(); + if writes_ignore_permissions(dir.path()) { + eprintln!("skipped: running with privileges that ignore permission bits"); + return; + } + let reports = dir.path().join("reports"); + std::fs::create_dir(&reports).unwrap(); + let out = reports.join("report.json"); + std::fs::write(&out, "old report\n").unwrap(); + std::fs::set_permissions(&reports, std::fs::Permissions::from_mode(0o555)).unwrap(); + let output = assail_kyaml_attested(&project, &out); + std::fs::set_permissions(&reports, std::fs::Permissions::from_mode(0o755)).unwrap(); + assert!(!output.status.success()); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!(stderr.contains("nothing was written"), "{stderr}"); + assert_eq!(std::fs::read_to_string(&out).unwrap(), "old report\n"); + assert!(!reports.join("report.kyaml").exists()); +} From e12b823ea66439b3885b51bd6e3d741507c6be06 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 9 Nov 2026 22:50:55 +0000 Subject: [PATCH 04/11] fix(abi): type-check the Idris2 proofs in CI and re-prove line-comment stripping (#281) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Problem `PROOF-NEEDS.adoc` marks panic-attack's Idris2 ABI proofs ✅, but no CI job ever type-checked `src/abi/`. With Idris2 0.7.0, **none of the five modules checked as committed**: | Module | On main | |---|---| | all five | module name `PanicAttack.ABI.*` does not match the flat path `src/abi/*.idr` | | `Types.idr` | parse error at line 46; `Ord Severity` without `Eq Severity` | | `Stripping.idr` | stray `\|\|\|` block at line 222; **`stripLineCommentsIdempotent` does not type-check** (unsolved constraint in the `//` case; `decEq` not imported) | | `PatternCompleteness`, `ClassificationSoundness`, `AttestationUnforgeability` | check once staged at the right module path | ## Change - **`scripts/ci/check-idris-abi.sh`**: - stages a module-path view of the unchanged files as symlinks, so the bytes checked are the committed bytes; - runs `idris2 --check` on each module; - fails on any type error, a missing `%default total`, or `believe_me`, `assert_total`, `assert_smaller`, `postulate` or `idris_crash` in code. - **`proven-tests.yml`, new `idris-abi` job (blocking, every PR)**: - builds Idris2 from the `v0.7.0` tag, checking commit `27780073…`; - caches the build and runs the script; - records a PROOF-POLICY §6 deviation, because the Guix manifest and container lack Idris2. The workflow header explains why this departs from its "no Idris2 bootstrap on PRs" rule: a claim needs a receipt on every change, and the cache keeps it fast. - **`Stripping.idr`, re-proved:** - `stripIsIdOnClean`: stripping is the identity on text with no `//`. - `stripIsClean` / `bodyIsClean`: the strippers' output has no `//`, by mutual induction. - `stripLineCommentsIdempotent` and `bodyIsFixedPoint` follow. - The strippers are restated with explicit `Bool` steps, which can be reasoned about. - **`Types.idr`:** adds the missing `Eq Severity` and fixes the `LangId` declaration syntax. - **`PROOF-NEEDS.adoc`, `EXPLAINME.adoc`, `CHANGELOG.adoc`:** state that the earlier ✅ claims had no receipt, and name the new one. ## Validation (Idris2 0.7.0, built here from the same tag) - `scripts/ci/check-idris-abi.sh` passes all 5 modules. - The gate fails both ways: - on main's files: `Stripping` and `Types` FAIL; - on a planted `believe_me`: FAIL. - Stripping mutant (blank with `/` instead of a space): rejected at `bodyStepIsClean` and the sanity tests. - Differential run of the restated strippers against the original definitions: every string of length ≤ 9 over `{/, x, \n}` (29,524 inputs), **0 mismatches** and 0 non-idempotent outputs. - Nine closed sanity theorems check by evaluation. - `actionlint` 1.7.7 is clean, and `asciidoctor --failure-level=WARN` is clean on the edited `.adoc` files. ## Departure from the agreed plan The agreed option was "CI job + ipkg". I did not add an `.ipkg`, because one needs the files moved to `src/abi/PanicAttack/ABI/`. Their paths are cited in about 13 places, including legacy A2ML metadata that agents may not edit. The staging script gives the same check with no moves. Moving the files plus an `.ipkg` is the cleaner long-term layout and is a natural follow-up. --- _Generated by [Claude Code](https://claude.ai/code/session_01PFaqv6iBXrt3W5Akd1Fq5k)_ Co-authored-by: Claude --- .github/workflows/proven-tests.yml | 57 +++++ CHANGELOG.adoc | 18 ++ EXPLAINME.adoc | 2 +- PROOF-NEEDS.adoc | 39 +++- scripts/ci/check-idris-abi.sh | 68 ++++++ src/abi/Stripping.idr | 345 +++++++++++++++-------------- src/abi/Types.idr | 11 +- 7 files changed, 359 insertions(+), 181 deletions(-) create mode 100755 scripts/ci/check-idris-abi.sh diff --git a/.github/workflows/proven-tests.yml b/.github/workflows/proven-tests.yml index 82ab20c9..dcaa3e68 100644 --- a/.github/workflows/proven-tests.yml +++ b/.github/workflows/proven-tests.yml @@ -22,6 +22,12 @@ # and it would make every contributor wait on a compiler that cannot change # for them. # +# `idris-abi` (every PR, blocking): type-checks this repo's own Idris2 ABI +# proofs in src/abi/. This is a departure from the rule above, made because +# these proofs are claims in PROOF-NEEDS.adoc and a claim needs a receipt on +# every change. Idris2 is built from source once and cached by version and +# commit, so after the first run a PR waits only for the check itself. +# # WHY THE REPORT IS A FILE, NOT A PARAGRAPH: docs/INTEROP-PANIC-ATTACK.adoc in # proven-tests-and-benches fixes the mapping in prose; a report plus a verifier # is the same mapping with something at stake. @@ -191,3 +197,54 @@ jobs: path: .proven-tests/build/report/*.json retention-days: 30 if-no-files-found: warn + + # The Idris2 ABI proofs in src/abi/ (PA1, PA2, Layer 1.0 stripping, + # attestation unforgeability, ABI types). Until this job, none was ever + # type-checked. scripts/ci/check-idris-abi.sh fails on any type error, + # a missing `%default total`, or an escape hatch. + # + # Deviation (PROOF-POLICY §6), state: open. Leg: Idris2 ABI proofs. Tool: + # Idris2 0.7.0 built from the v0.7.0 tag (commit checked below) with the + # runner's Chez Scheme. Why not Guix or the container: panic-attack's Guix + # manifest and container do not provide Idris2 yet. + idris-abi: + name: Idris2 ABI proofs type-check + runs-on: ubuntu-latest + timeout-minutes: 45 + env: + IDRIS2_COMMIT: 27780073c8631826d846499840b3857d9b9a4fd5 + steps: + - name: Checkout this repository + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Cache the Idris2 toolchain built from source + id: idris2-cache + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.idris2-src-build + key: idris2-src-${{ env.IDRIS2_VERSION }}-${{ env.IDRIS2_COMMIT }}-${{ runner.os }} + + - name: Build Idris2 from the pinned tag (cache miss only) + if: steps.idris2-cache.outputs.cache-hit != 'true' + shell: bash + run: | + set -euo pipefail + sudo apt-get update -qq + sudo apt-get install -y -qq chezscheme + src=$(mktemp -d) + git clone --quiet --depth 1 --branch "v${IDRIS2_VERSION}" \ + https://github.com/idris-lang/Idris2.git "$src" + test "$(git -C "$src" rev-parse HEAD)" = "$IDRIS2_COMMIT" + make -C "$src" bootstrap SCHEME=scheme PREFIX="$HOME/.idris2-src-build" + make -C "$src" install PREFIX="$HOME/.idris2-src-build" + + - name: Type-check every module in src/abi + shell: bash + run: | + set -euo pipefail + # The cached build still needs Chez Scheme at run time. + command -v scheme >/dev/null || { sudo apt-get update -qq && sudo apt-get install -y -qq chezscheme; } + export PATH="$HOME/.idris2-src-build/bin:$PATH" + bash scripts/ci/check-idris-abi.sh diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index 59d763fb..6c93c735 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -2,6 +2,24 @@ === [Unreleased] +==== Fixed — the Idris2 ABI proofs are type-checked, and the stripping proof is re-proved + +* No CI job had ever type-checked `+src/abi/+`, and as committed none of its +five modules checked. A new `+idris-abi+` job (`+proven-tests.yml+`, using +`+scripts/ci/check-idris-abi.sh+`) now checks all five on every PR with Idris2 +0.7.0. It fails on a type error, on a missing `+%default total+`, or on +`+believe_me+`, `+assert_total+`, `+assert_smaller+`, `+postulate+` or +`+idris_crash+`. +* `+Stripping.idr+`: the 2026-06-02 proof of `+stripLineCommentsIdempotent+` +never type-checked, so it is re-proved through two facts: stripping is the +identity on text with no `+//+`, and the strippers' output has no `+//+`. The +strippers are restated with explicit `+Bool+` tests. An exhaustive run over +29,524 inputs shows they behave exactly as before. +* `+Types.idr+`: adds the missing `+Eq Severity+` that `+Ord+` needs, and fixes +the `+LangId+` declaration syntax. +* `+PROOF-NEEDS.adoc+` and `+EXPLAINME.adoc+` now say that the earlier claims had +no receipt. + ==== Added — `+assail --output-format kyaml+` writes a KYAML copy * With `+--output-format kyaml+` and `+--output report.json+`, `+assail+` writes diff --git a/EXPLAINME.adoc b/EXPLAINME.adoc index 8724d1ff..caf3293d 100644 --- a/EXPLAINME.adoc +++ b/EXPLAINME.adoc @@ -54,7 +54,7 @@ The README makes claims. This file backs them up. | `src/jit_context.rs` | JIT-framework classifier: Cranelift / Llvm / Wasm / Javascript / None; `transmute_targets_fn_ptr` recogniser for fn-ptr type-punning | `src/abi/PatternCompleteness.idr` | PA1 Idris2 proof — every Lang/Category has a detector | `src/abi/ClassificationSoundness.idr` | PA2 Idris2 proof — severity total order + maxSeverity commutativity -| `src/abi/Stripping.idr` | Layer 1.0 partial — line-comment-strip foundation (shape lemma + base cases of `stripLineCommentsIdempotent` Qed-closed) +| `src/abi/Stripping.idr` | Layer 1.0 — line-comment stripping is idempotent (`stripLineCommentsIdempotent`; re-proved 2026-11-09, type-checked by the `idris-abi` CI job) | `src/kanren/` | Logic engine (unification, fact database, taint, cross-lang) | `src/kanren/core.rs` | Term, substitution, unification, FactDB, forward chaining | `src/kanren/taint.rs` | Source→sink tracking (user input, network, deserialization) diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index e09653d0..c86680d9 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -2,12 +2,23 @@ === Current State +*Receipts (2026-11-09).* Until this date no CI job type-checked +`+src/abi/+`, and as committed none of the five modules checked: their module +names did not match their paths, `+Types.idr+` and `+Stripping.idr+` had syntax +errors, and `+Stripping.idr+`'s main theorem did not type-check at all. The ✅ +marks below therefore had no receipt before that date. Each module is now +checked on every PR by the `+idris-abi+` job in +`+.github/workflows/proven-tests.yml+` (`+scripts/ci/check-idris-abi.sh+`, Idris2 +0.7.0, `+%default total+`, no `+believe_me+`, `+assert_total+`, +`+assert_smaller+`, `+postulate+` or `+idris_crash+`). PROOF-POLICY §6 deviation, +state open: Idris2 is built from its v0.7.0 tag, because the repo's Guix +manifest and container do not provide it. + * **src/abi/*.idr**: 5 files — `+Types.idr+`, `+PatternCompleteness.idr+` (PA1 ✅ 2026-04-11), `+ClassificationSoundness.idr+` (PA2 ✅ 2026-04-11), `+Stripping.idr+` -(PROOF-PROGRAMME Layer 1.0 line-comment slice ✅ 2026-06-02 — -multi-comment semantics + slash-slash closure both Qed via -mutual-recursive `+bodyIsFixedPoint+`), +(PROOF-PROGRAMME Layer 1.0 line-comment slice: claimed 2026-06-02 but +never checked; re-proved and checked 2026-11-09), `+AttestationUnforgeability.idr+` (PROOF-PROGRAMME §3.2 ✅ 2026-06-04 — conditional unforgeability from Ed25519 EUF-CMA + chain-hash collision-resistance hypotheses) @@ -34,15 +45,19 @@ top-level theorem. preserves the ordering. |Layer 1.0 line-comment idempotence |`+src/abi/Stripping.idr+` -|*2026-06-02 close-out (issue #113)*: corrects PR #111’s single-comment -model to multi-comment via mutual recursion -(`+stripLineComments ↔ stripLineCommentBody+` — body calls back into -main after each preserved newline). Qed-closes -`+stripLineCommentsIdempotent+` for ALL cases including the slash-slash -inductive via the load-bearing `+bodyIsFixedPoint+` lemma proved by -mutual induction with the main theorem. 7 sanity-check theorems -including the two-comments-on-different-lines case PR #111 silently -mis-stripped. +|*Re-proved 2026-11-09.* The 2026-06-02 proof (issue #113) never +type-checked: its `+//+` case relied on reductions that do not hold for a +variable character, and it used `+decEq+` without importing it. The new proof +shows `+stripLineCommentsIdempotent+` (stripping twice equals stripping once) +through two facts: stripping is the identity on text with no `+//+` +(`+stripIsIdOnClean+`), and the strippers' output has no `+//+` +(`+stripIsClean+`/`+bodyIsClean+`, by mutual induction). `+bodyIsFixedPoint+` +follows. The strippers are restated with explicit `+Bool+` tests. An +exhaustive differential run over every string of length up to 9 on +`+{/, x, \n}+` (29,524 inputs) found no difference from the previous +definitions. A mutant that blanks with `+/+` instead of a space is rejected. +Nine closed sanity theorems, including two comments on different lines and +`+///+`. |Attestation chain unforgeability |`+src/abi/AttestationUnforgeability.idr+` |*PROOF-PROGRAMME §3.2 diff --git a/scripts/ci/check-idris-abi.sh b/scripts/ci/check-idris-abi.sh new file mode 100755 index 00000000..ac6641a4 --- /dev/null +++ b/scripts/ci/check-idris-abi.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell +# +# Type-check every Idris2 module in src/abi/ and fail loudly on any error or +# escape hatch. This is the receipt for the ABI proofs listed in +# PROOF-NEEDS.adoc: until this ran, none of them had ever been checked. +# +# The files declare `module PanicAttack.ABI.` but live flat in +# src/abi/, and Idris2 requires a module's path to match its name. Moving +# them would break paths cited in legacy A2ML metadata, which agents may not +# edit, so this script stages a module-path view of the unchanged files as +# symbolic links and checks that. The bytes checked are the committed bytes. +# +# Requirements per module: +# * `idris2 --check` succeeds under the pinned Idris2 version; +# * the file states `%default total`; +# * no believe_me, assert_total, assert_smaller, postulate or idris_crash +# outside comments. +# +# Usage: scripts/ci/check-idris-abi.sh (idris2 0.7.0 on PATH) +set -euo pipefail + +IDRIS2_VERSION=0.7.0 +root=$(cd "$(dirname "$0")/../.." && pwd) +abi="$root/src/abi" + +version=$(idris2 --version) +[[ "$version" == "Idris 2, version $IDRIS2_VERSION"* ]] || + { echo "check-idris-abi: need Idris2 $IDRIS2_VERSION, got: $version" >&2; exit 2; } + +stage=$(mktemp -d) +trap 'rm -rf "$stage"' EXIT +mkdir -p "$stage/PanicAttack/ABI" + +failures=0 +shopt -s nullglob +files=("$abi"/*.idr) +[[ ${#files[@]} -gt 0 ]] || { echo "check-idris-abi: no modules in $abi" >&2; exit 2; } + +for file in "${files[@]}"; do + name=$(basename "$file" .idr) + declared=$(sed -n 's/^module[[:space:]]\{1,\}\([A-Za-z0-9_.]*\).*/\1/p' "$file" | head -1) + if [[ "$declared" != "PanicAttack.ABI.$name" ]]; then + echo "FAIL $name: declares module '$declared', expected PanicAttack.ABI.$name" + failures=$((failures + 1)); continue + fi + ln -s "$file" "$stage/PanicAttack/ABI/$name.idr" +done + +for file in "${files[@]}"; do + name=$(basename "$file" .idr) + [[ -L "$stage/PanicAttack/ABI/$name.idr" ]] || continue + # Code lines only: drop `--` and `|||` comment lines before searching. + code=$(grep -vE '^[[:space:]]*(--|\|\|\|)' "$file" || true) + if ! grep -qE '^%default[[:space:]]+total' <<<"$code"; then + echo "FAIL $name: missing %default total"; failures=$((failures + 1)); continue + fi + if hatch=$(grep -nE '\b(believe_me|assert_total|assert_smaller|postulate|idris_crash)\b' <<<"$code"); then + echo "FAIL $name: escape hatch: $hatch"; failures=$((failures + 1)); continue + fi + if out=$(cd "$stage" && idris2 --no-color --check "PanicAttack/ABI/$name.idr" 2>&1); then + echo "PASS $name" + else + echo "FAIL $name"; printf '%s\n' "$out" | sed -n '1,40p'; failures=$((failures + 1)) + fi +done +exit "$failures" diff --git a/src/abi/Stripping.idr b/src/abi/Stripping.idr index 115fdeff..32bf438b 100644 --- a/src/abi/Stripping.idr +++ b/src/abi/Stripping.idr @@ -2,42 +2,36 @@ -- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell -- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) -||| Comment and String Stripping Idempotence (PROOF-PROGRAMME Layer 1.0) -||| -||| Closes the slash-slash inductive case left open by PR #111 (issue #113). +||| Line-comment stripping idempotence (PROOF-PROGRAMME Layer 1.0) ||| ||| Mechanises the foundation lemma for every Layer-1 per-category -||| soundness proof: the analyzer's comment-stripping pass is -||| **idempotent** — running it twice yields the same output as running -||| it once. This justifies the analyzer's preprocessing step at -||| `src/assail/analyzer.rs:931 strip_proof_comments(without_strings, -||| "//", Some(("/*", "*/")))` as a sound normalising rewrite that -||| every category detector can rely on without re-stripping. +||| soundness proof: the analyzer's line-comment stripping pass is +||| **idempotent**, so running it twice gives the same output as running +||| it once. That justifies the preprocessing step at +||| `src/assail/analyzer.rs` `strip_proof_comments(without_strings, "//", …)` +||| as a normalising rewrite that every category detector can rely on +||| without re-stripping. ||| -||| Differences from the PR #111 model: +||| The model strips ALL `//` comments: `stripLineCommentBody` hands back +||| to `stripLineComments` after each preserved newline (issue #113). ||| -||| 1. **Multi-comment semantics.** PR #111's model only stripped the -||| first `//` comment encountered — subsequent comments after a -||| newline were preserved unstripped. The Rust pass strips ALL -||| comments in the input. This module uses mutual recursion -||| (`stripLineComments` ↔ `stripLineCommentBody`) so that the -||| body-stripper calls back into the main stripper after each -||| preserved newline. +||| Re-proved 2026-11-09. The previous proof (PR #111, issue #113) never +||| type-checked: its `//` case relied on reductions that do not hold for a +||| variable character, and it used `decEq` without importing it. The proof +||| below goes through two facts instead: ||| -||| 2. **Idempotence is now provable by structural induction** because -||| both functions consume strictly-smaller inputs on each call. -||| Idris2's totality checker validates the recursion. +||| * `stripIsIdOnClean`: stripping is the identity on text with no `//`; +||| * `stripIsClean` / `bodyIsClean`: the strippers' output has no `//`. ||| -||| 3. **Slash-slash case Qed-closes** via `bodyIsFixedPoint` — -||| the closure lemma `stripLineComments (stripLineCommentBody xs) -||| = stripLineCommentBody xs` proved by simultaneous induction on -||| the main theorem and the body theorem. +||| Their composition is `stripLineCommentsIdempotent`. No `believe_me`, +||| `assert_total` or `postulate`; checked by Idris2 0.7.0 under +||| `%default total`. module PanicAttack.ABI.Stripping %default total -- ═══════════════════════════════════════════════════════════════════════ --- Helpers +-- Characters -- ═══════════════════════════════════════════════════════════════════════ ||| The space character used to overwrite stripped regions. @@ -45,201 +39,218 @@ public export sp : Char sp = ' ' -||| The newline character preserved during line-comment stripping (so -||| line numbers in the stripped view match line numbers in the -||| original — critical for the analyzer's location reporting). +||| The newline preserved during stripping, so line numbers in the +||| stripped view match the original (the analyzer reports locations). public export nl : Char nl = '\n' +||| Is this the comment-marker character `/`? +public export +isSlash : Char -> Bool +isSlash c = c == '/' + +||| Is this the newline that ends a line comment? +public export +isNl : Char -> Bool +isNl c = c == nl + +||| Does the list start with `/`? +public export +headIsSlash : List Char -> Bool +headIsSlash [] = False +headIsSlash (d :: _) = isSlash d + -- ═══════════════════════════════════════════════════════════════════════ --- Stripping functions (mirror src/assail/analyzer.rs:931 strip_proof_comments) +-- Stripping functions (mirror strip_proof_comments for "//") -- --- Mutual recursion: stripLineComments handles non-comment input and --- delegates comment bodies to stripLineCommentBody; stripLineCommentBody --- walks until the next newline and then returns control to the main --- stripper for the post-newline tail. +-- `stripLineComments` copies input until it meets `//`, which it replaces +-- with two spaces before handing the comment body to +-- `stripLineCommentBody`. That blanks every character up to the next +-- newline, keeps the newline, and hands the rest back. Each call works on +-- a strictly smaller suffix of the input. -- ═══════════════════════════════════════════════════════════════════════ mutual - ||| The main line-comment stripper. Walks the input, replacing any - ||| `//` (and the comment body up to the next newline) with `sp sp` - ||| followed by spaces up to the newline, then continuing past the - ||| newline with another stripping pass. + ||| The line-comment stripper. public export stripLineComments : List Char -> List Char stripLineComments [] = [] - stripLineComments ('/' :: '/' :: rest) = - sp :: sp :: stripLineCommentBody rest - stripLineComments (c :: rest) = c :: stripLineComments rest - - ||| `stripLineCommentBody` walks from JUST AFTER `//` until the - ||| next newline, replacing every character with `sp`. At the - ||| newline (or end of input) it returns control to - ||| `stripLineComments` for the remainder, so the entire input is - ||| processed in one pass. + stripLineComments (c :: tl) = stripStep (isSlash c) (headIsSlash tl) c tl + + ||| One step on `c` followed by `tl`. The flags say whether `c`, and + ||| the head of `tl`, are `/`: only both together start a comment. + public export + stripStep : Bool -> Bool -> Char -> List Char -> List Char + stripStep True True _ (_ :: rest) = sp :: sp :: stripLineCommentBody rest + stripStep _ _ c tl = c :: stripLineComments tl + + ||| The comment body, from just after `//` to the next newline. public export stripLineCommentBody : List Char -> List Char stripLineCommentBody [] = [] - stripLineCommentBody (c :: rest) = - if c == nl - then nl :: stripLineComments rest - else sp :: stripLineCommentBody rest + stripLineCommentBody (c :: rest) = bodyStep (isNl c) rest + + ||| One step of the body: keep the newline and hand back, or blank. + public export + bodyStep : Bool -> List Char -> List Char + bodyStep True rest = nl :: stripLineComments rest + bodyStep False rest = sp :: stripLineCommentBody rest + +-- ═══════════════════════════════════════════════════════════════════════ +-- "No `//`": text the stripper leaves unchanged +-- ═══════════════════════════════════════════════════════════════════════ + +||| `NoDoubleSlash xs`: no `/` in `xs` is immediately followed by `/`. +public export +data NoDoubleSlash : List Char -> Type where + NDSNil : NoDoubleSlash [] + NDSCons : (isSlash c = True -> headIsSlash tl = False) -> + NoDoubleSlash tl -> + NoDoubleSlash (c :: tl) + +spNeverSlash : isSlash Stripping.sp = True -> Void +spNeverSlash Refl impossible + +nlNeverSlash : isSlash Stripping.nl = True -> Void +nlNeverSlash Refl impossible + +-- ═══════════════════════════════════════════════════════════════════════ +-- Fact 1: stripping is the identity on text with no `//` +-- ═══════════════════════════════════════════════════════════════════════ + +||| The step agrees with consing when no comment starts at `c`. +stepIsCons : (b1, b2 : Bool) -> (c : Char) -> (tl : List Char) -> + (b1 = True -> b2 = False) -> + stripLineComments tl = tl -> + stripStep b1 b2 c tl = c :: tl +stepIsCons True True _ _ h _ = absurd (h Refl) +stepIsCons True False c tl _ ih = cong (c ::) ih +stepIsCons False _ c tl _ ih = cong (c ::) ih + +public export +stripIsIdOnClean : (xs : List Char) -> NoDoubleSlash xs -> stripLineComments xs = xs +stripIsIdOnClean [] NDSNil = Refl +stripIsIdOnClean (c :: tl) (NDSCons h rest) = + stepIsCons (isSlash c) (headIsSlash tl) c tl h (stripIsIdOnClean tl rest) -- ═══════════════════════════════════════════════════════════════════════ --- Idempotence — main theorem + body fixed-point lemma, mutual induction +-- Fact 2: the strippers' output has no `//` -- ═══════════════════════════════════════════════════════════════════════ +||| Stripping keeps a non-`/` head, so the output starts with no `/` either. +stripKeepsHead : (tl : List Char) -> headIsSlash tl = False -> + headIsSlash (stripLineComments tl) = False +stripKeepsHead [] _ = Refl +stripKeepsHead (d :: rest) pd = stepHead (isSlash d) (headIsSlash rest) d rest Refl pd + where + stepHead : (b1, b2 : Bool) -> (d : Char) -> (rest : List Char) -> + isSlash d = b1 -> b1 = False -> + headIsSlash (stripStep b1 b2 d rest) = False + stepHead False _ d rest pd' _ = pd' + stepHead True _ _ _ _ Refl impossible + mutual - ||| **Main theorem.** Applying the line-comment stripper twice - ||| equals applying it once. public export - stripLineCommentsIdempotent : (cs : List Char) -> - stripLineComments (stripLineComments cs) = stripLineComments cs - stripLineCommentsIdempotent [] = Refl - stripLineCommentsIdempotent ('/' :: '/' :: rest) = - -- LHS = strip (strip ('/' :: '/' :: rest)) - -- = strip (sp :: sp :: stripBody rest) [by definition] - -- = sp :: strip (sp :: stripBody rest) [sp /= '/'] - -- = sp :: sp :: strip (stripBody rest) [sp /= '/'] - -- = sp :: sp :: stripBody rest [by bodyIsFixedPoint] - -- RHS = sp :: sp :: stripBody rest [by definition] - cong (\xs => sp :: sp :: xs) (bodyIsFixedPoint rest) - stripLineCommentsIdempotent ('/' :: c :: rest) = - -- Not a comment marker (c is not the immediate-second '/'). - -- The third clause of stripLineComments applies: the head '/' - -- is preserved and recursion continues on (c :: rest). - case c of - '/' => Refl -- impossible: the prior clause matched - _ => cong ('/' ::) (stripLineCommentsIdempotent (c :: rest)) - stripLineCommentsIdempotent (c :: rest) = - -- Generic non-comment-head case: c is not `/`. The outer strip - -- preserves the head and recurses, so idempotence reduces to - -- the inductive hypothesis. - case c of - '/' => case rest of - [] => Refl - _ :: _ => Refl -- covered by the prior two clauses - _ => cong (c ::) (stripLineCommentsIdempotent rest) - - ||| **Body fixed-point lemma.** Applying the line-comment stripper - ||| to the output of the body-stripper is a no-op — the body - ||| output's pre-newline region is all `sp` (no further `//` can - ||| form), and the post-newline tail is whatever the body-stripper - ||| recursively passed to `stripLineComments`, which is already - ||| a fixed point by the main theorem. + stripIsClean : (xs : List Char) -> NoDoubleSlash (stripLineComments xs) + stripIsClean [] = NDSNil + stripIsClean (c :: tl) = + stepIsClean (isSlash c) (headIsSlash tl) c tl Refl Refl + + stepIsClean : (b1, b2 : Bool) -> (c : Char) -> (tl : List Char) -> + isSlash c = b1 -> headIsSlash tl = b2 -> + NoDoubleSlash (stripStep b1 b2 c tl) + stepIsClean True True _ (_ :: rest) _ _ = + NDSCons (\h => void (spNeverSlash h)) + (NDSCons (\h => void (spNeverSlash h)) (bodyIsClean rest)) + stepIsClean True True _ [] _ Refl impossible + stepIsClean True False c tl _ ptl = + NDSCons (\_ => stripKeepsHead tl ptl) (stripIsClean tl) + stepIsClean False _ c tl pc _ = + NDSCons (\h => absurd (trans (sym h) pc)) (stripIsClean tl) + public export - bodyIsFixedPoint : (cs : List Char) -> - stripLineComments (stripLineCommentBody cs) = stripLineCommentBody cs - bodyIsFixedPoint [] = Refl - bodyIsFixedPoint (c :: rest) = - -- Case split on whether c is the newline. - case decEq c nl of - Yes prf => - -- c = nl. Body returns `nl :: stripLineComments rest`. - -- strip applied to that = `nl :: strip (stripLineComments rest)` - -- (nl /= '/'). By main theorem on `rest`, that = - -- `nl :: stripLineComments rest`. - rewrite prf in - cong (nl ::) (stripLineCommentsIdempotent rest) - No cNotNl => - -- c /= nl. Body returns `sp :: stripLineCommentBody rest`. - -- strip applied to that = `sp :: strip (stripLineCommentBody rest)` - -- (sp /= '/'). By the body IH, that = `sp :: stripLineCommentBody rest`. - let - -- Establish that the body output starts with sp - -- under the c /= nl case. - bodyOutputShape : stripLineCommentBody (c :: rest) - = sp :: stripLineCommentBody rest - bodyOutputShape = case decEq c nl of - Yes y => absurd (cNotNl y) - No _ => Refl - in - rewrite bodyOutputShape in - cong (sp ::) (bodyIsFixedPoint rest) + bodyIsClean : (xs : List Char) -> NoDoubleSlash (stripLineCommentBody xs) + bodyIsClean [] = NDSNil + bodyIsClean (c :: rest) = bodyStepIsClean (isNl c) rest + + bodyStepIsClean : (b : Bool) -> (rest : List Char) -> + NoDoubleSlash (bodyStep b rest) + bodyStepIsClean True rest = + NDSCons (\h => void (nlNeverSlash h)) (stripIsClean rest) + bodyStepIsClean False rest = + NDSCons (\h => void (spNeverSlash h)) (bodyIsClean rest) -- ═══════════════════════════════════════════════════════════════════════ --- Sanity tests +-- Idempotence +-- ═══════════════════════════════════════════════════════════════════════ + +||| **Main theorem.** Stripping twice equals stripping once. +public export +stripLineCommentsIdempotent : (cs : List Char) -> + stripLineComments (stripLineComments cs) = stripLineComments cs +stripLineCommentsIdempotent cs = stripIsIdOnClean _ (stripIsClean cs) + +||| **Body fixed point.** Stripping the comment body's output changes nothing. +public export +bodyIsFixedPoint : (cs : List Char) -> + stripLineComments (stripLineCommentBody cs) = stripLineCommentBody cs +bodyIsFixedPoint cs = stripIsIdOnClean _ (bodyIsClean cs) + +-- ═══════════════════════════════════════════════════════════════════════ +-- Sanity tests (closed terms, checked by evaluation) -- ═══════════════════════════════════════════════════════════════════════ -||| Empty input: both sides reduce to []. private sanityEmpty : stripLineComments (stripLineComments []) = stripLineComments [] sanityEmpty = Refl -||| Single non-comment char: preserved through both passes. private sanityNonCommentSingleChar : - stripLineComments (stripLineComments ['x']) - = stripLineComments ['x'] + stripLineComments (stripLineComments ['x']) = stripLineComments ['x'] sanityNonCommentSingleChar = Refl -||| Bare `//`: produces `[sp, sp]` then `[sp, sp]` again. private sanityBareCommentMarker : - stripLineComments (stripLineComments ['/', '/']) - = stripLineComments ['/', '/'] + stripLineComments (stripLineComments ['/', '/']) = stripLineComments ['/', '/'] sanityBareCommentMarker = Refl -||| `x//`: char preserved, then sp+sp. private sanityCharPlusCommentMarker : stripLineComments (stripLineComments ['x', '/', '/']) = stripLineComments ['x', '/', '/'] sanityCharPlusCommentMarker = Refl -||| `//x`: comment body strips `x` to `sp`, total `[sp, sp, sp]`. Then -||| second pass is a no-op (no further `//` can form from spaces). +||| `//x` strips to three spaces. private -sanityCommentWithBody : - stripLineComments (stripLineComments ['/', '/', 'x']) - = stripLineComments ['/', '/', 'x'] +sanityCommentWithBody : stripLineComments ['/', '/', 'x'] = [' ', ' ', ' '] sanityCommentWithBody = Refl -||| `//x\\ny`: first pass produces `[sp, sp, sp, nl, y]`. Second pass -||| is the identity. The post-newline `y` is preserved because -||| stripLineCommentBody calls back into stripLineComments which leaves -||| non-comment chars alone. +||| `//x\ny`: the newline and the code after it survive. private sanityCommentThenCode : - stripLineComments (stripLineComments ['/', '/', 'x', '\n', 'y']) - = stripLineComments ['/', '/', 'x', '\n', 'y'] + stripLineComments ['/', '/', 'x', '\n', 'y'] = [' ', ' ', ' ', '\n', 'y'] sanityCommentThenCode = Refl -||| TWO comments separated by a newline. PR #111's broken single-comment -||| model would have left the second `//` unstripped; this corrected -||| mutual-recursion model strips both. +||| Two comments on different lines are both stripped (the case PR #111's +||| single-comment model got wrong). private sanityTwoComments : - stripLineComments (stripLineComments ['/', '/', 'a', '\n', '/', '/', 'b']) - = stripLineComments ['/', '/', 'a', '\n', '/', '/', 'b'] + stripLineComments ['/', '/', 'a', '\n', '/', '/', 'b'] + = [' ', ' ', ' ', '\n', ' ', ' ', ' '] sanityTwoComments = Refl --- ═══════════════════════════════════════════════════════════════════════ --- What still remains (Layer 1.0 future slices) --- ═══════════════════════════════════════════════════════════════════════ +||| A lone `/` is code, not a comment, and is kept. +private +sanityLoneSlash : stripLineComments ['a', '/', 'b'] = ['a', '/', 'b'] +sanityLoneSlash = Refl -||| Future Layer-1.0 work (recorded in PROOF-NEEDS.md + issue #114): -||| -||| 1. **Stripping_Block.idr** — `/* … */` block-comment stripper. Same -||| mutual-recursion shape. The closing `*/` plays the role newlines -||| played here. -||| -||| 2. **Stripping_Strings.idr** — `"…"` string-literal stripper with -||| `\"` escape handling. Same mutual-recursion shape. -||| -||| 3. **Stripping_Composition.idr** — `strip = stripLineComments . -||| stripBlockComments . stripStrings` is itself idempotent given -||| each component is. Requires cross-component lemmas (e.g. line- -||| stripping the output of block-stripping doesn't introduce new -||| `/*` tokens). -||| -||| 4. **Stripping_PositionPreservation.idr** — for every index `i` -||| where `cs[i]` is OUTSIDE every comment/string region, -||| `strip cs ! i = cs[i]`. Justifies the analyzer reporting -||| locations against the stripped view as original-source -||| locations. -||| -||| Once all five land, every Layer-1.1..1.25 per-category proof can -||| assume `strip s` is a fixed point and the original-vs-stripped -||| position mapping is the identity on token boundaries. +||| `///` is a comment whose body starts with `/`. +private +sanityTripleSlash : stripLineComments ['/', '/', '/'] = [' ', ' ', ' '] +sanityTripleSlash = Refl + +-- Future Layer-1.0 slices (PROOF-NEEDS, issue #114): block comments +-- (`/* … */`), string literals with escapes, the composition of the three +-- strippers, and position preservation (the stripped view has the same +-- length and newline positions as the source). diff --git a/src/abi/Types.idr b/src/abi/Types.idr index ee997b15..1afe5c0e 100644 --- a/src/abi/Types.idr +++ b/src/abi/Types.idr @@ -18,6 +18,15 @@ import Data.Fin public export data Severity = Info | Warning | Error | Critical +||| Equality on severity, required by `Ord`. +public export +Eq Severity where + Info == Info = True + Warning == Warning = True + Error == Error = True + Critical == Critical = True + _ == _ = False + ||| Total ordering on severity public export Ord Severity where @@ -43,7 +52,7 @@ data ScanOp ||| Language identifier (47 supported, bounded) public export -data LangId = MkLangId (n : Fin 47) +data LangId = MkLangId (Fin 47) -- ═══════════════════════════════════════════════════════════════════════ -- C ABI Exports From d4a68ede1a66e4fb091cf60e0d31db447186ceaf Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 9 Nov 2026 22:52:32 +0000 Subject: [PATCH 05/11] feat(publish): prototype generation-directory publish (not wired), with experiments and a model check (#275) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **Prototype, kept as a draft until the maintainer decides the open design questions.** Nothing calls it yet. ## Why On hyperpolymath/panic-attack#264 the maintainer chose to replace per-file output writes with a **generation-directory publish**. This prototype tests that design on evidence first. The publish is meant to close three gaps that exist in every output mode today: - **Check-then-write race:** a symbolic link swapped in between the preflight check and the write. - **Partial files after a crash.** - **Interleaving between concurrent runs.** ## What is here ### `src/publish.rs` For a report `D/S.json`, the layout is: - `D/S.gen//`: one complete generation per directory. - `D/S.gen/CURRENT`: a JCS manifest naming the current generation and the SHA-256 of each of its files. - Mirrors at `D/S.json` and the other output names: hard links into the current generation, with copies where linking fails. A publish runs in this order: 1. Write the generation into `.partial/`, syncing each file. 2. Rename it to ``. 3. Under a `File::lock`: - refuse if `CURRENT` names a newer generation (ids sort by run start); - otherwise replace `CURRENT` by temp file and rename; - refresh the mirrors; - collect old generations (default: keep 1 older). Readers take no lock. When the generation `CURRENT` named has been collected, they retry from `CURRENT`. ### `examples/publish_stress.rs` Compares three strategies: `inplace` (`fs::write`, as on main), `perfile` (rename each file) and `genset` (this prototype). It runs a multi-process race, kill -9 crash trials and a latency bench. ### `tests/publish_model.rs` An exhaustive explicit-state check of the protocol, reported as **TESTED** with these bounds: - up to 3 writers and 1 reader, 2 files; - a crash at every writer step; - garbage collection of crashed **and live** writers' partial generations; - 13,427 states at most. Results: - **P1–P3 hold:** `CURRENT` names only complete generations, it never moves backwards, and manifest readers never fail or return a partial set. - **P4 fails, as expected:** after a crash the mirrors can be torn. - **Three deliberate mutants are each caught**, showing that the invariants are not trivially true. ## Measured on Linux 6.18, ext4, 4 cores | | inplace | perfile | genset (mirrors) | genset (manifest) | |---|---|---|---|---| | race, 3 writers × 150 + readers: partial reads | 7931 / 8424 | 0 / 3209 | 0 / 6058 | **0 / 2500** | | race: torn sets | 182 | 3008 | 67 | **0** | | kill -9, 300 trials: partial / torn left behind | 176 / 13 | 0 / 155 | 0 / 3 | **0 / 0** | | 100 races: torn final state | 0 | 7 | 0 | **0** | | p50 latency, 1 KiB / 256 KiB / 4 MiB | 0.27 / 1.0 / 12.5 ms | 1.2 / 3.2 / 23.8 ms | 2.7 / 4.6 / 33.7 ms | | Defects the experiments found, each fixed here with a test that fails without the fix: - GC raced a writer renaming its own `.partial` directory, so a writer crashed with NotFound in 1 of 40 races. - A reader could lose a generation to GC: 1 in 2,500 reads. It now retries, and was 0 in about 11,200 reads afterwards. ## Open before this can be wired - **Mirrors torn after a crash** (P4). The options are to document it, to use one `current` symlink pointer (crash-consistent on Unix), or to drop the mirrors. - **Proof legs** under `PROOF-POLICY` §1.2 and §4: - a TLA+ spec checked by TLC (the policy's worked exception), which was not runnable in this session; - Creusot/Pearlite and Verus for the pure selection logic, still OPEN. - **Other questions:** - the generation id format (UUIDv7 or v8 per the estate ID policy); - Windows behaviour (no Windows run yet); - power-loss durability (fsync ordering is not tested; kill -9 tests process crashes only); - what a replacement keeps: ACLs, xattrs, labels and ownership. --- _Generated by [Claude Code](https://claude.ai/code/session_01PFaqv6iBXrt3W5Akd1Fq5k)_ --------- Co-authored-by: Claude --- .github/workflows/verify.yml | 29 + PROOF-NEEDS.adoc | 26 + docs/design/GENERATION-PUBLISH.adoc | 144 +++++ examples/publish_stress.rs | 298 ++++++++++ src/lib.rs | 1 + src/publish.rs | 510 ++++++++++++++++++ tests/publish_model.rs | 367 +++++++++++++ .../proofs/tlaplus/GenerationPublish.tla | 186 +++++++ verification/proofs/tlaplus/MirrorsCrash.cfg | 12 + .../proofs/tlaplus/MirrorsNoCrash.cfg | 12 + .../proofs/tlaplus/MutantCollectCurrent.cfg | 13 + .../proofs/tlaplus/MutantNoSupersede.cfg | 13 + .../tlaplus/MutantSwapBeforeComplete.cfg | 13 + verification/proofs/tlaplus/NoRetry.cfg | 12 + verification/proofs/tlaplus/Safety3Keep0.cfg | 13 + verification/proofs/tlaplus/Safety3Keep1.cfg | 13 + verification/proofs/tlaplus/check.sh | 55 ++ 17 files changed, 1717 insertions(+) create mode 100644 docs/design/GENERATION-PUBLISH.adoc create mode 100644 examples/publish_stress.rs create mode 100644 src/publish.rs create mode 100644 tests/publish_model.rs create mode 100644 verification/proofs/tlaplus/GenerationPublish.tla create mode 100644 verification/proofs/tlaplus/MirrorsCrash.cfg create mode 100644 verification/proofs/tlaplus/MirrorsNoCrash.cfg create mode 100644 verification/proofs/tlaplus/MutantCollectCurrent.cfg create mode 100644 verification/proofs/tlaplus/MutantNoSupersede.cfg create mode 100644 verification/proofs/tlaplus/MutantSwapBeforeComplete.cfg create mode 100644 verification/proofs/tlaplus/NoRetry.cfg create mode 100644 verification/proofs/tlaplus/Safety3Keep0.cfg create mode 100644 verification/proofs/tlaplus/Safety3Keep1.cfg create mode 100755 verification/proofs/tlaplus/check.sh diff --git a/.github/workflows/verify.yml b/.github/workflows/verify.yml index e133c955..54a38ccf 100644 --- a/.github/workflows/verify.yml +++ b/.github/workflows/verify.yml @@ -242,3 +242,32 @@ jobs: assert any(i.get('status') not in ('analyzed', 'excluded') for i in coverage['inputs']), coverage print(f"coverage={coverage['status']} (gap recorded)") PY + + # The generation-publish protocol (src/publish.rs) is model-checked with TLC, + # PROOF-POLICY §4's worked exception for concurrency protocols. Results are + # TESTED with the bounds in each .cfg, never PROVEN. Every config states its + # expected outcome; the expected violations (the documented mirror limit and + # three mutants) show the invariants can fail. + # + # Deviation (PROOF-POLICY §6), state: open. Leg: protocol model check. Tool: + # tla2tools.jar v1.7.4 (TLC 2.19), SHA-256 below, checked by check.sh. Why not + # Guix or the container: neither provides TLC yet. Java comes from the runner. + tlaplus: + name: TLA+ model check (GenerationPublish) + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Fetch the pinned TLC and model-check every config + shell: bash + run: | + set -euo pipefail + java -version + tools_dir=$(mktemp -d) + trap 'rm -rf "$tools_dir"' EXIT + curl --fail --silent --show-error --location --max-time 120 \ + https://github.com/tlaplus/tlaplus/releases/download/v1.7.4/tla2tools.jar \ + --output "$tools_dir/tla2tools.jar" + TLA2TOOLS="$tools_dir/tla2tools.jar" bash verification/proofs/tlaplus/check.sh diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index c86680d9..e351a613 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -81,11 +81,37 @@ obligation. The gateway octad projection is lossy by design, so this property also does not establish a faithful hexad↔octad map. |=== +=== Model-Checked (TESTED, bounded) + +[width="100%",cols="26%,30%,44%",options="header",] +|=== +|Obligation |Artefact |Status and bounds +|Generation-publish protocol P1–P3 (prototype, `src/publish.rs`; +design `docs/design/GENERATION-PUBLISH.adoc`) |`verification/proofs/tlaplus/GenerationPublish.tla` +(TLC 2.19, tla2tools v1.7.4, run by `verify.yml` job `tlaplus`) and +`tests/publish_model.rs` (independent explicit-state checker) |*TESTED*: up to +3 writers, 1 reader, 2 files, crash at every writer step, collection of crashed +and live writers' partial generations, `keep` 0 and 1. P1 (`CURRENT` names a +complete generation), P2 (never moves backwards) and P3 (manifest reader never +fails) hold. P4 (mirrors consistent once quiet) holds without crashes and +*fails* after a crash, the documented limit. Three mutants each violate P1 or +P2, so the invariants are not vacuous. *PROOF-POLICY §4 exception*: a +concurrency protocol over crash interleavings, which Creusot and Verus do not +state. *§6 deviation (open)*: TLC is fetched from its release, digest-checked, +because neither Guix nor the container provides it yet. +|=== + === What Still Needs Proving [width="100%",cols="51%,27%,22%",options="header",] |=== |Component |What |Why +|Generation publish: pure decisions (`src/publish.rs`) |*OPEN* (PROOF-POLICY +§1.2): Creusot/Pearlite and Verus contracts for the supersede comparison and +the collect selection (never removes the current or a newer generation; keeps +the `keep` newest older ones) |The model check covers the protocol, not the Rust +code; these tie the implementation to P1 and P2 + |*Layer 1.0 — stripBlockComments + Strings + Composition + Position-Preservation* (issue #114) |Block-comment (`+/* */+`) and string-literal (`+"..."+`) strippers with same mutual-recursive shape; diff --git a/docs/design/GENERATION-PUBLISH.adoc b/docs/design/GENERATION-PUBLISH.adoc new file mode 100644 index 00000000..c0e71d15 --- /dev/null +++ b/docs/design/GENERATION-PUBLISH.adoc @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell += Generation-directory publish +:toc: + +Status: *prototype* (`src/publish.rs`), not wired into any subcommand. + +== Why + +`assail` writes up to three files for one run: the JSON report, its KYAML +copy and the `--attest` sidecar. Writing them one at a time with `fs::write` +leaves three gaps: + +* a crash can leave a partial file; +* two runs into the same paths can interleave, leaving files from different + runs side by side; +* a symbolic link swapped in between the preflight check and the write is + written through. + +hyperpolymath/panic-attack#264 chose to close these for every output mode +with one design: publish each run as a *generation*, and move a single +pointer to it. + +== Layout + +For a report `D/S.json`: + +[cols="1,3"] +|=== +|`D/S.gen/.partial/` |A generation still being written. +|`D/S.gen//` |One complete generation: `S.json`, `S.kyaml`, `S.attestation.json`. +|`D/S.gen/CURRENT` |The pointer: a JCS manifest naming the current generation and the SHA-256 of each file in it. +|`D/S.gen/lock` |An advisory lock (`File::lock`) taken by publishers only. +|`D/S.json`, `D/S.kyaml`, … |Mirrors: hard links into the current generation (copies where linking fails). +|=== + +`` is 16 hex digits of Unix nanoseconds at the run's start followed by 16 +random hex digits, so ids sort by start time. + +== Protocol + +. *Write.* Create `.partial/`. Create each file new inside it and sync it. +. *Complete.* Sync the directory, rename it to ``, sync `S.gen`. +. *Publish, under the lock:* +.. if `CURRENT` names a newer generation, this run is *superseded*: remove + its generation and stop; +.. otherwise replace `CURRENT` (temporary file, sync, rename, sync `S.gen`); +.. refresh each mirror by hard link and rename; +.. collect: remove complete generations older than the current one, except + the newest `keep` (default 1), and partial generations older than the + grace period (default 1 hour). + +*Readers* take no lock. A reader that needs one consistent set reads +`CURRENT`, then the generation's files, and checks each against the +manifest's digest. If the generation was collected in between, it starts +again, but only if `CURRENT` has moved; otherwise the loss is real and it +reports an error. + +== What is guaranteed + +Checked by TLC (`verification/proofs/tlaplus/`) and, independently, by the +explicit-state checker in `tests/publish_model.rs`. Both are `TESTED` under +PROOF-POLICY §4, with the bounds in each configuration: up to 3 writers, +1 reader and 2 files, a crash at every writer step, and collection of the +partial generations of crashed and live writers. + +[cols="1,4,1"] +|=== +|Property |Statement |Result + +|P1 |`CURRENT` only ever names a complete generation. |holds +|P2 |`CURRENT` never moves to an older generation. |holds +|P3 |A manifest reader never fails or returns a mixed set. |holds +|P4 |When no writer is running, every mirror names the current generation. |holds without crashes; *fails after a crash* (see below) +|=== + +Each checker also runs three mutants of the protocol, each with one +defect. Each mutant violates P1 or P2, which shows the invariants are not +vacuous: + +* no supersede check; +* publish before completion; +* collect the current generation. + +Process crashes only: a SIGKILL stops the process and the kernel releases +its lock. Power loss, and the ordering of `fsync` against it, is not modelled +or tested. + +== The mirror limit (owner decision: hard links, documented) + +Mirrors are refreshed one at a time after `CURRENT` moves. A crash between +two refreshes leaves the mirrors from different generations: each file is +complete, but the set is mixed. The checkers find this (P4), and the crash +experiment measured it: 3 torn mirror sets in 300 kills, against none for +manifest readers. + +This is accepted and documented, because: + +* each mirror is always a complete file, never a partial one; +* the attestation sidecar seals the report's SHA-256, so + `attest verify --report` detects a mixed report and sidecar; +* the next successful publish repairs the mirrors; +* a reader that needs a consistent set reads through `CURRENT`. + +The alternative considered was a single `current` symlink with fixed +top-level symlinks through it. It is crash-consistent on Unix, but needs +symlinks (Windows falls back) and must exempt its own links from `assail`'s +symlink refusal. It was not chosen. + +== Measurements + +Measured on Linux 6.18, ext4, 4 cores, with `examples/publish_stress.rs`. +Three strategies are compared: `fs::write` in place, a rename per file, and +generation publish. + +[cols="2,1,1,1,1"] +|=== +| |in place |rename per file |generation, mirrors |generation, manifest + +|race, 3 writers × 150 runs: partial reads |7931 / 8424 |0 / 3209 |0 / 6058 |0 / 2500 +|race: mixed sets read |182 |3008 |67 |0 +|kill -9, 300 trials: partial / mixed left |176 / 13 |0 / 155 |0 / 3 |0 / 0 +|100 races: mixed final state |0 |7 |0 |0 +|p50 latency, 1 KiB / 256 KiB / 4 MiB |0.27 / 1.0 / 12.5 ms |1.2 / 3.2 / 23.8 ms |2.7 / 4.6 / 33.7 ms | +|=== + +The extra cost is mostly the directory syncs. Against a scan that takes +seconds, a few milliseconds is negligible. + +== Open before wiring + +* *Rust verification* (PROOF-POLICY §1.2): Creusot/Pearlite and Verus for + the pure decisions: the supersede comparison, and the collect selection + (never the current generation or a newer one). `OPEN`. +* *Generation id format*: align with the estate identifier policy (UUIDv7 or + v8), keeping the start-time ordering. +* *Windows*: there is no Windows run yet. `std::fs::rename` replaces + directories differently there, and a file that is open cannot be removed. +* *What a replacement keeps*: new files inherit the directory's default ACLs, + not the previous file's ACLs, xattrs, labels or owner. A private report + stays private because each file takes the mode of its predecessor in the + current generation. +* *Wiring*: `assail` (all output modes) and the preflight in + hyperpolymath/panic-attack#264 and hyperpolymath/panic-attack#274. diff --git a/examples/publish_stress.rs b/examples/publish_stress.rs new file mode 100644 index 00000000..be478c56 --- /dev/null +++ b/examples/publish_stress.rs @@ -0,0 +1,298 @@ +// SPDX-License-Identifier: MPL-2.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell + +//! Experiments for `docs/design/GENERATION-PUBLISH.adoc`. +//! +//! Three ways to write the report set `S.json`, `S.kyaml`, +//! `S.attestation.json` are compared: +//! +//! - `inplace`: `fs::write` each file (main today); +//! - `perfile`: temp file + rename for each file (the withdrawn +//! `atomic_write` approach); +//! - `genset`: `panic_attack::publish` generation directories. +//! +//! ```text +//! cargo run --release --example publish_stress -- race +//! cargo run --release --example publish_stress -- crash +//! cargo run --release --example publish_stress -- bench +//! ``` +//! +//! Every count printed is measured; none is estimated. + +use anyhow::{bail, Context, Result}; +use panic_attack::publish::{self, Generation, Options}; +use sha2::{Digest, Sha256}; +use std::fs; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +const NAMES: [&str; 3] = ["S.json", "S.kyaml", "S.attestation.json"]; +const PAD: usize = 256 * 1024; + +fn main() -> Result<()> { + let args: Vec = std::env::args().skip(1).collect(); + let a = |i: usize| args.get(i).map(String::as_str).unwrap_or(""); + let n = |i: usize| -> Result { + a(i).parse::() + .with_context(|| format!("argument {i} must be a number, got {:?}", a(i))) + }; + match a(0) { + "race" => race(a(1), n(2)?, n(3)?, n(4)?), + "crash" => crash(a(1), n(2)?), + "bench" => bench(a(1), usize::try_from(n(2)?)?, n(3)?), + "writer" => writer(a(1), Path::new(a(2)), a(3), n(4)?, PAD), + "reader" => reader(a(1), Path::new(a(2)), n(3)?), + _ => bail!("usage: see the module comment"), + } +} + +fn set_for(run: &str, pad: usize) -> [(String, Vec); 3] { + let json = format!("{{\"pad\":\"{}\",\"run\":\"{run}\"}}\n", "x".repeat(pad)); + let sha = hex::encode(Sha256::digest(json.as_bytes())); + let kyaml = format!( + "{{\n pad: \"{}\",\n run: \"{run}\",\n}}\n", + "x".repeat(pad) + ); + let sidecar = format!("{{\"report_sha256\":\"{sha}\",\"run\":\"{run}\"}}\n"); + [ + (NAMES[0].into(), json.into_bytes()), + (NAMES[1].into(), kyaml.into_bytes()), + (NAMES[2].into(), sidecar.into_bytes()), + ] +} + +fn write_set(mode: &str, dir: &Path, run: &str, pad: usize) -> Result<()> { + let files = set_for(run, pad); + match mode { + "inplace" => { + for (name, bytes) in &files { + fs::write(dir.join(name), bytes)?; + } + } + "perfile" => { + for (name, bytes) in &files { + let tmp = dir.join(format!(".{name}.{}.tmp", publish::new_id()?)); + let mut f = fs::File::create_new(&tmp)?; + std::io::Write::write_all(&mut f, bytes)?; + f.sync_all()?; + fs::rename(&tmp, dir.join(name))?; + } + } + "genset" => { + let mut g = Generation::begin(&dir.join("S.json"), publish::new_id()?)?; + for (name, bytes) in &files { + g.write(name, bytes)?; + } + g.publish(&Options::default())?; + } + other => bail!("unknown mode {other}"), + } + Ok(()) +} + +fn writer(mode: &str, dir: &Path, id: &str, iters: u64, pad: usize) -> Result<()> { + for i in 0..iters { + write_set(mode, dir, &format!("{id}-{i}"), pad)?; + } + Ok(()) +} + +#[derive(Default, Debug)] +struct Tally { + reads: u64, + partial: u64, + torn: u64, + missing: u64, +} + +/// Classify one set of three byte strings. +fn classify(files: &[Vec], t: &mut Tally) { + t.reads += 1; + let json: Option = serde_json::from_slice(&files[0]).ok(); + let side: Option = serde_json::from_slice(&files[2]).ok(); + let kyaml_ok = files[1].ends_with(b"}\n"); + let (Some(json), Some(side), true) = (json, side, kyaml_ok) else { + t.partial += 1; + return; + }; + let kyaml = String::from_utf8_lossy(&files[1]); + let run = json["run"].as_str().unwrap_or(""); + let sha = hex::encode(Sha256::digest(&files[0])); + if side["run"] != run + || side["report_sha256"] != sha.as_str() + || !kyaml.contains(&format!("run: \"{run}\"")) + { + t.torn += 1; + } +} + +fn read_once(mode: &str, dir: &Path, t: &mut Tally) { + if mode == "genset-consistent" { + match publish::read_consistent(&publish::root_for(&dir.join("S.json"))) { + Ok(Some((_, files))) => { + let v: Vec> = NAMES + .iter() + .map(|n| files.get(*n).cloned().unwrap_or_default()) + .collect(); + classify(&v, t) + } + Ok(None) => {} + Err(_) => t.missing += 1, + } + return; + } + let mut v = Vec::new(); + for name in NAMES { + match fs::read(dir.join(name)) { + Ok(b) => v.push(b), + Err(_) => { + t.missing += 1; + return; + } + } + } + classify(&v, t); +} + +fn reader(mode: &str, dir: &Path, millis: u64) -> Result<()> { + let mut t = Tally::default(); + let end = Instant::now() + Duration::from_millis(millis); + while Instant::now() < end && !dir.join(".done").exists() { + read_once(mode, dir, &mut t); + } + println!("{} {} {} {}", t.reads, t.partial, t.torn, t.missing); + Ok(()) +} + +fn me() -> Result { + Ok(std::env::current_exe()?) +} + +fn race(mode: &str, writers: u64, readers: u64, iters: u64) -> Result<()> { + let dir = tempfile::tempdir()?; + write_set(mode, dir.path(), "seed-0", PAD)?; + let start = Instant::now(); + let ws = (0..writers) + .map(|w| { + Command::new(me()?) + .args(["writer", mode]) + .arg(dir.path()) + .args([format!("w{w}"), iters.to_string()]) + .spawn() + .context("spawning a writer") + }) + .collect::>>()?; + // Readers of the plain paths, and for genset also manifest readers. + let mut read_modes = vec![mode.to_string()]; + if mode == "genset" { + read_modes.push("genset-consistent".into()); + } + let mut rs = Vec::new(); + for rm in &read_modes { + for _ in 0..readers { + let child = Command::new(me()?) + .args(["reader", rm]) + .arg(dir.path()) + .arg("120000") + .stdout(Stdio::piped()) + .spawn()?; + rs.push((rm.clone(), child)); + } + } + for mut w in ws { + let status = w.wait()?; + if !status.success() { + bail!("a writer failed: {status}"); + } + } + let wall = start.elapsed(); + fs::write(dir.path().join(".done"), b"")?; + for rm in &read_modes { + let mut sum = [0u64; 4]; + for (_, r) in rs.iter_mut().filter(|(m, _)| m == rm) { + let out = r.stdout.take().context("reader stdout")?; + let s = std::io::read_to_string(out)?; + for (slot, x) in sum.iter_mut().zip(s.split_whitespace()) { + *slot += x.parse::()?; + } + r.wait()?; + } + println!( + "race mode={mode} readers-of={rm} writers={writers} iters={iters} \ + writer_wall_ms={} reads={} partial={} torn={} missing={}", + wall.as_millis(), + sum[0], + sum[1], + sum[2], + sum[3] + ); + } + let mut t = Tally::default(); + read_once(mode, dir.path(), &mut t); + println!( + "race mode={mode} final-state partial={} torn={} missing={}", + t.partial, t.torn, t.missing + ); + Ok(()) +} + +fn crash(mode: &str, trials: u64) -> Result<()> { + let mut bad = Tally::default(); + let mut consistent_bad = 0u64; + for trial in 0..trials { + let dir = tempfile::tempdir()?; + write_set(mode, dir.path(), "seed-0", PAD)?; + let mut child = Command::new(me()?) + .args(["writer", mode]) + .arg(dir.path()) + .args([format!("c{trial}"), "1000".into()]) + .spawn()?; + // Kill somewhere inside the write loop: 1–40 ms in. + std::thread::sleep(Duration::from_micros(1000 + (trial * 7919) % 39000)); + child.kill()?; + child.wait()?; + read_once(mode, dir.path(), &mut bad); + if mode == "genset" { + match publish::read_consistent(&publish::root_for(&dir.path().join("S.json"))) { + Ok(Some(_)) => {} + _ => consistent_bad += 1, + } + } + } + println!( + "crash mode={mode} trials={trials} after-kill: partial={} torn={} missing={}{}", + bad.partial, + bad.torn, + bad.missing, + if mode == "genset" { + format!(" manifest-read-failures={consistent_bad}") + } else { + String::new() + } + ); + Ok(()) +} + +fn bench(mode: &str, bytes: usize, iters: u64) -> Result<()> { + let dir = tempfile::tempdir()?; + let mut samples = Vec::new(); + for i in 0..iters { + let t0 = Instant::now(); + write_set(mode, dir.path(), &format!("b-{i}"), bytes)?; + samples.push(t0.elapsed().as_micros()); + } + samples.sort_unstable(); + let pct = |p: usize| { + samples + .get(samples.len().saturating_sub(1) * p / 100) + .copied() + }; + println!( + "bench mode={mode} pad_bytes={bytes} iters={iters} p50_us={:?} p90_us={:?} max_us={:?}", + pct(50), + pct(90), + samples.last() + ); + Ok(()) +} diff --git a/src/lib.rs b/src/lib.rs index 67604985..5bfc4f04 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -40,6 +40,7 @@ pub mod kyaml; pub mod mass_panic; pub mod notify; pub mod panll; +pub mod publish; pub mod query; pub mod report; pub mod safe_path; diff --git a/src/publish.rs b/src/publish.rs new file mode 100644 index 00000000..7f6d4bfd --- /dev/null +++ b/src/publish.rs @@ -0,0 +1,510 @@ +// SPDX-License-Identifier: MPL-2.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell + +//! PROTOTYPE: publish a set of report files as one generation. +//! +//! Status: experimental, not wired into any subcommand. It exists to test +//! the design in `docs/design/GENERATION-PUBLISH.adoc` before anything uses +//! it. The design note records the open questions; the stress harness in +//! `examples/publish_stress.rs` holds the experiments. +//! +//! # Layout +//! +//! For a report named `S.json` in directory `D`: +//! +//! ```text +//! D/S.json, D/S.kyaml, ... mirrors: regular files, hard links into the +//! current generation (copies where links fail) +//! D/S.gen/lock advisory lock taken by publishers +//! D/S.gen/CURRENT JCS manifest naming the current generation +//! and the SHA-256 of each of its files +//! D/S.gen//S.json, ... one complete generation per directory +//! D/S.gen/.partial/ a generation still being written +//! ``` +//! +//! # Protocol +//! +//! 1. `begin`: create `.partial/`, `id` ordered by the run's start time. +//! 2. `write`: each file is created new inside it and synced. +//! 3. `publish`: sync the directory, rename it to `` (complete), then +//! under the lock: refuse if `CURRENT` names a newer generation, else +//! write `CURRENT` by temp-file + rename, sync `S.gen`, refresh the +//! mirrors by link + rename, and collect old generations. +//! +//! The invariants the protocol is meant to keep are stated in the design +//! note and model-checked with TLC in +//! `verification/proofs/tlaplus/GenerationPublish.tla` and, independently, by +//! the explicit-state checker in `tests/publish_model.rs`. + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use std::collections::BTreeMap; +use std::fs::{self, File, OpenOptions}; +use std::io::{self, Read, Write}; +use std::path::{Path, PathBuf}; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; + +/// Name of the pointer file inside the generation root. +pub const CURRENT: &str = "CURRENT"; +const LOCK: &str = "lock"; +const PARTIAL: &str = ".partial"; + +/// The `CURRENT` manifest: which generation is current, and what it holds. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct Manifest { + /// The generation's directory name, which orders generations. + pub generation: String, + /// File name → lowercase hex SHA-256 of its bytes. + pub files: BTreeMap, +} + +/// A current generation read whole: its manifest and each file's bytes. +pub type Snapshot = (Manifest, BTreeMap>); + +/// What `publish` did. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Outcome { + /// This generation is now current. + Published, + /// A newer generation was already current; this one was not published. + Superseded { current: String }, +} + +/// Options for `publish`. +#[derive(Debug, Clone)] +pub struct Options { + /// Complete generations kept besides the current one. + pub keep: usize, + /// Partial generations older than this are treated as abandoned. + pub partial_grace: Duration, + /// Refresh the top-level mirrors after moving the pointer. + pub mirror: bool, +} + +impl Default for Options { + fn default() -> Self { + Self { + keep: 1, + partial_grace: Duration::from_secs(3600), + mirror: true, + } + } +} + +/// A generation being written. +#[derive(Debug)] +pub struct Generation { + base_dir: PathBuf, + root: PathBuf, + id: String, + partial: PathBuf, + files: BTreeMap, +} + +/// The generation root for a report at `report`: `.gen` beside it. +pub fn root_for(report: &Path) -> PathBuf { + let stem = report.file_stem().unwrap_or_default().to_string_lossy(); + parent_dir(report).join(format!("{stem}.gen")) +} + +/// A new generation id: 16 hex digits of Unix nanoseconds at the run's +/// start, then 16 random hex digits. Ids sort by start time as strings. +pub fn new_id() -> io::Result { + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map_err(io::Error::other)? + .as_nanos() as u64; + let mut random = [0u8; 8]; + getrandom::fill(&mut random).map_err(io::Error::other)?; + Ok(format!("{nanos:016x}{}", hex::encode(random))) +} + +impl Generation { + /// Start a generation for the report at `report`, with id `id` (from + /// [`new_id`], taken when the run started). + pub fn begin(report: &Path, id: String) -> io::Result { + let base_dir = parent_dir(report).to_path_buf(); + let root = root_for(report); + fs::create_dir_all(&root)?; + let partial = root.join(format!("{id}{PARTIAL}")); + fs::create_dir(&partial)?; + Ok(Self { + base_dir, + root, + id, + partial, + files: BTreeMap::new(), + }) + } + + /// The generation id. + pub fn id(&self) -> &str { + &self.id + } + + /// Write one file of the generation. `name` is a bare file name. + pub fn write(&mut self, name: &str, bytes: &[u8]) -> io::Result<()> { + if name.is_empty() || name.contains(['/', '\\']) || name == "." || name == ".." { + return Err(io::Error::new(io::ErrorKind::InvalidInput, "bad file name")); + } + let path = self.partial.join(name); + let mut file = new_file(&path, self.mode_of_current(name))?; + file.write_all(bytes)?; + file.sync_all()?; + self.files + .insert(name.to_owned(), hex::encode(Sha256::digest(bytes))); + Ok(()) + } + + /// The permission bits of `name` in the current generation, so a report + /// made private stays private in the next one. + fn mode_of_current(&self, name: &str) -> Option { + let current = read_current(&self.root).ok()??; + mode_of(&self.root.join(¤t.generation).join(name)) + } + + /// Complete the generation and make it current unless a newer one is. + pub fn publish(self, options: &Options) -> io::Result { + sync_dir(&self.partial)?; + let complete = self.root.join(&self.id); + fs::rename(&self.partial, &complete)?; + sync_dir(&self.root)?; + + let lock = OpenOptions::new() + .create(true) + .truncate(false) + .write(true) + .open(self.root.join(LOCK))?; + lock.lock()?; + + let outcome: io::Result = (|| { + if let Some(current) = read_current(&self.root)? { + if current.generation > self.id { + return Ok(Outcome::Superseded { + current: current.generation, + }); + } + } + let manifest = Manifest { + generation: self.id.clone(), + files: self.files.clone(), + }; + let bytes = crate::json::to_canonical_string(&manifest).map_err(io::Error::other)?; + replace_file(&self.root, CURRENT, bytes.as_bytes())?; + if options.mirror { + for name in self.files.keys() { + mirror(&complete.join(name), &self.base_dir, name)?; + } + sync_dir(&self.base_dir)?; + } + Ok(Outcome::Published) + })(); + if let Ok(Outcome::Superseded { .. }) = &outcome { + let _ = fs::remove_dir_all(&complete); + } + let collected = collect(&self.root, options); + let _ = lock.unlock(); + let outcome = outcome?; + collected?; + Ok(outcome) + } +} + +/// Read the `CURRENT` manifest, or `None` before the first publish. +pub fn read_current(root: &Path) -> io::Result> { + match fs::read(root.join(CURRENT)) { + Ok(bytes) => serde_json::from_slice(&bytes) + .map(Some) + .map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e)), + Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(None), + Err(e) => Err(e), + } +} + +/// Read the whole current generation and check it against its manifest. +/// A reader that wants one consistent set of files uses this, not the +/// mirrors. +/// +/// Readers take no lock. A generation can be collected between reading +/// `CURRENT` and opening its files, but only after a newer one became +/// current, so on "not found" the read starts again from `CURRENT`. It +/// gives up when `CURRENT` has not moved, since then the loss is real. +pub fn read_consistent(root: &Path) -> io::Result> { + const ATTEMPTS: usize = 16; + let mut last_error = None; + for _ in 0..ATTEMPTS { + let Some(manifest) = read_current(root)? else { + return Ok(None); + }; + match read_generation(root, &manifest) { + Ok(files) => return Ok(Some((manifest, files))), + Err(e) if e.kind() == io::ErrorKind::NotFound => { + let moved = + read_current(root)?.map(|m| m.generation) != Some(manifest.generation.clone()); + if !moved { + return Err(e); + } + last_error = Some(e); + } + Err(e) => return Err(e), + } + } + Err(last_error.unwrap_or_else(|| io::Error::other("generation kept moving"))) +} + +fn read_generation(root: &Path, manifest: &Manifest) -> io::Result>> { + let dir = root.join(&manifest.generation); + let mut files = BTreeMap::new(); + for (name, digest) in &manifest.files { + let mut bytes = Vec::new(); + File::open(dir.join(name))?.read_to_end(&mut bytes)?; + if hex::encode(Sha256::digest(&bytes)) != *digest { + return Err(io::Error::new( + io::ErrorKind::InvalidData, + format!("{name} does not match the manifest"), + )); + } + files.insert(name.clone(), bytes); + } + Ok(files) +} + +/// Point `dir/name` at `target` by hard link + rename; copy + rename where +/// the filesystem refuses the link. +fn mirror(target: &Path, dir: &Path, name: &str) -> io::Result<()> { + let tmp = dir.join(format!(".{name}.{}.mirror", new_id()?)); + if fs::hard_link(target, &tmp).is_err() { + fs::copy(target, &tmp)?; + File::open(&tmp)?.sync_all()?; + } + fs::rename(&tmp, dir.join(name)).inspect_err(|_| { + let _ = fs::remove_file(&tmp); + }) +} + +/// Replace `dir/name` with `bytes` by temp file + rename, then sync `dir`. +fn replace_file(dir: &Path, name: &str, bytes: &[u8]) -> io::Result<()> { + let tmp = dir.join(format!(".{name}.{}.tmp", new_id()?)); + let result: io::Result<()> = (|| { + let mut file = new_file(&tmp, None)?; + file.write_all(bytes)?; + file.sync_all()?; + fs::rename(&tmp, dir.join(name)) + })(); + if result.is_err() { + let _ = fs::remove_file(&tmp); + } + result?; + sync_dir(dir) +} + +/// Remove complete generations beyond `keep` older ones, and partial ones +/// past the grace period. Never removes the current generation. Called +/// with the lock held, but other writers create and rename their own +/// generations without it, so an entry can vanish between listing and use: +/// that is treated as already gone, not as an error. +fn collect(root: &Path, options: &Options) -> io::Result<()> { + let current = read_current(root)?.map(|m| m.generation); + let mut complete = Vec::new(); + for entry in fs::read_dir(root)? { + let entry = entry?; + let Some(file_type) = unless_gone(entry.file_type())? else { + continue; + }; + if !file_type.is_dir() { + continue; + } + let name = entry.file_name().to_string_lossy().into_owned(); + if let Some(id) = name.strip_suffix(PARTIAL) { + let Some(meta) = unless_gone(entry.metadata())? else { + continue; + }; + let age = meta.modified()?.elapsed().unwrap_or(Duration::ZERO); + if age > options.partial_grace && Some(id) != current.as_deref() { + unless_gone(fs::remove_dir_all(entry.path()))?; + } + } else { + complete.push(name); + } + } + complete.sort(); + let Some(current) = current else { + return Ok(()); + }; + // Keep the current generation and the `keep` newest older than it. + // Newer complete generations belong to writers waiting for the lock and + // are left alone. An older one not yet published can only be superseded + // (the pointer never moves backwards), so removing it is safe. + let older: Vec<_> = complete.iter().filter(|g| **g < current).collect(); + let keep_from = older.len().saturating_sub(options.keep); + for (i, g) in older.iter().enumerate() { + if i < keep_from { + unless_gone(fs::remove_dir_all(root.join(g)))?; + } + } + Ok(()) +} + +/// `Ok(None)` for "not found", so a vanished entry is skipped. +fn unless_gone(result: io::Result) -> io::Result> { + match result { + Ok(value) => Ok(Some(value)), + Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(None), + Err(e) => Err(e), + } +} + +fn parent_dir(path: &Path) -> &Path { + match path.parent() { + Some(p) if !p.as_os_str().is_empty() => p, + _ => Path::new("."), + } +} + +fn new_file(path: &Path, mode: Option) -> io::Result { + let mut options = OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(mode.unwrap_or(0o666) & 0o777); + } + #[cfg(not(unix))] + let _ = mode; + options.open(path) +} + +fn mode_of(path: &Path) -> Option { + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let meta = fs::symlink_metadata(path).ok()?; + meta.is_file().then(|| meta.permissions().mode() & 0o777) + } + #[cfg(not(unix))] + { + let _ = path; + None + } +} + +fn sync_dir(dir: &Path) -> io::Result<()> { + #[cfg(unix)] + { + File::open(dir)?.sync_all() + } + #[cfg(not(unix))] + { + let _ = dir; + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn publish_set(report: &Path, tag: &str) -> Outcome { + let mut generation = Generation::begin(report, new_id().unwrap()).unwrap(); + generation + .write("S.json", format!("{{\"run\":\"{tag}\"}}\n").as_bytes()) + .unwrap(); + generation + .write("S.kyaml", format!("{{run: \"{tag}\"}}\n").as_bytes()) + .unwrap(); + generation.publish(&Options::default()).unwrap() + } + + #[test] + fn publishes_mirrors_and_manifest() { + let dir = tempfile::tempdir().unwrap(); + let report = dir.path().join("S.json"); + assert_eq!(publish_set(&report, "a"), Outcome::Published); + assert_eq!(fs::read_to_string(&report).unwrap(), "{\"run\":\"a\"}\n"); + let (manifest, files) = read_consistent(&root_for(&report)).unwrap().unwrap(); + assert_eq!(files.len(), 2); + assert_eq!(manifest.files.len(), 2); + assert!(fs::symlink_metadata(&report).unwrap().is_file()); + } + + #[test] + fn a_stale_generation_is_superseded() { + let dir = tempfile::tempdir().unwrap(); + let report = dir.path().join("S.json"); + let old_id = new_id().unwrap(); + std::thread::sleep(Duration::from_millis(2)); + publish_set(&report, "new"); + let mut stale = Generation::begin(&report, old_id).unwrap(); + stale.write("S.json", b"{\"run\":\"old\"}\n").unwrap(); + let outcome = stale.publish(&Options::default()).unwrap(); + assert!(matches!(outcome, Outcome::Superseded { .. })); + assert_eq!(fs::read_to_string(&report).unwrap(), "{\"run\":\"new\"}\n"); + } + + #[test] + fn keeps_current_and_one_older_generation() { + let dir = tempfile::tempdir().unwrap(); + let report = dir.path().join("S.json"); + for tag in ["a", "b", "c", "d"] { + publish_set(&report, tag); + } + let root = root_for(&report); + let gens = fs::read_dir(&root) + .unwrap() + .filter(|e| e.as_ref().unwrap().file_type().unwrap().is_dir()) + .count(); + assert_eq!(gens, 2); + } + + #[cfg(unix)] + #[test] + fn a_private_report_stays_private() { + use std::os::unix::fs::PermissionsExt; + let dir = tempfile::tempdir().unwrap(); + let report = dir.path().join("S.json"); + publish_set(&report, "a"); + fs::set_permissions(&report, fs::Permissions::from_mode(0o600)).unwrap(); + publish_set(&report, "b"); + let mode = fs::metadata(&report).unwrap().permissions().mode() & 0o777; + assert_eq!(mode, 0o600); + } + + #[test] + fn collect_skips_generations_renamed_while_it_lists() { + let dir = tempfile::tempdir().unwrap(); + let report = dir.path().join("S.json"); + publish_set(&report, "seed"); + let root = root_for(&report); + let options = Options { + partial_grace: Duration::ZERO, + ..Options::default() + }; + let stop = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); + let renamer = { + let (root, stop) = (root.clone(), stop.clone()); + std::thread::spawn(move || { + while !stop.load(std::sync::atomic::Ordering::Relaxed) { + let id = new_id().unwrap(); + let partial = root.join(format!("{id}{PARTIAL}")); + if fs::create_dir(&partial).is_ok() { + let _ = fs::rename(&partial, root.join(format!("{id}x"))); + let _ = fs::remove_dir(root.join(format!("{id}x"))); + } + } + }) + }; + let result = (0..20_000).try_for_each(|_| collect(&root, &options)); + stop.store(true, std::sync::atomic::Ordering::Relaxed); + renamer.join().unwrap(); + result.unwrap(); + } + + #[test] + fn rejects_path_names() { + let dir = tempfile::tempdir().unwrap(); + let mut generation = + Generation::begin(&dir.path().join("S.json"), new_id().unwrap()).unwrap(); + assert!(generation.write("../x", b"").is_err()); + assert!(generation.write("..", b"").is_err()); + } +} diff --git a/tests/publish_model.rs b/tests/publish_model.rs new file mode 100644 index 00000000..881b10d2 --- /dev/null +++ b/tests/publish_model.rs @@ -0,0 +1,367 @@ +// SPDX-License-Identifier: MPL-2.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell + +//! Exhaustive explicit-state check of the generation-publish protocol in +//! `src/publish.rs` (design: `docs/design/GENERATION-PUBLISH.adoc`). The same +//! model is checked with TLC in `verification/proofs/tlaplus/`; the two are +//! kept step for step alike so each cross-checks the other. +//! +//! This is an abstract model, not the code: each step below stands for one +//! filesystem operation the code performs, and every interleaving of up to +//! three writers and one reader is explored, with a crash (SIGKILL: the +//! process stops, the kernel drops its lock) allowed at every writer step. +//! It checks process crashes only. Power loss and fsync ordering are out of +//! scope here. +//! +//! Invariants checked in every reachable state: +//! - P1: `CURRENT` only ever names a complete generation. +//! - P2: `CURRENT` never moves to an older generation. +//! - P3: the manifest reader never returns a generation that is not +//! complete, and never fails while `CURRENT` names a readable one. +//! - P4 (expected to FAIL with crashes, see the design note): once no writer +//! is running, the top-level mirrors all name the current generation. + +use std::collections::{HashSet, VecDeque}; + +const FILES: usize = 2; + +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +enum Gen { + Absent, + Partial, + Complete, + Removed, +} + +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +enum Pc { + Start, + Writing, + Renamed, + Locked, + Swapped, + Mirrored(u8), + Superseded, + Collected, + Done, + Crashed, + /// Its partial generation was collected under it, so its next write or + /// rename failed with "not found" and it exited with an error. + Failed, +} + +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +enum Reader { + Idle, + Got(u8), + Missed(u8), + Ok(u8), + Failed, +} + +#[derive(Clone, PartialEq, Eq, Hash, Debug)] +struct State { + gens: Vec, + pc: Vec, + current: Option, + lock: Option, + mirrors: [Option; FILES], + reader: Reader, + retries: u8, +} + +struct Config { + writers: u8, + keep: usize, + crashes: bool, + reader_retries: bool, + /// A deliberate defect, to show the invariants can fail (see the + /// `mutant_*` tests). `None` is the protocol as implemented. + mutant: Option, +} + +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Mutant { + /// Move `CURRENT` without comparing generation ids. + NoSupersedeCheck, + /// Move `CURRENT` as soon as the partial generation is created. + SwapBeforeComplete, + /// Collect the current generation along with the older ones. + CollectCurrent, +} + +/// Writer `i` has generation id `i`: ids order generations by start time, +/// and the interleavings decide who finishes first. +fn successors(s: &State, c: &Config) -> Vec { + let mut out = Vec::new(); + for i in 0..c.writers { + let w = i as usize; + let mut n = s.clone(); + let step = match s.pc[w] { + Pc::Start => { + n.gens[w] = Gen::Partial; + if c.mutant == Some(Mutant::SwapBeforeComplete) { + // Publish the generation while it is still partial. + n.current = Some(i); + } + n.pc[w] = Pc::Writing; + true + } + Pc::Writing => { + if s.gens[w] == Gen::Removed { + n.pc[w] = Pc::Failed; + } else { + n.gens[w] = Gen::Complete; + n.pc[w] = Pc::Renamed; + } + true + } + Pc::Renamed if s.lock.is_none() => { + n.lock = Some(i); + n.pc[w] = Pc::Locked; + true + } + Pc::Locked => { + if c.mutant != Some(Mutant::NoSupersedeCheck) && s.current.is_some_and(|g| g > i) { + n.gens[w] = Gen::Removed; + n.pc[w] = Pc::Superseded; + } else { + n.current = Some(i); + n.pc[w] = Pc::Swapped; + } + true + } + Pc::Swapped => { + n.mirrors[0] = Some(i); + n.pc[w] = Pc::Mirrored(1); + true + } + Pc::Mirrored(k) if (k as usize) < FILES => { + n.mirrors[k as usize] = Some(i); + n.pc[w] = Pc::Mirrored(k + 1); + true + } + Pc::Mirrored(_) | Pc::Superseded => { + collect(&mut n, c.keep, c.mutant == Some(Mutant::CollectCurrent)); + n.pc[w] = Pc::Collected; + true + } + Pc::Collected => { + n.lock = None; + n.pc[w] = Pc::Done; + true + } + _ => false, + }; + if step { + out.push(n); + } + if c.crashes && !matches!(s.pc[w], Pc::Done | Pc::Crashed | Pc::Failed) { + let mut n = s.clone(); + n.pc[w] = Pc::Crashed; + if n.lock == Some(i) { + n.lock = None; + } + out.push(n); + } + // Grace-period removal of a partial generation. `collect` cannot tell + // a crashed writer from one that has stalled past the grace period, so + // a live writer's partial generation can be removed too. + if s.gens[w] == Gen::Partial { + let mut n = s.clone(); + n.gens[w] = Gen::Removed; + out.push(n); + } + } + // Reader. + let mut n = s.clone(); + let reader_step = match s.reader { + Reader::Idle => match s.current { + Some(g) => { + n.reader = Reader::Got(g); + true + } + None => false, + }, + Reader::Got(g) => { + n.reader = if s.gens[g as usize] == Gen::Complete { + Reader::Ok(g) + } else { + Reader::Missed(g) + }; + true + } + Reader::Missed(g) => { + n.reader = if c.reader_retries && s.current != Some(g) && s.retries < 3 { + n.retries += 1; + Reader::Idle + } else { + Reader::Failed + }; + true + } + Reader::Ok(_) | Reader::Failed => false, + }; + if reader_step { + out.push(n); + } + out +} + +fn collect(s: &mut State, keep: usize, also_current: bool) { + if also_current { + if let Some(g) = s.current { + s.gens[g as usize] = Gen::Removed; + } + } + let Some(current) = s.current else { return }; + let older: Vec = (0..s.gens.len()) + .filter(|&g| (g as u8) < current && s.gens[g] == Gen::Complete) + .collect(); + let keep_from = older.len().saturating_sub(keep); + for &g in &older[..keep_from] { + s.gens[g] = Gen::Removed; + } +} + +#[derive(Debug, Default)] +struct Report { + states: usize, + p1: Option, + p2: Option<(State, State)>, + p3: Option, + p4: Option, +} + +fn explore(c: &Config) -> Report { + let w = c.writers as usize; + let init = State { + gens: vec![Gen::Absent; w], + pc: vec![Pc::Start; w], + current: None, + lock: None, + mirrors: [None; FILES], + reader: Reader::Idle, + retries: 0, + }; + let mut seen = HashSet::new(); + let mut queue = VecDeque::from([init.clone()]); + seen.insert(init); + let mut r = Report::default(); + while let Some(s) = queue.pop_front() { + r.states += 1; + if let Some(g) = s.current { + if s.gens[g as usize] != Gen::Complete && r.p1.is_none() { + r.p1 = Some(s.clone()); + } + } + if s.reader == Reader::Failed && r.p3.is_none() { + r.p3 = Some(s.clone()); + } + let quiet = s.lock.is_none() + && s.pc + .iter() + .all(|p| matches!(p, Pc::Done | Pc::Crashed | Pc::Failed | Pc::Start)); + if quiet + && s.current.is_some() + && s.mirrors.iter().any(|m| *m != s.current) + && r.p4.is_none() + { + r.p4 = Some(s.clone()); + } + for n in successors(&s, c) { + if let (Some(a), Some(b)) = (s.current, n.current) { + if b < a && r.p2.is_none() { + r.p2 = Some((s.clone(), n.clone())); + } + } + if seen.insert(n.clone()) { + queue.push_back(n); + } + } + } + r +} + +#[test] +fn p1_p2_p3_hold_for_every_interleaving_with_crashes() { + for keep in [0, 1] { + for writers in [2, 3] { + let r = explore(&Config { + writers, + keep, + crashes: true, + reader_retries: true, + mutant: None, + }); + eprintln!("writers={writers} keep={keep} states={}", r.states); + assert!(r.p1.is_none(), "P1 violated: {:?}", r.p1); + assert!(r.p2.is_none(), "P2 violated: {:?}", r.p2); + assert!(r.p3.is_none(), "P3 violated: {:?}", r.p3); + } + } +} + +#[test] +fn without_the_retry_a_reader_can_fail() { + let r = explore(&Config { + writers: 2, + keep: 0, + crashes: false, + reader_retries: false, + mutant: None, + }); + assert!(r.p3.is_some(), "expected a P3 counterexample without retry"); +} + +#[test] +fn mirrors_are_consistent_when_quiet_without_crashes() { + let r = explore(&Config { + writers: 3, + keep: 1, + crashes: false, + reader_retries: true, + mutant: None, + }); + assert!(r.p4.is_none(), "P4 violated: {:?}", r.p4); +} + +#[test] +fn a_crash_can_leave_the_mirrors_torn() { + let r = explore(&Config { + writers: 2, + keep: 1, + crashes: true, + reader_retries: true, + mutant: None, + }); + let s = r.p4.expect("expected the known P4 counterexample"); + eprintln!("P4 counterexample: {s:?}"); +} + +/// Each mutant breaks one step of the protocol. The checker must find a +/// counterexample for each, or the invariants above would prove nothing. +fn mutant(m: Mutant) -> Report { + explore(&Config { + writers: 2, + keep: 1, + crashes: true, + reader_retries: true, + mutant: Some(m), + }) +} + +#[test] +fn mutant_without_the_supersede_check_moves_current_backwards() { + assert!(mutant(Mutant::NoSupersedeCheck).p2.is_some()); +} + +#[test] +fn mutant_swapping_before_completion_points_at_a_partial_generation() { + assert!(mutant(Mutant::SwapBeforeComplete).p1.is_some()); +} + +#[test] +fn mutant_collecting_the_current_generation_breaks_the_pointer() { + assert!(mutant(Mutant::CollectCurrent).p1.is_some()); +} diff --git a/verification/proofs/tlaplus/GenerationPublish.tla b/verification/proofs/tlaplus/GenerationPublish.tla new file mode 100644 index 00000000..f8a46d16 --- /dev/null +++ b/verification/proofs/tlaplus/GenerationPublish.tla @@ -0,0 +1,186 @@ +---------------------------- MODULE GenerationPublish ---------------------------- +\* SPDX-License-Identifier: MPL-2.0 +\* SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell +\* +\* The generation-publish protocol of src/publish.rs, for TLC. +\* +\* PROOF-POLICY section 4 exception: TLA+ model checking of a concurrency +\* protocol, the policy's worked example. The obligation is a property of +\* interleavings of several processes and of crashes between file-system +\* operations; neither Creusot nor Verus states that. A finite check is +\* reported as TESTED with the bounds in the .cfg files, never as PROVEN. +\* tests/publish_model.rs checks the same model independently in Rust. +\* +\* Each step is one file-system operation of the code. Writer w's generation +\* id is w, so ids order generations by start time and the interleavings +\* decide who finishes first. A crash is SIGKILL: the process stops and the +\* kernel releases its lock. Power loss and fsync ordering are out of scope. + +EXTENDS Naturals, FiniteSets + +CONSTANTS + Writers, \* writer ids, e.g. 1..3 + NFiles, \* files per generation, mirrored one at a time + Keep, \* complete generations kept besides the current one + Crashes, \* TRUE: a writer may crash at any step + Retry, \* TRUE: the reader retries when its generation vanished + MaxRetries, + Mutant \* "none", or one deliberate defect (see the Mutant* configs) + +VARIABLES gen, pc, mi, current, lock, mirror, rpc, rgen, retries + +vars == <> + +None == 0 +Files == 1..NFiles + +TypeOK == + /\ gen \in [Writers -> {"absent", "partial", "complete", "removed"}] + /\ pc \in [Writers -> {"start", "writing", "renamed", "locked", "mirroring", + "superseded", "collected", "done", "crashed", "failed"}] + /\ mi \in [Writers -> 0..NFiles] + /\ current \in Writers \cup {None} + /\ lock \in Writers \cup {None} + /\ mirror \in [Files -> Writers \cup {None}] + /\ rpc \in {"idle", "got", "missed", "ok", "failed"} + /\ rgen \in Writers \cup {None} + /\ retries \in 0..MaxRetries + +Init == + /\ gen = [w \in Writers |-> "absent"] + /\ pc = [w \in Writers |-> "start"] + /\ mi = [w \in Writers |-> 0] + /\ current = None + /\ lock = None + /\ mirror = [f \in Files |-> None] + /\ rpc = "idle" + /\ rgen = None + /\ retries = 0 + +\* Collect: remove complete generations older than the current one, except +\* the Keep newest of them. +Collected(g, cur) == + LET older == {h \in Writers : h < cur /\ g[h] = "complete"} + doomed == {h \in older : Cardinality({k \in older : k > h}) >= Keep} + IN [h \in Writers |-> + IF h \in doomed \/ (Mutant = "collectcurrent" /\ h = cur) + THEN "removed" ELSE g[h]] + +Begin(w) == + /\ pc[w] = "start" + /\ gen' = [gen EXCEPT ![w] = "partial"] + /\ pc' = [pc EXCEPT ![w] = "writing"] + \* Mutant: publish the generation while it is still partial. + /\ current' = IF Mutant = "swapbeforecomplete" THEN w ELSE current + /\ UNCHANGED <> + +\* Rename .partial to . Fails if collect removed the partial +\* generation under a writer that stalled past the grace period. +Rename(w) == + /\ pc[w] = "writing" + /\ IF gen[w] = "removed" + THEN /\ pc' = [pc EXCEPT ![w] = "failed"] + /\ UNCHANGED gen + ELSE /\ gen' = [gen EXCEPT ![w] = "complete"] + /\ pc' = [pc EXCEPT ![w] = "renamed"] + /\ UNCHANGED <> + +Acquire(w) == + /\ pc[w] = "renamed" + /\ lock = None + /\ lock' = w + /\ pc' = [pc EXCEPT ![w] = "locked"] + /\ UNCHANGED <> + +\* Under the lock: refuse if a newer generation is current, else move CURRENT. +Swap(w) == + /\ pc[w] = "locked" + /\ IF Mutant /= "nosupersede" /\ current /= None /\ current > w + THEN /\ gen' = [gen EXCEPT ![w] = "removed"] + /\ pc' = [pc EXCEPT ![w] = "superseded"] + /\ UNCHANGED current + ELSE /\ current' = w + /\ pc' = [pc EXCEPT ![w] = "mirroring"] + /\ UNCHANGED gen + /\ UNCHANGED <> + +MirrorStep(w) == + /\ pc[w] = "mirroring" + /\ mi[w] < NFiles + /\ mirror' = [mirror EXCEPT ![mi[w] + 1] = w] + /\ mi' = [mi EXCEPT ![w] = mi[w] + 1] + /\ UNCHANGED <> + +Collect(w) == + /\ \/ pc[w] = "mirroring" /\ mi[w] = NFiles + \/ pc[w] = "superseded" + /\ gen' = IF current = None THEN gen ELSE Collected(gen, current) + /\ pc' = [pc EXCEPT ![w] = "collected"] + /\ UNCHANGED <> + +Release(w) == + /\ pc[w] = "collected" + /\ lock' = None + /\ pc' = [pc EXCEPT ![w] = "done"] + /\ UNCHANGED <> + +Crash(w) == + /\ Crashes + /\ pc[w] \notin {"done", "crashed", "failed"} + /\ pc' = [pc EXCEPT ![w] = "crashed"] + /\ lock' = IF lock = w THEN None ELSE lock + /\ UNCHANGED <> + +\* Grace-period removal of a partial generation, crashed or merely stalled. +GcPartial(w) == + /\ gen[w] = "partial" + /\ gen' = [gen EXCEPT ![w] = "removed"] + /\ UNCHANGED <> + +\* The lock-free manifest reader: read CURRENT, open its generation, and on +\* "not found" start again only if CURRENT has moved. +ReadPointer == + /\ rpc = "idle" + /\ current /= None + /\ rpc' = "got" + /\ rgen' = current + /\ UNCHANGED <> + +OpenGeneration == + /\ rpc = "got" + /\ rpc' = IF gen[rgen] = "complete" THEN "ok" ELSE "missed" + /\ UNCHANGED <> + +Recheck == + /\ rpc = "missed" + /\ IF Retry /\ current /= rgen /\ retries < MaxRetries + THEN /\ rpc' = "idle" + /\ retries' = retries + 1 + ELSE /\ rpc' = "failed" + /\ UNCHANGED retries + /\ UNCHANGED <> + +Next == + \/ \E w \in Writers : + Begin(w) \/ Rename(w) \/ Acquire(w) \/ Swap(w) \/ MirrorStep(w) + \/ Collect(w) \/ Release(w) \/ Crash(w) \/ GcPartial(w) + \/ ReadPointer \/ OpenGeneration \/ Recheck + +Spec == Init /\ [][Next]_vars + +----------------------------------------------------------------------------- +\* P1: CURRENT only ever names a complete generation. +CurrentIsComplete == current /= None => gen[current] = "complete" + +\* P2: CURRENT never moves to an older generation (an action property). +CurrentMonotone == [][current /= None => current' /= None /\ current' >= current]_vars + +\* P3: the manifest reader never fails. +ReaderNeverFails == rpc /= "failed" + +\* P4: once no writer runs, every mirror names the current generation. +\* Expected to FAIL when Crashes is TRUE (the documented mirror limit). +Quiet == lock = None /\ \A w \in Writers : pc[w] \in {"start", "done", "crashed", "failed"} +MirrorsConsistentWhenQuiet == + (Quiet /\ current /= None) => \A f \in Files : mirror[f] = current +============================================================================= diff --git a/verification/proofs/tlaplus/MirrorsCrash.cfg b/verification/proofs/tlaplus/MirrorsCrash.cfg new file mode 100644 index 00000000..98352fa8 --- /dev/null +++ b/verification/proofs/tlaplus/MirrorsCrash.cfg @@ -0,0 +1,12 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: violation of MirrorsConsistentWhenQuiet (documented mirror limit) +CONSTANTS + Writers = {1, 2} + NFiles = 2 + Keep = 1 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "none" +SPECIFICATION Spec +INVARIANTS MirrorsConsistentWhenQuiet diff --git a/verification/proofs/tlaplus/MirrorsNoCrash.cfg b/verification/proofs/tlaplus/MirrorsNoCrash.cfg new file mode 100644 index 00000000..0bf8ac4d --- /dev/null +++ b/verification/proofs/tlaplus/MirrorsNoCrash.cfg @@ -0,0 +1,12 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: pass +CONSTANTS + Writers = {1, 2, 3} + NFiles = 2 + Keep = 1 + Crashes = FALSE + Retry = TRUE + MaxRetries = 3 + Mutant = "none" +SPECIFICATION Spec +INVARIANTS TypeOK MirrorsConsistentWhenQuiet diff --git a/verification/proofs/tlaplus/MutantCollectCurrent.cfg b/verification/proofs/tlaplus/MutantCollectCurrent.cfg new file mode 100644 index 00000000..6b556f41 --- /dev/null +++ b/verification/proofs/tlaplus/MutantCollectCurrent.cfg @@ -0,0 +1,13 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: violation of CurrentIsComplete +CONSTANTS + Writers = {1, 2} + NFiles = 2 + Keep = 1 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "collectcurrent" +SPECIFICATION Spec +INVARIANTS CurrentIsComplete +PROPERTIES CurrentMonotone diff --git a/verification/proofs/tlaplus/MutantNoSupersede.cfg b/verification/proofs/tlaplus/MutantNoSupersede.cfg new file mode 100644 index 00000000..cf5c1231 --- /dev/null +++ b/verification/proofs/tlaplus/MutantNoSupersede.cfg @@ -0,0 +1,13 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: violation of CurrentMonotone +CONSTANTS + Writers = {1, 2} + NFiles = 2 + Keep = 1 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "nosupersede" +SPECIFICATION Spec +INVARIANTS CurrentIsComplete +PROPERTIES CurrentMonotone diff --git a/verification/proofs/tlaplus/MutantSwapBeforeComplete.cfg b/verification/proofs/tlaplus/MutantSwapBeforeComplete.cfg new file mode 100644 index 00000000..31d8b835 --- /dev/null +++ b/verification/proofs/tlaplus/MutantSwapBeforeComplete.cfg @@ -0,0 +1,13 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: violation of CurrentIsComplete +CONSTANTS + Writers = {1, 2} + NFiles = 2 + Keep = 1 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "swapbeforecomplete" +SPECIFICATION Spec +INVARIANTS CurrentIsComplete +PROPERTIES CurrentMonotone diff --git a/verification/proofs/tlaplus/NoRetry.cfg b/verification/proofs/tlaplus/NoRetry.cfg new file mode 100644 index 00000000..e5c3a3ed --- /dev/null +++ b/verification/proofs/tlaplus/NoRetry.cfg @@ -0,0 +1,12 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: violation of ReaderNeverFails +CONSTANTS + Writers = {1, 2} + NFiles = 2 + Keep = 0 + Crashes = FALSE + Retry = FALSE + MaxRetries = 3 + Mutant = "none" +SPECIFICATION Spec +INVARIANTS ReaderNeverFails diff --git a/verification/proofs/tlaplus/Safety3Keep0.cfg b/verification/proofs/tlaplus/Safety3Keep0.cfg new file mode 100644 index 00000000..086633d8 --- /dev/null +++ b/verification/proofs/tlaplus/Safety3Keep0.cfg @@ -0,0 +1,13 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: pass +CONSTANTS + Writers = {1, 2, 3} + NFiles = 2 + Keep = 0 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "none" +SPECIFICATION Spec +INVARIANTS TypeOK CurrentIsComplete ReaderNeverFails +PROPERTIES CurrentMonotone diff --git a/verification/proofs/tlaplus/Safety3Keep1.cfg b/verification/proofs/tlaplus/Safety3Keep1.cfg new file mode 100644 index 00000000..7c270989 --- /dev/null +++ b/verification/proofs/tlaplus/Safety3Keep1.cfg @@ -0,0 +1,13 @@ +\* SPDX-License-Identifier: MPL-2.0 +\* Expected: pass +CONSTANTS + Writers = {1, 2, 3} + NFiles = 2 + Keep = 1 + Crashes = TRUE + Retry = TRUE + MaxRetries = 3 + Mutant = "none" +SPECIFICATION Spec +INVARIANTS TypeOK CurrentIsComplete ReaderNeverFails +PROPERTIES CurrentMonotone diff --git a/verification/proofs/tlaplus/check.sh b/verification/proofs/tlaplus/check.sh new file mode 100755 index 00000000..a390e8c0 --- /dev/null +++ b/verification/proofs/tlaplus/check.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell +# +# Model-check GenerationPublish.tla with every .cfg beside it. +# +# Each .cfg states its expected outcome on its second line: +# "\* Expected: pass" TLC must finish with no error; +# "\* Expected: violation of ..." TLC must report violated. +# The expected violations are the documented mirror limit, the reader +# without a retry, and three deliberate mutants: they show the invariants +# can fail. A config that does anything else fails the run. +# +# Usage: TLA2TOOLS=/path/to/tla2tools.jar verification/proofs/tlaplus/check.sh +# The jar must be the pinned release below; its digest is checked first. +set -euo pipefail + +TLA2TOOLS_VERSION=v1.7.4 +TLA2TOOLS_SHA256=936a262061c914694dfd669a543be24573c45d5aa0ff20a8b96b23d01e050e88 + +jar=${TLA2TOOLS:?set TLA2TOOLS to tla2tools.jar ($TLA2TOOLS_VERSION)} +dir=$(cd "$(dirname "$0")" && pwd) +echo "$TLA2TOOLS_SHA256 $jar" | sha256sum --check --quiet + +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT +failures=0 +for cfg in "$dir"/*.cfg; do + name=$(basename "$cfg" .cfg) + expected=$(sed -n '2s/^\\\* Expected: //p' "$cfg") + log="$work/$name.log" + set +e + java -XX:+UseParallelGC -cp "$jar" tlc2.TLC -workers auto -deadlock \ + -metadir "$work/$name.states" -config "$cfg" "$dir/GenerationPublish.tla" >"$log" 2>&1 + set -e + states=$(grep -oE '[0-9]+ distinct states found' "$log" | tail -1 || true) + case "$expected" in + pass) + if grep -q 'No error has been found' "$log"; then + echo "PASS $name: no error ($states)" + else + echo "FAIL $name: expected no error"; sed -n '1,40p' "$log"; failures=$((failures + 1)) + fi ;; + "violation of "*) + prop=${expected#violation of }; prop=${prop%% *} + if grep -qE "(Invariant|Action property) $prop is violated" "$log"; then + echo "PASS $name: $prop violated, as expected ($states)" + else + echo "FAIL $name: expected $prop to be violated"; sed -n '1,40p' "$log"; failures=$((failures + 1)) + fi ;; + *) + echo "FAIL $name: no 'Expected:' line"; failures=$((failures + 1)) ;; + esac +done +exit "$failures" From 36f1055f5b527f085ef70c3a80d42744fc27b2fd Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 10 Nov 2026 02:00:44 +0000 Subject: [PATCH 06/11] feat(assail): line-anchor Lean/Isabelle sorry findings so inline markers apply (#278) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Why Lean and Isabelle `sorry` produced one ProofDrift finding per file, with a count and `line: None`. The inline `panic-attack: accepted` pass only runs on findings that carry a line. So the only way to record an audited `sorry` was the A2ML classification registry, which is retired estate-wide and matches a whole file and category. That left hyperpolymath/echidna's static analysis gate red with no supported way out. Its corpus adapters keep deliberate detector fixtures: a `sorry` they must flag. hyperpolymath/echidna#478 bumps echidna to the scanner commit that can complete coverage, and these two fixtures are then its only Critical findings. ## What - **One finding per occurrence.** Each Lean or Isabelle `sorry` is its own Critical ProofDrift finding with its `line`. `stats.unsafe_blocks` counts the same as before. A severity gate's pass/fail is unchanged, though a file with N `sorry` now contributes N findings. - **Position-preserving strippers.** `strip_proof_comments` and `strip_isabelle_prose` now keep every newline of a removed comment or prose span, and the line-comment pass splits on `'\n'` instead of `lines()`. Line numbers taken from the stripped text are therefore line numbers in the file (cf. #114). The new `occurrence_lines` maps matches to 1-indexed lines. - **`(*` marker leader**, recognised at the start of a line only, for Isabelle, Coq/Rocq, OCaml, SML and F*. A trailing `*)` is dropped from the reason. Mid-line `(*` is excluded because it is also a C dereference or a Python splat. - **Fix: markers when the scan root is not the working directory.** Locations are relative to the scan root, but the marker pass read them relative to the process working directory. So `assail ` run from anywhere else silently ignored every inline marker, in all languages. It now resolves against the scan root first and falls back to the working directory. ## Evidence - `cargo test --features http`: **791 passed, 0 failed**. `cargo clippy --features http --all-targets`: **0 warnings**. `cargo fmt` is clean. - New tests: - a `proptest` that both strippers preserve every newline, over generated inputs built from comment, prose and cartouche tokens (including unterminated blocks and `\r\n`); - unit tests for `occurrence_lines`, for a `sorry` after multi-line comments and prose, and for the new leader (start-of-line only, previous-line suppression); - an end-to-end `analyze()` over a temp dir: one marked and one unmarked `sorry` per language, behind multi-line comments and prose. It runs from the crate root, so it also covers the root resolution. - Mutants, each **killed** by the new tests: | Mutant | Result | |---|---| | markers-read-cwd-only | killed | | block-drops-newlines | killed | | prose-drops-newlines | killed | | no-ml-leader | killed | | unterminated-drops-lines | killed | - **Against echidna `main`**, with its two fixtures annotated (`(* panic-attack: accepted (…) *)` above the Isabelle `sorry`, `-- panic-attack: accepted (…)` above the Lean one), scanned from `/tmp`: - both findings are suppressed at `Smoke.thy:21` and `algebra.lean:11`; - echidna's own `static-analysis-evidence.sh scan` + `gate` passes: `status: ran`, exit 0, `critical: 0`, total 109, which is 111 minus the two fixtures. ## Not in this PR - Other proof escape hatches (Isabelle `oops`, Coq `Admitted`, Agda `postulate`, Idris `believe_me`, and so on) are still per-file and unanchored. The same `occurrence_lines` pattern applies, but each needs its own position check, so they are left to a follow-up rather than widening this PR. - A deed-format successor to the A2ML classification registry. --- _Generated by [Claude Code](https://claude.ai/code/session_01JHfuPCv1pEiqkQGG5DQASb)_ --------- Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com> Co-authored-by: Claude --- src/assail/analyzer.rs | 189 ++++++++++++++++++++++++++++++++--------- src/assail/mod.rs | 78 ++++++++++++++++- src/comment_marker.rs | 44 ++++++++++ 3 files changed, 269 insertions(+), 42 deletions(-) diff --git a/src/assail/analyzer.rs b/src/assail/analyzer.rs index fb0af3a8..c7eecc39 100644 --- a/src/assail/analyzer.rs +++ b/src/assail/analyzer.rs @@ -3668,25 +3668,18 @@ impl Analyzer { // matching so that doc-comments documenting "no sorry" do not produce // false positives. let code = strip_proof_comments(content, "--", Some(("/-", "-/"))); - // sorry — banned proof escape hatch: admits unproven propositions - let sorry_count = code.matches("sorry").count(); - if sorry_count > 0 { - stats.unsafe_blocks += sorry_count; - weak_points.push(WeakPoint { - file: None, - line: None, - category: WeakPointCategory::ProofDrift, - location: Some(file_path.to_string()), - severity: Severity::Critical, - description: format!( - "{} sorry (banned proof escape hatch — admits unproven proposition) in {}", - sorry_count, file_path - ), - recommended_attack: vec![AttackAxis::Cpu], - suppressed: false, - test_context: None, - }); - } + // sorry — banned proof escape hatch: admits unproven propositions. + // One finding per occurrence, anchored to its line, so an audited + // occurrence can carry an inline `panic-attack: accepted` marker + // without hiding any other sorry in the same file. + push_line_anchored_proof_drift( + &code, + "sorry", + "1 sorry (banned proof escape hatch — admits unproven proposition)", + file_path, + stats, + weak_points, + ); // unsafeNativeIO — bypasses IO monad discipline let unsafe_io_count = @@ -3847,25 +3840,16 @@ impl Analyzer { // proof-escape hatches. See #43. let without_blocks = strip_proof_comments(content, "", Some(("(*", "*)"))); let code = strip_isabelle_prose(&without_blocks); - // sorry — Isabelle's admitted-proof escape hatch (banned estate-wide) - let sorry_count = code.matches("sorry").count(); - if sorry_count > 0 { - stats.unsafe_blocks += sorry_count; - weak_points.push(WeakPoint { - file: None, - line: None, - category: WeakPointCategory::ProofDrift, - location: Some(file_path.to_string()), - severity: Severity::Critical, - description: format!( - "{} sorry (banned — unfinished Isabelle proof admitted without verification) in {}", - sorry_count, file_path - ), - recommended_attack: vec![AttackAxis::Cpu], - suppressed: false, - test_context: None, - }); - } + // sorry — Isabelle's admitted-proof escape hatch (banned estate-wide). + // One line-anchored finding per occurrence (see analyze_lean). + push_line_anchored_proof_drift( + &code, + "sorry", + "1 sorry (banned — unfinished Isabelle proof admitted without verification)", + file_path, + stats, + weak_points, + ); // oops — abandons an unfinished proof without even admitting it let oops_count = code.matches("oops").count(); @@ -5899,6 +5883,52 @@ fn strip_simple_double_quoted_strings(content: &str) -> String { out } +/// Append only the newlines of a span that is being removed, so text after +/// the span keeps its original line number. +fn push_newlines_of(out: &mut String, removed: &str) { + out.extend(removed.chars().filter(|c| *c == '\n')); +} + +/// 1-indexed line of every occurrence of `needle` in `code`. `code` must be +/// a position-preserving rewrite of the file (every newline kept), as +/// `strip_proof_comments` and `strip_isabelle_prose` produce. +fn occurrence_lines(code: &str, needle: &str) -> Vec { + code.match_indices(needle) + .map(|(at, _)| code[..at].matches('\n').count() as u32 + 1) + .collect() +} + +/// One Critical ProofDrift finding per occurrence of `needle` in `code`, each +/// anchored to its line so an inline `panic-attack: accepted` marker can +/// suppress exactly that occurrence. `code` must be position-preserving +/// (see `occurrence_lines`). +fn push_line_anchored_proof_drift( + code: &str, + needle: &str, + what: &str, + file_path: &str, + stats: &mut ProgramStatistics, + weak_points: &mut Vec, +) { + let lines = occurrence_lines(code, needle); + stats.unsafe_blocks += lines.len(); + for line in lines { + weak_points.push(WeakPoint { + file: None, + line: Some(line), + category: WeakPointCategory::ProofDrift, + location: Some(file_path.to_string()), + severity: Severity::Critical, + description: format!("{what} in {file_path}:{line}"), + recommended_attack: vec![AttackAxis::Cpu], + suppressed: false, + test_context: None, + }); + } +} + +/// Remove proof-language comments while keeping every newline, so a line +/// number taken from the result is a line number in `content`. fn strip_proof_comments(content: &str, line_marker: &str, block: Option<(&str, &str)>) -> String { // First strip block comments greedily, left-to-right. let mut without_blocks = String::with_capacity(content.len()); @@ -5908,9 +5938,11 @@ fn strip_proof_comments(content: &str, line_marker: &str, block: Option<(&str, & without_blocks.push_str(&rest[..start]); let after_open = &rest[start + open.len()..]; if let Some(end) = after_open.find(close) { + push_newlines_of(&mut without_blocks, &after_open[..end]); rest = &after_open[end + close.len()..]; } else { - // Unterminated block comment — drop the rest. + // Unterminated block comment — drop the rest, keep its lines. + push_newlines_of(&mut without_blocks, after_open); rest = ""; break; } @@ -5920,12 +5952,13 @@ fn strip_proof_comments(content: &str, line_marker: &str, block: Option<(&str, & without_blocks.push_str(content); } - // Then strip line comments. + // Then strip line comments. Split on '\n' (not `lines()`) so a trailing + // newline survives and the newline count is exactly preserved. if line_marker.is_empty() { return without_blocks; } let mut out = String::with_capacity(without_blocks.len()); - for (i, line) in without_blocks.lines().enumerate() { + for (i, line) in without_blocks.split('\n').enumerate() { let kept = match line.find(line_marker) { Some(idx) => &line[..idx], None => line, @@ -6050,6 +6083,7 @@ fn strip_isabelle_prose(content: &str) -> String { out.push_str(&rest[..next.0]); out.push(' '); + push_newlines_of(&mut out, &rest[next.0..next.1]); rest = &rest[next.1..]; } } @@ -8361,3 +8395,76 @@ pub fn safe_get_x() -> Option { ); } } + +#[cfg(test)] +mod proof_line_anchor_tests { + use super::*; + use proptest::prelude::*; + + /// Inputs built from the tokens the proof strippers react to, so the + /// generator actually reaches unterminated blocks, nested cartouches and + /// comments that span lines. + fn proof_source() -> impl Strategy { + let tokens = [ + "(*", + "*)", + "/-", + "-/", + "--", + "//", + "text ", + "\\", + "\\", + "@{text ", + "}", + "{", + "\n", + "\r\n", + "sorry", + "a", + " ", + ]; + proptest::collection::vec(proptest::sample::select(tokens.to_vec()), 0..60) + .prop_map(|parts| parts.concat()) + } + + proptest! { + // The invariant line anchoring rests on: stripping never adds, + // drops or reorders a newline, so the line of any surviving + // character is its line in the original file. + #[test] + fn proof_strippers_preserve_every_newline(src in proof_source()) { + let newlines = src.matches('\n').count(); + for (line_marker, block) in [ + ("--", Some(("/-", "-/"))), + ("", Some(("(*", "*)"))), + ("//", None), + ] { + let out = strip_proof_comments(&src, line_marker, block); + prop_assert_eq!(out.matches('\n').count(), newlines); + } + let isabelle = strip_isabelle_prose(&strip_proof_comments(&src, "", Some(("(*", "*)")))); + prop_assert_eq!(isabelle.matches('\n').count(), newlines); + } + } + + #[test] + fn occurrence_lines_are_one_indexed_per_occurrence() { + assert_eq!( + occurrence_lines("sorry\nx\n sorry sorry\n", "sorry"), + vec![1, 3, 3] + ); + assert!(occurrence_lines("no escape here\n", "sorry").is_empty()); + } + + #[test] + fn sorry_after_multiline_comment_keeps_its_line() { + let lean = "/- a\n b -/\ntheorem t : True := by\n sorry\n"; + let code = strip_proof_comments(lean, "--", Some(("/-", "-/"))); + assert_eq!(occurrence_lines(&code, "sorry"), vec![4]); + + let thy = "(* a\n b *)\ntext \\prose\n sorry in prose\\\nlemma l: True\n sorry\n"; + let code = strip_isabelle_prose(&strip_proof_comments(thy, "", Some(("(*", "*)")))); + assert_eq!(occurrence_lines(&code, "sorry"), vec![6]); + } +} diff --git a/src/assail/mod.rs b/src/assail/mod.rs index 3a628465..5e561611 100644 --- a/src/assail/mod.rs +++ b/src/assail/mod.rs @@ -436,6 +436,15 @@ pub fn apply_v255_context_suppression(report: &mut AssailReport) -> usize { let mut newly_suppressed = 0usize; let mut content_cache: std::collections::HashMap> = std::collections::HashMap::new(); + // Finding locations are relative to the scan root, not to the process + // working directory, so resolve them against the root. Without this, + // `assail ` run from anywhere but silently ignored every + // inline marker. + let scan_root: Option = if report.program_path.is_dir() { + Some(report.program_path.clone()) + } else { + report.program_path.parent().map(Path::to_path_buf) + }; for wp in &mut report.weak_points { // Determine the file path; prefer the structured `file` field, @@ -455,7 +464,14 @@ pub fn apply_v255_context_suppression(report: &mut AssailReport) -> usize { // None so we don't repeatedly retry. let content_opt = content_cache .entry(file.clone()) - .or_insert_with(|| std::fs::read_to_string(&file).ok()) + .or_insert_with(|| { + let rel = Path::new(&file); + scan_root + .as_deref() + .filter(|_| rel.is_relative()) + .and_then(|root| std::fs::read_to_string(root.join(rel)).ok()) + .or_else(|| std::fs::read_to_string(rel).ok()) + }) .clone(); // Always set test_context — even when the finding isn't a @@ -891,3 +907,63 @@ mod v255_context_suppression_tests { ); } } + +#[cfg(test)] +mod proof_marker_tests { + use super::*; + use crate::types::WeakPointCategory; + use std::fs; + use tempfile::TempDir; + + /// End to end through `analyze()`: each `sorry` is its own finding on + /// its own line, an inline marker suppresses only the occurrence it sits + /// above, and markers resolve against the scan root rather than the + /// process working directory (the test runs from the crate root). + #[test] + fn sorry_markers_suppress_only_the_marked_occurrence() { + let tmp = TempDir::new().unwrap(); + fs::write( + tmp.path().join("Fix.lean"), + "theorem a : True := by\n -- panic-attack: accepted (detector fixture)\n sorry\n\n\ + /- a\n multi-line\n comment -/\ntheorem b : True := by\n sorry\n", + ) + .unwrap(); + fs::write( + tmp.path().join("Fix.thy"), + "theory Fix imports Main begin\n(* a\n multi-line comment *)\n\ + text \\prose\n over lines\\\nlemma a: True\n \ + (* panic-attack: accepted (detector fixture) *)\n sorry\nlemma b: True\n sorry\nend\n", + ) + .unwrap(); + + let report = analyze(tmp.path()).unwrap(); + let mut got: Vec<(String, u32, bool)> = report + .weak_points + .iter() + .filter(|w| matches!(w.category, WeakPointCategory::ProofDrift)) + .filter(|w| w.description.contains("sorry")) + .map(|w| { + let loc = w.location.clone().unwrap_or_default(); + let name = Path::new(&loc) + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .unwrap_or(loc); + ( + name, + w.line.expect("sorry findings are line-anchored"), + w.suppressed, + ) + }) + .collect(); + got.sort(); + assert_eq!( + got, + vec![ + ("Fix.lean".to_string(), 3, true), + ("Fix.lean".to_string(), 9, false), + ("Fix.thy".to_string(), 8, true), + ("Fix.thy".to_string(), 10, false), + ] + ); + } +} diff --git a/src/comment_marker.rs b/src/comment_marker.rs index eab824eb..81a91449 100644 --- a/src/comment_marker.rs +++ b/src/comment_marker.rs @@ -21,6 +21,8 @@ //! `;` Lisp / Scheme / Racket / Clojure / Common Lisp / Assembly //! `--` Haskell / Idris / SQL / Lua / Ada / Elm //! `%` Erlang / Prolog / MATLAB / LaTeX +//! `(*` Isabelle / Coq / Rocq / OCaml / SML / F* (start of line only; +//! a trailing `*)` is dropped from the reason) //! //! After the comment leader, we accept (whitespace tolerant): //! @@ -157,6 +159,12 @@ fn strip_to_after_comment_leader(line: &str) -> Option<&str> { if let Some(rest) = trimmed_start.strip_prefix("%") { return Some(rest); } + // ML-family block comment: Isabelle, Coq/Rocq, OCaml, SML, F*. + // Start-of-line only, like the leaders above: mid-line `(*` is also a + // C dereference or a Python splat. + if let Some(rest) = trimmed_start.strip_prefix("(*") { + return Some(rest); + } // Mid-line: only C-style `//` is safe — the others have non-comment // valid uses in expressions. @@ -196,6 +204,10 @@ fn strip_to_after_comment_leader(line: &str) -> Option<&str> { /// Clean the reason text: strip leading `:`, ` - `, parens, whitespace. fn clean_reason(s: &str) -> String { let mut s = s.trim(); + // A `(* ... *)` marker carries its closer on the same line. + if let Some(stripped) = s.strip_suffix("*)") { + s = stripped.trim(); + } if let Some(stripped) = s.strip_prefix(':') { s = stripped.trim(); } @@ -338,4 +350,36 @@ mod tests { assert!(is_suppressed_at(content, 0).is_none()); assert!(is_suppressed_at(content, 99).is_none()); } + + #[test] + fn ml_block_comment_marker_drops_its_closer() { + let m = parse_accepted_marker_in_line(" (* panic-attack: accepted (detector fixture) *)"); + assert_eq!(m, Some("(detector fixture)".to_string())); + let bare = parse_accepted_marker_in_line("(* panic-attack: accepted *)"); + assert_eq!(bare, Some(String::new())); + } + + #[test] + fn ml_block_comment_leader_is_start_of_line_only() { + // Mid-line `(*` is a C dereference or a Python splat, never a leader. + assert_eq!( + parse_accepted_marker_in_line(" sorry (* panic-attack: accepted *)"), + None + ); + assert_eq!( + parse_accepted_marker_in_line("x = (*p); /* panic-attack */"), + None + ); + assert_eq!( + parse_accepted_marker_in_line("(* an ordinary comment *)"), + None + ); + } + + #[test] + fn ml_marker_on_previous_line_suppresses() { + let content = "lemma a: True\n (* panic-attack: accepted *)\n sorry\n"; + assert!(is_suppressed_at(content, 3).is_some()); + assert!(is_suppressed_at(content, 1).is_none()); + } } From 9986ca4e7c25d787934274efd3e5ac2f417fc195 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 10 Nov 2026 16:40:16 +0000 Subject: [PATCH 07/11] fix(abi): repair the Idris2 proofs and record proof status against the policy (#277) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I re-checked panic-attack's Idris2 proofs with Idris2 0.7.0, the version `proven-tests.yml` pins. No CI job type-checks `src/abi/`, so the ✅ marks in `PROOF-NEEDS.adoc` had no receipt behind them. ## What the check found on `main` | Module | Result on `main` | Design | |---|---|---| | `ClassificationSoundness.idr` (PA2) | passes | Sound and non-trivial for its model | | `AttestationUnforgeability.idr` | passes | Sound relative to its hypotheses. Two of them are symbolic-model idealisations, and the header overstated them as EUF-CMA and collision resistance. | | `PatternCompleteness.idr` (PA1) | passes | **Vacuous**: `GenericAnalyzer : HasAnalyzer lang` inhabits every language. The model is also stale (see below). | | `Stripping.idr` | **does not compile** | The theorem is true, but the proof had two type errors, hidden behind a parse error. | | `Types.idr` | **does not compile** | No theorems, despite a header claiming three | PA1's staleness: its `Lang` has 50 languages against 52 in Rust, missing `C` and `Php`. `crossLangAlwaysApplied` also says cross-language checks run on every file, but the Rust code now skips them for configuration files. ## Commits 1. **`3dc4728` fix(abi): make the Stripping and Types Idris2 modules type-check** - **`Stripping.idr`:** matching on a `Char` literal only reduces when the character is a literal, so the proof stuck on a variable head. The model now classifies each character once (`Slash`, `Newline`, `Other`) and steps on the class, and each proof carries `cls c = k`. - The theorem statements are unchanged. - All seven sanity checks are unchanged and pass. - The new model equals the old one on all 349,525 strings of length ≤ 9 over `/`, newline, `x` and space. - **`Types.idr`:** syntax-only fixes. Its header now says it declares types only, and notes its drift from `src/types.rs`. 2. **`b1b1419` docs(proofs): record proof status against the estate proof policy** - `PROOF-NEEDS.adoc` now uses the proof policy's §5 tokens and §1.2 verifier mapping, pinned to `metadatastician/standards` `9bdf864`. - It records a receipt for the Idris2 check and an honest status per obligation. - The implementation obligations are listed as `OPEN` for Creusot/Pearlite and Verus: stripping, Patch Bridge reachability and taint analysis. ## Verification - All five `src/abi` modules type-check. The pass criterion is exit 0 and no `Error:` line; `idris2 --check` can exit 0 while printing an error. - **Negative controls:** a planted false lemma and a planted non-terminating definition are both rejected. - **No escape hatches:** every module sets `%default total` and uses none of `believe_me`, `assert_total`, `postulate` or holes. The one `believe_me` match is in a doc comment that bans it. - **Kani note:** `cargo clippy --lib --bins --tests --benches --all-features -- -F unsafe_code` compiles. The only `unsafe` is in `examples/vulnerable_program.rs`, a deliberately buggy scan target. - `PROOF-NEEDS.adoc` renders with Asciidoctor at `--failure-level WARN`. ## Not in this PR - **CI gate for `src/abi`:** the modules are named `PanicAttack.ABI.*` but live in `src/abi/`, so a plain `idris2 --check src/abi/X.idr` fails on the module name. The status file records the gate as `OPEN`, with the invocation to use. - **PA1's redesign or retirement:** recorded as `OPEN`. --- _Generated by [Claude Code](https://claude.ai/code/session_01Jma5ixtpeZyJRmfoSsC98u)_ --------- Co-authored-by: Claude --- PROOF-NEEDS.adoc | 262 +++++++++++++++++++++++++++++++++------------- src/abi/Types.idr | 22 ++-- 2 files changed, 202 insertions(+), 82 deletions(-) diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index e351a613..b89e4041 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -1,6 +1,13 @@ -== PROOF-NEEDS.md — panic-attack += Proof status: panic-attack -=== Current State +This file records each proof obligation in panic-attack, the verifier it +belongs to, and its status. The rules are the estate proof policy, +https://github.com/metadatastician/standards/blob/9bdf864cc7f2c8e24b4df84bf00d04158eee3ad5/3-practice/PROOF-POLICY.adoc[`standards/3-practice/PROOF-POLICY.adoc` +at `9bdf864cc7f2c8e24b4df84bf00d04158eee3ad5`]. Statuses use its closed §5 +vocabulary: `PROVEN`, `PROVEN-WITH-AXIOMS`, `TESTED`, `ASSUMED`, `OPEN`. + +None of these obligations is a theory theorem (§1), so none carries a Proof +Witness Grade. *Receipts (2026-11-09).* Until this date no CI job type-checked `+src/abi/+`, and as committed none of the five modules checked: their module @@ -28,23 +35,65 @@ that DETECTS believe_me in other repos); 282 `+unwrap()+` calls * *ABI layer*: Idris2 with completeness + soundness proofs + Layer-1.0 stripping foundation -=== Completed Proofs +== Where each kind of obligation belongs + +* *Behaviour of first-party Rust code* (stripping, reachability, taint + propagation, dispatch): Creusot with Pearlite contracts, and Verus, on the + implementation (§1.2). An Idris2 model of a Rust function is not evidence + that the function is correct (§1, §2.2). +* *Interfaces, ABI encodings and cross-language protocols* (the attestation + chain, the C integer encodings): Idris2. +* *Statements true of any implementation* (for example, a taint lattice + and its fixed point stated abstractly): Agda, rechecked in Rocq and Lean + (§2). + +No Kani obligation arises (§4.1). With `unsafe_code` forbidden, +`cargo clippy --lib --bins --tests --benches --all-features -- -F unsafe_code` +compiles at `f2a6a70eaec0cb02331e3264e05f185b5faab0f7`. The only `unsafe` +is in `examples/vulnerable_program.rs`, a deliberately buggy program that +exists to be scanned. + +== Receipt for the Idris2 modules + +* *Checker:* Idris2 0.7.0, the version `proven-tests.yml` pins, built from + tag `v0.7.0` (`27780073c8631826d846499840b3857d9b9a4fd5`) on Chez Scheme + 9.5.8. +* *Source:* `src/abi/*.idr` as committed. `Stripping.idr` is the re-proof + from #281 (see the ledger row below). +* *Invocation:* the modules are named `PanicAttack.ABI.` but live in + `src/abi/`, so `idris2 --check src/abi/.idr` fails on a module-name + mismatch from any directory. `scripts/ci/check-idris-abi.sh` stages the + committed files at `PanicAttack/ABI/.idr` as links and checks those. + A pass is exit 0 *and* no `Error:` line, because `idris2 --check` can + exit 0 while printing an error. +* *Result:* all five modules pass. Every module sets `%default total` and + uses no `believe_me`, `assert_total`, `assert_smaller`, `postulate` or + `idris_crash`. +* *Negative controls:* against this PR's earlier `Stripping.idr` + (`3dc4728`), a planted false lemma and a planted non-terminating + definition were both rejected, so the check can fail. +* *Gate:* the `idris-abi` job in `.github/workflows/proven-tests.yml` runs + this check on every PR and blocks on failure (#281). The statuses here + rest on that job's receipt for the commit in question; it passed on + `main` at `36f1055`. -[width="100%",cols="26%,21%,53%",options="header",] +== Ledger + +[cols="3,2,2,5",options="header"] |=== -|Proof |File |What it proves -|PA1 Pattern detection completeness |`+src/abi/PatternCompleteness.idr+` -|All 49 `+Lang+` constructors have an analyzer; all 20 `+WPCategory+` -constructors have at least one detector; cross-language checks applied -unconditionally to all languages. `+completeScanForAll+` is the -top-level theorem. - -|PA2 Classification soundness |`+src/abi/ClassificationSoundness.idr+` -|Severity (Low/Medium/High/Critical) is totally ordered (`+LTE+`); -`+maxSeverity+` is commutative and idempotent; numeric ABI encoding -preserves the ordering. - -|Layer 1.0 line-comment idempotence |`+src/abi/Stripping.idr+` +|Obligation |Artefact |Status |Trust boundary and notes + +|Severity order: `Low < Medium < High < Critical` is a total order; +`sevMax` is an upper bound; the numeric encoding is monotone (PA2) +|`src/abi/ClassificationSoundness.idr` +|`PROVEN` (model) +|The model's `Severity` matches the variant names and order of +`src/types.rs`; that correspondence is `ASSUMED` (a hand transcription). +The Rust order itself comes from `#[derive(PartialOrd, Ord)]`. + +|Line-comment stripping is idempotent: `strip (strip s) = strip s` +|`src/abi/Stripping.idr` +|`PROVEN` (model) |*Re-proved 2026-11-09.* The 2026-06-02 proof (issue #113) never type-checked: its `+//+` case relied on reductions that do not hold for a variable character, and it used `+decEq+` without importing it. The new proof @@ -59,29 +108,81 @@ definitions. A mutant that blanks with `+/+` instead of a space is rejected. Nine closed sanity theorems, including two comments on different lines and `+///+`. -|Attestation chain unforgeability -|`+src/abi/AttestationUnforgeability.idr+` |*PROOF-PROGRAMME §3.2 -(2026-06-04, issue #123)*: models the intent→evidence→seal chain -(`+chain_hash = H(intent‖evidence‖report)+`, Ed25519-signed) with the -cryptographic facts as a `+parameters+` block (hypotheses, *not* -`+postulate+`). Under `+%default total+` Qed-closes `+integrity+` -(tampering any phase invalidates the seal), `+authenticity+` (a -verifying seal comes from the matching secret key), `+nonRepudiation+` -(a genuine seal verifies), plus 2 corollaries. Conditional on chain-hash -collision-resistance + Ed25519 EUF-CMA (message- and signer-binding) + -signature correctness. - -|Hexad↔Octad persistence round-trip |`+src/storage/mod.rs+` (proptest) -|*PROOF-PROGRAMME §3.1 (2026-06-04, #122)*: -`+hexad_json_roundtrip_is_identity+` tests an in-memory -`+serde_json::Value+` serialize/deserialize/serialize round-trip on generated -samples, not by a universal theorem. It does not call `+write_*_hexad+` or -`+load_hexad_dir+`; on-disk persistence correspondence remains a separate -obligation. The gateway octad projection is lossy by design, so this property -also does not establish a faithful hexad↔octad map. +The model does *not* match `strip_proof_comments`, so it is evidence about +the model only. It replaces comment characters with spaces and keeps every +newline. The Rust function deletes comment text and drops a trailing +newline: on `//x` the model gives three spaces and Rust gives the empty +string. The model also covers line comments only; block comments and string +literals are not modelled. + +|Attestation chain: integrity, authenticity, non-repudiation +|`src/abi/AttestationUnforgeability.idr` +|`PROVEN` (model), relative to the `ASSUMED` hypotheses listed here +|The module's `parameters` block assumes: + +* `chainCR`: the chain hash is *injective*. +* `sigCorrect`: an honest signature verifies. +* `sigBindsMessage`: a signature verifies only on the message signed. +* `sigOnlySigner`: for every verifying signature there is a secret key + whose public key is the verifying key and whose signature on that digest + is that signature. + +`sigOnlySigner` says such a key *exists*. It says nothing about who +produced the signature, so `authenticity` inherits that limit: it shows the +seal equals one a matching key would make, not that the key holder made it. +`chainCR` and `sigOnlySigner` are idealisations, stronger than collision +resistance and EUF-CMA, and false of any real fixed-length hash or +signature scheme. So these are theorems of a symbolic (Dolev-Yao style) +model, not a computational security reduction. Model to +`src/attestation/`: `ASSUMED`. A computational proof is not claimed. + +|Every language reaches an analyzer; every weak-point category has a +detector; cross-language checks run on every file (PA1) +|`src/abi/PatternCompleteness.idr` +|`PROVEN` (model); supports no claim about the Rust code +|The theorem is vacuous: `GenericAnalyzer : HasAnalyzer lang` inhabits +the witness type for every language, so it holds whatever the dispatch +does. Rust gives the narrower guarantee that actually holds: +`match file_lang` in `src/assail/analyzer.rs` ends in +`_ => analyze_generic`, and `rustc` checks the match is exhaustive. That +covers every file that reaches the match. A file with no source language +and no configuration adapter (`Language::Unknown`, no adapter) is marked +`unsupported` in the coverage ledger and skipped first, so not every file +reaches an analyzer. + +The model has also drifted. Its `Lang` has 50 constructors and +`src/types.rs` has 52; `C` and `Php` are missing. `crossLangAlwaysApplied` +says cross-language checks run on every file, but the Rust code skips them +when `configuration.is_some()`. That model claim is false of the +implementation. + +|Declarations for the ABI-facing types and their C integer encodings +|`src/abi/Types.idr` +|No theorem +|Repaired in `3dc4728` so it compiles. It contains no proof and nothing +imports it. Drift: its `Severity` is `Info / Warning / Error / Critical` +where `src/types.rs` has `Low / Medium / High / Critical`, and `LangId` is +bounded at 47 against 52 variants. + +|Hexad JSON round-trip is the identity +|`src/storage/mod.rs`, `hexad_json_roundtrip_is_identity` (proptest) +|`TESTED` +|A serialize, deserialize, serialize round trip on proptest-generated +hexads, compared as `serde_json::Value`. Bounds: proptest's default 256 +cases (the repository sets no override). `id` and `program_path` are 0 to +48 characters from `[A-Za-z0-9:_./-]`; `language` 0 to 12 lowercase +letters; `version` 1 to 8 characters from `[0-9.]`; the four counts are +below 10,000; `robustness_score` is in [0, 1]; `categories` holds 0 to 5 +strings of 1 to 16 letters; `attestation_hash` is absent or up to 64 hex +digits. Fixed, not generated: `schema`, `created_at`, `tool`, and +`document` (`{"weak_points": [], "n": total}`), and the `migration`, +`finding`, `campaign` and `crosslang` facets are all `None`. It does not call +`write_*_hexad` or `load_hexad_dir`, so on-disk persistence is a separate +obligation. The gateway octad projection is lossy by design, so this does +not establish a faithful hexad-to-octad map. |=== -=== Model-Checked (TESTED, bounded) +== Model-checked (TESTED, bounded) [width="100%",cols="26%,30%,44%",options="header",] |=== @@ -101,43 +202,60 @@ state. *§6 deviation (open)*: TLC is fetched from its release, digest-checked, because neither Guix nor the container provides it yet. |=== -=== What Still Needs Proving +== Open obligations -[width="100%",cols="51%,27%,22%",options="header",] -|=== -|Component |What |Why -|Generation publish: pure decisions (`src/publish.rs`) |*OPEN* (PROOF-POLICY -§1.2): Creusot/Pearlite and Verus contracts for the supersede comparison and -the collect selection (never removes the current or a newer generation; keeps -the `keep` newest older ones) |The model check covers the protocol, not the Rust -code; these tie the implementation to P1 and P2 - -|*Layer 1.0 — stripBlockComments + Strings + Composition + -Position-Preservation* (issue #114) |Block-comment (`+/* */+`) and -string-literal (`+"..."+`) strippers with same mutual-recursive shape; -composition theorem proving the full pipeline is idempotent given each -component is; position-preservation theorem justifying analyzer -location-reporting against the stripped view as if it were -original-source. |Four slices: `+Stripping_Block.idr+`, -`+Stripping_Strings.idr+`, `+Stripping_Composition.idr+`, -`+Stripping_PositionPreservation.idr+`. - -|Bridge reachability soundness |Reachability analysis is sound (no -reachable dep wrongly classified as phantom) |Unreachable code marked -reachable wastes effort; reachable missed = security gap - -|Kanren taint analysis |Taint propagation tracks all tainted data flows -|Missed taint flow means missed vulnerability +[cols="3,2,5",options="header"] |=== +|Obligation |Verifier (§1, §1.2) |Notes + +|Generation publish: pure decisions (`src/publish.rs`). The supersede +comparison and the collect selection never remove the current or a newer +generation, and keep the `keep` newest older ones +|Creusot/Pearlite and Verus (§1.2) +|*OPEN*. The model check above covers the protocol, not the Rust code; +these contracts tie the implementation to P1 and P2. + +|`strip_proof_comments` is idempotent and preserves positions outside +comments and strings +|Fix the implementation first; then Creusot/Pearlite and Verus +|Neither property holds of the current Rust function, so neither can be +proved until it changes. All three cases below were run against the +function as it is on `main`: -=== Recommended Prover +* Not idempotent: `a//*x*/*b*/c` gives `a/*b*/c`, and a second pass gives + `ac`. +* Positions move: `a/*x*/b` gives `ab`, moving `b` from offset 6 to 1. +* Ordering bug: block comments are stripped before line comments, so a + block opener inside a line comment swallows the code after it. + `code // see /* note`, then `hidden();`, then `*/ tail`, gives `code `. + The callers that pass both a line and a block marker are affected: + `analyze_rust`, `is_rust_ffi_safe_wrapper`, `analyze_idris`, + `analyze_lean` and `analyze_agda`. In the Idris, Lean and Agda ProofDrift + detectors such a comment can hide a later `sorry`, `believe_me` or + postulate. `analyze_isabelle` and `analyze_coq` pass no line marker and + are not affected. -*Idris2* — Already in use. Taint analysis correctness proofs could use -*Agda* with relational semantics. The 282 unwrap() calls are a -significant debt (but separate from the proof obligations). +The Idris2 model above can guide the contracts once the function is +fixed; it does not discharge them. The model's own remaining slices +(block comments, strings, composition, position preservation; issue #114) +are optional once the implementation proof exists. -=== Priority +|Patch Bridge reachability is sound: no reachable dependency is classified +as phantom +|Creusot/Pearlite and Verus +|`src/bridge/reachability.rs`. A missed reachable dependency is a security +gap. -*MEDIUM* (was HIGH) — PA1 and PA2 completed 2026-04-11. The highest-risk -false-negative scenario (analyzer dispatch completeness) is now formally -proved. Remaining proofs are deeper semantic properties. +|Kanren taint analysis tracks every tainted flow +|Creusot/Pearlite and Verus; Agda for the abstract theory +|Any implementation-independent statement (the lattice, monotonicity, the +fixed point) is theory and belongs in Agda with Rocq and Lean rechecks. +Its realisation in `src/kanren/` is a Rust obligation. + +|Replace or retire PA1 +|Creusot/Pearlite and Verus, or retire +|A non-vacuous statement would be, for example, that each language with +a dedicated analyzer is routed to it. If that is not wanted, retire the +module, because `rustc`'s exhaustiveness check already gives the current +guarantee. +|=== diff --git a/src/abi/Types.idr b/src/abi/Types.idr index 1afe5c0e..e9cefc31 100644 --- a/src/abi/Types.idr +++ b/src/abi/Types.idr @@ -3,11 +3,13 @@ -- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) ||| ABI Types for panic-attack static analysis engine ||| -||| Defines the formal interface for the scanner's operations. -||| Proves that: -||| 1. Severity levels are totally ordered -||| 2. Scan results cannot be fabricated (constructive) -||| 3. Language coverage is bounded +||| Declares ABI-facing types and their C integer encodings. It contains no +||| theorems, and no other module imports it. The severity order is proved +||| in ClassificationSoundness.idr, over its own Severity type. +||| +||| Known drift (see PROOF-NEEDS.adoc): this Severity type does not match +||| src/types.rs, and LangId is bounded at 47 while src/types.rs has 52 +||| Language variants. module PanicAttack.ABI.Types import Data.Fin @@ -18,14 +20,14 @@ import Data.Fin public export data Severity = Info | Warning | Error | Critical -||| Equality on severity, required by `Ord`. public export +||| Equality on severity, required by `Ord`. Eq Severity where - Info == Info = True - Warning == Warning = True - Error == Error = True + Info == Info = True + Warning == Warning = True + Error == Error = True Critical == Critical = True - _ == _ = False + _ == _ = False ||| Total ordering on severity public export From 7070cb8856ade6e41ebba08325b69f33c37975e7 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 10 Nov 2026 16:42:20 +0000 Subject: [PATCH 08/11] fix(abi): put the Eq doc comment above public export The merge placed '||| Equality on severity' between 'public export' and 'Eq Severity where'. Idris2 then rejects 'public export' as a declaration start (idris-abi on b332b5f). Doc comment first, as on main. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0136P7ZnmBahbhfkFEP6ark4 --- src/abi/Types.idr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/abi/Types.idr b/src/abi/Types.idr index e9cefc31..f3eef792 100644 --- a/src/abi/Types.idr +++ b/src/abi/Types.idr @@ -20,8 +20,8 @@ import Data.Fin public export data Severity = Info | Warning | Error | Critical -public export ||| Equality on severity, required by `Ord`. +public export Eq Severity where Info == Info = True Warning == Warning = True From c7b1c44ffe25b36100790bb1a98ad07d40d4e931 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 10 Nov 2026 16:53:15 +0000 Subject: [PATCH 09/11] fix(abi): put the Eq doc comment above public export in Types.idr (#285) #277 merged at b332b5f. The fix for its one failing check (7070cb8) was pushed to the branch but did not land. ## Problem On main, `src/abi/Types.idr` has the `|||` doc comment between `public export` and `Eq Severity where`. Idris2 0.7.0 rejects it: ``` Error: Keyword 'public' is not a valid start to a declaration. PanicAttack.ABI.Types:23:1--23:7 ``` So the required job "Idris2 ABI proofs type-check" fails on main and on every PR based on it ([failing run](https://github.com/hyperpolymath/panic-attack/actions/runs/42856302372/job/129970151246)). ## Change Move the doc line above `public export`. One line, no change to any declaration. ## Verification The same commit is on #277's branch. The other 4 modules passed in the failing run; this is the only error. --- _Generated by [Claude Code](https://claude.ai/code/session_0136P7ZnmBahbhfkFEP6ark4)_ Co-authored-by: Claude --- src/abi/Types.idr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/abi/Types.idr b/src/abi/Types.idr index e9cefc31..f3eef792 100644 --- a/src/abi/Types.idr +++ b/src/abi/Types.idr @@ -20,8 +20,8 @@ import Data.Fin public export data Severity = Info | Warning | Error | Critical -public export ||| Equality on severity, required by `Ord`. +public export Eq Severity where Info == Info = True Warning == Warning = True From 4f51cca94eec4bc4714cecfbd8d4ac4fa3e34b8b Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 10 Nov 2026 17:14:52 +0000 Subject: [PATCH 10/11] fix(ci): build the scanner, not the caller, in scan-and-report.yml (#286) Refs #276 and #212. This pull request does not close #276 by itself: the first acceptance criterion needs a hosted run from an external caller (see "Not shown yet"). ## Symptom Every repository that calls `scan-and-report.yml` as a reusable workflow fails at **Run scan** with exit 127: `./target/release/panic-attack` does not exist. Observed on hyperpolymath/echidna `main`, Security Scan run 42351376316. ## Cause In a reusable workflow, `actions/checkout` with no `repository:` checks out the **caller**. The step `cargo build --locked --release` therefore built the caller's crate. It worked only when this repository ran the workflow on itself. ## Change The workflow now fetches the scanner source at the commit of this workflow file and builds that. - **Source:** `job.workflow_repository` at `job.workflow_sha`. GitHub documents both as the repository and commit of the workflow file that defines the job. - A caller that pins `scan-and-report.yml@` gets the scanner at that SHA. It is never `main` by name, which keeps the #212 property. - A run of this workflow in its own repository still builds the commit under test. - **Location:** the runner's temp directory, outside the workspace. The scan cannot report on the scanner's own files. - **Refusals:** the fetch step exits 2 for a repository that is not `owner/name` and for a commit that is not a full SHA. It also checks that the fetched `HEAD` is that SHA. - **Cache:** the cache step keys on the scanner's workspace, not the caller's. - **Scan step:** it takes the target path from the environment and quotes it. No `uses:` reference changes, so `actions.lock` is unchanged. One difference from the proposal in #276: the proposal names `hyperpolymath/panic-attack` as a literal. This change takes the repository from `job.workflow_repository`, so a rename of this repository cannot break the fetch. ## Acceptance criteria | Criterion | State | |---|---| | An external caller reaches `Run scan`, and the binary exists | Local run only. Needs a hosted run from an external caller. | | This repository's own run still builds the commit under test | Hosted `workflow_dispatch` run on this branch: see the comment below when it completes. | | The report contains no findings from the scanner source | Holds by construction: the source is outside the workspace. The local report names no scanner path. | ## Evidence (local) I extracted the changed steps from the workflow file and ran them on x86_64 Linux with cargo 1.97.0. - **Fetch:** succeeds at `c7b1c44` (`main` at that time). It is refused with exit 2 for `main`, a short SHA, a malformed repository and an empty repository. - **Build:** 3 min 14 s. The binary exists at the path that the scan step uses. - **Scan:** a checkout of hyperpolymath/echidna as the caller. Exit 0, schema 2.6 report, 951 of 951 files analysed, 130 weak points. The report names no scanner source path, and no scanner source is inside the workspace. ## Not shown yet That `job.workflow_sha` names this repository's commit inside a call from another repository. The local run cannot show it. A caller must pin a commit that contains this change; echidna's `security-scan.yml` still pins `be6ac46`. --- _Generated by [Claude Code](https://claude.ai/code/session_0136P7ZmAsYpFA9gxfQLF3F6)_ Signed-off-by: Claude Co-authored-by: Claude --- .github/workflows/scan-and-report.yml | 61 ++++++++++++++++++++++----- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/.github/workflows/scan-and-report.yml b/.github/workflows/scan-and-report.yml index 233c012d..eaa3a7ba 100644 --- a/.github/workflows/scan-and-report.yml +++ b/.github/workflows/scan-and-report.yml @@ -36,8 +36,37 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 20 steps: + # The scan target. In a reusable workflow, a checkout with no + # `repository:` is the CALLER, not this repository. - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # The scanner source, at the commit of THIS workflow file. `job.workflow_*` + # names the workflow that defines the job: for a caller that pins + # `scan-and-report.yml@` it is this repository at that SHA, and for a + # run of this file in its own repository it is the commit under test. It is + # never `main` by name. + # + # The source goes to the runner's temp directory, outside the workspace, + # so the scan below cannot report on the scanner's own files. + - name: Fetch the scanner source at this workflow's commit + env: + SCANNER_REPOSITORY: ${{ job.workflow_repository }} + SCANNER_SHA: ${{ job.workflow_sha }} + SCANNER_SRC: ${{ runner.temp }}/panic-attack-src + GIT_TERMINAL_PROMPT: '0' + run: | + set -euo pipefail + [[ "$SCANNER_REPOSITORY" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]] \ + || { echo "::error::scanner repository is not owner/name (got '${SCANNER_REPOSITORY}'); no scanner was built"; exit 2; } + [[ "$SCANNER_SHA" =~ ^[0-9a-f]{40}$ ]] \ + || { echo "::error::scanner commit is not a full SHA (got '${SCANNER_SHA}'); no scanner was built"; exit 2; } + git init -q "$SCANNER_SRC" + git -C "$SCANNER_SRC" fetch -q --depth 1 "https://github.com/${SCANNER_REPOSITORY}.git" "$SCANNER_SHA" + git -C "$SCANNER_SRC" checkout -q --detach FETCH_HEAD + [[ "$(git -C "$SCANNER_SRC" rev-parse HEAD)" == "$SCANNER_SHA" ]] \ + || { echo "::error::fetched scanner source is not ${SCANNER_SHA}"; exit 2; } + echo "scanner source: ${SCANNER_REPOSITORY}@${SCANNER_SHA}" + - name: Install Rust uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # v1 with: @@ -45,20 +74,32 @@ jobs: - name: Cache Rust dependencies uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 - - # Build the commit under test, not `main`. The previous step was - # `cargo install --git … --branch main`, so every pull request was scanned - # by whatever happened to be on main at that moment: the workflow reported - # on code it had not checked out, and a PR that broke the scanner could - # pass with the last good binary. - - name: Build the commit under test - run: cargo build --locked --release + with: + # Key and save the scanner's build, not the caller's workspace. + workspaces: ${{ runner.temp }}/panic-attack-src -> target + + # Build the scanner at the fetched commit, not `main` and not the caller. + # History of this step: `cargo install --git … --branch main` scanned + # every pull request with whatever was on main (#212). Its replacement, + # `cargo build` in the workspace, built the CALLER, so every external + # caller failed at "Run scan" with exit 127 (#276). + - name: Build the scanner at this workflow's commit + env: + SCANNER_SRC: ${{ runner.temp }}/panic-attack-src + run: | + set -euo pipefail + cargo build --locked --release --manifest-path "$SCANNER_SRC/Cargo.toml" + test -x "$SCANNER_SRC/target/release/panic-attack" - name: Run scan id: scan + env: + SCANNER: ${{ runner.temp }}/panic-attack-src/target/release/panic-attack + REPO_PATH: ${{ inputs.repo_path }} run: | - ./target/release/panic-attack assail ${{ inputs.repo_path }} --output scan-result.json - echo "scan_complete=true" >> $GITHUB_OUTPUT + set -euo pipefail + "$SCANNER" assail "$REPO_PATH" --output scan-result.json + echo "scan_complete=true" >> "$GITHUB_OUTPUT" - name: Enforce severity gate if: steps.scan.outputs.scan_complete == 'true' && inputs.fail_on_severity != 'none' From 55ffcc9a240e937edd968487af15ab45c6f0522f Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 10 Nov 2026 17:22:35 +0000 Subject: [PATCH 11/11] fix(ci): give a manual run of scan-and-report.yml its inputs (#287) Follow-up to #286. Refs #276. ## What failed Run 42862049822 was the first run of `scan-and-report.yml` by `workflow_dispatch`. It ran on the #286 branch before the merge. - The new steps of #286 passed: the fetch of the scanner source and the build (2 min 33 s). - **Run scan** failed with exit 2: `a value is required for '' but none was supplied`. ## Cause `repo_path` and `fail_on_severity` are declared under `workflow_call` only. A `workflow_dispatch` run has its own `inputs` context, so both values were empty strings. - The scan had no target. - The severity gate would then reject the empty threshold (`fail_on_severity must be one of ...`). The step before #286 expanded the same empty input, so this path did not work before #286 either. #286 did not cause it. The first manual run showed it. ## Change Declare both inputs under `workflow_dispatch`, with the same names and the same defaults (`.` and `none`). The expressions in the steps do not change. An explicit bad value from a caller is still refused. ## Evidence A hosted `workflow_dispatch` run on this branch is the proof. I add its result in a comment when it completes. This pull request does not show the external caller case of #276. That still needs a caller that pins a commit with #286, for example echidna's `security-scan.yml`. --- _Generated by [Claude Code](https://claude.ai/code/session_0136P7ZmAsYpFA9gxfQLF3F6)_ Signed-off-by: Claude Co-authored-by: Claude --- .github/workflows/scan-and-report.yml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/workflows/scan-and-report.yml b/.github/workflows/scan-and-report.yml index eaa3a7ba..1c8e0be6 100644 --- a/.github/workflows/scan-and-report.yml +++ b/.github/workflows/scan-and-report.yml @@ -26,7 +26,20 @@ on: VERISIMDB_PAT: description: 'PAT with repo scope for cross-repo dispatch to verisimdb-data' required: false + # A run of this file by hand has its own `inputs`: the workflow_call inputs + # above do not exist for it. Without these two, `inputs.repo_path` and + # `inputs.fail_on_severity` are empty strings, "Run scan" has no target and + # the severity gate rejects the empty threshold. Same names, same defaults. workflow_dispatch: + inputs: + repo_path: + description: 'Path to scan (default: .)' + default: '.' + type: string + fail_on_severity: + description: 'Fail on an unsuppressed finding at or above: none | low | medium | high | critical' + default: 'none' + type: string permissions: contents: read