Repository navigation
Bound cancellation during Standard 14 text layout #541
Description
Context
Review of #538 found that the shared Standard 14 layout code can scan an input-sized paragraph without polling its operation budget: newline splitting, a single linebreaks iterator step, and trailing whitespace trimming each may traverse long input without an intervening poll. DESIGN.md F5/F6 and section 9 require long input-derived operations to observe the caller's deadline and cancellation.
Goal
Make Standard 14 text layout interruptible and budgeted during long individual paragraphs while preserving line-breaking and valid layout behavior.
Scope
In: shared Standard 14 layout and its underlying line-break traversal if needed, with deterministic long-input tests. Also cover the shared PDF text decoder used by forms where it scans long UTF-8 input without polling. Out: annotation-specific rendering and embedded-font shaping.
Paths
crates/pyropdf-engine/src/standard/,crates/pyropdf-engine/src/forms/text.rs, and narrowly related shared text code;- targeted Rust layout tests and relevant text tests;
- manifests only if a dependency change is necessary.
Acceptance
- Long paragraphs, whitespace runs, newline scans, and UTF-8 decoding poll cancellation and deadline during processing, including inside line-break discovery, without silent truncation.
- Deterministic mid-operation cancellation and deadline tests return typed errors; ordinary layout matches existing results.
- Full project checks and current-head CI pass.
Notes
See the review of #538. Checking only between lines does not interrupt a single input-sized iterator step. Do not duplicate the parser or silently change Unicode line-breaking semantics.
Activity
yhay81 commentedon Nov 1, 2026
Claimed on branch
claude/541-standard14-layout-cancel.yhay81 commentedon Nov 1, 2026
Handoff: branch
claude/541-standard14-layout-cancel, PR #546 (Part of #541; waiting for owner review, not merged).Changed: new
standard/scan.rswith polled, step-bounded scans. These cover newline search, trailing-whitespace trimming, line-break discovery (linebreaksrun one step at a time behind a context that restarts at breaks or safe pairs, with same-class runs cut to their ends), and grapheme discovery.layout,single_line,graphemes,grapheme_count(it now takes&Budget),ops::push_string, and the forms UTF-8 decoderappend_lossynow use these scans. NewLayoutError::BreakContextis returned for a break context longer thanMAX_BREAK_CONTEXT(1024 chars) after compression.Checks:
python3 tools/check.pyis green locally. Tests added: differential proptests againstlinebreaks, the grapheme iterator, andfrom_utf8_lossy; deterministic mid-scan cancellation and deadline tests on each scan and on publiclayout. CI on the PR head is pending.Still open (why the PR says Part of): inside one call,
unicode-segmentation'sGraphemeCursorscans back over the preceding run for emoji ZWJ, conjunct, and regional-indicator decisions, and that scan does not poll. Chunks that do not start at offset 0 would avoid it, butGraphemeCursor1.13.3 then gets RI parity wrong (proptest seed🇦🇦🇦🇦, step 1). Next action: owner review of #546. The owner also needs to decide whether that residual should become an upstream issue or be accepted, and whether theBreakContextrefusal is acceptable.yhay81 commentedon Nov 1, 2026
Review of PR #546 at
98f86e26b5c107bc06f55df324990a1078c5474dfound additional work before #541's acceptance can be met:standard/scan.rsgrows the input-derivedcontextandwindowstrings without charging their allocations to the operation's memory budget. Bound and charge the buffers, and test a constrained memory budget.standard/layout.rscopies the entire prefix preceding the first newline with onepush_str; a long prefix is not interruptible during that copy. Copy it in polled steps and test cancellation during the prefix copy.GraphemeCursorbackward scan still cannot poll during a long emoji, Indic conjunct, or regional-indicator context. The current mid-operation tests check the forward polling but do not establish cancellation inside that backward scan.BreakContextrejects some previously valid input at a fixed threshold and maps the refusal to a generic PDF error. Its compatibility and error semantics remain to be resolved.The PR is a partial change, does not close this issue, and should not unblock #168 until the remaining conditions are addressed. The owner should decide any upstream proposal before publication to the dependency repository.