[andrewpark-scaleai]

Why

A run whose grader, snapshot or agent step failed but was tolerated (fail_task_on_error=False) completes and counts as a success. A reader of status cannot tell such a run from a clean one.

Stacked on #692. That PR records each failed step's role and cause in failed_steps. This PR sums them up on the run, so a reader can check one field instead of decoding the list. It changes no behavior.

What

• RunQuality(reasons) with a derived degraded flag, and run_quality(failed_steps) in agent_env/task/outcome.py. reasons holds the distinct roles of tolerated failures that were not retried and not resolved, most severe first: grader, agent, evidence, other.
• The completion CAS in TaskInstanceStore writes quality = {"degraded": ..., "reasons": [...]} in the same update that sets status="completed". It derives it from the document as this write leaves it, so a sibling step's earlier failure counts too.
• TaskInstance.quality reads it back. A run completed before this change has no quality and reads as clean, which is what every reader assumes today.
• FailedStep.from_dict reads stored entries. An entry written before step_role and cause existed reads as other and unknown, so it still degrades the run.

Design decisions

• Two fields, status and quality, instead of a new status value. status is lifecycle: did the run finish? Every existing reader branches on its four values, and a new degraded value would turn degraded runs into "not completed" for all of them at once. A separate quality lets each reader opt in. The cost is one more field a reader must check to know a run is clean.
• Derive in the completion write. The store already holds every step's result when it sets completed. Writing quality in the same CAS means no reader ever sees a completed run without it. The cost is one copy of the context per run, at completion only.
• degraded is derived from reasons. It is a dataclass field so asdict matches the stored form, but it is never passed in, so it cannot disagree with reasons.
• Severity order puts grader first. A run with no grade cannot be scored at all, while a missing snapshot still leaves a graded run. A reader that keeps one reason takes the first.

Test

uv run --extra dev pytest tst/unit/task/run_quality_test.py -q
9 passed

Without RunQuality the test module fails to import. With the new outcome.py but the old store.py, 3 of 9 fail: the completed run has no quality, and TaskInstance has no quality to read.

uv run --extra dev pytest tst/unit/task tst/unit/task_step tst/unit/bundle tst/unit/cli tst/unit/runner tst/unit/explorer -q -n 4
5261 passed, 3 skipped in 652.56s