[andrewpark-scaleai]

Why

A run whose grader, snapshot or agent step failed but was tolerated (fail_task_on_error=False) completes and counts as a success. Nothing on the run says what it lost, so no reader can flag it, hold it, or repair it.

Today a context.metadata.failed_steps entry records the step id, type, error text and whether it was fatal. It does not say whether the failed step was the grader or an optional snapshot, and it does not say whether the error was a model key out of budget (permanent until the budget resets) or a timeout (worth retrying). This PR adds those two facts. It changes no behavior. The next PR derives the run's quality from them.

What

• StepRole (grader, evidence, agent, other) on TaskStep as step_role. Every step type that writes context.metadata["verifications"] declares grader. snapshot_env, snapshot_agent_state and collect_artifacts declare evidence. prompt_agent declares agent. Every other step stays other.
• A step can override its type's role in config with step_role, so a run_code step that grades can say so. It serializes only when it differs from the type's role, so existing step documents are unchanged. An unknown value is refused when the task loads.
• New agent_env/task/outcome.py with FailureCause (key_budget, key_auth, transient, unknown), classify_failure(exc) and the FailedStep entry with FailedStep.from_exception(...).
• classify_failure reads the type names and messages of the exception's whole __cause__ / __context__ chain. A key cause wins over a transient one, because a timeout that wraps an out-of-budget error still fails on every retry.
• The DAG runner builds its failed_steps entry with FailedStep.from_exception. The entry keeps every existing key and adds step_role and cause.
• attach_retry_config becomes attach_base_fields, since it now hydrates step_role as well. All five callers are migrated.

Design decisions

• Declare the role on the step, not in a central table. The step type is the one place that knows whether it grades, and a new step type declares its role when it is written. A table would need every consumer to keep a copy in sync. The cost is a one-line declaration in 31 step classes. A test scans every registered step type and fails if one that writes verifications is not a grader, so a new verifier cannot forget.
• Name it step_role, not role. deploy_agent and modify_env_tool_access already use a role config field for their own meaning. A base role field would collide with both in config and on the instance.
• Match error text in one function. A judge or agent error crosses A2A and the model proxy as a message string, so its original exception type is gone by the time a step raises. Text matching is unavoidable. Keeping it in classify_failure means one place to fix when a provider changes its wording. The cost is that an unfamiliar message reads as unknown, which is the safe default.
• StepRole lives in task_step/task_step.py and outcome.py re-exports it. task_step cannot import from agent_env.task without an import cycle through agent_env/task/__init__.py.

Test

uv run --extra dev pytest tst/unit/task/outcome_test.py -q
25 passed

Without the change the module does not exist and collection fails. With only outcome.py present and the runner and step classes reverted, 6 of 25 fail: the tolerated-grader run records no step_role or cause, the step types declare no role, and the config override does not load.

uv run --extra dev pytest tst/unit/task tst/unit/task_step tst/unit/bundle tst/unit/cli tst/unit/runner -q -n 4
5142 passed, 3 skipped in 542.59s