A model that costs 40% less per token can still cost more to ship with once steering, failed checks, and reviewer corrections enter the path. Fullbeam compares both Stack Releases on repeated private work and reports observed attempt cost per accepted-task equivalent without inventing missing engineering-effort data.
Each result stays tied to the Stack Release, workload, repository state, and coverage that produced it. If evidence is missing, the gap remains visible in the release call.
Measure observed model and runtime spend per accepted-task equivalent. Show engineer steering, retries, fallback calls, CI recovery, review changes, rework, and abandoned attempts separately, including unavailable states.
A candidate may match the baseline on backend maintenance and fail three database migrations, and Fullbeam keeps those outcomes separate instead of averaging them into one flattering score.
Prepare a canary only for workloads that passed. Production route, outcomes, cost, and engineering effort require their own observed evidence before broader promotion.
Annualized savings stay unavailable until baseline spend, eligible workload volume, outcome coverage, and cost attribution support the number. If those inputs are missing, the forecast stays blank.
Move the workloads that keep the saving. Leave migrations, authentication, or any weakly covered class on the current release until the evidence changes.
