Troubleshooting and FAQ

A private qualification cannot start

Confirm that the current and candidate Stack Releases, benchmark version, frozen release policy, decision owner, and valid provider credential are present. Managed launch fails closed when the Daytona API key, supported snapshot, retained egress-attestation digest, harness/provider/model compatibility, or budget authorization is missing. Workload scope comes from the frozen benchmark task taxonomy and Release Policy; changing it requires a new benchmark or policy version.

Cost per accepted-task equivalent is unavailable

• Confirm cost coverage for every decision-driving valid attempt.
• Confirm that the arm has a nonzero accepted-task-equivalent denominator after the frozen repetition-aggregation policy is applied.
• Do not substitute token count or raw scheduled-work cost for the missing metric.
• Use the attempt IDs and coverage reason codes to repair ingestion, then rerun or continue evaluation according to policy.

The annualized saving is unavailable

This is expected until baseline annual spend or eligible workload volume, currency, pricing or FX provenance, projection horizon, and evidence coverage are supplied. Fullbeam does not invent a financial projection from an offline cost ratio.

A pull request does not appear in supporting evidence

1. Confirm the repository is included in the GitHub App installation.
2. Open Settings → GitHub and check that the installation belongs to the correct Fullbeam team.
3. Update the pull request or start a repository backfill.
4. In GitHub, inspect recent App webhook deliveries and confirm that pull_request events receive a successful response.

PR evidence analysis is still running

Analysis runs after the webhook is accepted. Large diffs or provider retries can take longer. If the state becomes failed, use the displayed failure category and run ID instead of pushing empty commits.

PR evidence analysis failed

• Provider: the AI provider did not complete the request. Check credentials and provider availability, then retry.
• Validation: the result did not satisfy Fullbeam's chapter, atom, or citation contract. Retry once; report persistent failures with the run ID.
• Budget: the configured analysis budget was reached.
• Fetch or GitHub: Fullbeam could not read the required revision. Check repository access and GitHub availability.

Earlier review marks remain available when a newer analysis fails.

The GitHub comment is missing

Confirm that the GitHub App can write pull-request issue comments and that the repository is still selected. The story remains available from the Fullbeam check and Pull Requests inbox.

Managed runtime evidence is missing

Open the attempt evidence and compare declared, materialized, and observed runtime identity. A harness can complete while the effective model remains partial or unknown when its output does not expose that identity. Missing runtime evidence does not become a match and can block qualification credit.

A Stack Release is out of date or drifted

Compare the assigned Stack Release with the effective-runtime observation in Releases or Rollouts. Apply or roll back configuration through your coding-agent vendor, GitOps, MDM, or internal platform, then start a new instrumented attempt so Fullbeam can observe the result.

Frequently asked questions

Is Fullbeam a token dashboard or routing service?

No. Vendor and gateway usage data are inputs. Fullbeam determines whether a proposed cost-saving Stack Release satisfies acceptance and engineering guardrails, and which workloads should receive it.

Do I need to connect coding-agent execution?

Yes for private baseline-versus-candidate qualification and effective-runtime confirmation. The GitHub App alone remains enough for supporting PR evidence.

Does Fullbeam work only with agent-written PRs?

No. It can organize human-written, agent-written, and mixed pull requests.

Does Fullbeam approve or reject PRs?

No. Fullbeam qualifies the system producing changes. Final review actions for each pull request remain in GitHub.

Will Fullbeam add a new comment after every push?

No. It maintains one summary comment and updates its story and chapter links.

Why can one chapter contain several files?

Chapters represent outcomes or decisions. Several files can work together to produce one behavior.

Why is a detail missing from the explanation?

The story intentionally omits routine and obvious details. The complete associated diff remains beside the chapter.

Can Fullbeam prove that a model, prompt, or skill caused a better result?

Private paired qualification supports a policy-bound comparison under controlled conditions. Production canary results remain observational and are reported with sample size, scope, and coverage rather than universal causal claims.

What happens after the PR changes?

Fullbeam compares the new revision with your checkpoint. Unchanged chapters keep continuity; affected chapters require another look.

Can someone use a copied link without repository access?

No. Fullbeam checks authentication and workspace entitlements when the story loads.

Can we select only some repositories?

Yes. Choose selected repositories during installation and change the selection later in GitHub.

What should I include in a support report?

Include the repository, pull-request number, Fullbeam run ID, and failure category. Never include secrets, device credentials, or private prompt content.