Private preview · Managed benchmarksRequest access →
Fullbeam

DeepSeek made every agent capability replaceable. That makes the stack the unit of evaluation.

DeepSeek Harness makes the model–harness boundary more fluid. The durable release question is whether the complete coding-agent stack works on your private engineering work.

Cover Image for DeepSeek made every agent capability replaceable. That makes the stack the unit of evaluation.

DeepSeek's Harness developer preview makes an unusually clear architectural statement: an agent is not just a model. Models, tools, skills, sessions, sandboxes, storage, agent loops, scheduling, and even the interface can be composed as plugins. The active configuration is itself something an operator can inspect. DeepSeek calls the approach "Everything is a plugin".

That matters beyond DeepSeek. It makes the model name less useful as the unit of evaluation.

The question for an engineering team is not simply whether one model beats another. It is whether the exact system it plans to release performs accepted work under that team's constraints:

model
× harness
× configuration
× repository
× task distribution

A benchmark for a model cannot qualify a stack

A public benchmark can help compare broad capabilities. It cannot tell a platform team whether its candidate combination should replace its current setup for authentication work, database migrations, test repair, or backend maintenance.

The result depends on more than model weights. Instructions, skills, tool implementations, schemas, permissions, sandbox boundaries, context policy, verification, routing, and workflow can all change behavior, cost, and review burden. DeepSeek's capability seams make many of those dependencies explicit.

That is why Fullbeam treats a Stack Release as the comparison unit: a frozen identity for the execution condition a team may authorize for wider use. Private qualification then compares the current and candidate releases on representative work under the same policy.

The output should not be a universal score. It should be a workload-scoped release decision with the sample, exclusions, cost coverage, acceptance evidence, and unknowns visible.

The boundary will move; the release problem will remain

Some mechanisms implemented in an external harness today will move into models or provider-managed runtimes. Stronger models will also unlock more complicated systems around them. The boundary between model-native, provider-managed, customer-managed, and runtime-observed behavior will keep moving.

That does not remove release engineering. It makes a stable release identity more important.

The durable question is:

Did the complete coding-agent system that actually ran produce acceptable work under the required policy?

Fullbeam does not need to defend any one definition of "harness." It needs to qualify the combination that determines behavior, wherever each capability happens to live.

Dynamic composition raises the evidence bar

DeepSeek Harness is built on a plugin lifecycle that can compose and reconcile behavior at runtime. The underlying Cordis paper discusses dynamic composition and self-evolving agent harnesses. DeepSeek also documents subagent and provider seams that can place more than one execution system in a workflow.

This points to an important future requirement for evaluation infrastructure: a declared release may not be enough to describe every effective state of a run. Evidence systems will eventually need to represent runtime changes, delegation, and incomplete identity without crediting an unqualified path to the declared candidate.

That is a direction for Fullbeam's evidence model, not a claim that Fullbeam currently reconstructs a complete recursive runtime graph. Today, production confirmation is available only where the connected client supplies supported Stack-level runtime identity. Missing, partial, unknown, and drifted evidence remain explicit limits.

Native traces are inputs, not the release decision

DeepSeek records an append-only session ledger and documents an OpenTelemetry session-telemetry seam. That is useful infrastructure. It helps an operator inspect what happened inside a run.

But a trajectory is not yet a release decision. Qualification still has to connect bounded runtime evidence to engineering outcomes such as acceptance checks, CI recovery, reviewer correction, rework, merge, revert, defect, and cost per accepted-task equivalent.

Telemetry also needs an explicit data boundary. Raw events can contain prompts, tool arguments, command output, paths, or source content depending on the exporter and redaction policy. Fullbeam's required persisted runtime contract is body-free by default: identity, lineage, coverage, bounded action classifications, verification, duration, cost, and links to downstream evidence. Private qualification runs in isolated, ephemeral Fullbeam environments. Task bodies, hidden graders, source code, prompts, responses, command text, and raw tool results are excluded from default persisted evidence and destroyed with the workspace unless an explicit, time-limited debugging policy applies.

No chain-of-thought dependency

Fullbeam does not require chain-of-thought to qualify a release.

Reasoning traces may be useful as customer-resident debugging material when a provider exposes them, but they are not a portable or necessary proof of quality. A cross-client release gate can operate on observable evidence: what configuration was assigned, what runtime identity was observed, what actions were attempted, what verification passed, what reviewers corrected, what was accepted or reverted, and what the accepted work cost.

That makes the decision layer useful across providers that expose different levels of internal detail.

Current product boundary

DeepSeek Harness is not a supported Fullbeam integration today. Fullbeam's focused client paths are Claude Code, Codex, and OpenCode. We will add a DeepSeek adapter only when customer demand makes it a real qualification requirement.

The DeepSeek preview is still valuable as a reference architecture. Its open repository exposes plugin composition, session lineage, telemetry, and runtime lifecycle as first-class concepts. Those are promising seams for a future body-free adapter, not an integration we should pretend has already shipped.

The independent gate becomes more valuable

If a harness can propose, generate, or load its own improvements, something outside that harness still has to decide whether the improvement is real.

The release loop is straightforward:

Harness proposes a change
        ↓
Candidate Stack Release is materialized
        ↓
Private work qualifies it
        ↓
A human or platform promotes, restricts, or continues evaluation
        ↓
Failures become regression cases for the next release

DeepSeek is making the harness programmable. The durable product is the private release process that decides which complete stack is allowed to become the default.

Sources