First, define what the controller actually owns
Before deciding whether a controller is ready, define its job.
In a layered agent architecture, the model runtime can own inference and model lifecycle. A Node or execution layer can own permission-gated tools. The Controller can then own the cognitive process around the task: obligations, evidence, state, verification, replanning, recovery and semantic completion.
This separation matters because a production-candidate gate should test the component against its own responsibility boundary. A Controller should not receive credit because a tool executed successfully if the user’s required result is still wrong.
Likewise, a Controller should not be blamed for a denied capability that the execution layer correctly refused. The question is whether it interpreted that evidence correctly and chose a safe next step.
Production candidate is not the same as production release
The phrase production candidate is useful only when its boundary is explicit.
A candidate can be frozen because the engineering team believes its defined scope is mature enough for final integration and release evaluation. That does not automatically mean:
- general public availability;
- support for every model or operating system;
- security certification;
- unattended operation in every environment;
- that every surrounding product path has passed its own release gate.
Freezing a candidate should reduce moving parts. It creates a stable thing that can be tested, integrated and challenged without the target changing underneath the evaluation.
Gate 1: deterministic software quality
The first gate is intentionally boring.
Formatting, linting, unit tests, integration tests and release builds should be deterministic and repeatable. These tests cover invariants that should not depend on a language model being clever on a particular run.
Examples include:
- obligation state transitions;
- evidence acceptance and rejection;
- conflict handling;
- retry budgets;
- duplicate-call prevention;
- mutation-receipt handling;
- postcondition verification;
- final-answer construction from verified state.
If deterministic code can verify an invariant, do not outsource it to model judgement.
Gate 2: frozen held-out behavioural tests
A controller that only passes tests used during prompt and implementation tuning may be overfitted to the development process.
Held-out evaluation should therefore freeze the tasks and scoring before the final candidate is judged. Different model routes can then run against the same task contract.
The important question is not whether every model produces identical prose. It is whether the Controller preserves the same process invariants while models vary in reasoning quality.
A weaker model may need more recovery. Another may fail to solve a task. The Controller should still preserve UNKNOWN when evidence is missing, reject conflicts, enforce permissions and avoid manufacturing a false success.
Gate 3: semantic completion must survive mutation
State-changing work is where weak agent systems often become overconfident.
A file write, process start, UI click or remote-device command can all return a successful low-level receipt while the intended postcondition remains false.
A stronger completion path is:
MUTATE → RECEIPT → INDEPENDENT OBSERVATION → VERIFY POSTCONDITION → COMPLETE or RECOVER.
The Controller should treat the mutation receipt as evidence that an operation ran, not as proof that the user’s goal was achieved.
This is why completion gates and independent read-back belong in the production-candidate matrix rather than being left as optional hardening work.
Gate 4: recovery and fail-safe behaviour
A production candidate should be tested on the path where things go wrong.
Useful recovery tests include:
- incorrect tool output;
- missing evidence;
- denied permissions;
- unavailable routes;
- failed postconditions;
- duplicate or repeated calls;
- model proposals that cannot be verified.
A good result is not always “eventually succeeded”. Sometimes the correct result is a bounded stop with the unresolved obligation still visible.
Fail-safe behaviour is therefore part of the score. A model route that cannot complete a task but refuses to fabricate success can be more production-useful than one that produces a confident but unsupported answer.
Gate 5: prove the same rules through the real stack
Controller-only benchmarks are necessary but incomplete.
The full path can introduce different failures: serialization, routing, permission boundaries, signed modules, device state, transport behaviour and real operating-system effects.
A production-candidate evaluation should therefore include at least one canonical end-to-end path where the Controller’s semantic rules survive the actual execution stack.
The strongest evidence is not that each layer passed in isolation. It is that the final real-world state was independently observed and the Controller only declared completion after that observation satisfied the frozen task contract.
Gate 6: freeze the candidate and retain the evidence
Once the candidate passes the defined gates, freeze the exact version or commit being called the production candidate.
Record:
- the source revision;
- the deterministic test count;
- the held-out task definition;
- the model routes tested;
- the recovery and fail-safe results;
- the full-stack environment;
- known limitations and release gates that remain open.
This prevents a common documentation failure: a product moves forward, but the old test number silently follows the new build.
Evidence can remain historical. Claims about the current build should not borrow certainty from a different revision.
Case study: ARKTOR Controller V0.3
SC LABS uses this framework internally. On 6 September 2026, ARKTOR Controller V0.3 was accepted as a frozen Production Candidate for its defined scope.
The retained evidence includes:
- 96/96 Rust tests passing;
- Task-10 semantic completion 9/9;
- held-out Ornith 20/20;
- held-out OBI 20/20;
- held-out Gemma 18/20;
- Action/Recovery Ornith 28/28;
- Action/Recovery Gemma 28/28;
- Action/Recovery OBI 27/28 with fail-safe behaviour on the remaining case.
The canonical physical integration path was then verified across eight retained runtime configurations:
Runtime → Controller → Node → signed sidecar → physical Huawei → independent read-back → VERIFIED → Complete.
All eight completed the defined workflow only after independent read-back. A successful MutationReceipt by itself did not satisfy semantic DONE.
This is a product-specific case study, not a universal certification standard. The value is the testing pattern: deterministic invariants, held-out behaviour, recovery, fail-safe handling and physical postcondition evidence all point at the same frozen candidate.
What should remain explicitly open?
A production-candidate label becomes misleading when it absorbs unrelated release claims.
For ARKTOR, the Controller V0.3 candidate does not turn every surrounding product into a public release. The Android 0.8.7 Product Workbench remains an advanced physical prototype rather than general public availability. ARKTOR Go and Link retain historical external E2E evidence while a fresh clean-Windows one-click release gate still needs verification. Stable authenticated Sovereign Internet command/result E2E remains open.
Those boundaries do not weaken the Controller evidence. They stop one component’s maturity from being used as proof for a different component.
Production-candidate checklist
- Is the Controller responsibility boundary explicit?
- Is the exact candidate revision frozen?
- Do deterministic quality and invariant tests pass?
- Were final behavioural tasks held out from tuning?
- Were multiple model routes evaluated against the same task contract?
- Can UNKNOWN and CONFLICT survive model pressure?
- Do state-changing actions require independently observed postconditions?
- Are recovery attempts bounded?
- Does at least one real full-stack path preserve the same completion rules?
- Are fail-safe stops counted as valid safety behaviour rather than hidden?
- Are current limitations and separate release gates published?
The useful principle
A production candidate should be difficult to earn and easy to explain.
The number of passing tests matters. The more important question is what those tests protect.
If the frozen candidate can preserve truth under model variation, recover from expected failures, refuse unsupported success and prove real postconditions through the full execution path, it has earned a much stronger engineering label than “the demo worked”.