Why the word “agent” hides too much
Many AI systems are described as one agent even when several very different mechanisms are involved. A model generates the next action. A runtime loads the weights. Another process checks permissions. Some code remembers which subtask remains. A tool changes the outside world. Something else decides whether the work is finished.
When those responsibilities are collapsed into one box, a failure becomes difficult to classify. Did the model reason badly? Did the provider return an incompatible output? Did the tool execute against the wrong target? Did the workflow lose state? Did a successful operation get mistaken for a completed goal?
A cleaner architecture starts by naming the jobs separately.
1. Model Runtime = inference
The Model Runtime is responsible for executing the model itself. Typical responsibilities include:
- load and unload model weights;
- apply the correct tokenizer and chat template;
- manage context limits and KV cache;
- run CPU/GPU inference;
- apply sampling and stop conditions;
- support constrained JSON or schema output;
- handle cancellation and timeouts;
- report latency, token and memory metrics;
- normalise the output contract presented to the caller.
The Model Runtime should not need to know whether a user’s business task is complete. It can return one constrained decision without owning the multi-step workflow around that decision.
2. Controller = cognitive and process control
The Controller owns the state of the task outside the model. It answers questions such as:
- What must be true before this job is complete?
- Which obligation is still open?
- Which facts are UNKNOWN, VERIFIED or in CONFLICT?
- Which evidence supports a fact?
- Has this tool route already failed?
- Should a retry use the same route or a different one?
- Can deterministic code do this calculation instead of the model?
- Does the current state satisfy the completion gate?
The model can still reason. The Controller does not need to become a second LLM. Its value comes from owning workflow invariants in deterministic code.
3. Node / agent runtime = execution control
The execution runtime owns the boundary between intended action and real authority. Typical responsibilities include:
- registered tool manifests;
- workspace and target boundaries;
- read, write, process or other capability grants;
- timeouts and output limits;
- signed extension or sidecar validation;
- tool dispatch;
- structured execution result;
- audit evidence.
A request should not be able to grant itself more authority. The Controller may decide that a file write is the next required action, but Node should still refuse the action if the active capability does not permit it.
The three-layer flow
A useful composition looks like this:
Goal → Controller → Model Runtime → Model decision → Controller → Node → Tool → Evidence → Controller → next step or DONE.
The exact message order can vary, but the trust boundaries should remain recognisable.
Why not put the Controller inside Node?
Because the two layers change for different reasons.
A permission rule may change because a new filesystem capability is introduced. A Controller rule may change because completion semantics need stronger postcondition verification. Those changes should not require the same component to be rebuilt or trusted for both jobs.
Keeping Node narrow also makes it independently useful. A human-written workflow or a different Controller can still use the same execution boundary.
Why not let the model own the Controller job?
A model can often remember a short plan and decide when an answer looks complete. That is useful behaviour, but it is not a reliable invariant.
Longer workflows benefit from explicit obligations, deterministic retry budgets, conflict handling and a completion gate the model cannot simply talk around. If a required fact is UNKNOWN, software can keep it UNKNOWN even when the model sounds confident.
Why not put tool permissions inside the Model Runtime?
Because inference backends should remain replaceable. A local GGUF runtime, Ollama adapter or cloud provider can all produce the same high-level decision. None of them should define whether a Windows process, file or browser action is authorised.
Provider independence becomes easier when model execution and machine authority are different contracts.
Where deterministic operations belong
Simple extraction, filtering, counting, sorting, numeric comparison and version ordering usually belong in deterministic code near the Controller, not in the model.
This reduces unnecessary inference calls and makes smaller models more useful because they are asked to reason about ambiguity instead of repeatedly re-computing facts software can derive exactly.
A real example from ARKTOR
SC LABS currently applies this separation as:
ARKTOR Model Runtime — direct local inference, model lifecycle and constrained output.
ARKTOR Controller — TaskContract, obligations, evidence, deterministic resolver, retries, verification and completion.
ARKTOR Node — permission-gated execution through registered tools and signed sidecars.
In retained real-Node tests, this separation let the same Node execution path work with models that had different native tool/output conventions. A focused Qwen recheck moved from 0/3 on the free Node loop to 3/3 through Controller + Node after a model-adapter compatibility issue was normalised. That is mechanism evidence, not a claim that every model always improves.
Architecture checklist
- Can you replace the model/provider without changing filesystem permissions?
- Can you improve completion logic without rebuilding the tool runtime?
- Can Node reject an action even when Controller wants it?
- Can Controller keep a fact UNKNOWN even when the model claims confidence?
- Can the Model Runtime unload a model without losing task state?
- Can a new tool capability be added without teaching the model runtime about its security policy?
- Can deterministic operations run without an LLM call?
- Can each component be tested independently?
The simplest mental model
Runtime: how the model runs.
Controller: what the task still needs and when it is truly complete.
Node: what the system is allowed to do in the real world.
If those three answers are owned by different, explicit contracts, an AI agent becomes easier to reason about — even when the complete system is capable.