In the last post I argued that a frozen language model in a loop can't pass the realization layer for a structural reason: the only state it carries that isn't already in its inputs is its own sampling noise, and noise isn't a belief. I also promised the positive half — a digital system that does carry, built to make the point that realization isn't about biology, it's about whether the structure lives on the running dynamics.
This is that system. But before I describe it, I have to describe the test, because the order matters.
Locking the test before building the thing
There's a failure mode in this kind of work that I wanted to make impossible for myself: building a system, seeing what it does, and then defining "success" as whatever it happened to do. So I wrote the test first and froze it — a substrate-agnostic gate, specified before I had a carrier that could pass it, so the verdict language couldn't be quietly reshaped by the results.
The test is a closed-loop world. There's a hidden quantity that drifts over time. The system never sees it directly — only a noisy reading — and it has to act, each step, to keep something on target. To do well, it has to maintain a running estimate of the hidden state and improve that estimate as evidence accumulates. It's the same family of task I put the language model through, and for the same reason: it's a task where a genuinely carried belief earns its keep and a reconstructed one doesn't.
Around that world I built a gate with three things the realization layer demands, plus one I insist on for honesty:
- Carrier-dependent — the behavior has to depend on internal running state, not just on the current input.
- Non-reconstructable — and this is the hard one — you must not be able to rebuild the system's operative structure from a bounded window of its recent inputs.
- Dynamically sustained — the structure persists and evolves on its own, rather than being re-derived each step.
- Productive — the fourth, mine: the structure has to actually be good at the task. An opaque system that's non-reconstructable because it's useless tells you nothing. I wanted the thing that's hard to reconstruct to be the very thing doing the work.
To measure non-reconstructability concretely, I run a window sweep. I hand a reconstructor the last W inputs the system saw and let it rebuild as good an estimate as it can, then compare. The carried system has to beat that reconstruction — and the advantage has to persist as the window grows. A self you can rebuild from a long-enough window of inputs isn't carried; it's just slow to copy.
The carrier
The system that passes is nothing like a transformer.
It's a continuously running carrier — picture a small dynamical system that never resets and never answers a "prompt." It just runs, ingesting the noisy stream, and it does two things at two speeds. Fast: it maintains a working estimate of the hidden world right now. Slow: across many episodes, it consolidates — it gradually tunes its own internal machinery into a model of how this world behaves, a model that lives in the running dynamics and the slowly-learned weights rather than in any stored transcript.
That two-speed structure is the whole trick, and it's where the realization property comes from. The fast state alone is reconstructable — it's close to the recent inputs. The slowly consolidated model is not, because it's the integrated residue of a long history of episodes, compressed into the system's own parameters. You can't read it off the last W observations, no matter how large W you allow, because it isn't a function of any bounded window. It's a function of the system's whole life so far.
The trade-off I didn't see coming
Here's the part I think is the actual content of the result, and the part I didn't anticipate when I started.
It is easy to be non-reconstructable. Be random. Be chaotic. Carry a state nobody can rebuild because it doesn't mean anything. And it is easy to be productive. Be a clean, simple function of the recent inputs — a good little estimator you could reconstruct in a line of code. What's hard — what I came to think is the real thing the realization layer is pointing at — is to be both at once: to have a single structure that is simultaneously the productive driver of competent behavior and unreconstructable from any bounded window.
Those two pull against each other. The more of your competence comes from a simple, legible function of recent input, the more reconstructable you are. The more you push your competence into something a window can't rebuild, the more you risk it being noise that doesn't help. For a long stretch I could get one or the other and not both, and I started to suspect the conjunction might be impossible — which would have been its own interesting result, a kind of impossibility at the heart of the layer.
It isn't impossible. But getting both requires a specific recipe, and the recipe is the finding:
- a rich enough world — one whose hidden structure genuinely takes many noisy observations to pin down, so that integrating a long history actually beats reacting to the last input;
- a carrier that identifies that structure — that builds an internal model of the world rather than just smoothing the input; and
- cross-episode consolidation — memory that carries the learned model forward across resets of the working state, so the operative self is the accumulated model, not the current scratchpad.
When all three are present, the same learned structure is both the thing doing the work and the thing you can't reconstruct. Take any one away and the conjunction collapses — and I checked that directly with two control conditions. Turn off the cross-episode consolidation (let each episode start fresh): the non-reconstructability advantage falls to zero. Swap the identifying carrier for a simple one: same collapse. The property isn't a lucky artifact of one system; it's what those three ingredients produce together, and nothing less does.
The honest boundary: operational, not absolute
I have to flag the limit, because it's the kind of thing that, left unflagged, makes the whole claim look like more than it is.
You might ask: fine, it's not reconstructable from a window — but is it reconstructable from the system's complete history, every input it ever saw? And the answer is yes. If you replay the entire causal history deterministically, you recover the structure exactly. I checked, and I want to be the one to say it: absolute non-reconstructability — irreducibility even to the full history — is not achievable for a productive system. Every mechanism I tried that broke full-history reconstruction also broke competence. You can't have a productive self that even God-with-the-logs couldn't rebuild.
That sounds like a retreat. I've come to think it's the opposite — it's the layer telling me what it actually is. The right notion of realization isn't "unrecoverable from the complete causal history." It's unrecoverable from any bounded window of current input, though fixed by the whole history. And that is exactly the biological situation. Your memories are not readable off your current sensory input — no window of what's hitting your retina right now reconstructs them — yet they are fully determined by your entire developmental history. Operational, within-window non-reconstructability isn't a weak stand-in for some stronger absolute version. It's the correct notion, the one a brain satisfies, and the continuous carrier has it.
I'll add, briefly, that this isn't one fragile setup. The same conjunction shows up across different task types, several different world families, and more than one kind of carrier — including a genuinely continuous-time one whose state evolves between observations rather than in discrete ticks. The recipe travels. That mattered to me, because a result that only works on the one world you tuned it on isn't a result, it's a coincidence.
So: a digital system, not a language model, carrying a productive self-structure on its own dynamics, unreconstructable from any window — passing the gate I locked before I built it. That clears the realization layer in the way I needed it cleared. But there's a sterner test, and it's the one I care most about, because it's my own gold standard — the same one the language models failed cold. The last post is whether the substrate passes that.