EP 08 · How AI Actually Works
How Looped Transformers Think 132 Layers Deep Without Words: Huginn-0125 on an A100
Huginn-0125 on an A100: one 4-layer core reused up to 132 layers deep, when each problem locks in, and what thinking in vectors costs.
TL;DR
A looped transformer reuses one small block of shared layers many times, so depth becomes a setting you choose at run time. Huginn-0125 loops a 4-layer core up to 132 effective layers inside 7.13 GB. Easy facts settle early, carry arithmetic needs about 8 loops, and the hidden state converges to a fixed point, which gives a natural rule for when to stop.
- 8 → 132effective layers
- 7.13 GBfixed memory
- 93.80%carry solved at T = 8
- 300×collapse without the prompt anchor
- 195.4 vs 995.3 ms32 loops vs 32 CoT tokens
Summary
A standard transformer's depth is fixed by how many distinct layers it stores. A looped transformer reuses a small block of shared layers again and again, so depth becomes a dial you turn at inference time. This article runs Huginn-0125, a 3.56B-parameter open model from the University of Maryland, on an A100 GPU and watches what happens inside. It has a 2-layer Prelude, a 4-layer core that repeats, and a 2-layer Coda. Looping the core 32 times yields 132 effective layers while memory stays at 7.13 GB. For contrast it also looks at Nanbeige 4.2, which simply runs its whole 22-layer stack twice.
Reading the Coda's prediction after each loop shows that different problems need different depth. A capital-city fact peaks early, at loop 6. Two-digit addition with a carry fails for the first few loops, then jumps to 93.80% at loop 8 and 99.73% at loop 16, and a doubling pattern locks in at loop 12. The hidden state settles toward a fixed point: the change between steps falls from 0.9472 to 0.0145 by loop 32, so a simple stop-when-the-change-is-small rule halts at loop 21 and skips 34.4% of the core work. Removing the re-injected prompt embedding at loop 2 makes confidence collapse about 300 times in a single step.
Thinking in vectors also carries far more information than thinking in words: each 5,280-number state holds 84,480 bits against 16 bits for a sampled token. Thirty-two latent loops took 195.4 ms against 995.3 ms for 32 chain-of-thought tokens, although at matched compute the gap shrinks to about 1.27x. Looping saves memory, not arithmetic.
This page is a short summary written for this site. Every figure, table, code sample and reference is in the original article on Towards AI.
Questions this article answers
What is a looped transformer?
It is a transformer that applies the same block of layers repeatedly instead of stacking many different ones. Tokens are still processed in parallel with attention, but the model iterates on its hidden state vertically, through depth. Because the weights are shared, you can choose more loops for harder inputs at inference time without storing any extra layers, trading extra compute for extra reasoning depth.
How does Huginn-0125 reach 132 layers in only 7.13 GB of GPU memory?
Only the middle block repeats. Huginn-0125 has a 2-layer Prelude that encodes the prompt, a shared 4-layer core, and a 2-layer Coda that produces the answer. Running the core 32 times gives 2 plus 128 plus 2, or 132 effective layers, with the same weights in memory the whole time. A conventional model would need separate weights for every one of those layers.
How does a looped transformer know when to stop thinking?
It can watch its own hidden state. In the article's trace the relative change between consecutive loops shrank steadily as the state approached a fixed point. Stopping once that change fell below 5% halted at loop 21 instead of 32, saving 34.4% of the core computation, with no access to the correct answer. Huginn is trained with random loop counts so this kind of convergence is learned.
Is latent reasoning faster than chain-of-thought?
In this test, 32 latent loops took 195.4 ms while generating 32 chain-of-thought tokens took 995.3 ms, about 5x slower. That comparison is not fully fair, because the token run also did four times as many core passes. Matching the compute, the advantage drops to roughly 1.27x. The bigger difference is that latent steps add no tokens to the KV cache, while visible reasoning stays auditable.
How is a looped transformer different from an RNN?
An RNN loops sideways across the sequence, reading one token after another, which prevents parallel processing of the prompt. A looped transformer handles all tokens at once with attention and loops upward through computation depth instead. Both reuse weights, but they repeat along different axes, which is why looped transformers keep the GPU-friendly parallelism that made transformers replace RNNs.