A recurrent path across the sequence
A new language-model architecture proposal called the Recurrent Looped Transformer, or RLT, describes a way to carry one continuous computation from a prompt through every token in a response. The design pairs a causal encoder with a recurrent decoder, retaining the decoder's final hidden state and its layer-specific sliding-window attention cache as generation proceeds.
The project authors present the system as a route to latent reasoning with an extensible temporal depth. That phrase does not mean an unlimited amount of computation occurs for each token. Instead, every new token lengthens the total recurrent path through the sequence, while the amount of decoder work assigned to an individual token remains fixed. In the concrete configuration described by the project, the encoder and decoder each have 48 layers. After a sequence reaches t tokens, its recurrent path has passed through 48 times t decoder blocks.
The encoder creates a global key-value memory from known tokens. The decoder then processes tokens in order, consulting that memory and maintaining a recent cache at each sliding-window attention layer. The previous decoder output feeds the next update. This structure is intended to remove a structural break between prompt processing and response generation, allowing both to participate in the same state transition.
Training and execution requirements
The proposal also outlines execution techniques intended to make the architecture practical. Known prompt tokens can be processed in a causal batch, and independent encoder work can run in parallel. Weight and memory reuse, sequence batching and activation checkpointing are positioned around the recurrent core. Decoder updates still have to occur in token order because each update constructs state needed by the next one.
Training exactness imposes additional constraints. Full backpropagation through time must preserve gradient paths through recurrent outputs, decoder caches and encoder memory; detaching any of those paths changes the gradient. After a parameter update, previously computed caches are no longer exact representations of the current policy. The authors therefore say current-policy replay must rebuild the complete history, including prompt states and decoder attention caches, while behavior probabilities must correspond to the sampler that actually produced the tokens.
The publication is an architecture proposal, not evidence that the approach has already delivered better reasoning or lower hardware costs. Its authors explicitly leave realized reasoning gains, hardware efficiency and reinforcement-learning scaling to future validation. The immediate contribution is a specific, testable design for making a transformer's computational history grow with its generated sequence without increasing the number of logical blocks executed for each new token.
