Anyone remember Universal Transformers paper (Dehghani et al) from back in 2018? Recurrent transformers have a history as long as transformers themselves.
Somewhat unclear how particularly novel this is vs a way to save compute.
>In contrast to a classic RNN, there's no unbounded hidden state accumulating across an entire trajectory
I don' understand this line. In a classic RNN hidden state is bounded dimension. In fact it's transformers that technically have unbounded hidden state.
You can't parallelize classic nonlinear RNNs for various reasons but in training both RNN and Transformer depend on the entire sequence history in a way that is unbounded. Of course in practice you just train on a max sequence length.
it's just an experimental optimization. Implementation detail...
Irrelevant to "safety". I mean its going to have to go in that direction anyway... eventually the models will just be constantly thinking, refining their internal thoughts / weights... External input and output will be rare, just as it is for most humans.
I like the idea of more reccurance in the transformer level. Chain of thought always seemed so clunky. Its just not the way the human brain processes information. Its an extrmeely crude approximation at best
>This suggests that deeper isn't always better for looped transformers, which leaves me less worried about a race to the bottom toward looped transformers with hundreds of recurrent loops.
I disagree with this. Deeper will always be at least as good because the extra loops can exit early or just no-op. Any performance degradation they're seeing at higher loop counts today is merely training stability issues, which can be overcome.
Deeper almost certainly is better, and we will probably see not just hundreds but millions of recurrent loops in the future.
> A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".
> It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.
> About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."
> Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.
> In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.
> In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.
> Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)
> While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.
> In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.
To add a bit more constructive feedback, think of the "AI cornucopia" and "Superintelligence destroys humanity" as opposite ends of all possible outcomes distribution (low probability event).
The most likely one is the "business as usual, but with AI" - some things will get better, some things will get worse, but overall state of affairs will remain mostly the same.
Anyone remember Universal Transformers paper (Dehghani et al) from back in 2018? Recurrent transformers have a history as long as transformers themselves.
Somewhat unclear how particularly novel this is vs a way to save compute.
>In contrast to a classic RNN, there's no unbounded hidden state accumulating across an entire trajectory
I don' understand this line. In a classic RNN hidden state is bounded dimension. In fact it's transformers that technically have unbounded hidden state.
You can't parallelize classic nonlinear RNNs for various reasons but in training both RNN and Transformer depend on the entire sequence history in a way that is unbounded. Of course in practice you just train on a max sequence length.
RNN xhat[t+1]=f(x[t],h[t])
Transformer/self-attention xhat[t+1]=f(x[t],h[t],h[t-1],...,h[1])
On further thought, I think the author's intent was to say that classic RNNs have "unbounded temporal accumulation in the hidden state".
it's just an experimental optimization. Implementation detail... Irrelevant to "safety". I mean its going to have to go in that direction anyway... eventually the models will just be constantly thinking, refining their internal thoughts / weights... External input and output will be rare, just as it is for most humans.
I like the idea of more reccurance in the transformer level. Chain of thought always seemed so clunky. Its just not the way the human brain processes information. Its an extrmeely crude approximation at best
it is what I do to solve hard problems through.
easy stuff happens by itself, but with a system large enough you need a scratchpad and a rubber duck.
>This suggests that deeper isn't always better for looped transformers, which leaves me less worried about a race to the bottom toward looped transformers with hundreds of recurrent loops.
I disagree with this. Deeper will always be at least as good because the extra loops can exit early or just no-op. Any performance degradation they're seeing at higher loop counts today is merely training stability issues, which can be overcome.
Deeper almost certainly is better, and we will probably see not just hundreds but millions of recurrent loops in the future.
Sebastian Raschka posted about this architecture:
> A lot of hype around OpenAI's Astra model here on my timeline today. Apparently, this goes back to a new article from The Information, which said Astra is a "recurrent depth or looped transformer".
> It's always interesting to read about new or different approaches (including rumors about what the closed labs may be up to), but let's debunk this a bit.
> About 2 months ago, I shared the architecture details of Nanbeige, for example, where "Nanbeige4.2-3B is pretrained from scratch on 28T tokens with a Looped Transformer that reuses the layer stack to increase capacity without adding parameters."
> Yes, that's it. The looped transformer idea is just reusing layers in the transformer block.
> In the case of Nanbeige, the main idea is to reuse the same 22-layer stack (=transformer block) twice instead of once. So, effectively it extends the 22-layer architecture to 44 layers, but without duplicating the weights.
> In simple terms, this roughly doubles the size of the model (if we ignore the embedding and output layers for a second). But instead of requiring 2x the storage and RAM to host this model, it stays at the same size since we reuse the components. However, it's almost 2x as expensive in terms of compute, because we run the embedded text through almost 2x as many layers.
> Why? In the Nanbeige 4.2 technical report, the researchers found that two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. (More passes gave barely any gains but made the training much slower and much more expensive.)
> While, as far as I know, Nanbeige 4.2 is the first notable open-weight model that adopted this approach, the idea goes back to the NeurIPS paper "Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation". Actually, this paper proposes a mechanism that is a bit more sophisticated by adding a learned router that determines whether each token receives one, two, or more passes. So, easy tokens can exit early while harder tokens receive additional computation.
> In sum, Astra may be a really good model, but this shouldn't be about this "looped transformer aspect," which is just a tiny architectural tweak.
https://x.com/rasbt/status/2095141254958858496
Related ongoing thread:
OpenAI's new reasoning technique alarms AI safety experts - https://news.ycombinator.com/item?id=49552395
A debate between grossly incompetent against grossly immoral, honestly.
You should ignore anything LessWrong or OpenAI says and do your own research.
To add a bit more constructive feedback, think of the "AI cornucopia" and "Superintelligence destroys humanity" as opposite ends of all possible outcomes distribution (low probability event).
The most likely one is the "business as usual, but with AI" - some things will get better, some things will get worse, but overall state of affairs will remain mostly the same.