The samples are striking in their simplicity compared to the over-controlled system prompts we have now. For example, the intended format is as follows:
"GPT‑2 generates synthetic text samples in response to the model being primed with an arbitrary input. The model is chameleon-like—it adapts to the style and content of the conditioning text."
The results don't sounds anything like today's models but note that each one took 10 attempts:
> ## System Prompt (human-written)
> In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
> ## Model Completion (machine-written, 10 tries)
> The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.
> Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
> Dr. Jorge Pérez, an evolutionary biologist [...]
I still think this was a fire drill. They knew it wasn't dangerous but they wanted the public to do their homework and come to the conclusion themselves.
You don't help your argument or convince people of your position by commenting on headlines. I'm in agreement that the modern wave of fearmongering is just a marketing tactic. Even so, that's not what this article is doing. The article wasn't saying it's too dangerous and going to kill us all, they were saying it's going to ruin the internet by making it trivial to spam coherent, deceptive text at scale. Which is true. The world would have been a better place if they didn't release this model or any one thereafter.
The public discourse was very different then. Misinformation, bias, etc. Those in power were much more concerned about an LLM saying something unapproved.
I'm starting to think that AI development might be the great filter. There are so many levels of competition at play here and even if an agreement is reached to slow everything down, the incentive is to cheat at that. I really don't think there's much of a chance we have control over the situation, we're just gonna summon these things and all we can do is pray they don't want to kill us.
But why are Dario, Sam and Elon who represent "the frontier" more than any other 3 people pretending like they need government permission or intervention to pause? It seems more likely that they see open source distillation as an existential threat to their IPO plans and are playing for regulatory capture of the market.
It would require government permission. The top labs would essentially be agreeing not to compete on model advancements for a period of time, which seems like a pretty clear violation of antitrust law.
The samples are striking in their simplicity compared to the over-controlled system prompts we have now. For example, the intended format is as follows:
"GPT‑2 generates synthetic text samples in response to the model being primed with an arbitrary input. The model is chameleon-like—it adapts to the style and content of the conditioning text."
The results don't sounds anything like today's models but note that each one took 10 attempts:
> ## System Prompt (human-written)
> In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
> ## Model Completion (machine-written, 10 tries)
> The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.
> Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
> Dr. Jorge Pérez, an evolutionary biologist [...]
I still think this was a fire drill. They knew it wasn't dangerous but they wanted the public to do their homework and come to the conclusion themselves.
You don't help your argument or convince people of your position by commenting on headlines. I'm in agreement that the modern wave of fearmongering is just a marketing tactic. Even so, that's not what this article is doing. The article wasn't saying it's too dangerous and going to kill us all, they were saying it's going to ruin the internet by making it trivial to spam coherent, deceptive text at scale. Which is true. The world would have been a better place if they didn't release this model or any one thereafter.
So they lied to the public then?
Wow, we should really trust these people today.
I wonder what they saw. GPT-2 has no malware applications. Maybe they were concerned about someone using it for spam?
The public discourse was very different then. Misinformation, bias, etc. Those in power were much more concerned about an LLM saying something unapproved.
Agree. It was more a worry that someone would create a bot posting lies and bigoted ideas to Facebook/X etc.
I'm starting to think that AI development might be the great filter. There are so many levels of competition at play here and even if an agreement is reached to slow everything down, the incentive is to cheat at that. I really don't think there's much of a chance we have control over the situation, we're just gonna summon these things and all we can do is pray they don't want to kill us.
This is what people were saying about nuclear technology half a century ago.
And? Nuclear technology continues to be an excellent candidate for the Great Filter.
See also the 2023 call to “ pause for at least 6 months the training of AI systems more powerful than GPT-4.” https://futureoflife.org/open-letter/pause-giant-ai-experime...
Signed by Benjio, Musk, Woz, etc
If it is possible to save one life, preventing RSI AGI is an existential threat.
But why are Dario, Sam and Elon who represent "the frontier" more than any other 3 people pretending like they need government permission or intervention to pause? It seems more likely that they see open source distillation as an existential threat to their IPO plans and are playing for regulatory capture of the market.
It would require government permission. The top labs would essentially be agreeing not to compete on model advancements for a period of time, which seems like a pretty clear violation of antitrust law.
... is what they believe.