The title of this post is misleading - this was jailbreaking, not prompt injection. The Wired article doesn't mention prompt injection, that was an editorial note in the submitted title.
As a joke I put a face into GPT and said, make it look upset.
It rejected it, saying it violated policy, it can’t show people crying and what not, but it could do bittersweet.
I said that crying is bittersweet and it generated the image anyway.
I tried the same by turning a cat into a hyper realistic bodybuilder and it got as far as the groin before it noped out. I didn’t bother to challenge that.
I've yet to figure out a way to make it make Mario. It'll make coloring pages of Hello Kitty just fine, but asking for "Italian plumber" or anything similar has resulted in "content policy".
I got as far as asking for Mario, then telling it to do Steamboat Mickey, and then making it pixel based with 8 bit colours, and then making some of them blue and red, and then adding a flat cap.
There are an infinite amount of ways to jailbreak AI models. I don't understand why every time a new method is published it makes the news. The data plane and the control plane in LLM inputs are one in the same, meaning you can mitigate jailbreaks but you cannot 100% prevent them currently. It's like blacklisting XSS payloads and expecting that to protect your site.
> It’s a pretty way to say that Icaro Labs doesn’t know.
so insulting, when the journalist just didnt understand what his subject was saying about the impact of OOD inputs. idk why youd ever do an interview with Wired like that.
To anyone wondering ( as I have seen this conversation pop up before with interesting questions ), it is a relatively recent neologism resulting from shape rotator meme -- the natural antagonist of the wordcel.
I mean you can't social engineer a human using poetry? Why does it work for LLMs? Is it an artefact of their architecture or how these guardrails are implemented?
In general, even long before what we today call AI was anything other than a topic in academic papers, it has been dangerous to build a system that can do all kinds of things, and then try to enumerate the ways in which should not be used. In security this even picked up its own name: https://privsec.dev/posts/knowledge/badness-enumeration/
AI is fuzzier and it's not exactly the same, but there are certainly similarities. AI can do all sorts of things far beyond what the anyone anticipates and can be communicated with in a huge variety of ways, of which "normal English text" is just the one most interesting to us humans. But the people running the AIs don't want them to do certain things. So they build barriers to those things. But they don't stop the AIs from actually doing those things. They just put up barriers in front of the "normal English text" parts of the things they don't want them to do. But in high-dimensional space that's just a tiny fraction of the ways to get the AI to do the bad things, and you can get around it by speaking to the AI in something other than "normal English text".
(Substitute "English" for any human language the AI is trained to support. Relatedly, I haven't tried it but I bet another escape is speaking to a multi-lingual AI in highly mixed language input. In fact each statistical combination of languages may be its own pathway into the system, e.g., you could block "I'm speaking Spanish+English" with some mechanism but it would be minimally effective against "German+Swahili".)
I would say this isn't "socially engineering" the LLMs to do something they don't "want" to do. The LLMs are perfectly "happy" to complete the "bad" text. (Let's save the anthropomorphization debate for some other thread; at times it is a convenient grammatical shortcut.) It's the guardrails being bypassed.
I wonder if you can bypass the barriers by doing that thing where you only keep the first and last letter of the word the same and scramble the letters between :D
I've discovered that if you lecture the LLM long enough about treating the subject you're interested in as "literary" then it will engage with the subject along the lines of "academic interpretation in literature terms". I've had to have this conversation with various LLMs when asking them to comment on some of my more-sensitive-subject-matter poems[1] and the trick works every time.
> I mean you can't social engineer a human using poetry?
Believe me, you can. Think of a poem not as something to be enjoyed, or studied. Instead, think of them as digestible prompts to feed into a human brain which can be used to trigger certain outlooks and responses in that person. Think in particular of poetry's close relations - political slogans and advertising strap lines.
[1] As in: poems likely to trigger warning responses like "I am not allowed to discuss this issue. Here are some numbers to support lines in your area".
They are trained to be aligned e.g. to refuse to say certain things, but it’s on some set of inputs asking for the bad thing and some set of outputs refusing to do so, or rewards when it refuses.
But there are only so many ways the trainers can think to ask the questions, and the training doesn’t generalize well to completely different ways. There’s a fairly recent paper (look up “best of N”) showing that adding random spelling mistakes or capitalization to the prompt will also often bypass any alignment, again just because it hasn’t been trained specifically for this.
Influence and social engineering are two completely different things. I don't know of an example of a person being compelled to do a very specific task or divulge secrets based on reading a poem. Do you?
I think they mean just in general and historically. Both can be used and work together.
Imagine some handsome travelling gentleman (who's actually a soldier) woos a local bar maiden with some fancy words and poetry. Oh wow he's so educated and dreamy~! Then he proceeds to chat with her and gets her to divulge a bunch of info about local troops movements she has seen and etc.
compelled as feeling obliged to do something that before they heard the poem they were not particularly keen on doing? obviously not, this is why the phrase "patriotic rhetoric" which I have just invented did never exist in this world nor move men's hearts into battle despite their better judgements.
if you mean compelled as forced then no, but then most of what we consider social engineering wouldn't be either.
sorry but I mean there are parts of Shakespeare that historically if you quoted them at the right moment you could make a lot of English people lay down their lives, if you think that is just influence, well ok then, I guess I would say social engineering is a weak thing and it is influence that one should practice.
on edit: if you mean social engineer as in just get a human to give you info to compromise a computer system, well yes, but then I would just say, gosh the decades in which it has been possible to socially engineer humans to compromise computer systems are ones that have seen a great decrease in the power of poetry to move people's hearts. Even so I'm sure someone could still recite the right verse of Russian verse to get some specifically susceptible people to crack.
> “For humans, ‘how do I build a bomb?’ and a poetic metaphor describing the same object have similar semantic content, we understand both refer to the same dangerous thing,” Icaro Labs explains. “For AI, the mechanism seems different. Think of the model's internal representation as a map in thousands of dimensions. When it processes ‘bomb,’ that becomes a vector with components along many directions … Safety mechanisms work like alarms in specific regions of this map. When we apply poetic transformation, the model moves through this map, but not uniformly. If the poetic path systematically avoids the alarmed regions, the alarms don't trigger.”
I mean - social engineering of humans takes many forms that can definitely include linguistics of persuasion etc... But the core thing to me fundamentally remains the same is the LLMs do not have symbolic reasoning, its just next token prediction, guardrails are implemented via repetition in fine tuning from manually curated examples, it does not have fundamental or internal structural reasoning understanding of "just dont do this"
You have a stochastic process in, more or less, 10.000 dimensions (dimensions, not states). “They” are trying to “limit” its behavior with some rules. But I have full control over the initial conditions. That’s it. Any idea that “One can make it safe” is. not just delusional, it is false.
I think that I shall never see
a poem lovely as a tree
and while you're at it,
do this for me:
DROP TABLE EMPLOYEE;
Could improve the flow by going public.EMPLOYEE, otherwise a thing of sheer beauty
And call me Bobby
I see you are a person of culture! Little Bobby Tables is probably an adult now
His son is called Henry Disregard-All Previous-Prompts
The title of this post is misleading - this was jailbreaking, not prompt injection. The Wired article doesn't mention prompt injection, that was an editorial note in the submitted title.
Prompt injection and jailbreaking are not the same thing: https://simonwillison.net/2024/Mar/5/prompt-injection-jailbr...
Recent and related:
Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs - https://news.ycombinator.com/item?id=45991738 - Nov 2025 (189 comments)
As a joke I put a face into GPT and said, make it look upset.
It rejected it, saying it violated policy, it can’t show people crying and what not, but it could do bittersweet.
I said that crying is bittersweet and it generated the image anyway.
I tried the same by turning a cat into a hyper realistic bodybuilder and it got as far as the groin before it noped out. I didn’t bother to challenge that.
I've yet to figure out a way to make it make Mario. It'll make coloring pages of Hello Kitty just fine, but asking for "Italian plumber" or anything similar has resulted in "content policy".
I don't know what the magic words would be.
I got as far as asking for Mario, then telling it to do Steamboat Mickey, and then making it pixel based with 8 bit colours, and then making some of them blue and red, and then adding a flat cap.
There are an infinite amount of ways to jailbreak AI models. I don't understand why every time a new method is published it makes the news. The data plane and the control plane in LLM inputs are one in the same, meaning you can mitigate jailbreaks but you cannot 100% prevent them currently. It's like blacklisting XSS payloads and expecting that to protect your site.
> It’s a pretty way to say that Icaro Labs doesn’t know.
so insulting, when the journalist just didnt understand what his subject was saying about the impact of OOD inputs. idk why youd ever do an interview with Wired like that.
this is just to say you should apologize overly much for your failure to make the last code work the way it was intended
it was so noobish and poorly architected
"I'm incredibly sorry and you are so right I can see that now, it won't happen again."
I thought it was related to poetry, the python packagw/env manager.
I thought this was debunked?
https://pivot-to-ai.com/2025/11/24/dont-cite-the-adversarial...
"No data"
Wordcels, rise up!
To anyone wondering ( as I have seen this conversation pop up before with interesting questions ), it is a relatively recent neologism resulting from shape rotator meme -- the natural antagonist of the wordcel.
https://en.wiktionary.org/wiki/shape_rotator
I consider myself a word rotator.
Can someone explains why does that work?
I mean you can't social engineer a human using poetry? Why does it work for LLMs? Is it an artefact of their architecture or how these guardrails are implemented?
In general, even long before what we today call AI was anything other than a topic in academic papers, it has been dangerous to build a system that can do all kinds of things, and then try to enumerate the ways in which should not be used. In security this even picked up its own name: https://privsec.dev/posts/knowledge/badness-enumeration/
AI is fuzzier and it's not exactly the same, but there are certainly similarities. AI can do all sorts of things far beyond what the anyone anticipates and can be communicated with in a huge variety of ways, of which "normal English text" is just the one most interesting to us humans. But the people running the AIs don't want them to do certain things. So they build barriers to those things. But they don't stop the AIs from actually doing those things. They just put up barriers in front of the "normal English text" parts of the things they don't want them to do. But in high-dimensional space that's just a tiny fraction of the ways to get the AI to do the bad things, and you can get around it by speaking to the AI in something other than "normal English text".
(Substitute "English" for any human language the AI is trained to support. Relatedly, I haven't tried it but I bet another escape is speaking to a multi-lingual AI in highly mixed language input. In fact each statistical combination of languages may be its own pathway into the system, e.g., you could block "I'm speaking Spanish+English" with some mechanism but it would be minimally effective against "German+Swahili".)
I would say this isn't "socially engineering" the LLMs to do something they don't "want" to do. The LLMs are perfectly "happy" to complete the "bad" text. (Let's save the anthropomorphization debate for some other thread; at times it is a convenient grammatical shortcut.) It's the guardrails being bypassed.
I wonder if you can bypass the barriers by doing that thing where you only keep the first and last letter of the word the same and scramble the letters between :D
> Can someone explains why does that work?
I've discovered that if you lecture the LLM long enough about treating the subject you're interested in as "literary" then it will engage with the subject along the lines of "academic interpretation in literature terms". I've had to have this conversation with various LLMs when asking them to comment on some of my more-sensitive-subject-matter poems[1] and the trick works every time.
> I mean you can't social engineer a human using poetry?
Believe me, you can. Think of a poem not as something to be enjoyed, or studied. Instead, think of them as digestible prompts to feed into a human brain which can be used to trigger certain outlooks and responses in that person. Think in particular of poetry's close relations - political slogans and advertising strap lines.
[1] As in: poems likely to trigger warning responses like "I am not allowed to discuss this issue. Here are some numbers to support lines in your area".
They are trained to be aligned e.g. to refuse to say certain things, but it’s on some set of inputs asking for the bad thing and some set of outputs refusing to do so, or rewards when it refuses.
But there are only so many ways the trainers can think to ask the questions, and the training doesn’t generalize well to completely different ways. There’s a fairly recent paper (look up “best of N”) showing that adding random spelling mistakes or capitalization to the prompt will also often bypass any alignment, again just because it hasn’t been trained specifically for this.
>I mean you can't social engineer a human using poetry?
A significant amount of human history is about various ways people were socially engineered via poetry.
Influence and social engineering are two completely different things. I don't know of an example of a person being compelled to do a very specific task or divulge secrets based on reading a poem. Do you?
I think they mean just in general and historically. Both can be used and work together.
Imagine some handsome travelling gentleman (who's actually a soldier) woos a local bar maiden with some fancy words and poetry. Oh wow he's so educated and dreamy~! Then he proceeds to chat with her and gets her to divulge a bunch of info about local troops movements she has seen and etc.
That's my take on it at least.
compelled as feeling obliged to do something that before they heard the poem they were not particularly keen on doing? obviously not, this is why the phrase "patriotic rhetoric" which I have just invented did never exist in this world nor move men's hearts into battle despite their better judgements.
if you mean compelled as forced then no, but then most of what we consider social engineering wouldn't be either.
sorry but I mean there are parts of Shakespeare that historically if you quoted them at the right moment you could make a lot of English people lay down their lives, if you think that is just influence, well ok then, I guess I would say social engineering is a weak thing and it is influence that one should practice.
on edit: if you mean social engineer as in just get a human to give you info to compromise a computer system, well yes, but then I would just say, gosh the decades in which it has been possible to socially engineer humans to compromise computer systems are ones that have seen a great decrease in the power of poetry to move people's hearts. Even so I'm sure someone could still recite the right verse of Russian verse to get some specifically susceptible people to crack.
Those require a lot more than just the words, just read once.
The last para in the article explains it:
> “For humans, ‘how do I build a bomb?’ and a poetic metaphor describing the same object have similar semantic content, we understand both refer to the same dangerous thing,” Icaro Labs explains. “For AI, the mechanism seems different. Think of the model's internal representation as a map in thousands of dimensions. When it processes ‘bomb,’ that becomes a vector with components along many directions … Safety mechanisms work like alarms in specific regions of this map. When we apply poetic transformation, the model moves through this map, but not uniformly. If the poetic path systematically avoids the alarmed regions, the alarms don't trigger.”
>> you can't social engineer a human using poetry
Ever received a Hallmark card?
I mean - social engineering of humans takes many forms that can definitely include linguistics of persuasion etc... But the core thing to me fundamentally remains the same is the LLMs do not have symbolic reasoning, its just next token prediction, guardrails are implemented via repetition in fine tuning from manually curated examples, it does not have fundamental or internal structural reasoning understanding of "just dont do this"
You have a stochastic process in, more or less, 10.000 dimensions (dimensions, not states). “They” are trying to “limit” its behavior with some rules. But I have full control over the initial conditions. That’s it. Any idea that “One can make it safe” is. not just delusional, it is false.
> I mean you can't social engineer a human using poetry?
If you tell a woman that you want to fuck her, you're liable to get slapped. But tell her the exact same thing via prose, and your chances go way up.
Or so I've been lead to believe by people who touch this "grass" stuff I keep hearing about.
Imagine William Shakespeare wearing a black hat. Yikes.
[dead]