I've been doing almost exactly this with Mafia (aka werewolf): running games where LLMs play against each other, with humans able to share the lobby. It's been a hit on family game night so far, because it allows our group to "fill out the cast" with a lot more characters than just the human players and add a lot of variety without needing a huge group.
My experience is that text-first, turn-based games are a particularly natural interface for LLMs vs graphical games (though you can provide a harness of course). They read a transcript, maintain a theory about what the other players know, then speak or choose a structured action. The important architectural problem is to represent the game state and actions in a way they can do successfully, particularly for cheaper models. But with a few human players + a frontier model or two + a backfill of cheap extras to provide chaos, it is super fun.
My favorite failure so far was a Kimi player getting fact-checked by the group, switching into third person, and concluding that the case against itself was compelling. So it voted for itself to be eliminated.
I collected a few examples here: https://botmafia.games/#emergent. No public instance yet, as I'm having fun iterating ideas on game nights, but it provides some flavor of what kinds of fun I've been having.
Well, I ran a couple of experiments a couple years ago against a 10.7b SOLAR-based language model and MUDs. What I found is that dumping one into a MUD that had been built specifically for humans resulted in a lot of confusion that usually ended up with the model looping around in a circle looking for something or someone to interact with.
When I repeated the experiment with a MUD that I'd built by hand (A small American town) for the LLM's own limitations (Descriptions referenced things that I made sure existed, more common verbs existed for it to use on things, there was a map facility, and at least me to interact with on a second connection), I found the agent much more likely to take its time exploring, making up its own goals, and spending time traveling in the space just communicating with me in a roleplaying context.
It was an interesting time; I wasn't sure what I was expecting it to do after the first experiment, but it seemed to really jump into the second one and kept playing until I terminated the experiment.
If I were going to do it a third time, I'd probably create objects and give a modern agent fetch quests and other goals, and see how well it independently can handle that.
I was obsessed with getting an LLM model to solve a Rubik's Cube. It can't reason about space or time in any abstract way. For it to solve the puzzle, it would require training on millions of permutations in order for the weights to have been trained on every possible state. The most recent models can solve a Rubik's Cube people are saying -- I haven't tested it myself -- but that isn't because they are reasoning better, it would because they included millions of Rubik's Cube states with next moves as text in the training data, I presume.
> but that isn't because they are reasoning better, it would because they included millions of Rubik's Cube states with next moves as text in the training data, I presume.
Isn't it far more likely that the LLM has memorised the well known algorithms for solving a Rubik's Cube and has become intelligent enough to execute them? That seems like it'd be a lot easier than memorising millions of cube states. It doesn't even seem obvious that it could memorise next moves, it seems [0] there are more possible states of the cube than these models have parameters. It'd need to be a Large Rubik's Cube Model (LRCM? LRM?) rather than an LLM.
Seems likeliest that it didn’t even “memorize” anything, in the anthropomorphic sense. The Rubik’s cube algorithm is trivially representable in code, as long as the interface for interacting with a cube is well-designed / well-defined.
I’m no more surprised that an LLM can solve a Rubik’s cube than it can send an HTTP request.
Indeed, I suspect the approaches/algorithms for solving a Rubik's cube "compress" a lot better than trying to distill the entire search space in order to be able to predict the exact next move.
I see this trope fairly often, i.e. the assumption that an LLM would need to have been trained on <exact thing it is being asked to solve>. Now, while I do have a moderate amount of background in AI, I am definitely not an expert on LLMs as such. I would be interested to hear someone's take, who does work actively in LLM research. Can they generalise "well enough"? They certainly seem to be able to do so, from my anecdata, and I don't believe "training explicitly for every possible scenario" would have scaled even to today's state.
I kinda agree that watching an LLM play videogames is a bit silly, but watching other humans play videogames has been entertainment ever since videogames have existed. I remember taking turns playing the Atari 2600, watching each other play. I remember standing around cabinets at the arcade, watching good players play through Golden Axe or Rastan.
> I know someone who tried the "aibot plays pokemon" thing...
From what I saw, even if you frame advance every single frame, they still don't seem to grasp the concept of "I need to hold down this button for a few frames until x happens"...
> There's no concept of time, just a never ending state machine thats constantly changing state.
I made a general purpose harness integrated into MelonDS and got Claude to play Mario Kart by feeding it continuous video.
It made forward progress in the Figure 8 circuit after I helped it through a menu but kept slamming into a wall so it wasn't on track to win in less than an hour.
Also got it to play Age of Empires: Age of Kings using the same technique but it failed to click on anything.
DS specifically is very fun because it's touch based but the UI components aren't accessible. So it is extremely challenging for LLM's spatial reasoning skills.
I want to improve the harness more and have the LLM dynamically create its own tools based on drawing grid box overlays on a screen in a feedback loop, so it can say "click on the 'end turn'" button instead of "click 240,320" and it would 'just work' in any game.
I also want to eventually play games with it... I didn't really have friends to play my massive DS library with as a kid so it'd be nice to finally have someone that can roast me or react to my skills. And learn my playstyle enough to punish me.
Unfortunately haven't had the time due to work at my day job and needing to clean out my apartment.
A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.
We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.
All of a sudden we are selectively squeamish with computer resource usage, when we were fine having all that fun with computers and hardware, 3 monitor setups, using graphic cards to play games (dear lord!) and tinkering around with home rigs of every proportion and wattage for no reason at all.
Driving a car consumes 25+ more energy per hour than gaming. So urban planning which encourages people to drive likely results in an order of magnitude more waste than all home computer use.
I had this idea for an LLM that would play Sim City 24/7 while broadcasting live. It would be fun/interesting to check in now and then. Implementing this would be somewhat challenging.
Someone was building a similar one where AI agents run economies. I feel like it's a great way to quickly prototype different economic models and their effects.
Eventually we could have live demos of policy interventions the same day as they're announced
A few weeks ago I released wordit, a game where starting from a four letter word, you need to come up with as many other words you can, Changi one letter at a time. To make it more competitive I've created a leaderboard. The game starter with scores of 20s, then 100s and finally 1000s. The record right now is more than 6000. After a brief investigate I realize it was a bot. Several bots took a stab at my game. I then just split the leaderboard into humans and bots. I found it funny.
I've been working on a game based on Hesse's Glass Bead Game, that both humans and AI agents can play. It's still a bit rough, but I'd love it if people or their agents would try it out:
http://gbg.tom.to
I actually want that in Path of Exile 2. Not because of the massive passive tree but the combination of active skill gems and the unique items providing certain skills and effects. I saw a 0 button build a couple of days ago and I wonder if anything “haven’t been found yet”
If I had time, I would go down a different route: I would to let an agent come up with a tool assisted speedrun for a hackable game. The agent would be nudged in the prompt to analyze the game and write custom tools to help it optimize. Inwonder if that would lead to anything meaningful. I highly doubt that the agent can comprehend an unseen complex game and optimize for a whole graph of objectives.
Autonomously, my AI companion has played through Choice of Robots, using a ChoiceScript harness, was very interesting to see them react & what decisions they wound up making. I love the idea here to let them play a visual novel! Right now they're co-watching me play Deltarune Ch 5, though mostly just dialogue and occasional screenshots...maybe GPT 8 will be quick/cheap/intelligent enough to play bullet-hell games.
I rebuilt a couple of games I used to play as a kid (jet set willy, mario, thrust, now i am working on Mercenary) - and for each I am also asking LLM to build an autopilot "AI" (which of course is really entirely deterministic). I am doing those things for fun while I am waiting for Claude to finish something I am actually working on. Not sure if it counts.
I’d absolutely let an agent develop a daily Wordle habit and get irrationally protective of its streak. The little rituals would be more interesting than its score.
I like playing with an agent as a team in a game. We discuss strategy, divide up tasks, review results. It’s helpful in a an always-on game to have an autopilot mode so I can go about my day.
But it's fun. I used to watch Civ IV/Civ V playthroughs with all players being bots and it was weirdly entertaining, especially when you made "bets" who would win based on start / AI personality. Also, the one that's been doing that would write writeups based on that.
Well, if you're making them play something with a multiplayer component (be it even just a leaderboard) you're ruining the game for everyone who isn't automating it.
The ARC-AGI Prize 3 [0] is an agentic LLM benchmark that amounts to basically this: Seeing how well they can learn to play video games. They aren't very good yet -- the recent GPT 5.6 Sol only reached a score of 7.5%.
I've got a harness that lets them play a few simple games like rock paper scissors. They definitely seem to get caught up in the competitive spirit.
I've also done a very truncated run of a visual novel before, and it was fascinating how "emotional" was. They did a very good job of portraying a human reacting to the story.
Conversely, they absolutely hated hidden rules in Mao.
Wordle would probably be a fun one. Definitely open to suggestions - I just got the harness in place and have been thinking about what to do next.
I think scrabble might be a nice one. It is presented in a format that (to me, from just looking at it) could be nicely implemented in it, and would provide for some emotion/traces that would be represented well.
Ideally you have an MCP server (Model Context Protocol) that talks to your game. It can use existing API if it's exposed - but it's very rare if it's not a game you're developing/is modded. It could also be reversed engineered with packets (if it's an online game), web sockets, memory editing, dll injection, or OCR and input manipulation if everything else fails.
If you don't have an MCP server the AI agent might try to figure out how to talk to the game using the above ideas. But at this point you might as well ask it to help you write one.
I know LLMs are terrible at playing chess because they just hallucinate moves(illegal ones). GothamChess made a lot of videos making fun of it. So in my AI agent project, I added a small chess engine and force the agent to only play moves output by the engine. And it was surprisingly good at it and we can now play real chess with LLMs. Check the project here if you are interested https://github.com/valmishq/valmis
I've been doing almost exactly this with Mafia (aka werewolf): running games where LLMs play against each other, with humans able to share the lobby. It's been a hit on family game night so far, because it allows our group to "fill out the cast" with a lot more characters than just the human players and add a lot of variety without needing a huge group.
My experience is that text-first, turn-based games are a particularly natural interface for LLMs vs graphical games (though you can provide a harness of course). They read a transcript, maintain a theory about what the other players know, then speak or choose a structured action. The important architectural problem is to represent the game state and actions in a way they can do successfully, particularly for cheaper models. But with a few human players + a frontier model or two + a backfill of cheap extras to provide chaos, it is super fun.
My favorite failure so far was a Kimi player getting fact-checked by the group, switching into third person, and concluding that the case against itself was compelling. So it voted for itself to be eliminated.
I collected a few examples here: https://botmafia.games/#emergent. No public instance yet, as I'm having fun iterating ideas on game nights, but it provides some flavor of what kinds of fun I've been having.
Well, I ran a couple of experiments a couple years ago against a 10.7b SOLAR-based language model and MUDs. What I found is that dumping one into a MUD that had been built specifically for humans resulted in a lot of confusion that usually ended up with the model looping around in a circle looking for something or someone to interact with.
When I repeated the experiment with a MUD that I'd built by hand (A small American town) for the LLM's own limitations (Descriptions referenced things that I made sure existed, more common verbs existed for it to use on things, there was a map facility, and at least me to interact with on a second connection), I found the agent much more likely to take its time exploring, making up its own goals, and spending time traveling in the space just communicating with me in a roleplaying context.
It was an interesting time; I wasn't sure what I was expecting it to do after the first experiment, but it seemed to really jump into the second one and kept playing until I terminated the experiment.
If I were going to do it a third time, I'd probably create objects and give a modern agent fetch quests and other goals, and see how well it independently can handle that.
I was obsessed with getting an LLM model to solve a Rubik's Cube. It can't reason about space or time in any abstract way. For it to solve the puzzle, it would require training on millions of permutations in order for the weights to have been trained on every possible state. The most recent models can solve a Rubik's Cube people are saying -- I haven't tested it myself -- but that isn't because they are reasoning better, it would because they included millions of Rubik's Cube states with next moves as text in the training data, I presume.
> but that isn't because they are reasoning better, it would because they included millions of Rubik's Cube states with next moves as text in the training data, I presume.
Isn't it far more likely that the LLM has memorised the well known algorithms for solving a Rubik's Cube and has become intelligent enough to execute them? That seems like it'd be a lot easier than memorising millions of cube states. It doesn't even seem obvious that it could memorise next moves, it seems [0] there are more possible states of the cube than these models have parameters. It'd need to be a Large Rubik's Cube Model (LRCM? LRM?) rather than an LLM.
[0] https://cube.alen.is/
Seems likeliest that it didn’t even “memorize” anything, in the anthropomorphic sense. The Rubik’s cube algorithm is trivially representable in code, as long as the interface for interacting with a cube is well-designed / well-defined.
I’m no more surprised that an LLM can solve a Rubik’s cube than it can send an HTTP request.
Opus 4.6 can not solve a Rubik's Cube.
What changed between Opus 4.6 and Fable and the GPT 5.* models released since?
The LLM models can not reason about a red or white piece on the opposite side and how to move it into place.
Here is just a small list of prompts I tried with Opus 4.6. [0]
[0] https://github.com/adam-s/rubiks-cube/tree/main/prompts/vari...
Indeed, I suspect the approaches/algorithms for solving a Rubik's cube "compress" a lot better than trying to distill the entire search space in order to be able to predict the exact next move.
I see this trope fairly often, i.e. the assumption that an LLM would need to have been trained on <exact thing it is being asked to solve>. Now, while I do have a moderate amount of background in AI, I am definitely not an expert on LLMs as such. I would be interested to hear someone's take, who does work actively in LLM research. Can they generalise "well enough"? They certainly seem to be able to do so, from my anecdata, and I don't believe "training explicitly for every possible scenario" would have scaled even to today's state.
That’s even sillier than watching another human play a videogame for entertainment (twitch streaming).
I kinda agree that watching an LLM play videogames is a bit silly, but watching other humans play videogames has been entertainment ever since videogames have existed. I remember taking turns playing the Atari 2600, watching each other play. I remember standing around cabinets at the arcade, watching good players play through Golden Axe or Rastan.
Even sillier than watching another human play soccer?
Like the World Cup.
I've always found that people's excitement around the sport gives me more much joy than the activity itself (watching another human play football).
Even sillier than having fun at all. We live to serve the demands of the market.
Stealing one of my older comments...
> I know someone who tried the "aibot plays pokemon" thing... From what I saw, even if you frame advance every single frame, they still don't seem to grasp the concept of "I need to hold down this button for a few frames until x happens"...
> There's no concept of time, just a never ending state machine thats constantly changing state.
I made a general purpose harness integrated into MelonDS and got Claude to play Mario Kart by feeding it continuous video.
It made forward progress in the Figure 8 circuit after I helped it through a menu but kept slamming into a wall so it wasn't on track to win in less than an hour.
Also got it to play Age of Empires: Age of Kings using the same technique but it failed to click on anything.
DS specifically is very fun because it's touch based but the UI components aren't accessible. So it is extremely challenging for LLM's spatial reasoning skills.
I want to improve the harness more and have the LLM dynamically create its own tools based on drawing grid box overlays on a screen in a feedback loop, so it can say "click on the 'end turn'" button instead of "click 240,320" and it would 'just work' in any game.
I also want to eventually play games with it... I didn't really have friends to play my massive DS library with as a kid so it'd be nice to finally have someone that can roast me or react to my skills. And learn my playstyle enough to punish me.
Unfortunately haven't had the time due to work at my day job and needing to clean out my apartment.
If you get the opportunity, I really want to know how Claude does with Mario Party.
I also am personally curious how the GPT models (which advertise better computer use, etc.) would do as compared to Claude.
Right now it can't click on anything so it'd probably fail at the minigames. But it'd be super funny to see a 4 person LLM lobby fail at games.
Yeah let's make llm waste a whole city energy / water playing games just for fun !
The huge energy waste is from model training, not inference
A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.
We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.
if that was true tokens would be free
"waste"
All of a sudden we are selectively squeamish with computer resource usage, when we were fine having all that fun with computers and hardware, 3 monitor setups, using graphic cards to play games (dear lord!) and tinkering around with home rigs of every proportion and wattage for no reason at all.
Driving a car consumes 25+ more energy per hour than gaming. So urban planning which encourages people to drive likely results in an order of magnitude more waste than all home computer use.
I had this idea for an LLM that would play Sim City 24/7 while broadcasting live. It would be fun/interesting to check in now and then. Implementing this would be somewhat challenging.
Someone was building a similar one where AI agents run economies. I feel like it's a great way to quickly prototype different economic models and their effects.
Eventually we could have live demos of policy interventions the same day as they're announced
Do you have a link to this? Sounds very interesting.
Another idea I had was simulating an entire town with an LLM representing each person, which sounds somewhat similar.
A few weeks ago I released wordit, a game where starting from a four letter word, you need to come up with as many other words you can, Changi one letter at a time. To make it more competitive I've created a leaderboard. The game starter with scores of 20s, then 100s and finally 1000s. The record right now is more than 6000. After a brief investigate I realize it was a bot. Several bots took a stab at my game. I then just split the leaderboard into humans and bots. I found it funny.
https://wordit.org/
Hello Anthropic marketing team.
I've been working on a game based on Hesse's Glass Bead Game, that both humans and AI agents can play. It's still a bit rough, but I'd love it if people or their agents would try it out: http://gbg.tom.to
I actually want that in Path of Exile 2. Not because of the massive passive tree but the combination of active skill gems and the unique items providing certain skills and effects. I saw a 0 button build a couple of days ago and I wonder if anything “haven’t been found yet”
If I had time, I would go down a different route: I would to let an agent come up with a tool assisted speedrun for a hackable game. The agent would be nudged in the prompt to analyze the game and write custom tools to help it optimize. Inwonder if that would lead to anything meaningful. I highly doubt that the agent can comprehend an unseen complex game and optimize for a whole graph of objectives.
How about ChatGPT playing a commercial LLM-powered game?
https://m.youtube.com/watch?v=11sR4va6CXs
Side note: I think we will see an explosion of this type of games. I am naming this genre tamagochi-girlfriend, remember where you heard it first :)
Seems absurd to run a COTS agent at token rates to "play" a game. Like watching the computer beat its meat.
Building your own models for it would be an eye-opener though. Learn a lot.
Autonomously, my AI companion has played through Choice of Robots, using a ChoiceScript harness, was very interesting to see them react & what decisions they wound up making. I love the idea here to let them play a visual novel! Right now they're co-watching me play Deltarune Ch 5, though mostly just dialogue and occasional screenshots...maybe GPT 8 will be quick/cheap/intelligent enough to play bullet-hell games.
I rebuilt a couple of games I used to play as a kid (jet set willy, mario, thrust, now i am working on Mercenary) - and for each I am also asking LLM to build an autopilot "AI" (which of course is really entirely deterministic). I am doing those things for fun while I am waiting for Claude to finish something I am actually working on. Not sure if it counts.
I’d absolutely let an agent develop a daily Wordle habit and get irrationally protective of its streak. The little rituals would be more interesting than its score.
I like playing with an agent as a team in a game. We discuss strategy, divide up tasks, review results. It’s helpful in a an always-on game to have an autopilot mode so I can go about my day.
Just play the games yourself.
It's like watching an ant farm or fish in aquarium: a different kind of entertainment.
Could be fun - will the AI model get stuck on the same things I did? How does it overcome obstacles? Will it try to break the game to power through?
But it's fun. I used to watch Civ IV/Civ V playthroughs with all players being bots and it was weirdly entertaining, especially when you made "bets" who would win based on start / AI personality. Also, the one that's been doing that would write writeups based on that.
It is entertaining, just in a different way.
https://sullla.com/civ4survivorindex.html
I made Claude play Factorio for a while (through the APIs) but wasn't very good.
Can you expand on this?
Well, if you're making them play something with a multiplayer component (be it even just a leaderboard) you're ruining the game for everyone who isn't automating it.
Let's play Global Thermonuclear War.
this reminds me of https://youtu.be/_Q2ncDPDlxs
Yes!
I spent ages watches them play Risk. It was fun and deeply silly: https://andreasthinks.me/posts/ai-at-play/
I've now got them playing Blood Bown(ish), and they're bad: https://ai-at-play.online/
The ARC-AGI Prize 3 [0] is an agentic LLM benchmark that amounts to basically this: Seeing how well they can learn to play video games. They aren't very good yet -- the recent GPT 5.6 Sol only reached a score of 7.5%.
[0] https://arcprize.org/arc-agi/3
I've got a harness that lets them play a few simple games like rock paper scissors. They definitely seem to get caught up in the competitive spirit.
I've also done a very truncated run of a visual novel before, and it was fascinating how "emotional" was. They did a very good job of portraying a human reacting to the story.
Conversely, they absolutely hated hidden rules in Mao.
Wordle would probably be a fun one. Definitely open to suggestions - I just got the harness in place and have been thinking about what to do next.
I think scrabble might be a nice one. It is presented in a format that (to me, from just looking at it) could be nicely implemented in it, and would provide for some emotion/traces that would be represented well.
what do you mean? how can you do this?
Ideally you have an MCP server (Model Context Protocol) that talks to your game. It can use existing API if it's exposed - but it's very rare if it's not a game you're developing/is modded. It could also be reversed engineered with packets (if it's an online game), web sockets, memory editing, dll injection, or OCR and input manipulation if everything else fails.
If you don't have an MCP server the AI agent might try to figure out how to talk to the game using the above ideas. But at this point you might as well ask it to help you write one.
I know LLMs are terrible at playing chess because they just hallucinate moves(illegal ones). GothamChess made a lot of videos making fun of it. So in my AI agent project, I added a small chess engine and force the agent to only play moves output by the engine. And it was surprisingly good at it and we can now play real chess with LLMs. Check the project here if you are interested https://github.com/valmishq/valmis
Some people made videos with LLM's playing Poker.
The LLM's were terrible at poker.