I think tool calling (such as MCP) is really underrated. People think of it as a way to connect to Google Drive etc but the potential for MCP to power actual in-chat apps is very under-explored.
I've been working on something called Liveclip which is like tables in chat: just a few hours ago I implemented a way for ChatGPT (etc) to surface sentences within the US Constitution: https://x.com/firasd/status/2049187034066198658
I've also thought that games are an obvious way to explore chat-native MCP apps. Hangman for example. Also small utilities like a clock. In fact I have a no-auth remote MCP clock that can be added to any client right now deployed straight from https://github.com/firasd/mcpclock to https://mcpclock.firasd.workers.dev/mcp <-- works in Claude .ai, Claude iOS, Claude Code, Codex, etc right now
Cool, but yes you can render whatever you want as long as you register an MCP resource and use it as part of a tool call. It is just html in an iframe.
i am increasingly going schizo, where every single thing I post/see posted gets copied and karma farmed on social media. further, any novelty I share with an llm gets eaten/absorbed by the harness as a feature.
Great work. I wonder what the future "can it run doom ?" Will it be: Can it run AGI on a new quantum processor/ robot / bio-engineered cell. Or will it be Doom all the way down...
Was a little disappointed that this didn't result in the agent playing Doom itself over MCP.
Still this is very cool! (And if that had been what you'd implemented, my disappointment would have been that it wasn't the reverse. :P Can it do both?)
Snake and DOOM were two of our early tests (for filter functions and MCP) when we stood up Open WebUI for internal chat/agent use. Sometimes games are the best way to limit-test new tech.
The title is misleading. This is in ChatGPT but not on ChatGPT. It's not running on AI. It's running on a site that serves an AI chatbot.
Still very cool though.
I suppose most people don't know this, but you can render web resources into chat harnesses via MCP apps [1]. The actual DOOM part of this runs on [2].
It took me a reread and some thinking to realize what was going on. The 'MCP app' he's referring to here is basically a browser front-end replacement like electron. So what he's doing here is running DOOM as apart of that browser run-time and passing it through to the front-end. It's less "playing DOOM through the AI" and more "Playing DOOM while the AI can watch".
Thank you for this explanation. So the AI can respond based on the DOOM game that it’s watching? E.g. Could you ask the chatbot for advice on what to do next on the current game level?
Kind of? Even watching is probably a bit of a stretch here. The point of an MCP server is to be a sort of AI translator for whatever you're inputting. Here we're inputting an iframe that's running a wasm binary. So I imagine in theory all the AI sees is the actual iframe and whatever is in memory currently for the wasm game. Funny enough without some sort of screenshot tool on top of this I'm not sure the AI can actually 'see' the game at all.
Interesting experiment!
I think tool calling (such as MCP) is really underrated. People think of it as a way to connect to Google Drive etc but the potential for MCP to power actual in-chat apps is very under-explored.
I've been working on something called Liveclip which is like tables in chat: just a few hours ago I implemented a way for ChatGPT (etc) to surface sentences within the US Constitution: https://x.com/firasd/status/2049187034066198658
I've also thought that games are an obvious way to explore chat-native MCP apps. Hangman for example. Also small utilities like a clock. In fact I have a no-auth remote MCP clock that can be added to any client right now deployed straight from https://github.com/firasd/mcpclock to https://mcpclock.firasd.workers.dev/mcp <-- works in Claude .ai, Claude iOS, Claude Code, Codex, etc right now
Cool, but yes you can render whatever you want as long as you register an MCP resource and use it as part of a tool call. It is just html in an iframe.
I put Bad Apple in an MCP App two weeks ago https://youtu.be/YFF5H886slQ
Apart from a cool project, this evolved my perspective on what an MCP is, along with some cool architecture insights and inspiring ideas. Thank you!
Glad to hear it! This stuff is fascinating and rapidly evolving. I’ve been learning by doing. Happy hacking.
somebody did this a month ago https://www.youtube.com/watch?v=fdbXNWkpPMY
i am increasingly going schizo, where every single thing I post/see posted gets copied and karma farmed on social media. further, any novelty I share with an llm gets eaten/absorbed by the harness as a feature.
Timestamp?
I am not sure if it aligns to the approach in OP's article but it's the last ~minute or two of the linked video.
Great work. I wonder what the future "can it run doom ?" Will it be: Can it run AGI on a new quantum processor/ robot / bio-engineered cell. Or will it be Doom all the way down...
Was a little disappointed that this didn't result in the agent playing Doom itself over MCP.
Still this is very cool! (And if that had been what you'd implemented, my disappointment would have been that it wasn't the reverse. :P Can it do both?)
Snake and DOOM were two of our early tests (for filter functions and MCP) when we stood up Open WebUI for internal chat/agent use. Sometimes games are the best way to limit-test new tech.
That’s awesome. Yes!
The title is misleading. This is in ChatGPT but not on ChatGPT. It's not running on AI. It's running on a site that serves an AI chatbot.
Still very cool though.
I suppose most people don't know this, but you can render web resources into chat harnesses via MCP apps [1]. The actual DOOM part of this runs on [2].
[1] https://modelcontextprotocol.io/extensions/apps/overview [2] https://github.com/cloudflare/doom-wasm
Ok, we've switched the title above to a hopefully-clearer phrase from the article's first sentence.
Works for me! Thanks for updating.
Title changed from “DOOM runs in ChatGPT and Claude” to “A playable DOOM MCP app”.
I was not too surprised to see DOOM running on a pregnancy test before but seeing it on an MCP was really unexpected ;)
It's still fun but it feels gross. Like here is this monstrosity of compute running the thing we are running in toothbrushes.
I read this and I did not understand it. An MCP is synthesizing DOOM frames from a training set? What is this doing
It took me a reread and some thinking to realize what was going on. The 'MCP app' he's referring to here is basically a browser front-end replacement like electron. So what he's doing here is running DOOM as apart of that browser run-time and passing it through to the front-end. It's less "playing DOOM through the AI" and more "Playing DOOM while the AI can watch".
Thank you for this explanation. So the AI can respond based on the DOOM game that it’s watching? E.g. Could you ask the chatbot for advice on what to do next on the current game level?
Kind of? Even watching is probably a bit of a stretch here. The point of an MCP server is to be a sort of AI translator for whatever you're inputting. Here we're inputting an iframe that's running a wasm binary. So I imagine in theory all the AI sees is the actual iframe and whatever is in memory currently for the wasm game. Funny enough without some sort of screenshot tool on top of this I'm not sure the AI can actually 'see' the game at all.
its basically an iframe embedding doom on a remote webpage
Best capability test there is
simply great!
Appreciate it. I had fun building.
Love it! MCP apps is still new and under-utilised and we need to understand it's boundaries and compatibility.
For sure.