This is one of the most artfully positioned, full-throated, thoughtful, and capable takedowns of the AI hype (not necessarily LLMs as tools) I've read. It doesn't try to speak doom to AI companies, it doesn't suggest the tools are worthless at things they're good at. Instead, it disassembles the assumptions underlying ideas about frontier AI labs' promises (of both utopia and doom) by explaining in simple terms what the real accomplishment they've made is:
They are thieves who turned their stolen goods into a pretty neat invention.
They are not prophets of the end times or messianic figures bringing in a new age of human relaxation or the biggest trillionaires since Tim Cook and Elon Musk. They are people with one good tool that they think can and should be used to do everything. They won't accept that their tool is just useful, or helpful. They want it to be _more important than the humans and human work it was based on_, and nothing less will do. They want to replace effort and thought with simulacra of both and be applauded for it.
Lest this go down as an anti-AI post: I use AI tools. I use LLMs every day. They are useful... but they can't do on their own what we did not demonstrate how to do first, because they are tools.
I could be wrong, but I don't know that I see this changing.
> but they can't do on their own what we did not demonstrate how to do first, because they are tools.
Funny thing to say here, where an agent swarm (1M hours agent time) just did something no human did. If someone else solved NS blowup before OpenAI we would have heard their claims already. And we did hear partial claims, "they rushed in the same direction" kind of claims, not "they copied my fully developed idea". I think this is Move 37 moment for math, when we realize we might not be the top anymore.
That is very much the case at the current iteration of AI, but that should change with the introduction of world-mapping, computer vision, and world interacting AI. If machines are built out so that they are constructing accurate world-maps of how the physical world works and how to operate in it, then AI will begin to be able to out-invent human tool concepts and push past the horizon of what humans have so far been able to dream
> or input generated by them as derivatives of human input
Trained mostly on their own data, not ours. The mix changed, from 10% to 90% synthetic data. RL training is now the largest part of the process. The AI work is searching, and I don't think we can claim this process to be credited to us anymore. The origin point is, but not the journey.
That’s the thing. AI won’t be limited to being trained on only human input, why would it? Humans are trained by interacting with their environment plus a genetic world-mapping initial seed. There is no reason AI will not be the same, just at a much more extreme level
It depends on the knowledge domain. I would argue that much of human knowledge is individual to the person acquired by interacting with their environment
> History judges researchers who work with the bad guys ... To them I say: learn from history and get out now. You’ll still be ahead of the curve. The current backlash against AI may seem huge, but it’s nothing compared to what I think the backlash will be in coming years.
> I chose 7 years because I see certain parallels between the situation with researchers and Big AI today and the relationship between researchers and Epstein in 2019.
Look, I absolutely love Vi Hart's stuff. And I mean love love, and I have for almost 20 years.
But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I thought the original Mathathon sounded like a great idea. They shut that down, and I think the revised one seems like a half decent idea. But if I get involved in it, in 7 years it'll be the same as if I had been involved with Jeffrey Epstein?
I feel like the discourse on this is starting to get way out of control. I say that with no less love for Vi Hart's ideas than I have always had. But surely this technology can be used for some good?
> Imagine asking an AI to generate the whole numbers in order, from 0 on up. You could spend trillions of dollars having it fill up datacenter after datacenter with lists of numbers. The more we spend on the project, the more the list would start including some very big and important numbers. But what would actually be accomplished?
> Consider this and then consider how it applies to every number in any dataset and everything an AI generates: with our normal human understanding of a number, the number theoretically contains all of the meaning of that number. A number intrinsically contains the fact of whether it's prime, whether it's irrational, whether it's a counterexample to the Riemann Hypothesis, all facts known and unknown. You don't need a human to interpret the number in order for the number to mean what it means. In fact, the longer a human spends interpreting, the more meaning they might find that a number intrinsically has in and of itself. And yet... can you think of a less meaningful way to fill up a datacenter than with an ordered list of numbers?
> Listing all possible proofs out of context, whether in order or randomly, would hardly be more meaningful than listing numbers. Almost all possible proofs are trivial and meaningless, mathematicians select for the fun stuff.
> But I will say, since it seems every player in the US government and economy have committed to all jumping off a cliff together in a stubborn wish to cover the country in datacenters, I could think of worse applications than to fill them up with every possible mathematical proof below a certain size.
> You'd end up with a Library of Babel of proofs.
> You can imagine the quality of literature you'd get if you trained an LLM on the Library of Babel, which contains every possible book (or combination of characters that fit in 410 pages). The library, by itself, functionally contains no information whatsoever. The LLM trained on it would be a literal smooth-brain with no correlations to wrinkle up those gradients.
> Without the context of language and how it’s used by people, a great work of literature is just another possible random combination of letters. This can be represented as a point in multidimensional space, and a computer can guess that maybe there's more things of interest to humans near where the stuff we already like is, but only humans know how to push further into unknown directions of multidimensional space that are interesting and worthwhile.
> Whenever a company or person shares something “AI generated” that seems pretty good, that tells me that the AI must necessarily be using a lot of human-created data in that same area. The AI can not have leapt there on its own, even if it might look like it did to a person not aware of the scope of human-made work that went into it.
> The Mathlib of Babel problem is one reason why, in my view, formal math and AI are just one tool that might be useful for some math sometimes, not the end of mathematics or the future of mathematics. Formal mathematics and algorithmically-assisted proofs are not my personal aesthetic but I think it should be somebody's, and it's not like our fragmented field doesn't know how to handle another specific expertise joining the mix. We've already folded them in.
Found this interesting and thought-provoking. Why do we care about these proofs? Why do we care about this art, or this code? Because it is what is useful and interesting to humans.
Any expensive search in an exponential space meets cost very soon, but Vi is a Platonist, she thinks AI generate meaningless tokens, such as enumerating all proofs by LEAN. No, that search is interacting with its own costs, it is not platonic search scaled to infinity.
This is one of the most artfully positioned, full-throated, thoughtful, and capable takedowns of the AI hype (not necessarily LLMs as tools) I've read. It doesn't try to speak doom to AI companies, it doesn't suggest the tools are worthless at things they're good at. Instead, it disassembles the assumptions underlying ideas about frontier AI labs' promises (of both utopia and doom) by explaining in simple terms what the real accomplishment they've made is:
They are thieves who turned their stolen goods into a pretty neat invention.
They are not prophets of the end times or messianic figures bringing in a new age of human relaxation or the biggest trillionaires since Tim Cook and Elon Musk. They are people with one good tool that they think can and should be used to do everything. They won't accept that their tool is just useful, or helpful. They want it to be _more important than the humans and human work it was based on_, and nothing less will do. They want to replace effort and thought with simulacra of both and be applauded for it.
Lest this go down as an anti-AI post: I use AI tools. I use LLMs every day. They are useful... but they can't do on their own what we did not demonstrate how to do first, because they are tools.
I could be wrong, but I don't know that I see this changing.
> but they can't do on their own what we did not demonstrate how to do first, because they are tools.
Funny thing to say here, where an agent swarm (1M hours agent time) just did something no human did. If someone else solved NS blowup before OpenAI we would have heard their claims already. And we did hear partial claims, "they rushed in the same direction" kind of claims, not "they copied my fully developed idea". I think this is Move 37 moment for math, when we realize we might not be the top anymore.
Did you read the article? It goes into a lot of detail about how oversold the Navier-Stokes discovery is.
That is very much the case at the current iteration of AI, but that should change with the introduction of world-mapping, computer vision, and world interacting AI. If machines are built out so that they are constructing accurate world-maps of how the physical world works and how to operate in it, then AI will begin to be able to out-invent human tool concepts and push past the horizon of what humans have so far been able to dream
Why? They're still only going to be trained on human input or input generated by them as derivatives of human input.
Modality change might expand their usefulness, but I don't think it changes the basic parameters of their operation.
Learning to swim might let you see a coral reef, but it won't turn you into one.
> or input generated by them as derivatives of human input
Trained mostly on their own data, not ours. The mix changed, from 10% to 90% synthetic data. RL training is now the largest part of the process. The AI work is searching, and I don't think we can claim this process to be credited to us anymore. The origin point is, but not the journey.
That’s the thing. AI won’t be limited to being trained on only human input, why would it? Humans are trained by interacting with their environment plus a genetic world-mapping initial seed. There is no reason AI will not be the same, just at a much more extreme level
Aren’t humans also mostly trained on human input?
It depends on the knowledge domain. I would argue that much of human knowledge is individual to the person acquired by interacting with their environment
> History judges researchers who work with the bad guys ... To them I say: learn from history and get out now. You’ll still be ahead of the curve. The current backlash against AI may seem huge, but it’s nothing compared to what I think the backlash will be in coming years.
> I chose 7 years because I see certain parallels between the situation with researchers and Big AI today and the relationship between researchers and Epstein in 2019.
Look, I absolutely love Vi Hart's stuff. And I mean love love, and I have for almost 20 years.
But do people really believe this? That associating with AI companies, using their product, etc, is similar to taking money from a child sex trafficker?
I thought the original Mathathon sounded like a great idea. They shut that down, and I think the revised one seems like a half decent idea. But if I get involved in it, in 7 years it'll be the same as if I had been involved with Jeffrey Epstein?
I feel like the discourse on this is starting to get way out of control. I say that with no less love for Vi Hart's ideas than I have always had. But surely this technology can be used for some good?
The more intelligent someone is, can just lead to more creative ways to go crazy.
> Imagine asking an AI to generate the whole numbers in order, from 0 on up. You could spend trillions of dollars having it fill up datacenter after datacenter with lists of numbers. The more we spend on the project, the more the list would start including some very big and important numbers. But what would actually be accomplished?
> Consider this and then consider how it applies to every number in any dataset and everything an AI generates: with our normal human understanding of a number, the number theoretically contains all of the meaning of that number. A number intrinsically contains the fact of whether it's prime, whether it's irrational, whether it's a counterexample to the Riemann Hypothesis, all facts known and unknown. You don't need a human to interpret the number in order for the number to mean what it means. In fact, the longer a human spends interpreting, the more meaning they might find that a number intrinsically has in and of itself. And yet... can you think of a less meaningful way to fill up a datacenter than with an ordered list of numbers?
> Listing all possible proofs out of context, whether in order or randomly, would hardly be more meaningful than listing numbers. Almost all possible proofs are trivial and meaningless, mathematicians select for the fun stuff.
> But I will say, since it seems every player in the US government and economy have committed to all jumping off a cliff together in a stubborn wish to cover the country in datacenters, I could think of worse applications than to fill them up with every possible mathematical proof below a certain size.
> You'd end up with a Library of Babel of proofs.
> You can imagine the quality of literature you'd get if you trained an LLM on the Library of Babel, which contains every possible book (or combination of characters that fit in 410 pages). The library, by itself, functionally contains no information whatsoever. The LLM trained on it would be a literal smooth-brain with no correlations to wrinkle up those gradients.
> Without the context of language and how it’s used by people, a great work of literature is just another possible random combination of letters. This can be represented as a point in multidimensional space, and a computer can guess that maybe there's more things of interest to humans near where the stuff we already like is, but only humans know how to push further into unknown directions of multidimensional space that are interesting and worthwhile.
> Whenever a company or person shares something “AI generated” that seems pretty good, that tells me that the AI must necessarily be using a lot of human-created data in that same area. The AI can not have leapt there on its own, even if it might look like it did to a person not aware of the scope of human-made work that went into it.
> The Mathlib of Babel problem is one reason why, in my view, formal math and AI are just one tool that might be useful for some math sometimes, not the end of mathematics or the future of mathematics. Formal mathematics and algorithmically-assisted proofs are not my personal aesthetic but I think it should be somebody's, and it's not like our fragmented field doesn't know how to handle another specific expertise joining the mix. We've already folded them in.
Found this interesting and thought-provoking. Why do we care about these proofs? Why do we care about this art, or this code? Because it is what is useful and interesting to humans.
Any expensive search in an exponential space meets cost very soon, but Vi is a Platonist, she thinks AI generate meaningless tokens, such as enumerating all proofs by LEAN. No, that search is interacting with its own costs, it is not platonic search scaled to infinity.