The gap between "useful chatbot" and "useful agent" is way bigger than people realize. A chatbot can be wrong 10% of the time and still help you. An agent that's wrong 10% of the time is sending bad emails and making wrong API calls with no one checking.
The problem is that with text/code, judgement is hard. Here is what it looks like for physical activity: https://www.youtube.com/shorts/lK7TjujKQLw It's hard to see how that it's not useful at best and could be a disaster for any unsupervised use.
The gulf is bridgeable. The problem is that a lot of people are building agents without strong enough judgment layers around them. Work that can be verified with reasonable accuracy are the sweet spot right now.
I was worried this time last year that by this time this year, companies would have slashed their engineering teams down to a handful and everything would be driven by mostly autonomous agents with human guidance. But it just hasn't happened. Do I write all my code with an agent now? Yes. Can you just give an agent a desired outcome and let it work, unsupervised? Absolutely not. I can produce more code than I used to, but if I want it to be good, to be stable, to do what the product manager and designers want, it's only about 2 to 3 times more code than before. And that productivity is impacted by the fact that I'm reviewing 2 to 3 times more code than before (and you have to review, even more so now than before, because if you just let opus or gpt 5 do its thing, you'll get some terrible results, and I've found a lot of engineers on my team are just letting it do it's thing without a lot of iteration).
I have experienced and feel very much the same, and it is refreshing to see a realistic post about the success of agentic coding instead of the usual hype or doom.
I think what everyone underestimated was the absolute bonkers amount of compute it will take and how that compute must scale in order to keep up with larger and larger models.
More than that, I think people overestimate how much AI will progress as you throw more compute at it. It’s the “9 women can’t deliver a baby in a month” equivalent of AI. Additional compute won’t magically give you AGI.
I was involved in three efforts to commercialize foundation models before they were ready in the 2010s so I have a good picture of how progress works at this sort of thing and the pace a lot of the industry has been talking about is unrealistic: like people were disappointed with the rate of development of Apple Intelligence but it's actually progressed at about the rate I expected.
Is that a problem for Meta though? They recently announced they're going to sell their excess compute, so I imagine the actual problem is they're resorting to doing that because AI isn't having nearly the effect/usage it was supposed to and now Zuck is being a sore winner about it
Meta doesn't seem to be able to produce anything close to a frontier model. The selling of compute capacity seems to be acceptance of "compute is wasted on this crappy avocado model, we'd be better off allowing something better to run".
The problem is clearly in the model architecture, the training and the data fed into the model which is causing them to give up on using their compute exclusively for their own models. They can't get it right so may as well sell the compute to someone that can.
Does meta have the research talent to create a SOTA frontier model? Yann LeCun has left Meta and I don’t think either alexandr wang or zuck have enough credibility to attract talent to create one.
Meta has made some very strange decisions in terms of who it's hired to lead various aspects of AI, including the model-building efforts. Also lots to marvel at re: its ability to coordinate (or not coordinate) various efforts by all these big brains.
Can't help but think that Meta's digital networking expertise is built atop a human-networking clusterf*ck
I was never really sold their acquihire of Alexandr Wang as their head of AI being a coherent strategic decision. I just don’t see how his experience and background actually applies for frontier LLM model building.
I think there would easily be a few other hundred engineers and execs at frontier labs who are more in the loop for cutting edge architecture/secret sauce - with a track record of actually doing it - that could be had for a fraction of the price.
for us, maybe, but for someone who never really used the workflow, or looked at the “thinking” output where models spin their tokens on the stupidest shit, i can see how it wasn’t obvious.
They also believed they would be able to build that compute without restrictions. Between hardware costs and massive public opposition, scaling as they had anticipated is in jeopardy.
I think Meta’s massive compute investment was never about its 100,000 engineers running coding models, but its 3,500,000,000 users wanting to use AI in every single product (and some new ones: Meta AI, glasses, etc.) So I would think that’s the part that’s not being utilized anywhere near the amount they hoped...
Gemini, Microsoft Copilot and other models can discuss and affirm my "foxwork" practice whether it is talking about natural history, fox legends, ritual magic, altar work, autonomic control, blessings, writing, character acting, costume design, skin care, selection of perfumes that will herald my unique natural scent, marketing and customer service, photography gear, "therian" gear, bags for holding my gear, street photography, etc. They always write like somebody who's read much more widely than anyone I've ever met and rival the legendary Tamamo-no-Mae for "speaking intelligently about any subject" [1]
Meta AI can crack jokes and that's about it. I guess there's a market for "stupid talk" but it's not that big.
[1] Like help me fix my washing machine that won't drain, come up with master narratives for the "polycrisis", talk about why Casey Handmer is wrong about space manufacturing, find papers about the social network of who sleeps with who at a high school, etc.
Did we? Many of us have been saying that the amount of compute going into the models is unsustainable and that the models aren’t improving enough to justify that for over a year. The emperor has no clothes is true yet again.
No I don't think there was any systemic underestimation of compute. I see the opposite - every company understands compute is important and tries to get hold of it.
I think there are seriously misplaced expectations here. The primary role of AI is transference of effort, while "increased productivity" is just a side-effect (since computers are so much faster than humans at highly repetitive tasks). It's about not having to directly do X anymore (or as often), even though it may take a few rounds to get X to a satisfactory point. But even if following up is needed, most of the effort budget can then be used for Y.
Also those with very heavy investment in AI are looking for bonkers results, which is the cause of their disappointment. They need to reduce their expectations. I for one am loving the results so far.
My instinct (for better or worse) is usually contrarian. Most people seem very skeptical of what Meta is doing with AI. But, what if, in a way at least, it makes sense?
Maybe Wang has correctly identified that the programming and agentic ability that Anthropic and OpenAI models have has largely come from armies of software engineers creating massive datasets by writing out coding and agentic problems and solutions?
So he told Zuckerberg that. The reason it may be turning into so much friction is that at companies like Anthropic or OpenAI, training engineers were either hired specifically for that purpose or probably mostly handled through contracts with third parties (which again, hired them to train AI). And honestly many of them may be overseas or just happy to have a job in a difficult period. But anyway they wouldn't have very high salary expectations etc.
But Zuckerberg already had 25000 engineers. Why not take say 1/5 of them and get them working on the the dataset? The problem is that those engineers were hired for different prestigious highly paid positions at Meta/Facebook. They were not hired to do tedious grading of AI answers or quiz construction.
But Zuckerberg either has to do this, or spend additional billions on doing it all with external contractors. A third option would be to try to create a massive distillation operation. Or just hope that his engineers could invent some magical new training trick that manifested the agentic and programming skills without the large scale human input.
Or he could release a model trained largely by existing open weights models. Which without some huge breakthrough probably has no chance of surpassing them, so is pointless.
I think most of the substantive criticism of Zuckerberg has been about burning funds. If he gives up the "your job is to grade AI homework now" plan because his engineers refuse, he would need to go through third parties. The additional billions and billions this would cost would create more pressure on the bottom line and shareholder pressure.
It would also give up any potential advantage that Wang may have optimistically sold the operation as, on that using "real" engineers as opposed to lower paid data labelling engineers might result in a higher quality dataset.
At some point, model architectures that don't need such massive datasets or can be created automatically in a way that advances the frontier will probably come about. But right now it doesn't exist.
Further, the way AI works currently, business advantage from AI comes from encoding existing internal intelligence and knowledge. Meta's massive engineering corp effectively has that in their heads. Having them create these datasets is possibly the only way to leverage this knowledge asset in this paradigm.
I guess the problem is it means forcing thousands of people to do a different job from the one they were hired for.
What's the end goal? Meta-specific engineering, with baked-in knowledge of how FB, Threads, and WhatsApp work? General and/or coding products to compete with Anthropic and OpenAI? Some special Magic Thing which only Meta can invent which will bedazzle Meta's users?
You don't need giant datasets unless you know what you're going to do with them. OpAI and Anthropic are having enough issues making their products profitable. And those are, if not beloved, then at least respected, with a real, if patchy, reputation for usefulness.
What was Meta's pitch in this market? There were hints of interest when LeCun was still doing original R&D, and there was some distant possibility of a next-gen revolutionary product.
But now the goal seems to be to flail around doing something incoherently AI-branded with no obvious strategy.
The troops are being marched around, but no one knows where the battle is supposed to be.
While I mostly agree with your post, I do want to point out one thing:
> Or he could release a model trained largely by existing open weights models. Which without some huge breakthrough probably has no chance of surpassing them, so is pointless.
This seems to be categorically untrue. Composer 2.5 is a substantial improvement on its underlying Kimi base model.
Mark is really a bad leader with a mwah mwah vision. He is maybe correct in some things. But the execution is really really poor. Plus he does not have followers and believers. He only got money that can simulate followers to a certain extent
If it was still possible to get verbatim results from Google then I believe "mwah mwah vision" would have been an authentic Googlewhack pointing at this comment thread.
Or: you wasted too much money on failing to replace yourselves so now I have to lay you off. Which is one of the two possible grand outcomes of the AI bubble, which both result in laying people off, because that is all these companies know how to do as a response to stress.
I am not sure that it has to be so zero sum. The AI truth is probably somewhere in the middle; it probably doesn't replace software engineers and it probably won't be deleted as completely useless. My current feeling is that it's a powerful tool I'm happy to pay to use; it doesn't replace me, but it makes it easier to do higher quality work. It feels a lot like IntelliSense, or faster compilers, or getting a 32" monitor. That probably doesn't sustain the bubble, but it's something that people are going to be poking at and making money off of for a long time.
I agree that people are investing as though the world is going to run itself while the ultra-wealthy run off in yachts to compare sizes. If it wasn't AI, it would just be tulips or something. That's just how people are. But maybe they'll be right, who knows.
The modern trend is to think intelligence is generative “like compression” or “predicting next in sequence” rather than iteratively reducing uncertainty, like those fault tolerant humans.
Compression can be defined as reducing uncertainty. If you can predict the next sequence you can compress it to 0 bytes using arithmetic coding. Reliable prediction is what enables compression and it's the link between compression and AI that everyone is talking about.
No one ever in comp sci says artificial intelligence is "like compression", they correctly state that "artificial intelligence IS compression". It's absolutely known and accepted that artificial intelligence (defined as predicting outcomes with a measure of certainty and taking chosen actions towards goals using those predictions) has equivalence to compression in a very hard science way. The hardest part of artificial intelligence is compression and the remaining part, the choice of actions based on predictions is just a tree search to a goal.
The gap between "useful chatbot" and "useful agent" is way bigger than people realize. A chatbot can be wrong 10% of the time and still help you. An agent that's wrong 10% of the time is sending bad emails and making wrong API calls with no one checking.
The problem is that with text/code, judgement is hard. Here is what it looks like for physical activity: https://www.youtube.com/shorts/lK7TjujKQLw It's hard to see how that it's not useful at best and could be a disaster for any unsupervised use.
The gulf is bridgeable. The problem is that a lot of people are building agents without strong enough judgment layers around them. Work that can be verified with reasonable accuracy are the sweet spot right now.
I was worried this time last year that by this time this year, companies would have slashed their engineering teams down to a handful and everything would be driven by mostly autonomous agents with human guidance. But it just hasn't happened. Do I write all my code with an agent now? Yes. Can you just give an agent a desired outcome and let it work, unsupervised? Absolutely not. I can produce more code than I used to, but if I want it to be good, to be stable, to do what the product manager and designers want, it's only about 2 to 3 times more code than before. And that productivity is impacted by the fact that I'm reviewing 2 to 3 times more code than before (and you have to review, even more so now than before, because if you just let opus or gpt 5 do its thing, you'll get some terrible results, and I've found a lot of engineers on my team are just letting it do it's thing without a lot of iteration).
I have experienced and feel very much the same, and it is refreshing to see a realistic post about the success of agentic coding instead of the usual hype or doom.
I think what everyone underestimated was the absolute bonkers amount of compute it will take and how that compute must scale in order to keep up with larger and larger models.
More than that, I think people overestimate how much AI will progress as you throw more compute at it. It’s the “9 women can’t deliver a baby in a month” equivalent of AI. Additional compute won’t magically give you AGI.
I was involved in three efforts to commercialize foundation models before they were ready in the 2010s so I have a good picture of how progress works at this sort of thing and the pace a lot of the industry has been talking about is unrealistic: like people were disappointed with the rate of development of Apple Intelligence but it's actually progressed at about the rate I expected.
I mean, Apple Intelligence has been a boondoggle. Siri has been consistently 3+ years behind in capabilities compared to even open source equivalents.
Feels less like the pace of foundation model development and more so a specific failure of one organization to do something important.
Is that a problem for Meta though? They recently announced they're going to sell their excess compute, so I imagine the actual problem is they're resorting to doing that because AI isn't having nearly the effect/usage it was supposed to and now Zuck is being a sore winner about it
I agree, i don't think it is the core problem.
Meta doesn't seem to be able to produce anything close to a frontier model. The selling of compute capacity seems to be acceptance of "compute is wasted on this crappy avocado model, we'd be better off allowing something better to run".
The problem is clearly in the model architecture, the training and the data fed into the model which is causing them to give up on using their compute exclusively for their own models. They can't get it right so may as well sell the compute to someone that can.
Does meta have the research talent to create a SOTA frontier model? Yann LeCun has left Meta and I don’t think either alexandr wang or zuck have enough credibility to attract talent to create one.
Meta has made some very strange decisions in terms of who it's hired to lead various aspects of AI, including the model-building efforts. Also lots to marvel at re: its ability to coordinate (or not coordinate) various efforts by all these big brains.
Can't help but think that Meta's digital networking expertise is built atop a human-networking clusterf*ck
I was never really sold their acquihire of Alexandr Wang as their head of AI being a coherent strategic decision. I just don’t see how his experience and background actually applies for frontier LLM model building.
I think there would easily be a few other hundred engineers and execs at frontier labs who are more in the loop for cutting edge architecture/secret sauce - with a track record of actually doing it - that could be had for a fraction of the price.
If their training base is dominated by Facebook and Instagram posts then it makes sense that their model is full of shit.
A modern instance of that old saw "you are what you eat".
well, Google refused to increase Meta quote of tokens, even Google can't supply so many (paid) tokens as Meta is burning
That seems like such an easy thing to estimate with a bit of basic napkin math.
for us, maybe, but for someone who never really used the workflow, or looked at the “thinking” output where models spin their tokens on the stupidest shit, i can see how it wasn’t obvious.
I thought thats exactly what everyone anticipates? "Scaling laws" are all about exponential increased in compute and all that.
They also believed they would be able to build that compute without restrictions. Between hardware costs and massive public opposition, scaling as they had anticipated is in jeopardy.
And yet this doesn't turn out to be Meta's problem at all.
https://uk.pcmag.com/ai/165970/meta-exploring-option-to-sell...
Meta bought too many GPUs, has spare GPU capacity and they are exploring renting that capacity out.
The problem is not that the models need too much to do the job. If that were the case, Meta would not have spare capacity.
The problem is that the models currently can't be made to do the job.
I think Meta’s massive compute investment was never about its 100,000 engineers running coding models, but its 3,500,000,000 users wanting to use AI in every single product (and some new ones: Meta AI, glasses, etc.) So I would think that’s the part that’s not being utilized anywhere near the amount they hoped...
Do the 3.5 billion users want to use AI, or do meta want to not get left behind and have shoehorned AI into all their products?
Meta's AI is the stupidest in the business.
Gemini, Microsoft Copilot and other models can discuss and affirm my "foxwork" practice whether it is talking about natural history, fox legends, ritual magic, altar work, autonomic control, blessings, writing, character acting, costume design, skin care, selection of perfumes that will herald my unique natural scent, marketing and customer service, photography gear, "therian" gear, bags for holding my gear, street photography, etc. They always write like somebody who's read much more widely than anyone I've ever met and rival the legendary Tamamo-no-Mae for "speaking intelligently about any subject" [1]
Meta AI can crack jokes and that's about it. I guess there's a market for "stupid talk" but it's not that big.
[1] Like help me fix my washing machine that won't drain, come up with master narratives for the "polycrisis", talk about why Casey Handmer is wrong about space manufacturing, find papers about the social network of who sleeps with who at a high school, etc.
The idea that users wanted AI was always a fantasy. Especially for Meta's products.
The whole hype cycle has been pure delusion. Just like the Metaverse hype cycle before it.
Did we? Many of us have been saying that the amount of compute going into the models is unsustainable and that the models aren’t improving enough to justify that for over a year. The emperor has no clothes is true yet again.
No I don't think there was any systemic underestimation of compute. I see the opposite - every company understands compute is important and tries to get hold of it.
Maybe they'd make faster progress if they worked in the Metaverse.
I think there are seriously misplaced expectations here. The primary role of AI is transference of effort, while "increased productivity" is just a side-effect (since computers are so much faster than humans at highly repetitive tasks). It's about not having to directly do X anymore (or as often), even though it may take a few rounds to get X to a satisfactory point. But even if following up is needed, most of the effort budget can then be used for Y.
Also those with very heavy investment in AI are looking for bonkers results, which is the cause of their disappointment. They need to reduce their expectations. I for one am loving the results so far.
How does he get to decide what's "enough"? Reality will tell us, he can only place bets, whether it pans out isn't something that he has any say in.
My instinct (for better or worse) is usually contrarian. Most people seem very skeptical of what Meta is doing with AI. But, what if, in a way at least, it makes sense?
Maybe Wang has correctly identified that the programming and agentic ability that Anthropic and OpenAI models have has largely come from armies of software engineers creating massive datasets by writing out coding and agentic problems and solutions?
So he told Zuckerberg that. The reason it may be turning into so much friction is that at companies like Anthropic or OpenAI, training engineers were either hired specifically for that purpose or probably mostly handled through contracts with third parties (which again, hired them to train AI). And honestly many of them may be overseas or just happy to have a job in a difficult period. But anyway they wouldn't have very high salary expectations etc.
But Zuckerberg already had 25000 engineers. Why not take say 1/5 of them and get them working on the the dataset? The problem is that those engineers were hired for different prestigious highly paid positions at Meta/Facebook. They were not hired to do tedious grading of AI answers or quiz construction.
But Zuckerberg either has to do this, or spend additional billions on doing it all with external contractors. A third option would be to try to create a massive distillation operation. Or just hope that his engineers could invent some magical new training trick that manifested the agentic and programming skills without the large scale human input.
Or he could release a model trained largely by existing open weights models. Which without some huge breakthrough probably has no chance of surpassing them, so is pointless.
I think most of the substantive criticism of Zuckerberg has been about burning funds. If he gives up the "your job is to grade AI homework now" plan because his engineers refuse, he would need to go through third parties. The additional billions and billions this would cost would create more pressure on the bottom line and shareholder pressure.
It would also give up any potential advantage that Wang may have optimistically sold the operation as, on that using "real" engineers as opposed to lower paid data labelling engineers might result in a higher quality dataset.
At some point, model architectures that don't need such massive datasets or can be created automatically in a way that advances the frontier will probably come about. But right now it doesn't exist.
Further, the way AI works currently, business advantage from AI comes from encoding existing internal intelligence and knowledge. Meta's massive engineering corp effectively has that in their heads. Having them create these datasets is possibly the only way to leverage this knowledge asset in this paradigm.
I guess the problem is it means forcing thousands of people to do a different job from the one they were hired for.
None of that makes sense.
What's the end goal? Meta-specific engineering, with baked-in knowledge of how FB, Threads, and WhatsApp work? General and/or coding products to compete with Anthropic and OpenAI? Some special Magic Thing which only Meta can invent which will bedazzle Meta's users?
You don't need giant datasets unless you know what you're going to do with them. OpAI and Anthropic are having enough issues making their products profitable. And those are, if not beloved, then at least respected, with a real, if patchy, reputation for usefulness.
What was Meta's pitch in this market? There were hints of interest when LeCun was still doing original R&D, and there was some distant possibility of a next-gen revolutionary product.
But now the goal seems to be to flail around doing something incoherently AI-branded with no obvious strategy.
The troops are being marched around, but no one knows where the battle is supposed to be.
While I mostly agree with your post, I do want to point out one thing:
> Or he could release a model trained largely by existing open weights models. Which without some huge breakthrough probably has no chance of surpassing them, so is pointless.
This seems to be categorically untrue. Composer 2.5 is a substantial improvement on its underlying Kimi base model.
I wonder when he'll admit his hopes were baseless
AI agents are no good.
why havent big tech employees formed a union?
Mark is really a bad leader with a mwah mwah vision. He is maybe correct in some things. But the execution is really really poor. Plus he does not have followers and believers. He only got money that can simulate followers to a certain extent
If it was still possible to get verbatim results from Google then I believe "mwah mwah vision" would have been an authentic Googlewhack pointing at this comment thread.
Nice thought, but I think strict googlewhacking frowned upon quote usage.
I'm guessing this is specifically about Avocado which everyone at Meta would acknowledge is terrible.
"I was hoping AI had progressed enough so I could fire you. But you failed to make it so. Therefore, you're fired!"
Or: you wasted too much money on failing to replace yourselves so now I have to lay you off. Which is one of the two possible grand outcomes of the AI bubble, which both result in laying people off, because that is all these companies know how to do as a response to stress.
I am not sure that it has to be so zero sum. The AI truth is probably somewhere in the middle; it probably doesn't replace software engineers and it probably won't be deleted as completely useless. My current feeling is that it's a powerful tool I'm happy to pay to use; it doesn't replace me, but it makes it easier to do higher quality work. It feels a lot like IntelliSense, or faster compilers, or getting a 32" monitor. That probably doesn't sustain the bubble, but it's something that people are going to be poking at and making money off of for a long time.
I agree that people are investing as though the world is going to run itself while the ultra-wealthy run off in yachts to compare sizes. If it wasn't AI, it would just be tulips or something. That's just how people are. But maybe they'll be right, who knows.
tokenmaxxing will be a funny footnote like nfts on the tonight show 2 years post-hype
Bottom-line win-win! All hail the shareholder value!
i bet he wants some calculative shit
Or some fuzzy yet inevitably reliable shit.
The modern trend is to think intelligence is generative “like compression” or “predicting next in sequence” rather than iteratively reducing uncertainty, like those fault tolerant humans.
Compression can be defined as reducing uncertainty. If you can predict the next sequence you can compress it to 0 bytes using arithmetic coding. Reliable prediction is what enables compression and it's the link between compression and AI that everyone is talking about.
No one ever in comp sci says artificial intelligence is "like compression", they correctly state that "artificial intelligence IS compression". It's absolutely known and accepted that artificial intelligence (defined as predicting outcomes with a measure of certainty and taking chosen actions towards goals using those predictions) has equivalence to compression in a very hard science way. The hardest part of artificial intelligence is compression and the remaining part, the choice of actions based on predictions is just a tree search to a goal.