With the top AI personnel leaving Google, I wonder what will happen to their Gemini. Right now they still have a lead in inference hardware with TPUs but with both Anthropic and OpenAI developing their own chips, I wonder how long until either one will catch up.
2023: Google is doomed. They're not building AI. No wonder the "Attention" authors left. Google just sat on this. Their PMs are steering them into oblivion. OpenAI is winning while Google takes no risks. Innovator's dilemma. Google Search is doomed.
2024: Google is on fire. Sergey coming back helped. Gemini, Veo. They've caught up. OpenAI is doomed.
2025: Google is seriously winning now. Nano Banana!! Google was destined to be the true winner of AI.
2026 H1: Google is slow as hell. Where is Gemini? Google is not launching anything, meanwhile just look at everyone else. Anthropic! And open source. What the hell are they doing over there?
2026 H2: Everyone is leaving Google. Google is doomed. PMs are destroying the company. They don't take risks.
It seems Google has to play both offense and defense: competing against frontier labs' models while protecting search and ads. They also have to be mindful of not doing anything that could hurt their own search or ads. That seems harder than a frontier lab just doing offense on both models and search/ads.
Having said that, I think Google's moat is still strong with Cloud, Gmail, YouTube, Android, Chrome, etc.
To be honest, this feels more like a lifestyle business (aka hobby) than a startup. They truly deserve it, but I don't expect a huge success as a business.
That said, I hope they write cool papers with various peers across the industry without worrying too much about the competing dynamics. That'd be a blessing for humanity, and good for their spirit.
A public benefit corp shouldn’t be a start up. The primary goal of a start up is to grow as quickly as possible which is rarely benefits the public.
Very silly to call every non start up a lifestyle business. It’s just a business. Start up are the weird thing that almost always an obscene waste of time and money, but sometime creates google.
A PBC is not a charity or a non-profit. PBCs are for-profit businesses with the goal of making money. In day-to-day business they're indistinguishable with other for-profit corporations, including fundraising and investment. The only real practical difference is that they give directors a little more leeway in their fiduciary duties to say "no" to doing evil things.
The advantage to a PBC is protecting founders from a serious problem with standard corporations: you might bring on investors who could subsequently demand you pollute, exploit people, and/or do other immoral activities for profit. You don't have to do these things to grow a business quickly.
How can you benefit the public if you are small? Shouldn't scaling up as fast as possible maximize the amount of benefit the company is able to deliver?
> The primary goal of a start up is to grow as quickly as possible which is rarely benefits the public.
Why is that so? Fast growth, when achieved honestly, is a result of solving user pain that others haven't. Maybe you think so because users != the public, but I think in totality the public is a collection of users who all have needs they want met.
For certain amount of fast growth: yes. Then there's continued "growth-hacking" and enshittification to keep fast revenue growing after the pain point has been solved with dark patterns and questionable tactics.
Why? Because investors poured a bunch of money in to support fast growth and now they want their money back. And incremental growth won't do. Since 9 out of 10 of the investments fail, the surviving one has to continue to growth-hacking revenues.
Gemini has done absolutely nothing for me. I can't even shut off the navigation feature on my phone using only hands free, when I get close to my destination. I have to take my eyes off the road, look down, and tap to exit.
Google's advanced AI cannot even exit a mobile app.
Antigravity + Gemini Pro absolutely RIPS through fullstack react + react-native apps / systems. I pay ~$20/month and I basically don't have to do my real work anymore. My time is freed up to learn systems programming and blender.
Those are all fake, part of an internal Google narrative that overstates individual contribution, and obscures the work of large engineering teams.
Here are some Jeff Dean well sourced facts:
- Already part of engineering of Google indexing systems that lacked basic checksums and ran on non-ECC hardware, allowing silent data corruption.
- One of the authors of LevelDB a database with so many documented crash-consistency, recovery, and data-loss weaknesses for years. Just check their Github project. LevelDB current tracker contains unresolved crash consistency, recovery and corruption reports going back almost 12 years on GitHub
- In AI engineering technical lead, let TensorFlow lose researcher mind share to PyTorch, and caused Google fragmented landscape across TensorFlow and JAX.
- Had the people at Google who invented the Transformer architecture, but failed, to turn that lead into the first dominant public LLM.
- As AI engineering and VP management let Google Brain and DeepMind remain duplicated and internally competitive for too long.
- Let Noam Shazeer leave and then spent heavily to bring him back with nothing to show for.
- Part of Technical VP leadership who had Bard rushed to launch with factual errors in Google own promotional material.
- The first Gemini demonstration overstated how real-time and interactive the system actually was, being basically a fake.
- Part of the VP and AI technical leadership who had Google AI Overviews launched with weak source quality controls and repeated satire and low-quality web content as factual advice.
- Part of teams that launched AlphaChip performance claims that were difficult for outside researchers to reproduce and remain technically disputed.
- Jeff Dean public explanation of Gebru departure was contested and damaged confidence in Google scientific governance.
- Jeff Dean was part of the team at Google that removed or marginalized prominent internal AI ethics critics shortly before many of their warnings became product problems.
- Jeff Dean was one of the managers behind Project Dragonfly supporting censorship.
- Jeff Dean is part of the VP technical leadership approving Project Nimbus supporting an ongoing genocide.
I actually think there's a low probability of that.
The four have so much influence within the company that they could have trivially set this up as part of Alphabet, if they wanted to. They are also ridiculously wealthy.
Only California employers with 15 or more employees have to post a pay range (Senate Bill 1162 [1], effective Jan 1, 2023) [2][3]. Discovery Loop employs only four people (that we know of), so it isn’t required to disclose a salary range in its job posts, of which there is only one [4].
I've seen tiny tiny hints from the outside that Jeff Dean was dealing with too much internal BS. Two examples that come to mind: Having to deal with Timnit Gebru fiasco, and even chips in the TPU series getting marketing names (Trillium and Ironwood) before switching back to more standard numbering.
I have no doubt that internal Google friction is one of the reasons they are moving. But the Gebru incident was almost seven years ago now. It is very unlikely to be a proximate cause.
I doubt numbering vs names on TPU releases even crosses Jeff's radar. It's not the kind of thing he cares about.
I wish them well, but this firm will likely fail miserably. The reason is that the value is in having access to real world hardware platforms that AI can control, not in the harness that controls them. There exist plenty of harnesses already. These people couldn't even get Google to build a top LLM. Before you dismiss and downvote, I dare you to counter it.
hmm not really, and it probably isnt the origin, but im glad to see him vindicated by having both C-level google leadership people who targeted to try and either stop google from doing the pentagon deal, both stepping down the same day.
At least it makes me feel like he reaching out might have made a difference on remembering this people killer robots are bad, and being part of it might not be on the best of their interests to go down on history for
As LLM coding agents plateau— at least for the average engineer without tens of thousands of dollars or swarms of agents to run —I’d say that, from here on it’s going to be about ASICs, specialized LoRA/or-equivalent models, or a Ruby on Rails for LLM context engineering and orchestration, which LangChain and others seems well position, including Google as they own the entire stack. LLM free lunch has been over for a while, perhaps since the ReAct loop, and has been official since Ilya mentioned it at NeurIPS.
I feel the most exciting development these days is self-evolving agents. Especially if you have a way to verify their outputs with a formal system, or with a system developed since the 60s by armies of PhDs.
DeepMinds Gnome is a good example, where they use DFT to verify outputs. Approximating NP-problems is always fun for those who dare.
I am also building in this space. Its a mix between HPC, AI, and hard science. Pretty fun compared to waking everyday to LLM news that seem more like marketing stunts.
LLM coding isn't even close to plateauing. Right now, the major players are in a consolidation step, focusing more on economic efficiency but still not at the point where we're ready to start burning models to hardware and freezing the line.
They are straddling the line between pushing it forward, and justifying the business case. It's hard to do both at the same time.
LLMs are hungry for tokens. Every day I see more startups claiming token usage at 50-100k per month. You can always brute-force your way in - just see HuggingFace's recent attacks.
If throwing more money at inference while accumulating compounding technical debt is the new norm, then we are not solving the problem, and the solution space is already covered.
Perhaps there are marginal gains at the expense of quadrillion-params LLM models with 10x the cost and energy. We are simply making inefficiency more expensive, camouflaged by VC money and great marketing.
If that is not plateauing, then I guess I will have to reconsider what plateauing means.
This is such unbelievable revisionism! Can you imagine in 2022 saying "Oh of course you can brute force your way to AGI if you spend enough money per month". Nobody thought that! Come on!
With the top AI personnel leaving Google, I wonder what will happen to their Gemini. Right now they still have a lead in inference hardware with TPUs but with both Anthropic and OpenAI developing their own chips, I wonder how long until either one will catch up.
2023: Google is doomed. They're not building AI. No wonder the "Attention" authors left. Google just sat on this. Their PMs are steering them into oblivion. OpenAI is winning while Google takes no risks. Innovator's dilemma. Google Search is doomed.
2024: Google is on fire. Sergey coming back helped. Gemini, Veo. They've caught up. OpenAI is doomed.
2025: Google is seriously winning now. Nano Banana!! Google was destined to be the true winner of AI.
2026 H1: Google is slow as hell. Where is Gemini? Google is not launching anything, meanwhile just look at everyone else. Anthropic! And open source. What the hell are they doing over there?
2026 H2: Everyone is leaving Google. Google is doomed. PMs are destroying the company. They don't take risks.
Meanwhile, Microsoft: "Copilot!"
It seems Google has to play both offense and defense: competing against frontier labs' models while protecting search and ads. They also have to be mindful of not doing anything that could hurt their own search or ads. That seems harder than a frontier lab just doing offense on both models and search/ads.
Having said that, I think Google's moat is still strong with Cloud, Gmail, YouTube, Android, Chrome, etc.
>Meanwhile, Microsoft: "Copilot!"
The difference is nobody expects anything better from Microsoft. Teams didn't exactly set the bar high.
My predictions about everything ever, summarized perfectly. I need an “inverse me ETF”.
To be honest, this feels more like a lifestyle business (aka hobby) than a startup. They truly deserve it, but I don't expect a huge success as a business.
That said, I hope they write cool papers with various peers across the industry without worrying too much about the competing dynamics. That'd be a blessing for humanity, and good for their spirit.
A public benefit corp shouldn’t be a start up. The primary goal of a start up is to grow as quickly as possible which is rarely benefits the public.
Very silly to call every non start up a lifestyle business. It’s just a business. Start up are the weird thing that almost always an obscene waste of time and money, but sometime creates google.
A PBC is not a charity or a non-profit. PBCs are for-profit businesses with the goal of making money. In day-to-day business they're indistinguishable with other for-profit corporations, including fundraising and investment. The only real practical difference is that they give directors a little more leeway in their fiduciary duties to say "no" to doing evil things.
The advantage to a PBC is protecting founders from a serious problem with standard corporations: you might bring on investors who could subsequently demand you pollute, exploit people, and/or do other immoral activities for profit. You don't have to do these things to grow a business quickly.
How can you benefit the public if you are small? Shouldn't scaling up as fast as possible maximize the amount of benefit the company is able to deliver?
> The primary goal of a start up is to grow as quickly as possible which is rarely benefits the public.
Why is that so? Fast growth, when achieved honestly, is a result of solving user pain that others haven't. Maybe you think so because users != the public, but I think in totality the public is a collection of users who all have needs they want met.
For certain amount of fast growth: yes. Then there's continued "growth-hacking" and enshittification to keep fast revenue growing after the pain point has been solved with dark patterns and questionable tactics.
Why? Because investors poured a bunch of money in to support fast growth and now they want their money back. And incremental growth won't do. Since 9 out of 10 of the investments fail, the surviving one has to continue to growth-hacking revenues.
Fair point. Their page doesn't list any investors.
The Times article lists several.
I don't see how not? A theoretical physicist can do all the thinking they want but if they can't test an idea against nature it's not super useful.
> To be honest, this feels more like a lifestyle business (aka hobby) than a startup
They're also incredibly productive and can build/deliver really good stuff, so who knows :)
Jeff Dean, Sanjay, et al have achieved so much. I'm very happy for them. Truly deserving.
Sometimes I couldn't resist wondering if I'll ever do work that has a tenth of the impact of theirs.
Gemini has done absolutely nothing for me. I can't even shut off the navigation feature on my phone using only hands free, when I get close to my destination. I have to take my eyes off the road, look down, and tap to exit.
Google's advanced AI cannot even exit a mobile app.
Antigravity + Gemini Pro absolutely RIPS through fullstack react + react-native apps / systems. I pay ~$20/month and I basically don't have to do my real work anymore. My time is freed up to learn systems programming and blender.
I wonder if Jeff Dean facts [1] (I hope people remember the reference) will carry over to the new startup.
[1] https://github.com/LRitzdorf/TheJeffDeanFacts
«Jeff Dean's PIN is the last 4 digits of pi.»
I had not read this before, but told many students the same about my PIN code and I a quiz about the last digits. Love it.
0000 in base pi. Oh Jeff.
Those are all fake, part of an internal Google narrative that overstates individual contribution, and obscures the work of large engineering teams.
Here are some Jeff Dean well sourced facts:
- Already part of engineering of Google indexing systems that lacked basic checksums and ran on non-ECC hardware, allowing silent data corruption.
- One of the authors of LevelDB a database with so many documented crash-consistency, recovery, and data-loss weaknesses for years. Just check their Github project. LevelDB current tracker contains unresolved crash consistency, recovery and corruption reports going back almost 12 years on GitHub
- In AI engineering technical lead, let TensorFlow lose researcher mind share to PyTorch, and caused Google fragmented landscape across TensorFlow and JAX.
- Had the people at Google who invented the Transformer architecture, but failed, to turn that lead into the first dominant public LLM.
- As AI engineering and VP management let Google Brain and DeepMind remain duplicated and internally competitive for too long.
- Let Noam Shazeer leave and then spent heavily to bring him back with nothing to show for.
- Part of Technical VP leadership who had Bard rushed to launch with factual errors in Google own promotional material.
- The first Gemini demonstration overstated how real-time and interactive the system actually was, being basically a fake.
- Part of the VP and AI technical leadership who had Google AI Overviews launched with weak source quality controls and repeated satire and low-quality web content as factual advice.
- Part of teams that launched AlphaChip performance claims that were difficult for outside researchers to reproduce and remain technically disputed.
- Jeff Dean public explanation of Gebru departure was contested and damaged confidence in Google scientific governance.
- Jeff Dean was part of the team at Google that removed or marginalized prominent internal AI ethics critics shortly before many of their warnings became product problems.
- Jeff Dean was one of the managers behind Project Dragonfly supporting censorship.
- Jeff Dean is part of the VP technical leadership approving Project Nimbus supporting an ongoing genocide.
I suppose you think Chuck Norris Facts are fake too.
You sound like you're quite jealous of him.
Actually the list is technically accurate. So maybe it's sour grapes, but it's correct sour grapes.
I had never heard of many of these, was surprised, went to research, and so far, the list seems correct.
not sure how you got that from a long list of criticisms
https://archive.ph/Pogl7
Just remember to make sure you turn in your devices.
End of an era for Alphabet.
I give it a 51% chance of Discovery Loop being acquired by Google
I actually think there's a low probability of that.
The four have so much influence within the company that they could have trivially set this up as part of Alphabet, if they wanted to. They are also ridiculously wealthy.
They are already partially funded by Google.
I wonder if this has anything to do with it
https://www.lesswrong.com/posts/iKm2FhpWkuuBojm82/why-i-left...
Doubtful. His destination is both funded by Google and has a compute deal with Google.
Careers page (if anyone is interested) → https://jobs.ashbyhq.com/Discovery-Loop
I don't see the salary (on mobile). Isn't there a law stating it must be added?
Only California employers with 15 or more employees have to post a pay range (Senate Bill 1162 [1], effective Jan 1, 2023) [2][3]. Discovery Loop employs only four people (that we know of), so it isn’t required to disclose a salary range in its job posts, of which there is only one [4].
[1] https://hr.ucmerced.edu/hr-units/talent-acquisition/senate-b...
[2] https://www.adp.com/spark/articles/2023/03/pay-transparency-...
[3] https://www.jazzhr.com/blog/pay-transparency
[4] https://jobs.ashbyhq.com/Discovery-Loop
Did the ycombinator podcast which included giving advice to startup founders just a few days ago:
https://www.ycombinator.com/library/Vy-jeff-dean-the-1-rule-...
As always, a very good presentation.
Jeff Dean leaving... as is Sir Demis...
What's happening?!
Have they had enough of Google?
I've had enough of Google and I don't even work there.
They're structuring the new company as public benefit corporation.
Does this mean anything besides for corporate virtue signaling?
Same day Google is rolling out layoffs.
Alphabet has almost 200k employees, it would be surprising if they weren't constantly laying off a ton of people (and hiring them).
"The speed of light in a vacuum used to be about 35 mph. Then Jeff Dean spent a weekend optimizing physics."
In 2018 there was a beautiful New Yorker article on Jeff Dean and Sanjay https://www.newyorker.com/magazine/2018/12/10/the-friendship...
Related:
The next chapter of our AI momentum
https://news.ycombinator.com/item?id=49184755
I've seen tiny tiny hints from the outside that Jeff Dean was dealing with too much internal BS. Two examples that come to mind: Having to deal with Timnit Gebru fiasco, and even chips in the TPU series getting marketing names (Trillium and Ironwood) before switching back to more standard numbering.
I have no doubt that internal Google friction is one of the reasons they are moving. But the Gebru incident was almost seven years ago now. It is very unlikely to be a proximate cause.
I doubt numbering vs names on TPU releases even crosses Jeff's radar. It's not the kind of thing he cares about.
Not through conscience.
Related:
Discovery Loop
https://news.ycombinator.com/item?id=49184960
Well, damn. Jeff, Sanjay and Demis leaving all at once, it's hard not to read that as someone fucked up pretty bad.
Hassabis isn't leaving
Ah, you're right. Thanks for the correction. I read that other post too fast.
Jeff and Sanjay both leaving!
Imagine being a VC and getting to this slide on their pitch deck, "involved in creating the following":
https://x.com/JeffDean/status/2085036253263921218
I wish them well, but this firm will likely fail miserably. The reason is that the value is in having access to real world hardware platforms that AI can control, not in the harness that controls them. There exist plenty of harnesses already. These people couldn't even get Google to build a top LLM. Before you dismiss and downvote, I dare you to counter it.
Oh wow demis and Jeff, that Ethics engineer who left GDM, and reached to both and posted on lesswrong a couple weeks [1] ago did make a difference?
1. https://www.lesswrong.com/posts/iKm2FhpWkuuBojm82/why-i-left...
...you think they've only been planning this for a couple weeks?
hmm not really, and it probably isnt the origin, but im glad to see him vindicated by having both C-level google leadership people who targeted to try and either stop google from doing the pentagon deal, both stepping down the same day.
At least it makes me feel like he reaching out might have made a difference on remembering this people killer robots are bad, and being part of it might not be on the best of their interests to go down on history for
Alex Turner discusses a bit here at this timestamp:
https://www.youtube.com/watch?v=pGlJQHVeKIA&t=14m27s
what was the post?
https://www.lesswrong.com/posts/iKm2FhpWkuuBojm82/why-i-left...
As LLM coding agents plateau— at least for the average engineer without tens of thousands of dollars or swarms of agents to run —I’d say that, from here on it’s going to be about ASICs, specialized LoRA/or-equivalent models, or a Ruby on Rails for LLM context engineering and orchestration, which LangChain and others seems well position, including Google as they own the entire stack. LLM free lunch has been over for a while, perhaps since the ReAct loop, and has been official since Ilya mentioned it at NeurIPS.
I feel the most exciting development these days is self-evolving agents. Especially if you have a way to verify their outputs with a formal system, or with a system developed since the 60s by armies of PhDs.
DeepMinds Gnome is a good example, where they use DFT to verify outputs. Approximating NP-problems is always fun for those who dare.
I am also building in this space. Its a mix between HPC, AI, and hard science. Pretty fun compared to waking everyday to LLM news that seem more like marketing stunts.
Sounds like you get your news from 2024 when people thought things were plateauing after GPT4?
we definitely haven't hit plateau yet. I think a lot of people latch on to anti-LLM narratives without really thinking things through.
LLM coding isn't even close to plateauing. Right now, the major players are in a consolidation step, focusing more on economic efficiency but still not at the point where we're ready to start burning models to hardware and freezing the line.
They are straddling the line between pushing it forward, and justifying the business case. It's hard to do both at the same time.
LLMs are hungry for tokens. Every day I see more startups claiming token usage at 50-100k per month. You can always brute-force your way in - just see HuggingFace's recent attacks.
If throwing more money at inference while accumulating compounding technical debt is the new norm, then we are not solving the problem, and the solution space is already covered.
Perhaps there are marginal gains at the expense of quadrillion-params LLM models with 10x the cost and energy. We are simply making inefficiency more expensive, camouflaged by VC money and great marketing.
If that is not plateauing, then I guess I will have to reconsider what plateauing means.
> You can always brute-force your way in
This is such unbelievable revisionism! Can you imagine in 2022 saying "Oh of course you can brute force your way to AGI if you spend enough money per month". Nobody thought that! Come on!