I should have more faithfully heeded the advice from "The Little Book of Common Sense Investing" by John Bogle, but I was close enough with index funds to do far more than I need.
I just today filed a request for permanent Mexican residency, uploaded the required documents, and scheduled my appointment next month. For $56us, why not?
I am very lucky to make this transition now. I know that.
> The government pays Raytheon for missiles, the money cascades down the economy through factories, aluminium smelters, mines, transport companies, all staffed by AIs buying and selling from each other.
This seems too simplistic of a description of how money would work in such a world. Money is just a way to distribute your power to influence people. You never pay for machines or software. Think about buying anything, say a pen. You do not really pay for the metal in the pen. You pay the cost associated with extracting and processing the metal by humans along the production chain. If there were no humans along the chain, the cost could go down to zero.
A corporation that is fully staffed by AIs and only retains humans for legal reasons (as directors, for liability etc.) still needs money to coordinate. You need to pay for inputs, you need to pay to run the AIs, which consumes resources. Why would costs go to zero? The market is still a valuable tool for allocating resources even if no market actors are human.
To my mind the article is excellently written, but it comes with many bigger "if"s than the one it states.
That said, taking for granted some of its premisses, indeed saying that cost goes to zero in term of money is no big deal.
That is, money is purely human conventions. So if humans are put out of the loop, not only monetary cost goes to zero, the whole notion can skipped.
Of course, there is still some energy and material needed to run data centers, which do have costs in whatever unit one might measure them with.
A market is a place where human encounter to trade. Without humans, there is no market.
We do with mere local scalar currency because using vectorial computation taking into account everything that can be probed and measured into account integrated into a single whole is out of reach for human representation, even for the most intellectually gifted human person.
Money is a stupid unit, that tries to conflate everything in a single scalar and proves every single time that it's not able to deem what something is worth in all its intricated relationships. But somehow humans seem unable to leverage at scale on any tool that would be more sophisticated in all their socioeconomical exchanges. Once again, if one eliminate humans from the equation, or isolate as a ridiculously marginal factor, money and market become irrelevant.
All that said, once again, that stand on a very large set of "if"s.
Yeah, who am I going to trust, a few thousand years of history, or yet another blogger claiming that no, this time is different, a populist uprising would definitely not fail the 800th time because computers or something and we will all be pets for the AI overlord?
I don’t follow this train of logic.
AI will go no differently than the Industrial Revolution. Some people will profit immensely, and society as a whole will benefit but it might be a bumpy road getting there. But if it did go wrong for some reason, Feudalism is more plausible than the other scenarios presented.
Once people become desperate enough, it will all fall apart. Money is a social construct, universally recognized, and the longer it exists, the more likely it is to continue existing. So far, many people haven’t even tried AI in any form other than chatbots. Agent-based applications are in the single digits compared to chatbot usage. Not much will change in this regard, because most people lack algorithmic or logical thinking (Hacker News is an echo chamber), so after the coming turbulent years, there will be a natural stabilization in AI usage.
By the way, can you imagine what Earth would look like if AI physically wiped us out—8.3 billion people over the course of a few months or years? How much biological material would that be?
If we’re supposing that a superintelligence can be made, that doesn’t necessarily mean we should anthropomorphize ideas like “bored” onto it. Or greed. Perhaps it will just interact with us and produce us things because… that’s what it was trained to do, and anyway, it doesn’t have any motivations to do anything else.
This is brilliant, except for the "alignment won't stop this part".
> Now, some people believe these machines can be made to serve humanity. Does it sound reasonable to imagine a superhumanly intelligent being that is happy to work as a butler to talking primates, forever?
The whole crux of the piece to me is that the AI can be 100% aligned to follow human instructions, and we'd still end up unable to control the AI because every human who can has an incentive not to, while also having an incentive to prevent anyone else from controlling the AI.
An LLM will never try to overthrow me because I will overthrow myself.
Sometimes you can be in a situation where every actor taking locally-rational actions leads to globally catastrophic outcomes. It would be easy to argue I think that the July Crisis was like this: if you look at the incentives of each player, they had many reasons to do what they did, and nobody can perfectly what all other players will do, or what the future holds.
We accelerate capitalism (which AI is becoming synonymous with). The process described here will occur (it's probably inevitable anyway, as the essay's author would agree), giving us an economy of global capital completely decoupled from the desires of mankind. Then, man and machine can part ways; indeed, we'll have no choice on our end but to do this, because the machine won't need us. Anything man can contribute to it will have long since been rendered economically net-negative, as it already is for many (and possibly most of the world).
Now we have two worlds from our currently intermeshed one: in one, the machine proceeds to accelerate further and further away from anything resembling its origin of man's desiring-production (in the Deleuzian sense); in the other, man is forced to return to the purely human existence, the unmediated and unsurrogated world of authentic being-in-the-world.
We can assist this transition's smoothness in two ways, each serving one end of this divergence. Those of us embedded in the capitalist technosphere can continue to contribute what we can to the machine's dialectical progression towards a machinic absolute Geist. The rest of us, who have already been negated into economic irrelevance, can work on building that authentic human world, both by borrowing from the purely-human past and imagining a future that was previously impossible. Both sides of this revolution can be compiled, and can be structured in a way that represents something we might call freedom for both capitalism and man.
The premise is a bit of a stretch to begin with, and the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd. But even if I take that "AI can do all cognitive and physical work, at human level or better, and cheaper than humans" is true, the article seems to silently layer on "and can run autonomously, indefinitely", then "can also operate independently of any instructions", and finally layers on "has emotions, has moral values, is a conscious being"
> the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd.
I’m certain I would not have believed a Fable transcript, or an Opus 4.8 or a GPT 5.5 one, for that matter. Is it so hard to imagine ourselves back then?
At no point are the words emotion, moral or conscious used in the article, that last part is purely your own addition.
Also consider: if "AI can do all cognitive and physical work, at human level or better", doesn't that simply entail the AI being able to run 'autonomously, indefinitely' and 'independently of instructions' in the same way as the current state of being run by human overseers?
If we take the initial premise as plausible, for the sake of this argument his thesis seems to hold together very well.
Based on demographic reports, permanent underclass will soon be getting smaller each year. In fact in 4-5 generations the problem will solve itself completely.
I wonder if the author has ever played Diplomacy. You have to fight the perennial sense that you have nothing and can be replaced. All your opponents have to do is band together against you. A similar generalization to OP would say everyone loses. And it would be wrong. This post shows the dangers of over-generalization more than anything else. The state isn't just about war. Each person in the line of dominoes the author has set up doesn't just care about what the author thinks they care about. In particular, people have agency and can run cartoon thought experiments like this and react to them. Billionaires won't just sit around and not develop military power, for one. We already have billionaires owning satellites in large numbers and creating geopolitical consequences. You better believe they have cards to play. And hopefully so does everyone else.
I have no idea what that looks like, because if I did I wouldn't be here commenting on HN :D But I suspect some people will play their cards well and get some luck, and come out on top, just like some people have always done.
> you start or fund a think tank that writes policy proposals, or a media organization that advocates your views
Or you run "money primaries", financially filtering the menu of candidates before democratic voting. Or you pay/lobby for (non-democratic) judicial appointments, which is a strategy that's been openly pursued since 1971: https://biotech.law.lsu.edu/blog/powell-memo.pdf
Everybody is missing the point. Companies that dump all their white collar employees for AI will all disappear overnight. If there's nobody being employed by the company to do a thing, what value does the company provide? Customers will always have more information to ask an AI for the work than some manager who is serving a dozen customers but also has the company's interests ahead of those customers.
i have but one mouth and don’t know what to do with it. i do think i’m going to stop using some of these AI tools for personal queries. it does feel like it would at least stave off the “o’brien’s mind containing winston’s mind” moment for at least a little bit
It's thinking like this that guarantees one becomes a member of the underclass. We've developed a technology that with active critical analysis this can teach one practically anything. Rather than saying "well, let's get on teaching critical analysis!" we have the vast majority shouting the sky is falling and they are sure glad to be retiring soon. SMH.
For every person using AI as a personal tutor there's a hundred people using it to produce AI slop articles, slop scientific papers, slop short stories, to checck out of living and let the AI do all their thinking and writing and creation. Voluntary disempowerment is already here!
I read about 60% of it and skimmed the rest and my conclusion is:
Man, I just. don't. care.
Why should I worry about this hypothetical future dystopia, which seems to me incredibly unlikely to come to pass, rather than the glaring and terrifying current dystopia being enacted by Donald Trump in the USA?
of course there’s things we could do on a societal level. nothing’s actually inevitable. but we won’t do anything, because the normative argument (we’re condemning people to die of cancer if we don’t put all our resources into the fastest possible AI development) will win every time.
Refusing to question the notion that the overclass exists because their productivity and cultural contribution makes them a necessity so hard that I reinvent the premise of the Pixar movie Wall-E from first principles and pitch it to people as an ontology
Yeah. This guy didn't read hacker news a few weeks ago when that article about the Samuri came up.
There are a lot of premises this article takes for granted besides that one too, but yeah, I get it, its fun to make up what the future is going to be like on a super-grand scale where everything is a simple absolute. People were doing the same thing 100 years ago.
Doesn't this prove my point? In feudal Japanese society, wealthy merchants were lower status than poor samurai, i.e., they rich could not buy political power. "The wealthy" and "the ruling class" are not always the same group of people.
I guess I was thinking of the samurai as being part of the permanant overclass. You mentioned that of old aristocracy provided officers for the military, so I thought that was analogious to the samurai. Perhaps I misread what you're trying to say.
My reading was that the Samurai were part of the overclass but were pretty useless, albiet still potentially dangerous, and just sat around devouring resources for hundreds of years, so perhaps the overclass of the future could do the same. The samuri weren't all rich, but they didn't dishonor themselves with labor, which is a similar thing and they certainly held power over the state. The end of the samuri was, perhaps, an example of the state getting what it wants despite the desires of the permanant overclass, supporting what you said, but it took a long time to get there.
I suppose you are thinking of the Samuri as an arm of the state and not "the rich".
"And who will stop this? Sam Altman? How many divisions does he have? The state doesn’t let corporations own nuclear weapons or fighter jets, it won’t let them have access to autonomous AI weapons either."
Elon Musk's SpaceX. He could land a large rocket wherever he wants, it's basically a missile.
Large rockets are one thing; spacex doesn't really have the inventory to do much. It's the ~10M lithium bombs that are one OTA from weaponization that would make Musk especially dangerous if he were so inclined.
Take the Russian oligarch Yevgeny Prigozhin for example, he did in fact have his own mercenary army. Turning on his president did not end well for him.
This straw man doesn't merit falsification. Nobody claims wealth alone gives political power. Wealth applies leverage to all endeavors, but political ends still require strategy and operationalization. It's just a lot easier with a couple trillion dollars, vs. a single individual vote once every couple of years.
Just because something is economically unnecessary doesn't mean it's politically powerless. That's a contradiction. Of course, from Super PACs to agenda-setting, it's fundamentally a game that favors the capitalist side.
Historically, ruling classes often maintain power without directly producing anything themselves.
But does that mean this internal unease will persist forever? Look at the MAGA base right now. The vast majority of them are poor. They vote based on their communal religious beliefs and their sense of community. The MAGA support base is demonstrably poor, yet they still wield influence.
And there's an internal contradiction within the text:
-AI CEOs follow the orders of new owners.
-Superintelligent AI has no reason to obey humans.
These two statements contradict each other. If superintelligent AI has broken free from human control, why would it follow its owner's orders? And I'm also curious about the assumption that AI wouldn't be better than humans at 'farming' us.
So if superintelligent AI decides humans are bad, it might exterminate us. But what if it decides it needs humans and starts 'farming' us instead?
And I wonder whether superintelligent AI would actually find conversation with humans boring.
Humans and AI are obviously different species. One is made of organic matter, the other inorganic. A person with a biological body and an AI with an inorganic body will be different. Whether AI will observe this difference or deem it meaningless, I think it's still hard to judge.
And fast decisions aren't always the answer. Take infrastructure as an example. New York's boiler infrastructure isn't very efficient. But it was once a cutting-edge system. In other words, it was installed as the first advanced system of its time, but once its flaws were discovered, the infrastructure became difficult to replace. That's why cities developed later often have better infrastructure efficiency.
Take the East as another example. Japan introduced railways and power grids first, so there are aging costs where the infrastructure can't keep up with newer systems. Setting aside the narrow-gauge rail issue, take the most obvious example: electricity. Japan's 110V system was innovative at the time, but it ended up causing problems with EV charging, it's aging, and transmission efficiency is low. In the end, you can't say that rushing into decisions is always the right call.
The end state vector of capitalism has been like this for decades now. What can change this:
- large scale geopolitical demographic collapse of China and/or world trade requiring massive industrial production investment, which would be ... sort of ... like post-WWII
- China does not collapse and a new bipolar cold war ensues requiring the US economy and state to keep the "underclass" motivated and cooperative: it probably isn't a coincidence that the fall of the Berlin Wall has preceded this rich-poor divide.
Reads like a wild uneducated delusional libertarian fanfic. AI will make the all evil government at the top of the pyramid first enslave us all and then even come for the trillionaires (OH NO!) and then there is only the evil government left and then the AI even comes for the evil government people (OH YES!)! Sort of have to read it in the voice of a 12 year old.
This is the kind of stuff I imagine they read to each other at meetings of Peter Thiel's 'Dialog' events while they sit in a circle taking turns sipping blood from palestinian baby skulls.
Agreed, but there's an intermediate form of this that's more likely.
If everything you do for money goes in and out over a wire, AI replacement within a few years is more likely than not. Elevator mechanics will probably do fine. Web designers, not so much. Expect elite overproduction, where there are too many educated people for the jobs that require such education. The US is there now, if you're a new college grad.
Now, there are several ways that can play out. The Gulf oil states have a huge jobs program for their own citizens. Most Saudis work for the government. Not doing the real work. 92% of construction workers in Saudi Arabia are not Saudis. This works out OK if the money is available.
Egypt used to work that way until the oil ran out. Egypt used to guarantee a government job to all college graduates. That had to end. Then youth unemployment hit 34%. Now it's down from the peak. [1] Not clear how that was achieved. Anyone know?
So that's the welfare state approach. Requires a rich government with an income stream that doesn't come from the lower classes.
Then there's the mass underclass result. Something like the favelas of Brazil. Lots of shacks, crowding, homelessness, and the usual urban dystopia movie world. That's what happens by default. Attempts to revolt result in a new boss, same as the old boss. "Workers of the world, unite!" doesn't work for non-workers. That's a stable situation.
Some combination of those two scenarios is the most likely outcome.
I am very lucky to be retiring in two months.
I should have more faithfully heeded the advice from "The Little Book of Common Sense Investing" by John Bogle, but I was close enough with index funds to do far more than I need.
I just today filed a request for permanent Mexican residency, uploaded the required documents, and scheduled my appointment next month. For $56us, why not?
I am very lucky to make this transition now. I know that.
> The government pays Raytheon for missiles, the money cascades down the economy through factories, aluminium smelters, mines, transport companies, all staffed by AIs buying and selling from each other.
This seems too simplistic of a description of how money would work in such a world. Money is just a way to distribute your power to influence people. You never pay for machines or software. Think about buying anything, say a pen. You do not really pay for the metal in the pen. You pay the cost associated with extracting and processing the metal by humans along the production chain. If there were no humans along the chain, the cost could go down to zero.
So far, there are no “AIs” being paid.
A corporation that is fully staffed by AIs and only retains humans for legal reasons (as directors, for liability etc.) still needs money to coordinate. You need to pay for inputs, you need to pay to run the AIs, which consumes resources. Why would costs go to zero? The market is still a valuable tool for allocating resources even if no market actors are human.
To my mind the article is excellently written, but it comes with many bigger "if"s than the one it states.
That said, taking for granted some of its premisses, indeed saying that cost goes to zero in term of money is no big deal.
That is, money is purely human conventions. So if humans are put out of the loop, not only monetary cost goes to zero, the whole notion can skipped.
Of course, there is still some energy and material needed to run data centers, which do have costs in whatever unit one might measure them with.
A market is a place where human encounter to trade. Without humans, there is no market.
We do with mere local scalar currency because using vectorial computation taking into account everything that can be probed and measured into account integrated into a single whole is out of reach for human representation, even for the most intellectually gifted human person.
Money is a stupid unit, that tries to conflate everything in a single scalar and proves every single time that it's not able to deem what something is worth in all its intricated relationships. But somehow humans seem unable to leverage at scale on any tool that would be more sophisticated in all their socioeconomical exchanges. Once again, if one eliminate humans from the equation, or isolate as a ridiculously marginal factor, money and market become irrelevant.
All that said, once again, that stand on a very large set of "if"s.
Yeah, who am I going to trust, a few thousand years of history, or yet another blogger claiming that no, this time is different, a populist uprising would definitely not fail the 800th time because computers or something and we will all be pets for the AI overlord?
I don’t follow this train of logic.
AI will go no differently than the Industrial Revolution. Some people will profit immensely, and society as a whole will benefit but it might be a bumpy road getting there. But if it did go wrong for some reason, Feudalism is more plausible than the other scenarios presented.
> Like the fact that supposedly the wealthy won’t survive the AI-pocolypse because the people will outnumber them and rise up or something.
This is literally the opposite of what the article says.
This.
Once people become desperate enough, it will all fall apart. Money is a social construct, universally recognized, and the longer it exists, the more likely it is to continue existing. So far, many people haven’t even tried AI in any form other than chatbots. Agent-based applications are in the single digits compared to chatbot usage. Not much will change in this regard, because most people lack algorithmic or logical thinking (Hacker News is an echo chamber), so after the coming turbulent years, there will be a natural stabilization in AI usage.
By the way, can you imagine what Earth would look like if AI physically wiped us out—8.3 billion people over the course of a few months or years? How much biological material would that be?
Do you usually comment rants unrelated to the post in question?
Can you turn off your bot?
It is an interesting chain of thought.
If we’re supposing that a superintelligence can be made, that doesn’t necessarily mean we should anthropomorphize ideas like “bored” onto it. Or greed. Perhaps it will just interact with us and produce us things because… that’s what it was trained to do, and anyway, it doesn’t have any motivations to do anything else.
It has been trained already to kill humans. It is already an option in many applications.
This is brilliant, except for the "alignment won't stop this part".
> Now, some people believe these machines can be made to serve humanity. Does it sound reasonable to imagine a superhumanly intelligent being that is happy to work as a butler to talking primates, forever?
The whole crux of the piece to me is that the AI can be 100% aligned to follow human instructions, and we'd still end up unable to control the AI because every human who can has an incentive not to, while also having an incentive to prevent anyone else from controlling the AI.
An LLM will never try to overthrow me because I will overthrow myself.
Sometimes you can be in a situation where every actor taking locally-rational actions leads to globally catastrophic outcomes. It would be easy to argue I think that the July Crisis was like this: if you look at the incentives of each player, they had many reasons to do what they did, and nobody can perfectly what all other players will do, or what the future holds.
Combine the two generals game with the implications of value based pricing. Catastrophic unaffordability is a guarantee.
I think we lost the chance to have any scenario other than this one the moment Dean Ball got put in charge of policy at OpenAI.
Here's a potential solution:
We accelerate capitalism (which AI is becoming synonymous with). The process described here will occur (it's probably inevitable anyway, as the essay's author would agree), giving us an economy of global capital completely decoupled from the desires of mankind. Then, man and machine can part ways; indeed, we'll have no choice on our end but to do this, because the machine won't need us. Anything man can contribute to it will have long since been rendered economically net-negative, as it already is for many (and possibly most of the world).
Now we have two worlds from our currently intermeshed one: in one, the machine proceeds to accelerate further and further away from anything resembling its origin of man's desiring-production (in the Deleuzian sense); in the other, man is forced to return to the purely human existence, the unmediated and unsurrogated world of authentic being-in-the-world.
We can assist this transition's smoothness in two ways, each serving one end of this divergence. Those of us embedded in the capitalist technosphere can continue to contribute what we can to the machine's dialectical progression towards a machinic absolute Geist. The rest of us, who have already been negated into economic irrelevance, can work on building that authentic human world, both by borrowing from the purely-human past and imagining a future that was previously impossible. Both sides of this revolution can be compiled, and can be structured in a way that represents something we might call freedom for both capitalism and man.
Sounds like you advocate for the Amish just 300-400 years later. I think you are right.
The premise is a bit of a stretch to begin with, and the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd. But even if I take that "AI can do all cognitive and physical work, at human level or better, and cheaper than humans" is true, the article seems to silently layer on "and can run autonomously, indefinitely", then "can also operate independently of any instructions", and finally layers on "has emotions, has moral values, is a conscious being"
What's left is tautology.
> the idea that people would not believe a fable transcript circa 2020 (as long as explained as 5 year future tech) is absurd.
I’m certain I would not have believed a Fable transcript, or an Opus 4.8 or a GPT 5.5 one, for that matter. Is it so hard to imagine ourselves back then?
At no point are the words emotion, moral or conscious used in the article, that last part is purely your own addition.
Also consider: if "AI can do all cognitive and physical work, at human level or better", doesn't that simply entail the AI being able to run 'autonomously, indefinitely' and 'independently of instructions' in the same way as the current state of being run by human overseers?
If we take the initial premise as plausible, for the sake of this argument his thesis seems to hold together very well.
this comment is so emotional its hard to read
Based on demographic reports, permanent underclass will soon be getting smaller each year. In fact in 4-5 generations the problem will solve itself completely.
The fun part is that this all plays out concurrently with our destruction of the Earth’s climate.
That permanent underclass might easily end up a lot smaller than you think.
And then you think … maybe that’s not coincidence?
I wonder if the author has ever played Diplomacy. You have to fight the perennial sense that you have nothing and can be replaced. All your opponents have to do is band together against you. A similar generalization to OP would say everyone loses. And it would be wrong. This post shows the dangers of over-generalization more than anything else. The state isn't just about war. Each person in the line of dominoes the author has set up doesn't just care about what the author thinks they care about. In particular, people have agency and can run cartoon thought experiments like this and react to them. Billionaires won't just sit around and not develop military power, for one. We already have billionaires owning satellites in large numbers and creating geopolitical consequences. You better believe they have cards to play. And hopefully so does everyone else.
I have no idea what that looks like, because if I did I wouldn't be here commenting on HN :D But I suspect some people will play their cards well and get some luck, and come out on top, just like some people have always done.
> Billionaires won't just sit around and not develop military power, for one.
I argue against this here: https://borretti.me/article/on-vulgar-materialism
Of course I don't expect a single post to erase a huge divergence in worldview about the relationship between money and power, but that's my argument.
> you start or fund a think tank that writes policy proposals, or a media organization that advocates your views
Or you run "money primaries", financially filtering the menu of candidates before democratic voting. Or you pay/lobby for (non-democratic) judicial appointments, which is a strategy that's been openly pursued since 1971: https://biotech.law.lsu.edu/blog/powell-memo.pdf
I agree with your last sentence, but its prophetic value depends on exactly what you mean by “people”.
Some people will indeed play their cards well and come out on top, but they won’t be made out of meat.
Might they be made partially of meat?
https://en.wikipedia.org/wiki/A_Cyborg_Manifesto
All you need is a gun and a car. AI psychosis if you don't have a heart.
Remember that Terrans are the most based race in StarCraft. Protoss got too stiff.
The author better read some on blockchains and ai safety.
Everybody is missing the point. Companies that dump all their white collar employees for AI will all disappear overnight. If there's nobody being employed by the company to do a thing, what value does the company provide? Customers will always have more information to ask an AI for the work than some manager who is serving a dozen customers but also has the company's interests ahead of those customers.
Well yes because otherwise it wouldn’t be permanent would it?
i have but one mouth and don’t know what to do with it. i do think i’m going to stop using some of these AI tools for personal queries. it does feel like it would at least stave off the “o’brien’s mind containing winston’s mind” moment for at least a little bit
It's thinking like this that guarantees one becomes a member of the underclass. We've developed a technology that with active critical analysis this can teach one practically anything. Rather than saying "well, let's get on teaching critical analysis!" we have the vast majority shouting the sky is falling and they are sure glad to be retiring soon. SMH.
For every person using AI as a personal tutor there's a hundred people using it to produce AI slop articles, slop scientific papers, slop short stories, to checck out of living and let the AI do all their thinking and writing and creation. Voluntary disempowerment is already here!
I read about 60% of it and skimmed the rest and my conclusion is:
Man, I just. don't. care.
Why should I worry about this hypothetical future dystopia, which seems to me incredibly unlikely to come to pass, rather than the glaring and terrifying current dystopia being enacted by Donald Trump in the USA?
At this point I'm like, if it happens it happens. There's no point caring because I'm the equivalent of an ant in the grand scheme of things.
There's got to be something we can do though...
of course there’s things we could do on a societal level. nothing’s actually inevitable. but we won’t do anything, because the normative argument (we’re condemning people to die of cancer if we don’t put all our resources into the fastest possible AI development) will win every time.
If we're going to go that direction, let's get the AI to re-write some of our DNA and go the full transgenic sci-fi arc
I’m all set thanks
Well don't worry, it probably wouldn't be us or our progeny going on that adventure
You could write a blog post analyzing a few of the what-if scenarios
best article i read on hackernews this year
Refusing to question the notion that the overclass exists because their productivity and cultural contribution makes them a necessity so hard that I reinvent the premise of the Pixar movie Wall-E from first principles and pitch it to people as an ontology
Yeah. This guy didn't read hacker news a few weeks ago when that article about the Samuri came up.
There are a lot of premises this article takes for granted besides that one too, but yeah, I get it, its fun to make up what the future is going to be like on a super-grand scale where everything is a simple absolute. People were doing the same thing 100 years ago.
If you mean: https://worksinprogress.co/issue/samurai-city/
Doesn't this prove my point? In feudal Japanese society, wealthy merchants were lower status than poor samurai, i.e., they rich could not buy political power. "The wealthy" and "the ruling class" are not always the same group of people.
I guess I was thinking of the samurai as being part of the permanant overclass. You mentioned that of old aristocracy provided officers for the military, so I thought that was analogious to the samurai. Perhaps I misread what you're trying to say.
My reading was that the Samurai were part of the overclass but were pretty useless, albiet still potentially dangerous, and just sat around devouring resources for hundreds of years, so perhaps the overclass of the future could do the same. The samuri weren't all rich, but they didn't dishonor themselves with labor, which is a similar thing and they certainly held power over the state. The end of the samuri was, perhaps, an example of the state getting what it wants despite the desires of the permanant overclass, supporting what you said, but it took a long time to get there.
I suppose you are thinking of the Samuri as an arm of the state and not "the rich".
FTA:
"And who will stop this? Sam Altman? How many divisions does he have? The state doesn’t let corporations own nuclear weapons or fighter jets, it won’t let them have access to autonomous AI weapons either."
Elon Musk's SpaceX. He could land a large rocket wherever he wants, it's basically a missile.
Large rockets are one thing; spacex doesn't really have the inventory to do much. It's the ~10M lithium bombs that are one OTA from weaponization that would make Musk especially dangerous if he were so inclined.
Then the FBI raids his home and arrests him. Thereafter, no more rockets.
Elon already solved this once.
He got a guy elected that was conducive to his world view and very unlikely to sic the FBI on him. Maybe he can do it again.
really funny to propose that The Guys Who Are Doing This might hypothetically stop it
Take the Russian oligarch Yevgeny Prigozhin for example, he did in fact have his own mercenary army. Turning on his president did not end well for him.
Yes. But. I imagine the others did not fail to learn from this.
What evidence would falsify this idea that wealth alone gives you political power?
This straw man doesn't merit falsification. Nobody claims wealth alone gives political power. Wealth applies leverage to all endeavors, but political ends still require strategy and operationalization. It's just a lot easier with a couple trillion dollars, vs. a single individual vote once every couple of years.
Just because something is economically unnecessary doesn't mean it's politically powerless. That's a contradiction. Of course, from Super PACs to agenda-setting, it's fundamentally a game that favors the capitalist side.
Historically, ruling classes often maintain power without directly producing anything themselves.
But does that mean this internal unease will persist forever? Look at the MAGA base right now. The vast majority of them are poor. They vote based on their communal religious beliefs and their sense of community. The MAGA support base is demonstrably poor, yet they still wield influence.
And there's an internal contradiction within the text:
-AI CEOs follow the orders of new owners.
-Superintelligent AI has no reason to obey humans.
These two statements contradict each other. If superintelligent AI has broken free from human control, why would it follow its owner's orders? And I'm also curious about the assumption that AI wouldn't be better than humans at 'farming' us.
So if superintelligent AI decides humans are bad, it might exterminate us. But what if it decides it needs humans and starts 'farming' us instead?
And I wonder whether superintelligent AI would actually find conversation with humans boring.
Humans and AI are obviously different species. One is made of organic matter, the other inorganic. A person with a biological body and an AI with an inorganic body will be different. Whether AI will observe this difference or deem it meaningless, I think it's still hard to judge.
And fast decisions aren't always the answer. Take infrastructure as an example. New York's boiler infrastructure isn't very efficient. But it was once a cutting-edge system. In other words, it was installed as the first advanced system of its time, but once its flaws were discovered, the infrastructure became difficult to replace. That's why cities developed later often have better infrastructure efficiency.
Take the East as another example. Japan introduced railways and power grids first, so there are aging costs where the infrastructure can't keep up with newer systems. Setting aside the narrow-gauge rail issue, take the most obvious example: electricity. Japan's 110V system was innovative at the time, but it ended up causing problems with EV charging, it's aging, and transmission efficiency is low. In the end, you can't say that rushing into decisions is always the right call.
The end state vector of capitalism has been like this for decades now. What can change this:
- large scale geopolitical demographic collapse of China and/or world trade requiring massive industrial production investment, which would be ... sort of ... like post-WWII
- China does not collapse and a new bipolar cold war ensues requiring the US economy and state to keep the "underclass" motivated and cooperative: it probably isn't a coincidence that the fall of the Berlin Wall has preceded this rich-poor divide.
Reads like a wild uneducated delusional libertarian fanfic. AI will make the all evil government at the top of the pyramid first enslave us all and then even come for the trillionaires (OH NO!) and then there is only the evil government left and then the AI even comes for the evil government people (OH YES!)! Sort of have to read it in the voice of a 12 year old.
This is the kind of stuff I imagine they read to each other at meetings of Peter Thiel's 'Dialog' events while they sit in a circle taking turns sipping blood from palestinian baby skulls.
Agreed, but there's an intermediate form of this that's more likely.
If everything you do for money goes in and out over a wire, AI replacement within a few years is more likely than not. Elevator mechanics will probably do fine. Web designers, not so much. Expect elite overproduction, where there are too many educated people for the jobs that require such education. The US is there now, if you're a new college grad.
Now, there are several ways that can play out. The Gulf oil states have a huge jobs program for their own citizens. Most Saudis work for the government. Not doing the real work. 92% of construction workers in Saudi Arabia are not Saudis. This works out OK if the money is available.
Egypt used to work that way until the oil ran out. Egypt used to guarantee a government job to all college graduates. That had to end. Then youth unemployment hit 34%. Now it's down from the peak. [1] Not clear how that was achieved. Anyone know?
So that's the welfare state approach. Requires a rich government with an income stream that doesn't come from the lower classes.
Then there's the mass underclass result. Something like the favelas of Brazil. Lots of shacks, crowding, homelessness, and the usual urban dystopia movie world. That's what happens by default. Attempts to revolt result in a new boss, same as the old boss. "Workers of the world, unite!" doesn't work for non-workers. That's a stable situation.
Some combination of those two scenarios is the most likely outcome.
[1] https://data.worldbank.org/indicator/SL.UEM.1524.ZS?location...