I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce.
I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to <Big Cloud Provider> with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.
If you're planning to run in clouds, committing to not buy hardware (during the contract term, presumably) isn't a big imposition. Maybe you switch to a different cloud, and you wouldn't buy hardware for that.
If you want to switch back to on prem, there's probably a way to structure acquiring hardware so it doesn't break the contract. Maybe you lease it, maybe the purchase happens through a related company, maybe there was no way for the contracted cloud to find out...
There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets
It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU
GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build.
Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy that line of reasoning.
Maybe relatedly, I expect (hope) the hardware manufacturers will ramp up supply in the meanwhile which would also put downward pressure on GPUs. Right now though this hardware crunch is making me sad, not even because of GPUs but also because of general memory / disk.
so are buybacks. you choose to sign the contract. there's no way they didn't have an escape clause, although likely it meant not using the cloud provider anymore
They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).
I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is.
And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.
The GPU alone has a TDP of 700W, together with everything else (CPU, RAM, storage, fans) you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...
That's actually less bad than I assumed. High-end gaming GPUs are already almost at 600W with peak usage above that. I assumed it was much worse than that; that seems absolutely feasible for running at home.
In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained. To use higher wattage appliances, we have to upgrade our outlets to 20A (2000/1600W) or up our voltage to 240V, but that carries a different set of plugs and outlets as well.
Usually the home service is 200+ amps these days which is 24 KW total across all circuits, though you're right that most individual circuits are only 10-15 amp.
In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)
The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power.
You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.
Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still.
What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.
All indications are there will be a lot of repossessed GPUs appearing on the market before too long. Likely to be messy for a while but will open up a lot of possibilities when it’s easy to get your hands on some secondhand GPUs.
> These are not catastrophic events. They are the steady state.
> There is no GPU futures market, no standardized residual value curve, and no way to lock in a forward rental rate. The premium is is the price of underwriting in the dark.
The headings are also AI like, a lot of essays before usually did not have titled sections but now they do and they all feel like these.
In addition the diagrams themselves look pretty AI generated.
I like Meg a lot a human, but Meg is all doom and gloom. Every single post she makes is about how GPUs fail [0] and now she's onto how financing is a big thing just waiting to crash and "nobody knows what a used GPU cluster is worth"...
Actually, we do, people offer them to me all the time. A used box of MI300x is $257k. "There is no GPU futures market"... actually there are a few of them that people have pitched to me.
This article is a lot of words from someone who isn't actually buying or deploying compute. My point is... take it all with a grain of salt.
Somehow at any given moment every possible topic to discuss on HN belongs to either the set of "it's amazing and nobody can say anything bad about it" or the set of "this thing sucks and nobody is allowed to say anything positive about it" and I never have ANY idea which one any given topic will be in on any given day.
latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the real, lived experience of such a business than the post seems to have.
Liquidate accelerators, liquidate stocks, liquidate ai hype farmers, liquidate datacenters. It will purge the rottenness out of the system. High costs of training and high ram will come down. AI will work harder, live a more local life. Values will be adjusted, and enterprising people will pick up the wrecks from less competent people
If anything, you might have to pay to have them disposed of, they don't really have any meaningful used eBay market outside of the randos that want to do high end extreme local inference in their basement.
Also, as for RAMmageddon, the inference SBCs that all of the AI bros bought don't have DIMMs, they're not even the right chip: its all GDDR and LPDDR. The only DDR DIMMs being consumed are for regular non-inference machines that help run the business and service infrastructure behind the scenes.
Would it make economic sense to strip it down and sell for parts? That’s how it’s done now for older data centers, where the obsolete equipment is sent off to China, stripped down for parts, and sold on the secondary market. I’ve picked up several older but still useful RAID hardware cards off eBay this way.
I’m mainly interested in getting some DDR4/5 and RTX5090s on the cheap :).
Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price.
Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct.
If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash.
I like the Shrek Movie correlation theory, as they always happen just before Debt-backed investors get hit hard... And the new film is due out in 2027. =3
so you're telling me you can't use 1 or 2% percent of 5 billions dollars to rebuild a team that runs GPU clusters for like a couple of years ?! and the guy that borrowed billions from these banks would want to mess up that relationship for what? I mean he is a stupid narcissist but not to that degree. This whole article makes no sense to me.
I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce.
I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to <Big Cloud Provider> with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.
How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.
> and not purchase any more.
why would anyone sign such a contract?
If you're planning to run in clouds, committing to not buy hardware (during the contract term, presumably) isn't a big imposition. Maybe you switch to a different cloud, and you wouldn't buy hardware for that.
If you want to switch back to on prem, there's probably a way to structure acquiring hardware so it doesn't break the contract. Maybe you lease it, maybe the purchase happens through a related company, maybe there was no way for the contracted cloud to find out...
There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets
It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU
GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build.
Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy that line of reasoning.
Maybe relatedly, I expect (hope) the hardware manufacturers will ramp up supply in the meanwhile which would also put downward pressure on GPUs. Right now though this hardware crunch is making me sad, not even because of GPUs but also because of general memory / disk.
That should be illegal. Sounds like a very fraudulent business tactic.
Fraudulent, I wouldn't say so. Anti-consumer or anti-competitive? Sounds like.
Would you call a trade-in a fraudulent tactic?
Trade-ins are voluntary.
so are buybacks. you choose to sign the contract. there's no way they didn't have an escape clause, although likely it meant not using the cloud provider anymore
The Grift Economy places all legalities on the marks and their inability to form legal fights.
Sounds like capitalism at its finest.
Good old "win-win-lose"
Wouldn't that be crazy - a hobbyist market for H100s?
Maybe someone could start a business buying up and rehousing these.
They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).
I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is.
And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.
This is also true of the older P100 and hobbyists do this stuff now, today
Or you can secure yourself a DLC one and try to feed it with the correct regime (liquid composition, temperature and flow rate). I'd say good luck.
These things get hot and are fussy about their requirements.
The GPU alone has a TDP of 700W, together with everything else (CPU, RAM, storage, fans) you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...
That's actually less bad than I assumed. High-end gaming GPUs are already almost at 600W with peak usage above that. I assumed it was much worse than that; that seems absolutely feasible for running at home.
Hell no. 1.5kw is half a socket capacity. A heat pump or many kitchen appliances are using much more. Our car charger uses more as well.
That's why they said "depending on country"
In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained. To use higher wattage appliances, we have to upgrade our outlets to 20A (2000/1600W) or up our voltage to 240V, but that carries a different set of plugs and outlets as well.
Usually the home service is 200+ amps these days which is 24 KW total across all circuits, though you're right that most individual circuits are only 10-15 amp.
> In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained.
It's 1800W for short periods and 1500W sustained.
In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)
A typical hair dryer uses about 1500W. I guess GPUs are mechanically not much different from a hair dryer.
Install it in your bathroom, use it like a restroom hand dryer.
The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power.
You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.
Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still.
What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.
supposedly, the next step is into fiber & optics.
but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space.
You tell them we're not living on mars any time soon and they'll bring up christopher columbus.
I wonder what shovels were worth after the Gold Rush faltered. Blacksmiths probably had all the scrap iron they could ever care for.
All indications are there will be a lot of repossessed GPUs appearing on the market before too long. Likely to be messy for a while but will open up a lot of possibilities when it’s easy to get your hands on some secondhand GPUs.
This sounds like weapons market after the crash of soviet republic. Never has been a better time to get a functioning tank or parts of nukes.
Maybe this article has something useful to say but the painfully LLM-generated prose is too distracting to make it evident.
Did an AI make all the text and kerning so ugly? Why do web sites look like this now?
Isn't this just Substack?
How do you know? I feel that it's just poorly edited.
In my experience, AI is easier to read than this was.
Some sentences feel pretty AI like:
> These are not catastrophic events. They are the steady state.
> There is no GPU futures market, no standardized residual value curve, and no way to lock in a forward rental rate. The premium is is the price of underwriting in the dark.
The headings are also AI like, a lot of essays before usually did not have titled sections but now they do and they all feel like these.
In addition the diagrams themselves look pretty AI generated.
Used GPU hosting for pension funds.
You go to ebay search for a used GPU. You get a price.
Neither is used servers a new thing or used routers. There are established used server companies.
https://en.wikipedia.org/wiki/The_Emperor%27s_New_Clothes
One data point: 512GB Mac Studios are selling on ebay for double what they were selling for new at the beginning of the year.
Time to dig up the spreadsheet models of what dark fiber is worth.
Nothing, because those "GPU's" are special proprietary hardware and are not what most people are capable of plugging in into anything.
People are selling adapter boards to plug some data center GPUs into a regular pcie slot.
The GPU cluster's RAM is probably worth more than the flops these days.
I like Meg a lot a human, but Meg is all doom and gloom. Every single post she makes is about how GPUs fail [0] and now she's onto how financing is a big thing just waiting to crash and "nobody knows what a used GPU cluster is worth"...
Actually, we do, people offer them to me all the time. A used box of MI300x is $257k. "There is no GPU futures market"... actually there are a few of them that people have pitched to me.
This article is a lot of words from someone who isn't actually buying or deploying compute. My point is... take it all with a grain of salt.
[0] https://x.com/meggmcnulty/status/2040851080066859386
Aren't the AI Datacenters for the Digital Control Grid, so govt contracts?
Somehow at any given moment every possible topic to discuss on HN belongs to either the set of "it's amazing and nobody can say anything bad about it" or the set of "this thing sucks and nobody is allowed to say anything positive about it" and I never have ANY idea which one any given topic will be in on any given day.
latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the business of it than the post does.
That's great, their comment should get more attention then.
I was complaining that it's obviously incorrect that nobody knows what used GPUs are worth, not about latchkey.
I have NO idea why anyone is upvoting a post titled "nobody knows what a used GPU cluster is worth", that is a WILD claim.
she makes a lot of wild claims for clicks and gets to the front page with them... sigh.
latchkey is being restrained. He runs a data center filled with AMD GPUs. He's got a lot more insight to the real, lived experience of such a business than the post seems to have.
Liquidate accelerators, liquidate stocks, liquidate ai hype farmers, liquidate datacenters. It will purge the rottenness out of the system. High costs of training and high ram will come down. AI will work harder, live a more local life. Values will be adjusted, and enterprising people will pick up the wrecks from less competent people
GPU clusters have, largely, no actual value.
If anything, you might have to pay to have them disposed of, they don't really have any meaningful used eBay market outside of the randos that want to do high end extreme local inference in their basement.
Also, as for RAMmageddon, the inference SBCs that all of the AI bros bought don't have DIMMs, they're not even the right chip: its all GDDR and LPDDR. The only DDR DIMMs being consumed are for regular non-inference machines that help run the business and service infrastructure behind the scenes.
so when will xAI default on its debt?
Would it make economic sense to strip it down and sell for parts? That’s how it’s done now for older data centers, where the obsolete equipment is sent off to China, stripped down for parts, and sold on the secondary market. I’ve picked up several older but still useful RAID hardware cards off eBay this way.
I’m mainly interested in getting some DDR4/5 and RTX5090s on the cheap :).
Given the price for a rack of bc-250 after the crypto hype cycle, the expected value of the hardware will be around 5% to 10% of the original retail price.
Without other market influences, that is a >90% expected discount when the over-provisioned market must inevitably self-correct.
If the Market follows what Samsung/SK Hynix did to the South Korean exchange this week, than the "AI" bubble will hit harder than the dot com crash.
I like the Shrek Movie correlation theory, as they always happen just before Debt-backed investors get hit hard... And the new film is due out in 2027. =3
so you're telling me you can't use 1 or 2% percent of 5 billions dollars to rebuild a team that runs GPU clusters for like a couple of years ?! and the guy that borrowed billions from these banks would want to mess up that relationship for what? I mean he is a stupid narcissist but not to that degree. This whole article makes no sense to me.
If nobody knows what a used GPU cluster is worth it means nobody is doing anything important with GPUs - time to short everything. Do you believe it?