I'm posting from a machine with 8GiB of DDR3-1333. The modern web is slow, but I avoid the modern web as much as possible anyway. The software I actually enjoy using is mostly fast enough. Swap on zram (LZ4 compression, vm.page-cluster = 0, earlyoom userspace OOM killer, no disk swap) makes it feel like 9GiB with no perceptible downside.
I have a laptop with soldered RAM. The motherboard upgrade to a 16GB variant was 120 EUR. Same price as the RAM soldered on it. It came with a CPU upgrade as well. I'm happy.
Heh well, forget DDR4, DDR3 rides again https://www.youtube.com/watch?v=e4c2FSSaCsw
I would be interested to see a comparison of the inflation caused by the AI rollout versus the actual value generated by the use of AI. I really wouldn’t be surprised if the later is dwarfed by the former (that’s assuming integrating AI in a company generates significant economic value in the first place, which is still speculative as far as I’m aware)
As an owner of a 1700 socket mainboard and 64GB of DDR4 (recently acquired a second 32GB kit on eBay), I am very pleased to see this.
I am still using the machine I built in 2017.. Prices for GPUs have been consistently insane since shortly after then, (NFT-idiots, crypto-bros, now AI-twats) and now ram.. My work setup uses 100% of my ram now though. I'd like to double my ram by getting 2 more of the same sticks I have in it.. But the exact same sticks of DDR4 now cost more than they did in 2017.
This will drive linux adoption like nothing else.
Hmm. That might push me further towards just decomissioning this machine and using the laptop with a dock instead of rebuilding with Linux. The 64Gb of DDR4 in it might be worth a fair bit second-hand more than I assumed. I'll have to scan my box of old parts too, I'm sure there is some DDR4 in there (or maybe that is old enough to be DDR3…).
I would like to see a Raspberry PI with DDR3 RAM. The price is insane right now.
Only a couple of my work machines use DDR4. Most of my own kit, both for work and for play, uses DDR3 which is cheap and plentiful.
I avoid dedicated GPUs which have always been a money-pit. I find modern video games dull and boring, and this is not offset by them being very very pretty. Gaming for me peaked with Quake II and there's been nothing of real interest since Portal and Untitled Goose Game.
Don’t call it a comeback, I’ve been here for years!
I have a Dell R740xd that is the core of my home lab, with 384 GB of DDR4 ECC. That was by far the most expensive part of the server. I get a lot of use out of it, but it certainly contributes to a higher-than-average power bill. I'm working on solar at some point next year.
I just sold 4x8GB of DDR4 that I wasn't using for $200 on Ebay so this article is quite timely
I hope optimization comes back into fashion
My idea was taking 20+ year old designs that are patent-free, implementing them on any modern nodes that work (esp MOSIS-supported), their performance per watt will probably be better, and just use those. Some of the old designs, like NonStop architecture or MPP clusters, still have amazing benefits.
Fact is that performance intensive code, games or whatever, has always optimised locality and so RAM speed shouldn't usually be a 1:1 impact on performance. Using older generations of slower RAM will be acceptable for many games and workloads.
I was eyeing a 96GB ddr5 so-dimm upgrade for my laptop last year, it was about 200-250 euro at the time. The only offerings I find for it now are in the range of 2000 euro. Regretting that I didn't max out ssd/ram at the time, but I'm optimistic the situation will get better in a year or so.
row, row, row your hammer, gently down the bit flip stream...
DDR4 never left. DDR4 RAM pricing has also been spiking, just not quite as hard as DDR5.
If my DDR4 RAM go kaput I'm screwed just as I would be on DDR5.
It is not just DDR4; I know of at least one supplier sampling units with DDR3 for their ONTs.
I just bought a ~10 years old SuperMicro 1U server with 2x 24 cores of Intel's E5. For ... 250 Euros. Really cheap, hehe. Then I had to buy second-hand 128GB DDR4 ECC for 400 Euro and 4x 3,84 TB Datacenter SSDs for 2000 Euros, second-hand of course.
I mean it somehow still feels cheep, if I watch the prices for new hardware.
But it took me days to create this upper mentioned system from all the different sellers and different sources on eBay. There is no "cheap" one-click solution with included guarantee anymore.
Not even DDR4... Things are this bad:
It's quite preposterous to think that an old 256GB RAM DDR3-using dual xeon server (128GB per NUMA node) that I had sitting around on a shelf gathering dust significantly appreciated in value over the past 12 months. But it's factually correct, looking at what comparables sell for for on eBay.
>But at least it’s a good time to hang onto your DDR4 RAM.
A year and a half ago I bought 2 new 8G DDR4 for 47 USD. I was going to use it to upgrade my W541 from 16G to 32G. The W541 has 4 slots. But seems the one I have maxs out at 16.
Now I have another reason t hang on to the RAM :)
Bring back DDR4? Journalists need to check their "I got new shit" privilege. I work in e-waste recycling. I've sold terabytes of RAM over the past year and a half. DDR4 never went away, and for that matter, neither has DDR3. (DDR2 and older isn't as common.)
This article saying DDR4 is making a comeback, and all the comments here saying DDR4 never went away, reminds me of the submarine ("Suits make a corporate comeback").
The crazy thing is knowing so many of the produced GPUs are just sitting in storage collecting dust because they don't have anywhere to plug them in and even where they have space do not have the power to run them. So basically the whole RAM situation is driven by things that are sitting dark in storage.
Not GPUs. AI chips. no benefit to the customer
Love the link preview tagline. Thanks TheVerge
"Somehow, DDR4 has returned."
__
Besides that, wasn't DDR4 good enough for everything we did, anyway?
At least I did not notice all that much of a difference when migrating to my current DDR5 system other than the CPU being massively more performant. Though I am sure that the memory did play some role there.
Clanker claims that the felt absolute latency between decent DDR4 and DDR5 should be similar, while bandwidth might be lower, apparently? Something something CAS latency.
For daily office use there is much more benefit in having more memory than having faster memory. Because apps all started using more memory. Especially anything Google Chrome related.
If you have some unused extra memory it will be used as disk cache, which also helps in the responsiveness of a desktop. Waiting for the disk is orders of magnitude slower than reading from memory cache.
The big companies owe us money here. I call it a heist.
Also, the article makes no sense. Why would I want to pay more for DDR4 now? They should tax those AI companies that are responsible for this.
[dead]
[flagged]
It’s making ddr3 relevant again as well. Bought 256GB DDR3 ECC for ~200 usd a couple weeks ago for my ai server (wip).
These old workstation platforms seem to be perfectly fine for hosting GPUs for local llms.
256GB means you are going to be using system RAM instead of completely offloading to GPU, I assume? If so, you should make sure your CPU has AVX2.
I think it'd be mostly for engrams. It seems LLMs are moving in this direction, with 50GB in qwen 3.8 flash, and 200GB in deepseek 4.1 flash.
My $20 xeons 2696v2 and 2670 are tool old for real offloading, and sata ssd drives are likely a bit slow for ssd streaming :)
"hosting GPUs for local llms" meaning pci-express bandwidth and memory bandwidth are not issues for them, and CPU features probably also don't make a difference.
My Home PC is from 2019, it has DDR4, it gets most things done, and I have never felt it was sluggish or anything. Maybe not the best gaming rig, but I get by.
I built my current home pc on the cusp of the transition of ddr4 to ddr5 and ddr5 was at the time out of my budget so I went with ddr4 and I can still play modern games fine with 32gb even at 2k. I might not max out everything but that was never a priority for me. Usually besides resolution I seldom touch the graphical settings if the game is running ok. In fact the only thing I upgraded one it until now was my GPU since that was a holdover from my previous one.
Home server also runs ddr4 and does ok, but that has been hitting more cpu bottlenecks now rather than memory.
That said its kinda sad personal users are getting tossed back a generation due to current generation ram being almost all gobbled up for AI.
We are getting tossed back a generation, but we are also becoming as rare as chicken teeth. Most people do not own a Desktop PC, most people don't even own a Laptop any longer. To do the most good for the most people requires datacentres at the moment, and it will probably remain like that for the forseeable future.
lol give it a year … now that the datacenter buildout is stalled out because of electricity and ddr6 is being rolled out in China, I bet this looks different in a year.
CXMT is planning expansion for the ram capacity, they’ll do that faster than the west, I beleive they said end of year ish? (Could be wrong).
once it comes online and they increase their yields, this whole thing changes.
Also new datacenters mean increased supply so cheaper tokens now that parameter size is a non-sequitor.
I’m legit curious how long China holds back… but I doubt it’ll be long.
CXMT is not part of the cartel that is probably once again coordinating to keep prices high for as long as possible, so we'll hopefully get some actual competition soon.
I can’t blame them for making back their costs for any manufacturing and maybe a little extra for profit selling at the current rate. Let’s see what they do though :)
Could totally be one of those situations where the current cartel outsource all their production for MoW outside the US to them and then just pocket the investment money lol
However … without the US datacenter buildout ramping continuously … the ram will eventually make its way from warehouses to the market lol. We really need a way to use server dram in consumer boards… but idk if that will happen.
Although it would take a ton of engineering and probably a new chip … a custom motherboard would that did that would be legit haha :) would solve a lot of issues.
I’m trying to find a use for DDR3 RAM which is like $50 or less for 64GB but I found out most things are incompatible
As this is hacker news, I expect a hacker to accept the challenge and design an adapter board.
Well it’s only incompatible with new parts, you just put the best things from 10+ years ago together and it works. It’s quite cheap now, but power consumption would be high. Also basically impossible to upgrade in case you need more compute or RAM.
Ok, so the real title should read:
The RAM shortage is bringing back old compute hardware
A friend and I bought RAM for our aging servers at eBay in 2024. That was 512GB (16x32GB -- we split it equally) of DDR4 ECC 2133MHz, registered. For… $536.86 plus shipping.
Somehow it feels like we got a pretty good deal, in retrospect. (My server has been since replaced, and I can tell you, 256GB of DDR5 ECC was more than that! There's been a further doubling or two of prices since I bought, though. I think I could maybe sell half the RAM and finance the entire server…)
2x16GB costs around $190 right now, or closer to $375 if you want ECC.
Not much cheaper than DDR5.
Currently 128GB of normal non-ECC DDR5 as a 2x64 set is around 2000 euro and increasing...
One would think just some classes and forms of RAM increased in price. I have 128 GB (2 x 64GB DDR5 SO-DIMM) in my home NAS. Home NAS. Those nice ugreen DXP8800 Plus thingamabobs. It'd be over 2k euros for RAM alone now. Ridiculous. Add to that two SN850X 4TB SSDs, and 8 (+1 unpacked spare) IronWolves Pro and it's now a serious worth.
No end in sight.
I know the only manufacturer of such 64GB modules: Crucial (Micron), but Micron doesn't sell new chips to Crucial, thus almost no chance to get them anymore (though few months ago there were some). Kingston (using Micron, Hynix and Samsung) still does not have 64GB.
Soon Zen 6 should become available (well, or not so soon). With 7200 MTs memory controller? I haven't seen such JEDEC modules at all. Are they possible without the additional clock driver (like on 6400 CUDIMMs)?
Interesting, that explains why getting 4x48 = 192gb is around 2400 euros, so 1.2x the price for 1.5x the capacity.
Speed has indeed decreased for the modules you can actually buy, while historically you would get faster and faster modules at lower prices in the years after the introduction is a new DDR standard.
[flagged]
Please consider being more civil when you post, you’ll probably find it more enjoyable in the long run.
The archive link was posted [1] long before you posted. Also from the HN FAQ [2]:
> Are paywalls ok?
It's ok to post stories from sites with paywalls that have workarounds.
In comments, it's ok to ask how to read an article and to help other users do so. But please don't post complaints about paywalls. Those are off topic.
Yes, let's encourage absolute minimum effort when posting!
The archive link looks dead.
Regardless of AI demand: as it currently stands, it seems like that DDR5 prices will drop substantially in ~24 months as additional wafer capacity comes online between SK Hynix's fabs M15X and Yongin Fab 1, Samsung's fabs Pyeongtaek P4 and P5 Fab 1, and Micron's fabs Boise ID1 and Tongluo (+ whatever CXMT do as well), and HBM production hits HBM packaging bottlenecks (HBM wafer capacity exceeds HBM packaging capacity) which means the wafer capacity will start getting dumped into DDR5 instead (unless the HBM packaging bottleneck is solved in the next 24 months).
A bit of a tangent, but when can we expect storage prices to go down?
Assuming demand does not asymptote as we approach singularity.
Which it to say, if models eventually do useful work, demand for substrate will continue to outpace production. I don’t envy the fabs that have to forecast how much memory the world will desire in 2030
This is a supply constraint and requires no assumption about demand - HBM packaging capacity constrains the amount of wafers that can go towards HBM (regardless of price in the near term), and excess wafer capacity will be dumped into DDR5 and crash DDR5 margins.
The biggest advantage is that the older technology can also produced by Chinese manufacturers like CXMT. They're ramping up production to claim a sizeable portion of the market.
They are already able to produce state-of-the-art DRAM, not only old DDR4 or slower DDR5, but they have difficulties to ramp up production for such DRAM, for not being able to buy adequate equipment due to the US "sanctions".
For consumer use DDR4 is fine and price is the real obstacle as is availability. So if they can ramp up DDR4 production that would be far more useful than limited quantities of DDR5.
Update on the 1600 dollar laptop I bought 10 months ago. Now retails for 6,300. Fun times! Thanks!
Can you elaborate? I have a M4 Max with 128 GB RAM that I bought in 2024 with AppleCare+ (€500) for €7k. I estimate I would get €4k to €5k if I sold it second hand today. So it hasn't appreciated in value in any meaningful sense but it has very slowly depreciated in value. Normally a 2 year old laptop would strictly lose at least 50% of it's value.
Lenovo was selling AMD HX Pro with 96 GB DDR5 and 2TB of SSD end of last year 1600. Same model now 6300.
Was expecting the article to go deep into DDR3 land by tone of the title...
Something like 3200mhz DDR4 is still perfectly fine for 90% of consumer use. You really don't need more to look at cat pictures on facebook.
Actually, because it is absolute fine DDR4 prices are also rising. The only thing I found I could find for a decent price is unbuffered server ram or laptop ram.
I think the reason why those processors are produced are because it rather makes sense to upgrade your old rig with them. Seems to be the only way actually to increase CPU sales beyond laptops ans smart phones.
I don't see more DDR4 ram being produced. Chinese brands are already starting to produce DDR5 and soon HBM. 'Bringing back DDR4' is IMHO a too bold claim.
My old laptop has this exact RAM. Funny thing is that in terms of throughput it's on par with my current one, where I put just one 32GB stick, with one slot left for upgrades.
I still use it occasionally and it's really the old CPU and worn out SSD which slow it down.
I actually recently bought DDR3 memory: $150 for 32GB for an older computer. The price is so much cheaper than what I expected. (My expectation of current RAM price is probably pretty out of whack due to market conditions.)
You were ripped hard. $150 is 32GB of 2133/2400 DDR4. DDR3 should be like $50 for 2x16GB set.
Sorry.
That 90% probably also includes computer games. I'm not sure whether those need fast main memory. They certainly need fast VRAM.
Most games do not, unless you want the best graphics. But then you’re out of the 90% use cases
Apparently mainly CPU (rather than GPU) limited games tend to benefit from DDR5: https://www.techspot.com/review/3059-ddr5-vs-ddr4-gaming/
DDR4 have similar latency as DDR5 and that's what matters in all games, while raw bandwidth only matters in some games
And raw bandwidth is a function of how many channels you've got as well. So a threadripper with 8 channels of DDR4 will run just fine even on heavy workloads.
The article doesn't go into detail about it, but if you go to eBay and find a really old Dell rack server that that does use DDR3 ECC Registered (such as an R910 or R920 with 512GB in it), they have significantly gained in value over the past 12-18 months. Or like an R720 with 256 or 512GB in it. Also major price jump in resale value. That's what we have come to now.
Yeah but TBH an old server with DDR3 ECC is still way sufficient to serve something like 99.9% of all the servers out there's jobs.
That's the plain truth: most sites/servers simply serve very very very little and have not anywhere near the need for 256 GB of RAM, let alone 512 GB.
These old servers aren't sufficient for the Amazon/Apple/Microsoft/Meta/Google/Oracle of this world, but for 99.9% of the rest of the world, they're more than plenty.
And the world and technology shall keep on marching: we'll have DDR6 and as soon as that happens, suddenly all the hyperscalers shall consider DDR5 to be oh-so-lame and dumped by the truckloads in landfills.
We may have two more painful years ahead for RAM but we'll then have plenty of used enterprise/datacenters gear on the market.
Depending on your location and power costs though, before the recent crises, it was probably generally worth it get newer hardware for the power efficiency gains. I have an old DDR3 dual-socket XEON E5-2650 system, and while it's more than sufficient for everything I run on it (and then some), its idle power consumption is pretty bad. If prices do ever come down, I'll probably try to replace it purely for that reason, even though I'm not sure I'll ever need more raw horsepower than it's got.
It was obvious AI would further stratify the haves and have-nots, but I was too naive to realize the tech of the have-nots could literally just regress. Has such a thing ever happened before in tech?
I don’t see that at all obvious.
Businesses are competing like crazy to make algorithms that are in high demand. AI could theoretically let people and create new things that they would never be able to otherwise.
It is entitled to thing that people wanting GPUs and new PCs for gaming and hobbies is a more valid use. And if someone values it, all they have to do is pay the market price.
This is the crux of a lot of the anti-AI sentiment especially in the gaming market. They feel entitled to access to the latest chips at a subsidized price below what industry is willing to pay. Many of them then take it a step further and claim some conspiracy (like a few commenters here).
I get why they're upset (it's not fun to be crowded out) but it's a fundamentally selfish position.
Yes, the previous fad (crypto) killed consumer GPU prices for good. I don't believe anything has compared on price-performance to a 1080 in the 10 years since.
Also the enshittification of software might count? You used to own Photoshop for life, now you're expected to pay a subscription for life. And legal onslaught, too. Encryption and consumer VPNs are on the chopping block in much of the world at the rate things are going. The end goal of governments and corporations alike is to remove the ability of consumers to own hardware they control. We are rapidly moving towards a future where you will only be able to access the internet and government services with a "trusted device", which will be a thin client with a TPM, biometric verification, telemetry of everything you do on it, and which will require paying an ongoing subscription to use.
As far as I can tell, in terms of MSRP, 980ti was $649 at launch. 1080 and 1080ti were both $699 when they hit the market.
$699 in 2015 (when the 1080 was new) is ~$975 today, adjusted for inflation.
$975 is pretty close to the launch MSRP of a 5080, which is "just" $999. (We can't actually buy them for $999, of course, but we couldn't usually get 1080 cards at MSRP during the mining frenzy a decade ago either.)
The big difference here is that by the time the 2080 had saturated the market, you could regularly find a 1080 for ~$300 (I paid $275 for an EVGA B-stock model).
Whereas today, with the 5000 series in the market for quite a while already, a 4080 is still well over MSRP
Back in the day 1080 was top of the line. For couple of generations now it's xx90 lineup.
You could say that there was Titan but in practice they were not seriously considered. At least for gaming.
Sorta, kinda. For a long time, a new PC tended to be built with new parts. New motherboard, new storage, new CPU, new graphics card. New.
During every mining crunch that made GPUs very expensive, we sometimes went backwards on parts of new builds. A used GPU from years ago was often a more-viable choice than a new model in any kind of condition.
edit to add: A lot of folks in 3D printing like to have a small computer driving the machine, especially with tinkerer-friendly hobbyist printers of yore. That often meant some manner of Raspberry Pi because it was inexpensive-enough, small, and a known quantity that was simple to deal with in an appliance-like way.
During the covid-era chip shortages, Pi hardware was expensive, hard to find, or often both. So folks started driving their printers with things like old cell phones or cheap Android TV boxes instead.
I remember getting a Radeon card on sale for about $180AUD in 2012ish. And 4 years later looking for an upgrade and discovering the cheapest higher performing card was at least $300 Aud.
I eventually upgraded to an RX 570 in 2018, and hit the same problem 5 years later.
So as far as loe to mid range cards are concerned you can wait 5 years and still have to pay more to replace it for an upgrade.
This mentions DDR4, but I used to be in electronic recycling and I have still so much 1GB and 2GB DDR2 and DDR3 RAM. Happy to ship a couple of sticks if you guys could find a use. It’s going to scrap otherwise.
I have some old machines that take DDR3. What do you have?
I have laptop, desktop, and server RAM. Which did you want? I can send those pictures.
Desktop. I have 3 old machines, just decommissioned at the beginning of the year, that I've been kicking around the idea of making into a cluster.
I recently reactivated a 10 years old Kindle as an AI-Story-Generator (https://github.com/donmb1/ai-story-generator). Old hardware is valueable and with AI you can do great things with it.
unlesss it's for children, just read books.
and if it is for your child, read them books.
Please. There are so many human stories out there—read those.
This is interesting. Maybe add a couple examples of generated stories.
So LLMs are simultaneously making new hardware unaffordable, while also making old hardware usable again.
That too sounds like an actually good development all things considered. For the planet, anyway. And with the big caveat of that cloud tether, though just for scaffolding.
Produce rots in the fields because there was no way to get a profit from it
> That too sounds like an actually good development all things considered
It would be good if we would use less resources.
But we are not.
It's just that some people don't get the newer resources.
Also making old hardware unaffordable too.
Old e-readers were already usable. Generating extruded story-like product on them when they're already perfectly capable of displaying the vast majority of human literature doesn't make them any more usable.
Rather, the LLMs will make the OS on those old e-readers much more maintainable.
Please don't raise your kids on AI slop. They'll grow up without knowing anything else.
It's a shame this story isn't better known. I was lucky enough to read it in grade school, just before the first usable 8-bit computers came out.
Thanks for sharing, maybe it's time to buy some more old Kindles... I use bunch as smart paintings (with changing pictures generated locally with stable diffusion), and of course Home Assistant dashboard. They are awesome little machines.
Look at CrowPanel, you can get nice e-ink devices made for development, at a reasonable price. I have a small 2.13in esp32 board, I spent some time yesterday making it into a pocket flashcard device for my German courses. The energy consumption is minimal, small form factor, easy to flash via esptool or arduino utils
Thanks for the tip, I've seen those small e-ink display on sale in Aliexpress, but wasn't sure about their quality. Maybe I'll get one for my next project, I can see many use cases.
I didn’t notice any quality issue. And the crowpanel docs and demo repos make it simple to quickly get results. Though I had troubles getting macOS to establish a serial connection, no problem under Linux and windows
How are all these related news regarding silicon shortages surprising at all? It was well predicted that while 2023 chip turmoil[1] was happening, big AI aimed to secure almost all of the demand[2] intended for consumer markets regarding RAM chips so as to win the AI race going forward from that date. They cannibalized almost the entirety of the supply chain, so that they could sell you the incredibly subsidized AIaaS[3] with still considerably higher inference and astronomically higher training process without as of late 2026, failing to achieve any meaningful productivity at all[4]. The serfs now will be happy to own their decade old DDR4 architecture.
[1]: https://www.electronicdesign.com/technologies/embedded/artic...
[2]: https://www.reuters.com/technology/nvidia-supplier-sk-hynix-...
[3]: https://arxiv.org/abs/2606.24616
[4]: https://www.apollo.com/insights-news/insights/daily-spark/No...
No productivity from AI is a crazy conclusion to make given the impact we already know it's having on software development. Just because you can't see it in the data yet doesn't mean it's not there, which is what the article you cited says.
Remember too that these models only really became useful less than a year ago with Opus 4.6 being an inflection point. It takes a long time for the economy to reorganize itself so it's not surprising we're not immediately seeing the payoff in economic data.
Of course, when we talk about utilizing AI as an efficient tooling, it is well expected to see the very outcomes you desire. But the point here is the disconnect between the perceived productivity gains and their corresponding measuring metrics. Remember the Goodhart's Law[1]? It becomes fairly unreliable when you try to measure productivity by a set of standards instead of actually producing productivity. When it comes to future about this issue, I'd hope to see the progress actually happening in open source, decentralized AI instead of the current way of using it.
Yeah, I talk to normies and nobody still understands the utility of just having a ChatGPT desktop app being able to basically drive your computer and do whatever needs to be done.
Well is there any productivity gain from chatbots? So far, I mostly see a deluge of misleading content disguised as documentation for things that don't even exist. If anything, the amount of slop out there is slowing me down…
I can't imagine it can even feel productive to anyone, unless you use agents to automate re-rolling your slop machines, but then you're just going to end up with an unmaintainable ball of mud. Either way, it's definitely net negative.
Kind of same outcome as if you just generated lots and lots of random code (perhaps at AST level; but don't tell your Claude, that'd trick it into making itself obsolete tech!) and compiled it in parallel till it passed tests, but this time around, the computer also tells you that you hit the nail on its head, that you're totally right <s>and that you're such a good puppy. agent uses tool `clicker`</s>
I was in this camp until a few months ago, but LLMs have really gotten that good now that you can actually trust an agent with a ticket and it will do it cromulently. The code quality isn't any worse than if a junior did it. Sure, you'll sometimes want to refactor something, but then you can just tell the agent and it will do it for you.
Update your priors.
Yep. To the point it makes you wonder if the "AI bubble" is totally misunderstood. It's less "AI bubble" and much more "Monopolize computing". Buy up computers until most companies have no choice but to use cloud, then buy out chip fabs entirely, then never give anyone a working free-programmable computer again. All software goes through approval from Google/Apple/Microsoft.
You know, what Microsoft has been trying with Palladium (can't risk general purpose computers! People might commit, oh the horror, copyright infringement, with those!)
Imho, as hardware keeps getting better, it seems to me this is the only thing that can make cloud investments worth it in the long term (which is only 2-3 years away) for hyperscalers. It's the only explanation. If OpenAI/SpaceX/Musk/... really thought AI would take over all jobs, they wouldn't be investing in compute, they'd be buying up farms, land, harbors, mines, bankrupt factories, fucking tax prep businesses ...
Make compute only possible within Goog/Amazon/Microsoft/... walled gardens ... that's the play. And this AI is a cute technology that conveniently needs lots of compute, but it's not even the actual core of the play. A very simple very old style monopoly grab, you know, the business plan people got convicted for trying in ancient Persia, 5000 years ago.
> Yep. To the point it makes you wonder if the "AI bubble" is totally misunderstood. It's less "AI bubble" and much more "Monopolize computing". Buy up computers until most companies have no choice but to use cloud.
Unless prices get categorically higher, I don't think that's likely. Like 10x more than it is now.
Companies were buying computers for all their employees in the 90's, where a mediocre laptop could pretty easily cost two grand in 1990s money. My mom just bought a middle-of-the-road laptop for about a thousand dollars in today-money, and that was a consumer buying it; companies can generally spend more. "No choice but to use the cloud" feels like it's way too strong of a conclusion to draw.
I think they'll keep cheap consumer electronics for the most part. Not as cheap as this last decade, but still cheap. They just want to control the entire software chain from boot to network-accessed resources.
Are you human? Then you must verify it from a "trusted computer", preferably with your driver's license. Gets governments and corporations as much information as they want, along with a guarantee that you can't "steal" their content and can't avoid their ads.
Oh I don’t disagree with that, I was responding to the very specific point made by OP about companies being “forced” to use the cloud.
I have no doubt that Google and Microsoft will increasingly do things to force us to provide more data so they can do awful things to us.
A "monopoly" grab from a conspiracy spanning all of the major tech firms? You really think that's the most likely explanation? Not just that the firms are competing in AI and all realize they need massive compute?
Prices are up a bit but there are plenty of "free-programmable computers" from a dozen or more manufacturers on the market. That's not going away.
The word conspiracy is misused there. That is not what a conspiracy is. This would be all those companies acting in their own best interest, separate from each other. The word for that isn't conspiracy but collusion, and it doesn't even require those companies to actually cooperate with one another as implied by the word conspiracy. They would all be acting in their common interest, with no or limited agreements between them.
I don't think you have your definitions straight since collusion does mean conspiring to work together. Your original post pointed towards AI just being cover for a plan to create monopoly power amongst the tech firms. I don't see how you can make that argument with a straight face. Apple, HP, Dell, Asus, etc are just going to stop selling computers?
I take your point that AI is pushing up the price of computers, at least temporarily, but you're leaning way into conspiracy and cynicism in your original post.
> ... AI is pushing up the price of computers ...
"AI" is really 2 companies, by extension funded/owned mostly by FANG, and is not pushing up the price of computers, it's buying ALL DDR5 capacity, processors, memory, and of course GPUs, including the fab capacity to build them in the first place. I don't know if you've recently tried to convince your boss recently to buy a new machine, but ...
> the business plan people got convicted for trying in ancient Persia, 5000 years ago.
Excuse my ignorance, do you have more details on this?
Persia didn't even come into being until 500BC
What?
Really, I'm not sure how much simpler and clearer their comment could be:
The Persian empire did not exist until round about 550 BCE.
Thus, the upthread statement,
> the business plan people got convicted for trying in ancient Persia, 5000 years ago.
Is somewhat nonsensical: 5000 years ago, Persia did not exist. I suppose we could be talking about whatever existed in that locale of the world around 3000 BCE. But that's what they're saying: "what do you mean, 'in ancient Persia, 5000 years ago', Persia did not exist then".
It didn't exist 5,000 years ago
I firmly believe this shortage is artificially produced and I really hope chinese manufacturers flood the market and make memory as cheap as toilet paper
It's not artificial as in "RAM vendors lie about real demand and refuse to invest in more capacity".
There is definitely some exaggeration around timelines (2030 instead of 2027) and the industry has a cartel but most of it is real demand. From AI companies that don't have the money to pay for it and probably won't ever receive actual shipments but they locked down contracts ahead of time.
> I firmly believe this shortage is artificially produced and I really hope chinese manufacturers flood the market and make memory as cheap as toilet paper
I hope so too, but I'm afraid that if China ever blockades Taiwan, then current RAM/GPU prices will look like an incredible bargain.
My guess is it may happen after the election in November. Trump has everything tied up in a pointless war in Iran right now, so it would be a good time for China to make a move, before the 2028 election gives us a more competent government again.
It is not artificial constraint. The build out is titanic; the supply of AI compute is doubling every 6.8 months. To feed that much growth hyperscalers have to price everyone else out of the market.
To put the AI buildout in perspective, it exceeds the GDP percentage that was put into the railroads or interstate highway system.
It's only "artificial" in the sense that getting this much money on the AI companies that don't even make a profit is not normal.
It's actaully "super" in the sense super intelligence is causing the shortage
I don't think that an abundance of any computing resource is a good thing. Cloud, storage, memory: this idea that you can always get lots of it leads to all sorts of undesirable outcomes.
What? All a shortage does is ensure those with deep pockets control the entire ecosystem. Bye bye personal computing. The people with money don’t want you to own anything, they’d rather you rent everything from them. How is that not an undesirable outcome?
Qualcomm CEO was talking demand for chips that can run a 100B param model on a phone.
There is a Q about how much RAM these phones will need and the average selling price.
People are seriously addicted to doom scrolling. But what are they willing to pay for a phone?
How so?
Price crashes, builders go bankrupt, price oscillates heavily, pace of improvement goes way down.
Honestly I think a slowdown in the "pace of improvement" might be positive. These days it feels like you can't catch your breath.
I think in upcoming days everything gonna be making comeback previously like one or two years ago I was thinking the mobile price are so dropped that mobile phones gonna be like as cheap as possible but now after the AI their price are going as high as possible and same with the computers and laptops and all.
We can get refurbished phones with 8 GB LPDDR5 for less than regular DDR5 modules.
LPDDR5 is slower but maybe a small cluster could be useful?
I mean arguably, this is good.
The amount of waste produced due to phones not receiving new batteries and software updates was utterly insane, given what goes into making them.
100% agree. This is generally a good news story. The e-waste issue is one of the darker aspects of tech which is mostly ignored. We should promote brands like Fairphone and Framework that are trying to make a difference here. Reuse more of your kit!
It's good if you're a tech guy in silicon valley with a good salary who used to upgrade every year to the latest pro max.
It's terrible if you're a labourer in India who buys phones under 10k INR (~100 USD) when your salary is 20k INR per month.
Unfortunately there are way more in the latter category than the former, and most of them won't know what ram is. They'll just be paying more and getting less going forward for a few years.
(Source, the salaries and phones I've seen of the staff at office. And we're in a major region you've likely heard of, not some tiny rural village.)
The wages of the West's deliberate deindustrialization and deliberate dependence on Asian manufacturing.
30 refineries were closed in Europe in the past 15 years about, and that is just a small example of a pathological situation.
I was reading that oil from the Middle East had to go to South Korea to be refined before being shipped as diesel to Australia.
Cost of hiring an oil tanker has ballooned massively (like 40 fold) since America decided to bomb its supply chain, so these long supply chains are massively expensive.
The Republican Party has decided that running a world of global free trade was no longer desired. America wont pay the price for another decade or two, but it will.
Europeans can lament the choices the Americans made all day, but Europeans deliberately chose over dependency on the Americans, Chinese and Russians.
Europe can easily be more dominant than the Americans if we chose it.
Well capitalism is responsible for that.
I think in a few hundred years we'll have learnt what a disgusting, poisonous system it is.
Europeans and Americans choose to make things unsustainable and unprofitable to do in Europe, and chose to make it possible to import things cheaply from places like China and India who use slave labour and destroy the environment … and then blame Capitalism.
As you like … but blaming capitalism is not going to bring industry back, and it won't stop China from using slave labour or India from using their rivers as landfills.
I think in a few hundred years the very small number of survivors of humanity will be living in caves and huts in the high Arctic and in a greening slowly-reafforesting Antarctic, they'll be hunter-gatherers with maybe some simple small-scale unmechanised agriculture, and they won't have money. Maybe a barter system for a few.
Among the many other things they won't have is an economic model.
Maybe more efficient code will make a comeback.
How many prompts include "make it as efficient as possible"? Or will efficiency clash with shipping as many features as possible(not even humanly anymore).
I have bridge to sell you if you think that is likely.
If the trend to disregard performance with ever-increasing CPU power had not stopped, C++ and Rust wouldn't be as popular as they are. Something did change with the end of the free lunch.
They are popular among a certain hobbyist community, just as they were in 2024. They are not popular among application developers. Remember how FAANG app developers keep bumping into platform limits, such as the old Android limit of 65k classes in an app, and necessitating workarounds?
What will change is more and more software will be moved over to a cloud based subscription model with the end user device just being a crippled thin client.
And I can already see the shitwank bean counters salivating over this prospect. Imagine the possibilities, we can stuff as much DRM as we want, charge as much as possible monthly, arbitrarily restrict the terms of usage in the future. What will the user do? Run the same heavy duty software on their Intel N150 + 4GB RAM shitbox? Blah.
I mean you could always vibe code a replacement in 30 minutes. Thankfully I don’t see DRM being a problem when you can’t DRM an idea. They will try, but how is that going for the games industry currently?
My AI model queried itself just this morning to verify cluster status, and it says although DDR5 has been available since 2020, that 2+2 is still equal to 4 for those of us still running DDR4. V
In all seriousness, anyone remember when every library, business, government agency, and institution, etc was buying $20,000 cisco routers to "get on the internet" during the dot com boom/bust in the late 90's ?
Only to see them all end up on the surplus market, unused, for a couple hundred ?
We're in the AI boom/bust cycle, and its my understanding DRAM fab is specialized enough, that the supply is getting squeezed.
As for "doing it on purpose" ? It's being milked for all its worth. That's what companies do. They milk the market for what it will pay.
Also, for everyone who is trying to do local AI, on Xeon or EPYC platforms, memory bandwidth scales with module count, making them that much more in demand in terms of processing capacity / wafer area.
Plus it's the same ram cartel players since the early 2000s.
I'm here for the scraps.
> EPYC platforms, memory bandwidth scales with module count, making them that much more in demand in terms of processing capacity / wafer area.
Why is so hard to get self-hosted bare metal which uses all of the modules? Most server providers go for 1 stick and call it a day....
ffs
>My AI model queried itself just this morning to verify cluster status
>Only to see them all end up on the surplus market, unused, for a couple hundred ?
You realize to people that aren't you, this looks like holding two competing ideas at once.
And the comparison makes zero logic. Your library, government, employer are on the internet, who gives a fuck what dump their first modem went into
That's a witty joke, we used to make them all the time ...
1/ Not apparent, there are lots of guys on hn that have real AI setups costing > $50K
Lots of business owners here too that build their own clusters
2/ If it's a boom and bust cycle as he said, where's the bust? It's not coming. Humanity will use exponentially more memory and computing, as we've seen. That's not going away.
Let's have interesting conversations about the next steps for memory, is it quantum? what's up next? is humanity really beholden to a monopoly of private chip makers
> As for "doing it on purpose" ? It's being milked for all its worth. That's what companies do. They milk the market for what it will pay.
It’s what everyone does, from company to individual. When you sell your used car or list your house, you price it according to what the market will pay. When companies sell a product, they list it for what the market will pay.
The conspiracy theories and accusations of collusion were flying everywhere until CXMT came online. It felt like every comment section and headline was predicting that CXMT’s arrival was going to crush the cartel and smash the “price fixing” when a new competitor arrived.
Then CXMT memory arrived at nearly the same price as everything else because, to the surprise of nobody with basic economic understanding, they were selling their products at market rate too.
The high price is being driven by demand combined with a supply chain that takes years to increase output. Just prior to this demand explosion, RAM and SSDs were so cheap that producers couldn’t justify building out more capacity. It would have been a bad choice at the time unless they had a crystal ball that could have predicted the future.
For what it’s worth, I’m willing to buy as much RAM from everyone else at 50% of what these companies are selling it at. If everyone works together, we can cut the market price by selling it to me. I will provide the much needed capital in dollars to shift the price and others can provide the RAM to me.
I’m willing to spend a lot to drive down the price. As a show of good faith, I promise not to actually pay any more than 50% of what you can get on the used market.
It's not really clear to me how your comment functions as a response to the parent.
Obviously, spending additional dollars on purchasing RAM will drive the price of RAM up.
Just as obviously, spending additional dollars on manufacturing RAM will drive the price of RAM down.
The reason why the memory cartel fucked everyone over isn't exactly the pricing. It's the supply. It's that the established manufacturers all agreed on "AI demand is a fluke, we wouldn't rush to online new manufacturing capacity" back in 2022.
A calculated move. If they're right and the AI demand is a fluke, they don't overinvest in expensive manufacturing infra, they don't get saddled with more capacity than they need, and there is no memory glut eating their margins in the future. If they're wrong and the AI demand isn't a fluke, they get to pad their margins big time, and the supply bottleneck isn't their problem. As long as they shake hands and bet on "fluke" in sync, the downside is very limited for all of them.
They shook hands, they bet in sync, and they were wrong. Now the supply bottleneck is everyone's problem.
Potentially even theirs - if Chinese vendors can use it to wedge themselves into the market when it's at its most lucrative, and invest in growth. Which they almost certainly will.
> The reason why the memory cartel fucked everyone over isn't exactly the pricing
Let's not rewrite history: Sam Altman kicked this off by trying to corner the memory market to get an edge over his competition...in return for only a year's supply of memory. The resulting supply squeeze has been very profitable to the RAM manufactures, and will last many multiples of OpenAI's initial contracts, so they can't be too mad at him for bot disclosing his other wafer deals during negotiations. They can afford to gift him a thank-you yacht named the "Adverse Supply Shock"
The important question isn’t so much “are people trying to get the best deal possible for themselves?” but rather “how well is this market functioning?” Or phrased another way, “how competitive is this market?”
This particular market is pretty clearly not functioning well, and there are textbook Econ 101 reasons why a market may not function well. Of those textbook reasons, I suspect the most obviously applicable one here is large barriers to entry.
> and there are textbook Econ 101 reasons why a market may not function well. Of those textbook reasons, I suspect the most obviously applicable one here is large barriers to entry.
Econ 101 would be understanding that you can’t snap your fingers and have a new DRAM factory online overnight, and that this isn’t an indicator that the market is not functioning.
You’re defining a market “functioning well” as meaning the price is low for you and all demand is satisfied immediately without raising prices.
That’s not really what is meant by functioning markets. A functioning market does allow the price to increase as this is what spurs investment into new capacity and new developments.
It also allows the DRAM to go to the most profitable activities. This is actually what the whole Georgism and Land Value Tax people advocate for with property taxes, which bizarrely many people think they like: The idea is to raise the price of something (in that case, your property tax) to ensure it’s not being “wasted” on an economic activity that isn’t optimal. The most optimal economic activity is the one that can afford to pay the most, which is the one that captures the resource.
It’s not fun when your use case isn’t the one that can pay the most, but that’s literally how markets work.
> Econ 101 would be understanding that you can’t snap your fingers and have a new DRAM factory online overnight, and that this isn’t an indicator that the market is not functioning.
You can quibble over my usage of “functioning,” but all I’m referring to are the textbook characteristics of perfect competition. I’m not claiming that someone is to blame if any of these conditions don’t hold. Of course they never hold perfectly in practice!
I’m not trying to blame anyone for the fact that it takes a lot of money and time to get a new DRAM factory online. I’m just saying that this fact is bad for essentially everyone except for the small number of current DRAM producers.
> You’re defining a market “functioning well” as meaning the price is low for you and all demand is satisfied immediately without raising prices.
Well, yes, that’s pretty much the only reason to advocate for competitive markets, isn’t it? If you don’t want allocative efficiency and productive efficiency, then what do you want?
> Econ 101 would be understanding that you can’t snap your fingers and have a new DRAM factory online overnight, and that this isn’t an indicator that the market is not functioning.
Wasn't there this whole legal thing a couple of years ago involving most of the dram manufacturers?
From what I've read, albeit from Reddit, they're not planning to sell or shelve those racks, they're planning to destroy them due to the fact it will flood the market. They have no intention of letting others have them.
Who is "they"? Certainly we have some concentration of consumers of this hardware. Are you saying they'd rather destroy the hardware to protect their supplier's market rather than allow the secondary market to function as normal? Maybe this is written into contracts? We're in a very sad state of affairs if that's the case.
There is such demand for the GPUs, for example, it seems that they can pick the winners and demand they abide with terms.
Aren't servers depreciating assets owned by the company and tracked on the balance sheet?
If so, on a liquidation event, I'd imagine they'd have to be sold to compensate the shareholders.
There are a lot of companies that will buy your servers and lease them back to you.
Idk yes its meant to depreciate and perhaps is from an accounting perspective but all my computer hardware has been rapidly appreciating… My 8x RTX Pro 6000 cluster has easily doubled in value in the last 12 months.
Lots of precedent for that: In the EU WEEE has incentivized companies to crush working electronics rather than selling them, because it is way less bureaucracy to hand something to a certified recycler than it is to legally sell the stuff. And once something has been entered into the WEEE system as waste it's pretty much impossible to legally recover it: the recyclers are literally not allowed to simply pick working equipment out of the pile and sell it. Technically WEEE is supposed to enable reuse, but it's way easier and less bureaucracy to just crush stuff. The necessary per-item paperwork, testing and certification (it has to actually work, broken/non-working equipment cannot be removed from the WEEE system, because it is waste, legally) and logistics required for reuse exceed the residual value anyway.
That's why used enterprise/industrial electronics are way less common on EU eBay compared to other places, despite the industrial base being so large (or used to being so large). You can actually see when WEEE came into force because very little used T&M equipment entered circulation in the EU after the early 2000s.
Another factor is lease agreements, where companies don't buy equipment and the lessor "recycles" (i.e. crushes) the equipment to avoid creating a used/second hand market as an additional benefit.
The liquidators will squeeze every penny out. All that ram will hit someone who can take the chips out and repackage for consumers.
Huh. This is a crime against humanity. The planet is burning.
Eh. Destroying the racks is a waste of resources. Most of the metal components will be recycled. The boards scrapped for gold. The net positive is that many the data centers would shut down leading to a better balance overall.
An interesting downstream effect of hypothetical highly-effective robotic AI labour would bring is the ability to strip, sort and test e-waste down to the component level.
A company can either generate a small new revenue stream from on-selling old gear, or they can pay to destroy and dispose of it.
Which option is a profit-oriented, not-totally-profitable AI outfit going to choose?
People seem confused here, but liquidation is very common. That's what private equity often does. It's also why debt collectors call non-stop, even if they are calling the wrong person (fuck you AI debt collectors!). They buy up the debt for pennies on the door and the original debt owner writes off a loss on their taxes. It's also the reason how I bought a Herman Miller for 40% retail.
It's very simple: some is better than none
I am not sure that it would be profitable to sell equipment that has been depreciated to 0$.
Clothing companies destroy unsold stock all the time.
Clothes are typically too cheap to liquidate. And how does Nike liquidate a Nike branded shirt. The buyer would need to remove the logo.
Clothing companies do typically put items on sale first. They'll even sell "at a loss"
Well, if they were bought on credit, and must be returned when you go insolvent to the company that makes the chips, whose market - and therefore stock - would collapse in value should the market be flooded with used equipment, and said company already dealt with a similar situation in the past few years... I think the answer is obvious.
Liquidators aren't going to care about "the market", they will try and get in first and profit on on-selling equipment at a lucrative price.
I don’t think you got the point. The liquidator in this case is highly incentivized to keep these off the market
there's a big difference between now and then. we built up capacity. dark fiber. and couldn't use until we got broadband. In general. ai capacity is being used right now today.
This does not seem to be the case. The very first VeraRubin NVL72 systems are barely installed and won't be running real workloads until beginning of '27. Colossus II appears to be the largest user of Blackwell Ultra NVL72s and those are also only recently on-line in September of this year with the remaining 3/4 of them not until end of the year. The delays in datacenter deployment are very long. Even their GB200s didn't come on-line until January of '26. Hopper H100/200s still dominate.
The other hyperscalers are even further delayed. CoreWeave has some VeraRubin, but not in volume (as I understand).
That’s one big difference. The other big difference is that capital back then was mainly spent by startups with no or little traction.
Today, the "old" corporations like google, meta, amazon are also spending a lot of money.
We’re doing a similar thing now too, building tons of extra power capacity and links to data centers.
After the crash no one will need to build a new data center for 20 years, and there will be plentiful power connections
> ai capacity is being used right now today.
A lot of its current use is just basic questions that could be handled by local llms just as well, yet they are being processed on GB200s on the other side of the world.
That’s not exactly true, a lot of GPUs are sitting in warehouses because they are lacking prerequisites to being installed, in some cases this is even just power.
https://www.tomshardware.com/tech-industry/data-centers-in-n...
… and cooling
I don’t think the article you cite supports your point with confidence you convey. Yes other dimensions can complicate deployment, but incentives to work around those issues are high, given the cost of GPUs
That’s not the same. Those GPUs are allocated to real projects with real demand. If they could turn them on today they would do it and they would have customers.
It’s not the same as dark fiber capacity being overbuilt for demand that wasn’t there yet.
There appear to be a lot of uninstalled GB200s (looking at you Azure), but the power efficiency of those is ~10x worse than a Vera Rubin. Does it really make sense to install them, when your data center is power/cooling limited, and it's not built yet?
Maybe somebody will use them, but not the original owners.
> If they could turn them on today they would do it and they would have customers.
Is this really true? Other than the occasional outages, the frontier labs don't seem to be getting overloaded by API pricing customers at peak times, and with Opus 5.5 using even less tokens (which translates into less GPU time and less money spent on the same tasks), it seems actual capacity isn't literally strained right now. If current needs are met by currently available capacity, where is this demand going to come from?
Yes. I have 3 GPUs in the other room, sitting in boxes, waiting to be powered on, because I dont have the electrical circuit installed in the room yet.
I can directly confirm this fact, and I'm just some random dude out here in suburbia, who can only imagine what the Indiana Jones warehouse of glowing GPUs looks like in a data center.
We're all salivating over this imagined future where an AI toilet seat greets us every morning.
Who here has already asked AI how to make your own DRAM ? I know you have. Thats a sign of peak interest in a subject or market.
> We're all salivating over this imagined future where an AI toilet seat greets us every morning.
I'm not, but I can certainly believe the surveillance industry vectoralists are salivating over a future where their AI toilet seats greet each of their data subjects every morning.
If only AI had been watching Epsteins toilet, then we would all finally know what happened to that turd. :D
> In general. ai capacity is being used right now today.
Not according to this article[1], which says:
"I believe we are now in an inevitable overbuild situation, one with no neat, tidy Dot-Com Bubble-style exit story. Demand for NVIDIA GPUs — and those from Broadcom, AMD and other semiconductor companies — is driven by speculative capital believing that the AI industry will become magnitudes larger than it is today, largely driven by the fact that everybody believes there’s far more demand for compute capacity than actually exists. ... NVIDIA has created a remarkable illusion perpetuated by the media — that GPU sales are a direct measurement of the actual demand for AI compute, rather than a measurement of how a few companies are willing to invest in an idea two years in advance, using circular financing as a means of creating the sense that you must buy these GPUs now, or you’ll miss out on the future. ... At the very least, hyperscalers are going to be burdened with brutal depreciation charges or onerous write-offs for years to come, whether their capacity turns into revenue or not. ... I don’t see how 90%+ of NVIDIA’s sales ever end up generating a single dollar of revenue, and considering the amount of project financing-backed data center debt deals, there’s very little that exists to protect investors if AI compute demand never arrives."
That's Ed Zitron, and his calls on AI matters routinely get outperformed by that of a broken clock.
The man saw the market demand for saying "AI bubble is going to pop and AI tech is going to wither away and die", and went to meet it on the supply side - truth be damned.
So far, AI companies still keep getting bottlenecked on compute, AI utilization increases - driven by, among other things, increased price-performance of AI making it viable in more and more roles. And the demand for both AI inference and AI hardware in general shows no signs of stopping.
In 5 years from now every device. Even your oven or washing machine has somekind of local LLM running for sensor reading and smart decision models. I can already think of quite some useful use cases for those devices. So I do believe that AI becomes magnitudes bigger than today. It's a logical step in our digital journey.
Why do you need an LLM to read a sensor? This is truly AI pilled delusion.
Why would you need the LLM to be running on the device itself? Wouldn't it be more efficient for the intelligence to be more centralized? Maybe in one central computer per home, or (probably even more desirable for corporations) for the devices to phone home to a centralized server at the service provider in the cloud somewhere.
Oh of course thats certainly also gonna happen. But i believe there its still useful to have local dedicated AI with low latency. For example regulators that need constant adjustments. Also in cameras local AI could be extremely useful. Same for audio processing and other sensor processing. Cloud solutions wont work in those cases.
> As for "doing it on purpose" ? It's being milked for all its worth. That's what companies do. They milk the market for what it will pay.
In supply constrained markets the bidders set the price floor, not the manufacturer. A manufacturer produces a supply; they tender the supply for bids. The highest bidder gets the opportunity to purchase the supply. Everyone else has to wait for the next bid, the waiting is their opportunity cost, and is priced into their bid.
That's what we mean when we say "milking". Just because you can explain something in economic terms does not make it justifiable.
To add onto this: it is generally not possible for suppliers to sell below market rate. If DDR5 is going for $400/16GB, and you want to sell 16GB for $200, people are going to buy your cheap RAM until you run out and resell it for more money. This is called arbitrage, and it's the market rewarding others for being greedy where you weren't. The only way to make the price go down is to make the market's price go down: produce so much supply that you extinguish all of the arbitrage-driven demand.
There are some wrinkles to this. If CXMT's RAM was somehow inferior, then the market could sustain two different prices for two different products. But it's difficult to separate the market like this without just making unusable trash. The one separation we do have - DDR5 vs HBM - works against the consumer's favor. One layer down in the supply chain, RAM manufacturers have to buy silicon wafers to etch circuits into, and HBM takes up treble more wafer space than DDR5 per gigabyte. In fact, this is specifically the scarcity that allowed Sam Altman to engineer a RAM crisis, by buying up a bunch of wafer supply that he could then redirect to more AI-optimized HBM.
We can also infer from all of this that CXMT - while producing lots of memory - does not have the scale necessary to actually quench the inflated demand. Either because their yields are shit, or because China wants to run a business and made the same calculation as the other RAM producers that the AI bubble would pop before they could get additional equipment online to service demand.
Counterpoint: Arbitrageurs cannot outsupply the OEM, and with enough supply, whatever delta the arbitrage itself has will be recursively arbitraged in turn until a stable low price is reached, cf. Chinese solar industry dominance, and keep an eye out for Chinese EVs.
You have described the market milking process very accurately.
Theres machine learning algorithms in the mix, and it all works perfectly smoothly, until it doesn't.
Smoothly here, being defined as the markets emotional feelings about it all, not, the price went up.
The price went up because it could, as you pointed out.
The price went up because demand went up. If you want incentives to build more infrastructure to produce more supply so the price can come down, it has to first go up.
Everything, everyone, has ever said, to another human, was a market signal.
I agree completely.
Bad faith response.
When I said "milking" it was an attempt to summarize market mechanics.
It failed, and I feel we are all the better for it. :D
Routers don't do productive work.
This boom is only superficially like the dot com bubble.
There are absolutely parallels.
But, a GPU that can help me write emails, code, articles, etc. has intrinsic value that a Cisco router simply didn't provide.
I don't believe GPUs are a new asset as Jensen Huang claims. (His take seems very bubble-ish.) But, I feel these elevated prices are going to last until production catches up.
>Routers don't do productive work.
I'm sorry, what?! Communications is a valuable thing. How is all that AI slop going to hit the Internet if there's no routers?
Without routers, your email isn't going anywhere, it doesn't matter who wrote it.
If you're growing onions in the backyard, you don't need a router.
If you're writing emails, you're gonna need to send them through a router or two before someone reads them.
> I don't believe GPUs are a new asset as Jensen Huang claims.
He sells GPU's, so taken in context, he's promoting the financing of GPU's, by anyone who is listening to his sales pitch. He travels in CEO circles, so, he's talking to other CEO's, and the banking industry.
The phrasing of it also directly pushes back against the articles accusing the AI industry of circularly financing itself to inflate everyones balance sheet.
And the other thing this idea of it being an asset that you invest in, like a house presumably, is that it primes all of us for 2 future realities: price increases and minimal performance gains ( "its useful for longer so finance it" ).
As to economic value proposition to any individual or group, it boils down to how they are going to use it to recoup the expense.
Don't just take the sales guys word for it, ever.
There have been many videos lately, on whats happening in the automotive market. The industry has been adding more and more tech, to the point where its now financially extractive via mandatory dealer interaction aka fees, and people are becomming fed up with it all, wanting something simpler for less money.
Translate this to the big fat GPU card industry, and how long is it before AMD/Intel add just enough AI ops into every CPU that noone cares anymore and they start asking themselves "Why am I buying 96GB of RAM and a GPU, and 96GB RAM and a CPU ? I'm buying ram twice ! It's a conspiracy !"
Unified architectures seem to be inevitable, GPU's would no longer need seperate ram, then shrinkflation is gonna hit your video card, and you're gonna pay GPU+RAM prices for just a GPU. The upside to this is that you could fit more GPU's in your box and they won't need 2.5 slots each. For now, they are hoping the DGX Spark type boxes fill the market demand, and allow them to sell higher priced products. Unified architectures would dent demand for those.
Just my 2 cents on it all. Enjoy the ride.
> If you're growing onions in the backyard, you don't need a router. If you're writing emails, you're gonna need to send them through a router or two before someone reads
That’s what the cable company wants you to believe. We sent their damned boxes back. They wanted us renting a modem that never hangs up, then buying a router just so our own computers could talk to each other. Mine used to call yours directly over the telephone!
Now there’s a computer inside the modem, another pretending to be a router, and a mess of switches, gateways, and subnet masks to do what two modems and a telephone number used to accomplish.
They took the party line, put it on a television channel, called it broadband, and convinced everybody to rent the equipment.
In my day, when somebody tried to break into your computer, you could hear the bastard dialing. And when you were done with the Internet, you hung up. Now they want you connected twenty-four hours a day.
Just wait until somebody breaks into your television through that cable. You’ll wish you’d kept your rabbit ears.
I for one find it simpler and better to have a single router for multiple computers to talk among themselves, rather than daisy chaining serial ports (or how did you accomplish that back in the day ie before Ethernet?)
[flagged]
> I for one find it simpler and better to have a single router for multiple computers to talk among themselves
That's not what a router does. A router provides communication between two (or, theoretically, more) different networks.
Multiple computers talking among themselves would generally all be on the same network. (Or, if they're not, you'd put a whole mesh of redundant routers between them.)
Adjadent to the article, there are some interesting builds with pulled parts from decommissioned servers in places where hardware is expensive or heavily taxed.
There's an entire parallel ecosystem around Chinese X79 and X99 boards [1](brands like Huananzhi and Machinist). They exist almost entirely to soak up scrap-tier Xeons and decommissioned server DDR3 and DDR4 ECC RAM that get pulled from datacenters for next to nothing. You see them online in Russian/Chinese/SE Asian online communities as budget builds, so nearly all the BIOS mods and documentation are in Russian or Chinese.
I think in NA we'd tend to write this hardware off as e-waste unless you're a homelabber (though I know Kimsufi still includes DDR3 builds in their catalogue). But in emerging markets, people have been squeezing real utility out of discarded enterprise silicon for a long time out of pure economic necessity. I've been tempted to do my own build but I already seek to have collected an overwhelmingly unnecessary amount of 2012-era workstations that I'm not sure my partner would appreciate an addition to.
About six years ago (as a holdover until I went full homelab, which I never did) I built a machine with some spare parts, a $40 Xeon E5-2650v2 (8 core Ivy Bridge), and one of those Machinist motherboards.
Initially it was just for home server stuff (ZFS array for storage/backups, Plex, Vaultwarden, etc) but then I wanted to experiment with a remote dev environment using LXC/LXD.
I now do all my web development (mostly back-end, Python and C#) on that machine via tmux and VS Code remote SSH. I'm sure if I was running my browser on it it might painful given the slow single-core performance, but honestly I've felt zero need to upgrade it. Once the RAM-pocalypse is over I'll probably look at doing so just for reliability's sake.
I have one of these Machinist boards sitting in my closet, which I haven’t set up mostly out of laziness. What I really want is a PCI-E card that can take DDR3 RAM to be used as a RAM disk for containers that do a heavy amount of heavy I/O.
One or two DDR3 ramdisk cards were produced but they’re basically non-existent today. There are DDR4/5 ones (CXL), but they they’re restricted to specific enterprise motherboards.
There are significant performance/energy ratio problems with older hardware, particularly with the rise of ARM.
It's fun to think about running datacenter hardware at home, until you get the electricity bill.
Until about 6 years ago I also mostly used ex-server "ewaste" hardware, because it's always been cheaper, except for electricity use which I wasn't paying for myself. My desktop in ~2015 was built from 2006-era hardware, with 32GB of ECC DDR2, which felt like a lot at the time.
My "6 years ago" build was with more typical current-gen consumer-grade hardware, because at that point I was on the hook for the electricity bill. I haven't thought about what my next build will look like because it's still going strong (aside from a flakey DIMM... so my next system will have ECC).
My homeserver is essentially a top of line gaming PC from 2012 with a Quadro RTX 4000 from 2018. It was not top of the line in 2013 when I bought everything on sale but after many inexpensive ebay upgrades, it's now what I wish I could have afforded back then. Most of the time, it's perfectly fine since it's main job is jellyfin. I'm sure I could be saving some electricity with newer hardware but with the current market, it's just not worth it. I'm missing AVX2 support which Claude Code has recently made a requirement but now I just have Claude ssh into it from another PC that is a few years newer.
I've been extremely pleased with my R320 in a NAS role. Idles at 40 watts, is quiet, has ECC, and the drives stay cool.
Maybe that's still too much power for some people, but I'm happy with the tradeoffs.
The E5-2670v1 was goated for this back in 2016-2017. You could get them for like 25-50 bucks. I still have one of the supermicro boards I got from work in hardware decommissioning and one of the chinese franken-boards with a modded bios.
I just bought a Xeon E5-2690 v4 for $16 and a Sparkle A770 for like $350. Going to try some local models on there. I think it will be plenty good for my use cases.
Yeah, I have an old X99 old asrock Fatality (lol) board, micro ATX with a Xeon E5 2640 and 128GB of ECC DDR4 running 24/7 as a Truenas server with 80TB of storage in it. It seemed like an overkill when I put it together two years ago when I replaced my 5820 CPU that had been my daily desktop but now with storage and everything prices exploding it has turned into a very useful thing. I still don't like the fact that it sits at around 130 watts. But given that it has so many hard drives in it, it's not that bad. My whole basement cloud now feeds into it.
Typing this from an Asus X99 WS 3.1 with a E5-2696v4 . I got 256GB of RAM, sees only 128GB but I am sure with some overvoltage and lowering the ram speed it could see all 256GB.
if anybody got a tip, i'd be more than glad to listen, please.
> Typing this from an Asus X99 WS 3.1 with a E5-2696v4.
Yeah the Xeon E5 v4 to me are really an amazing spot. These things, today, are basically free. If you look a bit you can buy stuff like four 20 cores E5 v4 for $60, making them effectively $15 each. Basically free.
And here's the trick: I don't need my Xeon to be up 24/7. 8/7 is sufficient, so I save 2/3rd of the electricity cost. I set up my network so that when I sleep, my main server doesn't need to be on. The services that really need be on 24/7 (say my DNS for when I'm sleeping and my wife/kid need Internet) are running on a Raspberry Pi (which consumes nothing).
And when I need the shitloads of cores / ZFS etc.: I turn the Xeon on.
Now it's still a server in my case: my main machine is a recent AMD.
> if anybody got a tip, i'd be more than glad to listen, please.
Slap 128 GB of the 256 in a $50 used Xeon server and use that has a backup / to run VMs / ZFS / etc.
Thanks for this! Your post inspired me to look at my X99-UD4 system that was put in a closet some years ago. The board currently has a 5960x CPU, and I did not know I could put an E5-2697v4 CPUs in it. I have a couple of these v4 CPUs and tons of DDR4 modules from a recent data center decomm event. Time to hack again - it may revive my old PC!
I have 2 Huananzhi EPYC boards.
At about $350 each, I had to try them. Everything else was double.
Only thing I can say so far is, they boot linux on second hand EPYC 7002 series CPU's.
Haven't put them through their paces yet. V
Oh, I remember quirk #1 ... apparently, only 1 of the case fan headers is speed controllable, the others run at full speed all the time, with no bios setting.
You can fix it with a fan hub that supports pwm so not a show stopper by any means, for those wanting to try these motherboards.
What's up with the "V"? I've seen a couple of your comments that have a lone V at the end of the comment or paragraph.
peace V
I guess its a substitute for "just so you know" or "fyi" or any other "I'm an authority on something" ending. :D
I end up with B sometimes because I use iPhone double-space to get a period and accidentally triple space but I miss the third and hit B. Maybe V is the same.
Awesome! Technology to the rescue. Glad it's not just me.
I imagine those countries have sane energy prices compared to Western Europe.
Regardless if the market allows consumers access to negative solar prices or not, consumers can attach solar panels to their servers (independent of the grid), so the LLM is there when the sun is shining.
> when the sun is shining.
So no LLMs for 3-5 months a year in most of Europe?
personally I think we should look at 12 km tall floating balloons piping a suitable vapor at ground temperature up, dumps the heat a bit above the tropopauze, condenses the vapor so it rains back down (perhaps in a loop, perhaps as a naive thermosiphon).
I believe we can generate baseload energy by cooling the planet... nearly everywhere, no fuels needed.
Energy is a commodity so the price will be very similar for the same goods. What they tack on as taxes might be different.
Electricity cannot be shipped from one place to another in bulk quantities.
And data can be.
Which is why I suspect most nominal "data centers" are fronts for crypto mining, an activity that seeks out the cheapest energy.
You're thinking the whole AI boom is just a front for crypto mining?
Energy means electricity in this case, and the EU prices are about 2-3x those paid in China AFAIK, for both residential and commercial.
EU is big and prices vary. (Transfer capacity is limited)
Afaik electricity is around 7c/kWh in China. I got 8c/kWh 2y contract a couple weeks ago here in Finland. Though have to add 4c/kWh for transfer to that (no clue if the Chinese prices I could find include that). Commercial prices are less then that (no VAT and possibly lower power tax bracket. The 8c includes all taxes/fees)
The cheapest price in Europe at best equaling the average price in China sounds like it just proves my point?
No. You're comparing apples and oranges.
Me too.
EU average is just under 29c/kWh (non-contract price) [0]
In the UK I got 27.5c/kWh (+ 58c/day fixed charge) on an 18-month contract a month ago
[0] https://ec.europa.eu/eurostat/statistics-explained/index.php...
My thought was that the EU average can't be that high, but apparently Finland is just the outlier here.
I think in Quebec they've been paying me to use power.
Good for you
That is expensive. If it makes you feel better, U.S. utilities' published kWh rates don't include a whole bunch of other things that make bills much higher than you'd imagine. For example, I used 280 kWh last month for my apartment, at a nominal rate of either 11 or 13 c/kWh (the rate changed from 13 to 11 partway through the month as it got cooler).
But after all the "sales true ups" and "affordability charges" and "resource adjustments" and "fuel cost charges" and "interim rate adjustments", not to mention the city fees and sales tax, the final bill divided by my usage gave me an effective rate of 23 c/kWh.
That sounds horrible, even in Europe deceptive and abusive tactics towards taking people's money are a constant hazard. I can't imagine trying to exist without even what little protection we have.
There is a state utility regulator that is supposed to ensure this stuff is fair, and it is disclosed ahead of time. Regardless, for small energy users like me, the fixed costs and extra fees are a huge percentage of the total. Thankfully the total electricity bill is not terribly consequential for my household budget.
Utilities are actually highly regulated in the US. Those rates and fees are publicly available and explained when you get your service, for what it’s worth.
Utilities have to argue for rate increases and get them approved by regulators, so they’re not just dropping fees on top of everything as they please from month to month.
Really nobody should assume that their service is flat rate with a slope down to $0/month if you consume 0kWh. There are fixed and marginal costs in every service like this.
> Utilities have to argue for rate increases and get them approved by regulators, so they’re not just dropping fees on top of everything as they please from month to month.
Do regulators ever reject rate increases that the utilities ask for?