Hacker News

I have a hard time believing ANYONE thinks this is a selling point. Literally anyone, including marketing and execs at Microsoft. I think they have just sunk too much money in it to quit, so they keep doubling down.

"The problem I see with the AM5 socket is simply the fact that DD5 RAM to support it is just too expensive."

DDR4 is basically just as expensive. At least DDR5 gives on-chip error correction (not as good as full ECC).

For a geneal computer, there's not that much difference between AM4 and AM5 unless you really want the extra speed of DDR5, PCI 5, and the newest processors. You can build a very capable AM4 machine for slightly less money, but that savings is found on the CPU and motherboard, not on the RAM.

If I can use it with Linux in any meaningful way, that would be a better selling point.

Now all the features you don't use can perform 20% faster!

Upgrading to AM5 wasn't compelling to me even last summer before things went bonkers; I'm still very content with my 5800x and 64GB of DDR4.

Trying to take the plunge on that now sounds like a nightmare.

To really take advantage of those gpu cores you need memory bandwidth. Modern transformer based LLMs are really bandwidth hungry. I am really happy to see this first push. NVIDIA having discrete GPU/memory/etc is an option, but not great for a lot of different reasons. Unified memory architectures like what AMD and Apple have are the way to go for the future. Put 256GB of ram on the main board and be able to access it at speed for LLM use please.

Just like the previous generation of AI PC, consumers just need a usb/pcie NPU,

Mass adoption won't happen until we get those cheap, because there are no mass prosumers making software for them that is massively popular.

Minisforum MS-S1 is the same chip but has a PCIe slot suitable for a network card.

The RAM shortage is... a shortage. It's temporary by nature. RAM didn't suddenly get 4x more expensive to produce, it's just in high demand right now. Supply will eventually catch up even if it takes a few years.

They can't fix stupid.

Let me describe this in the most simple terms possible: You have speculators speculating about AI products. The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution.

So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM. Which of these RAMs are the ones that they need to worry about? Exclusively HBM, which is a minority in production, DDR and GDDR dominate production.

If you're into inference, and have older machines, you're buying Hxxx or Bxxx cards that use HBM, fit into dual slot x16 configurations, and you're jamming (optimally) 8 of them in. If you're into hardware that is newer, somewhere in the middle of the inference boom, you're using MXM cards. In either situation, the host machine has DDR, but if you're OpenAI, Anthropic, Microsoft, or Google, you're not building (more) inference machines like this.

The first two are buying Nvidia's all in one SBC solution: unified HBM, onboard ARM CPU to babysit the dual GPUs, has its own dual QSFP network controller that can RDMA, etc. No DDR or GDDR involved. Any machines built before this platform are being phased out entirely.

Microsoft is doing the same, but with AMD's products, the MI series that co-locates Epyc-grade Zen 4/5 CCDs with CDNA compute chiplets, running the entire thing off HBM, thus also unified and no DDR/GDDR needed. They, too, are phasing out machines older than this.

Google has a mix: they offer Nvidia all in one SBCs as part of GCP for legacy inference tasks (so your stack that can't run on AMD yet still can run), but also offer the same MI products that Microsoft offers via Azure's inference product, but also has their own TPUs that some of Gemini runs on; the TPUs run on HBM afiact. No DDR or GDDR here.

So, what does AMD or Intel do here? Lets say they waste fab time to make their own dies on the wrong process (TSMC and Intel-Foundry do not have for-RAM optimized processes)... they would be producing DDR and GDDR for a market that almost has its entire demand met. Intel lacks the die stacking technology required to build HBM, and TSMC I think can't do it for that many layers (HBM has 8 to 16 layers in current gen stuff iirc).

Micron, for example, already is bringing two large factories online here in the US to meet the projected growth in demand for the next 20+ years. When these factories finally start producing, it will not change the minds of speculators: they still seem to think AI datacenters need RAM, of any kind, and refuse to understand even the most basics of nuance. Also, when they come online, HBM will be a minority product; the AI inference boom is still just a bump in the road for them.

Nvidia kinda screwed their consumer partners, btw: they no longer bundle the GDDR required for the card with the purchase of the die. There is a slight short term bump in GDDR spot prices as partners are building up warchests to push series 60 GPUs into production, and once that is done, spot prices return to normal (outside of the wild speculation manipulation).

One last thing: what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it. I consider it as a sub-category of DDR (and some dies seem to work as either DDR or LPDDR as of DDR5, due to the merger of the specs by JEDEC), but since it isn't something you find in datacenters, it seems to have avoided speculation.

The Ryzen Max CPUs mentioned in the linked article? Uses LPDDR. Doubling down on the Ryzen Max product line might be a brilliant move.

I find the panic over RAM prices to be overestimated. 32GB DDR5 RAM is around $500 which is comparable to to the 9800x3D. Sure it sucks that it increases by around 4x, but when you factor in the overall price of a top end PC at around 1000-2000, especially for the lion's sum of the GPU, the increase is marginal.

This only effects a very narrow slice of highly budget conscious consumers trying to build high end PCs at razor thin margins.

That's a ridiculously implausible and sensationalized rumor. At most, Microsoft may make a NPU a requirement for OEMs to use the Windows 12 logo on new PCs. Actually refusing to support the existing install base of recent and hughly-capable desktops is not at all likely. It would be far more drastic than the hardware deprecations brought by Windows 11, which were already quite controversial (and loosely enforced).

Does a cache help with inference workloads anyway?

I don't know much about it but my mental model is that for transformers you need random access to billions of parameters.

Are we going to see more memory channels for consumer desktop at some point from AMD or Intel? Apple seems to be the only one that offers it.

NPUs are more useful for prefill than decode anyway. Memory bandwidth is not the bottleneck for prefill.

it's a distinct piece of hardware based on AMD XDNA architecture, which, coincidentally, much like CPUs, can tap into your RAM pool. there are XDNA drivers (`amdxdna`) for Linux.

Well, as long as it's one product of many, they may be able to segment the market and suffer no drawback.

We are dealing with a hype, but the reality is that AI would change everything we do. Local models will start being helpful in [more] unobtrusive ways. Machines with decent local NPUs would be usable for longer before they feel too slow.

Doubt it, and competition does the same so not like they are going intel coz of that

I'd actually love to have an NPU that isn't useless on my 285k.

Or hoping to get ahead of the Windows 12 requirements.

Indeed, I was buying a laptop for my wife, and she was viscerally against "Ryzen AI": I don't want a CPU with builtin AI to spy on my screen all the time!

They can just buy a regular Ryzen 9000 series CPU, then. Maybe add a real graphics card if they're into gaming.

That is already the case with datacenter "GPUs". A A100, MI300 or Intel PVC/Gaudi does not have useful graphics performance nor capabilities. Coprocessors ala NPU/VPU are also on the rise again for CPUs.

I'm expecting in the not so distant future we'll even have LLMs baked into an ASIC, in our PCs.

That’s already the norm no?

Pretty much every hardware vendor has an NPU

Yes, this has already been the case for years on mobile devices, CoPilot+ PC design requires this approach as well.

Additionally, GPUs are going back to the early days, by becoming general purpose parallel compute devices, where you can use the old software rendering techniques, now hardware accelerated.

We kinda already have it with NPU/TPUs, tho they are usually attached to CPUs (and for some reason come with near zero proper documentation).

I can see separate cards for datacenter use but for consumers they will probably come on same SOC as CPU

Yes.

Even the latest NVIDIA Blackwell GPUs are general purpose, albeit with negligible "graphics" capabilites. They can run fairly arbitrary C/C++ code with only some limitations, and the area of the chip dedicated to matrix products (the "tensor units") is relatively small: less than 20% of the area!

Conversely, the Google TPUs dedicate a large area of each chip to pure tensor ops, hence the name.

This is partly why Google's Gemini is 4x cheaper than OpenAI's GPT5 models to serve.

Jensen Huang has said in recent interviews that he stands by the decision to keep the NVIDIA GPUs more general purpose, because this makes them flexible and able to be adapted to future AI designs, not just the current architectures.

That may or may not pan out.

I strongly suspect that the winning chip architecture will have about 80% of its area dedicated to tensor units, very little onboard cache, and model weights streamed in from High Bandwidth Flash (HBF). This would be dramatically lower power and cost compared to the current hardware that's typically used.

Something to consider is that as the size of matrices scales up in a model, the compute needed to perform matrix multiplications goes up as the cube of their size, but the other miscellaneous operations such as softmax, relu, etc.. scale up linearly with the size of the vectors being multiplied.

Hence, as models scale into the trillions of parameters, the matrix multiplications ("tensor" ops) dominate everything else.

I guess it depends on what games you play. I have an AI Max 395 (Framework Desktop) and it runs every game in my library flawlessly. I'm sure if I played this year's most resource-intensive games it might stutter, but I don't. For me, it's an amazing low power minpc doing triple duty as a gaming PC, development box, and running my self-hosted services for the rest of the house.

Presumably that's why the subheading is:

>First wave of Ryzen AI desktop CPUs targets business PCs rather than DIYers.

As far as I can find, Plex does not support AMD iGPU for transcoding. Jellyfin will work, but support seems rather spotty. For other AI/ML work, it seems like ROCm is up and coming, but support - e.g. for Frigate object detection - is still a work in progress, especially for newer chips.

My proxmox server with a few VMs works perfectly fine with much less compute.

My homelab setup runs out of memory much faster than it does CPU cores.

Maybe also Immich, for face and object recognition.

Apple has their high-bandwidth chips, the rest of the commercial desktop market is effectively running Windows, and Microsoft has no incentive to move towards local AI, their ideal case is that you use their cloud-based services and pay for them forever (you being enterprise clients with thousands of PCs).

Putting Strix Halo into the AM5 socket would make no sense. Half the memory controllers would be orphaned and the GPU would be severely bandwidth-starved (assuming that the memory controller on Strix Halo actually supports DDR5 and not just LPDDR5).

Yeah the next generation of Strix Halo is what would get me excited. I think right now TSMC has no capacity, so maybe we have to wait another year. Kinda ironic that all CPU/RAM capacity is being sold to LLM companies, and as a result we can't get the hardware needed for good local LLMs.

Since this is for desktop, the NPU is irrelevant. Consumer-grade NPUs are not made for high performance. They are optimized for low power consumption, which makes sense when you are trying to run basic AI tasks on a laptop without turning it into a frying pan. On consumer-grade gear, the GPU will outperform the NPU, and since a desktop is not constrained by battery and can dissipate far more heat, the NPU is mostly irrelevant (aside from using it to develop software for laptop NPUs).

Narrator: The RAM prices did not, in fact, go down.

Actually it is Qualcom, as they keep trying to push for ARM, but due to the way PC ecosystem has been going since the IBM PC clones started, no one is rushing out to adopt ARM.