Hacker News

Newer process nodes are not the main avenue of improvement. What those transistors are used for is more important and it’s plausible that improvements between generations can increase performance by multiples on a specific task. All of the improvements aren’t necessarily in the chip itself either.

E.g. the next gen might have hardware inference for lower bits, more memory bandwidth, etc.

Almost seems as if microchips are approaching their "B-52 age":

"Those things are still flying! Introduced in 1955!"

"But that was the B version, all those that are still flying are the H version, so many iterations between them!"

"Welcome to 1962"

> but an 5x efficiency gain in a single generation feels like its too much, especially considering how newer process nodes have been yielding less and less improvements

The efficiency is in other areas too e.g. memory, network, etc. It's TOTAL.

> Here's a synthethic benchmark page listing every GPU in recent memory

We don't have the GPU gains not because of process nodes. Nvidia and later AMD stopped investing in that direction. They started optimizing for AI not graphics.

Xeons have a much longer shelf life and diverse workloads. If you order hardware specifically for LLM inference and then some new hardware/model combination is much better at that (which it will be, because a lot of people are working on that), you might be in trouble.

It's like setting up a warehouse of GPUs to mine bitcoin while others are switching to ASICs.

Infiniband and coherent fabric.

But this is exactly why Oracle is the wrong partner. They don’t get it. They never will. To them, it’s just a “workload”.

And on the DC side they will be building to a power and heat budget. If Vera Rubin changes the power density per rack equation that may have some impact. But thinking rationally if the flops per kw-sq ft are lower than Blackwell, no problem. If they are a lot better then even if the kw per sq ft is higher you can just space the racks out a little

It might be 400% in the one thing everyone is interested in.

you think in FP16. nobody uses FP16 for inference anymore. 400% probably for FP4/INT4 computation.

I think the difference is that the other hyperscalars are doing this out of the enormous cash rivers produced by their other profitable businesses, at a rate less than that at which profits are flowing in, whereas Oracle is funding it out of debt with AI capex in 2026 projected to reach levels nearly as high as their expected revenue (not profits) in the same period.

If the hardware refresh rate makes a substantial share of data center cost function more like opex than capex, the companies funding it out of operations (especially from operations of what are essentially monopoly businesses, in the sense pricing power), even if it isn’t the operations it power specifically, are fine in the near-to-intemediate term (barring exogenous shocks to those other businesses), whereas Oracle, funding it by a debt bonanza, is in a different position.

Google, Amazon, Meta, etc don't have to wait 12 or 24 months for their big data center to open. They already have lots of DCs to cram all the NVidia cards into, right now.

Only the ones that require profit from the GPUs.

All DCs are big concrete rooms that can supply so much power per sq area and remove so much heat per sq area (the two related of course since the heat comes from dissipating the power). Variation is just in density of whatever sort of fancy resistor you plan to put in the concrete room.

I also don't want the next Bari Weiss in charge of WB

On the contrary, Netflix would have been decent because WB is bigger than them (in terms of IP, existing content, brands, etc) and would have probably (at least that's what they said publicly) left it mostly alone. And it's weird how people assume that just because Netflix produce a ton of content, all of it is low quality. A lot of it is for people half paying attention, but there is plenty of actually good stuff. Them having WB would improve them on both fronts.

Nepo baby is coming with a political angle and wants control of the news media part of WB. The American media landscape is already without much competition nor diversity in political views, now there would practically be none.

> The only consolation is the lawnmower is 81 and thus is going to be dead soon

Reminder to lay up your treasures in heaven.

> In our current capitalist society once you're into the billions you're "too big to fail" and you have unlocked the infinite money glitch.

That's not how any of this works. "Too big to fail" can be applied to companies, but I don't know of any examples of it being applied to people.

> At his level of personal wealth there is no realistic scenario that leads to personal bankruptcy. In our current capitalist society once you're into the billions you're "too big to fail" and you have unlocked the infinite money glitch.

This is plainly false. There are plenty of example, even recently, of billionaires losing their fortunes or going bankrupt. Often they come with criminal prosecution because they get desperate and try illegal ways to hang on to their wealth. Sam Bankman-Fried, Elizabeth Holmes, and several other examples come to mind.

There are a lot of stories of billionaires getting too risky with their investments or too concentrated in businesses and losing the majority of their wealth. The Barclay story, Jim Justice, the old Peloton CEO.

It’s not a common outcome because you have to try hard to screw up that badly when you have over a billion dollars in wealth. Parking it anywhere in common investments would leave you and your ancestors set forever.

> The only consolation is the lawnmower is 81 and thus is going to be dead soon (even the mega-wealthy can't plastic surgery themselves out of this outcome, at least not yet)

Haven't been following Bryan Johnson, eh?

With respect to consumption, it’s pretty efficient vs older traditional servers, though I know workloads like that aren’t completely fungible. Nonetheless it bears keeping in mind that a single GB200 NVL72 rack provides 1.4 ExaFLOPS of AI compute (at FP4 precision, ideal circumstances, but this is envelope math all around). So it’s power efficient, for what it is.

Since you and me and everyone else will foot the elecricy bill. Energy consumption or efficiency is not a concern.

The compute power is also ridiculous.

Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.

There are box trucks with less power consumption.

Something is probably happening but I don't know what it is. Maybe this is really a negotiation over price.

I agree with you more than I agree with the parent comment.

To use the hit HBO TV show silicon valley analogy, it is far more likely that "the bear is sticky with honey" will happen at Oracle than at Open AI. Some kind of game of telephone gone wrong at some point and now the people responsible at Oracle must double down in order to kick the can to the next quarter and not appear clueless.

Statutory disclaimer: I am not affiliated with either Open AI or Oracle and have no insider information. All of this is mere conjecture and has no basis in reality.

OpenAI is a unreliable narrator as long as Sam is in charge. Full stop. EM_DASH.

>cnbc are fabricating stories while drunk?

Don't forget the possibility that it's AI slop.

> CNBC are clueleess?

That sounds about right.

> People at openai are lying to cnbc?

Remove "to cnbc" and that's a yes.

> cnbc are fabricating stories while drunk?

Maybe not drunk but likely high.

> Unless Nvidia can launch a new chip every 2-3 years with massively improved performance-per-watt at a lower price no one is going to rush to recycle the old one.

That's exactly the point.

Performance/watt is increasing so much gen-to-gen that it makes no longer sense to run older hardware.

Not my words, Jensen's.

you can absolutely run e.g. datacenter-level A100 at home, there are adapters from the SXM to the PCIe socket. Haven't seen people running SXM versions of H100s this way but this could be due to the price factor only

> I doubt anyone has the setup to run a H200 in their home rig.

There are PCIe versions of these right? And another comment is saying there are PCI adapters too. It "only" requires 600 to 700W. It's not out of reach for everybody.

If the used regular server market is any indication, you can find, after a few years, a lot of enterprise gear at totally discounted prices. CPU costing $4K brand new for $100 after a few years: stuff like that.

A friend has got a 42U rack and so do some homelab'ers. People have been running GPU farms mining cryptocurrencies or doing "transcoding" (for money).

It's not just CPUs at 1/40th of their brand new price: network gear too. And ECC RAM (before the recent RAM craze).

I'm pretty sure that if H200 begin to flood the used market, people shall quickly adapt.

> Unless Nvidia can launch a new chip every 2-3 years with massively improved performance-per-watt at a lower price no one is going to rush to recycle the old one.

I agree with that. But if they resell old H200s, people are resourceful and shall find a way to run these.

K80's are up on eBay.

9% is an absurd failure rate for solid state electronics. Particularly considering the profit margins. I assume it's related to the power densities involved. Would you happen to recall the source?

We already have that. It's called ebay.

That might work to some extent with the DDR5 DIMMs connected to the CPUs, but is thoroughly impossible for the HBM DRAM stacks packaged with the GPUs.

I wish there to be an active market like what Gamer's Nexus covered in China:

https://www.youtube.com/watch?v=1H3xQaf7BFI&t=1577s

in the States.

I previously ran 150,000 AMD gpus in all conditions at 100% utilization for years. I currently have a multi-million $ cluster of enterprise AMD GPUs.

A couple real world points:

1. They generally don't just fail. More likely a repairable component on a board fails and you can send it out to be repaired.

2. For my current stuff, I have a 3 year pro support contract that can be extended. Anything happens, Dell goes and fixes it. We also haven't had someone in our cage at the DC in over 6 months now.

V100 and A100 are over 5 years old and you can find used ones on eBay.

The data center owners aren't the ones selling new GPUs.

Last I looked (about a year ago,) IBM is 3 things: Software licensing, services and Big Iron. The companies' revenue is split roughly equally among these three areas.

An interesting perspective of IBM is its relative position. It's leveled off at about $60b/y, after a lengthy decline. It is far overmatched by many big tech companies today in terms of revenue.

It's a niche business, serving niche demands. I think IBM's moat is that most of its business is highly uninteresting: industrialized box ticking work, deeply entangled by contracts and a strong need for continuity by its customers.

I actually had a recent encounter with one of IBM's products. A commercial B2B REST API I created was analyzed by an IBM vulnerability scanning platform on behalf of a major US municipality. It didn't find anything actually critical, but there were some worthwhile points in the report, and working around a false positive was a frustration. The product, in this case, is diffusion of responsibility.

On the Big Iron end, IBM isn't really selling hardware. They selling an ecosystem: services, software, support, continuity (over decades,) etc. It pleases me that they chose to stick to Power: it's nice to know Itanium didn't kill off every enterprise RISC platform.

Maybe, one day, some major quantum computing breakthrough happens at IBM. As far as I can see, that's the only play they have that could change their trajectory. In the meantime, they have a large software portfolio and plenty of institutions that will keep signing contracts long after I'm gone.

iirc they still do a lot of chip research and fabrication equipment. They were able to keep that side of the business going at least.

They're one ~ 3 main companies in the US that will sell you quantum computers, and the only one offering a quantum PaaS.

They do a lot of stuff. Also own Hashicorp now, so they have things like: Ansible / RedHat Linux (already owned), Terraform, Consul, Nomad, Packer, etc. A lot of "let's build modern infra" tooling.

Mainframes aren't going anywhere.

Yeah, it used to be true that server GPUs at least somewhat resembled their gaming counterparts (i.e. Nvidia Tesla server components from 12+ years ago); they were still PCIe cards, just with server-optimized coolers, and fundamentally shared the same dies that the gaming and professional cards used.

That stopped being true many years ago though, and the divergence has only accelerated with the advent of AI datacenter usage. The form factor is now fundamentally different (SXM instead of PCIe); you can adapt an SXM card to PCIe with some effort [1], but that may not even be worthwhile because 1. the power and cooling requirements for the SXM cards are radically different than a desktop part and more importantly 2. the dies are no longer even close to being the same. IIRC, Blackwell AI chips straight up don't have rasterization hardware onboard at all; internally they look like a moderate number of general SMs attached to a huge number of tensor core. Modern AI GPUs are fundamentally optimized for, well, mat-mults, which is not at all what you want for gaming or really any non-AI application.

[1] https://l4rz.net/running-nvidia-sxm-gpus-in-consumer-pcs/

I never thought I would see the day, but my stodgy, lumbering company just banned new Oracle databases. Everyone hates Oracle, and only does business out of necessity. I think more and more companies are trying to extricate themselves from Oracle legal, so Oracle needs a new way to leech onto corporations for the coming decades. AI is the best play in sight.

It’s a huge gamble but they have no choice but to take it. Most their software will be rendered obsolete by AI (I’ve vibecoded replacements saving millions already, companies everywhere are doing this right now).

So they have to hope they’re a part of the future in the AI capacity because their SaaS business is going to take a big hit.

YTD performance didn’t fully bake this reality in. It was seen as them having 2 huge revenue streams, the market is realizing that AI is a threat to SaaS and baking that into stonks

Anthropic in 2026 only added several billions of revenue. This is insanely fast. In my company llm cost are already eating hiring budgets to a certain extent. We don’t buy gpu. We are paying to those who will.

> None of these investments are immediately profitable.

If you're OpenAI spending $100M on a training run they're not.

But if you're Oracle renting out GPUs to little guys doing inference, they are.

25 bln from just one company . There will be 6-7 companies like this . And they just scratched the surface . The penetration in many areas is almost 0. Yet.

I'm definitively not interested in buying oracle servers but I imagine AWS and Microsoft are also building out aggressively.

The inevitable bailout.

Would you want the commercial company use your name, or the name of your relative?