Hacker News

Yup. Also regardless of price they need to spend more and more as the project collapses under the inevitable incidental complexity of 30k lines of code a day.

It's similar to how if you know what you're doing you can manage a simple VPS and scale a lot more cost effectively than something like vercel.

In a saturated market margins are everything. You can't necessarily afford to be giving all your margins to anthropic and vercel.

> The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up.

It's not that clear. Sure, hardware prices are going up due to the extremely tight supply, but AI models are also improving quickly to the point where a cheap mid-level model today does what the frontier model did a year ago. For the very largest models, I think the latter effect dominates quite easily.

> Not being dependent on LLMs for your fundamental product’s value

I think more specifically not being dependent on someone else's LLM hardware. IMO having OSS models on dedicated hardware could still be plenty viable for many businesses, granted it'll be some time before future OSS reaches today's SOTA models in performance.

What's weird though is the bifurcation in pricing in the market: aka if your app can function on a non-frontier level AI you can use last years model at a fraction of the cost.

That'll be (part of) the big market correction, but also speaking broadly; as investor money dries up and said investors want to see results, many new businesses or products will realise they're not financially viable.

On a small scale that's a tragedy, but there's plenty of analysts that predict an economic crash and recession because there's trillions invested in this technology.

No shit. People are just figuring this out now?

This is the “Building my entire livelihood on Facebook, oh no what?” all over again.

Oh no sorry I forgot, your laptops LLM can draw a potato, let me invest in you.

in fact I am betting opposite. frontier models are getting not THAT much better anymore at all, for common business needs at least. but the OSS models keep closing the gap. which means if trajectories hold there will be a near future moment probably where the big provider costs suddenly drop shaerply once the first viable local models consistently can take over tasks normally on reasonable hardware. Right now probably frontier providers rush for as much money as they possible can before LLMs become a true commodity for the 80% usecases outside of deep expert areas they will have an edge over as specialist juggernauts (iE a cybersecurity premium model).

So its all a house of cards now, and the moment the bubble bursts is when local open inference has closed the gap. looks like chinese and smaller players already go hard into this direction.

Absolutely. Pricing exposure is the quiet story under all the waves of AI hype. Build for convenience → subsidise for dependence → meter for margin is a well-worn playbook, and AI-dependent companies are about to find out what phase three feels like.

Hyperscalers are spending a fortune so we think AI = API, but renting intelligence is a business model, not a technical inevitability.

Shameless link to my post on this: https://mjeggleton.com/blog/AIs-mainframe-moment

How is that surprising? We've been taking that into account for any LLM related tooling for over a year now that we either can drop it, or have it designed in a way that we can switch to a selfhosted model when throwing money at hardware would pay for itself quickly.

It's just another instance of cloud dependency, and people should've learned something from that over the last two decades.

Any company that has become dependant on AI will struggle to survive from here on. By the time many teams realise it'll be too late.

Marginal costs matter in this world.

The companies that are entirely AI-dependent may need to raise prices dramatically as AI prices go up

Or they'll price the true cost in from the start, and make massive profits until the VC subsidies end... I know which one I'd do.

I wonder if it could be that they won't because the real mechanism is that AI wrapper pricing power is weak (switching costs near zero) but state of the art models makes it difficult to lower prices due to higher cost.

Also: AI dependance could be explicit AI API usage by the product itself, but also anything else, like: AI assisted coding, AI used by humans in other surrounding workflows, etc.

This was as clear as the sky when the first llm based businesses popped up. How did you realize this only now?

Not really, the next move is to establish standards groups requiring the use of AI in product development. A mix of industry and governmental mandates. What you view are viewing as COGS instead becomes instead a barrier to entry.

China already operates like this. Low cost specialized models are the name of the game. Cheaper to train, easy to deploy.

The US has a problem of too much money leading to wasteful spending.

If we go back to the 80s/90s, remember OS/2 vs Windows. OS/2 had more resources, more money behind it, more developers, and they built a bigger system that took more resources to run.

Mac vs Lisa. Mac team had constraints, Lisa team didn't.

Unlimited budgets are dangerous.

Harness is a big one, Claude Code still has trouble editing files with tabs. I wonder how many tokens per day are wasted on Claude attempting multiple times to edit a file.

Yep.

As a recent example in AI space itself. China had scarce GPU resources, quite obvious why => DeepSeek training team had to invent some wheels and jump through some hoops => some of those methods have since become 'industry standard' and adopted by western labs who are now jumping through the same hoops despite enjoying massive computeresources, for the sake of added efficiency.

Absolutely. Anyone working on inference token level knows how wasteful it all is especially in multimodal tokens.

Could not agree more, this will spur innovation in all aspects of local models is my hunch.

What do you mean by harness here?

[dead]

It feels like a repeat of the dot com infrastructure buildup that spurred the whole 2005 explosion in affordable hosting and new companies. This will probably leave us massive access to affordable compute in a couple of years.

OpenAI has an absurdly high valuation given their cash burn vs RRR.

Anthropic's is far more reasonable.

It makes no sense to lump these two companies together when talking about valuation. They have completely different financial dynamics

> would resort to writing code by hand because that's a very skilled and hardworking tribe they've been doing this all their lives

Shush, don't tell that to the AI coding acolytes.

I agree and I am amazed at how much money some individuals and also a friend's company burn on token costs. I get huge benefits from this tech just using gemini-cli and Antigravity a few times a week, briefly. I also currently invest about $15/month in GLM-5.1 running Hermes Agent on a small dedicated VPS - fantastically good value for getting stuff done and this requires little of my time besides planning what I need done.

I think the token burners are doing it wrong. I think that long term it is better to move a little slower, do most analysis and thinking myself, and just use AI when the benefits are large while taking little of my time and money to use the tools.

The fun times will also end for consumers soon enough, when the oligopolies are established and investors start asking for returns.

China is installing something like 500 GW of wind and solar per year now. Even if they're only able to build and otherwise access chips that have half the SoTA performance per watt, they will win.

China's domestic chips are increasingly close to state-of-the-art. The US electrical grid is... not.

US energy is constrained by the utility monopolies/oligopolies which have to extract more rents, specifically by increasing costs. Their profit is a percentage of cost, these perverse incentives + oligopolies will make it increasingly expensive to make anything (including AI) in US.

The dynamics vastly favor China, part of the reason the US sprinting towards "ASI" isn't totally boneheaded is that the US and its industry needs a hail mary play to "win" the game, if they play it safe they lose for sure.

On X I had seen him mostly posting memes so this post seems par for the course

> Infra is always limited, even at hyper scalers

I think maybe infra is limited only at hyperscalers. For the rest of us it's just how much capacity to we want to rent from the hyperscalars.

It's kind of a recent cloud-native mindset, since back in the day when you ran your own hardware scaling and capacity was always top of mind. Looks like AI compute might be like that again, for the time being.

One graph, about 100 words, AI in the title: Hacker News front page.

Not bad for a coffee break of effort.

Jensen just said that if the signal/commitments are there, ASML can scale in 2-3 years.

I've seen this claimed, but I'm not sure it's been true for my use cases? I should try a more involved analysis but so far open models seem much less even in their skills. I think this makes sense if a lot of them are built based on distillations of larger models. It seems likely that with task specific fine tuning this is true?

The mainframe/PC analogy is spot on. And the hardware floor keeps dropping - you can grab a mini PC with 32-64GB RAM for a few hundred bucks and run surprisingly capable quantized models locally. Something like https://terminalbytes.com/best-mini-pcs-for-home-lab-2025/ shows the kind of hardware that's now available at consumer prices. The "scarcity" framing only makes sense if you assume everyone needs frontier-tier models for everything.

Buy new Macs from where? There is a shortage of RAM, SSD, GPUs, and the CPU shortage just started.

That’s nonsense. Local models don’t have any of the nuance in text responses. I find them more akin to GPT 3.5 than even 4.x

But that concerned mostly only gamers and cryptominers. AI is supposed to be replacing traditional software development, which affects everything.

> In 5 years consumer chips and model inference will be so good you won't need a server for SOTA.

Naw man, you crazy. If you tell me that in 5 years, consumer chips will be so good that I can run GPT-5.4-level AI on my phone, I'd find that plausible (I buy cheap phones). If you're telling me that in 5 years we won't need _servers_ because our _phones and/or desktops_ will be powerful enough to run the biggest newest LLMs in existence, I question your judgment, I think that prediction shows a deep uncreativity about how massively compute-hungry SOTA models will get.

The valuable things to do with inference will keep being a server niche because they'll keep being 1-2 OOM more compute-hungry than whatever consumer hardware can handle. Like gaming: my laptop can run games from 2015 at max settings no problem but the games actually worth getting excited about in 2026 still melt a $2k GPU, because whatever headroom the hardware gains, developers immediately spend on ray tracing and Nanite and modelling individual skin cells or whatever. I don't see any plausible reason to expect that the ceiling on "valuable server-side compute" or "inference capacity" will rise any more slowly than the on-device capability is rising.

My assumption is that in 2031, SOTA top-intelligence AI will be hosted on cloud servers like it is today, offering dirt-cheap access to capabilities we can't even dream of today, while your Android will be running some open-source GPT-5+ equivalent.

There is a major logic flaw in what you're saying.

'If I am a grocery store that pays $1 for oranges and sells them for $0.50, I can't say, "I don't have enough oranges."'

How about 'if I'm a grocery store and I see no limit on demand for oranges at $.50 but they are currently $1, I can say 'if oranges were cheaper I could sell orders of magnitude more of them'.

Buying oranges for $1 and selling for $0.5 is an investment into acquiring market share and customer relationships and a gamble on the price of oranges falling in the future.

The grocery store analogy works if compute is the orange.

But labs arent buying oranges — theyre buying the only orchard on the island, hoping it yields a fruit no ones grown yet. Burning $1B to net $500M isnt "I have too few oranges." Its "Im betting the farm Ill find a new one."

Both can be irrational. Theyre irrational in different ways.

You can if you're exhausting the global production of oranges.

If there were more oranges you’d pay less to buy them and your economics would work out.

You misunderstand.

"I built a ship to go to the Indies and bring back tea."

"Bro, the ship cost 100,000 pounds sterling and only brought back 50,000 pounds of tea. I don't care if you paid 12,500 pounds for the tea itself, you're losing money."

There is a very rational reason labs are spending everything they can get for more compute right now. The tea (inference) pays 60%+ margins. And that is rising. And that number is AFTER hyper scalars make their margins. There is an immense amount of profit floating around this system, and strategics at the edge believing they can build and control the demand through combined spend on training and inference in the proper ratios.

ASML only makes a certain number of machines a year that can do extreme ultra-violet lithography.

Also - turbine blades limit power, according to Elon.

Between them - we cannot chip fabs past a certain rate, and we cannot stand up the datacenter to run these desired chips past a certain rate. Different people believe one or the other is the 'true' current bottleneck. The turbine supply chain scaling looks much more tractable -- EUV is essentially the most complicated production process humans have ever devised.

> and compare expenditure vs actually delivered value

Which means that the hype production will be driven up another few notches to make people doubt their rational findings and keep them in irrational territory just a tad longer. Every minute converts to dollars spent on tokens.

Distillation doesn't give you an equivalent model.

The economic hypothesis that has dominated the past hundred years is that economic growth is infinite because resources are infinite and (almost) free. We all know this is unrealistic and disconnected from our human condition.

Regarding "innovation", I agree with your idea. I even think that the major innovation will be to transpose models locally, using reduced infrastructures that will still be sufficient for the majority of use cases.

> Does this also mean ram prices are not coming down anytime soon?

One person replies "yes". Another replies "no".

This concludes our press conference.

<3 HN

yes, and it will keep increasing

they already are

The human brain is incredibly efficient (Approximately 20W of energy consumption¹). These AI systems use many orders of magnitude more energy than human equivalents.

1. https://pmc.ncbi.nlm.nih.gov/articles/PMC8364152/

The problem with this idea is that someone can, and likely will, come up with the next best architecture that leapfrogs the current frontier models at least once a year, likely faster, for the foreseeable future. This means by the time you've manufactured your LLM on an ASIC, it's 4-5 generations behind, and probably much less efficient than current SOTA model at scale.

It won't make sense for ASIC LLMs to manifest until things start to plateau, otherwise it'll be cheaper to get smarter tokens on the cloud for almost all use cases.

That said, a 10 trillion parameter model on a bespoke compute platform overcomes a lot of efficiency and FOOM aspects of the market fit, so the angle is "when will models that can be run on an asic be good enough that people will still want them for various things even if the frontier models are 10x smarter and more efficient"

I think we're probably a decade of iteration on LLMs out, at least, and the entire market could pivot if the right breakthrough happens - some GPT-2 moment demonstrating some novel architecture that convinces the industry to make the move could happen any time now.

I don't think so. GB200 prices are GOING UP. A100s are still expensive. This implies massive utilization and demand, no? These machines are not sitting idle, or prices would drop in the very competitive hyperscaler environment.

That worked out, for the founders of frontier labs at least.