Hacker News

I'm not a very smart person, so take what I say with a grain of salt.

I think the path forward will have agents that use models that are individually specialized tasks (some might use a bigger model, some might use smaller models), then orchestrators that are good at knowing when to use which agent type.

I've played around with this in my own tiny coding agents, for TTRPG NPCs, and even a small experiment where LLMs controlled a MUD client as an NPC that played the game with you (only 5 rooms in the experiment).

Basically, break the tasks down into chunks so you don't have to use generalist models for everything, and can chose the right model for the job.

I'm also running all of this locally, where a generalist foundation model doesn't work, and heavily quantized models don't perform well for all tasks, so for unlimited token budgets, my solution is probably overkill.

Mythos is 20x more expensive though

It doesn't make sense to include the capex cost to train a model in this kind of discussion, because that cost is fixed.

Consider a model that costs $100m to train.

If the vendor then prices it such that each inference token has a margin of 10% over the variable costs to serve (power + server costs), whether or not they cover their costs is based entirely on how many tokens they can sell.

If they sell less than $1bn of tokens, they lose money - the break even point is 10x100m = $1bn.

If they sell $10bn of tokens they make a ton of money.

This also means you can't credibly calculate how much of the fixed training expense is covered by your token spend, because until the model is retired and you can account for how much inference it ran you don't know what percentage of the training cost each sold token was responsible for.

They are losing money because they are training new models and building new data centers. The claim of the video is that they're losing money just serving current AI models. There's just no evidence of that.

i mean this is difficult to calculate because of prompt cacheing, the ratio of input/output token etc, but if you just do some napkin math, i find it hard to believe people are getting this many tokens on a $20 plan.

heres some napkin math

gpt oss 120b is in/out price at 0.039/ 0.18 per million on open router. heres some assumptions.

1. the ratio of input/ouput is about 25/1. (coding is mostly grep and fairly low outpu)

2. you are getting 75% prompt cache reads

Case B: 50% Prompt Caching Discount (Standard Provider Rate)At 75% Prompt Caching:Total Tokens Obtained: 658,749,010 (approx. 659 Million tokens)

Input: ~633mil

~475 mil cached at 50% input pricing = ~$9.25

~158 mil uncached = ~$6.15

tokensOutput: 25mil tokens ($4.5)

This doesnt even account for profit margins on inference providers, or the fact that openAI probably has a much more efficient inference stack.

its really hard to know what these companies are actually paying, but from everything im hearing, people are reporting API inference pricing is 50% margin.

If you're claiming "AI inference is sold at a loss", it's on you to prove it.

All we have actual evidence of is: some users use enough AI that the subscription is sold at a loss to them (up to degenerate cases: usage maxed out at all times), if billed by API metrics, while some other users are, by the same metrics, profitable (down to degenerate cases: a forgotten subscription with $20 a month and 0 usage).

We don't know how API prices relate to costs - we only have estimates. And we certainly don't know how much inference does an average subscription user spend.

If you have some sort of information that would decisively prove that the aggregate is "AI company N is losing money on subscriptions", then, show it.

Or is it you who's blinded by faith? Like some sort of AI bubble cultist? The bubble is real, you just have to believe in it?

You don't know the actual margins -- you only know the API rate. If their API rate has huge margins and the average subscriber isn't coming anywhere close to their limits, the subscription can be very profitable. If they're only near peak capacity in peak working hours (when API traffic is most active) and subscription 5h limits help them redistribute a lot of use outside peak hours when they've got spare capacity, that alone could make a massive difference in profitability.

I think there are unknowns in the calculation:

- What are the margins of Anthropic over their API pricing? Without this, all we're saying is the API is more expensive for heavy usage

- How have their price margins changed over time? I imagine built into their commercial model is the expectation for inference to get cheaper over time

- They have tighter usage windows than they used to, now having both 5-hour and weekly limits, they also seem to experiment with their usage quite often, this probably affects user's average utilisation, do they have any other levers they can pull here? eg how does changing to an 8-hour window affect it, or limiting certain models to API-only usage based on capacity like Fable

I know there is a certain level of subsidised usage built into their subscriptions, the VC-funded company playbook, but I don't think anyone from the outside knows for sure how much it is and I imagine it's lower than most people think, and reducing over time.

Many of us would love to, and the models are there, but we're constrained by heavily inflated hardware costs.

If big AI does crash out, it would be an absolute gold-mine for local LLM. Cheap, efficient, Nvidia GPUs, and RAM that can run the best local models already available, will be a real boon.

PS - And as great as Qwen3.6-27B is, how large you can scale it (i.e. how big of a context/project) is mostly hardware constrained.

Err, yes, until the surplus kills off all other competition and allows the supplier to jack prices up sky high, or otherwise bend consumers to their will. There's a reason most countries will stop foreign firms from doing this to them.

Okay ... and for all of Europe?