Hacker News

Benedict Evans may be right after all; frontier models look more and more like telecom companies in the 90s. Billions and billions of investment in infrastructure while others further up the stack captured all the value.

> while keeping control of the UX.

Extremely tangential, but this is my favourite upshot of AI. For decades, companies have been walling off their services and forcing us into their fuckass UIs. Now over the course of the last twelve months, suddenly everything has an MCP and I can use it through my command line chat interface.

Any company that doesn't adapt gets so hammered by people's AI-DIY web scrapers that they have no choice but to cave.

Does “the best machine for AI use” apply here considering these models are still server-side?

I think there is an opportunity for a new hardware company to enter the market. I know this is just hypothetical but I believe that AI is revolutionary enough where a new approach to hardware and UI/UX will enable far more value to be derived from AI. I think the incumbents like Apple will stick to their familiar platforms and could get beaten out by a new competitor that is AI native to the core. Maybe? ¯\_(ツ)_/¯

It's been clear for years now that eventually ai will be embedded at the os level. Apple even recognized it way back when they first introduced Apple Intelligence. Yes they're commoditizing llms or whatever. But this has been a user facing feature they've been iterating on for years now

Now we only need to commoditize the hardware.

How is this Apple keeping control of the UX?

Check out this WWDC session. Obviously not going to compete with the frontier models (and I think 8GB is too small anyways), but Apple did demo MLX + OpenCode.

https://developer.apple.com/videos/play/wwdc2026/232/ https://www.youtube.com/watch?v=wykPErJ8M-8

You can use OpenCode or Pi with SSD streaming so it technically will have all the features, just unbearably slow.

I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.

> all of the existing features of Claude Code but somehow running locally on my laptop's neural engine

You can use environment variables to have claude code query literally any endpoint you choose as long as it has a compatible API.

I think that's what they are trying to avoid. If you need on-device intelligence, their pitch was "The model the device already has is best", and if you need something more specific an adapter (aka, a fine-tune/lora) is best.

They were wrong when their on-device model was way behind. They still might be right in the long term.

While multiple app I use might need Gemma 4 E4B, I use dozens of apps and app devs can choose from hundreds of models. A shared cache might reduce size a little when there's overlap, but the core problem still exists. If each app chooses a model disk and memory-swapping explode.

Its probably be better for device manufacturers to bake in a default. I'm not proposing they limit you from using others, but one shared default might be best developer/user experience for 99% of apps.

- Being warm in memory is the single biggest perf speedup you can get, and a default is much more likely to be warm.

- "Best model" is usually "best model for this device" given both RAM and compute. A developer can't test every device but Apple can/will.

- Each model needs to be optimized for the hardware (what's running on ANE, what's running on Metal, what's running on CPU). The default gets optimized.

- If you need custom model, a Lora is probably best (30MB, benefits from all of the above)

You could say the default should be swappable, but that's more a linux ideal than an Apple one so I doubt we ever see that. Plus there are real downsides: intentional or not, prompts end up optimized to the model they are developed for, so swapping the default system model would degrade every app.

That's a great opportunity for Apple to provide a universal unique model ID protocol and some shared storage space to allow devs to register models.

Check out “Bring an LLM provider to the Foundation Models framework” - https://developer.apple.com/videos/play/wwdc2026/339

That is exactly what foundation models are, yes. Same in Android with AICore which uses Gemma underneath, apps can query the LLM and receive responses back rather than bundling in their own model.

The apps can use the system provided on-device model using the same framework and APIs; but there's no affordances to deduplicate custom models between apps.

Ok but don't expect Anthropic to help with local models, that'll be something apple rolls out themselves if at all

Do you guys not have phones (with at least 1TB of storage)?

Sounds ripe for block-level deduplication. :D Or an API that lets you request a model and handles caching.

The important part is Apple rebranding “OpenAI-compatible API” to “language model protocol” and I think we should all rally around this immediately before we’re cursed with that awful tongue twister.

Apple has some clever mechanics to protect user data. I had to work with App tracking stuff lately and their approach to keeping user details private with anonymized cohorts (SKAN, Differential Privacy) before reporting tracking events to third party platforms was surprisingly well thought out. There is value in having them in your loop if you care about privacy.

This is support for a new framework that ships with reality/mac/iPad/watch/tv/iOS 27 (and that they've promised to open-source later in the year, so presumably you'll also be able to lean on this if you ship Swift on your backend).

The framework's whole deal is that it lets you use the same API to target either the device built-in models, the Apple-hosted online models (Private Cloud Computer), or write your own shims to call out to arbitrarily hosted online models.

You can then dynamically route your calls to a different kind of model/provider, using system APIs, without having to write your own abstraction layer over "I want to use local model for this, but I want to use Claude for that", or having to integrate your own API integration with Anthropic/OpenAI APIs.

It abstracts things like tool calling in one place; and has a bunch of other niceties/oddities (it keeps the same "transcript" going, even if you dynamically switch providers/models during a session) and some other things.

A dark, but not totally unfair take: It makes it easier for Apple to take payment for the models others provide, and even allows Apple, if they want to, to use the data to build a dataset for training their own models based on how users use third party models. It's only on Apple devices this API is used, so they split up the market by not letting developers use the same system if they want things to work on iOS, locking users even more in.

The cynic (or realist?) in my thinks this abstraction layer is Apple's way of making sure that users give their own Apple Intelligence credit for the underlying LLM functionality, even if another company is actually providing the LLM.

There are already on-device models that you can use through this framework as a developer. Claude would just be an additional one.

Maybe they plan to have the providers pay for being the default model? So basically, what Google is doing right now for search engines. The difference however is that Google is making money with additional search requests while AIs are (as of now) losing money with additional requests. I don't see the business case for them yet though.

> which I think we’ve heard they’ve been spending lots of money on training and might be somehow involved with Siri or current Apple AI

Lol bro this is literally it this is the model they've been training (was Apple Foundation model not a big enough hint?)

UX is just another word for ecosystem building, which is what Apple does best in comparison to their competition and also doesn’t hurt to do hardware to go along with it. Microsoft and Nvidia aren’t teaming up for nothing.

> That'll give apps better UX and save developers money on a bill that Apple doesn't get a cut of.

With other words, it's unlikely to happen as there is no money in it. Better for Apple to create some new subscription "AI" and "AI-lite" plans people can subscribe to, and since Apple is a company and we all know what those care about, it's unlikely to become a utopia of local models running on your phone.

How does using Gemini lead to better on-device models?

The even bigger hurdle is selling token based pricing to normal (non-dev) users.

"You pay an indeterminant amount of money to ask a question and you might not even get the response you want without spending even more money" doesn't appeal to most people who aren't gamblers and explaining how "thank you" at the end of a long exchange can be expensive due to context is an even harder thing for an average person to swallow.

Token cost going up/down like a yo-yo also doesn't help. Normal users NEED fixed costs and don't want to expend energy constantly keeping up with the AI meta. "My subscription lasted much longer last month" isn't a winning problem either.

I think Apple is correct that Local LLM for most things is the future.

Ugh. It really is. I have allihat.com which is the only safari extension (i think still) that talks to claude. And it's well sought for. But you as a user have to enter a friggin claude api key. :( And I still don't grok their TOS around this. Like you can still type: ```setup-token Set up a long-lived authentication token (requires Claude subscription)``` but this seems like a trap? :) Whose using this? Doesn't this like insta break their TOS if you use that anywhere?

Right now for allihat.com I just let people use the Apple model locally if you don't feel like using the claude key. And my conversions to paying user shot up like 3x! But it really isn't a replacement obviously to claude. I was hoping Apple would make proxying to Claude some kind of thing they do for me so I also don't have to proxy to my own server just to try and manage API to Claude usage.

> For production, route requests through your own back end with .proxied

Apple is offering developers with less than 2 million downloads free AI models via their servers https://techcrunch.com/2026/06/08/apple-bets-cheaper-ai-will...

Users don’t give a API key. The docs show how to set up your backend proxy.

The same way you did it before — by proxying the requests to your backend.

Why?

You wouldn't use this when building a coding agent.

How does it work now though? There is a Claude app on iOS

it says:

Proxy (production)

For production, route requests through your own back end with .proxied. The relay at baseURL adds the Claude API credential server-side, so the app ships no key. The headers you provide are sent on every request so your proxy can authorize the caller.

https://platform.claude.com/docs/en/cli-sdks-libraries/libra...

>Model identifiers are values of ClaudeModel. Use a compiled-in constant, or construct one with explicit capabilities for an ID that isn't compiled in yet (see Capabilities):

Special emphasis on the "isn't compiled in yet" and "or construct one" bit.

https://platform.claude.com/docs/en/cli-sdks-libraries/libra...

it says put into your API layer and proxy it.

> It made it sound like they were training their own.

They are.

The pricing squeezes will continue until token spend improves!

Shared daemon (as others pointed out), and, later shared revenue, probably with Apple receiving payments to ship ad-laden, “editorialized” models. Hopefully, it’ll go the other way, and Apple will subsidize high quality model training.

OpenAI is worse from a public policy standpoint, and apparently Fable was yanked at Amazon’s request.

Enough is enough. I’m seriously evaluating open models this week.

Read the linked article? It is absolutely a cloud service. Neither Apple nor Anthropic is suggesting otherwise

it's cloud, the doc is explicit that requests go straight to api.anthropic.com with Apple not in the way.

so Claude via FM dies offline while Apple's on-device SystemLanguageModel (the ~3B one) keeps working. It isn't a hybrid really: the framework just has both implement the same LanguageModelSession protocol so "local 3B" and "remote frontier model" become a one-argument swap.

IMHO what's worth internalising is that the two share an API but nothing else: the on-device path runs on Apple's Neural Engine and costs battery (you can watch ANE power ramp while it works) while the cloud path costs API credits/tokens and does zero local compute. Same code, opposite cost model.