Hacker News

Apple’s Photos ecosystem is privacy oriented and as a result some (now) baseline features require the use of local models.

The Foundation models keep a surprisingly decent set of data available for fast filtering by a local LLM.

I’ve found a slower, second pass, is needed by Qwen to clean it up. (I am running on the new Mac Mini m5 pro 64gb.)

I’ve run my own tests using a date range, a short description of a red cedar planter box build I did last year, and a set of known correct photos.

There is some cool stuff you can do without sending your family’s pics up to any of the frontier model providers.

Exactly my thought as I read it.

They make and sell the hardware that will actually make AI something everyone can used without needing to rely on OpenAI/Anthropic et al.

Apple has never had a moat, anyone else could have built this sort of hardware but they haven't.

So they don't care which model wins, because it will always move out to the edge of the network, and they own a large chunk of that edge.

Nvidia is also selling the shovle but they train their own LLM and by doing so they can fine tune their CUDA/Chip to have a better shovel than others.

Nvidia's own model does not have to beat OpenAI or any SOTA model. Nvidia just need to build up the in house expert and knowledge to make a better chip since hyperscalers and none hyperscaler like Meta is building their own AI chip now.

Apple can chose not to train their own model to learn how to sell a better shovel.

> They are out here selling the shovels

just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).

That's just BS cope. They tried and miserably failed - and still are failing badly. Remember, they were the ones to push for AI with "Siri". Many firings later and 2 CEOs later we are still exactly in square one. The fucking thing can't even get who I want to call, let alone set an alarm without touching my phone FFS.

apple is pretty well set up for this because they do all their hardware in house. if they wanted to, they could pivot in this direction. Ternus is also a hardware guy.

They have been sold out of a lot of units, due to people developing AI, so apparently, it is significant.

I think the bet is that it will get bigger, both at the consumer level, and with orgs eventually discovering ready-made solutions for having their own on-prem inference.

100,000 units a year would be $2 billion annually.

And usually, most of a company's revenue comes from the heaviest spenders.

So figure there are some out there who will by tens of these because they have some "idea to change the world". (Whether true or not.) Some startups, etc.

I'd wager, across planet earth right now, it should be trivial to push 100,000 units per year. Probably closer to 500,000 units per year. So that's an easy $10 billion a year.

$2-10 billion is meaningful for any company. Even one as large is Apple.

Especially if it's a first step towards a stronger foothold in the AI inference space.

They can then expand down market to grow the pie further.

For example, with truly dedicated inference boxes.

And given the demand for $7,000 Sparks and sold out $5,000 GPUs, the market for these $20,000 machines is likely much higher than my 100,000 unit lower bound. (Earth is a big place. It doesn't take much excitement to generate massive sales in a space with currently insatiable demand.)

idk but naive me sees the article say the model needed 158GB of RAM so I guess 10x 16GB GPUs? Apparently retail price is $1k per 5080 16GB but they're actually selling at $1.5k so I guess $15k for just the GPUs.

I disagree.

I think the iOS App Store is going to be in steep decline in a few years. People will generally have a few apps: ChatGPT, Instagram, messaging, and something like a Photos/Camera app.

Everything else will be done in your AI app.

I've been hooking up more and more data and plugins into ChatGPT/Claude. I only want to use it. I don't want to use other apps if I don't have to. For example, research, buy, plan. I prefer not to use other apps.

I think all it takes is for OpenAI to make a phone from the ground up for this and Apple is going to have serious competition.

If we're being frank here, Apple's AI strategy lags behind Nvidia, AMD and even Google. Nvidia has CUDA, AMD ships CDNA, Google has the TPU family, and Apple... well, Apple discontinued the rackmount Mac Pro, forfeit the HPC ARM datacenter to Nvidia's Grace CPUs, refuses to sign CUDA drivers, and spent half a decade designing raster GPUs instead of GPGPUs.

All of their competition has real inference and training stacks. Apple ships an iGPU and a smartphone NPU, they're reliant on Nvidia and Google hardware to deliver Apple Intelligence.

Even in the unlikely case that does happen, there are plenty of other open weight models like Qwen, GLM, and Mistral to name a few

I find it bizarre of Apple to do anything publicly in partnership with Meta. I always thought the reasons Apple snubbed Facebook were out of values and principles not business. That could be wrong. Now that Meta has a line of business that Apple can finally charge a platform tax on there is new reason to start sleeping with them.

They don’t lead with it. The title of the actual article is “Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio”.

The submitter changed it to “Apple ML Research: Muse Spark Open with MLX on Mac Studio”

But there are definitely people out there that will need to run these models locally (contractual or legal requirements). By validating a Mac Studio as an option for running local models, it might be the more appealing option.

For most of us though, you're right -- the OpenRouter option is far more cost effective.

It seems Apple figured they could sell more Mac Studios on the basis of frontier local AI models, profit to be made while users haven't figured out the math yet.

Puts into perspective how cheap the frontier models are via subs or even API.

There was the M3 Ultra 512GB Mac Studio, though.

Imagine if you had 4 of them!

Most people are probably looking at: https://artificialanalysis.ai/models/comparisons?compare=mus...

This is still a welcome model, even if I won't be running it at home

Do note that those are en-dashes and not em-dashes.