H3-metal – Native MiniMax-H3 inference for Apple Silicon (github.com)

293 points by swyx 10 hours ago

Meleagris 8 hours ago

I've been using MiniMax H3 on my M5 Pro 64GB MacBook Pro through ComfyUI. It works extremely well.

I had to modify the default ComfyUI workflows to use a GGUF quant (city96's ComfyUI-GGUF custom node, UnetLoaderGGUF in place of the stock loader) [0].

I use the model labeled Q5_K_M. There is Q8_0 available as well, which is 34GB and fits fine in 64GB unified memory if you keep resolution modest.

The main issue is speed, a ~9-second 480x864 clip at 20 steps takes me a bit over an hour. So this will be cool to try for the speed up alone.

There's a lot of great information and workflows available to follow on the r/StableDiffusion subreddit.

[0] https://huggingface.co/Abiray/MiniMax-H3-GGUF/tree/main/unet

antirez 3 hours ago

This implementation is much faster on my M5 Max, like a few minutes for the same video, but on an M5 Max with 128GB, didn't test on M5 Pro. About memory, could be executed on 64GB with a few changes.

Manfrednotfunny 3 hours ago

Memory bandwidtih between pro and max is double. 300gb/s vs. 600gb/s btw.

antirez 3 hours ago

alexgoodhart 5 hours ago

I wonder how much faster your m5 pro is compared to my M1 Max @ 64gb

yieldcrv 4 hours ago

wait till the M7 bro you’re almost there, rumor has it that Apple is skipping the M6 but it still might be 2028

ignoramous an hour ago

jonplackett 4 hours ago

What is the quality of the output like compared to something like Veo?

Myzura 3 hours ago

How much free space do you have left after running this llm model? Have you tried to develop your own model with the M5?

antirez 5 hours ago

In the AMA Minimax said that H3 could support sparse attention, that would be a huge speedup! I wonder if there are any news on that. H3 is very cool. EDIT: testing a --sparse-attention optional mode based on what they said in the Reddit post.

linzhangrun 4 hours ago

On my 128GB M4 Max Mac Studio, generating a 15s 480p video with MiniMax H3 in ComfyUI takes an hour and a half.

Put Codex to work on deploying it now, hoping the speed can improve quite a lot :-) Thanks anyway

embedding-shape 2 hours ago

> On my 128GB M4 Max Mac Studio, generating a 15s 480p video with MiniMax H3 in ComfyUI takes an hour and a half.

That's crazy, a RTX Pro 6000 does that in in 2-3 minutes (give or take, depending on your exact settings). LLMs don't make the difference between standalone GPU vs unified memory + CPU so obvious as diffusion models seems to do.

whywhywhywhy an hour ago

It’s always been the case, it’s more the anomaly that LLMs work at comparable speeds on M series because almost all other ML runs way faster on Nvidia cards.

sscarduzio 4 hours ago

Please keep up posted about the results!

linzhangrun 2 hours ago

First batch of quick test results: approximately 1/5 speed improvement

diddid 7 hours ago

This is where the DGX spark makes up a bit of the ground it loses on llm work, diffusion and cuda go together like peanut butter and jelly.

embedding-shape an hour ago

cough DiffusionGemma cough

Seriously, very dumb model compared to what you can run locally, but holy moly is it FAST on one GPU, seriously impressive. Can't wait for those to be scaled up a bit to fit perfectly within 96GB VRAM, then they'll be competitive.

TechSquidTV 8 hours ago

This still requires 128Gb of memory, right? Me and my lowly 96Gb, like a commoner; missing out on the fun.

thehamkercat 8 hours ago

From README:

> On the 128 GB M5 Max, clean end-to-end image+audio and embedded-video+audio renders completed in 74.58 and 76.99 seconds respectively, each with about a 40.1 GB peak physical footprint and zero swaps.

Looks like it uses 40GB? So your 96GB mac setup should work fine i guess (Model itself is 33B)

vunderba 7 hours ago

This repo looks neat, but I hope they add some more clear benchmarks because that time (74.58s) is pretty meaningless given that the it/s (and total time) is highly dependent on mode (T2V vs I2V vs REF2V), resolution (0.4, 0.6mp, etc), duration (5-15 seconds), etc.

embedding-shape an hour ago

It shouldn't? Unless you're using BF16 for all weights (I'm using NVFP4 for the text encoder, otherwise everything BF16 (and audio F32)) you'll fit it all within 96GB VRAM, bugs non-with-standing :) I've been fitting this within 96GB VRAM without issues.

Arn_Thor an hour ago

*notwithstanding

Anyway, good input!

embedding-shape 37 minutes ago

pferdone an hour ago

you should have a look at https://github.com/deepbeepmeep/Wan2GP which is the goto tool for "gpu poor", although as people below already pointed out you should be fine with comfyui's standard setup aswell

embedding-shape an hour ago

First, I think they're not even talking about GPUs, this is macOS hardware so unified memory. Secondly, if they were talking about GPUs, then 96GB VRAM is hardly what people refer to when they say "gpu poor".

aaqaishtyaq 36 minutes ago

Anyone tried it with M4 Pro, 48GB of memory?

iamyatin an hour ago

Noob question to all, is there any open source coding model that I can run on Mac mini 16gb?

jckahn an hour ago

Qwen 9B

c0rruptbytes 8 hours ago

wow antirez does not sleep

punnerud 5 hours ago

Wow, had to check some of his other repos; his the one behind dump1090

matheusmoreira 4 hours ago

He's the one who wrote Kilo too!

v-mdev an hour ago

mlmonkey 7 hours ago

Understatement of the year :-D

behnamoh 7 hours ago

when you have enough money to not have to worry about anything, you can go back to your hobbies. in this case, his hobby is programming.

azinman2 6 hours ago

Being a world class talent is independent of financial situation.

afavour 4 hours ago

moshun 3 hours ago

menaerus an hour ago

behnamoh 6 hours ago

stressback 7 hours ago

"enough money not to worry about anything" haha

myshapeprotocol 2 hours ago

Native inference optimization for Apple Silicon is such a game-changer for local-first workflows. Incredible performance work.

abhinai 8 hours ago

How similar are Jeff Dean and Salvatore Sanfilippo?

onionisafruit 8 hours ago

My favorite Jeff Dean fact is that he’s also antirez. Which reminds me of my favorite Salvatore Sanfilippo fact. He’s also Jeff Dean

hnfong 3 hours ago

The claims seem fitting from a user named "onionisafruit"...

tomcam 7 hours ago

I'm totally following this

ralusek 7 hours ago

Is that what the identity function is?

silisili 5 hours ago

robotresearcher 5 hours ago

People are really good at stuff.

I noticed on a bar TV the other day that some of the Chromecast screensaver landscape photo credits were to Peter Norvig. They were really lovely pictures.

SwellJoe 3 hours ago

I've run into Peter Norvig twice. Once at a YC event; the other when I parked my motorhome in front of his house in Palo Alto for a couple of days while visiting a friend who happened to live on the same street (not on purpose, I didn't know it was his house, it was just where I found sufficient open street parking for a huge motorhome, big houses with fewer cars on the street than on my friend's block). I ran into him while walking my dog, he asked about the motorhome and we talked travel. He was lovely both times. Not everyone is nice about a big motorhome parking on their block, especially in California, but he was friendly.

tipiirai 6 hours ago

I'd love to know what the alternatives are and how this is better

v-mdev an hour ago

I totally agree with you.

luciana1u an hour ago

neat — now I just need a machine with the memory bandwidth to render the three-second clip of my cat before the cat itself forgets what happened

yieldcrv 4 hours ago

Alright I’ve been afraid to ask but have been having trouble finding

What are some adult entertainment workflows in comfyui, I need best loras, best prompts to start with

and the communities, are they on telegram or something?

antirez an hour ago

H3 is quite uncensored, but was not trained on p0rn, so it has no anatomy clues needed to generate that kind of stuff. For softer adult content it is reported to be fine on Reddit.

bahmboo 3 hours ago

This is a healthy question. We want to use these tools for regular old human needs and desires.

Havoc 3 hours ago

There will Reddit subs for it though couldn’t tell you which off top of my head

I’d personally steer clear of messaging platforms for this - who knows what one might stumble into there

embedding-shape an hour ago

> I’d personally steer clear of messaging platforms for this - who knows what one might stumble into there

Personally I have no interest, but sometime browse stuff out of curiosity. But this got more of my curiosity, what kind of "stuff" are you implying they might stumble upon on the open, public internet? Sure, some NSFW, horror and otherwise weird stuff is there, especially around AI generation, but hardly something that will leave you traumatized, unless I misunderstand what you're implying?

Havoc 8 minutes ago

pferdone an hour ago

a friend told me there's a reddit called: unstable diffusion