Huawei releases MACaQ, its internal quantizer for local inference (github.com)
119 points by thefox96 16 hours ago
lukasc-ch 13 hours ago
People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh
Schlagbohrer 2 hours ago
The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
Rabbit504030201 15 hours ago
https://huawei-csl.com/blog/macaq/
Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.
nout 15 hours ago
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
gt0 11 hours ago
This is pretty much the reason that if I read about a new product, I generally just go to the Wikipedia page. I just want the no-bullshit version, no "Say hello to blah" video or marketing word salad.
andsoitis 9 hours ago
In this particular case I think you'd be pleased with it https://huawei-csl.com/blog/macaq/
It is from their lab in Switzerland, after all.
lukasc-ch 13 hours ago
Looks like many of their repos there have been around for a while. Also not sure why we would sanction free stuff from them
skinfaxi 11 hours ago
Well I'm not a lawyer but are you allowed to provide material support to sanctioned entities?
sroussey 10 hours ago
mocchibird 15 hours ago
Did they release any model? Seems good for Strata
minraws 9 hours ago
Qwen 3.8 27B has a macaq quant available
utternerd 16 hours ago
What a project name.
QuantumNomad_ 16 hours ago
Probably intended to reference macaque monkeys.
keyle 15 hours ago
I forgot my French, but isn't it incredibly rude?
PorciiVorbesc 14 hours ago
IsTom 14 hours ago
0xb0565e486 15 hours ago
xD
adrian_b 15 hours ago
I do not think that macaques would mind.
ebx 15 hours ago
Just enough plausible deniability :P