Hacker News

Huawei releases MACaQ, its internal quantizer for local inference

135 points by thefox96 ago | 33 comments

lukasc-ch |next [-]

People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh

alfiedotwtf |root |parent |next [-]

Sorry, I've deleted my Reddit account since they locked out users. Could you please post links to the GGUFs here :)

adrian_b |root |parent |previous [-]

The results shown there are very good, much better than for the previously widely used quantizers.

Nice for Huawei to share it.

Rabbit504030201 |next |previous [-]

https://huawei-csl.com/blog/macaq/

Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.

nout |root |parent |next [-]

I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.

gt0 |root |parent |next [-]

This is pretty much the reason that if I read about a new product, I generally just go to the Wikipedia page. I just want the no-bullshit version, no "Say hello to blah" video or marketing word salad.

andsoitis |root |parent |previous [-]

In this particular case I think you'd be pleased with it https://huawei-csl.com/blog/macaq/

It is from their lab in Switzerland, after all.

lukasc-ch |root |parent |next |previous [-]

Looks like many of their repos there have been around for a while. Also not sure why we would sanction free stuff from them

skinfaxi |root |parent [-]

Well I'm not a lawyer but are you allowed to provide material support to sanctioned entities?

sroussey |root |parent [-]

It’s a commerce department sanction not a treasury department sanction (which would be far more serious).

skinfaxi |root |parent [-]

I thought Huawei was on the ofac list.

xpil11 |root |parent |previous [-]

[dead]

Schlagbohrer |next |previous [-]

The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.

xpil11 |root |parent [-]

SINQ was created last year. It was for weights only and it was calibration-free quantization. Then KVarN is for KV-cache quantization to get longer context. MACaQ is multimodal, activation and weights calibrated quantization. 3 different projects

mocchibird |next |previous [-]

Did they release any model? Seems good for Strata

minraws |root |parent [-]

Qwen 3.8 27B has a macaq quant available

speedupagi |next |previous [-]

[flagged]

p-s- |next |previous [-]

[dead]

|next |previous [-]

thefox96 |next |previous [-]

[dead]

utternerd |previous [-]

[flagged]

QuantumNomad_ |root |parent |next [-]

Probably intended to reference macaque monkeys.

keyle |root |parent |next [-]

I forgot my French, but isn't it incredibly rude?

PorciiVorbesc |root |parent |next [-]

Why?

dmonitor |root |parent [-]

I think the immature commenters are referring to the pronunciation being similar to "my cock"

PorciiVorbesc |root |parent |next [-]

Aaa, thanks I didn't think of that.

arcanemachiner |root |parent [-]

Congratulations on not being a total degenerate.

stdbrouw |root |parent |next |previous [-]

It's also a racial slur in certain languages.

yonatan8070 |root |parent |next [-]

It's also "Roach" in Hebrew (מקק)

onemoresoop |root |parent |previous [-]

What are those?

|root |parent [-]

keyle |root |parent |previous [-]

No, no, not that.

IsTom |root |parent |previous [-]

I'd assume the French know something about naming projects in the vein of Coq.

thenthenthen |root |parent [-]

Coq au vin?

xpil11 |root |parent |previous [-]

[dead]

0xb0565e486 |root |parent |next |previous [-]

xD

wolrah |root |parent |next |previous [-]

First thing my mind went to.

"Huawei releases MACaQ"

"Just like Jensen released his..."

adrian_b |root |parent |next |previous [-]

I do not think that macaques would mind.

ebx |root |parent |previous [-]

Just enough plausible deniability :P

|root |parent [-]