Hacker News
LittleBit: Ultra Low-Bit Quantization via Latent Factorization
4 points by kakadu
ago
|
1 comments
stev1983
[-]
I think the difficult aspect is that you have to re-train the model after conversion.
I assume you could use this during training (QAT) similar to ternary quantization.
It’s a bit beyond me, but it seems promising–at least for general knowledge models.