Hacker News

LittleBit: Ultra Low-Bit Quantization via Latent Factorization

4 points by kakadu ago | 1 comments

stev1983 [-]

I think the difficult aspect is that you have to re-train the model after conversion. I assume you could use this during training (QAT) similar to ternary quantization.

It’s a bit beyond me, but it seems promising–at least for general knowledge models.