4 articles tagged “Quantization”.
A correction to what I wrote about TurboQuant in May. A new paper from Mariano Tepper and Ted Willke makes the case that the rotation in front of a scalar quantiser should be learned from your data, and that learning it is cheap enough that I no longer think calibration cost belongs in the argument.
Ben Trent and Thomas Veasey shipped another DiskBBQ optimisation today: quantising queries against coarser parent centroids instead of per-document ones. 5x off the quantisation stack with no meaningful recall loss. The insight underneath is that the query path and the document path don’t have to be treated symmetrically.
TurboQuant landed as a KV cache result, but the more interesting application might be ColBERT-style late interaction. Here’s the case, and the open questions.
A new quantisation method out of Google Research is making the rounds. Qdrant shipped it. Elastic ran the benchmarks and politely declined. Both responses tell you something useful.
