aisar-labs / turboquant-rs

What is turboquant-rs?

Turboquant-rs is a Rust crate implementing TurboQuant, a data-oblivious vector quantization method from Google Research (ICLR 2026), compressing high-dim vectors like LLM KV caches or embeddings to 1-4 bits per dimension without any training data. Developers get a simple API to instantiate a quantizer by dimension and bit width, then quantize Vec inputs, dequantize for reconstruction, or estimate inner products unbiasedly. It includes built-in benchmarks via cargo bench that reproduce the paper's distortion tables, integrated with rust github actions and rust github ci workflows.

Why is it gaining traction?

Unlike product quantization or GPTQ needing calibration data, TurboQuant works instantly on any vector via random rotation and precomputed codebooks, slashing KV cache memory 6x for longer contexts or more users per GPU. Early rust github trending interest stems from its clarity—f64 precision, 45 tests passing via cargo test—and relevance to edge AI, where on-device RAG fits in MBs. As a pure rust implementation language crate, it dependencies like nalgebra play nice in rust github dependency graphs and rust github actions cache for fast builds.

Who should use this?

AI researchers reproducing the 2026 Google paper's results on distortion or throughput. LLM serving engineers prototyping KV cache compression in rust github workflow pipelines. Vector DB devs testing instant 4-bit indexing for RAG without rust implementation is not general enough pitfalls in data-dependent methods.

Verdict

Grab it for experimentation—solid docs, tests, and cargo bench make it a credible rust github client for learning extreme compression, despite 1.0% credibility score and 12 stars signaling early research stage. Skip for production until SIMD optimizations and stability land.

(178 words)