TL;DR
Rust-based turbovec implements Google's TurboQuant algorithm, compressing 31GB vector corpus to 4GB while delivering 3.4x faster search than FAISS with no training phase.
Key Points
- 10M document corpus: 31GB → 4GB at float32 with 2-4 bit quantization
- Search speed: 3.4x faster at 4-bit, 20-26% faster at 2-bit vs FAISS IndexPQFastScan across ARM and x86
- Online ingest: zero training, parameter tuning, or rebuilds; single add() at 6.3-19.7µs, 100-vector batch at 4.6-16.3µs/vector
- Incremental persistence: fsync-per-call, crash-safe, O(1) removes at 0.44-1.22µs vs FAISS's 0.19-1.02s per delete
Why It Matters
For teams building RAG systems, vector search infrastructure, or embedding-heavy applications, turbovec eliminates the memory/latency tradeoff that made FAISS dominant. Data-oblivious quantization (no corpus-dependent training) plus runtime filtering and drop-in framework integration (LangChain, LlamaIndex, Haystack) make this immediately deployable for privacy-critical, resource-constrained, or high-throughput workloads.
Source: github.com