TurboQuant Memory#

Compress embedding vectors 5-8x with 98%+ search accuracy using TurboQuant (Google, ICLR 2026).

Quick Start#

1. Run tests#

bash

python3 scripts/turboquant.py

15 built-in tests: FWHT correctness, MSE distortion, IP correlation, recall, compression ratio, determinism.

2. Validate on your data#

bash

python3 scripts/validate.py --db /path/to/memory.sqlite --auto-detect --bits 5

Auto-detects sqlite-vec vec0 tables, analyzes distribution, reports quantization quality and recall.

3. Quantize a memory database#

bash

python3 scripts/memory_quantize.py --db /path/to/memory.db --bits 5 --benchmark
python3 scripts/memory_quantize.py --db /path/to/memory.db --bits 5 --migrate

4. Integrate into code#

python

from turboquant import TurboQuantMSE

# Initialize (deterministic — same seed = same quantization)
tq = TurboQuantMSE(dim=3072, bits=5)

# Quantize for storage
stored = tq.quantize(embedding_vector)  # float32 → compressed

# Reconstruct
reconstructed = tq.dequantize(stored)   # compressed → float32

# Search: query stays float32, database is quantized
q_rot = tq.rotation.apply(query)
for doc in database:
    score = doc['norm'] * doc['scale'] * np.dot(q_rot, tq.codebook[doc['indices']])

Recommended Configuration#

PresetModeBitsR@1CompressionUse CaseDefaultMSE5****98%****6.4xMost memory/RAG searchConservativeMSE698%+5.3xHigh-fidelity retrievalAggressiveMSE492%8.0xLarge-scale, storage-constrained

Parameters#

ParameterDefaultDescriptiondimauto-detectEmbedding dimension (768, 1536, 3072, etc.)bits5Bits per coordinate. See table above.seed42Rotation seed. Same seed = reproducible quantization.

Algorithm#

Blockwise Hadamard Rotation → Lloyd-Max Scalar Quantization

1. Split vector into power-of-2 blocks (e.g., 3072 = 3 × 1024)
2. Per block: random sign flip + Fast Walsh-Hadamard Transform (fully invertible)
3. Per-vector scale normalization
4. Lloyd-Max optimal scalar quantizer per coordinate (precomputed codebook for N(0,1))
5. Pack indices into compact bit representation

Key properties:

• Data-oblivious: no training or calibration needed
• Fully invertible: zero information loss from rotation
• Near-optimal: within 2.7x of Shannon information-theoretic lower bound
• Deterministic: same seed = same output

See references/algorithm.md for full details.

Benchmark (Gemini embedding-001, 3072-dim, 112 vectors)#

BitsMSECosineR@1R@5R@10Bytes/vecCompression31.1e-50.98288%90%91%1,16010.6x43.2e-60.99592%93%93%1,5448.0x58.2e-70.999****98%****96%****96%1,9286.4x62.2e-71.00096%98%98%2,3125.3x78e-81.000100%98%99%2,6964.6x83e-81.00098%98%99%3,0804.0x

Compatibility#

• Python 3.9+, numpy only (no scipy, no GPU)
• Any embedding dimension ≥ 128
• Any embedding model (Gemini, OpenAI, Cohere, sentence-transformers, etc.)
• SQLite / sqlite-vec vec0 tables (auto-detected)

References#

• TurboQuant paper: arXiv:2504.19874 (ICLR 2026)
• PolarQuant paper: arXiv:2502.02617 (AISTATS 2026)