SUB-MS VECTOR SEARCH ENGINE
A from-scratch ANN engine with an HNSW index, SIMD distance kernels, and a lock-free query path. Serves nearest-neighbour lookups over 50M embeddings without breaking a millisecond at p99.
A nearest-neighbour search engine built from the ground up in Rust, designed around a single goal: never let a similarity lookup cross a millisecond at the 99th percentile, even over tens of millions of vectors.
The problem
Off-the-shelf vector databases were comfortable at the mean but unpredictable at the tail. Under concurrent load, GC pauses and lock contention pushed p99 into the tens of milliseconds — unacceptable for an online ranking path sitting in the request critical section.
Approach
- An HNSW graph index with a memory layout tuned for cache locality on the search path.
- SIMD distance kernels (AVX2/AVX-512) for batched cosine + dot-product.
- A lock-free read path so queries never block behind a writer.
- Tokio for async I/O, with query work pinned off the accept threads.
Results
0.82ms p99 over 50M embeddings at 1.2M queries/sec on a single node, holding 0.991 recall@10. Tail latency stayed flat as concurrency climbed — the whole point.