STREAMING FEATURE STORE

Exactly-once feature pipelines feeding online inference, with point-in-time correctness and a columnar hot cache that keeps reads in single-digit microseconds.

The problem

Training/serving skew is a silent model killer: features computed one way offline and another way online quietly degrade accuracy. And online reads have to be fast enough to sit in the inference path without becoming the bottleneck.

Approach

• One transformation definition, executed by Flink for both backfill and streaming.
• Point-in-time joins so training never sees a feature value from the future.
• A columnar hot cache (DuckDB + Redis) for microsecond online reads.

Results

Sub-200ms feature freshness, 8µs online reads, and point-in-time-exact correctness — training and serving finally agreed.