← ALL PROJECTS
P-062025Lead Engineer
DISTRIBUTED RAG ORCHESTRATOR
A DAG-based orchestration layer for retrieval-augmented agents: typed tool calls, speculative retrieval, automatic eval gating, and full request tracing across every hop.
+37%
GROUNDED ANSWERS
-41%
TOKEN COST
100%
TRACE COVERAGE
An orchestration layer that treats a retrieval-augmented agent as a typed dataflow graph rather than a pile of prompt strings — so every hop is observable, testable, and individually optimisable.
The problem
Agent pipelines were opaque. When an answer went wrong, there was no way to tell which hop caused it, and no way to change one step without risking the rest. Cost and latency were both invisible until the bill arrived.
Approach
- A DAG of typed tool calls, each with explicit inputs/outputs and its own eval gate.
- Speculative retrieval that prefetches likely context while earlier hops run.
- Automatic eval gating: a step can't ship if it regresses its offline metric.
- OpenTelemetry tracing across every hop, so latency and token cost are attributable.
Results
Grounded-answer rate up 37%, token cost down 41%, and 100% trace coverage — debugging went from guesswork to reading a span.
/ STACK
PythonasynciopgvectorTemporalOTel