Performance

The time the relay adds.

We measure the time Renio adds: the difference between going through the relay and calling the provider directly, under the same load.

These historical local-loop measurements do not measure the current durable journal or total-budget path. These add persistent writes and database access before the provider call. Actual latency depends on the region and budget mode.

The measurement

~0.5 msmedian overhead, one request at a time.

This is a local-loop measurement: relay and mock provider on the same machine, where network latency is negligible. A measurement on deployed European infrastructure will be published at production launch. The product target remains < 15 ms P95.

Standard request (non-streaming)

End-to-end latency of a complete request. At low load, overhead is under a millisecond; it increases with concurrency due to scheduling and copying streams.

Non-streaming request latency in local tests, milliseconds
Concurrent requests	Direct (median)	Via Renio (median)	Median overhead	P95 overhead
1	2.45 ms	2.91 ms	0.46 ms	0.91 ms
16	2.94 ms	5.69 ms	2.75 ms	5.40 ms
64	3.27 ms	14.39 ms	11.12 ms	18.23 ms

Streaming (time to first token)

The delay before the first token — the metric that determines whether a response feels immediate. Each chunk is forwarded without buffering.

Streaming request latency in local tests, milliseconds
Concurrent requests	Direct (median)	Via Renio (median)	First-token overhead	P95 overhead
1	2.49 ms	2.90 ms	0.41 ms	0.74 ms
16	2.94 ms	3.05 ms	0.11 ms	1.58 ms
64	2.68 ms	4.70 ms	2.02 ms	4.01 ms

Methodology

Overhead = latency through the proxy minus latency directly to a mock provider, at exactly the same load.
3,000 measured requests per point (+ 300 warm-up requests), upstream HTTP/2, Go 1.25.
The mock provider responds after 2 ms, with 1 ms between streaming chunks — isolating the relay’s cost, not the model’s.
Proxy and mock provider run on the same machine. A dated measurement on deployed EU infrastructure will replace these local results.

source: infra/bench · reproducible results in the repository