Cache and architecture for a sports betting API
Work by a team member before SIMYL Research; the provenance is at the end of the entry.
Problem
A real-time sports betting API received many repeated requests that recomputed prices at the origin. Under traffic spikes, the thundering-herd pattern multiplied load and latency.
Intervention
An in-memory cache with a single-flight protocol (one computation per key while the other threads wait for the result), hash-based routing across multiple gateways so that requests for the same key reach the same node, and a second shared cache layer. The concurrency protocol was designed in C++ in 1998; its Go version, gateway_cache, was published in 2020 and has been in production since then on the sports data platform of Genius Sports, the company where the responsible member worked and which publishes the code in geniussportsgroup/gateway_cache. The C++20 reimplementation, cpp-cache, is open source; its 2026 measurements are in Benchmarks.
Recorded results
In 2023 tests under controlled conditions, with the Go implementation:
- Up to 100,000 requests per second with p99 below 500 ms.
- Over 95% first-layer hits; about 0.25% of traffic reached the origin.
- Approximate latencies of 50 µs in the first layer and 500 µs in the second, hardware-dependent.
Limits
The figures come from load tests, not real production traffic, and the full description of the historical hardware and configuration is not preserved. They are not extrapolated to cpp-cache. The procedure we measure with today is in How we measure. The architecture and lessons are documented in 100K requests per second and When a cache does more than cache; the code is in gateway_cache and cpp-cache.
Provenance
Prior work by Leandro Rabindranath León, member of SIMYL Research, carried out at Genius Sports, where he worked: protocol and architecture design. The protocol is published as open source: gateway_cache (Go, 2020) and cpp-cache (C++20). The figures correspond to the Go implementation and the 2023 tests.