Caching

2 articles tagged with "Caching"

Part 2. TANREN — 11 Wins to 1 Against Twenty Years of Cache Classics, on the Future of a Production Trace Series

Part 2. TANREN — 11 Wins to 1 Against Twenty Years of Cache Classics, on the Future of a Production Trace

Cache eviction — deciding what to drop when a cache is full — has been studied for more than twenty years. On that ground, a 53-line evolved Python function beats ARC, LIRS, and W-TinyLFU on future segments of Twitter's production traces. This post covers how the first attempt lost by overfitting to one time period, the two-window training that fixed it, and the public code anyone can rerun.

Read →
Part 3. TANREN — Patching the Evolved Cache Policy into a Real vLLM and Measuring It Series

Part 3. TANREN — Patching the Evolved Cache Policy into a Real vLLM and Measuring It

The cache policy that won in simulation is patched into a live vLLM and measured on real hardware. The simulated win vanished at first; after fixing where and what to measure, tail latency (p99 TTFT) came out 13-16% lower under memory pressure. The prototype's costs and the conditions where it does not help are reported as measured.

Read →