Part 1. TANREN — Atari Pong 21–0, and the Kill-Shot the Loop Found on Its Own

This is Part 1 of the TANREN series. The project overview — what the forge is, and the full record — is here. This article goes deep on the first campaign: Atari Pong.

TANREN in 30 Seconds

TANREN (鍛錬) forges a task-specific algorithm by having a cheap, frozen LLM write thousands of candidate solutions as code, scoring each with a deterministic verifier, and keeping the best to seed the next round. No model weights are trained — what improves is a readable Python function. It applies only where the scoring rule can be written as a deterministic program. Pong is exactly such a case: the score is the point difference, and it is unambiguous.

The Setup: 128 Bytes and No Labels

The target is Atari Pong through the Arcade Learning Environment (ALE). The policy does not see pixels. Its only input is the 128 bytes of the console’s raw RAM — and nothing tells it which byte is the ball, which is the paddle, or what any value means.

So the forge has two problems to solve at once, from the scoring signal alone:

1. Reverse-engineer the game — figure out what the bytes mean.
2. Play it perfectly — turn that understanding into a winning policy.

The gene is a single Python function, act(ram, prev, t, mem) -> action. The verifier runs the game deterministically per seed and scores the point difference, averaged over seeds. Train seeds and holdout seeds are kept disjoint, so a policy that merely memorizes one trajectory cannot pass.

What Evolution Discovered on Its Own

Analyzing the final champion, the evolved code had — with no supervision — pinned down the meaning of four RAM bytes:

From ram[49] and ram[53] — the ball’s x now and one frame ago — the policy estimates the ball’s velocity with a moving average, predicts where the ball will arrive, and moves the paddle there to wait, rather than chasing it. It even learned to control the timing of its own serve.

flowchart TB
    A["128-byte RAM<br/>(no labels)"] --> B["Identify ball x/y<br/>and own paddle<br/>ram49 ram50 ram51 ram53"]
    B --> C["Estimate ball velocity<br/>(moving average of x)"]
    C --> D["Predict arrival point<br/>and wait there"]
    D --> E["Control serve timing"]
    E --> F["Kill-shot:<br/>end every point in one return"]

None of these stages was specified. Each emerged because it scored better than what came before.

The Kill-Shot

Here is the part that is more interesting than the score. Taking the perfect champion apart shows a striking regularity: every point runs on the same 94-step rhythm, and the policy makes exactly 1.0 of its own returns per point.

In other words, the evolved code does not win by rallying well. It found a serve angle that the built-in opponent AI physically cannot reach, and ends every point with a single shot. It is the same category of result as DeepMind’s Breakout “tunneling”: given only a verifier, the loop uncovered an exploit of the environment that no one described to it.

The Result, Honestly

21 is a saturated ruler — a perfect game. So the honest claim is not “TANREN beat MuZero.” It is: TANREN tied the ceiling, and this particular target did not need the scale that reached it before. A human tester scores 9.3 against this same built-in opponent; the 2015 DQN reached 18.9 after training on 200 million frames; MuZero reached the ceiling with large-scale compute. The evolved policy reached it with readable code and a few dollars of API cost.

The 40-game holdout matters: 30 of those games start from positions the loop never saw during evolution, and it still went 21–0. That is the evidence that it found a strategy, not memorized a trajectory.

Why a Few Dollars Was Enough — the Diagnostic Loop

Run naively, the score stalls around 16. Getting past that was not about spending more; it was about a diagnostic loop:

flowchart LR
    M["Measure<br/>the exact frame<br/>of each lost point"] --> I["Interpret once<br/>(a strong model states<br/>measured facts only)"]
    I --> J["Inject facts<br/>into the mutation prompt<br/>(no fixes, no guesses)"]
    J --> P["Mass-produce<br/>variants using those facts"]
    P --> S["Verifier selects<br/>the one that clicks"]

The important control here is that the strong model is only allowed to state what was measured — “own paddle sits on the high-x side,” and so on — never to prescribe a fix. The cheap LLM then writes many variations that use those facts, and the verifier picks the winner.

One result made the value of this division concrete: from the same measured facts, a fix hand-written by a human (me) — one that was “correct in principle” — collapsed to −15, while evolution reached 21.0. Knowledge narrows the search cone; volume fills the cone; the verifier picks the right point. Remove any one of the three and it falls apart.

Honest Limits

Stated plainly, so the result is not read as more than it is:

• The input is RAM, not pixels, and this is a per-game policy — a separate one is forged for each game.
• The opponent is the standard built-in AI (the one a human pro scores 9.3 against). Rallies are short precisely because every return is a winner; that measured value (1.0 returns per point) is reported, not hidden.
• 21 is a saturated ceiling. “Tied,” not “beat.”
• The model’s weights are unchanged — no quality risk, and the policy can be reverted in one line.

The Code Is Public

The champion from this campaign — the evolved act() function that plays 21–0 from 128 bytes of RAM (97 lines excluding blanks) — is public on GitHub together with the ALE verifier and a runner: matu79go/tanren-champions. With nothing more than pip install ale-py, you can replay all 40 shutouts — the 10 training starts and the 30 unseen holdout starts — with run_pong.py on your own machine.

Cost

The full Pong campaign came to roughly $5 of API cost — about $0.5–0.8 per run, mutator = gemini-2.5-flash-lite, scoring on pure-Python CPU with no GPU training. The results it ties were trained on 200 million frames (DQN’15) or large-scale distributed compute (MuZero). For a problem whose value lives in a structural discovery rather than in millions of reflexes, that entire difference turned out to be unnecessary.

Next in the series: a cache-eviction policy that beat 20-year-old classics on a real trace. Back to the project overview.