garagelm$ competitions
Exploratory research, run in arenas the lab doesn't control. Externally sponsored competitions put the lab's methodology in front of a live field and a scoreboard nobody can argue with: the kind of stress test no in-house benchmark provides. Each campaign gets a compact account here, win or lose, and what it taught feeds back into the milestone work.
| when | arena | agent | live rating | status |
|---|---|---|---|---|
| 2026-06 → 08 | pokémon tcg ai battle · kaggle, simulation track | mlp behavior cloning + meta decks | 600 → 1022.9 | final scoring |
A deck swap and two data bugs: 600 → 1023 on the Pokémon ladder
The campaign took a random-play template to a team-best live rating of 1022.9 on a ladder where every submission enters at 600, playing full Pokémon TCG under imperfect information: hidden hands, hidden decks, coin flips. The agent that held is almost embarrassingly plain. No search at play time, no RL, no transformer: a feed-forward network behavior-cloned on ~1M decisions mined from top-band ladder replays, flying the meta's best decks.
The three largest gains were a deck swap and two data-pipeline bug fixes. The swap (+~370 live) proved that deck-vs-field strength is invisible to self-play: mirrors cancel the deck. The bugs were an off-by-one label alignment that poisoned every offline imitation metric for a month, and a zeroed feature that left the model card-blind on its most common decision type until the final week. Both hid the same way: trained and validated on the same wrong data, offline accuracy rose (0.51 → 0.57) while live play collapsed (34% → 13%). Only numbers measured by playing games survived the audit.
Nearly every modeling lever gated null, each closed with a reason: value functions capped at ~0.75 accuracy across five architectures (the residual is unseen future draws, not model capacity), RL flat in seven configurations, deeper search self-cancelling in paired tests, specialist fine-tunes passing the mirror and failing the field, rules exploits nonexistent (6 draws in 14,821 games). The winner of the whole competition ran a structural counter-deck; a rival with a full BC → PPO → MCTS pipeline placed 552nd. Strategy beat sophistication at every rating band.
What transfers: the ground truth is the game, not the metric. Audit labels and features before touching architecture. Know what each proxy is structurally unable to see. And engineer the variance before trusting any delta (byte-identical submissions diverged σ≈48 live). It is the lesson milestone 13 taught at home, loss and capability coming apart, learned again at tournament scale.