garagelm is an independent, open-source artificial intelligence research lab started in 2026, built on the hardware you'd find in a garage rather than a datacenter. The work is open source first: code, training recipes, write-ups, and weights are published wherever possible, and negative results ship alongside the wins.
lab objective 1: research
the metal · one apple m4 pro
Capability-per-dollar and per-watt: real models trained end-to-end on one consumer Mac, measured against published baselines under a fair-comparison rule, weights released. Written up win or lose.
The same ideas at readable scale: working models shrunk until every matrix fits on screen, probes on every number, permission to break things. Nothing canned; it all recomputes live.
est. 2026 · independent · open-source focused · no cluster · no CUDA · no tracking. just a browser tab and one Mac.
$ research
The core of the lab chases one question: how much intelligence can you train per dollar, per watt, per hour? Every experiment in attention variants, data quality, and training-cycle efficiency runs on a single Apple M4 Pro (48GB, no CUDA) under a strict fair-comparison rule, and gets written up win or lose.
headline result · hybrid-gpt-232m
vs pythia-160mat-or-above on all four tasks, at 300× fewer tokensvs gpt2clearly ahead on arc-easy; even elsewherekv cache~30% of full attentiondecode~310 tok/s fp16 · 530+ at 4-bit (mlx)
02baseline: 114M GQA/RoPE/SwiGLU control [control]03–04attention shootout: sliding-window "wins" twice [artifact]05data frontier: refined corpus flips it: hybrid beats full attention [win]07flagship: matches pythia-160m at 600× fewer tokens [600×]08solidify: hybrid > full attention in 3/3 seeds [3/3]09flagship-2: 1B tokens; the token axis saturates [saturating]10sft + release: chat tuning, zero regression, weights published [shipped]11efficiency: four levers, four gated negatives; the recipe stands [0-for-4]12post-training: alignment tax measured; dpo passes at zero tax; chat v2 ships [v2]12bbudget ablation: hybrid ahead at every budget; the edge decays as training grows [curve]13moe: 2.5× the params bought loss, not capability; both big arms decode slower [flat]14babylm: the data-scarcity test of the hybrid prior, prediction pinned, parked untrained [on hold]15harness: an open 9b model inside an inference harness vs dated frontier scores [active]
Exploratory research in arenas the lab doesn't control: externally sponsored competitions, where the methodology faces a live field and a scoreboard nobody can argue with. What each campaign teaches feeds back into the milestone work.
campaign 01 · pokémon tcg ai battle (kaggle, jun–aug 2026)
result600 → 1022.9 live team best on the skill ladderthe agentmlp behavior cloning on ~1m corrected replay decisionsthe lessona deck swap and two data bugs beat every modeling lever
A complete decoder-only transformer at d=4 · 2 heads · 188 parameters, all of them visible. Trace embeddings → attention → logits, hover any cell for its lineage, then step gradient descent and watch it overfit your sentence. Seven views: about, architecture, pipeline, math, residual stream, generate, training.
The second act of training, end to end on real numbers: sft, preference pairs, the dpo loss with an interactive gradient explorer, pre-registered gates, and the honest eval where "preferred" and "obedient" part ways.
[03]kv-cache visualizer(planned)
Full vs sliding-window vs hybrid cache growth: the research result above, animated.
[04]attention-head zoo(planned)
Every head's pattern across prompts: induction, positional, syntax.
garagelm