This started with a fairly reasonable request: could we make a modern version of the old Windows Space Cadet pinball game?
Then the ball got stuck. Then it kept disappearing down the sides. Then the effects were too much. Then, once those things were improving, I realised the table was still a bit boring.
So I kept pushing it.
Somewhere along the way it became Agent Pinball: a browser game with an agentic AI theme, a fork bomb that releases more balls, and a small model learning to work the flippers. It is now playable here on the site. You can play it yourself or watch the AI have a go.
The scope has expanded slightly.
First, make it worth playing
I have been building this with Codex, steering the work through a lot of play, feedback, fixes, and another go. The useful feedback was usually quite blunt. The table needed more going on. The flippers felt hectic. The layout had too much text. Holding the launch button should actually pull the plunger back.
The agent theme gave it a direction of its own. Tasks, forks, and swarms became things the table could do. A bonus could turn one ball into a much less manageable situation.
That was fun. Triggering it too easily was less fun. If every moment is chaos, the big moment stops being a big moment.
There has been a lot of that sort of iteration: better returns from the sides, calmer effects, a simpler arcade layout, and trying to make the exciting bits feel earned. It is still a work in progress, but it has become something I want to keep playing with.
Then, teach something to play it
The next question was whether we could train a model locally to play the game. And, if we could, whether we could keep the footage and measurements as it learned.
That turned into a second project inside the first one.
The coding assistant and the player are separate things. Codex helps build the game and the training setup. The player is a small reinforcement learning model, trained with PPO. It gets 76 numbers describing the game state and chooses between four controls: left flipper, right flipper, both, or neither.
It makes that choice 30 times a second. Launching is handled by a script; it has not learned nudging. It reads the game state directly, so this is not a model watching the screen the way a person would.
Training runs the same physics without drawing the table. The model plays, gets feedback, and adjusts. Saved versions let us go back and see what changed.
I asked about using my RTX 4070 Ti. The actual machine has a 4070 Ti SUPER, but the CPU was faster in our small benchmark. The network is tiny, and moving work to the GPU did not pay off for this setup. A useful reminder to measure the thing in front of you.
Better scores, with some fairly large asterisks
The current player is at just under 10 million training decisions along its training path. That is decisions, not games, and it does not include all the work spent on alternative branches.
The most recent successful experiment added a training reward for completing ramp and portal shots with a single ball in play. We picked a candidate on one set of games, froze it, then compared it with the previous player on 128 fresh, matched games.
| Score | Previous player | Current player |
|---|---|---|
| Average | 9.50 million | 14.68 million |
| Median | 6.02 million | 10.55 million |
That is about 54% higher average score and 75% higher median score. The new player won 76 of the 128 comparisons, with one tie.
Those tests used normal game points, with the extra training rewards switched off. Each game ended after three lost balls or two simulated minutes. The individual scores are here.
That is enough evidence to keep this version. It is not enough to call it an expert. These are two fixed models on one table, not a human tournament. The new player still scored under a million in 39 of those 128 games. A good average can hide some fairly terrible afternoons.
The failed runs are part of the story
One recurring problem is the flippers. The model can be busy in a way that looks ridiculous.
Making movement more expensive sometimes produced a quieter player that was worse at pinball. Rewarding something easy to measure does not guarantee you get the behaviour you actually wanted.
Even more training is not automatically progress. The latest follow-up added another 3.15 million decisions across three branches. All three failed the score-consistency requirement, so the existing champion stayed installed.
I want to keep those results. We now have 113 saved replay checkpoints and 17 raw video takes, alongside the charts and experiment records. The bigger video should show the failures and the changes of direction as well as the good runs. Otherwise it would be very easy to edit this into a much neater story than it was.
Small enough to live in the browser
The exported player is about 190 KB and runs locally in JavaScript. Watching it play does not make an AI API call for every flipper movement. The site serves the game and the model, and your browser does the work.
That makes it a fairly practical thing to put online. There is normal hosting and bandwidth involved, but no inference server sitting there playing every visitor’s game.
I would still like it to become very hard to beat. The next useful work is making the bad games less bad, getting more deliberate flipper control, and measuring it against people. Table changes will also need fresh checks; a player trained on one layout should not be assumed to understand another.
For now, have a go at Agent Pinball. If you find a repeatable way to embarrass the AI, tell me about it. That would be useful data, and probably good footage.
Video text alternative
The reel opens with the shot-learning experiment: three branches trained with different rewards. It shows a saved replay of the previous, 7.86-million-decision player, followed by a diagram of the ball state and the probabilities of its four flipper controls.
The focused, 9.96-million-decision candidate appears next. Its selection results were 14.73 million average points and 11.85 million median points. A chart compares the selection branches. Another recorded state diagram shows the candidate's control probabilities; these are model outputs, not an explanation of its intentions.
The final chart uses the separate 128-game confirmation set: 14.68 million average and 10.55 million median for the candidate, against 9.50 million and 6.02 million for the previous player. The candidate won 76 comparisons and passed the recorded requirements. The closing card notes that human difficulty remains unmeasured. There is no audio.