My Laptop Played 3.5 Million Hands of Poker Against Itself. Here’s the Bot it Became.
Henry Anderson • July 2026 • 14 minute read
I wanted to know whether a machine could teach itself poker from absolutely nothing. No hand histories, no human advice, no “always raise pocket aces.” Just the rules and millions of hands against a copy of itself. Twelve and a half hours later I had a heads-up No-Limit Hold’em bot that bluffs, folds, and mixes its bet sizes like a person. This is the story of the three rebuilds it took to get there, and what the finished strategy actually does.
The whole project in one curve. “Regret” measures how much the bot wishes it had played differently. It spikes early as the bot discovers new ways to bet, then falls for three million hands straight as the strategy settles. By the end it has dropped to 0.076.
“Before we start... this is a hobby solver, not Libratus. It plays a deliberately simplified version of poker (more on that below), it’s trained at one stack size, and a sharp human can absolutely beat it. Nothing here is gambling advice, and the dollar amounts in the game are play money. Treat it as a window into how self-play learning works, not a system for beating anyone for real.”
Why I did this
I find it a little uncanny that a program can become good at a game nobody taught it. Modern poker AIs do not study how humans play. They sit in a room and play themselves, and a balanced, almost unbeatable strategy falls out the other end. I kept reading about that and not really believing it in my gut, so I decided the only way to believe it was to build one and watch the thing happen on my own machine.
There is a self-interested angle too. I want to work in quantitative finance, and the math under the hood here, equilibria, regret minimization, learning under uncertainty, is the same math that shows up in market making. Poker is the friendly, visual version of it. If I can explain why my bot bleeds chips to a player who never folds, I can explain a lot of other things.
The false start that taught me the most
My first poker bot was not this one. It was an eight-player bot that learned by Q-learning, the classic “try things, remember what paid off” method. It improved against the opponents I trained it on and then folded the moment a new style sat down. I rebuilt it as an eight-player CFR bot, switching to the algorithm the famous poker AIs use, and it still drifted and never settled.
The problem was not my code. It was the table. The convergence guarantee that makes these algorithms work only holds for two-player zero-sum games, where one player’s win is exactly the other’s loss. Put eight players in the pot and the math no longer promises anything: three opponents can quietly gang up on a fourth, and there is no single unbeatable strategy to converge to. I had spent weeks fighting a theorem.
So I threw the table out and rebuilt around heads-up, one-on-one poker. This is the setting where the algorithm is on solid ground, and not by coincidence it is the exact game the Libratus AI used to beat top human pros in 2017. Shrinking from eight seats to two also collapsed the number of distinct situations the bot has to learn by roughly six times, so every situation gets far more practice for the same training budget. The lesson stuck with me: picking the right problem did more than any clever code I wrote afterward.
What CFR actually does, in plain English
The algorithm is called Counterfactual Regret Minimization. The word that matters is regret. Every time the bot reaches a decision, it later asks a simple question: “Knowing how that hand played out, how much better off would I have been if I had always taken a different action right here?” If raising would have earned more than what it actually did, it racks up regret for not raising. The next time the same situation comes up, it leans a little harder toward the actions it regrets skipping.
Run that loop over millions of hands and something subtle happens. There is one twist that surprised me: the strategy the bot is playing at any given moment never stops wobbling. The thing that converges is the average of every strategy it has ever played. You let it thrash around, and you keep a running tally of the average, and that average settles onto the unbeatable strategy. The bot is noisy; its memory is calm.
“The one idea to hold onto... this bot is not trying to beat you. It is trying to be impossible to beat by anyone. Those are different goals, and the difference explains almost everything strange about how it plays.”
Three rebuilds
The bot you can play at the bottom is the third major version. Each rebuild fixed something the last one got wrong, and each one made the bot’s little world a bit richer.
1. v1 and the calling station
The first heads-up version technically worked and was deeply boring to play against. Trained on a short run, it had learned exactly one move: call. Almost every situation came back about ninety-nine percent “call,” the poker equivalent of a person who will not fold but will not bet either. It had not played enough hands to discover that raising and folding are good ideas. The fix was not cleverness, it was patience, but first the training had to get fast enough that patience was affordable.
2. v2 and the race to make training cheap
The thing eating all the time was not the poker logic. It was the bot estimating, hand after hand, “how strong am I right now?” by dealing out thousands of imaginary futures. About ninety percent of training was spent on that one calculation. Two changes attacked it. The first was a cache: a hand’s strength before the flop depends only on the two cards you hold, and there are just 169 distinct two-card hands, so I compute each once and reuse it forever. The second was parallelism, running one trainer on every processor core and merging what they learn each round.
Hands trained per second. The richer betting in v3 made each hand about three times more expensive to learn from, which is why parallelism stopped being a nice-to-have. Spreading the work across six cores took the bot from eight hands a second back up to nearly eighty.
. v3, the “feel like a real person” pass
v2 was strong but robotic. My goal for v3 was that playing it should feel like sitting across from a thoughtful human, so I widened the choices it could make. The biggest change was bet sizing. Earlier versions could only bet half the pot, the full pot, or shove everything in. A human does not jump from a half-pot bet straight to their whole stack, so I added a three-quarter-pot bet and let the bot mix all four. I also upgraded the learning rule to a faster variant called CFR+, and gave the bot a small but important piece of memory: whether it was the one doing the raising before the flop. That one bit lets it tell “I raised and I’m following through” apart from “I just called and now I’m taking a stab,” which keeps its story straight across a hand.
Two ways v3’s world got bigger: one more bet size, and twice as many possible situations to learn. The number of situations the bot can imagine doubled; the number it ever actually meets in real play turned out to be only about a thousand.
Not every idea survived. I tried adding a fifth, tiny one-third-pot bet, and training throughput collapsed by about five times. A small bet invites a small re-raise, which invites another, and the tree of possible betting sequences exploded. I cut it. Knowing when to remove an option is its own skill.
The fifth bet size that did not make it. Adding a one-third-pot option made betting sequences so long that the bot trained roughly five times slower for no real gain in strength.
What the finished bot does
After all that, here is the part I actually cared about. With the strategy fully trained, does it play like a person or like a robot? The clearest answer is in how it bets. When the trained bot decides to put money in, it does not have a favorite button. It spreads its aggression across all four sizes, leaning on the smaller bets a little more, the way a real player mixes a thin value bet here and a pot-sized threat there.
Every raise the trained bot makes, sorted by size. No single size dominates. This is the one chart that convinced me the “feel like a person” pass had worked: it is neither a calling station nor a maniac who only shoves.
You can see the same balance before the flop. With weak hands the bot mostly folds, more readily out of the big blind than on the button. As its hand improves it shifts smoothly into calling and then into raising, and crucially it raises with a mix of sizes rather than a tell-tale single bet. Strong hands are not always jammed all-in, which is exactly what keeps a good player hard to read.
Before the flop, by hand strength and seat. Weak hands on the left are folded; strong hands on the right are played aggressively but with varied sizing. The bot is noticeably more willing to put chips in from the big blind, where it has already posted money in the pot.
And now the result that delighted me, because it looks at first like a failure. I tested the bot against three simple opponents as it trained: one that plays at random, one that only ever pushes or folds, and one that calls every single bet. It crushes the random player and edges out the push-or-fold player. But against the player who calls everything, the finished bot actually loses chips.
Chips won per hand against three opponents over the full run. The bot beats the random and push-or-fold players comfortably. Against a pure calling station (red) it slides into the red and stays there, and that is not a bug.
Here is why that is the most honest chart in the post. A player who never folds cannot be bluffed. The right way to beat them is to stop bluffing entirely and only bet your good hands. But my bot is not built to beat one specific person; it is built to be unexploitable against everyone, which means it keeps bluffing at a balanced rate. The calling station simply calls those bluffs and pockets them. The bot is not playing badly. It is playing a different game than the one in front of it, on purpose. That is the “unexploitable, not exploitative” idea from earlier, showing up as a real and slightly painful number.
Going deeper
Everything above is true but soft. Here is the machinery, for anyone who wants it.
Regret matching
At each decision the bot keeps a running total of regret for every action. To turn those totals into an actual strategy, it plays each action in proportion to its positive regret, and ignores actions whose regret is negative.
strategy[I][a] = max(0, R[I][a]) / Σ max(0, R[I][a]) )
If every regret is zero or below, it just plays uniformly at random. The version I use, CFR+, adds two refinements. It floors regret at zero after every update, so an action that has been bad for a long time can rejoin the strategy the instant it starts looking good again instead of climbing out of a deep negative hole. And it weights later, better-converged hands more heavily when forming that all-important average. These two lines are the difference between a bot that needs tens of millions of hands and one that is strong by a few hundred thousand.
You cannot walk the whole tree
Heads-up Hold’em has somewhere around a hundred trillion decision points. You cannot visit them all every hand, so I use external-sampling Monte Carlo CFR. At the bot’s own decisions it examines every legal action, because it needs each one’s value to compute regret. At the opponent’s decisions it samples just one action. At the deck it deals one random outcome. Each “hand” of training is therefore one random deal, walked once from each player’s point of view. It is unbiased over many hands and astronomically cheaper than the full tree.
The abstraction, which is the real trick
The honest reason any of this fits on a laptop is that the bot does not solve real poker. It solves a blurry compression of it. Every situation gets boiled down to a short key built from six features:'
Multiply those out and there are 3,200 possible buckets. Hand strength is the interesting one: rather than track your exact two cards plus the board, the bot estimates the probability that hand wins against a random holding and files it into one of ten strength bins. Two genuinely different hands with the same chance of winning are treated as identical. That is what shrinks a hundred-trillion-point game down to something a laptop can actually grind through.
| Feature | What It Represents | Bins |
|---|---|---|
| Hand strength | Win probability vs. a random hand (Monte Carlo simulation) | 10 |
| Street | Preflop, flop, turn, or river | 4 |
| Pot odds | Cost of calling relative to the pot | 5 |
| Bet size | Current bet size relative to the pot | 4 |
| Position | Button or big blind | 2 |
| Preflop aggressor | Whether you raised before the flop | 2 |
The action side is compressed the same way. A real No-Limit game lets you bet any amount; my bot picks from six moves, fold, call, raise half-pot, raise three-quarter-pot, raise pot, or all-in, with a cap of three raises per street so the betting cannot spiral. In the playable game below you get a full slider and can bet anything you like, while the bot sticks to its trained sizes. That asymmetry is exactly how the professional solvers work: they have their sizes, you do whatever you want.
The overnight run
The publishable model came from a single long run: 3.5 million hands, six cores, launched in the evening and finished by morning. Before committing twelve hours of compute I ran a small adversarial audit of the whole pipeline to make sure nothing would crash at hour ten and lose the lot. The training stayed steady throughout.
Hands per second across the full run. No slow leak, no thermal cliff, just a flat line for twelve hours. Boring, in the way you want a long compute job to be boring.
One number on that run says a lot. Of the 3,200 buckets the bot could in principle visit, only about a thousand ever came up in real play. Many combinations simply never co-occur. That means each situation the bot does meet got roughly 3,500 hands of practice, which is why the strategy is as sharp as it is despite the run sounding modest next to an industrial solver.
How many distinct situations the bot had encountered, over the run. It saturates almost immediately near a thousand and never approaches the 3,200 it could theoretically reach. The bot learns the poker that actually happens, not the poker that could.
Where it falls short
The compression is coarse. Filing hands into ten strength bins throws away real information. Two hands that win equally often can play very differently, and my bot cannot tell them apart. A true solver keeps far more detail.
It is blind to stack depth. The bot only ever trained with both players holding a hundred big blinds. Hand it a short stack and it is playing a situation it has never seen, so the game below resets to a fresh hundred big blinds every hand rather than pretending the bot knows how to play short.
Its bet sizes are still blunt. Four sizes feel human, but a strong player varies their sizing far more finely. The one-third-pot experiment showed the cost of adding even one more, and doing it properly needs a different sampling method than the one I used.
The strength estimate is noisy. Because hand strength comes from random rollouts, the same hand can occasionally land in neighboring bins on different visits. It averages out over millions of hands but slows things down getting there.
Play the bot
“What this is... the actual 3.5-million-hand model, running entirely in your browser. You sit at the bottom, the bot at the top. Stacks reset to 100 big blinds each hand because that is the only depth it knows. Your running result is saved locally on your machine and nowhere else. It is play money, and the bot can be beaten, so go find the holes in it.”
Play heads-up against the CFR bot
No-limit Texas Hold'em, one on one, against a bot trained on 3.5 million hands of self-play. You each start with $500; every hand is dealt 100 big blinds deep — the depth the bot learned — and your bankrolls carry across hands. Play until someone busts.
What I actually learned
The moment that stays with me is not the final win rate. It is that losing chart against the calling station. I built something whose entire purpose is to be unbeatable, and the proof that it succeeded is that a brick-simple opponent can take money off it. Being impossible to exploit and being maximally profitable are not the same thing, and a strategy can be correct and still leave money on the table against a fool. That is a genuinely useful thing to understand about any adversarial system, markets very much included.
The other thing is just how little I had to teach it. I wrote down the rules of poker and a way to measure regret, and walked away for twelve hours. What came back bluffs, sizes its bets, folds its weak hands, and plays differently from the button than the big blind, none of which I ever specified. I only built the room. It worked out how to play in it.
