Artificial Intelligence Taking over the World of Poker
Rack up another win for the machines.
In this article, we’re going to talk about how Artificial Intelligence called Pluribus has finally cracked the biggest challenge in poker — winning a multiplayer professional tournament.
However, to help you get a better understanding of just how special this milestone was, we’ll also show you:
We won’t keep you waiting as we’re sure that you want to find out more about this incredible story. So, let’s get down to it.
Another Win for Artificial Intelligence
Although this victory was an important milestone for AI, it wasn’t the first time a poker-playing program managed to beat poker pros. Namely, during the 2017 casino tournament, an AI program called Libratus defeated four professional players in 120,000 hands of two-player poker.
However, at that time, the program’s co-creator Tuomas Sandholm (pictured left) was convinced that AI couldn’t achieve similar performance when playing against several players at a time.
Only two years later, he has proven himself wrong. His creation, an AI program called Pluribus, has won the no-limit Texas Hold’Em tournament, during which it played with five human professionals at a time. In over 20,000 hands of online poker, Pluribus has won a virtual $48,000, beating 15 of the world’s top poker players. Five elite human players who agreed to take on the challenge were selected each day from a pool. Each of the pros has won more than $1 million professionally playing the game.
What Makes This Milestone so Special?
Ai has already surpassed humans in two-team or two-player games such as chess, Go, checkers, and two-player no-limit poker. All these games are zero-sum — there are just one winning and one losing side. Also, in these games, players can see the positions of all the pieces. Six-poker, on the other hand, comes much closer to resembling real-life situations — one player must make decisions without knowing anything about the multiple opponents’ resources and decision making processes.
The Pluribus program learned poker by playing against copies of itself. In the beginning, it played six-player games which featured just one human and five independent versions of itself. Later on, it went on to win the tournament against five professional human players in 10,000 hands of poker and 12 days of games. Although Pluribus didn’t reach a win rate as high as Libertus, it still notched a very respectable win rate.
Although there was some evidence that the AI technique used in two-player poker would work well in the three-player competition, it wasn’t clear whether or not it would suffice to reach the highest professional level of play. The fact that it worked so effectively for a six-player poker is undoubtedly a notable milestone. Tuomas Sandholm said,
The ability to beat five players at a time in such a complex game of bluff and hidden information opened up new opportunities for AI to tackle real-world problems
How It All Went Down
To reach this level, Pluribus had started from scratch. It first played randomly, but as time went on, the program steadily improved its performance. After a week or so, it had developed a strategy called blueprint that was used for the first round of betting. The colossal breakthrough that led to beating poker pros in a six-player tournament came from its depth-limited search feature. This feature allows the AI to predict several moves and figure out a profitable strategy for the rest of the game, based on possible opponents decisions.
Many other poker-playing programs have used similar search components, but most of them were designed for zero-sum matches where the number of possible outcomes is somewhat limited. Playing against five opponents is much more complex, and it would require an absurd amount of computing memory. Simply put, in the six-poker game, there are far too many scenarios to predict, based on what cards each player supposedly holds, what each of them believes other players have, as well as all the betting decisions based on those predictions.
Libratus got around this roadblock only by using searches in the final two (out of four) betting rounds. Nevertheless, that solution still required the use of 100 CPUs (central processing units) in a game with only two players.
So Pluribus deployed its depth-limited search. When using this technique, the AI first considers a couple of next moves. Beyond that, it narrows down each simulated player’s choices to only four outcomes. This type of modified search is the reason why the program required less computing resources and memory compared with past superhuman achievements in gaming AIs. Specifically, Pluribus ran on a machine with “just” two central CPUs and 128 gigabytes of memory. To put things into perspective, Deep Mind’s famous AlphaGo program ran on 1,920 CPUs when it beat the professional Go player Lee Sedol.
Carnegie Mellon University
To master Texas Hold’em, Pluribus deployed some surprising strategies which would have otherwise been used by the professionals it played. To start with, it used different bet sizes — an approach humans seem to find hard to do. Also, the program embraced a strategy that humans usually avoid — the “donk betting” strategy. The donk betting is a practice of ending the first round of betting with a call and opening the next round with a bet. And while donk betting is regarded as a weak move that doesn’t make sense among poker pros, Pluribus found otherwise.
Sean Ruane, one of the players who played against Pluribus, explained why playing against a program is such a grueling task. According to him, for humans, poker is a game that rewards you for your mental discipline, consistency, and focus, and punishes you when you lack any of the three. On the other hand, AI bot doesn’t have to worry about these shortcomings, and this is what makes it a tough opponent.
While Pluribus outperformed human poker players, there was no guarantee it would always win, according to a game theory principle called the Nash equilibrium.
The Nash equilibrium occurs in non-cooperative games where each player has a list of strategies, and no player can improve their performance by implementing a different approach. While the Nash equilibrium is still unbeatable in Heads Up Texas Hold’Em, developers still have to find one for the six-player variant of the game.
Do We Stand a Chance?
Jason Les, one of the players who probably has more experience battling poker AI systems than any other poker professional in the world, was stunned by Pluribus’ victory. He claimed to know all the spots for weaknesses, and all the tricks necessary to take advantage of AI’s shortcomings, and yet came up short despite his best efforts. According to him, Pluribus used a sound, game-theory optimal strategy that can only be seen from top human professionals, which was unexpected. To conclude his statement, he said that he wouldn’t participate in poker tournaments where this AI bot was at the table.
Many pros and scientists think that poker AIs might kill the very game they are trying to conquer. Indeed, AI maybe has already killed the heads-up limit. Poker is like a pyramid scheme: it needs a wide range of skill levels to support the pros playing for the big bucks. As humans learn quickly from the bots, everyone becomes good, the skill levels become evened out, the pyramid collapses downward, and the game dies.
Unfortunately, there is a lot of merit to these assumptions. Life-altering developments are fundamentally changing the way humans interact, grow, and survive. Artificial Intelligence is changing the world, and all we can do is adapt and try to make the most out of it.
When it comes to games that involve a lot of probability calculations and math such as Chess or Poker, computers will always be better than us. Our brains are just not wired to perform vast amounts of mathematical calculations in such a short period of time. Does that mean that AI will ruin poker? Not at all, it just means that the game will evolve, like everything else.
What do you think about all this? Does this victory of the machines marks the end of poker? We would be delighted to hear your impressions and thoughts, so don’t hesitate to start a debate in the comment section.