Online Poker: Machine Learning's Ultimate Testing Ground?

It may have taken a bit longer than some expected, but the poker industry is finally beginning to feel the impact of AI in much the same way that so many other industries, in the gaming world and beyond, have felt it. In fact, change has occurred so rapidly that many now regard online poker as among the industries mostreceptive to AI- and machine learning-based innovation. It’s been a remarkable shift. 

For both professional and casual poker players as well as investors and entrepreneurs, AI has begun to deliver much-needed stability and credibility, and also a major shot of confidence. Up until 10 or 15 years ago, players had only imperfect tools at their disposal in their efforts to boost their performance. Strategy was rooted in classical pot-odds and game theory, with the result that almost all attempts to develop a winning strategy that could be used by all players, regardless of skill level, failed miserably. No-limit Poker, the ultimate “imperfect information” game, but also one that offers players a near-infinite set of possible choices, proved stubbornly resistant to any sort of detailed analysis or statistical modeling.

Efforts develop more effective strategies picked up between the late 1970s and the late 2000s, when a number of poker professionals published landmark books offering winning strategies based on their own experiences playing the professional circuit. Broadly speaking, these strategies looked to maximize favorable situations and avoid unfavorable ones as early as possible. But they were deeply flawed. Their proponents sought to exploit poker conventions of the time and worked hard to simplify the game; the resulting strategies eschewed complexity in their quest for clarity and direction. However, if someone (or something) ever succeeded in developing a winning strategy by embracing complexity and successfully processing vast amounts of data with ease, the limitations of the earlier strategies (and their proponents) would be exposed almost immediately. 

And this is exactly what happened once big data and statistical modeling came to the fore in the early-to-mid 2010s. Software relying on hand-histories, simulations and probability theory let players know exactly which strategies were profitable or unprofitable in specific situations. Psychological warfare at the tables and on-the-fly probability judgments gave way to detailed study, memorization and the application of insights. The advent of modeling and simulation gave rise to an entire industry centered around research, software and IP--one that continues to grow by leaps and bounds. At the same time, as the game became more stable and trust levels grew, player numbers exploded, hastened in part by the growth of online poker. Data scientists, financial experts, and game developers were especially well-represented. And the inevitable result of the shift to online gambling generally, and online poker specifically, was that security began to be taken far more seriously. Advances in cyber-security, random number generation, and cryptography were rapid. A quick look at the home pages of PokerStars, partypoker, GGPoker, and similar sites reveals detailed explanations of how these businesses employ cutting-edge technologies, in varying forms, to reassure players (and especially current and potential investors) of the basic fairness and reliability of the games on offer. The inevitable result? More players and more investment and the chance to spend even more on technological innovation. 

AI and machine learning began to make its presence felt around 2015 when a number of academics and data scientists pitted an AI program they’d developed, named “Claudico,” against a small team of renowned professional poker players at Rivers Casino in Pittsburgh, PA. They played heads-up no limit (HUNL) poker almost ceaselessly over the course of 13 days. The end result? A remarkable human victory--one that echoed the way the IBM supercomputer Deep Blue initially failed to best mercurial chess Grandmaster Garry Kasparov in 1989. But just as a re-tooled Deep Blue defeated Kasparov convincingly in a rematch in 1997, an upgraded AI poker program known as “Libratus” thrashed four of the world’s best poker players in early 2017. Significantly, it was the ability of Libratus to spend its evenings during the 20-day challenge reviewing and learning from its mistakes, in a way no human being could ever match, that sealed its victory. 

Libratus’ architecture consisted of three main modules, each of which featured new algorithms. The first module was aimed at “pre-computing” solutions to theoretical game situations, ultimately forming a high-level strategy blueprint; the second module featured a new nested sub-game-solving algorithm that continually adjusted and refined strategy as the game advanced, and the third module--self-correcting--helped to supplement and influence the “pre-computed” strategy blueprint. Thus, in early 2017, it became abundantly clear that human cognition, at least where poker was concerned, was bumping up against its natural limits; what was equally clear, however, was that AI and machine learning seemed to have no natural limits at all. One of the players who went up against Libratus, Dong Kim, was shocked by the machine’s capabilities, noting that he “...felt like [he] was playing against someone who was cheating.” It seemed, he said, like the machine could see all his cards, though he was quick to add that he wasn’t accusing Libratus of any dishonesty; he was merely keen to point out just how powerful the system really was. 

Poker, to state the obvious, is a game. But given poker’s enormous complexity and the drive for competitive advantage and profit that surround the game, it should be regarded not only as a proxy for gaming as a whole but a proxy for the entire competitive marketplace. AI and machine learning are revolutionizing what can be achieved in terms of speed, efficiency and quality across almost every imaginable industry. Gaming is highly-visible, and poker -- with its often-enormous pots -- tends to capture the imagination of players and onlookers alike; however, the advances that have taken place in poker are being matched, and often exceeded, by advances in industries as diverse as finance, health care, education, transportation, IT and manufacturing, to name only a few. Entire industries are abandoning past practices--and past assumptions--in favor of AI-mediated processes and decision making. Poker is only one of these industries. And yet online poker has become a laboratory of sorts, and a key showcase, for cutting-edge AI and machine learning technologies. It’s a truly amazing development, and one that will ultimately benefit all of us, even if it means Libratus, or a future iteration of Libratus, continues to beat all comers in no-limit Texas hold’em.