next up previous contents
Next: Bayesian Poker Player Up: Previous Computer Science Studies Previous: Koller and Pfeffer

Automated Texas Hold'em

Presently, Billings et al. Billings95,Papp98,Pena99,Billings98a,Billings98b,Schaeffer99,Billings99a,Billings99b are investigating the automation of the poker game Texas Hold'em with their program Poki. Their goal is to develop a world class poker playing program capable of defeating the best human players. They combine probability tables for the current hand strength, the probability of a hand improving during play, frequency data for opponent behaviour, and playing heuristics. Their work models opponents in a weighting scheme relating to hand strengths and betting behaviour and does quite well at observing each opponent's actions over time and adjusting its play accordingly.

Poki has been competing against human opponents across the internet on a server dedicated for poker play. It achieves highly promising results and performs at the level of an intermediate poker player. After the initial version of Poki was created, Loki, a number of weaknesses were identified in the program design. The rigid, deterministic, hand-tuned betting strategy that always returned the ``best'' betting action was being identified and exploited by stronger players. The program was then modified by introducing a new betting strategy which uses three probabilities, one for each betting action (fold, call/pass, raise/bet), determined using an evaluation function which attempts to determine the most profitable action in a given situation. Poki then randomly select the betting decision based on this probability distribution. This strategy added unpredictability to the programs play without sacrificing too much in terms of expected reward.

To determine the most profitable action for a given situation a simulation-based approach was used. The approach consisted of simulating the outcome of a hand many times. In every simulation trial, a likely instance of the opponents' hands was generated, and the hand was played out once for each betting alternative performed as the first action. The results of all the trials are averaged and the betting action with the highest expectation was used. These modifications improved the playing ability of the program notably. It consistently out plays the deterministic version, Loki, and makes a significantly higher profit against human opponents than the initial program design. The work being performed by Billings et al. in still in progress and is continually being developed and improved. Their research appears highly promising is invaluable in the goal of producing a world class poker playing program. This research has provided a number of useful ideas for computer poker, not in the least, that the use of some form of randomised betting strategy should probably be employed to ensure that the programs actions do not make it easily predictable and exploitable. Opponent modeling is also seen to be a necessity. While Poki's opponent modeling is still under development, it has been identified as an essential feature for a strong poker playing program.


next up previous contents
Next: Bayesian Poker Player Up: Previous Computer Science Studies Previous: Koller and Pfeffer
Jason R Carlton
2000-11-13