The challenge of poker NDHU CSIE AI Lab

The challenge of poker NDHU CSIE AI Lab 羅仲耘

Outline • • Introduction Texas Hold’em rules Poki’s architecture Betting Strategy – Pre-flop betting strategy – Basic betting strategy • Reference 2004/11/04 the challenge of poker 2

Introduction • Bridge and poker involve imperfect information • Traditional methods like deep search can’t play these game well 2004/11/04 the challenge of poker 3

Texas Hold’em rules (1) 1. Opening deal – Each player is dealt two cards face down, which are known as hole cards or pocket cards. 2. First round of betting – Starting with the player to the left of the big blind, each player can call the big blind, raise, or fold. The big blind has the option to raise an otherwise unraised pot. 2004/11/04 the challenge of poker 4

Texas Hold’em rules (2) 3. The flop – The dealer burns a card, and then deals three community cards face up. The first three cards are referred to as the flop, while all of the community cards are collectively called the board. 2004/11/04 the challenge of poker 5

Texas Hold’em rules (3) 4. Second round of betting – Starting with the player to the left of the dealer button, each player can check or bet. Once a bet has been made, each player can raise, call, or fold. 2004/11/04 the challenge of poker 6

Texas Hold’em rules (4) 5. The turn – The dealer burns another card, and then adds a fourth card face-up to the community cards. This fourth card is known as the turn card, or fourth street. 2004/11/04 the challenge of poker 7

Texas Hold’em rules (5) 6. Third round of betting – It follows the same format as the second round, but the size of the bets have usually doubled in limit games. 2004/11/04 the challenge of poker 8

Texas Hold’em rules (6) 7. The river – The dealer burns another card, and then adds a fifth and final card to the community cards. This fifth card is known as the river card, or fifth street. 2004/11/04 the challenge of poker 9

Texas Hold’em rules (7) 8. Final round of betting – It follows the same format as the second and third rounds. 9. The showdown – Using the best fivecard combination of their hole cards and the community cards, the remaining players show their hands, with the bettor or last raiser showing first. The highest five-card hand wins the pot. (In case of a tie, the pot is evenly split among the winning hands. ) 2004/11/04 the challenge of poker 10

Poki’s architecture 2004/11/04 the challenge of poker 11

Pre-flop betting strategy(1) • Income rate: base on roll simulation • Roll simulation: an off-line computation that consists of playing several million hands (trials) where all players call the first bet, and then all the remaining cards are dealt out without any further betting • David Sklansky wrote books on this game and classify the hand cards. 2004/11/04 the challenge of poker 12

Pre-flop betting strategy(2) 2004/11/04 the challenge of poker 13

Pre-flop betting strategy(3) • Prefer roll-out simulation than David Sklansky’s classification – Roll-out simulation information is quantitative rather than qualitative – Can apply many difference specific situation 2004/11/04 the challenge of poker 14

Basic betting strategy(1) • Hand strength (HS): is the probability that a given hand is better than that of an active opponent • For example: hand is A♦-Q♣ and the flop is J♥-4♣-3♥. 47 remaining unknown cards and therefore {47 choose 2} = 1081 possible hands an opponent might hold. 444 cases are better than hand, 9 are equal, and 628 are worse than hand. • There is a 58. 5% chance that hand is better than a random hand. 5 • If there are 5 opponents, the HS 5 is 0. 585 = 6. 9% 2004/11/04 the challenge of poker 15

Basic betting strategy(2) • Hand potential – Positive potential (PPot): Chance that a hand which is not currently the best improves to win at the showdown – Negative potential (NPot): Chance that a currently leading hand ends up losing • All possible opponent cards is {47 choose 2} = 1081, therefore remains {45 choose 2} = 990 possible turn and river cards to consider for each opponent 2004/11/04 the challenge of poker 16

Basic betting strategy(3) • For example: if the hand A♦-Q♣ is ahead against one opponent after 5 cards, then after 7 cards there is a 449005 / 621720 = 72% chance of still being ahead 2004/11/04 the challenge of poker 17

Basic betting strategy(4) • Effective hand strength (EHS): combines hand strength and potential to give a single measure of the relative strength of Poki’s hand against an opponent P(win) = P(ahead) x P(opponent does not improve) + P(behind) x P(we improve) = HS x (1 - NPot) + (1 - HS) x PPot 2004/11/04 the challenge of poker 18

Basic betting strategy(5) • Weighting the enumerations – The probability of each hand being played to a particular point in the game will very – For example: the probability that the opponent holds Ace-King is much higher than 7 -2 after the flop, because most will fold 7 -2 before the flop 2004/11/04 the challenge of poker 19

Basic betting strategy(6) • To account for this, Poki maintains a weight table for each opponent • The weights have a value in the range zero to one 2004/11/04 the challenge of poker 20

Basic betting strategy(7) • For example: [ A♠K♣, 0. 40 ], …[ Q♦ 2♦, 0. 20 ], … the Predict. Opponent. Action procedure generates probability distributions { P( fold), P( check / call), P( bet / raise)} A♠ - K♣ { 0. 0, 0. 7, 0. 3 } Q♦ - 2♦ { 0. 0, 0. 1, 0. 9 } • If the observed player action is a bet, after reweighting, the new table entry for A♠ - K♣ will be 0. 4 x 0. 3 = 0. 12 Q♦ - 2♦ will be 0. 2 x 0. 9 = 0. 18 2004/11/04 the challenge of poker 21

Basic betting strategy(8) • Probability triple PT = {f, c, r}, such that f + c + r = 1. 0, representing the probability distribution that the next betting action in a given context is fold, call, or raise, respectively 2004/11/04 the challenge of poker 22

Reference • • • The challenge of Poker / Darse Billings, Aaron Davidson, Jonathan Schaeffer, Duane Szafron / Artificial Intelligence 134 (2002) http: //www. cardplayer. com/rules-of-poker/texas-holdem. php http: //www. ultimatebet. com/rules-strategy/texas-holdem. html 2004/11/04 the challenge of poker 23

Thanks for your attention! 2004/11/04 the challenge of poker 24