Hybrid Keyword Auctions Ashish Goel Stanford University Joint

Hybrid Keyword Auctions Ashish Goel Stanford University Joint work with Kamesh Munagala, Duke University

Online Advertising Pricing Models r CPM (Cost per thousand impressions) r CPC (Cost per click) r CPA (Cost per acquisition) r Conversion rates: • Click-through-rate (CTR), conversion from clicks to acquisitions, … Differences between these pricing models: r Uncertainty in conversion rates: • Sparse data, changing rates, … r Stochastic fluctuations: • Even if the conversion rates were known exactly, the number of clicks/conversions would still vary, especially for small samples 1

Cost-Per-Click Auction Advertiser Bid = Cost per Click C Auctioneer (Search Engine) 2

Cost-Per-Click Auction Advertiser Bid = Cost per Click C Auctioneer (Search Engine) CTR estimate Q 3

Cost-Per-Click Auction Advertiser Bid = Cost per Click C Auctioneer (Search Engine) CTR estimate Q • Value/impression ordering: C 1 Q 1 > C 2 Q 2 > … • Give impression to bidder 1 at CPC = C 2 Q 2/Q 1 4

Cost-Per-Click Auction Advertiser Bid = Cost per Click C Auctioneer (Search Engine) CTR estimate Q • Value/impression ordering: C 1 Q 1 > C 2 Q 2 > … • Give impression to bidder 1 at CPC = C 2 Q 2/Q 1 VCG Mechanism: Truthful for a single slot, assuming static CTR estimates Can be made truthful for multiple slots [Vickrey-Clark-Groves, Myerson 81, AGM 06] This talk will focus on single slot for proofs/examples 5

When Does this Work Well? n High volume targets (keywords) q n Good estimates of CTR What fraction of searches are to high volume targets? q Folklore: a small fraction q Motivating problem: r How to better monetize the low volume keywords? 6

7

8

Possible Solutions n Coarse ad groups to predict CTR: q n n Use performance of advertiser on possibly unrelated keywords Predictive models q Regression analysis/feature extraction q Taxonomies/clustering q Collaborative filtering q Learn the human brain! Our approach: Richer pricing models + Learning 9

Hybrid Scheme: 2 -Dim Bid M = Cost per Impression C = Cost per Click Advertiser <M, C > Auctioneer (Search Engine) 10

Hybrid Scheme M = Cost per Impression C = Cost per Click Advertiser <M, C > Auctioneer (Search Engine) CTR estimate Q 11

Hybrid Scheme M = Cost per Impression C = Cost per Click Advertiser <M, C > Auctioneer (Search Engine) CTR estimate Q • Advertiser’s score Ri = max { Mi , Ci Qi } 12

Hybrid Scheme M = Cost per Impression C = Cost per Click Advertiser <M, C > Auctioneer (Search Engine) CTR estimate Q • Advertiser’s score Ri = max { Mi , Ci Qi } • Order by score: R 1 > R 2 > … 13

Hybrid Scheme M = Cost per Impression C = Cost per Click Advertiser <M, C > Auctioneer (Search Engine) CTR estimate Q • Advertiser’s score Ri = max { Mi , Ci Qi } • Order by score: R 1 > R 2 > … • Give impression to bidder 1: • If M 1 > C 1 Q 1 then charge R 2 per impression • If M 1 < C 1 Q 1 then charge R 2 / Q 1 per click 14

Example: CPC Auction Bidder 1 Bidder 2 Per click cost C 5 10 Conversion rate estimate Q 0. 1 0. 08 C*Q 0. 5 0. 8 CPC auction allocates to bidder 2 at CPC = 0. 5/0. 08 = 6. 25 15

Example: Hybrid Auction Bidder 1 Bidder 2 Per click cost C 5 10 Conversion rate estimate Q 0. 1 0. 08 C*Q 0. 5 0. 8 Per impression cost M 1. 0 0 Max {M, C * Q} 1. 0 0. 8 Hybrid auction allocates to bidder 1 at CPI = 0. 8 16

Why Such a Model? n Per-impression bid: q Advertiser’s estimate or “belief” of CTR q May or may not be an accurate reflection of the truth q Backward compatible with cost-per-click (CPC) bidding 17

Why Such a Model? n n Per-impression bid: q Advertiser’s estimate or “belief” of CTR q May or may not be an accurate reflection of the truth q Backward compatible with cost-per-click (CPC) bidding Why would the advertiser know any better? q Advertiser aggregates data from various publishers q Has domain specific models not available to auctioneer q Is willing to pay a premium for internal experiments 18

Provable Benefits 1. Search engine: Better monetization of low volume keywords n Typical case: Unbounded gain over CPC auction n Pathological worst case: Bounded loss over CPC auction 2. Advertiser: Opportunity to make the search engine converge to the correct CTR estimate without paying a premium 3. Technical: a) Truthful b) Accounts for risk characteristics of the advertiser c) Allows users to implement complex strategies 19

Key Point Implementing the properties need both per impression and per click bids 20

Multiple Slots n Show the top K scoring advertisers r n Assume R 1 > R 2 > … > RK+1… Generalized Second Price (GSP) mechanism: r For the ith advertiser, if: • If Mi > Qi. Ci then charge Ri+1 per impression • If Mi < Qi. Ci then charge Ri+1 / Qi per click 21

Multiple Slots n Show the top K scoring advertisers r n Assume R 1 > R 2 > … > RK+1… Generalized Second Price (GSP) mechanism: r For the ith advertiser, if: • If Mi > Qi. Ci then charge Ri+1 per impression • If Mi < Qi. Ci then charge Ri+1 / Qi per click n Can also implement VCG r r [Vickrey-Clark-Groves, Myerson 81, AGM 06] Need separable CTR assumption Details in the paper 22

Uncertainty Model for CTR n n For analyzing advantages of Hybrid, we need to model: q Available information about CTR q Asymmetry in information between advertiser and auctioneer q Evolution of this information over time We will use Bayesian model of information q Prior distributions q Specifically, Beta priors (more later) 23

Bayesian Model for CTR True underlying CTR = p Advertiser Auctioneer (Search Engine) 24

Bayesian Model for CTR True underlying CTR = p ri st di ) on ti bu lic ub c P au Advertiser iv r (P r o ri e) t a (P tr s di io on ti u ib Pr v P ad Auctioneer (Search Engine) 25

Bayesian Model for CTR True underlying CTR = p di CTR estimate ri st Q ) on ti bu lic c P au Advertiser r (P iv e) t a ub P r o ri s di (P tr M r Per-impression bid io on ti u ib Pr v P ad Auctioneer (Search Engine) 26

Bayesian Model for CTR True underlying CTR = p di CTR estimate ri st Q ) on ti bu lic c Advertiser P au r (P iv e) t a ub P r o ri s di (P tr M r Per-impression bid io on ti u ib Pr v P ad Auctioneer (Search Engine) Each agent optimizes based on its current “belief” or prior: Beliefs updated with every impression Over time, become sharply concentrated around true CTR 27

What is a Prior? n n n Simply models asymmetric information q Sharper prior More certain about true CTR p q E[ Prior ] need not be equal to p Main advantage of per-impression bids is when: q Advertiser’s prior is “more resolved” than auctioneer’s q Limiting case: Advertiser certain about CTR p Priors are only for purpose of analysis q Mechanism is well-defined regardless of modeling assumptions 28

Truthfulness n Advertiser assumes CTR follows distribution Padv n Wishes to maximize expected profit at current step q r E[Padv] = x = Expected belief about CTR r Click utility = C Expected profit = C x - Expected price 29

Truthfulness n Advertiser assumes CTR follows distribution Padv n Wishes to maximize expected profit at current step q r E[Padv] = x = Expected belief about CTR r Click utility = C Expected profit = C x - Expected price Bidding (Cx, C) is the dominant strategy Regardless of Q used by auctioneer and true CTR p Elicits advertiser’s “expected belief” about the CTR! Holds in many other settings (more later) 30

Specific Class of Priors

Conjugate Beta Priors n Auctioneer’s Pauc for advertiser i = Beta( , ) r , are positive integers r Conjugate of Bernoulli distribution (CTR) r Expected value = / ( + ) 32

Conjugate Beta Priors n n Auctioneer’s Pauc for advertiser i = Beta( , ) r , are positive integers r Conjugate of Bernoulli distribution (CTR) r Expected value = / ( + ) Bayesian update with each impression: r Probability of click = / ( + ) r If click, r If no click, new Pauc (posterior) = Beta( , ) new Pauc (posterior) = Beta( , ) 33

Evolution of Beta Priors Click 1, 1 No Click 1/2 2, 1 2/3 1, 2 1/3 3, 1 3/4 4, 1 Denotes Beta(1, 1) Uniform prior Uninformative 1/3 2, 2 1/4 1/2 E[Pauc] = 1/4 1, 3 1/2 3, 2 1/4 2, 3 3/4 1, 4 E[Pauc] = 2/5 34

Properties of Beta Priors n Larger , Sharper concentration around p q n Uninformative prior: Beta( , ) = Uniform[0, 1] Q = E[Pauc] = / ( + ) q Auctioneer’s expected “belief” about CTR q Could be different from true CTR p 35

Benefits to Auctioneer and Advertisers

Advertiser Certain of CTR True underlying CTR = p CTR estimate = ( ta Be Q = / ( + ) ) , lic ) Advertiser c r (P iv e) t a ub o n K p (P M = Cp ws P au Per-impression bid Auctioneer (Search Engine) 37

Properties of Auction n n Revenue properties for auctioneer: r Typical case benefit: log n times better than CPC scheme r Bounded pathological case loss: 36% of CPC scheme r Unbounded gain versus bounded loss! Flexibility for advertiser: r Can make Pauc converge to p without paying premium r But pays huge premium for achieving this in CPC auction 38

Better Monetization p 1 Adv. 1 Pa uc p 2 Adv. 2 p 3 Adv. 3 … pn Adv. n =B eta a( P auc et B = ( , Q ≈ 1 / log n log g , lo n) n) Auctioneer Low volume keyword: • Auctioneer’s prior has high variance • Some pi close to 1 with high probability 39

Better Monetization n Hybrid auction: Per-impression bid elicits high pi n CPC auction allocates slot to a random advertiser n Theorem: Hybrid auction generates log n times more revenue for auctioneer than CPC auction 40

Flexibility for Advertisers Assume C = 1 Per-impression bid M=p Advertiser Knows p Q = / ( + ) Pauc = Beta( , ) Auctioneer (Search Engine) 41

Flexibility for Advertisers Per-impression bid M=p Advertiser Knows p Q = / ( + ) Pauc = Beta( , ) Auctioneer (Search Engine) Wins on per impression bid Pays at most p per impression 42

Flexibility for Advertisers Per-impression bid M=p T impressions N clicks Advertiser Knows p g s ake Q = / ( + ) Pauc = Beta( , ) Auctioneer (Search Engine) o et M Q r e v con p Wins on per impression bid Pays at most p per impression 43

Flexibility for Advertisers Per-impression bid M=p T impressions N clicks Advertiser Knows p g s ake Q = / ( + ) Pauc = Beta( , ) Auctioneer (Search Engine) o et M Q r e v con p Now switch to CPC bidding 44

Flexibility for Advertisers n n If Q converges in T impressions resulting in N clicks: q ( N T) ≥ p q Since Q = /( + ) < p, this implies N ≥ T p q Value gain = N; Payment for T impressions at most T * p q No loss in utility to advertiser! In the existing CPC auction: q The advertiser would have to pay a huge premium for getting impressions and making the CTR converge 45

Dynamic Properties

Uncertain Advertisers n n Advertiser “wishes” CTR p to resolve to a high value q In that case, she can gain utility in the long run q … but CTR resolves only on obtaining impressions! Should pay premium now for possible future benefit q What should her dynamic bidding strategy be? 47

Uncertain Advertisers n n Advertiser “wishes” CTR p to resolve to a high value q In that case, she can gain utility in the long run q … but CTR resolves only on obtaining impressions! Should pay premium now for possible future benefit q n What should her dynamic bidding strategy be? Key contribution: q Defining a new Bayesian model for repeated auctions q Dominant strategy exists! 48

The Issue with Dynamics Padv 1 Adv. 1 Pau c 1 P auc 2 Padv 2 Auctioneer Adv. 2 [Bapna & Weber ‘ 05, Athey & Segal ‘ 06] Advertiser 1 underbids so that: • Advertiser 2 can obtain impressions • Advertiser 2 may resolve its CTR to a low value • Advertiser 1 can then obtain impressions cheaply 49

Semi-Myopic Advertiser n Maximizes utility in contiguous time when she wins the auction q Priors of other advertisers stay the same during this time q Once she stops getting impressions, cannot predict future … since future will depend on private information of other bidders! 50

Semi-Myopic Advertiser n Maximizes utility in contiguous time when she wins the auction q Priors of other advertisers stay the same during this time q Once she stops getting impressions, cannot predict future … since future will depend on private information of other bidders! n Advertiser always has a dominant hybrid strategy q Bidding Index: Computation similar to the Gittins index r q Advertiser can optimize her utility by dynamic programming Socially optimal in many reasonable scenarios r Implementation needs both per-impression and per-click bids 51

Summary n Allow both per-impression and per-click bids r Same ideas work for CPM/CPC + CPA n Significantly higher revenue for auctioneer n Easy to implement n r Hybrid advertisers can co-exist with pure per-click advertisers r Easy path to deployment/testing Many variants possible with common structure: r Optional hybrid bids r Impression + Click [S. Goel, Lahaie, Vassilvitskii, 2010] 52

Conclulsion and Open Questions n n Learning is important in Online advertising q Remember that there are multiple strategic participants q Design richer pricing and communication signals Some issues that need to be addressed: q Whitewashing: Re-entering when CTR is lower than the default q Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q r Switch to per click bidding when slot is “locked in” by the high Q 53

Open Questions n Some issues that need to be addressed: q Whitewashing: Re-entering when CTR is lower than the default q Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q r n Switch to per click bidding when slot is “locked in” by the high Q Analysis of semi-myopic model r Other applications of separate beliefs? 54

Open Questions n Some issues that need to be addressed: q Whitewashing: Re-entering when CTR is lower than the default q Fake Clicks: Bid per impression initially and generate false clicks to drive up CTR estimate Q r n Analysis of semi-myopic model r n Switch to per click bidding when slot is “locked in” by the high Q Other applications of separate beliefs? Connections of Bayesian mechanisms to: r Regret bounds and learning r Best-response dynamics [Nazerzadeh, Saberi, Vohra ‘ 08] [Edelman, Ostrovsky, Schwarz ‘ 05] 55

Truthfulness n Advertiser assumes CTR follows distribution Padv n Wishes to maximize expected profit at current step q r E[Padv] = x = Expected belief about CTR r Utility from click = C Expected profit = C x - Expected price Let Cy = Per impression bid R 2 = Highest other score If max(Cy, C Q) < R 2 then Price = 0 Else: If y < Q then: Price = x R 2 / Q If y > Q then: Price = R 2 56