Vasilis Syrgkanis 1 Microsoft Research NYC Joint with

Vasilis Syrgkanis 1 Microsoft Research, NYC Joint with: Yishay Mansour (Microsoft Research and Tel-Aviv Univ. ) Aleksandrs Slivkins (Microsoft Research)

2

§ Prior bias of users leads to lack of exploration § Can miss good options that a priori seem inferior (“hidden gems”) § System needs to “incentivize” exploration § This talk: incentivizing exploration through information asymmetry 3

Recommendation algorithm Users 1 Actions t 1 § T i K Prior on Mean Rewards Realized rewards noisy 4

Recommendation algorithm Users 1 Actions t 1 T i K Prior on Mean Rewards Realized rewards noisy 5

Users do not observe rewards or recommendations of previous users Unaware whether rewards of previous steps have made a priori better arms worse than a priori worse arms 8

Main question § 9

§ 10

§ 11

§ 12

Key Ideas 13

§ Hide exploration in a pool of exploitation Recommend arm 2 Pull arm 1 Pull posterior better arm • “Exploration” user doesn’t know if he is recommended action 2 because of it is the better posterior action or because of exploration • If L is large enough exploitation is most probable and hence in his own interest 14

Recommend arm 2 Pull arm 1 Ask Bandit algorithm Report back Pull posterior better arm … Initial Sampling Phase 15

• Use sample means instead of posterior means • Exploit arm is arm 2 only if sample mean higher with some confidence • After initial sampling phase do active arms elimination Recommend arm 2 Pull arm 1 Pull an exploit arm 16

§ Recommend arm 2 Pull arm 1 Pull posterior better arm from 1, , 2 Recommend arm 3 Pull posterior better arm from 1, 2, 3 Recommend arm 4 Pull posterior better arm from 1, 2, 3, 4 17

§ 18

§ 19