IUI 2004 Madeira Portugal Wed Jan 14 2004

IUI 2004, Madeira, Portugal – Wed Jan 14, 2004 Designing Example Critiquing Interaction Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani LIA Wed Jan 14, 2004 HCI Designing Example Critiquing Interaction

Outline n n Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion Wed Jan 14, 2004 Designing Example Critiquing Interaction 2

Motivation n Many real word applications require people to select a most preferred outcome from a large set of possibilities (electronic catalogs) Users are usually unable to correctly state their preferences up front People are greatly helped by seeing examples of actual solutions ¡ example critiquing Wed Jan 14, 2004 Designing Example Critiquing Interaction 3

Mixed Initiative Interaction initial preference The user critiques the solutions stating a new preference the system shows K solutions The user picks the final choice Wed Jan 14, 2004 Designing Example Critiquing Interaction 4

An implementation: reality trade-off between different criteria user critiques existing solutions Wed Jan 14, 2004 Designing Example Critiquing Interaction 5

What to show? n Standard approach show the best solutions ¡ assumption: user model is complete and accurate ¡ n Does not in general stimulate new preferences Wed Jan 14, 2004 Designing Example Critiquing Interaction 6

New approach n Display_set = stimulate_set + optimal_set Wed Jan 14, 2004 Designing Example Critiquing Interaction 7

What to show? n Stimulate set = solutions that ¡ ¡ n make the user aware of attributes diversity have high probability to become optimal if new preferences are stated Optimal set = solutions that ¡ are optimal given the current preferences Wed Jan 14, 2004 Designing Example Critiquing Interaction 8

Outline n n Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion Wed Jan 14, 2004 Designing Example Critiquing Interaction 9

Stimulating new preferences n Pareto optimality ¡ ¡ n general concept does not involve weights Dominated solution can become Pareto optimal if new preferences are stated show solutions that have higher probability of becoming Pareto optimal Wed Jan 14, 2004 Designing Example Critiquing Interaction 10

Dominance relation and Pareto optimality Penalty table, 2 preferences P 2 9 s 4 s 5 6 s 3 s 1 3 n s 2 n n 3 Wed Jan 14, 2004 6 9 n P 1 S 1 and S 2 are Pareto optimal S 3 is dominated by S 1 and S 2 S 4 is dominated by S 1 S 5 is dominated by S 1, S 2, S 3. Designing Example Critiquing Interaction 11

Pareto Optimal Filters n n Estimate the probability that a dominated solution can become Pareto optimal when new preferences are stated Different Pareto-filters: ¡ ¡ ¡ counting filter attribute filter probabilistic filter Wed Jan 14, 2004 Designing Example Critiquing Interaction 12

Counting filter n n n Counting Filter: number of Dominators Wed Jan 14, 2004 We count the number of dominators S 1 and S 2 are currently “optimal” S 4 more promising than S 3 and S 5 Designing Example Critiquing Interaction 13

A new preference is added n n New column with penalties S 4 becomes Pareto optimal even if the new penalty (0. 6) is worse than for S 3 (0. 5) and S 5 (0. 4) The counting filter predict that S 4 has better chances to become P. O. when a new preference is added. Wed Jan 14, 2004 Designing Example Critiquing Interaction 14

Hasse diagrams Pareto Optimal 1 2 4 4 3 1 2 3 5 User Model={p 1, p 2, p 3} 5 User Model={p 1, p 2} Adding preferences: Pareto optimal set grows, dominance relation becomes sparse Wed Jan 14, 2004 Designing Example Critiquing Interaction 15

Attribute filter n Solution: n attributes: A 1, . . , An ¡ ¡ n n D 1, . . , Dn domains for A 1, . . , An a solution is a complete assignment Preferences modeled as penalty functions defined on attribute domains Look at the attribute space ¡ if two values are the same, any penalty function defined on these values will be the same Wed Jan 14, 2004 Designing Example Critiquing Interaction 16

Attribute filter: motivation Preferences: on price (to minimize), on M 2 (to maximize) n n S 2 and S 3 are both dominated by S 1 If we add new preference ¡ ¡ ¡ on Location if North is preferred S 2 will be Pareto Optimal on Transport if Tramway is preferred to Bus then S 2 will be P. O. S 3 will always be dominated!! Wed Jan 14, 2004 Designing Example Critiquing Interaction 17

Attribute Filter n For the new preference, dominated solution s must have lower penalty than all dominant solutions for discrete domain, attribute values must be different ¡ for continuous domains, consider extreme values ¡ Wed Jan 14, 2004 Designing Example Critiquing Interaction 18

Probabilistic filter n penalty n Directly estimate probability of becoming P. O. The bigger the difference on a specific attribute, the more likely the penalties will be different 1 1 domain Wed Jan 14, 2004 Designing Example Critiquing Interaction domain 19

Experiments Database of actual accommodation offers (room for rent, studios, apartments) n Random datasets n 11 attributes of which n 4 continuous (price, duration, square meters, distance to university) ¡ 7 discrete (kitchen, kitchen type, bathroom, public transportation, . . ) ¡ Wed Jan 14, 2004 Designing Example Critiquing Interaction 20

Results (accommodation dataset) Wed Jan 14, 2004 Designing Example Critiquing Interaction 21

Average fraction of correct predictions Results (random dataset) number of preferences known Wed Jan 14, 2004 Designing Example Critiquing Interaction 22

Outline n n Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion Wed Jan 14, 2004 Designing Example Critiquing Interaction 23

Modelling n True preference model P* (unknown) ¡ ¡ n Estimated through a model P ¡ ¡ ¡ n P*={p*1, p*2, . . , p*k} st: target solution P={p 1, . . , pk} pi are built-in standard penalty functions assume limited difference between p and p* Penalty functions ¡ ¡ pi(ak): dk -> R write pi(s) instead of pi(aj(s)) Wed Jan 14, 2004 Designing Example Critiquing Interaction 24

Selecting displayed solutions n n n Dominance filters Utilitarian filters Egalitarian filters Wed Jan 14, 2004 Designing Example Critiquing Interaction 25

Optimal Set Filters Properties We want. . 1. To show a limited number of solutions ¡ 2. each filter selects k solutions to display To ensure that a Pareto-optimal solution in D is Pareto-optimal in S ¡ ¡ each filter satisfies this dominance filter (by definition), Utilitarian and Egalitarian (theorem) Wed Jan 14, 2004 Designing Example Critiquing Interaction 26

Optimal Set Filters Properties 3. To include target solutions ¡ ¡ only if target solution is included the user can choose it! probability to include the target solution in D depends on filter. Assumption Wed Jan 14, 2004 (1 -ε)pi ≤ pi* ≤ (1+ε)pi Designing Example Critiquing Interaction 27

Dominance filter n Display k solutions that are not dominated by another one Wed Jan 14, 2004 Designing Example Critiquing Interaction 28

Dominance filter: target solution n Plot of probability of target solution being included in D, as function of number of preferences ¡ |P|=3, . . , 12 ¡ K=30, 60 ¡ m=778, 6444 Wed Jan 14, 2004 Designing Example Critiquing Interaction 29

Utilitarian filter n n We minimize the un-weighted sum of penalties Efficiently computed by “branch & bound” Wed Jan 14, 2004 Designing Example Critiquing Interaction 30

Utilitarian filter: probability to find target solution n n Does not depend on m, the number of total solutions (proved analytically) Better than dominator filter Wed Jan 14, 2004 Designing Example Critiquing Interaction 31

Egalitarian filter n Minimize F(s) In case of equality, use lexicographic order: (0. 4, 0. 2) preferred to (0. 4, 0. 4) ¡ n Target solution inclusion probability similar to that of the Utilitarian filter. Wed Jan 14, 2004 Designing Example Critiquing Interaction 32

Robustness against violated assumption n n Wed Jan 14, 2004 Designing Example Critiquing Interaction Fraction of PO solutions shown within the best k ones Egalitarian, utilitarian filter 33

Outline n n Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion Wed Jan 14, 2004 Designing Example Critiquing Interaction 34

Conclusion n n Optimal and stimulation set Example critiquing on firmer mathematical ground Suggestions to system developer How to compensate an incomplete/inaccurate user model Experimental evaluation on real and random problems Wed Jan 14, 2004 Designing Example Critiquing Interaction 35

Questions Wed Jan 14, 2004 Designing Example Critiquing Interaction 36

Wed Jan 14, 2004 Designing Example Critiquing Interaction 37

Pareto Filters: conclusions Counting filter works already fairly well n Attribute filter works very well when only 1 or 2 preferences are missing, but generally probabilistic is the best n Impact of correlation between attributes can affect performance n Random Counting Attribute Probabilistic Complexity Wed Jan 14, 2004 Designing Example Critiquing Interaction 38

Attribute filter/2 n Continuous domains: best values are the extremes assumption: preference functions are monotonic penalty ¡ OR 1 domain Wed Jan 14, 2004 Designing Example Critiquing Interaction 39

1 domain ai(o 1) θ ai(o 2) ai(o 1) ai(o 2) m 1 1 domain ai(o 1) Wed Jan 14, 2004 θ ai(o 2) domain ai(o 1) Designing Example Critiquing Interaction θ ai(o 2) 40

1 domain ai(o) θ ai(o) li li m 2 m 1 1 1 ai(o) ai(o 1)-t θ Wed Jan 14, 2004 domain li θ+t θ-t gi Designing Example Critiquing Interaction θ ai(o) si 41

Wed Jan 14, 2004 Designing Example Critiquing Interaction 42

Theorem n n n Given a set of m solutions S={s 1, . . , sm} and a set of penalties {p 1, . . , pd} Let S’ be the best k solutions according to the utilitarian filter A solution s in S’ not dominated by any other of S’, is Pareto Optimal in S. Wed Jan 14, 2004 Designing Example Critiquing Interaction 43

Simplified Apartment Domain n A very simple example: ¡ ¡ A={Location, Rent, Rooms} DLocation={Centre, North, South, East, West} DRent={x|x integer x>0} DRooms={1, 2, 3. . } Preferences: location should be centre and rent less than 500 Wed Jan 14, 2004 Designing Example Critiquing Interaction 44

Penalty functions n P 1: = ¡ n if (Location==centre) then 0 else 1 P 2: = ¡ ¡ If (Rent > 500) then K*(Rent-500) Else 0 Wed Jan 14, 2004 Designing Example Critiquing Interaction 45

Electronic catalogues n K attributes: A 1, . . , Ak D 1, . . , Dk domains for A 1, . . , Ak ¡ a solution is a complete assignment ¡ write aj(s), value of S for attribute j ¡ n Solution set S ¡ n is a subset of D 1 x D 2 x D 3 x D 4 x. . . Preferences modeled as penalty functions defined on attribute domains Wed Jan 14, 2004 Designing Example Critiquing Interaction 46

Counting filter* The Dominator set for a solution s 1, is the subset of S of solution that dominates s 1. The counting filter orders solutions on the size of Sd(s 1 the dominator set. ) s 1 Wed Jan 14, 2004 Designing Example Critiquing Interaction 47

Probabilistic filter n penalty n Directly estimate probability of becoming P. O. The bigger the difference on a specific attribute, the more likely the penalties will be different 1 domain Wed Jan 14, 2004 Designing Example Critiquing Interaction 48