Multiagent Systems Rational Decisions and Utility Theory Manfred

  • Slides: 20
Download presentation
Multiagent Systems Rational Decisions and Utility Theory © Manfred Huber 2018 1

Multiagent Systems Rational Decisions and Utility Theory © Manfred Huber 2018 1

Rationality n Rationality describes a decision making criterion n A rational agent always takes

Rationality n Rationality describes a decision making criterion n A rational agent always takes the action that leads to the “best” outcome for it n In multiagent scenarios this implies n n n © Manfred Huber 2018 Actions are not taken to harm other agents Actions are taken in order to maximize the agent’s performance criterion But: actions do not necessarily only consider aspects important to the individual agent 2

Rationality n Rationality requires to define “best” n n n In multiagent systems what

Rationality n Rationality requires to define “best” n n n In multiagent systems what is “best” can be different for every agent What is “best” depends on the state of the agent What is “best” can be influenced by random effects and uncertainties © Manfred Huber 2018 3

Decision and Utility Theory n Decision Theory deals with optimal decision making in single

Decision and Utility Theory n Decision Theory deals with optimal decision making in single agent as well as multiagent systems n n A rational decision agent takes the action that leads to the highest expected payoff Utility Theory deals with the definition of “best”, “better”, and “payoff” n Utility defines the expected “payoff” © Manfred Huber 2018 4

Utility Theory n Utility: n n n quantifies the degree of preference across different

Utility Theory n Utility: n n n quantifies the degree of preference across different alternatives models the impact of uncertainty on preferences represents a mapping from an agent’s state (or situation) to the degree of happiness (expected future payoff) for being in this state © Manfred Huber 2018 5

Utility Theory n The application of utility theory requires: n n All preferences of

Utility Theory n The application of utility theory requires: n n All preferences of an agent have to be expressed with a single utility function independent of the complexity of the task, environment, and action set An agent’s decisions in the context of uncertainty are purely determined by the expected value of the utility © Manfred Huber 2018 6

Rational Preferences n To make rational decisions we have to define rational preferences: n

Rational Preferences n To make rational decisions we have to define rational preferences: n n Outcomes oi o 1 is at least as desirable as o 2 (the agent weakly prefers o 1 to o 2) n o 2 ( n ( © Manfred Huber 2018 the agent is indifferent between o 1 and ) the agent strictly prefers o 1 to o 2 and ) 7

Rational Preferences n Preferences have to be expressible in the context of uncertainty: n

Rational Preferences n Preferences have to be expressible in the context of uncertainty: n Lottery: n specifies the probability of each possible outcome [p 1 : o 1, p 2 : o 2, …, pn : on] n Lotteries represent uncertain outcomes © Manfred Huber 2018 8

Rational Preferences n For preferences to be rational (more precisely for them to lead

Rational Preferences n For preferences to be rational (more precisely for them to lead to rational decisions) they have to fulfill a number of axioms: n Completeness n Transitivity n Substitutability n Decomposability n Monotonicity n Continuity © Manfred Huber 2018 9

Preferences - Completeness n All outcomes have to be comparable n A preference relationship

Preferences - Completeness n All outcomes have to be comparable n A preference relationship has to be defined for every pair of outcomes © Manfred Huber 2018 10

Preferences - Transitivity n Preferences have to be transitive n This implies that there

Preferences - Transitivity n Preferences have to be transitive n This implies that there can not be any circular preference relationships © Manfred Huber 2018 11

Preferences - Substitutability n If an agent is indifferent between two outcomes then they

Preferences - Substitutability n If an agent is indifferent between two outcomes then they can be substituted for each other without change in the preference n Outcomes that are indifferent have to be indifferent in all contexts © Manfred Huber 2018 12

Preferences - Decomposability n An agent has to be indifferent between two choices that

Preferences - Decomposability n An agent has to be indifferent between two choices that lead to the same outcome n n The agent is indifferent between two lotteries that lead to the same outcome probabilities This implies that the agent has no preference for the mechanism that leads to an outcome but only for the outcome itself © Manfred Huber 2018 13

Preferences - Monotonicity n An agent prefers a larger chance at a better outcome

Preferences - Monotonicity n An agent prefers a larger chance at a better outcome to a smaller chance n This also implies that the agent does not have any preference between deterministic and probabilistic outcomes © Manfred Huber 2018 14

Preferences - Continuity n The preference relation of a lottery has to change continuously

Preferences - Continuity n The preference relation of a lottery has to change continuously with the change of the outcome probabilities © Manfred Huber 2018 15

From Rational Preferences to Utility n von Neumann and Morgenstern showed in 1944 that

From Rational Preferences to Utility n von Neumann and Morgenstern showed in 1944 that if preferences are rational (i. e. they obey the axioms), then there exists a scalar utility function that quantifies the preferences © Manfred Huber 2018 16

Utility Functions n A Utility function allows to quantify preferences for decision making n

Utility Functions n A Utility function allows to quantify preferences for decision making n Rational decisions are simply the ones that lead to the largest value of the utility function © Manfred Huber 2018 17

Utility Functions n There an infinite number of utility functions for each set of

Utility Functions n There an infinite number of utility functions for each set of rational preferences n n Utility can only be used to compare alternatives n n E. g. : Linear offsets and scaling of the utility function preserves preferences The absolute value of the utility is arbitrary The bounds on the values of the utility function are not necessary for rational decision making n Utilities can be arbitrary values (as long as they are finite) © Manfred Huber 2018 18

Utility vs. Money n Money does not behave like a utility for humans n

Utility vs. Money n Money does not behave like a utility for humans n Humans preferences react to “risk” as well as money From Russell and Norvig © Manfred Huber 2018 19

Utility vs. Money n That humans do not choose actions that maximize the expected

Utility vs. Money n That humans do not choose actions that maximize the expected amount of money implies either: n n Human preferences are not rational (i. e. People are irrational) Human preferences are not based solely on money (i. e. People are rational but money is not the utility function used) Artificial Intelligence usually prefers the second option © Manfred Huber 2018 20