Towards Realistic Models for Evolution of Cooperation LIK

  • Slides: 42
Download presentation
Towards Realistic Models for Evolution of Cooperation LIK MUI

Towards Realistic Models for Evolution of Cooperation LIK MUI

… about procedure … • Briefly go over the paper – Clarify major points

… about procedure … • Briefly go over the paper – Clarify major points • Describe simulations (not in paper)

Road. Map • • Introduction Cooperation Models Simulations Conclusion

Road. Map • • Introduction Cooperation Models Simulations Conclusion

Evolution of Cooperation • Animals cooperate • Two questions: – How does cooperation as

Evolution of Cooperation • Animals cooperate • Two questions: – How does cooperation as a strategy becomes stable evolutionarily? – How does cooperation arise in the first place?

Darwinian Natural Selection “Survival of the fittest” • If evolution is all about individual

Darwinian Natural Selection “Survival of the fittest” • If evolution is all about individual survival, how can cooperation be explained? • Fittest what?

Fittest what ? • Individual – Rational agency theory (Kreps, 1990) • Group –

Fittest what ? • Individual – Rational agency theory (Kreps, 1990) • Group – Group selection theory (Wilson, 1980) • Gene – Selfish gene hypothesis (Dawkins, 1979) • Organization – Classic organizational theory (Simon, 1969)

Road. Map • Introduction • Cooperation Models • • • Group Selection Kinship Theory

Road. Map • Introduction • Cooperation Models • • • Group Selection Kinship Theory Direct Reciprocity Indirect Reciprocity Social Learning • Simulations • Conclusion

Group Selection • Intuition: we ban cannibalism but not carnivorousness • Population/species: basic unit

Group Selection • Intuition: we ban cannibalism but not carnivorousness • Population/species: basic unit of natural selection • Problem: explain war, family feud, competition, etc.

Kinship Theory I • Intuition: nepotism • Hamilton’s Rule: – Individuals show less aggression

Kinship Theory I • Intuition: nepotism • Hamilton’s Rule: – Individuals show less aggression and more cooperation towards closer kin if rule is satisfied – Basis for most work on kinship theory • Wright’s Coefficient of Related: r – Self: r=1 – Siblings: r=0. 5 – Grandparent-grandchild: r=0. 25

Kinship Theory II • Cannot explain: – Competition in viscuous population – Symbioses –

Kinship Theory II • Cannot explain: – Competition in viscuous population – Symbioses – Dynamics of cooperation

Direct Reciprocity • Intuition: being nice to others who are nice • “Reciprocal Altruism”

Direct Reciprocity • Intuition: being nice to others who are nice • “Reciprocal Altruism” – Trivers (1971) • Tit-for-tat and PD tournament – Axelrod and Hamilton (1981) • Cannot explain: – We cooperate not only with people who cooperate with us

Indirect Reciprocity • Intuition: respect one who is famous • Social-biological justifications – Biology:

Indirect Reciprocity • Intuition: respect one who is famous • Social-biological justifications – Biology: generalized altruism (Trivers, 1971, 1985) – Sociobiology: Alexandar (1986) – Sociology: Ostrom (1998) • 3 types of indirect reciprocity: – Looped – Observer-based – Image-based

Indirect Reciprocity: Looped • Looped Indirect Reciprocity – Boyd and Richerson (1989)

Indirect Reciprocity: Looped • Looped Indirect Reciprocity – Boyd and Richerson (1989)

Indirect Reciprocity: Observers • Observer-based Reciprocity – Pollock and Dugatkin (1992)

Indirect Reciprocity: Observers • Observer-based Reciprocity – Pollock and Dugatkin (1992)

Indirect Reciprocity: Image • Image (reputation) based Reciprocity – Nowak and Sigmund (1998, 2000)

Indirect Reciprocity: Image • Image (reputation) based Reciprocity – Nowak and Sigmund (1998, 2000)

Social Learning • Intuition: imitate those who are successful • Cultural transmission – Boyd

Social Learning • Intuition: imitate those who are successful • Cultural transmission – Boyd and Richerson (1982) • Docility – Simon (1990, 1991)

Critiques of Existing Models • Many theories each explaining one or a few aspects

Critiques of Existing Models • Many theories each explaining one or a few aspects of cooperation • Unrealism of model assumptions

Unrealism for Existing Models • asexual, non-overlapping generations • simultaneous play for every interaction

Unrealism for Existing Models • asexual, non-overlapping generations • simultaneous play for every interaction – c. f. , Abell and Reyniers, 2000 • dyadic interactions • mostly predetermined behavior – c. f. , May, 1987 (lack of modeling stochasticity) • discrete actions (cooperate or defect) • social structure and cooperation – c. f. , Simon, 1991; Cohen, et al. , 2001 • extend social learning – c. f. , Simon, 1990

Road. Map • Introduction • Cooperation Models • Simulations • Nowak and Sigmund Game

Road. Map • Introduction • Cooperation Models • Simulations • Nowak and Sigmund Game • Prisoner’s Dilemma Game • Simon’s Docility Hypothesis • Conclusion

Nowak and Sigmund Game • Payoff Matrix C = 0. 1 B = 1.

Nowak and Sigmund Game • Payoff Matrix C = 0. 1 B = 1. 0 Interact Not interact Donor -C 0 Recipient B 0 Interact Not interact Donor A -A Recipient 0 0 • Image Adjustment A=1

Using Global Image: 1 Run

Using Global Image: 1 Run

Using Global Image: 100 Runs

Using Global Image: 100 Runs

Dynamics using Global Reputation

Dynamics using Global Reputation

Using 10 Observers/Interactions

Using 10 Observers/Interactions

Evolutionary PD Game • Repeated Prisoners’ Dilemma Game • Agent Actions: Action = {

Evolutionary PD Game • Repeated Prisoners’ Dilemma Game • Agent Actions: Action = { cooperate, defect } • Payoff Matrix: C D C 3/3 0/5 D 5/0 1/1

PD Game Agent Strategies • All defecting (All. D) • Tit-for-tat (TFT) • Reputational

PD Game Agent Strategies • All defecting (All. D) • Tit-for-tat (TFT) • Reputational Tit-for-tat (RTFT): using various notions of reputation

Base Case: PD Game

Base Case: PD Game

Simple Groups: social structures • Group structure affects members – Interactions, observations, and knowledge

Simple Groups: social structures • Group structure affects members – Interactions, observations, and knowledge – Persistent structure • Groups actions – Observed indirectly through member's actions

Group Membership • Member agents – Have public group identity – Directly associated with

Group Membership • Member agents – Have public group identity – Directly associated with one environment • Group Structure is a Tree – Least common ancestral node (LCAN) – Events occur with respect to a shared environment

Shared Environment Example Agents A 1, A 2 A 3, A 4 A 5,

Shared Environment Example Agents A 1, A 2 A 3, A 4 A 5, A 2 A 1, A 3 A 5, A 3 Group G 1 G 2 G 1 G 0

PD Game with Group Reputation (varying encounters per generation EPG)

PD Game with Group Reputation (varying encounters per generation EPG)

PD Game with Group Reputation (100 EPG; varying Inter-group interaction probability)

PD Game with Group Reputation (100 EPG; varying Inter-group interaction probability)

Groups/Organizations: bounded rationality explanation • Docility – Cooperation (altruism) as an explanation for the

Groups/Organizations: bounded rationality explanation • Docility – Cooperation (altruism) as an explanation for the formation of groups/organizations • Why individuals “identify” with a group? – boundedly rational individuals – increase their survival fitness (Simon, 1969, 1990, 1991)

PD Game with Docility (50 cooperators and 50 defectors; 100 EPG; 1. 0 IP)

PD Game with Docility (50 cooperators and 50 defectors; 100 EPG; 1. 0 IP)

Conclusion • Reviewed 5 major approaches to study evolution of cooperation • Provided 2

Conclusion • Reviewed 5 major approaches to study evolution of cooperation • Provided 2 main critiques for existing models • Constructed model extensions addressing the critiques

Implications for Computer Science • Artificial intelligence – Benevolent agents are not good enough

Implications for Computer Science • Artificial intelligence – Benevolent agents are not good enough (c. f. , multi-agents systems) – Learning theory can be used to study evolution of cooperation • Systems – Improve system design by understanding the dynamics of agents – Accountability substrate needed for distributed systems

Future Plan • • Extend the simple group social structure Overlapping generations Sexual reproduction

Future Plan • • Extend the simple group social structure Overlapping generations Sexual reproduction Extend social learning using realistic/robust learning model

Modeling Diploid Organisms

Modeling Diploid Organisms

Modeling Diploid Organisms

Modeling Diploid Organisms

Modeling Diploid Organisms Parental Chromosomes One of 2 Child Chromosomes

Modeling Diploid Organisms Parental Chromosomes One of 2 Child Chromosomes

Simulation Demo • Recall PD payoff matrix: C D R/R S/T T/S P/P •

Simulation Demo • Recall PD payoff matrix: C D R/R S/T T/S P/P • PD strategies: viewed as a probability vectors – – – Strategy: TFT: All. D: All. C: STFT: <PI, PT, PR, PP, PS> < 1, 1, 1, 0, 0 > < 0, 0, 0 > < 1, 1, 1 > < 0, 1, 1, 0, 0 >

Simulation: a search problem • Search Optimal PD Strategy – Search space: I, T,

Simulation: a search problem • Search Optimal PD Strategy – Search space: I, T, R, P, S probabilities