Lecture 1 Introduction CS 51106110 Multi Agent Systems

Lecture 1: Introduction CS 5110/6110 Multi. Agent Systems Slides adapted from Sobah Abbas Petersen (with permission)

• • Design Systems to Act for Groups of Humans Cooperation Task Sharing Decision Making Group Decisions Forming Coalitions Allocating scarce resources Bargaining Arguing 2

Consider a Smart City • Traffic delays take time and increase pollution • Personal GPS systems can direct users to take routes which spread traffic out based on current congestion. Some drivers may be directed to slower routes but community benefits. • The “city” has the ability to change speed limits, direction of traffic, change access permissions, or limit turns dynamically. • Drivers may want to query system to find best time to travel or best route to a group of destinations. 3

Visit with your Neighbor • Come up with two interesting aspect of this problem which deal with multi-agent issues: – – – – Cooperation Task Sharing Decision Making Group Decisions Forming Coalitions Allocating scarce resources Bargaining Arguing 4

Utilities • When we are directing a computer how to make decisions, we must create a system by which it can tell which results we prefer to others. • Utilities assign a numerical value to various decisions/results to communicate desirability. 5

Uber • Replacement for Taxis • App to request order. Told cost, when driver will arrive, what s/he will be driving • Surge pricing is used to attract more drivers during high demand. • Anyone can drive – if pass background check. • track the car on their smartphone. Email receipt. Paid automatically by credit card. • No cab at airport for hours hoping for long ride 6

In groups • Identify decisions the system must make • What utilities are assigned to various decisions? 7

Voting • In the US presidential election, the rules are very complicated. Delegates are selected. They may or may not be bound. • Winner is picked based on electoral college (which may be winner take all or plurality). If no one gets the majority (270), election goes to the house. 8

(2016) Electoral votes by state: 9

Voting Power Wyoming: 142741 people per electoral vote (highest power) New York: 519, 075 people per electoral vote (lowest power) 10

Notes • Use Piazza. A good way for you to ask questions of me and the class. Also a good way to share things you’ve discovered. • Find articles on digital library: https: //dl. acm. org/ • http: //ieeexplore. ieee. org/Xplore/home. jsp 11

Look at first programming assignment. Tutors: Justin, Landon, Mc. Kade, Pravallika, Steve. S 12

Consider a simpler problem • Suppose, the CS department has $100 K to give out in one time salary increase. • Suppose, a group of students has been tasked with giving out raises. • How would you decide who gets what? • Working in groups come up with a fair way. 13

Project • Hopefully these sample problems have piqued your interest. • Start on the process of picking a topic for your project. • What has been your experience of researching topics? What works for you? 14

Programming has progressed through • procedures & functions (no context) • abstract data types (data and operations combined) Reduces seen details • objects – data abstraction, encapsulation, modularity, polymorphism, and inheritance • Define those terms… • to agents 15

• The word 'agent' is widely used. Everyone knows what it means. We have press agents, travel agents, real estate agents, FBI agents, secret agents, double agents. Johnny Depp has an agent. Sometimes the word 'broker' refers to a kind of agent, for example, a stock broker. • But to actually define what the word, agent, means is not so easy. (Similar problems occur with other concepts such as intelligence, or life. ) There seem to be almost as many definitions of agent, or agency as there are people trying to define these concepts! • So we will look at a few of these attempts at definition. Take your pick or make up your own. 16

Why Agents? When a space probe makes its long flight from Earth to the outer planets, a ground crew is usually required to continually track its progress and decide how to deal with unexpected occurrences. • Expensive • if quick decisions needed, not practical • Nasa: make probes more autonomous ** difference between parallel and multiagent 17

Recent news blurb: • Steve Chien and Kiri Wagstaff from NASA's Jet Propulsion Laboratory think that machines will also have to learn as they go, adapting to what they find beyond the reaches of our most powerful telescopes. • "By making their own exploration decisions, robotic spacecraft can conduct traditional science investigations more efficiently and even achieve otherwise impossible observations, such as responding to a short-lived plume at a comet millions of miles from Earth, " write the researchers. • One example they give is AI that can tell the difference between a storm and normal weather conditions on a distant planet, making the readings that are being taken much more useful to scientists back home. 18

• Just like Google uses AI to recognize dogs and cats in photos, an explorer buggy could learn to tell the difference between snow and ice, or between running water and still water, adding extra value and meaning to the data it gathers. • The researchers suggest AI-enabled probes could reach as far as Alpha Centauri, some 4. 24 light-years away from Earth. Communications across that distance would be received by the generation after the scientists who launched the mission in the first place, so giving the probe a mind of its own would certainly speed up the decision-making process. • The next generation of AI robots will have to be able to detect "features of interest", detect unforeseen features, process and analyze data, and adapt their original plans where necessary, say the researchers. 19

Definitions of Agents. . . IBM: ”Intelligent agents are software entities that carry out some set of operations on behalf of a user or another program with some degree of independence or autonomy, and in doing so, employ some knowledge or representations of the user’s goals or desires”. Consider the Uber agent: How does definition apply? 20

Definitions of Agents. . . • An agent is autonomous: capable of acting independently, exhibiting control over its internal state. Ø An agent is a computer system capable of autonomous action in some environment. System Input Output Environment 21

• “An agent is a persistent software entity dedicated to a specific purpose. 'Persistent' distinguishes agents from subroutines; agents have their own ideas about how to accomplish tasks, their own agendas. 'Special purpose' distinguishes them from entire multifunction applications; agents are typically much smaller. (Smith, Cypher and Spohrer 1994) • Consider smart city app: what could the specific purpose be? 22

• It is widely recognized that interaction is probably the most important single characteristic of complex systems. • “Agents are programs that engage in dialogs, [and] negotiate and coordinate transfer of information. ” (Michael Cohen) • What is the difference between negotiate and coordinate? 23

• “Agent enjoys the following properties: • autonomy - agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state; • social ability - agents interact with other agents (and possibly humans) via some kind of agent-communication language; • reactivity: agents perceive their environment and respond in a timely fashion to changes that occur in it; • pro-activeness: agents do not simply act in response to their environment, they are able to exhibit goal-directed behaviour by taking initiative. ” (Wooldridge and Jennings, 1995) 24

Properties of Agents • Mentalistic notions, such as beliefs and intentions are often referred to as properties of strong agents. • Other properties are: – Mobility: the ability of an agent to move around a network. – Veracity: agent will not knowingly communicate false information. – Benevolence: an agent does not have self-conflicting goals and always tries to do what is asked of it. – Rationality: an agent will act in order to achieve its goals and will not act in such a way as to prevent its goals being achieved. How do these properties differ from people? 25

Ideas for agents • List an activity you engage in which could be taken over by a software agent… 26

Stable Matching with Uncertain Pairwise Preferences: Assigning graduates to jobs (AAMAS ‘ 17) • stable matching: there is no pair of man and woman that prefer each other rather than matched to their partners • Preferences may be uncertain: unable to differentiate between partners. Probablistic • May be cycles • May not be stable matching 27

Spread of Misinformation in Online Social Networks (AAMAS ‘ 17) The popularity of online social networks comes from their incredible efficiency in information dissemination and sharing based on trust relationships built among their users. (i) recognize misinformation; (ii) identify misinformation sources; given graph G and the set A of infected nodes, find a set S of nodes having maximum probability to be the sources of infection; (iii) limit the diffusion of misinformation. 28

Smart Thermostat (AAMAS ‘ 17) • Save Money or Feel Cozy? A Field Experiment • The smart thermostat enables the users to manage their heating given real-time prices. Moreover, our machine learning-based thermostats outperform a baseline without machine learning in terms of usability. • Analyze users’ economic behavior, including their reaction to price changes, their price sensitivity, and their comfortcost trade-offs. • Reduce the average energy consumption during peak hours by 38% 29

Overview • Five ongoing trends have marked the history of computing: 1. Ubiquity – everywhere at once – even more so with smart phones. • Reduction in the cost of computing capability 2. Interconnection • Computer systems are networked into large distributed systems 3. Intelligence (MAS) • The complexity of tasks that can be automated and delegated to computers 4. Delegation (MAS) • Judgment of computer systems are frequently accepted 5. Human-orientation • Use human concepts and metaphors to describe computers. Examples? 30

Concerns • Need for computer systems to act in our best interests • “The issues addressed in Multiagent systems have profound implications for our understanding of ourselves. ” Wooldridge 31

Trends… • These trends present major challenges to software developers. e. g. – Delegation – act independently. – Intelligence – act in a way that represents our best interests while interacting with other humans or systems. Ø Need systems that can act effectively on our behalf. • Systems must have the ability to cooperate and reach agreements with other systems. èImportant field: Multi-agent Systems 32

Two Key Problems 1. How do we build agents that are capable of independent, autonomous action in order to successfully carry out the tasks that we delegate to them? (Micro aspects) 2. How do we build agents that are capable of interacting (cooperating, coordinating, negotiating) with other agents in order to successfully carry out the tasks we delegate to them? (Macro aspects) 33

Problems • How many of you play games to take advantage of the novice? • What if the other agent is as smart as you? • What if YOU are the one being taken advantage of? 34

Example – Market for Lemons • Suppose the seller has ten cars of varying quality. Only the seller knows its true value. The buyer cannot tell by looking. • Incentives exist for the seller to pass off a low-quality good as a higher-quality one. • But buyer is as smart as the seller!!! • The buyer, however, takes this incentive into consideration, and takes the quality of the good to be uncertain. Only the average quality of the good will be considered, which in turn will have the side effect that goods that are above average in terms of quality will be driven out of the market. This mechanism is repeated until a no-trade equilibrium is reached. • Note the different perspectives of the negotiation 35

Cars range from 1 -10 in value (Only the seller knows value) Buyer decides only to offer average price (5. 5) Sellers of cars valued over 5. 5 decide not to sell. 36

Cars range from 1 -5 in value (Only the seller knows value) Buyer decides only to offer average price (3) Sellers of cars valued over 3 decide not to sell. 37

Cars range from 1 -3 in value (Only the seller knows value) Buyer decides only to offer average price (2) Sellers of cars valued over 2 decide not to sell. 38

Cars range from 1 -2 in value (Only the seller knows value) Buyer decides only to offer average price (1. 5) Sellers of cars valued over 1. 5 decide not to sell. 39

Cars range from 1 in value (Only the seller knows value) Buyer decides only to offer average price (1) Sellers decides to sell as that is a fair price. 40

Fields that inspired agents • Artificial Intelligence – Agent intelligence, micro aspects • Software Engineering – Agent as an abstraction • Distributed systems and Computer Networks – Agent architectures, multi-agent systems, coordination There are many definitions of agents – often too narrow or too general. 41

Agents and Objects • • • Remember the previous slide: procedures & functions (no context) abstract data types (data and operations combined) objects to agents • Are agents just objects by another name? 42

Agents and Objects do it for free… • Agents do it because they want to! • Agents do it for money! 43

Agents and Objects. . . Main differences: – Agents are autonomous: agents embody a stronger notion of autonomy than objects, in particular, agents decide for themselves whether or not to perform an action. – Agents are smart: capable of flexible (reactive, pro-active social) behaviour; standard object models do not have such behaviour. – Agents are active: a multi-agent system is inherently multithreaded in that each agent is assumed to have at least one thread of active control. 44

Need a pairing of agents (marriage). Assume each agent has a score (1 -10) for how it feels about the other agents in the system. The higher the score, the happier the agent will be in that relationship. • What are the differences in programming a centralized solution for all the agents – versus a distributed solution? • Consider – difficulty of programming – computational complexity – global utility versus local utility – communication bottlenecks 45

Why agents? • We have a distributed environment that cannot be completely specified due to dynamism– open environments. • Former paradigms, such as OOP, cannot completely satisfy our needs: – They were designed for constructing systems in a completely specified environment - a closed world. 46

In the following examples, what is “open”? • • navigation Face recognition travel agent (cheaptickets. com) Resource allocation 47

Open Environment • Navigation– environment can change • Going to Wal. Mart in an open environment • How can we work in an open environment? 48

How can we work in an Open Environment? • By copying human behaviour: – Perceive the environment – Affect the environment – Have a model of behaviour Agent Environment – Specify intentions and motivations to be fulfilled by implementing corresponding goals 49

Decentralization • An important concept – No central control; control is distributed. What is advantage? – Knowledge or information sources may also be distributed. Why? 50

Multi-agent Systems (MAS) Contains a number of agents which interact with one another through communication. The agents are able to act in an environment; where each agent will act upon or influence different parts of the environment. Reference: Wooldridge, An Introduction to Multiagent Systems Multi-agent System Environment 51

Motivation for MAS 1. To solve problems that are too large for a centralized agent 2. To allow interconnection and interoperation of multiple legacy systems 3. To provide a solution to inherently distributed problems 4. To provide solutions which draw from distributed information sources 5. To provide solutions where expertise is distributed (Why is expertise distributed? ) 6. To offer conceptual clarity and simplicity of design 7. To be able to hide strategies from competition 52

Benefits of MAS Why? • Faster problem solving • Decrease in communication • Flexibility • Increased reliability - fault tolerant • Natural to Application 53

Summary • An agent is a system that is capable of independent action on behalf of its user or owner. • A multi-agent system is one that consists of a number of agents which interact with one another. • In order to successfully interact, agents need ability to cooperate, coordinate and negotiate. 54