Intelligent Software Agents 1 What is Agent Definition

Intelligent Software Agents 1

What is Agent • Definition: – Something that acts—Webster • Basic Working Definition – An autonomous entity that can interact with its environment. • Software Agent – An autonomous software entity that can interact with its environment. 2

What is an (Intelligent) Agent? • Anything that can be viewed as perceiving its environment through sensors and acting upon that environment through its effectors to maximize progress towards its goals. • Task-specific & specialized: well-defined goals and environment • The notation of an agent is meant to be a tool for analyzing systems. Agent = Architecture + Program 3

Intelligent Agents • An Agent is anything that interact and perceives with environments through sensors and effectors • To be useful, agents should —to some extent—be autonomous, adaptive, and communicative. 4

Software Agents • They are agents implemented using software that – perceives its environment through its sensors, and – acts upon an environment with effectors. • They can interact with other kinds of entities—including humans, machines, and other software agents in various environments and platforms. • Function CALLs and object messages invoke operations— they do not involve perception. • Software perception involves receiving external messages (e. g. , requests, events, queries) on which the agent can choose to act. 5

Intelligent Agents and Artificial Intelligence • Human mind as network of thousands or millions of agents all working in parallel. To produce real artificial intelligence, this school holds, we should build computer systems that also contain many agents and systems for arbitrating among the agents' competing results. • Distributed decision-making and control • Challenges: – Action selection: What next action to choose – Conflict resolution 6

Definitions of Intelligent Agent • “Intelligent agents are software entities that carry out some set of operations on behalf of a user or another program, with some degree of independence or autonomy and in so doing, employ some knowledge or representation of the user’s goals or desires. ” (“The IBM Agent”) • Autonomous agents are computational systems that inhabit some complex dynamic environment, sense and act autonomously in this environment and by doing so realize a set of goals or tasks for which they are designed (Maes) 7

More Definitions • A persistent software entity dedicated to a specific purpose. “Persistent” distinguishes agents from subroutines; agents have their own ideas about how to accomplish tasks, e. g. , their own agenda. “Special purpose” distinguishes them from entire multifunction applications; agents are typically much smaller” (Smith) • An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors (Russell and Norvig) 8

More Definitions • Intelligent agents continuously perform three functions: perception of dynamic conditions in the environment; action to affect conditions in the environment; and reasoning to interpret perceptions, solve problems, draw inferences, and determine actions (Hayes-Roth, 1995) • An agent is a computer system that is situated in some environment, and that is capable of autonomous action in this environment to meet its design objectives. Wooldridge (1999). 9

Why Intelligent Agents? Information Overload • Data doubles annually in large enterprises – Can analyze only about 5% – Most efforts: discover patterns, not meaning, not what to do – Reduces decision making capabilities by 50% • Much caused by the Internet/Web – How to filter data – How to identify relevant sources of data • Intelligent agents can assist searching • Save time: agents decide what is relevant to the user 10

Reasons for Intelligent Agent Technology Growth • • • Decision support Front-line decision support Repetitive office activity Mundane personal activity Search and retrieval Domain experts • The agents can advise, alert, broadcast, browse, critique, distribute, empower, explain, filter, monitor, negotiate understand what we mean, find and use relevant information, automate routine tasks • adapt to our needs and desires, cooperate with other assistants 11

How is an Agent different from other software? • Agents are autonomous, that is they act on behalf of the user • Agents contain some level of intelligence, from fixed rules to learning engines that allow them to adapt to changes in the environment • Agents have social ability, that is they communicate with the user, the system, and other agents as required • Agents may also cooperate with other agents to carry out more complex tasks than they themselves can handle • Agents may migrate from one system to another to access remote resources or even to meet other agents 12

Objects vs agents • Objects – – state methods control method invocation • Agents – – goals knowledge planning negotiation 13

14

Agents Can Be Implemented In Many Ways 15

The Right Thing = The Rational Action • Rational Action: The action that maximizes the expected value of the performance measure given the percept sequence to date – Rational = Best ? – Rational = Optimal ? – Rational = Omniscience ? – Rational = Successful ? 16

The Right Thing = The Rational Action • Rational Action: The action that maximizes the expected value of the performance measure given the percept sequence to date – Rational = Best Yes, to the best of its knowledge – Rational = Optimal Yes, to the best of its abilities (incl. its constraints) – Rational Omniscience – Rational Successful 17

Definition of ideal rational agent Ideal Rational Agent: For each possible percept sequence, such an agent does whatever action is expected to maximize its performance measure, on the basis of the evidence provided by the percept sequence and whatever built-in knowledge the agent has. What do you think? Is this an acceptable definition? Not looking left when crossing the street: If I don’t see a car coming from the left, it is rational to cross the street? No. Should also consider taking information gathering actions. Limited/costly computation time Bounded rationality Limited/costly memory … 18

Rationality – do the right thing What is “rational” depends on: – Performance measures – Prior knowledge of the environment – Actions that the agent can perform – Percept sequence: everything the agent has perceived so far 19

Behavior and Performance of AIs • Perception (sequence) to Action Mapping: – Ideal mapping: specifies which actions an agent ought to take at any point in time • Performance measure: a subjective measure to characterize how successful an agent is (e. g. , speed, power usage, accuracy, money, etc. ) • (degree of) Autonomy: to what extent is the agent able to make decisions and actions on its own? 20

WARNING: Might not get what you ask for in the performance measure • Cleaning robot – Pick up as much trash as possible • Vehicle route optimization – Maximize utilizations => Driving fully loaded – Capitalizing on oddities in tariff list => Renegotiation – Don’t include solution method in the criterion 21

Intelligence Levels and Power Level 0: Straight orders Level 1: User initiated search by key words (search engines) Level 2: Have user profiles (software agents) Level 3: Have learning and deductive capabilities (learning or truly intelligent agents) 22

Possible Components of an Agent • • Owner Author Account Goal Subject description Creation and duration Background: (supporting information) Intelligent subsystem: (rule-base system) 23

24

Classification by Characteristics • Agency: a degree of autonomy of and authority vested in an agent and can be measured at least qualitatively by the nature of the interaction b/w the agents and the other entities of the system. At minimum an agent must run asynchronously. • Intelligence: is the degree of the reasoning and the learned behavior. Includes user model, planning, discover relationship, connections…. . • Mobility: is the degree to which the agents themselves travel through the network. – Static – Mobile scripts – Mobile with state 25

Systems & information integration Resource markets Multiagent systems Internet commerce Common metaphore? Recommender systems Agent-based systems Economics Software agents Assistant agents Mobile agents Sociology Simulated agents Robots Agents Artificial life Believable agents People. . . 26

Personal assistant agents • Reducing work and information overload • Examples – Apple’s Knowledge Navigator – MIT Media Lab (various) – Microsoft Paper Clip – Office of Naval Research Tactical Picture Agent 27

Mobile agents • Moving running software between computers • Purpose – overcome network latency (e. g. interact in real time) – use located resources (e. g. computing power, storage, special hardware) – see e. g. Seven Good Reasons for Mobile Agents by Lange and Oshima, CACM 3(42) • Problems – Security, trust, performance 28

AN AGENT SYSTEM • Agent System = <Agents, Environment, Coupling> • Agent = <State, Input, Output, Process> • State is the set of properties (values, true propositions) that completely describes the agent. • Input and Output are subsets of State whose variables are coupled to the environment. • Process is an autonomously executing process that changes the agent’s State. 29

• Environment = <State, Process> • The Environment has its own Process that can change its state, independent of the actions of its embedded agents. • Coupling is a mapping of an agent’s Input and Output from/to the Environment’s State. 30

PEAS • PEAS: Performance measure, Environment, Actuators, Sensors • Must first specify the setting for intelligent agent design • Consider, e. g. , the task of designing an automated taxi driver: – Performance measure – Environment 31

PEAS • Must first specify the setting for intelligent agent design • Consider, e. g. , the task of designing an automated taxi driver: – Performance measure: Safe, fast, legal, comfortable trip, maximize profits – Environment: Roads, other traffic, pedestrians, customers – Actuators: Steering wheel, accelerator, brake, signal, 32

PEAS • Agent: Medical diagnosis system • Performance measure: Healthy patient, minimize costs, lawsuits • Environment: Patient, hospital, staff • Actuators: Screen display (questions, tests, diagnoses, treatments, referrals) • Sensors: Keyboard (entry of symptoms, findings, patient's answers) 33

PEAS • Agent: Part-picking robot • Performance measure: Percentage of parts in correct bins • Environment: Conveyor belt with parts, bins • Actuators: Jointed arm and hand • Sensors: Camera, joint angle sensors 34

PEAS • Agent: Interactive English tutor • Performance measure: Maximize student's score on test • Environment: Set of students • Actuators: Screen display (exercises, suggestions, corrections) • Sensors: Keyboard 35

Environment types • Fully observable (vs. partially observable): An agent's sensors give it access to the complete state of the environment at each point in time. • Deterministic (vs. stochastic): The next state of the environment is completely determined by the current state and the action executed by the agent. (If the environment is deterministic except for the actions of other agents, then the environment is strategic) • Episodic (vs. sequential): The agent's experience is divided into atomic "episodes" (each episode consists of the agent perceiving and then performing a single action), and the 36

Environment types • Static (vs. dynamic): The environment is unchanged while an agent is deliberating. (The environment is semidynamic if the environment itself does not change with the passage of time but the agent's performance score does) • Discrete (vs. continuous): A limited number of distinct, clearly defined percepts and actions. • Single agent (vs. multiagent): An agent operating 37

Examples of agents in different types of applications Agent type Percepts Actions Goals Environment Medical diagnosis system Symptoms, findings, patient's answers Questions, tests, treatments Healthy patients, minimize costs Patient, hospital Satellite image analysis system Pixels of varying intensity, color Print a categorization of scene Correct categorization Images from orbiting satellite Part-picking robot Pixels of varying intensity Pick up parts and sort into bins Place parts in correct bins Conveyor belts with parts Refinery controller Temperature, pressure readings Maximize purity, yield, safety Refinery Open, close valves; adjust temperature Interactive English tutor Print exercises, suggestions, corrections Maximize student's score on test Set of students Typed words 38

40

AGENT PROCESSING OVERVIEW • The primary purpose of an agent: • is not just to interact with the environment, • but rather to process and interpret the perceived information and to achieve some goal(s). 41

42

AGENTS CAN MULTIPROCESS • encourages concurrency • minimizes single point of failure • processing threads can communicate directly – provides fast reaction capability – causes degradation when there is too much • intercommunication 43

• coordinators can be added to manage and • control multiple threads – reduces the need to embed all coordination • functionality in every thread – increases processing overhead 44

Agent Types • Four basic types in order of increasing generality – simple reflex agents – model based reflex agents (with state) – goal-based agents – utility-based agents • How to convert all these into learning agents 45

Skeleton agent function SKELETON-AGENT (percept) returns action static: memory, the agent’s memory of the world memory UPDATE-MEMORY(memory, percept) action CHOOSE-BEST-ACTION(memory) memory UPDATE-MEMORY(memory, action) return action On each invocation, the agent’s memory is updated to reflect the new percept, the best action is chosen, and the fact that the action was taken is also stored in the memory. The memory persists from one invocation to the next. Input = Percept, not history NOTE: Performance measure is not part of the agent 46

1. Table-driven agent function TABLE-DRIVEN-AGENT (percept) returns action static: percepts, a sequence, initially empty table, a table, indexed by percept sequences, initially fully specified append percept to the end of percepts action LOOKUP(percepts, table) return action An agent based on a prespecified lookup table. It keeps track of percept sequence and just looks up the best action • Problems – Huge number of possible percepts (consider an automated taxi with a camera as the sensor) => lookup table would be huge – Takes long time to build the table – Not adaptive to changes in the environment; requires entire table to 47 be updated if changes occur

Simple Reflex Agents A simple reflex agent works by finding a rule whose condition matches the current situation (as defined by the percept) and then doing the action associated with that rule. Rectangles: agent’s internal state Ovals: background information • Implementation: easy; Applicability: narrow 48

• function SIMPLE-REFLEX-AGENT(percept) returns action • static: rules, a set of condition-action rules • • state INTERPRET-INPUT (percept) rule RULE-MATCH (state, rules) action RULE-ACTION [rule] return action 49

Simple reflex agent… • • Table lookup of condition-action pairs defining all possible condition-action rules necessary to interact in an environment • e. g. if car-in-front-is-breaking then initiate breaking Problems – Table is still too big to generate and to store (e. g. taxi) – Takes long time to build the table – No knowledge of non-perceptual parts of the current state – Not adaptive to changes in the environment; requires entire table to be updated if changes occur – Looping: Can’t make actions conditional 50

Model-Based Reflex Agents (with state) • Sensor information alone is not sufficient in case of partial observability • Need to keep track of how the world evolves – Evolution: independently of the agent, or caused by the agent’s action – Knowledge about how the world works – Model of the world 51

• function REFLEX-AGENT-WITH-STATE (percept) returns action • static: state, a description of the current world state • rules, a set of condition-action rules • • • state UPDATE-STATE (state, percept) rule RULE-MATCH (state, rules) action RULE-ACTION [rule] state UPDATE-STATE (state, action) return action 52

Reflex agent with internal state … • Encode “internal state of the world to remember the past as contained in earlier percepts • Needed because sensors do no usually give the entire state of the world at each input, so perception of the environment is captured over time. “State” used to encode different “world states” that generate the same immediate percept • Requires ability to represent change in the world with/without the agent • one possibility is to represent just the latest state, but then cannot reason about hypothetical courses of action • Example: Rodney Brook’s Subsumption Architecture. Main idea: build complex intelligent robots by decomposing behaviors into a hierarchy of skills, each completely defining a complete percept-action cycle for one very specific task. For example, avoiding contact, wandering, exploring, recognizing doorways, etc. Each behavior is modeled by a finite-state machine with a few states (though each state may correspond to a complex function or module). Behaviors are loosely-coupled, asynchronous interactions 53

Goal-Based Agents • State and actions don’t tell where to go • Need goals to build sequences of actions (planning) • Goal-based: uses the same rules for different goals • Reflex: will need a complete set of rules for each goal 54

Agent with explicit goals … • Choose actions so as to achieve a (given or computed) goal = a description of desirable situations. e. g. where the taxi wants to go • Keeping track of the current state is often not enough – need to add goals to decide which situations are good • Deliberative instead of reactive • May have to consider long sequences of possible actions before deciding if goal is achieved – involves considerations of the future, “what will happen if I do…? ” (search and planning) • More flexible than reflex agent. (e. g. rain / new destination) In the reflex agent, the entire database of rules would have to be rewritten 55

Utility-Based Agents • Several action sequences to achieve some goal (binary process) • Need to select among actions and sequences (preferences) • Utility: state real number – express degree of satisfaction and specify trade-offs between conflicting goal 56

Learning Agents • Learning element: making improvements • Performance element: selecting external actions (entire former agents) • Critic: collecting feedback on how the agent is doing? • Problem generator: suggesting (exploratory) actions (experiments) 57

Classification by Application Area • • • Assist in workflow and administrative management Collaborate with other agents and individuals Support electronic commerce Support desktop applications Assist in information access and management Process mail and messages Control and manage the network access Manage systems and networks Create user interfaces 58

Internet-based Software Agents (Software Robots or Softbots) Major Categories • • • E-mail agents (mailbots) Web browsing assisting agents Frequently asked questions (FAQ) agents Intelligent search (or Indexing) agents Internet softbot for finding information Network Management and Monitoring 59

• E-mail agents (mailbots): monitors what the user does routinely with the e-mail and memorizes the user’s situationaction pairs. Prioritizing e-mail messages based on users’ preferences. • Web browsing assisting agents: facilitates the web browsing by offering the user a tour of the internet. Monitors the user activity and collects information about the user behavior. (Web Watcher) • Frequently asked questions (FAQ) agents: uses the text of the question to locate the answer. Because of the limited FAQ agents and semistructuredness of the questions, the reliability is high. 60

• Intelligent search (or Indexing) agents: traverse the web and perform tasks like information retrieval discovery. Normally uses Metasearch Engines. • Internet softbot for finding information: what the user wants and understand the contents of the information services. They are only used for structured information (simple model). 61

Network Management and Monitoring • Patrol Application Management: monitor & manage application, databases, middleware & underlay N/W resources • Tabriz: allow agents to perform tasks proactively. (mobile agents) • Watch. Guard: internet firewall. • Alert. View: monitor n/w resources, databases & e-mail system for threshold violations. • Inter. Ap: search the web and automate file transfer. 62

Mobile agents • • • Programs that can migrate from one machine to another. Execute in a platform-independent execution environment. Require agent execution environment (places). Mobility not necessary or sufficient condition for agenthood. Practical but non-functional advantages: – Reduced communication cost (e. g. , from PDA) – Asynchronous computing (when you are not connected) • Two types: – One-hop mobile agents (migrate to one other place) – Multi-hop mobile agents (roam the network from place to place) • Applications: – Distributed information retrieval. – Telecommunication network routing. 63

• • Information Agents Manage the explosive growth of information. Manipulate or collate information from many distributed sources. Information agents can be mobile or static. Examples: – Bargain. Finder comparison shops among Internet stores for CDs – FIDO the Shopping Doggie (out of service) – Internet Softbot infers which internet facilities (finger, ftp, gopher) to use and when from high-level search requests. • Challenge: ontologies for annotating Web pages (eg, SHOE). 64

65

66

67

68