ICT 619 Intelligent Systems Topic 2 Expert Systems

ICT 619 Intelligent Systems Topic 2: Expert Systems ICT 619 S 2 -05

Expert Systems PART A § Introduction Applications of expert systems § Structure of an expert system § An example rule base § Reasoning in a rule-based expert system § Reasoning using forward and backward chaining § Dealing with uncertainty PART B § Developing expert systems § Frame-based expert systems § Advantages and disadvantages of expert systems § Case studies ICT 619 S 2 -05 2

Expert Systems (ES) - what they are § Intelligent systems for emulating human experts § Used as decision support tools § Can be § consistent, unbiased substitutes for human experts § repository for domain-specific knowledge § Work by § capturing human expertise in a specific area § applying deductive reasoning to infer conclusions § Not self-adaptive § can not learn by themselves, all knowledge encoded by humans ICT 619 S 2 -05 3

Why expert systems? § Problem solving knowledge often expressed as heuristics or rules of thumb § Easier to represent heuristics using rules rather than algorithms § Knowledge engineer extracts knowledge and encodes them into rules § ES store knowledge usually in a rule base § Distinction from conventional programs: § Knowledge separated from its processing. § Easier to build and maintain as change in code does not affect knowledge. ICT 619 S 2 -05 4

Some advantages of ES § First developed in the 1970’s § Currently well established as intelligent systems § Expertise available 24 hours a day § Unlike human experts, do not retire, die or resign § Provide consistent unbiased recommendations § Multiple copies can be produced and distributed § Can justify conclusions by detailing chain of reasoning followed ICT 619 S 2 -05 5

Applications of expert systems § Early successful applications in medicine § MYCIN (mid 70 s) for diagnosing and recommending treatment for blood disease § In science and engineering § DENDRAL (chemistry) and PROSPECTOR (geology) § First major commercial application § XCON (Digital Equipment Corporation 1979) § Other commercial applications in § banking and finance, eg TARA (foreign currency trading) § manufacturing § personnel management ICT 619 S 2 -05 6

Applications of expert systems (cont’d) § US public sector agencies making use of expert systems § Environmental Protection Agency § Immigration and Naturalisation Service § Postal Service § Internal Revenue Service § Department of Energy § British National Health Service ES with 11, 200 rules used to evaluate the performance of medical care providers § American Express’s Authorizer’s Assistant helps decide approval of credit card charge ICT 619 S 2 -05 7

Structure of an expert system User Interface: Knowledgebase editor - Menudriven User - GUI Knowledge-base Inference engine - Natural language Working memory Explanation sub-system Three main components: 1. The rule-base, 2. The working memory and 3. The inference engine ICT 619 S 2 -05 8

Structure of an expert system (Cont’d) § Working memory stores initial facts specific to problem at hand, intermediate conclusions and hypotheses § Inference engine combines rules in knowledge base to arrive at final conclusion § User interface § Allows user to enter relevant facts by answering questions asked by the system § Enables use of the explanation sub-system by asking why and how questions § Knowledge-base editor used to create, debug and maintain rules ICT 619 S 2 -05 9

Knowledge representation using rules § Knowledge represented in rules having the form: IF <condition> THEN < conclusion> § Left hand side (LHS) antecedent § Right hand side (RHS) consequent § Propositional logic - basis of reasoning used in rulebased expert systems § Antecedents and consequents are examples of propositions or statements in propositional logic ICT 619 S 2 -05 10

Knowledge representation using rules (cont’d) § A rule can have more than one proposition in its antecedent or consequent § For example, in the rule IF rain is forecast AND outdoor activity anticipated THEN advice is ‘take rain coat’ the antecedent consists of two propositions combined using the logical AND connective ICT 619 S 2 -05 11

An example rule-base – the mortgage loan case (Zahedi 93) § The domain expertise needed for approving a mortgage loan contains the following knowledge base: § To get a mortgage loan, § § the applicant must have a steady job, acceptable income, good credit ratings; and the property should be acceptable. § If applicant does not have a steady job, then they must have adequate assets. § § The amount of loan cannot be more than 80 percent of the property value, and the applicant must have 20 percent of the property value in cash. ICT 619 S 2 -05 12

An example rule-base – the mortgage loan case (cont’d) § The definition of a steady job: § § The definition of adequate assets: § § Applicant’s properties must be valued at ten times the amount of the loan, or the applicant must have liquid assets valued at five times the amount of the loan. An acceptable property: § § Applicant should have been at the present job for more than two years. Either located in the bank’s lending zone with no legal constraints, or is on the bank’s exception list. The definition of adequate income: § § If applicant is single, then mortgage payment must be less than 70 percent of their net income. If applicant is married, then mortgage payment must be less than 60 percent of the family net income. ICT 619 S 2 -05 13

The mortgage loan case rule-base– 1. IF the applicant has a steady job AND the applicant has adequate income AND the property is acceptable AND the applicant has good credit ratings AND the amount of loan is less than 80% of the property value AND the applicant has 20% of the property value in cash THEN approve the loan 2. IF the applicant has adequate assets AND the applicant has adequate income AND the property is acceptable AND the applicant has good credit ratings AND the amount of loan is less than 80% of the property value AND the applicant has 20% of the property value in cash THEN approve the loan ICT 619 S 2 -05 14

The mortgage loan case rule-base– 3. IF the applicant has a job AND the applicant has been more than two years at the present job THEN the applicant has a steady job 4. IF the property is in the bank’s lending zone OR the property is on exception list THEN the property is acceptable 5. IF the family income is adequate OR the single income is adequate THEN the applicant has adequate income ICT 619 S 2 -05 15

The mortgage loan case rule-base– 6. IF the applicant is married AND mortgage payment is less than 60% of the family net income THEN the family income is adequate 7. IF NOT the applicant is married AND mortgage payment is less than 70% of net income THEN the single income is adequate applicant’s 8. IF applicant has properties with a value greater than 10 times the loan OR the applicant has liquid assets greater than 5 times the loan THEN the applicant has adequate assets. ICT 619 S 2 -05 16

Reasoning in a rule-based expert system § Inference performed through deductive reasoning § Deductive reasoning § Reasoning process starts with a set of premises already proven or accepted new facts or conclusions derived based on these premises and using rules of inference, § ES combines facts and pieces of knowledge (rules) to deductively infer new knowledge as conclusions and recommendations ICT 619 S 2 -05 17

Rules of inference used in expert system reasoning § “Reasoning” based on the following rules of inference borrowed from propositional logic: § Modus ponens § Hypothetical syllogism § Modus tollens and Boolean logic: True AND True = True, True AND False = False, False AND False = False, True OR False = True, True OR True = True, False OR False = False § Modus ponens Given a rule, if the antecedent is true, conclude that the consequent is also true. IF X THEN Y X is true Conclude: Y is true ICT 619 S 2 -05 18

Rules of inference used in expert system reasoning (cont’d) § Hypothetical syllogism When the consequent of one rule is the antecedent of a second rule, then we can establish a third rule whose antecedent is that of the first rule and whose consequent is that of the second. IF X THEN Y IF Y THEN Z Conclude: IF X THEN Z § Modus tollens (indirect proof) When the negation of a fact is established, given the consequent of a rule is not true, conclude that the antecedent is not true. IF X THEN Y Y is false Conclude: X is false ICT 619 S 2 -05 19

ES Reasoning process using multiple inferencing § Multiple inferencing involves use of more than one rules for drawing a conclusion § Inference engine matches facts in the working memory with rules in the rule-base to determine which rules apply § More than one rule may match a fact All matching rules are put in a conflict set by the inference engine § Inference engine selects one rule from the conflict set and “fires” (applies) it § As a result of applying a rule, a new fact may be inferred, which is added to the working memory ICT 619 S 2 -05 20

ES Reasoning – conflict resolution § The expert system may come to a halt or repeat the match-andfire cycle depending on the latest inference § Different order of selecting rules from the conflict set may result in different outcomes § Order of rule firing is determined by meta rules : Can be § independent of problem § specific to a problem ICT 619 S 2 -05 21

ES Reasoning – conflict resolution (cont’d) § Resolution independent of problem § Fire rules in the order they appear in the rule base § Fire rule matching most recently added fact (recency) Eg, (Negnevitsky 2005) § Rule 1: IF forecast is rain [08: 16 PM 11/25/96] THEN advice is ‘take an umbrella’ Rule 2: IF weather is wet [10: 18 AM 11/26/96] THEN advice is ‘stay home’ ICT 619 S 2 -05 22

ES Reasoning – conflict resolution (cont’d) § Fire rules with a large number of conditions on the LHS (specificity) Eg, (Negnevitsky 2005), § Rule 1: IF the season is autumn AND the sky is cloudy AND the forecast is rain THEN advice is ‘stay home’ § Rule 2: IF the season is autumn THEN advice is ‘take an umbrella’ § Choice of rule may be specific to a problem Eg, Favour rules dealing with high credit risk ICT 619 S 2 -05 23

Reasoning using forward and backward chaining § Two well-known methods of multiple inferencing. § In backward chaining, § § multiple inference starts with a goal It finds the rule whose consequent matches the goal and goes backward to the antecedent part of the rule It then tries to establish the truth value of the antecedent part of the rule § It does this by establishing the truth values of the propositions in the antecedent § This is done by finding rules having consequents matching the propositions § If no such rule is found in the rule base, the user is asked to provide information to establish the truth of the propositions ICT 619 S 2 -05 24

Backward chaining in the mortgage loan example Reasoning process starts with the goal: Approve loan for an applicant § This goal is the consequent of rules (1) and (2). § Rules are tested sequentially from the beginning of the knowledge base § Rule (1) is fired, and its propositions become the current goals. § The first proposition test is whether the applicant has a steady job. § This is in the consequent of rule (3). It asks the user if the applicant has a job. If the answer is no, the inference engine does not go any further in (3). Since ‘applicant has a steady job’ is not true, it does not go any further with rule (1) either § As the goal could not be reached from rule (1), the inference engine tries rule (2) next to establish the goal. ICT 619 S 2 -05 25

The mortgage loan case rulebase – 1. IF the applicant has a steady job AND the applicant has adequate income AND the property is acceptable AND the applicant has good credit ratings AND the amount of loan is less than 80% of the property value AND the applicant has 20% of the property value in cash THEN approve the loan 2. IF the applicant has adequate assets AND the applicant has adequate income AND the property is acceptable AND the applicant has good credit ratings AND the amount of loan is less than 80% of the property value AND the applicant has 20% of the property value in cash THEN approve the loan ICT 619 S 2 -05 26

The mortgage loan case rulebase – 3. IF the applicant has a job AND the applicant has been more than two years at the present job THEN the applicant has a steady job 4. IF the property is in the bank’s lending zone OR the property is on exception list THEN the property is acceptable 5. IF the family income is adequate OR the single income is adequate THEN the applicant has adequate income ICT 619 S 2 -05 27

The mortgage loan case rulebase – 6. IF the applicant is married AND mortgage payment is less than 60% of the family net income THEN the family income is adequate 7. IF NOT the applicant is married AND mortgage payment is less than 70% of net income THEN the single income is adequate applicant’s 8. IF applicant has properties with a value greater than 10 times the loan OR the applicant has liquid assets greater than 5 times the loan THEN the applicant has adequate assets. ICT 619 S 2 -05 28

Backward chaining in the mortgage loan example (cont’d) § The first proposition in rule (2) is if the applicant has adequate assets. This becomes the current goal. § § Inference engine searches the rule base from the beginning to see which rule has this proposition as its consequent. It finds rule (8). § First proposition of rule (8) becomes the current goal. § Inference engine searches entire rule base to see if any rule has the applicant’s properties as the consequent. It does not find any. § So it asks the user if the applicant has properties with a value greater than ten times the loan. ICT 619 S 2 -05 29

Backward chaining in the mortgage loan example (cont’d) § If the answer is yes, the inference engine concludes that the applicant has adequate assets and attempts to check the truth value of other conditions in rule (2). § If the answer is no, then the system asks whether the applicant has liquid assets greater than five times the loan. § If the user says no, the inference engine does not go any further because it has failed to establish the truth of the goal. § If the user’s answer is positive, then the inference engine attempts to check the truth value of the second proposition in rule (2), and so on (the rest of this multiple inference is left as an exercise). ICT 619 S 2 -05 30

Forward Chaining In forward chaining: § The system requires the user to provide facts pertaining to the problem § The inference engine tries to match each fact with the antecedent of a rule § If the match succeeds, the rule fires and the truth of the consequent of that rule is established, and is added to known facts of the case currently in working memory § This process continues until the inference engine has drawn all possible conclusions by matching facts to antecedents of rules in the knowledge base. ICT 619 S 2 -05 31

Forward Chaining example § Assume the knowledge base consists of the following rules: 1. 2. 3. 4. IF A THEN C IF D THEN E IF B AND C THEN F IF E OR F THEN G § If we start with known facts that A and B are true, then the inference engine uses A and rule (1) to conclude that C is true. § C is added to working memory as a known fact for the case § Then it uses B and C and rule (3) to conclude that F is true. § F added to working memory as a known fact for the case § It then uses the truth of F and the last rule to establish that G is true. ICT 619 S 2 -05 32

Forward vs. Backward Chaining § Forward chaining is data driven because it starts with the data about the case and moves forward from the antecedents of rules to conclude their consequents. § Backward chaining is goal driven since it starts with objective of satisfying a goal. § Backward chaining is useful when the number of goals is small § Forward chaining performs well when § the number of goals is large § the user has a given set of facts at the start of the inquiry, and wants to find the implications of these facts § Some expert system products allow for combining the two methods of multiple inference. ICT 619 S 2 -05 33

Dealing with uncertainty § Facts and inferences in logic are categorical - either true or false § But in expert systems applied to real life problems uncertainty may arise: § § § within the knowledge domain due to expert and knowledge engineer due to the user § Uncertainty in knowledge domain § § Knowledge may be incomplete and imperfect Knowledge may be vague Knowledge may become uncertain due to measurement error Uncertainty may be introduced by conflicting expert opinions. ICT 619 S 2 -05 34

Dealing with uncertainty (cont’d) § Uncertainty related to the expert and knowledge engineer § Expert may not be 100% certain of a rule § Knowledge engineer may not have 100% confidence in a rule expressed by expert. § Uncertainty related to the data input by user § User may be unsure about accuracy of data to be input ICT 619 S 2 -05 35

Dealing with uncertainty (cont’d) Use of certainty parameters in inference § Uncertainty propagates between rules as conclusion reached in an uncertain rule gets used in another rule § Degree of uncertainty in a rule or fact expressed numerically using the certainty or confidence factor cf, in the range 0 to 1 or -1 to +1 § Difficult to implement probability based uncertainty handling schemes § Ad hoc schemes, although mathematically unsound, seem to work ICT 619 S 2 -05 36

Calculation of rule confidence factor (cf) for uncertain facts A scheme for dealing with uncertainty § Let P 1 and P 2 be two propositions and cf(P 1) and cf(P 2), their certainty parameters § Then cf(P 1 AND P 2) cf(P 1 OR P 2) = = min(cf(P 1), cf(P 2)) max(cf(P 1), cf(P 2)) § Given the rule IF P 1 THEN P 2 (Rule cf=C) § Then certainty of the consequent P 2 is given by § cf(P 2) = cf(P 1) * C ICT 619 S 2 -05 37

Calculation of rule cf for uncertain facts (cont’d) § Example: § IF interest rates fall (cf=0. 6) AND taxes are reduced (cf=0. 8) THEN stock market rises (Rule cf=0. 9) § The cf of the conclusion that the stock market is rising can be calculated to be (min(0. 6, 0. 8) * 0. 9 = 0. 6 * 0. 9 = 0. 54 § If more than one rules lead to the same conclusion, the final conclusion is given maximum cf value of all these rules ICT 619 S 2 -05 38

Uncertainty handling using probability theory § There are schemes for handling uncertainty based on probability theory, but they suffer from practical limitations § Difficulty with using probability theory: § Difficult for human experts to express likelihood estimates in terms of probabilities § Not all information are available for correct probabilistic treatment of uncertainties, eg, to evaluate certainty of rule IF A OR B THEN C § needs probabilities of both A and B, as well as correlation between occurrences of A and B. § Uncertainty is handled much more effectively using fuzzy rather than traditional logic (Topic 3) ICT 619 S 2 -05 39

REFERENCES § AI Expert, October 1991 – presents applications of expert systems § Dhar, V. , & Stein, R. , Seven Methods for Transforming Corporate Data into Business Intelligence. , Prentice Hall 1997, Ch 7 § Giarratano, J. , & Riley, G. Expert Systems Principles and Programming, Thomson Course Technology, 2005. § Negnevitsky, M. Artificial Intelligence A Guide to Intelligent Systems, Addison-Wesley 2005. § Zahedi, F. , Intelligent systems for Business, Wadsworth Publishing, Belmont, California, 1993. ICT 619 S 2 -05 40