COMP 4200 Expert Systems Dr Christel Kemke Department

  • Slides: 44
Download presentation
COMP 4200: Expert Systems Dr. Christel Kemke Department of Computer Science University of Manitoba

COMP 4200: Expert Systems Dr. Christel Kemke Department of Computer Science University of Manitoba © C. Kemke XPS Implementation 1

Course Overview Introduction Knowledge Representation Semantic Nets, Frames, Logic Predicate Logic, Inference Methods, Resolution

Course Overview Introduction Knowledge Representation Semantic Nets, Frames, Logic Predicate Logic, Inference Methods, Resolution Probability, Bayesian Decision Making Expert System Design © C. Kemke Variables, Functions, Expressions, Constraints Expert System Implementation Concepts, Notation, Usage Pattern Matching Reasoning with Uncertainty Reasoning and Inference CLIPS Overview Salience, Rete Algorithm Expert System Examples Conclusions and Outlook ES Life Cycle XPS Implementation 2

Overview Implementation of Rule-Based Systems Motivation Objectives Chapter Introduction Basic Idea Unification Pattern Matching

Overview Implementation of Rule-Based Systems Motivation Objectives Chapter Introduction Basic Idea Unification Pattern Matching in Rule. Based Systems Overview Rete Network Assert and Retract Optimizations Improvements Rule Formulation © C. Kemke Rete Algorithm Important Concepts Performance Aspects Pattern Matching General vs. Specific Rules Simple vs. Complex Rules Loading and Saving Facts Important Concepts and Terms Chapter Summary XPS Implementation 3

Motivation pattern matching and unification are powerful operations to determine the similarity and consistency

Motivation pattern matching and unification are powerful operations to determine the similarity and consistency of complex structures they are at the core of many rule-based and predicate logic mechanisms their application goes beyond rule-based systems study concepts and methods that are critical for the functionality and performance of rule-based systems pattern matching and the Rete algorithm use and formulation of rules © C. Kemke XPS Implementation 4

Objectives comprehend the mechanics of pattern matching in rule-based systems basic concepts and techniques

Objectives comprehend the mechanics of pattern matching in rule-based systems basic concepts and techniques Rete algorithm understand the effects of matching and rule formulation on the performance of rule-based systems learn to write rule-based programs and implement systems in an efficient way analyze and evaluate the performance of rule-based programs and systems © C. Kemke identify bottlenecks formulate and implement strategies for performance improvements XPS Implementation 5

Overview Implementation of Rule-Based Systems due to their more declarative nature, it can be

Overview Implementation of Rule-Based Systems due to their more declarative nature, it can be difficult to evaluate and predict the performance of rule-based systems time to complete a task memory usage disk space usage pattern matching can be used to eliminate unsuitable rules and facts © C. Kemke but it can also introduce substantial overhead XPS Implementation 6

Chapter Introduction Important Concepts entities with internal structure data structures, objects, components terms, sentences,

Chapter Introduction Important Concepts entities with internal structure data structures, objects, components terms, sentences, graphs diagrams, images concepts, hierarchies Performance Aspects somewhat different from conventional programs less control over the runtime behavior pattern matching can do a lot of the work © C. Kemke XPS Implementation 7

Pattern Matching determines if two or more complex entities (patterns) are compatible with each

Pattern Matching determines if two or more complex entities (patterns) are compatible with each other patterns can be (almost) anything that has a structure pictures: mugshot vs. person drawings: diagrams of systems expressions: words, sentences of a language, strings graphs are often used as the underlying representation the structure of the graphs must be compatible usually either identical, or one is a sub-graph of the other the individual parts must be compatible nodes must have identical or compatible values variables are very valuable links must indicate compatible relationships compatibility may be dependent on the domain or task © C. Kemke XPS 2004] Implementation 8 [Giarratano & Riley 1998, Friedmann-Hill 2003, Gonzalez & Dankel,

Pattern Matching Example images Do both images refer to the same individual? Do they

Pattern Matching Example images Do both images refer to the same individual? Do they have other commonalities? Bucky and Satchel ? ? ? test Satchel likes. Bucky likes fish © C. Kemke Bucky likes fish XPS Implementation 9

Pattern Matching Example shapes ? ? ? ? ? © C. Kemke XPS Implementation

Pattern Matching Example shapes ? ? ? ? ? © C. Kemke XPS Implementation 10

Pattern Matching Examples constants and variables “Hans” “Josef” “Franz” ? first_name last_name © C.

Pattern Matching Examples constants and variables “Hans” “Josef” “Franz” ? first_name last_name © C. Kemke “Joseph” ? “Joseph” XPS Implementation 11

Pattern Matching Examples terms composed of constants, variables, functions father(X) ? “Joseph” father(Y) father(X))

Pattern Matching Examples terms composed of constants, variables, functions father(X) ? “Joseph” father(Y) father(X)) © C. Kemke mother(X) ? ? grandfather(X) XPS Implementation 12

Unification formal specification for finding substitutions that make logical expressions identical the unification algorithm

Unification formal specification for finding substitutions that make logical expressions identical the unification algorithm takes two sentences and returns a unifier for them (if one exists) Unify(p, q) = if Subst( , p) = Subst( , q) if there is more than one such substitution, the most general unifier is returned used in logic programming, automated theorem proving possibly complex operation quadratic in the size of the expressions © C. Kemke “occur check” sometimes omitted determines if a variable is contained in the term against which it is unified XPS Implementation 13

Pattern Matching in Rule-Based Systems used to match rules with appropriate facts in working

Pattern Matching in Rule-Based Systems used to match rules with appropriate facts in working memory rules for which facts can be found are satisfied the combination of a rule with the facts that satisfy it is used to form activation records © C. Kemke one of the activation records is selected for execution XPS Implementation 14

Simplistic Rule-Based Pattern Matching go through the list of rules, and check the antecedent

Simplistic Rule-Based Pattern Matching go through the list of rules, and check the antecedent (LHS) of each rule against the facts in working memory create an activation record for each rule with a matching set of facts repeat after each rule firing very inefficient roughly (number of rules) * (number of facts)(number of patterns) the actual performance depends on the formulation of the rules and the contents of the working memory © C. Kemke XPS Implementation 15

Rete Algorithm in most cases, the set of rules in a rule-based system is

Rete Algorithm in most cases, the set of rules in a rule-based system is relatively constant the facts (contents of working memory) change frequently optimization of the matching algorithm most of the contents of working memory, however, don’t change every time remember previous results change only those matches that rely on facts that changed the Rete algorithm performs an improved matching of rules and facts © C. Kemke invented by Charles Forgy in the early 80 s basis for many rule-based expert system shells [ Friedmann-Hill 2003, Giarratano & Riley 1998, Gonzalez & Dankel, 2004] XPS Implementation 16

Rete Network the name comes from the latin word rete stands for net consists

Rete Network the name comes from the latin word rete stands for net consists of a network of interconnected nodes each node represents one or more tests on the LHS of a rule input nodes are at the top, output nodes at the bottom pattern nodes have one input, and check the names of facts join nodes have two inputs, and combine facts terminal node at the bottom of the network represent individual rules a rule is satisfied if there is a combination of facts that passes all the test nodes from the top to the output node at the bottom that represents the rule the Rete network effectively is the working memory for a rulebased system © C. Kemke XPS Implementation 17

Rete Network Example 1 (deftemplate x (slot a)) (deftemplate y (slot b)) ? =

Rete Network Example 1 (deftemplate x (slot a)) (deftemplate y (slot b)) ? = x ? = ? v 1 (defrule example-1 (x (a ? v 1)) (y (b ? v 1)) ==> ) y ? v 1 Left. 0. a ? = Right. b ? v 1 = ? v 1 example-1 © C. Kemke XPS Implementation 18

Rete Left and Right Memories left (alpha) memory ? = right (beta) memory contains

Rete Left and Right Memories left (alpha) memory ? = right (beta) memory contains the left input of a join node contains the right input of a join node notation: Left. p. q ? = Right. r compare the contents of slot q in fact p from the left memory with slot r in the fact from the right memory x ? = ? v 1 y ? v 1 Left. 0. a ? = Right. b ? v 1 = ? v 1 (deftemplate x (slot a)) (deftemplate y (slot b)) (defrule example-1 (x (a ? v 1)) (y (b ? v 1)) ==> ) © C. Kemke example-1 XPS Implementation 19

Running the Network only facts x and y are considered all facts where x.

Running the Network only facts x and y are considered all facts where x. a == y. b pass the join network all {x, y} tuples are fowarded to the next node compare the contents of slot q in fact p from the left memory with slot r in the fact from the right memory ? = x ? = ? v 1 y ? v 1 Left. 0. a ? = Right. b ? v 1 = ? v 1 (deftemplate x (slot a)) (deftemplate y (slot b)) (defrule example-1 (x (a ? v 1)) (y (b ? v 1)) ==> ) © C. Kemke example-1 XPS Implementation 20

Rete Network Example 2 shares some facts with Example 1 (deftemplate x (slot a))

Rete Network Example 2 shares some facts with Example 1 (deftemplate x (slot a)) (deftemplate y (slot b)) (deftemplate z (slot c)) (defrule example-2 (x (a ? v 2)) (y (b ? v 2)) (z) ==> ) ? = x ? = ? v 2 y ? = z ? v 2 Left. 0. a ? = Right. b ? v 2 = ? v 2 example-2 © C. Kemke XPS Implementation 21

Rete Network Example 2 with Assert additional fact asserted ? = (deftemplate x (slot

Rete Network Example 2 with Assert additional fact asserted ? = (deftemplate x (slot a)) (deftemplate y (slot b)) (deftemplate z (slot c)) (defrule example-2 (x (a ? v 2)) (y (b ? v 2)) (z) ==> ) x ? = ? v 2 y ? = ? v 2 z 17 Left. 0. a ? = Right. b ? v 2 = ? v 2 = 17 example-2 (assert (z (c 17)) © C. Kemke XPS Implementation 22

Assert and Retract with Rete asserting additional facts imposes some more constraints on the

Assert and Retract with Rete asserting additional facts imposes some more constraints on the network retracting facts indicates that some previously computed activation records are not valid anymore, and should be discarded in addition to the actual facts, tags are sent through the networks ADD to add facts (i. e. for assert) REMOVE to remove facts (i. e. for retract) CLEAR to flush the network memories (i. e. for reset) UPDATE to populate the join nodes of newly added rules © C. Kemke already existing join nodes can neglect these tokens XPS Implementation 23

Rete Network Optimization networks with shared facts can be combined ? = x ?

Rete Network Optimization networks with shared facts can be combined ? = x ? = y ? = z (deftemplate x (slot a)) (deftemplate y (slot b)) (deftemplate z (slot c)) Left. 0. a ? = Right. b (defrule example-1 (x (a ? v 1)) (y (b ? v 1)) ==> ) (defrule example-2 (x (a ? v 2)) (y (b ? v 2)) (z) ==> ) © C. Kemke example-1 example-2 XPS Implementation 24

Further Optimizations sophisticated data structures to optimize the network hash table to presort the

Further Optimizations sophisticated data structures to optimize the network hash table to presort the tokens before running the join node tests fine-tuning via parameters © C. Kemke frequently trade-off between memory usage and time XPS Implementation 25

Special Cases for Pattern Matching additional enhancements of the Rete network can be used

Special Cases for Pattern Matching additional enhancements of the Rete network can be used to implement specific methods backward chaining not conditional element indicates the absence of a fact requires special join nodes and special fields in the tokens passing through the network test conditional element © C. Kemke requires a signal indicating to the network that a particular fact is needed uses a special join node that ignores its right input the result of the function is passed on XPS Implementation 26

Exploring the Rete Network in Jess (watch compilations) function diagnostic output when rules are

Exploring the Rete Network in Jess (watch compilations) function diagnostic output when rules are compiled example-1: +1+1+1+2+t (view) function +1 one-input (pattern) node added to the Rete network +2 two-input (pattern) node added +t terminal node added graphical viewer of the Rete network in Jess (matches <rule-name>) function displays the contents of the left and right memories of the join nodes for a rule useful for examining unexpected rule behavior © C. Kemke XPS Implementation 27

Rule Formulation Pattern Order General vs. Specific Rules Simple vs. Complex Rules Loading and

Rule Formulation Pattern Order General vs. Specific Rules Simple vs. Complex Rules Loading and Saving Facts © C. Kemke [Giarratano & Riley 1998] XPS Implementation 28

Pattern Order since Rete saves information about rules and facts, it can be critical

Pattern Order since Rete saves information about rules and facts, it can be critical to order patterns in the right way © C. Kemke otherwise a potentially huge number of partial matches can be generated XPS Implementation 29

Example Pattern Order (deffacts information (find-match a c e g) f 1 (item a)

Example Pattern Order (deffacts information (find-match a c e g) f 1 (item a) (item b) (item c) (item d) (item e) (item f) (item g)) (defrule match-1 (find-match ? x ? y ? z ? w) P 1 (item ? x) (item ? y) (item ? z) (item ? w) ==> (assert (found-match ? x ? y ? z ? w)) © C. Kemke f 2 f 3 f 4 f 5 f 6 f 7 f 8 P 2 P 3 P 4 P 5 (deffacts information (find-match a c e g) (item a) (item b) (item c) (item d) (item e) (item f) (item g)) (defrule match-2 (item ? x) (item ? y) (item ? z) (item ? w) (find-match ? x ? y ? z ? w) ==> (assert (found-match ? x ? y ? z ? w)) [Giarratano & Riley 1998] XPS Implementation 30

Pattern Matches full matches P 1: f 1 P 2: f 2, f 3,

Pattern Matches full matches P 1: f 1 P 2: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 3: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 4: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 5: f 2, f 3, f 4, f 5, f 6, f 7, f 8 full matches P 1: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 2: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 3: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 4: f 2, f 3, f 4, f 5, f 6, f 7, f 8 P 5: f 1 partial matches P 1: [f 2, f 3, f 4, f 5, f 6, f 7, f 8] P 1: [f 1] P 1 -2: [f 2, f 2], [f 2, f 3], [f 2, f 4], [f 2, f 5], [f 2, f 6], [f 2, f 7], [f 2, f 8], P 1 -2: [f 1, f 2] [f 3, f 2], [f 3, f 3], [f 3, f 4], [f 3, f 5], P 1 -3: [f 1, f 2, f 4] [f 3, f 6], [f 3, f 7], [f 3, f 8], P 1 -4: [f 1, f 2, f 4, f 6]. . . P 1 -5: [f 1, f 2, f 4, f 6, f 8] P 1 -3, P 1 -4: . . . Total: 29 full, 5 partial matches P 1 -5: [f 2, f 4, f 6, f 8, f 1] Total: 29 full, 2801 partial matches © C. Kemke XPS Implementation 31

Adding another Fact what is the effect on the two cases if another fact

Adding another Fact what is the effect on the two cases if another fact (item h) is added? no significant changes for match-1 in particular, no additional partial matches major changes for match-2 © C. Kemke another 1880 partial matches XPS Implementation 32

Guidelines for Pattern Matches try to formulate your rule such that the number of

Guidelines for Pattern Matches try to formulate your rule such that the number of matches is low try to limit the number of old partial matches full and partial matches removing those also is time-consuming in general, the state of the system should be reasonably stable matches assertion, retraction, modification of facts © C. Kemke XPS Implementation 33

Guidelines for Pattern Ordering most specific patterns first smallest number of matching facts largest

Guidelines for Pattern Ordering most specific patterns first smallest number of matching facts largest number of variable bindings to constrain other facts patterns matching volatile facts go last facts that are changing frequently should be used by patterns late in the LHS smallest number of changes in partial matches may cause a dilemma with the above guideline patterns matching the fewest facts first © C. Kemke reduces the number of partial matches XPS Implementation 34

Multifield Variables multifield wildcards and multifield variables are very powerful, but possible very inefficient

Multifield Variables multifield wildcards and multifield variables are very powerful, but possible very inefficient should only be used when needed limit their number in a single slot of a pattern © C. Kemke XPS Implementation 35

Test Conditional Element the test conditional element should be placed as close to the

Test Conditional Element the test conditional element should be placed as close to the top of the rule as possible reduces the number of partial matches evaluation of expressions during pattern matching is usually more efficient © C. Kemke XPS Implementation 36

Built-In Pattern Matching Constraints the built-in constraints are always more efficient than the equivalent

Built-In Pattern Matching Constraints the built-in constraints are always more efficient than the equivalent expression not so good: (defrule primary-color ? x&: (or (eq ? x red) (eq ? x green) (eq ? x blue) ==> (assert (primary-color ? x))) better: (defrule primary-color ? x&red|green|blue) © C. Kemke ==> (assert (primary-color ? x))) XPS Implementation 37

General vs. Specific Rules some knowledge can be expressed through many specific, or a

General vs. Specific Rules some knowledge can be expressed through many specific, or a few general rules specific rules generate a top-heavy Rete network with many pattern nodes and fewer join nodes general rules offer better opportunities for sharing pattern and join nodes it usually is easier to write an inefficient general rule than an inefficient specific rule © C. Kemke XPS Implementation 38

Simple vs. Complex Rules simple rules are sometimes elegant, but not necessarily efficient storing

Simple vs. Complex Rules simple rules are sometimes elegant, but not necessarily efficient storing temporary facts can be very helpful © C. Kemke especially in recursive or repetitive programs XPS Implementation 39

Loading and Saving Facts facts can be kept in a file, and loaded into

Loading and Saving Facts facts can be kept in a file, and loaded into memory when needed (load-facts) and (save-facts) functions may lead to visibility or scoping problems if the respective deftemplates are not contained in the current module © C. Kemke XPS Implementation 40

Figure Example © C. Kemke XPS Implementation 41

Figure Example © C. Kemke XPS Implementation 41

Use of References [Giarratano & Riley 1998] [Russell & Norvig 1995] [Jackson 1999] [Durkin

Use of References [Giarratano & Riley 1998] [Russell & Norvig 1995] [Jackson 1999] [Durkin 1994] [Giarratano & Riley 1998] © C. Kemke XPS Implementation 42

Important Concepts and Terms © C. Kemke agenda assert backward chaining constant fact expert

Important Concepts and Terms © C. Kemke agenda assert backward chaining constant fact expert system (ES) expert system shell forward chaining join node knowledge base knowledge-based system left (alpha) memory matches matching pattern matching pattern node RETE algorithm retract right (beta) memory rule substitution term test conditional element unification variable view working memory XPS Implementation 43

Summary XPS Implementation for rule-based systems, an efficient method for pattern matching between the

Summary XPS Implementation for rule-based systems, an efficient method for pattern matching between the rule antecedents and suitable facts is very critical the Rete algorithm is used in many expert system shells matching every rule against all possible facts repeatedly is very inefficient it constructs a network from the facts and rules in the knowledge base since certain aspects of the knowledge base are quite static, repeated matching operations can be avoided a few strategies can be used by programmers to achieve better performance © C. Kemke most specific patterns first, patterns with volatile facts last careful use of multifield variables, general rules use of the test conditional element, built-in pattern constraints loading and saving of facts XPS Implementation 44