Classifier Systems Anil Shankar Classifier Systems Overview Introduction

Classifier Systems Anil Shankar Classifier Systems

Overview • • • Introduction and problem overview Architecture Component details Track a specific example Summary 2

The Learning Classifier System • Rule-based knowledge discovery and concept learning tool • Operates by means of evaluation, credit assignment, and discovery applied to a population of “chromosomes” (rules) each with a corresponding “phenotype” (outcome) 3

Components of a Learning Classifier System • Performance – Provides interaction between environment and rule base – Performs matching function • Reinforcement – Rewards accurate classifiers – Punishes inaccurate classifiers • Discovery – Uses the genetic algorithm to search for plausible rules 4

Knowledge Representation • Classifiers – IF-THEN rules • Condition=“genotype” • Action=“phenotype” – Strength metric – Encoded as bit strings or numerics • Population – Fixed size collection of classifiers 5

Low-level knowledge representation: The Classifier • Taxon is analogous to a condition (LHS) of an IF -THEN rule • Action bit is analogous to an action (RHS) of an IF-THEN rule • Strength is an internal fitness function 6

Problem Multiplexer Example Rule Address Signal # # # 0 0 0: 0 0 0 # # 0 1: 0 1 0 # # 1 0: 0 2 0 0 # # # 1 1: 0 3 0 # # # 1 0 0: 1 0 1 # # 1 # 0 1: 1 1 1 # # 1 0: 1 2 1 1 # # # 1 1: 1 3 1 Perfect Rule Set 7

Classifier System (C. S) • Learn simple string rules in an arbitrary environment • A classifier is a simple string rule • Components – Rule and Message System – Apportionment of credit system – Genetic Algorithm 8

Overview • General organization of a classifier system – performance system: rule based, messagepassing, highly standardized, and highly parallel – credit assignment: bucket brigade algorithms – rule discovery: genetic algorithms 9

• Definition of the basic elements – input interface: translate the current state of the environment into standard messages – classifiers (the rules used by the system): define the system’s procedures for processing messages – message list: contain all current messages (those generated by the input interface and those generated by satisfied rules) – output interface: translate some messages into effector actions that modify the state of the environment 10

Classifier Systems (2) – Basic parts of a classifier system 11

– Execution cycle Step 1. Add all messages from the input interface to the message list. Step 2. Compare all messages on the message list to all conditions of all classifiers and record all matches (satisfied conditions) Step 3. For each set of matches satisfying the condition part of some classifiers, post the message satisfied by its action part to a list of new messages. Step 4. Replace all messages on the message list by the list of new messages. Step 5. Translate messages on the message list to requirements on the output interface, thereby producing the system’s current output. Step 6. Return to Step 1. 12

Rule and Message System • Production system • Fixed size representation for rules • Parallel activation • Rating of a rule by an information-based economy • <message>: : = { 0, 1} • <classifier>: : = l <condition>: <message> • <condition>: : ={0, 1, #}l 13

Which classifier to choose? • Bucket Brigade Algorithm – For ranking or rating individual classifiers – Classifiers buy and sell the right to trade information (information-based economy) – Auction house and clearing house – If a classifier matches a message, it participates in an auction. – The bid (B) is proportional to its strength (S) – Once activated the winner pays its bid to other classifiers which also matched the message 14

Which classifier to choose? (contd…) • Notation – – – • • • S = Strength P = Payment T = Tax R = Reward Cbid = Bid Coefficient The ith classifier strength (at time step t) Si(t+1) = Si(t) – Pi(t) – Ti(t) + Ri(t) Bid Bi = Cbid * Si Taxi = Ctax * Si Effective Bid EBidi = Bi + N (σbid) In terms of strength S(t+1) = S(t) – Cbid*S(t) – Ctax*S(t) + R(t) 15

Generating better rules • Bucket brigade algorithm evaluates rules and decides among competing alternatives. • We could still inject new (possibly better rules), so use a Genetic Algorithm (GA) • A classifier’s strength (S) is used as its fitness • Similar to the simple genetic algorithm • Entire population is not replaced at the next generation (Generation gap ) • GA period (epoch) – Number of time steps between GA calls – Time step = rule-message cycle • Crowding to maintain diversity • Mutation over a ternary alphabet {1, 0, # } 16

Generating better rules • Selection is performed using roulettewheel selection • The GA is run according to the GA Period or when conditioned on particular events (lack of match or poor performance) 17

C. S in action (1) Index Classifier 1 01## : 0000 2 00#0 : 1100 3 T= 0 Index S 1 200 11## : 1000 2 200 4 ##00 : 0001 3 200 Environment (E) 0111 4 200 E 0 Strength (S) Messages (Msg) Match (M) Bid (B) CBid = 0. 1 CTax = 0. 0 Msg M B E 20 0111 01## 0000 18

C. S in action (2) Index 1 2 3 4 Environment (E) Classifier 01## : 0000 00#0 : 1100 11## : 1000 ##00 : 0001 0111 Strength (S) Messages (Msg) Match (M) Bid (B) CBid = 0. 1 CTax = 0. 0 T= 1 Index S Msg 1 180 0000 2 200 3 200 4 200 E 20 M B 1 20 00#0 ##00 1100 0001 19

C. S in action (3) Index 1 2 3 4 Environment (E) Classifier 01## : 0000 00#0 : 1100 11## : 1000 ##00 : 0001 0111 Strength (S) Messages (Msg) Match (M) Bid (B) T= 2 Index S Msg M B 1 220 2 180 3 200 2 20 4 180 0001 2 18 E 20 1100 CBid = 0. 1 CTax = 0. 0 20

C. S in action (4) Index 1 2 3 4 Environment (E) Classifier 01## : 0000 00#0 : 1100 11## : 1000 ##00 : 0001 0111 Strength (S) Messages (Msg) Match (M) Bid (B) T= 3 Index S Msg M 1 220 2 218 3 180 1000 4 162 0001 3 E 20 B 16 CBid = 0. 1 CTax = 0. 0 21

C. S in action (5) Index 1 2 3 4 Environment (E) Classifier 01## : 0000 00#0 : 1100 11## : 1000 ##00 : 0001 0111 Strength (S) Messages (Msg) Match (M) Bid (B) T= 4 Index S 1 220 2 208 3 196 4 156 E 20 Msg M B 0001 CBid = 0. 1 CTax = 0. 0 22

C. S in action (6) Index 1 2 3 4 Environment (E) Strength (S) Classifier 01## : 0000 00#0 : 1100 11## : 1000 ##00 : 0001 0111 T= 5 Index S 1 220 2 208 3 196 4 206 E 20 Payoff 50 CBid = 0. 1 CTax = 0. 0 23

Are these rule-sets the same? Rule Address Signal Rule # # # 0 0 0: 0 0 0 # # # 0 0 0: 0 # # 0 1: 0 1 0 # # 1 0: 0 2 0 0 # # # 1 1: 0 3 0 # # 1 0: 0 # # # 1 0 0: 1 0 # # # 1 1: 0 # # 1 # 0 1: 1 1 1 # # # : 1 # # 1 0: 1 2 1 1 # # # 1 1: 1 3 1 # # 0 1: 0 24

Multiplexer Example • Default Hierarchy – General rules cover general conditions and specific rules cover exceptions – Parsimony ###000 0 ##0#01 0 #0##10 0 0###11 0 ###### 1 • Fewer rules – Enlargement of the solution set • While the problem space remains the same 25

Summary • A classifier is a simple string rule • Classifier System – rule-message system, – apportionment of credit mechanism – GA • Advantages of CS – rules are simple – use fixed length representation – parallel activation – operate in an informationbased economy 26

Thank You Questions ? 27