2 Concept Learning 2 1 Introduction Concept Learning

  • Slides: 31
Download presentation
2. Concept Learning 2. 1 Introduction Concept Learning: Inferring a boolean-valued function from training

2. Concept Learning 2. 1 Introduction Concept Learning: Inferring a boolean-valued function from training examples of its inputs and outputs 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 2. 2 A Concept Learning Task: “Days in which Aldo enjoys

2. Concept Learning 2. 2 A Concept Learning Task: “Days in which Aldo enjoys his favorite water sport” 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Hypothesis Representation – Simple representation: Conjunction of constraints on the

2. Concept Learning • Hypothesis Representation – Simple representation: Conjunction of constraints on the 6 instance attributes • indicate by a “? ” that any value is acceptable • specify a single required value for the attribute • indicate by a “ ” that no value is acceptable Example: h = (? , Cold, High, ? , ? ) indicates that Aldo enjoys his favorite sport on cold days with high humidity (independent of the other attributes) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning – h(x)=1 if example x satisfies all the constraints h(x)=0 otherwise

2. Concept Learning – h(x)=1 if example x satisfies all the constraints h(x)=0 otherwise – Most general hypothesis: – Most specific hypothesis: (? , ? , ? , ? ) ( , , , ) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Notation – – – Set of instances X Target concept

2. Concept Learning • Notation – – – Set of instances X Target concept c : X {0, 1} (Enjoy. Sport) Training examples {x , c(x)} Data set D X Set of possible hypotheses H h : X {0, 1} Goal: Find h / h(x)=c(x) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Inductive Learning Hypothesis Any hypothesis h found to approximate the

2. Concept Learning • Inductive Learning Hypothesis Any hypothesis h found to approximate the target function c well over a sufficiently large set D of training examples x, will also approximate the target function well over other unobserved examples in X 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning “We have experience of past futures, but not of futures, and

2. Concept Learning “We have experience of past futures, but not of futures, and the question is: Will futures resemble past futures? ” Bertrand Russell, "On Induction" 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 2. 3 Concept Learning as Search – Distinct instances in X

2. Concept Learning 2. 3 Concept Learning as Search – Distinct instances in X : 3. 2. 2. 2 = 96 – Distinct hypotheses • syntactically • semantically 5. 4. 4. 4 = 5120 1 + (4. 3. 3. 3) = 973 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • General-to-Specific Ordering of hypotheses h 1=(sunny, ? , Strong, ?

2. Concept Learning • General-to-Specific Ordering of hypotheses h 1=(sunny, ? , Strong, ? ) h 2=(Sunny, ? , ? , ? ) Definition: h 2 is more_general_than_or_equal_to h 1 (written h 2 g h 1) if and only if ( x X) [ h 1(x)=1 h 2(x)=1] g defines a partial order over the hypotheses space for any concept learning problem 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 2. 4 Finding a Maximally Specific Hypothesis – Find-S Algorithm h

2. Concept Learning 2. 4 Finding a Maximally Specific Hypothesis – Find-S Algorithm h 1 ( , , , ) h 2 (Sunny, Warm, Normal, Strong, Warm, Same) h 3 (Sunny, Warm, ? , Strong, Warm, Same) h 4 (Sunny, Warm, ? , Strong, ? ) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Questions left unanswered: – – Has the learner converged to

2. Concept Learning • Questions left unanswered: – – Has the learner converged to the correct concept? Why prefer the most specific hypothesis? Are the training examples consistent? What is there are several maximally specific hypotheses? 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 2. 5 Version Spaces and the Candidate-Elimination Algorithm – The Candidate-Elimination

2. Concept Learning 2. 5 Version Spaces and the Candidate-Elimination Algorithm – The Candidate-Elimination Algorithm outputs a description of the set of all hypotheses consistent with the training examples – Representation • Consistent hypotheses Consistent(h, D) ( {x, c(x)} D) h(x) = c(x) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning – Version Space VSH, D {h H | Consistent(h, D)} –

2. Concept Learning – Version Space VSH, D {h H | Consistent(h, D)} – The List-Then-Eliminate Algorithm • Initialize the version space to H • Eliminate any hypothesis inconsistent with any training example the version space shrinks to the set of hypothesis consistent with the data 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Compact Representation for Version Spaces – General Boundary G(H, D):

2. Concept Learning • Compact Representation for Version Spaces – General Boundary G(H, D): Set of maximally general members of H consistent with D – Specific Boundary S(H, D): set of minimally general (i. e. , maximally specific) members of H consistent with D 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Theorem: Version Space Representation – For all X, H, c

2. Concept Learning • Theorem: Version Space Representation – For all X, H, c and D such that S and G are well defined, VSH, D {h H | ( s S) ( g G) (g g h g s )} 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Candidate-Elimination Learning Algorithm 1 er. Escela Red Pro. TIC -

2. Concept Learning • Candidate-Elimination Learning Algorithm 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning • Remarks – Will the Candidate-Elimination converge to the correct hypothesis?

2. Concept Learning • Remarks – Will the Candidate-Elimination converge to the correct hypothesis? – What training example should the learner request next? – How can partially learned concepts be used? 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning A=yes B=no C=1/2 yes - 1/2 no D=1/3 yes - 2/3

2. Concept Learning A=yes B=no C=1/2 yes - 1/2 no D=1/3 yes - 2/3 no 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 2. 7 Inductive Bias Can a hypothesis space that includes every

2. Concept Learning 2. 7 Inductive Bias Can a hypothesis space that includes every possible hypothesis be used ? – The hypothesis space previously considered for the Enjoy. Sport task is biased. For instance, it does not include disjunctive hypothesis like: Sky=Sunny or Sky=cloudy 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning An unbiased H must contain the power set of X Power.

2. Concept Learning An unbiased H must contain the power set of X Power. Set (X) = the set of all subsets of X |Power Set (X)| = 2|X | (= 296 ~1028 for Enjoy. Sport) • Unbiased Learning of Enjoy. Sport H =Power Set (X) 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning For example, “Sky=Sunny or Sky=Cloudy” H : (Sunny, ? , ?

2. Concept Learning For example, “Sky=Sunny or Sky=Cloudy” H : (Sunny, ? , ? , ? ) (Cloudy, ? , ? , ? ) Suppose x 1 , x 2 , x 3 are positive examples and x 4 , x 5 negative examples S: {(x 1 x 2 x 3)} G: { (x 4 x 5)} In order to converge to a single, final target concept, every instance in X has to be presented! 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning – Voting? Each unobserved instance will be classified positive by exactly

2. Concept Learning – Voting? Each unobserved instance will be classified positive by exactly half the hypotheses in the version space and negative by the other half !! • The Futility of Bias-Free Learning A learner that makes no a priori assumptions regarding the target concept has no rational basis for classifying unseen instances 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning Notation (Inductively inferred from): (Dc xi) L(xi, Dc) Definition Inductive Bias

2. Concept Learning Notation (Inductively inferred from): (Dc xi) L(xi, Dc) Definition Inductive Bias B: ( xi X) [(B Dc xi) L(xi, Dc)] Inductive bias of the Candidate-Elimination algorithm: The target concept c is contained in the hypothesis space H 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de

2. Concept Learning 1 er. Escela Red Pro. TIC - Tandil, 18 -28 de Abril, 2006

Forensic Reports Introduction Introduction Introduction Introduction Introduction IntroductionForensic Reports Introduction Introduction Introduction Introduction Introduction Introduction
Assembly Method Concepts Concept 2 Concept 1 ConceptAssembly Method Concepts Concept 2 Concept 1 Concept
Chapter 6 Concept Modeling Concept Modeling Contents ConceptChapter 6 Concept Modeling Concept Modeling Contents Concept
Today Eerste Concept Tweede Concept Defenitief Concept OnderzoekToday Eerste Concept Tweede Concept Defenitief Concept Onderzoek
Concept check Concept check Concept check Look atConcept check Concept check Concept check Look at
WELLNESS CONCEPT Concept Of Health And Wellness ConceptWELLNESS CONCEPT Concept Of Health And Wellness Concept
RHUMBA Project Concept RHUMBA Concept Green Project ConceptRHUMBA Project Concept RHUMBA Concept Green Project Concept
Concept Mapping What are Concept Maps Concept mapsConcept Mapping What are Concept Maps Concept maps
COGNITION THOUGHT Concept Formation Concepts Concept Formation ConceptCOGNITION THOUGHT Concept Formation Concepts Concept Formation Concept
PPTwww pptbz com Concept concept Concept 150 140PPTwww pptbz com Concept concept Concept 150 140
www 51 pptmoban com Concept concept Concept 150www 51 pptmoban com Concept concept Concept 150
Concept Maps Definitions of concept maps A conceptConcept Maps Definitions of concept maps A concept
Assignment 4 Concept Map CONCEPT MAP 1 CONCEPTAssignment 4 Concept Map CONCEPT MAP 1 CONCEPT
Concept Mapping What is Concept Mapping A conceptConcept Mapping What is Concept Mapping A concept
Concept Voorstellen Het Concept Ons definitieve concept isConcept Voorstellen Het Concept Ons definitieve concept is
Ch 2 Concept Learning S Visa Introduction ConceptCh 2 Concept Learning S Visa Introduction Concept
Concept Learning and the GeneraltoSpecific Ordering Concept LearningConcept Learning and the GeneraltoSpecific Ordering Concept Learning
Chapter 2 Concept learning Learning from examples ConceptChapter 2 Concept learning Learning from examples Concept
INTRODUCTION TO What is concept Concept is anINTRODUCTION TO What is concept Concept is an
Concept learning Overview What do we mean conceptConcept learning Overview What do we mean concept
Deuteronomy Introduction Deuteronomy Introduction Deuteronomy Introduction Deuteronomy IntroductionDeuteronomy Introduction Deuteronomy Introduction Deuteronomy Introduction Deuteronomy Introduction
I Introduction to the concept of prevention IntroductionI Introduction to the concept of prevention Introduction
Index Introduction to OOPS Oops Concept Introduction toIndex Introduction to OOPS Oops Concept Introduction to
Introduction Introduction and Basic Concept What is VerilogIntroduction Introduction and Basic Concept What is Verilog