Outline Classification 1142020 Visual Perception Modeling 1 Bayesian

Outline • Classification 11/4/2020 Visual Perception Modeling 1

Bayesian Decision Rule • A two-class example – 1 for sea bass – 2 for salmon • Prior probability – P( 1) – P( 2) 11/4/2020 Visual Perception Modeling 2

Bayesian Decision Rule – cont. • Class conditional probability density – P( 1 | x) – P( 2 | x) • Bayes formula 11/4/2020 Visual Perception Modeling 3

Bayesian Decision Rule – cont. • Bayes decision rule – Decide 1 if P( 1 | x) > P( 2 | x) – Otherwise decide 2 – The optimal decision rule • Minimize the average error we make 11/4/2020 Visual Perception Modeling 4

Feature Space • Feature space – The Euclidean space Rd if we use a ddimensional feature – Each possible feature is a point the Rd space 11/4/2020 Visual Perception Modeling 5

Loss Function • Loss function – States exactly how costly each action is – Is used to convert a probability determination into a decision – Allows us to treat situations where some kinds of classification mistakes are more costly than others • Equally costly is a special case 11/4/2020 Visual Perception Modeling 6

Loss Function – cont. • Suppose that there are c categories – { 1, 2, . . . , c} • There a possible actions – { 1, 2, . . . , c} • Loss function ( i | j} describe the loss incurred for taking action i when the state of nature is j 11/4/2020 Visual Perception Modeling 7

Loss Function – cont. • Bayes formula – where 11/4/2020 Visual Perception Modeling 8

Loss Function – cont. • The expected loss function given a particular observation x • The overall risk 11/4/2020 Visual Perception Modeling 9

Bayes Decision Rule • To minimize the overall risk, compute the conditional risk and select the action for the conditional risk is minimum – The resulting minimum overall risk is called the Bayes risk, which is the best performance 11/4/2020 Visual Perception Modeling 10

Two-Category Classification • Two categories • Two actions • We decide 1 if 11/4/2020 Visual Perception Modeling 11

Minimum-Error-Rate Classification • Zero-one loss • For minimum error rate, – Decide 1 if P( 1 | x) > P( 2 | x) – This is the Bayes decision rule 11/4/2020 Visual Perception Modeling 12

Discriminant Functions • The classifier is said to assign a feature vector x to class i if – gi(x) > gj(x) for all j i – This can be viewed as a network – If f(. ) is a monotonically increasing function, f(g(x)) and g(x) as discriminant function will give the same classification result 11/4/2020 Visual Perception Modeling 13

Decision Regions • The effect of decision rule is to divide the feature space into c decision regions – R 1, R 2, . . , Rc – The regions are separated by decision boundaries – Two-category case 11/4/2020 Visual Perception Modeling 14

Normal Density • Gaussian density – Properties • • 11/4/2020 Mean Variance Entropy Central limit theorem Visual Perception Modeling 15

Discriminant Functions for Normal Density • Minimum error rate classification for normal density – Special cases 11/4/2020 Visual Perception Modeling 16