ECE 471571 Lecture 2 Bayesian Decision Theory Pattern

ECE 471/571 – Lecture 2 Bayesian Decision Theory

Pattern Classification Statistical Approach Supervised Basic concepts: Baysian decision rule (MPP, LR, Discri. ) Non-Statistical Approach Unsupervised Basic concepts: Distance Agglomerative method Parameter estimate (ML, BL) k-means Non-Parametric learning (k. NN) Winner-takes-all LDF (Perceptron) Kohonen maps NN (BP) Mean-shift Decision-tree Syntactic approach Support Vector Machine Deep Learning (DL) Dimensionality Reduction FLD, PCA Performance Evaluation ROC curve (TP, TN, FP) cross validation Stochastic Methods local opt (GD) global opt (SA, GA) Classifier Fusion majority voting NB, BKS

Bayes’ formula (Bayes’ rule) Conditional probability density (pdf – probability density function) (likelihood) a-posteriori probability (posterior probability) From domain knowledge a-priori probability (prior probability) normalization constant (evidence) 3

pdf examples Gaussian distribution n n Bell curve Normal distribution Uniform distribution Rayleigh distribution 4

Bayes decision rule Maximum a-posteriori Probability (MAP): x 5

Conditional probability of error 6

Decision regions The effect of any decision rule is to partition the feature space into c decision regions 7

Overall probability of error Or unconditional risk, unconditional probability of error 8

The conditional risk Given x, the conditional risk of taking action ai is: lij is the loss when decide x belongs to class i while it should be j 9

Likelihood ratio - two category classification Likelihood ratio 10

Zero-One loss 11

Recap Bayes decision rule maximum aposteriori probability Conditional risk Likelihood ratio Decision regions How to calculate the overall probability of error 12

Recap Maximum a-posteriori Probability Likelihood ratio Overall probability of error 13