Bayesian Network CLASSIFIERS Weka Demonstration Lecture 11 Outline

Bayesian Network CLASSIFIERS & Weka Demonstration Lecture 11

Outline of Rule-Based Classification 1. Bayesian Network Classifier 2. Characteristics of Naïve Bayesian Classifier 3. Weka demonstration

Bayesian Networks �Probabilistic graphical model �Represents random variables and conditional dependencies using a directed acyclic graph (DAG) �Nodes of graph represent random variables

Bayesian Networks �Edges of graph represent conditional dependencies �Unconnected nodes conditionally independent of each other � Does not require all attributes to be conditionally independent

Bayesian Networks �Probability table associating each node to its immediate parent nodes �If node X has no immediate parents, table contains only prior probability P(X) �If one parent Y, table contains P(X|Y) �If multiple parents {Y 1, Y 2, ⋯ , Yn}, table contains P(X|Y 1, Y 2 , ⋯ , Y n)

Bayesian Networks

Bayesian Networks �Model encodes relevant probabilities from which probabilistic inferences can then be calculated �Joint probability: P(G, S, R) = P(R)P(S|R)*P(G|S, R) �G = “Grass wet” � S = “Sprinkler on” � R = “Raining”

Bayesian Networks �We can then calculate, for example:

Bayesian Networks �That is

Bayesian Networks �Building the model �Create network structure (graph) �Determine probability values of tables �Simplest case �Network defined by user �Most real-world cases �Defining network too complex �Use machine learning: many algorithms

Bayesian Networks �Algorithms built into Weka �User defined network �Conditional independence tests �Genetic search �Hill climber �K 2 �Simulated annealing �Maximum weight spanning tree �Tabu search

Bayesian Networks �Many other versions online �BNT (Bayes’ Net Tree) Matlab toolbox � Kevin Murphy, University of British Columbia � http: //www. cs. ubc. ca/~murphyk/Software/

2. Bayesian Classifiers in Weka Click on Explorer

Bayesian Classifiers in Weka Open file

Bayesian Classifiers in Weka Weather file loaded

Bayesian Classifiers in Weka Choose choose

Bayesian Classifiers in Weka Choose classify

Bayesian Classifiers in Weka Choose classify

Bayesian Classifiers in Weka Bayes’ Net Chosen -K 2 default Click inside To gain additional choices

Bayesian Classifiers in Weka Additional Options: Can choose Alternate Algorithms or Conditional Probability estimators

Bayesian Classifiers in Weka Yet more Options: Can change Maximum Number of parents

Bayesian Classifiers in Weka Start classification

Bayesian Classifiers in Weka Default results Right click to View network

Bayesian Classifiers in Weka Visualize graph

Bayesian Classifiers in Weka Results from Three parents maximum

Bayesian Classifiers in Weka Graph from Three parents maximum

Characteristics of Bayesian Network �BBN provides an approach for capturing the prior knowledge of a particular domain using a graphical model. The Network can also be used to encode causal dependencies among variables. �Constructing the network can be time consuming and requires a large amount of effort. However, once the network has been built, it is easy to add a new variable. �BBN are well suited to dealing with incomplete data. Instances with missing attributes can be interpolated �BBN is quite robust to model overfitting.