Comparison of Bayesian Neural Networks with TMVA classifiers

Comparison of Bayesian Neural Networks with TMVA classifiers Richa Sharma, Vipin Bhatnagar Panjab University, Chandigarh India-CMS March, 2009 Meeting, DU, Delhi

CONTENTS ● Multivariate techniques and their advantages ● BNN -Introduction -Algorithm -Software used ● Work done ● Comparison of BNN with some of the classifiers in TMVA

MULTIVARIATE TECHNIQUES AND THEIR ADVANTAGES ● Many statistical techniques focus on just one or two variables ● Multivariate analysis techniques allow more than two variables to be analyzed at once ● Useful when there is correlation among the variables ● More information is analyzed simultaneously, giving greater power ● Variable selection (e. g. , to give maximum signal/background discrimination)

TMVA(Toolkit for Multivariate Analysis) The Toolkit for Multivariate Analysis (TMVA) provides a ROOT-integrated machine learning environment for the processing and parallel evaluation of sophisticated multivariate classification techniques The package includes: Rectangular cut optimization Projective likelihood estimation (PDE approach) Multidimensional probability density estimation (PDE – range-search approach) Multidimensional k-nearest neighbour classifier Linear discriminant analysis (H-Matrix and Fisher discriminants) Function discriminant analysis (FDA) Artificial neural networks (three different implementations) Boosted/Bagged decision trees Predictive learning via rule ensembles (Rule. Fit) Support Vector Machine (SVM)

Neural Networks ● An extremely simplified model of the brain ● Transforms inputs into output to the best of its ability ● Output node The basic computational element is often called a Input nodes node or unit ● Each input has an associated weight w, which can be modified ● The unit computes some function ‘f ’ of the weighted sum of its inputs: Hidden nodes

Bayesian Neural Networks BNN is a model for classification based on Bayesian Statistics The goal of BNN is to approximate the posterior class conditional probabilities BAYES THEOREM P(Ck|x) = P(x|Ck)P(Ck)/P(x) x = (x 1, x 2, . . x. P) where P is the no. of variables Ck are the classes(signal and background) P(Ck|x) = conditional probability, that an object described by x belongs to class Ck P(x|Ck) = conditional probability that an object is of the form x

FBM(Flexible Bayesian Modelling Package) by Radford M. Neal The package is organized in a modular fashion ● UTIL - specify final portion of a probabilistic model that supports reading of numeric input from data files or other sources , & specify sets of training and test cases ● MC - provides support for MCMC methods ● DIST – contains programs for sampling from a distribution specified by formulas for the prior and likelihood for a Bayesian model ● NET – implements Bayesian learning based on MLP networks

WORK DONE ● Ported BNN package being used in D 0 to a local machine as a standalone package - macros modified to remove D 0 dependence - library dependence changed for compilation ● No D 0 environment specific dependence ● Tested it for independent samples ● Compared the efficiency of signal/background discrimination of BNN with some of the classifers in TMVA (Thanks to : S. Jain, OSU/D 0 for valuable inputs on BNN setup/testing)

STEPS INVOLVED IN MAKING AN ANALYSIS Make a list of signal and background files and variables for which to be trained Read the root files and write out the variables in a text file Run Markov Chain Run FBM code to get the Bayesian probabilities Average over the previous iterations BNN filter created, used in final analysis with data Create unweighted events from weighted ones Mix signal and background training files

BNN performance using D 0 MC data

Verification plots

BNN performance using a toy example

Plots showing efficiency of the BNN

Efficiency Plots of the classifiers in TMVA

COMPARISON BETWEEN BNN and TMVA CLASSIFIERS CLASSIFIER S / √(S+B) CUT 1. BDT 61. 77 -0. 14 2. LIKELIHOOD 59. 80 0. 18 3. PDERS 62. 68 0. 38 4. FISHER 63. 44 -0. 07 MLP 63. 49 -0. 22 BNN 63. 61 0. 35 6.

Results and future plan of work ● ● ● The efficiency of BNN is comparable to the classifiers in TMVA, therefore good enough to promote BNN for CMS Can be used as a cross check to the studies already done using TMVA Therefore we plan to test it on data samples available in our group and then use it for physics studies

The aim is to find an optimum classifier n(x, ) that minimizes the rate of misclassification ● Minimize the empirical risk function with respect to E( )=1/N S ([ti-n(xi, )])2 ● One can show that the error gets minimized when n(x, )=∫t p(t|x)dt ● This yields one sets of weights which corresponds to a mapping function n(x, ) ● Prone to overfitting ● Therefore we will use BNN i. e. training over the entire space n(x'|t, x) = ∫n(x', )p( |t, x)d ●Will produce a function n that is more stable and is less likely to be overtrained ●