Linear Model YI NG SHE N SSE TON
- Slides: 52
Linear Model YI NG SHE N SSE, TON GJI UNIVERSITY SEP. 2016
The basic form of the linear model 10/31/2020 PATTERN RECOGNITION 2
Linear regression Hours Spent Studying Math SAT Score 10/31/2020 4 9 10 14 4 7 12 22 1 3 8 390 580 650 730 410 530 600 790 350 400 590 PATTERN RECOGNITION 3
Linear regression 10/31/2020 PATTERN RECOGNITION 4
Linear regression Mean squared error (MSE) is a commonly used performance measure: We want to minimize MSE between f(xi) and yi: 10/31/2020 PATTERN RECOGNITION 5
Linear regression The method of determining the fitting model based on MSE is called the least square method In linear regression problem, the least square method aims to find a line such that the sum of distances of all the samples to it is the smallest. 10/31/2020 PATTERN RECOGNITION 6
Pre-requisite A stationary point of a differentiable function of one variable is a point of the domain of the function where the derivative is zero Single-variable function: f(x) is differentiable in (a, b). At x 0, Two-variables function: f(x, y) is differentiable in its domain. At (x 0, y 0), 10/31/2020 PATTERN RECOGNITION 7
Pre-requisite 10/31/2020 PATTERN RECOGNITION 8
Parameter estimation 10/31/2020 PATTERN RECOGNITION 9
Parameter estimation 10/31/2020 PATTERN RECOGNITION 10
Multivariate linear regression 10/31/2020 PATTERN RECOGNITION 11
Pre-requisite 10/31/2020 PATTERN RECOGNITION 12
Pre-requisite Matrix differentiation 2. Function is a matrix and the variable is a scalar Definition 10/31/2020 PATTERN RECOGNITION 13
Pre-requisite Matrix differentiation 3. Function is a scalar and the variable is a vector Definition In a similar way 10/31/2020 PATTERN RECOGNITION 14
Pre-requisite Matrix differentiation 4. Function is a vector and the variable is a vector Definition 10/31/2020 PATTERN RECOGNITION 15
Pre-requisite Matrix differentiation 5. Function is a vector and the variable is a vector In a similar way 10/31/2020 PATTERN RECOGNITION 16
Pre-requisite Matrix differentiation 5. Function is a vector and the variable is a vector Example 10/31/2020 PATTERN RECOGNITION 17
Pre-requisite 10/31/2020 PATTERN RECOGNITION 18
Pre-requisite 10/31/2020 PATTERN RECOGNITION 19
Multivariate linear regression 10/31/2020 PATTERN RECOGNITION 20
Multivariate linear regression 10/31/2020 PATTERN RECOGNITION 21
Generalized linear model 10/31/2020 PATTERN RECOGNITION 22
Logistic regression 10/31/2020 PATTERN RECOGNITION 23
Logistic regression 10/31/2020 PATTERN RECOGNITION 24
Logistic regression 10/31/2020 PATTERN RECOGNITION 25
Logistic regression Task: Determine w and b in Solution: Step 1. y → p ( y = 1 | x ) Step 2. Estimate w and b using maximum likelihood method 10/31/2020 PATTERN RECOGNITION 26
Pre-requisite: maximum likelihood estimation 10/31/2020 PATTERN RECOGNITION 27
Pre-requisite: maximum likelihood estimation 10/31/2020 PATTERN RECOGNITION 28
Pre-requisite: maximum likelihood estimation 10/31/2020 PATTERN RECOGNITION 29
Logistic regression 10/31/2020 PATTERN RECOGNITION 30
Logistic regression 10/31/2020 PATTERN RECOGNITION 31
Pre-requisite: Newton’s method The name "Newton's method" is derived from Isaac Newton's description of a special case of the method in De analysi per aequationes numero terminorum infinitas (written in 1669, published in 1711 by William Jones) and in De metodis fluxionum et serierum infinitarum(written in 1671, translated and published as Method of Fluxions in 1736 by John Colson). Newton’s method was first published in 1685 in A Treatise of Algebra both Historical and Practical by John Wallis. In 1690, Joseph Raphson published a simplified description in Analysis aequationum universalis. 10/31/2020 PATTERN RECOGNITION 32
Pre-requisite: Newton’s method 10/31/2020 PATTERN RECOGNITION 33
Pre-requisite: Newton’s method 10/31/2020 PATTERN RECOGNITION 34
Pre-requisite: Newton’s method 10/31/2020 PATTERN RECOGNITION 35
Pre-requisite: Newton’s method 10/31/2020 PATTERN RECOGNITION 36
Logistic regression 10/31/2020 PATTERN RECOGNITION 37
Logistic regression The first and second derivatives of are given in the book Assignment 1: Implemented logistic regression model using matlab (R, Python, or any language you are familiar) You can use any dataset in UCI repository to validate your model Select two attributes from the dataset and plot a figure like this → 10/31/2020 PATTERN RECOGNITION 38
Pre-requisite: Lagrange multiplier 10/31/2020 PATTERN RECOGNITION 39
Pre-requisite: Lagrange multiplier n+m equations! 10/31/2020 PATTERN RECOGNITION 40
Pre-requisite: Lagrange multiplier 10/31/2020 PATTERN RECOGNITION 41
Pre-requisite: Lagrange multiplier 10/31/2020 PATTERN RECOGNITION 42
Linear discriminant analysis In a two-classification problem, given n samples in a ddimensional feature space. There are n 1 samples belong to class 1 and n 2 samples belong to class 2. Goal: to find a vector w, and project the n samples on the axis y = w. Tx, so that the projected samples are well separated. 10/31/2020 PATTERN RECOGNITION 43
Linear discriminant analysis 10/31/2020 PATTERN RECOGNITION 44
Linear discriminant analysis 10/31/2020 PATTERN RECOGNITION 45
Linear discriminant analysis 10/31/2020 PATTERN RECOGNITION 46
Linear discriminant analysis 10/31/2020 PATTERN RECOGNITION 47
Multiclassification Ov. O “+” “-” C 1 C 2 C 1 C 3 C 1 C 4 C 2 C 3 C 2 C 4 C 3 C 4 10/31/2020 C 1 C 2 C 3 C 4 Ov. R “+” “-” C 1 C 2 C 3 C 4 “-” C 2 C 1 C 3 C 4 “-” C 3 C 1 C 2 C 4 “+” C 4 C 1 C 2 C 3 “-” N(N-1)/2 classifiers PATTERN RECOGNITION 48
Multiclassification Hamming distance test sample 10/31/2020 -1 +1 +1 +1 -1 -1 +1 +1 -1 -1 +1 PATTERN RECOGNITION Euclidian distance 49
Class-imbalance 10/31/2020 PATTERN RECOGNITION 50
Class-imbalance 10/31/2020 PATTERN RECOGNITION 51
Class-imbalance Undersampling ◦ Easy. Ensemble [Liu et al. , 2009] Oversampling ◦ SMOTE [Chawla et al. , 2002] Threshold-moving 10/31/2020 PATTERN RECOGNITION 52
- Sse linear regression
- Que ton aliment soit ton médicament
- Gama cromatica
- 0 856m = cm
- Contoh soal indeks marshall edgeworth
- Sse
- Simd
- Mmx sse
- Prof.sse
- Sse coverage map
- Sse-cmm
- Sse iphone app
- Nersc job script generator
- Ssecmm
- Trade and barter sse
- Sse can never be
- Direct control pointing devices
- Irny
- She is beautiful isn't she grammar
- He had had breakfast before she came
- Who she is or who is she
- Who was she? where was she? what was happening?
- Pack she back to she ma
- She looks pretty sick. i think she go to a doctor answer
- She got the job because she
- She worked hard. she made herself ill
- I like shopping
- Little a little few a few
- Limitations of linear model of communication
- Primal dual program linear
- Linear regression vs multiple regression
- Contoh soal metode biseksi
- Linear texts
- Linear vs nonlinear plot
- Contoh soal persamaan non linear
- Which pipeline is linear
- Linear vs nonlinear multimedia
- Right linear to left linear grammar
- Fungsi linier dan non linier
- Penerapan fungsi non linier dalam ekonomi
- What is a linear transformation linear algebra
- Linear algebra 1
- Titik penggal adalah
- Linear impulse
- Persamaan linier simultan
- Linear or non linear
- Linear or nonlinear tables
- Difference between linear and non linear equations
- Nonlinear video editing
- Persamaan linear simultan
- Closure properties of regular languages
- Textos textos narrativos
- Concentric zone model image