Exploring the Guttman Effect Statistics 300 Zhao Chen

The Guttman Effect • Introduction • The Guttman Effect and Bivariate Normal Distributions –

Introduction • The Guttman Effect, or horseshoe effect, occurs when two PC scores are

Introduction • Where does this effect come from? • Is it an artifact? How

Introduction Species Abundance versus Environment data, Boomer Lake (Oklahoma State University) The Guttman Effect.

Bivariate Normals The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Bivariate Normals • Is there an intuitive explanation? http: //phylonetworks. blogspot. com The Guttman

Bivariate Normals • Curvature reflects a metric that is only locally meaningful! http: //phylonetworks.

Bivariate Normals • However, it suggests existences of a gradient, or “extreme groups. ”

In-Depth Guttman on Political Data Diaconis et al. (2008) The Guttman Effect. Z. Chen,

In-Depth Guttman on Political Data The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data • Define A = 1 - P • A

Detrended Correspondence Analysis Two Options: - Force Linear and Quadratic Orthogonality between Dimensions -

Detrended Correspondence Analysis - Literature Recommends Subtracting Averages from Samples The Guttman Effect. Z.

Detrended Correspondence Analysis - Literature Recommends Subtracting Averages from Samples - Can be Improved

Detrended Correspondence Analysis - Can get rid of Gutman Effect - Can focus on

Conclusions - - - The Guttman Effect most commonly occurs on strongly correlated Gaussian

References - P Diaconis, S Goel, S Holmes (2008) Horseshoes in multidimensional scaling and

The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Slides: 40

Download presentation

Exploring the Guttman Effect Statistics 300 Zhao Chen Alexander Hristov Darvin Yi

The Guttman Effect • Introduction • The Guttman Effect and Bivariate Normal Distributions – An Application to our Mortality Data Set • In-Depth Guttman on Political Data – Connections to Linear Algebra • Detrended Correlation Analysis The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Introduction • The Guttman Effect, or horseshoe effect, occurs when two PC scores are related by a purely convex or concave function, despite the original relationship being more linear. Species Abundance versus Environment data, Boomer Lake (Oklahoma State University) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Introduction • Where does this effect come from? • Is it an artifact? How do we deal with it in our analyses? Species Abundance versus Environment data, Boomer Lake (Oklahoma State University) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Introduction Species Abundance versus Environment data, Boomer Lake (Oklahoma State University) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Bivariate Normals The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Bivariate Normals • Is there an intuitive explanation? http: //phylonetworks. blogspot. com The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Bivariate Normals • Curvature reflects a metric that is only locally meaningful! http: //phylonetworks. blogspot. com The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Bivariate Normals • However, it suggests existences of a gradient, or “extreme groups. ” http: //phylonetworks. blogspot. com The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data Diaconis et al. (2008) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data Diaconis et al. (2008) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data • Define A = 1 - P • A is Toeplitz. • A is totally positive. The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data • Define A = 1 - P • A is Toeplitz. => Eigenspace is split into odd and even functions • A is totally positive. => Even and odd functions are interlaced. The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Detrended Correspondence Analysis Two Options: - Force Linear and Quadratic Orthogonality between Dimensions - Difficult - Remove Quadratic Relationship (M. O. Hill 1980) The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Detrended Correspondence Analysis - Literature Recommends Subtracting Averages from Samples The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Detrended Correspondence Analysis - Literature Recommends Subtracting Averages from Samples - Can be Improved with HPF The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Detrended Correspondence Analysis - Can get rid of Gutman Effect - Can focus on Variance with Background Sub. - Assumes Guttman is Pure Artifact The Guttman Effect. Z. Chen, A. Hristov, D. Yi

Conclusions - - - The Guttman Effect most commonly occurs on strongly correlated Gaussian structure in the data matrix – a singledimensional “gradient” that opposes two extreme groups in our data. It tells us something about the underlying structure of our data, but the nonlinearities it seems to suggest is artifact. - The nonlinearities reflect failures of our distance metric to be reliable at large distances. Methods exist to artifically remove Guttman from data (e. g. DCA), but such methods are very ad-hoc and aggressive. The Guttman Effect. Z. Chen, A. Hristov, D. Yi

References - P Diaconis, S Goel, S Holmes (2008) Horseshoes in multidimensional scaling and local kernel methods. Annals of Applied Statistics. A. Baccini, H. Caussinus, A. de Falguerolles (1994) Diabolic Horseshoes. 9 th International Workshop on Statistical Modeling. J. De Leeuw (2008) A Horseshoe for Multidimensional Scaling. UCLA Department of Statistics Papers. M. O. Hill, H. G. Gauch Jr. (1980) Detrended correspondence analysis: An improved ordination technique. Plant Ecology. Principal Component Analysis. http: //ordination. okstate. edu/PCA. htm Distortions and artifacts in Principal Components Analysis – analysis of genome data. http: //phylonetworks. blogspot. com/2012/12/distortions-and-artifacts-in -pca. html M. Greenacre, J. Blasius. Multiple Correspondence Analysis and Related Methods. 2006. The Guttman Effect. Z. Chen, A. Hristov, D. Yi

The Guttman Effect. Z. Chen, A. Hristov, D. Yi

In-Depth Guttman on Political Data Diaconis et al. (2008) The Guttman Effect. Z. Chen, A. Hristov, D. Yi