# Principal Nested Spheres Backwards PCA Backwards PCA Backwards

- Slides: 78

Principal Nested Spheres •

Backwards PCA •

Backwards PCA •

Backwards PCA Euclidean Settings: Forwards PCA = Backwards PCA (Pythagorean Theorem, ANOVA Decomposition) So Not Interesting But Very Different in Non-Euclidean Settings (Backwards is Better !? !)

Backwards PCA •

Nonnegative Matrix Factorization •

Nonnegative Matrix Factorization Standard NMF (Projections All Inside Orthant)

Nonnegative Matrix Factorization Standard NMF But Note Not Nested No “Multi-scale” Analysis Possible (Scores Plot? !? )

Nonnegative Nested Cone Analysis Same Toy Data Set All Projections In Orthant

Nonnegative Nested Cone Analysis Same Toy Data Set Rank 1 Approx. Properly Nested

1 st Principal Curve Linear Reg’n Proj’s Reg’n Usual Smooth Princ’l Curve

Manifold Learning How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Perceived Major Challenge: How to find 2 nd Principal Curve?

Manifold Learning Key Component: Principal Surfaces Le. Blanc & Tibshirani (1996) Challenge: Can have any dimensional surface, But how to nest? ? ? Proposal: Backwards Approach

An Interesting Question How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Key Idea: Express Backwards PCA as Nested Series of Constraints

• General View of Backwards PCA

General View of Backwards PCA Define Nested Spaces via Constraints • Backwards PCA • Principal Nested Spheres • Principal Surfaces • Other Manifold Data Spaces Sub-Manifold Constraints? ? (Algebraic Geometry)

Detailed Look at PCA Now Study “Folklore” More Carefully • Back. Ground • History • Underpinnings (Mathematical & Computational) Good Overall Reference: Jolliffe (2002)

PCA: Rediscovery – Renaming Statistics: Principal Component Analysis (PCA) Social Sciences: Factor Analysis (PCA is a subset) Probability / Electrical Eng: Karhunen – Loeve expansion Applied Mathematics: Proper Orthogonal Decomposition (POD) Geo-Sciences: Empirical Orthogonal Functions (EOF)

An Interesting Historical Note The 1 st (? ) application of PCA to Functional Data Analysis: Rao (1958) 1 st Paper with “Curves as Data Objects” viewpoint

Detailed Look at PCA Three Important (& Interesting) Viewpoints: 1. Mathematics 2. Numerics 3. Statistics Goal: Study Interrelationships

Review of Linear Algebra (Cont. ) SVD Full Representation: = Diagonal Matrix of Singular Values

Review of Linear Algebra (Cont. )

Review of Linear Algebra (Cont. )

Review of Linear Algebra (Cont. ) Isometry (~Rotation) Coordinate Rescaling Isometry (~Rotation)

Covariance Matrices

Covariance Matrices Are Data Objects Essentially Inner Products of Rows

Covariance Matrices

Covariance Matrices

PCA as an Optimization Problem Find Direction of Greatest Variability: Centered At Mean

PCA as an Optimization Problem

PCA as Optimization (Cont. ) Same Notation as Above

PCA as Optimization (Cont. ) Sample Variance of Scores

PCA as Optimization (Cont. ) Looks Like “Products Of Rows” From Above Cov. Matrix Repr’n

PCA as Optimization (Cont. ) Diagonal Matrix of Eigenvalues

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. )

PCA as Optimization (Cont. ) Modes of Variation

Connect Math to Graphics 2 -d Toy Example From Much Earlier Class Meeting 2 -d Curves as Data In Object Space Simple, Visualizable Feature Space

Connect Math to Graphics 2 -d Toy Example

Connect Math to Graphics 2 -d Toy Example

PCA Redistribution of Energy Now for Scree Plots (Upper Right of FDA Anal. ) Carefully Look At: Ø Intuition Ø Relation to Eigenanalysis Ø Numerical Calculation

PCA Redist’n of Energy (Cont. )

PCA Redist’n of Energy (Cont. )

PCA Redist’n of Energy (Cont. ) Note, have already considered some of these Useful Plots: • Power Spectrum (as %s) • Cumulative Power Spectrum (%) Recall Common Terminology: Power Spectrum is Called “Scree Plot” Kruskal (1964) Cattell (1966) (all but name “scree”) (1 st Appearance of name? ? ? )

PCA vs. SVD Sometimes “SVD Analysis of Data” = Uncentered PCA = 0 -Centered PCA

PCA vs. SVD Sometimes “SVD Analysis of Data” = Uncentered PCA = 0 -Centered PCA Consequence: Skip this step

PCA vs. SVD •

PCA vs. SVD •

PCA vs. SVD Sometimes “SVD Analysis of Data” = Uncentered PCA = 0 -Centered PCA Investigate with Similar Toy Example

PCA vs. SVD 2 -d Toy Example Direction of “Maximal Variation”? ? ?

PCA vs. SVD 2 -d Toy Example Direction of “Maximal Variation”? ? ? PC 1 Solution (Mean Centered) Very Good!

PCA vs. SVD 2 -d Toy Example Direction of “Maximal Variation”? ? ? SV 1 Solution (Origin Centered) Poor Rep’n

PCA vs. SVD 2 -d Toy Example Look in Orthogonal Direction: PC 2 Solution (Mean Centered) Very Good!

PCA vs. SVD 2 -d Toy Example Look in Orthogonal Direction: SV 2 Solution (Origin Centered) Off Map!

PCA vs. SVD 2 -d Toy Example SV 2 Solution Larger Scale View: Not Representative of Data

PCA vs. SVD Sometimes “SVD Analysis of Data” = Uncentered PCA Investigate with Similar Toy Example: Conclusions: ü PCA Generally Better ü Unless “Origin Is Important” Deeper Look: Zhang et al (2007)

Different Views of PCA Solves several optimization problems: 1. Direction to maximize SS of 1 -d proj’d data

Different Views of PCA 2 -d Toy Example 1. Max SS of Projected Data

Different Views of PCA Solves several optimization problems: 1. Direction to maximize SS of 1 -d proj’d data 2. Direction to minimize SS of residuals

Different Views of PCA 2 -d Toy Example 1. Max SS of Projected Data 2. Min SS of Residuals

Different Views of PCA Solves several optimization problems: 1. Direction to maximize SS of 1 -d proj’d data 2. Direction to minimize SS of residuals (same, by Pythagorean Theorem) 3. “Best fit line” to data in “orthogonal sense” (vs. regression of Y on X = vertical sense & regression of X on Y = horizontal sense)

Different Views of PCA 2 -d Toy Example 1. Max SS of Projected Data 2. Min SS of Residuals 3. Best Fit Line

Different Views of PCA Toy Example Comparison of Fit Lines: Ø PC 1 Ø Regression of Y on X Ø Regression of X on Y

Different Views of PCA Normal Data ρ = 0. 3

Different Views of PCA Projected Residuals

Different Views of PCA Vertical Residuals (X predicts Y)

Different Views of PCA Horizontal Residuals (Y predicts X)

Different Views of PCA Projected Residuals (Balanced Treatment)

Different Views of PCA Toy Example Comparison of Fit Lines: Ø PC 1 Ø Regression of Y on X Ø Regression of X on Y Note: Big Difference Prediction Matters

Different Views of PCA Solves several optimization problems: 1. Direction to maximize SS of 1 -d proj’d data 2. Direction to minimize SS of residuals (same, by Pythagorean Theorem) 3. “Best fit line” to data in “orthogonal sense” (vs. regression of Y on X = vertical sense & regression of X on Y = horizontal sense) Use one that makes sense…

Participant Presentation Siyao Liu: Clustering Single Cell RNAseq Data

- Backwards intergration
- Backwards vagina
- How do you steer straight forward select all that apply
- Game theory in operation research
- Google backwards image search
- Lateral thinking riddle
- Backwards bending labour supply curve
- Working backwards problem solving examples
- Backwards chaining
- Backwards goal setting
- A fireman turns on a hose and is knocked backwards
- The one was a ship
- Sohcahtoa triangle
- Backwards induction
- Naming compounds working backwards
- Which bird can rotate its neck backwards to a large extent
- What is makaveli spelled backwards
- Months of the year backwards
- Backwards spelling bee
- Backwards design software
- Les accents in french
- Bass clef n
- Doti coin
- Two spheres of mass 35kg are 60m apart
- Cone volume ratio
- Earth's spheres foldable
- Networking spheres
- Four spheres of the earth
- Argument spheres
- Earth's spheres
- Polystyrene refractive index
- Lesson 9.3 volume of spheres answer key
- Finding the volume of cylinders pyramids cones and spheres
- How does biosphere interact with hydrosphere
- Starry spheres figure of speech
- 4 spheres of the earth poster
- Sphere of self sufficiency
- The 4 spheres of knowledge management are:
- Sphere of earth
- Geosphere drawing
- Four major spheres of the earth
- Clouds are not spheres
- Keypoint matching
- What are the three spheres of quality
- 13-1 volume of cylinders
- Sphere of consensus
- Identify which sphere of earth each image represents.
- How can you compare the terrarium to an earth system
- Volume maze
- Earth systems foldable
- Volume of prisms and cylinders
- Seven spheres of physical activity
- Cryosphere hydrosphere biosphere geosphere and atmosphere
- A 25 gram paper cup falls from rest
- 10-1 circles and circumference quiz answers
- Galatea of the spheres meaning
- Volume maille monoclinique
- 12-6 surface areas and volumes of spheres answers
- What is this
- 4 spheres of the earth
- Three stage nested design
- Features of 80186 microprocessor
- Dame meaning in baa black sheep
- Nested decision structures in python
- Flowchart nested if
- Contoh program nested loop c++
- Struct
- Overfitting loss curve
- Eecs 1019
- Nested loops python
- An array is a structured data type
- Nested state diagram
- Case control study
- Nested quantifiers exercises
- Nested anova spss
- Milazy
- Xml elements must be properly nested
- Contoh soal nested loop
- Rancangan tersarang