Regression as Moment Structure 1 Regression Equation Y

  • Slides: 25
Download presentation
Regression as Moment Structure 1

Regression as Moment Structure 1

Regression Equation Y = b X + v Observable Variables Y z = X

Regression Equation Y = b X + v Observable Variables Y z = X Moment matrix s. YY = s. YX s. XX Moment structure = (q) b 2 s. XX +svv bs. XX = bs. XX Parameter vector q = (b, s. XX, svv )’ 2

Sample: z 1, z 2, . . . , zn n iid Sample Moments

Sample: z 1, z 2, . . . , zn n iid Sample Moments S = n-1 zi zi’ S = syy syx sxx Fitting S to = (q) Estimator ^ q S close to =^ (q) ^ 3 moment equations • syy= b 2 s. XX +svv • syx= bs. XX • sxx= s. XX with 3 (unknown) parameters Parameter estimates ^ ^ q = (syx/sxx, s. XX, syy - (b )2 s. XX )’ ^ b is the same as the usual OLS estimate of b ! 3

Regression Equation Y = b x + v X = x + u Observable

Regression Equation Y = b x + v X = x + u Observable Variables Y z = X Moment structure = (q) b 2 s. XX +svv = bs. XX Parameter vector q = (b, s. XX, svv , suu )’ bs. XX + suu new parameter 4

Sample: z 1, z 2, . . . , zn n iid Sample Moments

Sample: z 1, z 2, . . . , zn n iid Sample Moments S : = S = n-1 zi zi’ syy syx sxx Fitting S to = (q) Estimator ^ q = S close to 3 moment equations • syy= b 2 sxx +svv • syx= bsxx • sxx= sxx + suu with 4 (unknown) parameters Parameter estimates ^ ^ = (q ^ ) ^ b is the same as the usual OLS estimate of b ! 5

The effect of measurement error in regression v b x X Y = b

The effect of measurement error in regression v b x X Y = b (X -u)+ v = b. X Y u + (v - bu) Note that w is correlated with X, = c. X + w, where w = v - bu unless u or b equals zero So, the classical LS estimate b of b is neither ubiased, neither consistent. In fact, b ---> s. YX/s. XX = b (sxx/s. XX )= kb k is the so called Fiability coefficient (reliability of X). Since 0 k 1 b suffers from downward bias 6

In multiple regression Regression Equation Y = b 1 x 1 + b 2

In multiple regression Regression Equation Y = b 1 x 1 + b 2 x 2. . . + b pxp+ v Xk = xk + uk Observable Variables b = SXX-1 SXY does not converge to b b* : = (SXX - Quu)-1 SXY Examples with EQS of regression with error in variables Using suplementary information to assessing the magnitude of variances of errors in variables. 7

Path analysis & covariance structure Example with ROS data 8

Path analysis & covariance structure Example with ROS data 8

Sample covariance matrix ROS 92 ROS 93 ROS 94 ROS 95 ROS 92 72.

Sample covariance matrix ROS 92 ROS 93 ROS 94 ROS 95 ROS 92 72. 07 29. 56 30. 21 27. 63 Mean: 6. 27 ROS 93 ROS 94 ROS 95 36. 21 31. 09 46. 51 24. 04 35. 19 46. 62 7. 35 10. 02 8. 80 n = 70 bj = ? F b 1 ROS 92 SEM: b 2 b 3 ROS 94 It is a valid model ? 9

Calculations b 1 b 2= 29. 56 b 1 b 3= 30. 21 b

Calculations b 1 b 2= 29. 56 b 1 b 3= 30. 21 b 2 b 3= 31. 09 b 1 b 2/b 1 b 3 = b 2/b 3 = 29. 56/30. 21 --> 31. 09 = b 2 b 3= b 3 (. 978 b 3) --> b 32= b 3 = 5. 64 b 2 =. 978 b 3 31. 09/. 978 In the same way, we obtain b 1=5. 34 b 2=5. 52 Model test in this case is CHI 2 = 0, df = 0 10

Fitted Model F 5. 34 5. 52 R 92 43. 34 R 93 5.

Fitted Model F 5. 34 5. 52 R 92 43. 34 R 93 5. 80 1 5. 64 R 94 14. 74 CHI 2 = 0, df = 0 11

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) /SPECIFICATIONS CAS=70; VAR=4; /LABEL V 1=ROS 92; V

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) /SPECIFICATIONS CAS=70; VAR=4; /LABEL V 1=ROS 92; V 2=ROS 93; V 3=ROS 94; V 4=ROS 95; /EQUATIONS V 1 = *F 1 + E 1; V 2 = *F 1 + E 2; V 3 = *F 1 + E 3; /VARIANCES F 1 = 1. 0; E 1 TO E 3 = *; /COVARIANCES /MATRIX 72. 07 29. 56 36. 21 30. 21 31. 09 46. 51 27. 63 24. 04 35. 19 46. 62 /END 12

ROS 92 =V 1 = 5. 359*F 1. 974 5. 504 + 1. 000

ROS 92 =V 1 = 5. 359*F 1. 974 5. 504 + 1. 000 E 1 ROS 93 =V 2 = 5. 516*F 1. 650 8. 482 + 1. 000 E 2 ROS 94 =V 3 = 5. 637*F 1 + 1. 000 E 3. 753 7. 482 VARIANCES OF INDEPENDENT VARIABLES -----------------E --E 1 -ROS 92 43. 347*I 8. 205 I 5. 283 I I E 2 -ROS 93 5. 789*I 3. 924 I 1. 475 I I E 3 -ROS 94 14. 736*I 4. 693 I 3. 140 I I D --I I I 13

… with the help of EQS RESIDUAL COVARIANCE MATRIX ROS 92 ROS 93 ROS

… with the help of EQS RESIDUAL COVARIANCE MATRIX ROS 92 ROS 93 ROS 94 V V V (S-SIGMA) : ROS 92 V 1 0. 000 1 2 3 CHI-SQUARE = ROS 93 V 2 0. 000 ROS 94 V 3 0. 000 BASED ON 0. 000 0 DEGREES OF FREEDOM STANDARDIZED SOLUTION: ROS 92 ROS 93 ROS 94 =V 1 =V 2 =V 3 = = = . 631*F 1. 917*F 1. 827*F 1 + + + . 776 E 1. 400 E 2. 563 E 3 14

one - factor four- indicators model F * R 92 * * R 93

one - factor four- indicators model F * R 92 * * R 93 * * * R 94 * R 95 * CHI 2 = ? , p-value = ? df = ? 15

… with the help of EQS /TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) ! This

… with the help of EQS /TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) ! This line is not read /SPECIFICATIONS CAS=70; VAR=4; /LABEL V 1=ROS 92; V 2=ROS 93; V 3=ROS 94; V 4=ROS 95; /EQUATIONS V 1 = *F 1 + E 1; V 2 = *F 1 + E 2; V 3 = *F 1 + E 3; V 4 = *F 1 + E 4; /VARIANCES F 1 = 1. 0; E 1 TO E 4 = *; /COVARIANCES /MATRIX 72. 07 29. 56 36. 21 30. 21 31. 09 46. 51 27. 63 24. 04 35. 19 46. 62 /END 16

… with the help of EQS ROS 92 =V 1 = ROS 93 =V

… with the help of EQS ROS 92 =V 1 = ROS 93 =V 2 = ROS 94 =V 3 = ROS 95 =V 4 = 4. 998*F 1. 966 5. 175 4. 837*F 1. 622 7. 779 + 1. 000 E 1 6. 417*F 1. 653 9. 833 + 1. 000 E 2 5. 393*F 1 + 1. 000 E 4. 710 7. 590 VARIANCES OF INDEPENDENT VARIABLES -----------------E --E 1 -ROS 92 47. 090*I 8. 437 I 5. 581 I I E 2 -ROS 93 12. 810*I 2. 775 I 4. 616 I I E 3 -ROS 94 5. 332*I 3. 017 I 1. 767 I I E 4 -ROS 95 17. 536*I 3. 682 I 4. 763 I D --- I I I I 17

Fitted Model F 4. 84 4. 99 R 92 47. 10 6. 42 R

Fitted Model F 4. 84 4. 99 R 92 47. 10 6. 42 R 93 12. 81 R 94 5. 33 5. 40 R 95 17. 54 CHI 2 = 6. 27, p-value =. 043 df = 2 19

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) /SPECIFICATIONS CAS=70; VAR=4; /LABEL V 1=ROS 92; V

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS) /SPECIFICATIONS CAS=70; VAR=4; /LABEL V 1=ROS 92; V 2=ROS 93; V 3=ROS 94; V 4=ROS 95; /EQUATIONS V 1 = *F 1 + E 1; V 2 = *F 1 + E 2; V 3 = *F 1 + E 3; V 4 = *F 1 + E 4; /VARIANCES F 1 = 1. 0; E 1 TO E 4 = *; /COVARIANCES /CONSTRAINTS (V 1, F 1)=(V 2, F 1)=(V 3, F 1)=(V 4, F 1); /MATRIX 72. 07 29. 56 36. 21 30. 21 31. 09 46. 51 27. 63 24. 04 35. 19 46. 62 /END 20

… estimation results ROS 92 =V 1 = 5. 521*F 1. 528 10. 450

… estimation results ROS 92 =V 1 = 5. 521*F 1. 528 10. 450 + 1. 000 E 1 ROS 93 =V 2 = 5. 521*F 1. 528 10. 450 + 1. 000 E 2 ROS 94 =V 3 = 5. 521*F 1. 528 10. 450 + 1. 000 E 3 ROS 95 =V 4 = 5. 521*F 1. 528 10. 450 + 1. 000 E 4 CHI-SQUARE = 12. 425 BASED ON 5 DEGREES OF FREEDOM PROBABILITY VALUE FOR THE CHI-SQUARE STATISTIC IS 0. 02941 21

. . . EQS use an iterative optimization method ITERATIVE SUMMARY ITERATION 1 2

. . . EQS use an iterative optimization method ITERATIVE SUMMARY ITERATION 1 2 3 4 5 6 7 8 9 10 PARAMETER ABS CHANGE 21. 878996 5. 741889 2. 309283 0. 477505 0. 147232 0. 056361 0. 014530 0. 005784 0. 001423 0. 000598 ALPHA 1. 00000 1. 00000 FUNCTION 1. 39447 0. 43985 0. 19638 0. 18079 0. 18014 0. 18008 0. 18007 22

Exercise: a) Write the covariance structure for the one - factor four- indicators model

Exercise: a) Write the covariance structure for the one - factor four- indicators model b) From the ML estimates of this model, shown in previous slides, compute the fitted covariance matrix. c) In relation with b), compute the residual covariance matrix Note: For c), use the following sample moments: ROS 92 ROS 93 ROS 94 ROS 95 ROS 92 72. 07 29. 56 30. 21 27. 63 Mean: 6. 27 n = 70 ROS 93 ROS 94 ROS 95 36. 21 31. 09 46. 51 24. 04 35. 19 46. 62 7. 35 10. 02 8. 80 23

one - factor four- indicators model with means F 1 * * R 92

one - factor four- indicators model with means F 1 * * R 92 * * R 93 * * * R 94 * * R 95 * CHI 2 = ? , p-value = ? df = ? 24

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS data) /SPECIFICATIONS CAS=70; VAR=4; ANALYSIS = MOMENT; /LABEL

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS data) /SPECIFICATIONS CAS=70; VAR=4; ANALYSIS = MOMENT; /LABEL V 1=ROS 92; V 2=ROS 93; V 3=ROS 94; V 4=ROS 95; /EQUATIONS V 1 = *V 999+ *F 1 + E 1; V 2 = *V 999+ *F 1 + E 2; V 3 = *V 999+ *F 1 + E 3; V 4 = *V 999+ *F 1 + E 4; /VARIANCES F 1 = 1. 0; E 1 TO E 4 = *; /COVARIANCES /CONSTRAINTS ! (V 1, F 1)=(V 2, F 1)=(V 3, F 1)=(V 4, F 1); /MATRIX 72. 07 29. 56 36. 21 30. 21 31. 09 46. 51 27. 63 24. 04 35. 19 46. 62 /MEANS 6. 27 7. 35 10. 02 8. 80 /END 25

ROS 92 =V 1 = 6. 270*V 999 + 4. 998*F 1 1. 022.

ROS 92 =V 1 = 6. 270*V 999 + 4. 998*F 1 1. 022. 966 6. 135 5. 175 ROS 93 =V 2 = 7. 350*V 999 + 4. 837*F 1. 724. 622 10. 146 7. 779 ROS 94 =V 3 = 10. 020*V 999 + 6. 417*F 1. 821. 653 12. 204 9. 833 ROS 95 =V 4 = 8. 800*V 999 + 5. 393*F 1. 822. 710 10. 706 7. 591 VARIANCES OF INDEPENDENT VARIABLES -----------------E --E 1 -ROS 92 47. 092*I 8. 437 I 5. 582 I I E 2 -ROS 93 12. 810*I 2. 775 I 4. 616 I I E 3 -ROS 94 5. 332*I 3. 017 I 1. 767 I I E 4 -ROS 95 17. 535*I 3. 682 I 4. 763 I + 1. 000 E 1 + 1. 000 E 2 + 1. 000 E 3 + 1. 000 E 4 D --I I I I 26

Moment dipolowy moment dipolowy wizania moment dipolowy czsteczkiMoment dipolowy moment dipolowy wizania moment dipolowy czsteczki
Moment dipolowy moment dipolowy wizania moment dipolowy czsteczkiMoment dipolowy moment dipolowy wizania moment dipolowy czsteczki
Couple Moment Couple moment Scalar formula Thus momentCouple Moment Couple moment Scalar formula Thus moment
Couple Moment Couple moment Scalar formula Thus momentCouple Moment Couple moment Scalar formula Thus moment
Moment Forces Moment The moment of a forceMoment Forces Moment The moment of a force
Moment of Inertia Moment of Inertia The momentMoment of Inertia Moment of Inertia The moment
Moment of Momentum Equation 1 Moment of aMoment of Momentum Equation 1 Moment of a
Gas Rate Equation General Equation Backpressure Equation LowpressureGas Rate Equation General Equation Backpressure Equation Lowpressure
Gas Rate Equation General Equation Backpressure Equation LowpressureGas Rate Equation General Equation Backpressure Equation Lowpressure
Balancing equation Chemical equation Word equation potassium waterBalancing equation Chemical equation Word equation potassium water
Structure Structure Structure 1 2 End Structure StructureStructure Structure Structure 1 2 End Structure Structure
Regression Analysis The regression equation is Sold 5Regression Analysis The regression equation is Sold 5
Y 4Y 5 Regression Analysis The regression equationY 4Y 5 Regression Analysis The regression equation
Least Squares Regression Line u Regression line equationLeast Squares Regression Line u Regression line equation
Regression Linear regression involves finding the equation ofRegression Linear regression involves finding the equation of
DIPOLE MOMENT Electronegativity Polarity of Bonds Dipole MomentDIPOLE MOMENT Electronegativity Polarity of Bonds Dipole Moment
Moment Tensor Inversion Symmetry of the Moment TensorMoment Tensor Inversion Symmetry of the Moment Tensor
MOMENT OF INERTIA Type of moment of inertiaMOMENT OF INERTIA Type of moment of inertia
Being in the Moment Being in the momentBeing in the Moment Being in the moment
MOMENT GENERATING FUNCTION AND STATISTICAL DISTRIBUTIONS 1 MOMENTMOMENT GENERATING FUNCTION AND STATISTICAL DISTRIBUTIONS 1 MOMENT
Moments Moment The moment of a force isMoments Moment The moment of a force is
Moment in 3 D Moment in 3 DMoment in 3 D Moment in 3 D
T1 MOMENT OF INERTIA Moment of Inertia TheT1 MOMENT OF INERTIA Moment of Inertia The
T1 MOMENT OF INERTIA Moment of Inertia TheT1 MOMENT OF INERTIA Moment of Inertia The