STAT MINIPROJECT NKU Executive Doctoral Cohort August 2012

  • Slides: 24
Download presentation
STAT MINIPROJECT NKU Executive Doctoral Co-hort August, 2012 Dot Perkins

STAT MINIPROJECT NKU Executive Doctoral Co-hort August, 2012 Dot Perkins

Gallatin County High School Implemented E-prep in 2011 -2012 to improve ACT results My

Gallatin County High School Implemented E-prep in 2011 -2012 to improve ACT results My hypothesis is that students who utilize EPREP will score statistically higher on the ACT in March 2012 than students who did not utilize E-PREP, i. e. , the previous three junior classes ( 2011, 2010, 2009)

 I will compare the 2012 junior class ACT scores to the previous three

I will compare the 2012 junior class ACT scores to the previous three years (2011, 2010, 2009) junior classes ACT scores. These classes did not have access to EPREP. I will also compare the sophomore PLAN scores to each classes’ ACT scores to determine possible student growth or decline

E-PREP An on-line tool to quickly improve skills and confidence on the ACT test.

E-PREP An on-line tool to quickly improve skills and confidence on the ACT test. Each course features engaging, expert and personalized instruction, available 24/7 through on-demand videos and interactive lessons Student had access 24/7, 365 The high school scheduled E-PREP three times during the school year to benchmark

PLAN & ACT CLASS OF 2013 Summary statistics: Column n PLAN 2013 95 ACT

PLAN & ACT CLASS OF 2013 Summary statistics: Column n PLAN 2013 95 ACT 2013 95 Mean Variance Std. Dev. Std. Err. Media Range n Min Max Q 1 Q 3 17. 09 12. 57603 3. 546 0. 363 4736 55 2706 8397 17 16 11 27 14 20 4. 802 0. 492 4817 7242 19 24 11 35 15 22 19 23. 06383 *Gain of 2 points typical) growth. Shows promise. Not conclusive.

Boxplot of All PLAN and ACT Results for Class of 2013, 2012, 2011

Boxplot of All PLAN and ACT Results for Class of 2013, 2012, 2011

CLASS OF 2013 ACT – PLAN DIFFERENCE COMPARED TO CLASS OF 2012 & 2011

CLASS OF 2013 ACT – PLAN DIFFERENCE COMPARED TO CLASS OF 2012 & 2011 ACT & PLAN DIFFERENCE Colum n n PLAN 2013 95 ACT 2013 95 Mean Variance Std. Dev. Std. Err. Media Range n Min Max Q 1 Q 3 17. 094 12. 57603 3. 546 0. 3638 736 55 2706 397 17 16 11 27 14 20 4. 802 0. 4927 4817 242 19 24 11 35 15 22 19 23. 06383 PLAN 2012 & 2011 187 16. 240 10. 88264 3. 298 0. 2412 643 1 8846 3834 16 19 10 29 14 18 ACT 2012 & 2011 187 18. 283 24. 37622 4. 937 0. 3610 422 8 2287 4593 18 33 0 33 14 21

PLAN ACT DIFFERENCE 2013(E-PREP Access) 17. 095 19 +1. 905 2012 & 2011 16.

PLAN ACT DIFFERENCE 2013(E-PREP Access) 17. 095 19 +1. 905 2012 & 2011 16. 24 18. 28 +2. 04

Here are the hypothesis tests for combined ACT scores and growth. Hypothesis test results:

Here are the hypothesis tests for combined ACT scores and growth. Hypothesis test results: μ 1 : mean of ACT 2013 μ 2 : mean of ACT 2012 & 2011 μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean 0. 71657753 Std. Err. 0. 61084473 DF 193. 80548 T-Stat 1. 1730927 P-value 0. 1211 VERY slight indication of significance. Indicates difference in sample mean ACT scores is slightly unusual, if there is truly no difference in mean ACT scores for all students in these grades. On average, E-PREP group. 7 higher than non-E-PREP group

Hypothesis test results: μ 1 : mean of 2013 growth μ 2 : mean

Hypothesis test results: μ 1 : mean of 2013 growth μ 2 : mean of 2011/12 Growth μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean -0. 13751759 Std. Err. DF 0. 32211766 240. 59476 T-Stat -0. 42691728 P-value 0. 6651 ACT Growth –(minus) PLAN: How much growth? Non-EPREP actually increased slightly more from PLAN To ACT (****Starter higher*****)

Here are the hypothesis tests for ACT and growth comparisons for each year (2013

Here are the hypothesis tests for ACT and growth comparisons for each year (2013 vs 2012, 2013 vs 2011) Hypothesis test results: μ 1 : mean of ACT 2013 μ 2 : mean of ACT 2012 μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean 0. 9292929 Std. Err. 0. 7377061 DF 190. 55873 T-Stat 1. 2597061 P-value 0. 1047 VERY slight indication of significance. Indicates difference in sample mean ACT scores is slightly unusual, if there is truly no difference in mean ACT scores for all students in these grades.

Hypothesis test results: μ 1 : mean of 2013 Growth μ 2 : mean

Hypothesis test results: μ 1 : mean of 2013 Growth μ 2 : mean of 2012 Growth μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean -0. 034130782 Std. Err. 0. 4205581 DF 169. 8504 T-Stat P-value -0. 081155926 P-value: No evidence that there is growth/change between the two groups E-PREP and Non-E-PREP groups. *Slight indication that ACT scores are higher but no Conclusive evidence that E-PREP impacted. 0. 5323

Hypothesis test results: μ 1 : mean of ACT 2013 μ 2 : mean

Hypothesis test results: μ 1 : mean of ACT 2013 μ 2 : mean of ACT 2011 μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean 0. 47727272 Std. Err. 0. 67212915 DF 180. 76358 T-Stat 0. 71009076 P-value 0. 2393 P-value: No indication of a difference between ACT of 2013 & 2011. Not unusual.

Hypothesis test results: μ 1 : mean of 2013 Growth μ 2 : mean

Hypothesis test results: μ 1 : mean of 2013 Growth μ 2 : mean of 2011 Growth μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 HA : μ 1 - μ 2 > 0 (without pooled variances) Difference μ 1 - μ 2 Sample Mean -0. 25382775 Std. Err. 0. 35007808 DF 177. 18616 T-Stat P-value -0. 7250604 P-value: . 7 No evidence of a difference in change between 2013 & 2011. 0. 7653

Quantitative Data Examination A summative sample of ACT test results data was obtained and

Quantitative Data Examination A summative sample of ACT test results data was obtained and compiled from Gallatin County High School Conclusions: *E-PREP was implemented with all juniors, sophomores, and freshmen. This on-line program has potential because EPREP group exhibited higher ACT scores. The data are also inconclusive because there is no difference in the growth. (There are higher ACT scores for the first E-PREP class. ***May just have had a good class last year)

Quantitative Data Examination Checking assumptions: samples were selected independently; number of individuals in each

Quantitative Data Examination Checking assumptions: samples were selected independently; number of individuals in each group exceeds 30; standard deviation of ACT and growth scores for all students in each class is unknown. Potential threat to validity is the lack of random selection. (How was the program used? Consistently? )

Qualitative Data Examination A focus group of eight randomly selected juniors was held to

Qualitative Data Examination A focus group of eight randomly selected juniors was held to discuss with them their impressions, observations, likes and dislikes of using E-PREP as a tool to improve their performance on the ACT. The conversation was recorded and transcribed.

QUESTIONS 1. How many times did you take E-PREP? How much time was involved?

QUESTIONS 1. How many times did you take E-PREP? How much time was involved? Did you ever utilize E-PREP at home? The public library? On your cell phone? 2. After experiencing E-PREP, did you find it rigorous/difficult/challenging? Why or why not? 3. Did you give E-PREP your best effort? Why or why not? 4. As a result of participating in E-PREP, do you think your ACT score improved? Why or why not? 5. Were there other strategies/things that you did to improve your ACT score along with using E_PREP? If so, what were these?

QUESTIONS 6. Did the technology work for you? 7. Did your teacher, principal, assistant

QUESTIONS 6. Did the technology work for you? 7. Did your teacher, principal, assistant principal, or guidance counselor discuss your E-PREP results with you? How? When? 8. If your teacher reviewed your E-PREP results with you, did instruction within your classroom change as a result of theses results in order to help you improve your ACT score? How? 9. If our current sophomores who will be juniors use E-PREP, do you think this program would benefit them in their ACT preparation? Why or why not? 10. What could be improved in using E-PREP if anything? Why?

Qualitative Data Conclusions After convening this focus group of students to discuss EPREP the

Qualitative Data Conclusions After convening this focus group of students to discuss EPREP the following are common observations and impressions: *Students desire a set schedule of when E-PREP will be given. Let them know when they will be expected to perform. *Give students time to do the tutoring and individual lessons that E-PREP offers to help students improve. *Individualize instruction so students can improve once they get their E-PREP results back. Students want teachers to go over their results with them (Feedback).

 *Students suggested counting E-PREP as part of grade. Recognize students who are working

*Students suggested counting E-PREP as part of grade. Recognize students who are working hard and doing what they’re supposed to do. *Time – Give students time to read questions they missed and the correct answers. *Need full implementation – E-PREP will do more. We don’t know how to use it all yet. *All teachers need to take E-PREP seriously

NOTE The variability in administration approaches and use of E-PREP from class to class

NOTE The variability in administration approaches and use of E-PREP from class to class may undermine the validity of statistical analyses. Suggest greater uniformity of use, if results are to be studied statistically.

Final Conclusions *Consensus of the focus group is that E-PREP is a valuable tool

Final Conclusions *Consensus of the focus group is that E-PREP is a valuable tool and does impact ACT results by giving students confidence in the types of questions that are asked on this assessment and the time constraints that are imposed with this test. *E-PREP will continue to be implemented next year and more data will be collected to determine value of this program on improving ACT scores among our juniors.