The Intended and Unintended Consequences of the Use

The Intended and Unintended Consequences of the Use of Alternate Assessments Claudia Flower, Lynn Ahlgrim. Delzell, and Diane Browder University of North Carolina at Charlotte Meagan Karvonen Western Carolina University [http: //education. uncc. edu/cpflower] 1

Access to the General Education Curriculum l l Individuals with Disabilities Education Act (IDEA) 1997 & 2004 NCLB – 2 alternate assessments for students with disabilities unable to participate in statewide assessments (1%)

Students with Significant Cognitive Disabilities l Traditionally the content focus was nonacademic – – – 3 Excluded Preschool (developmental) Functional

Inclusion in Accountability l Positive consequences – – 4 access to the general curriculum improved instructional methods additional resources higher expectations for student learning

Consequences of Test Use l Standards for Educational and Psychological Testing – 5 “It is the responsibility of those who mandate the use of tests to monitor their impact and to identify and minimize potential negative consequences. Consequences resulting from the uses of the test, both intended and unintended, should also be examined by the test user” (p. 145).

Intended Consequences l l 6 Unlimited number of intended & unintended consequences Gathering evidence of the intended effects is first necessary (Lane, Parke, & Stone, 1998).

From the Literature l Results of accountability systems are intended to impact the following: – – 7 (a) implemented curriculum; (b) the instructional content and strategies; (c) the content and format of classroom assessments; (d) student, teacher and administrator motivation and effort;

From the Literature (cont) l l l 8 (e) learning experienced by all students; (f) the nature of professional development support; (g) teacher participation in administration, development, and scoring of assessments; (h) student, teacher, administrator, and public awareness and beliefs about the assessment, criteria for judging performance, and the use of the assessment results; and (i) the use and nature of test preparation materials

From the Literature (cont) l 9 see Cizek, 2001; Frederiksen & Collins, 1989; Koretz, Barron, Mitchell & Stecher, 1996; Koretz, Stecher, Klein, & Mc. Caffrey, 1994; Linn, 1993; Linn, Baker & Dunbar, 2001; Messick, 1992; and many others

Inclusion of Students with Significant Cognitive Disabilities l 10 What do we know? – Several studies out of Kentucky & Massachusetts (Kleinert & Colleagues; Zatta et al. ) and others… – Both benefits & challenges

Benefits – – – 11 inclusion in state accountability system raised expectations for students with significant disabilities increased focus on the general education curriculum increase in professional development activities

Challenges l competed with teaching time and meeting individual student needs – l 12 What’s important? increased the paperwork burden

Purpose l l Previous studies used teachers as the data source This study – – 13 Perspective of special education administrators Hypothesized potential intended & unintended consequences

Survey Research Method – Procedures l l – Participants l l 14 Post card to ~4500 special education administrators Responded via the internet 708 surveys from 49 states and the District of Columbia Most district-level administrators (65%)

Instrument (28 Items) l Six domains served to guide in item development: – – – 15 (a) access to the general curriculum [Access] (b) improved instruction [Instruct] (c) increase student expectations [Expect] (d) increased educational resources and training [Resources], (e) support from principal [Principal], and (f) increased workload and stress [Workload].

Results l Top Rated Items – – 16 More paperwork IEP reflects student’s needs vs. AA Increased teacher stress level Increased amt of required training

Lowest Rated Items l l 17 Principals have thorough understanding of AA process Raised general education teachers expectations Changed views about SWSCD Increased principal’s involvement in exceptional children programs

Exploratory Factor Analysis (EFA) l l l 18 Split the sample in half Principal factors extraction Six factors were extracted with eigenvalues greater than 1. 0 Skee plot leveled off Used loadings of. 4 for interpretation

EFA Results l l l 19 (a) instruction, (b) principal, (c) training, (d) workload, (e) access, and (f) support

Confirmatory Factor Analysis (CFA) l l l 20 Using the other half of the sample Six factor model was hypothesized with items identified in the EFA as indictors of the factors LISREL

Results of CFA l Marginal support – – – 21 2 (df=237, N=356) = 401. 06, p<. 000001 root mean square error of approximation (RMSEA) =. 05, and comparative fix index (CFI) =. 93

Post hoc Model Modifications l l l 22 basis of the Lagrange multiplier test and theoretical relevance error variances between the observable variables were allowed to correlate 2 (df=233, N=356) =320. 56, p=. 00012, RMSEA=. 037, comparative fix index (CFI) =. 96

Correlations between Factors Instruct Principal Training Work Access Instruct 23 Principal . 49 Training . 37 . 32 Work -. 47 -. 24 -. 22 Access . 82 . 61 . 20 -. 28 Support . 27 . 31 . 87 -. 28 . 12

Summary l 24 Regardless of the accuracy and consistency of identifying proficient students and schools, it is meaningless if nothing happens to improve the work of schools (Marion & Gong, 2003).

Continuous Improvement l l l 25 Alternate assessments are relatively new Using results to improve educational systems for students Starting point for looking at consequences of alternate assessments