GAAP Guidelines for Accessible Assessments Project June 20

GAAP Project Overview • Focus is to develop research based guidelines for the audio and sign representation of CCSS test items. • Primary project activities: • Draft consensus based audio and sign guidelines • Conduct cognitive labs • Revise guidelines • Conduct experimentally designed research • Revise and finalize guidelines

The GAAP Team • Maryland State Department of Education • 17 Research and Partner States • WGBH National Center for Accessible Materials (audio experts) • National Center for Educational Outcomes (project evaluators) • National sign language experts and educators • Advisory Board Members • Measured Progress Innovation Lab

State Audio Guidelines for Assessment

State Audio Guidelines for Assessment States (and consortia) are making progress in improving reliability of audio accessibility! • Seven states provide oral/reader scripts that human administrators use to read tests to one more eligible student. • Eleven states provide pre-recorded versions of test content on CD or cassette that accompany paper-based test materials. • Three states provide a pre-programmed audio version of the test as part of their computer based test administration. • In 2012 Smarter Balanced contracted with the Measured Progress Innovation Lab to develop accessibility guidelines.

Framework for Audio Guidelines • Identify mathematics and ELA content categories • Select content that requires audio guideline • Identify current state guidelines for each content element • Work with WGBH/NCAM group to draft multiple guidelines for content that requires different representations based on the user (blind/low vision, print disability, etc) • Select content elements where there is not agreement across state guidelines to discuss as part of a series of consensus building webex sessions • Conduct research in 2013 -14 school year

Audio Guideline Issue Examples • How should the expression below be read? 3(x + 5) • How should the number below be read? 3. 2

Audio Guideline Item Example

Audio Guideline Item Example • Should audio feedback be provided for students with vision access needs?

Audio Item Examples • The GAAP Project website contains draft math and ELA audio guidelines and item examples • Item examples 1) describe an issue 2) present two items that show different ways of presenting the item • http: //nimbletools. com/gaap/ • Send us your feedback and opinions on the draft guidelines and different item representations!

State Sign Guidelines for Assessment • No states have content specific sign guidelines for assessment. Alaska, Massachusetts, and South Carolina provide a pre-recorded sign DVD for one or more grade levels/content areas. • Existing state documents provide guidelines for: • Qualification of sign language administrators • Warnings against cueing and elaboration • Use of “home signs” or “locally developed signs” • Testing environment (lighting, group size, etc).

Embedded Sign Support

Pre-Recorded ASL video interviews • Massachusetts provides an ASL DVD of grade 10 mathematics test. • Use a team development, recording, and review process. • Engage a deaf math educator and interpreter to develop sign videos along with MA content specialists, special education specialist, assessment vendor manager, and 23 interpreters. • Add some introductory and concluding text/signs to items to set the stage before signing item details. • Do not sign answer options. • Most frequent areas of disagreement are around finger spelling and any added text.

Pre-Recorded ASL video interviews • South Carolina provides an ASL DVD for all content areas in grades 5 -8, high school, and end of course assessments. • Use an independent three step development and review process: 1) script creation 2) video production 3) review • Engage a certified sign consultant to create scripts, interpreter to create sign videos, and a University professor with sign expertise for review. • Do not add introductory and concluding text/signs to items. • Do sign answer options. • Most frequent area of disagreement is around finger spelling.

Framework for Sign Guidelines • Introduction: rationale for guidelines, intended use • Team: roles and responsibilities in developing videos • Process: steps in creating high quality ASL videos • Content Specific Guidelines (use of space, plurality/repetition in presenting content, integration of prompt into answer options, use of “diamond” structure in representing items, direction on fingerspelling, specifying dimensionality (2 D, 3 D), etc). • Filming Considerations • ASL Glossary for Assessment Terms

Sign Item Examples • The GAAP Project website contains a sign report summarizing MA and SC interviews, the in-person sign meeting, and providing background on ASL as an accommodation. • Item examples 1) describe an issue 2) present two items that show different ways of presenting the item in ASL • http: //nimbletools. com/gaap/ • Send us your feedback and opinions on the draft sign report and different item representations!

Next Steps • Input from states and audio/sign teams on draft audio guidelines, sign report and item representations. • Conduct cognitive labs in Fall 2013. • Revise audio guidelines, draft sign guidelines. • Create item representations for experimentally designed research. • Recruit schools/students. • Conduct experimentally designed research in Winter 2013. • Release updated final version of audio and sign guidelines. The content of this presentation were developed under a grant from the U. S. Department of Education. However, those contents do not necessarily represent the policy of the U. S. Department of Education and you should not assume endorsement by the Federal government.