Assessment and Accountability in Kentucky Phyllis Shuttleworth Program
Assessment and Accountability in Kentucky Phyllis Shuttleworth Program Consultant Kentucky Department of Education
Assessment and Accountability in Kentucky n Assessment • Components • Standards n Accountability • • Raw/Scale/Cut Scores Accountability Indices Growth Chart Rewards/Assistance
COMMONWEALTH ACCOUNTABILITY TESTING SYSTEM
Assessment
Two Basic Types of Assessments w Criterion-referenced or Standards-Based • student performance is measured against a set standard • Norm-referenced - student performance is measured against the performance of other students
Kentucky’s Assessment Program
ELEMENTARY TESTING
MIDDLE SCHOOL TESTING
HIGH SCHOOL TESTING
WHO CREATED CATS? 4 CTB/MCGRAW-HILL COMPANY TESTING
WHO CREATED CATS? 4 OVER 300 TEACHERS INVOLVED: Blueprint Construction, Content Advisory Committee, NRT/Core Content Comparison, and Bias Screening 4 OVER 6200 EDUCATORS, PARENTS, & CITIZENS SHARED INPUT
WHO CREATED CATS? 4 ADVISORY GROUPS: EAARS - Education Assessment and Accountability Review Subcommittee NTAPAA - National Technical Advisory Panel on Assessment and Accountability OEA - Office of Education Accountability SCAAC - School Curriculum, Assessment and Accountability Council
WHO CREATED CATS?
WHO CREATED CATS? 4 STATE BOARD OF EDUCATION
Norm-Referenced Test n n n CTBS/5 Survey Edition Norm-Referenced Spring Administration Multiple-Choice Created by CTB Mc. Graw-Hill (testing contractor)
WHY HAVE NRT TESTS? 4 Tests basic skills 4 Counts in accountability formula 4 Measures basic skills across the country 4 Compares students nationally
Writing Portfolio n Standards-Based n Developed Over Time n Collection of Student’s Best Work n Created from Classroom Assignments
GRADE 4 PORTFOLIOS è 4 pieces includes: Reflective (letter to reviewer) Personal Literary Transactive è 1 piece from content area other than language arts
GRADE 7 PORTFOLIOS *5 Pieces: A. Reflective (letter to reviewer) B. C. D. E. Personal Literary Transactive Choice of B-D *1 piece must be from content area other than language arts/English
GRADE 12 PORTFOLIOS *5 Pieces: A. Reflective (letter to reviewer) B. Personal C. Literary D. Transactive E. Choice of B-D *2 pieces must be from content area other than language arts/English
Kentucky Core Content Test (KCCT) n n Standards-Based Spring Administration Includes Open. Response, Multiple. Choice, & Writing Prompts Created by Kentucky Teachers Based on KY’s Core Content for Assessment
KENTUCKY CORE CONTENT TESTS 4 * * 8 MULTIPLE CHOICE AND 2 OPEN RESPONSE IN EACH: ARTS & HUMANITIES PRACTICAL LIVING & VOCATIONAL STUDIES 4 * * 24 MULTIPLE CHOICE AND 6 OPEN RESPONSE IN EACH : READING MATH SCIENCE SOCIAL STUDIES ( MULTIPLE CHOICE ARE WEIGHTED 33% AND OPEN RESPONSE ARE WEIGHTED 67%)
WHAT ABOUT PRETEST ITEMS? 1 Found on all KY. Core Content Tests 1 Used to test items with Kentucky students 1 Mixed with regular test 1 Doesn’t count in CATS accountability 1 Items can be used on later forms of KY. Core Content Test after testing with students
Two Types of Standards Content Standards What Content
KCCT Test Blueprint for Science Grade 4 Grade 7 Grade 11 Physical Science 33% 30% 35% Earth/Space Science Life Science 33% 35% 30% 33% 35% 100% Sub-domain
Two Types of Standards n Performance Standards how well
Steps for Setting Performance Standards Draft Performance Jaeger-Mills Descriptions Contrasting Groups CTB Bookmark Synthesis KBE Decision
Steps for Setting Performance Standards Draft Performance Jaeger-Mills Descriptors • NOVICE Contrasting • APPRENTICE Groups CTB • PROFICIENT Bookmark • DISTINGUISHED KBE Decision Synthesis
WHAT ABOUT STUDENTS WITH SPECIAL NEEDS? ª ALL STUDENTS WITH DISABILITIES ARE INCLUDED IN CATS. ª THOSE WITH MILD DISABILITIES TAKE ASSESSMENTS-USE ACCOMMODATIONS FROM THEIR IEP.
WHAT ABOUT THOSE WITH MODERATE TO SEVERE NEEDS? ª FIRST - MAY BE DETERMINED TO BE EXEMPT FROM STANDARD ASSESSMENTS. ª NEXT – PRODUCE AN ALTERNATE PORTFOLIO (COLLECTION OF STUDENT WORK)
Alternate Portfolio n n Involves students with profound and severe disabilities in the assessment program Developed over time Collection of a student’s best work Created from classroom assignments
Accountability
Raw Scores Scale Scores Cut Scores
Raw Scores Reading 335 X 64 MATH Science Social Studies 6 open-response items 24 multiple-choice items Prac. Living Voc. Studies Arts and Humanities 2 open-response items 8 multiple-choice items
TEST FORM ONE TEST FORM TWO TEST FORM THREE TEST FORM FOUR TEST FORM FIVE TEST FORM SIX
Raw Scores FORM 6 FORM 4
Scale Scores FORM 4 FORM 6
Science Nov. 325 425 App. 525 Pro. 625 Scale Scores Dist 725 800
Following A Student through the Kentucky Core Content Test (KCCT) Student’s Science Answers Raw Score Form 2 Open Response Scale Score Form 1 OR-2 3 4 OR– 4 OR-5 4 3 OR-3 2 OR-6 19 x 2 38 3 Form 2 2 7 th grade Science Performance 800 D Levels 540 525 P P Form 3 53 517 Form 4 A Multiple-Choice -++-+++++-+-++--+++-++ Proficient Form 5 489 15 N Form 6 325
Accountability Index
Middle School Science % Nonperf. Novice-Mid Novice-High Appr-Low Appr-Mid Appr-High Proficient Distinguished 1. 42 4. 18 11. 99 15. 46 19. 30 20. 18 20. 76 6. 71 Weight 0 13 26 40 60 80 100 140 Calculation 0. 0142 0. 0418 0. 1199 0. 1546 0. 193 0. 2018 0. 2076 0. 0671 x 0= x 13 = x 26 = x 40 = x 60 = x 80 = x 100 = x 140 = Weights are Total applied & index is calculated Estimated Science Academic Index 0. 00 0. 54 3. 12 6. 18 11. 58 16. 14 20. 76 9. 39 67. 72 67. 7
ACCOUNTABILITY INDEX Academic and Non-academic measures are added together to create a school’s total score or Accountability Index.
Accountability Index Reading Non Academic Data 335 X 64 MATH Science Attendance + Social Studies Arts and Humanities Prac. Living Voc. Studies + + Retention Dropout Transition = Schools Accountability Index
Growth Chart *Every accountable school *1999/2000 Baseline *Used to set goals and determine progress
Novice Reduction Targets Baselin e Recognition Points Dropout Rate Goa l Standard Error Rewards Information
Rewards • Allocations based on school size • Meet dropout and novice reduction • School-based decision making council decides how to use A school meets or exceeds Goal Line 3 shares A school within the progressing area with an index greater than it had in the last biennium ½ share A school passes through the recognition points the first time (at 55, 66, 77, 88, and 100) 1 share Top five percent (exceeding 88 and 100) 1 share
Assistance All schools with indices below their assistance lines are divided into three levels. • Level 1 Conduct Scholastic Reviews and Selfstudies • Level 2 Receive Scholastic Reviews • Level 3 Receive Scholastic Audits and Assistance Teams with Highly Skilled Educators
Scholastic Audit The Kentucky Department of Education is required by House Bill 35 to conduct “audits” of schools that fail to meet achievement goals for each biennium. • Team members defined by law and come from outside the district. • Standards and Indicators for School Improvement document used as a measurement instrument. • Allows schools to focus on their specific needs. • Special funds available to help schools work on these needs.
WHY GIVE STATE TESTS? n GIVES A MEASUREMENT OF STUDENTS’ PERFORMANCE n TELLS WHERE STUDENTS NEED HELP n USE TO MAKE DECISIONS IN CLASSROOMS ABOUT LEARNING n INDICATES HOW SCHOOLS ARE PERFORMING
WHY ARE THESE TESTS SO IMPORTANT? n PROVIDES A PICTURE OF LEVEL OF LEARNING n DIRECTS LEARNING FOR ALL STUDENTS n IDENTIFIES WEAKNESSES AND NEEDS n RESULTS IN IMPROVED INSTRUCTION AND PERFORMANCE SUCCESS IN TODAY’S WORLD REQUIRES STUDENTS TO KNOW MORE
HELPFUL HINTS FOR SUCCESS n FOLLOW THE PROGRAM OF STUDIES n INCLUDE MULTIPLE CHOICE & OPEN RESPONSE IN ROUTINE ASSESSMENTS n KNOW THE CORE CONTENT AND ALIGN INSTRUCTION TO IT
OPEN RESPONSE INFO* n n Require students to combine content knowledge and application of process skills in communicating answers Open response are appropriate for all students X CONTENT X HIGHER ORDER THINKING *PROGRAM OF STUDIES DISK INFORMATION
OPEN RESPONSE INSTRUCTIONAL STRATEGIES* n Stress communication (explain & expand) n Give time to think & respond n Help to anticipate scoring of a question n Use practical contexts (realize relevancy of what they are learning) *PROGRAM OF STUDIES DISK INFORMATION n Encourage selfevaluation (look for evidence of quality & refinement of skills) n Provide successful experiences with higher level thinking n Evaluate frequently (helps progress by determining quality of own work)
OPEN RESPONSE ANSWERING STRATEGIES 4 UNDERLINE THE QUESTION 4 NUMBER PARTS 4 CIRCLE KEY VERBS 4 FOCUS ON QUALITY, NOT QUANTITY 4 LOOK FOR CORE CONTENT CLUES 4 BE CONCISE
OPEN RESPONSE FORMAT * STUDENTS’ ANSWERS ARE LIMITED TO ONE PAGE FOR EACH OPEN RESPONSE QUESTION (EXCEPT IN WRITING)
THINGS TO REMEMBER ABOUT OPEN RESPONSE $ SCORED STYLE $ NO FOR SUBSTANCE-NOT WRITING CREDIT FOR INCORRECT RESPONSE $ RESTATE FOCUS QUESTION ONLY IF HELPS KEEP $ ASSESSES $ NEED CONTENT, NOT WRITING TO MANAGE TEST-TAKING TIME
4 CHECKLIST FOR OPEN RESPONSE 4 ANSWERED ALL ELEMENTS 4 ANSWERED CONTENT OF PROMPT 4 ORGANIZED ANSWER 4 GAVE SUPPORT FOR ANSWER (EVIDENCE & EXAMPLES) NOTE: NOTHING CAN BE ON A STUDENT’S DESK DURING TESTING EXCEPT THE MATERIALS WHICH CAME WITH THE TEST, A PENCIL/PEN, AND BLANK SCRATCH PAPER.
KDE Website http: //www. kde. state. ky. us or http: //www. kentuckyschools. org
- Slides: 61