Assessment and Accountability in Kentucky Phyllis Shuttleworth Program

Assessment and Accountability in Kentucky Phyllis Shuttleworth Program Consultant Kentucky Department of Education

Assessment and Accountability in Kentucky n Assessment • Components • Standards n Accountability • • Raw/Scale/Cut Scores Accountability Indices Growth Chart Rewards/Assistance

COMMONWEALTH ACCOUNTABILITY TESTING SYSTEM

Assessment

Two Basic Types of Assessments w Criterion-referenced or Standards-Based • student performance is measured against a set standard • Norm-referenced - student performance is measured against the performance of other students

Kentucky’s Assessment Program

ELEMENTARY TESTING

MIDDLE SCHOOL TESTING

HIGH SCHOOL TESTING

WHO CREATED CATS? 4 CTB/MCGRAW-HILL COMPANY TESTING

WHO CREATED CATS? 4 OVER 300 TEACHERS INVOLVED: Blueprint Construction, Content Advisory Committee, NRT/Core Content Comparison, and Bias Screening 4 OVER 6200 EDUCATORS, PARENTS, & CITIZENS SHARED INPUT

WHO CREATED CATS? 4 ADVISORY GROUPS: EAARS - Education Assessment and Accountability Review Subcommittee NTAPAA - National Technical Advisory Panel on Assessment and Accountability OEA - Office of Education Accountability SCAAC - School Curriculum, Assessment and Accountability Council

WHO CREATED CATS?

WHO CREATED CATS? 4 STATE BOARD OF EDUCATION

Norm-Referenced Test n n n CTBS/5 Survey Edition Norm-Referenced Spring Administration Multiple-Choice Created by CTB Mc. Graw-Hill (testing contractor)

WHY HAVE NRT TESTS? 4 Tests basic skills 4 Counts in accountability formula 4 Measures basic skills across the country 4 Compares students nationally

Writing Portfolio n Standards-Based n Developed Over Time n Collection of Student’s Best Work n Created from Classroom Assignments

GRADE 4 PORTFOLIOS è 4 pieces includes: Reflective (letter to reviewer) Personal Literary Transactive è 1 piece from content area other than language arts

GRADE 7 PORTFOLIOS *5 Pieces: A. Reflective (letter to reviewer) B. C. D. E. Personal Literary Transactive Choice of B-D *1 piece must be from content area other than language arts/English

GRADE 12 PORTFOLIOS *5 Pieces: A. Reflective (letter to reviewer) B. Personal C. Literary D. Transactive E. Choice of B-D *2 pieces must be from content area other than language arts/English

Kentucky Core Content Test (KCCT) n n Standards-Based Spring Administration Includes Open. Response, Multiple. Choice, & Writing Prompts Created by Kentucky Teachers Based on KY’s Core Content for Assessment

KENTUCKY CORE CONTENT TESTS 4 * * 8 MULTIPLE CHOICE AND 2 OPEN RESPONSE IN EACH: ARTS & HUMANITIES PRACTICAL LIVING & VOCATIONAL STUDIES 4 * * 24 MULTIPLE CHOICE AND 6 OPEN RESPONSE IN EACH : READING MATH SCIENCE SOCIAL STUDIES ( MULTIPLE CHOICE ARE WEIGHTED 33% AND OPEN RESPONSE ARE WEIGHTED 67%)

WHAT ABOUT PRETEST ITEMS? 1 Found on all KY. Core Content Tests 1 Used to test items with Kentucky students 1 Mixed with regular test 1 Doesn’t count in CATS accountability 1 Items can be used on later forms of KY. Core Content Test after testing with students

Two Types of Standards Content Standards What Content

KCCT Test Blueprint for Science Grade 4 Grade 7 Grade 11 Physical Science 33% 30% 35% Earth/Space Science Life Science 33% 35% 30% 33% 35% 100% Sub-domain

Two Types of Standards n Performance Standards how well

Steps for Setting Performance Standards Draft Performance Jaeger-Mills Descriptions Contrasting Groups CTB Bookmark Synthesis KBE Decision

Steps for Setting Performance Standards Draft Performance Jaeger-Mills Descriptors • NOVICE Contrasting • APPRENTICE Groups CTB • PROFICIENT Bookmark • DISTINGUISHED KBE Decision Synthesis

WHAT ABOUT STUDENTS WITH SPECIAL NEEDS? ª ALL STUDENTS WITH DISABILITIES ARE INCLUDED IN CATS. ª THOSE WITH MILD DISABILITIES TAKE ASSESSMENTS-USE ACCOMMODATIONS FROM THEIR IEP.

WHAT ABOUT THOSE WITH MODERATE TO SEVERE NEEDS? ª FIRST - MAY BE DETERMINED TO BE EXEMPT FROM STANDARD ASSESSMENTS. ª NEXT – PRODUCE AN ALTERNATE PORTFOLIO (COLLECTION OF STUDENT WORK)

Alternate Portfolio n n Involves students with profound and severe disabilities in the assessment program Developed over time Collection of a student’s best work Created from classroom assignments

Accountability

Raw Scores Scale Scores Cut Scores

Raw Scores Reading 335 X 64 MATH Science Social Studies 6 open-response items 24 multiple-choice items Prac. Living Voc. Studies Arts and Humanities 2 open-response items 8 multiple-choice items

TEST FORM ONE TEST FORM TWO TEST FORM THREE TEST FORM FOUR TEST FORM FIVE TEST FORM SIX

Raw Scores FORM 6 FORM 4

Scale Scores FORM 4 FORM 6

Science Nov. 325 425 App. 525 Pro. 625 Scale Scores Dist 725 800

Following A Student through the Kentucky Core Content Test (KCCT) Student’s Science Answers Raw Score Form 2 Open Response Scale Score Form 1 OR-2 3 4 OR– 4 OR-5 4 3 OR-3 2 OR-6 19 x 2 38 3 Form 2 2 7 th grade Science Performance 800 D Levels 540 525 P P Form 3 53 517 Form 4 A Multiple-Choice -++-+++++-+-++--+++-++ Proficient Form 5 489 15 N Form 6 325

Accountability Index

Middle School Science % Nonperf. Novice-Mid Novice-High Appr-Low Appr-Mid Appr-High Proficient Distinguished 1. 42 4. 18 11. 99 15. 46 19. 30 20. 18 20. 76 6. 71 Weight 0 13 26 40 60 80 100 140 Calculation 0. 0142 0. 0418 0. 1199 0. 1546 0. 193 0. 2018 0. 2076 0. 0671 x 0= x 13 = x 26 = x 40 = x 60 = x 80 = x 100 = x 140 = Weights are Total applied & index is calculated Estimated Science Academic Index 0. 00 0. 54 3. 12 6. 18 11. 58 16. 14 20. 76 9. 39 67. 72 67. 7

ACCOUNTABILITY INDEX Academic and Non-academic measures are added together to create a school’s total score or Accountability Index.

Accountability Index Reading Non Academic Data 335 X 64 MATH Science Attendance + Social Studies Arts and Humanities Prac. Living Voc. Studies + + Retention Dropout Transition = Schools Accountability Index

Growth Chart *Every accountable school *1999/2000 Baseline *Used to set goals and determine progress

Novice Reduction Targets Baselin e Recognition Points Dropout Rate Goa l Standard Error Rewards Information

Rewards • Allocations based on school size • Meet dropout and novice reduction • School-based decision making council decides how to use A school meets or exceeds Goal Line 3 shares A school within the progressing area with an index greater than it had in the last biennium ½ share A school passes through the recognition points the first time (at 55, 66, 77, 88, and 100) 1 share Top five percent (exceeding 88 and 100) 1 share

Assistance All schools with indices below their assistance lines are divided into three levels. • Level 1 Conduct Scholastic Reviews and Selfstudies • Level 2 Receive Scholastic Reviews • Level 3 Receive Scholastic Audits and Assistance Teams with Highly Skilled Educators

Scholastic Audit The Kentucky Department of Education is required by House Bill 35 to conduct “audits” of schools that fail to meet achievement goals for each biennium. • Team members defined by law and come from outside the district. • Standards and Indicators for School Improvement document used as a measurement instrument. • Allows schools to focus on their specific needs. • Special funds available to help schools work on these needs.

WHY GIVE STATE TESTS? n GIVES A MEASUREMENT OF STUDENTS’ PERFORMANCE n TELLS WHERE STUDENTS NEED HELP n USE TO MAKE DECISIONS IN CLASSROOMS ABOUT LEARNING n INDICATES HOW SCHOOLS ARE PERFORMING

WHY ARE THESE TESTS SO IMPORTANT? n PROVIDES A PICTURE OF LEVEL OF LEARNING n DIRECTS LEARNING FOR ALL STUDENTS n IDENTIFIES WEAKNESSES AND NEEDS n RESULTS IN IMPROVED INSTRUCTION AND PERFORMANCE SUCCESS IN TODAY’S WORLD REQUIRES STUDENTS TO KNOW MORE

HELPFUL HINTS FOR SUCCESS n FOLLOW THE PROGRAM OF STUDIES n INCLUDE MULTIPLE CHOICE & OPEN RESPONSE IN ROUTINE ASSESSMENTS n KNOW THE CORE CONTENT AND ALIGN INSTRUCTION TO IT

OPEN RESPONSE INFO* n n Require students to combine content knowledge and application of process skills in communicating answers Open response are appropriate for all students X CONTENT X HIGHER ORDER THINKING *PROGRAM OF STUDIES DISK INFORMATION

OPEN RESPONSE INSTRUCTIONAL STRATEGIES* n Stress communication (explain & expand) n Give time to think & respond n Help to anticipate scoring of a question n Use practical contexts (realize relevancy of what they are learning) *PROGRAM OF STUDIES DISK INFORMATION n Encourage selfevaluation (look for evidence of quality & refinement of skills) n Provide successful experiences with higher level thinking n Evaluate frequently (helps progress by determining quality of own work)

OPEN RESPONSE ANSWERING STRATEGIES 4 UNDERLINE THE QUESTION 4 NUMBER PARTS 4 CIRCLE KEY VERBS 4 FOCUS ON QUALITY, NOT QUANTITY 4 LOOK FOR CORE CONTENT CLUES 4 BE CONCISE

OPEN RESPONSE FORMAT * STUDENTS’ ANSWERS ARE LIMITED TO ONE PAGE FOR EACH OPEN RESPONSE QUESTION (EXCEPT IN WRITING)

THINGS TO REMEMBER ABOUT OPEN RESPONSE $ SCORED STYLE $ NO FOR SUBSTANCE-NOT WRITING CREDIT FOR INCORRECT RESPONSE $ RESTATE FOCUS QUESTION ONLY IF HELPS KEEP $ ASSESSES $ NEED CONTENT, NOT WRITING TO MANAGE TEST-TAKING TIME

4 CHECKLIST FOR OPEN RESPONSE 4 ANSWERED ALL ELEMENTS 4 ANSWERED CONTENT OF PROMPT 4 ORGANIZED ANSWER 4 GAVE SUPPORT FOR ANSWER (EVIDENCE & EXAMPLES) NOTE: NOTHING CAN BE ON A STUDENT’S DESK DURING TESTING EXCEPT THE MATERIALS WHICH CAME WITH THE TEST, A PENCIL/PEN, AND BLANK SCRATCH PAPER.

KDE Website http: //www. kde. state. ky. us or http: //www. kentuckyschools. org