Music Teacher Evaluation in Michigan Dr Phillip M

Music Teacher Evaluation in Michigan Dr. Phillip M. Hash, Calvin College pmh 3@calvin. edu February 14, 2013

Overview of PM Workshop 1. 2. 3. 4. 5. New Legislation Current Trends Evaluation strategies Assessment Strategies Your Experience

Legislative Review • All Teachers Evaluated Annually • Percentage of Evaluation to Relate to Student Growth • National, State, And Local Assessments • Evaluations vs. Seniority in Personnel Decisions • Michigan Council On Educator Effectiveness 2

MDE Will Provide • Measures For every educator, regardless of subject taught, based on 2009 -10 and 2010 -11 data: – Student growth levels in reading and math – Student proficiency levels in math, reading, writing, science, social studies – Foundational measure of student proficiency and improvement (same for each teacher in a school) Understanding Michigan's Educator Evaluations, MDE (December 2010) • How will this data be used for arts educators? – Currently up to school districts – Might be specified by the state after this year

Performance-Based Compensation • A district shall implement a compensation method for teachers and administrators that includes “job performance and job accomplishments as a significant factor” to determine “compensation and additional compensation. ” MCL 380. 1250(1) • Meaning for arts educators?

New Prohibited Bargaining Subjects • • 1. Teacher Placement 2. Reduction in Force/Recall 3. Classroom Observation 4. Performance Evaluation 5. Teacher Discharge/Discipline 6. Performance-Based Compensation 7. Parent Notification

Pilot Programs • 2012 -13 Pilot 14 districts 4 evaluation models Standardized tests Local measures for nontested subjects – Recommendations by 2013 -14 school year – – 4

Current Trends in MI Teacher Evaluation

Frameworks, Methods, Systems Used as part of Local Evaluation

% Student Growth Counted in Teacher Evaluation (2011 -12) % of Growth in Local Evaluation Systems

Current Trends: Effectiveness Ratings for 2011 -12

Teacher Ratings & Student Growth

Evaluation Strategies

Evaluation Strategies • Always have lesson plans connecting to standards – See MI GLCE – Incorporate as many standards as make sense for your class – but not just perform and read notation • Study the evaluation form • Plan lessons using evaluation rubric as a guide • Be prepared to provide evidence of instructional & professional practices – Student work, rubrics, lesson plans, parent call log, etc. • Use a variety of instructional practices. • Focus on student engagement. • Don’t try to put on a show for evaluator • [Is it time to reconsider the number of performances per year? ? ]

Danielson Example

Student Engagement in Rehearsal http: //cart. bravomusicinc. com/ • http: //www. youtube. com/watch? v=VY 3 n. JXCh Wr. Y (student led warm-ups - breathing) • http: //www. youtube. com/watch? v=tgdksldrw kc (chorale) • http: //www. youtube. com/watch? v=_e. Oj. Yt 1 -4 -0 (student sectionals - feedback)

Developing Local Assessment Strategies

Creating an Assessment Plan • District Music Faculty (by area) – Est. curriculum based on MI Standards • What should students in each grade level know and be able to do? • How and when will objectives be assessed? – Perhaps not every grade every year • How will assessments show growth? (e. g. , difference in % b/w pre- post test, defined NP, P, HP? ) • Take plan to administration for approval – Law says that “with the involvement of teachers” • Pilot, Review, Revise, Implement

MI Grade Level Content Expectations (June 2011) • What students should know and be able to do in grades K-8, & HS • Aligned w/ VPAA & 21 st century skills • Standards, & benchmarks by grade level • Teachers evaluated on use of standards • [See handout]

Assessment Terms • Reliability = Consistency – Test/retest (regardless of yr. , location, etc. ) – Interrater (every judge the same) • Validity = the extent to which an assessment measures what they purport to measure • Authentic Assessment = Students demonstrate knowledge and skills in real-world context (e. g. , performance) • Quantitative – data is numerical (anything that can be counted, percentages) • Qualitative – data is in words (descriptions, written critiques) • Formative vs. Summative – • Formal vs. Informal -

Assessment Terms - RTTT • Rigorous – assessments that measure grade-level standards • Two points in time – pre- & post-test – Proficiency from one year to the next – Ongoing assessments of musical skills (steady beat, pitch matching, singing, recorder, instrumental performance, sight-reading, etc. ) • Comparable across classrooms – same for all teachers at a particular level or area – Assessments comparable in rigor to other subjects

Student Growth Measures

Rubistar http: //rubistar. 4 teachers. org/ • Create rubrics using existing descriptors • Search other teachers’ rubrics for samples – Edit to fit your needs

Rubrics • Types include: – Holistic (overall performance) – Analytic (specific dimensions of performance) – Additive • Descriptors must be valid (meaningful) • Scores – Must be reliable (consistent) – Should relate to actual levels of students learning • Can be used by students for self-assessment and to assess the performance of other students • Give to students b/f assessment 14

What does a rubric look like? TONE Beginning Basic Proficient Advanced Breathy; Unclear; Lacks focus; Unsupported Inconsistent; Beginning to be centered and clear; Breath support needs improvement Consistent Resonant; breath support; Centered and Vibrant; clear; Beginning Projecting to be resonant Features: • Scale includes rating points (at least 4). See handout for sample headings • Highest point represents exemplary performance • Criterion—based categories • Descriptors are provided for each level of student performance • Pre- and/or Post-test. Teacher, peer, & self assessment Adapted from: K. Dirth, Instituting Portfolio Assessment in Performing Ensembles, NYSSMA Winter Conference, Dec. 2, 1997. 13

Holistic Rubric

Holistic Rubric

Piano Rubric - Analytic Quiz #1 Scales Two octaves, hands together, ascending and descending ______ Keys 1 point 2 points 3 points 4 points Score Not Yet Successful Developing Satisfactorily Successful Highly Successful Fluency Student performs Students performs Student performs with many hesitations with few hesitations with almost no with no hesitations and quite a few and mistakes hesitations and very or mistakes few mistakes Correct Fingering Student performs with mostly incorrect with some incorrect with mostly correct fingerings Tempo Student performs with correct fingerings Student performs at a student performs at Student performs at very slow tempo a somewhat slow a moderate tempo a fast tempo (Largo) tempo (Adagio) (Andante) (Allegro) 1/6/12

Sample Rating Scale 12

Showing Growth w/ Rubrics (or any other pre- post-test) • Pre- & post-test • average class posttest % - average class pretest % = % growth Post 67 79 59 90 82 58 Pre 57 65 32 80 72 45 % growth 10 14 27 10 10 13 72. 5 58. 5 14

Est. Personal Reliability Record 10 students Grade w/ rubric Grade again in 2 weeks Measure the difference in score for each recording • Calculate average difference • Lower = better • • Trial 1 Trial 2 Difference 9 9 0 6 7 1 8 6 2 11 10 1 9 7 2 12 10 2 4 4 0 Av. Diff. 1. 14

Rate these 6 recorder performances on a scale of 1 -12 Rate the same examples using rubric in handout Trial 1 1 _____ 2 _____ 3 _____ 4 _____ 5 _____ 6 _____

Recorder Trial 2 • Use rubric on loose sheet • Add up score • Match score from Trial 1 to Scores from Trial 2 • Is there a difference? • In which scores are you most confident?

Elementary General Music – Grade 3 Pre- & Post Test Sample • • • [See handout] Paper/pencil, but relies on musical response Prompts can be different for pre-test Pre-test can be an abbreviated version Require 2 -3 class periods to complete Music supervisor could issue musical examples & prompts before the test (avoid teaching to the test)

Creating Similar Elementary General Music Assessment • For grades 3 -5, determine what GLCEs can be measured through paper/pencil response • Create question(s) for each benchmark – deliberately connect question to GLCEs (validity, rigor, comparable a/c classrooms) • Decide # of questions needed to determine competency • Create questions that fit different prompts

Performing Ensembles • Semester Exam [see handout] • Jason Lowe – Bay City HS Bands • Mandy Smith – Rockford HS Choirs

Watkins – Farnum Performance Scale Sight reading – band Published by Hal Leonard Reliable & valid assessment Forms A & B Easy to score as per directions in handout • 14 exercises worth X pts. • Score until student earns 0 on 2 consecutive exercises • • •

Royal Conservatory Music Development Program (see handout) • Recorder, strings, • Includes solos, etudes, woodwinds, brass, scales/arpeggios, ear percussion, voice training, sight reading, • Graded preparatory, 1 -10 theory • Curricula online – RC Grade 8 considered college entrance • Adapt for your program

Performing Ensembles

Excellence in Theory or Standard of Excellence Music Theory & History Workbooks • Kjos - publisher • 3 volumes (see handout sample) • Includes theory, ear training, history • Take MS & HS to complete 3 volumes • Students work on lessons during down time in rehearsal • Establish grade level expectations and written exam

Insuring Integrity

Insuring Integrity • • • Self created, administered, and graded assessments Colleagues & administrators will ask Standards Based assessments Comparable across classrooms Demonstrate validity & reliability – Explain/demonstrate process for creating, administering, & grading – Demonstrate connection b/w state standards and assessments – Archive recordings

www. vocaroo. com • Audio emails • Archived up to 5 months • Sends link to an email address • Download as. WAV or. Ogg • Useful for performance tests • Very easy! • http: //vocaroo. com/? media=v. Adx 5 RJr 1 DVC 7 up. Ic

Festival Ratings

NAf. ME Position Statement • Successful music teacher evaluation must, where the most easily observable outcomes of student learning in music are customarily measured in a collective manner (e. g. , adjudicated ratings of large ensemble performances), limit the use of these data to valid and reliable measures and should form only part of a teacher’s evaluation. (NAf. ME, 2011)

Festival Ratings: Advantages • Provide quantitative third party assessment • Can show growth over time in some circumstances – Individual judges’ ratings – Repertoire difficulty – 3 yr. period • Valid to the extent that they measure the quality of an ensemble’s performance of three selected pieces & sight reading at one point in time • Likely reliable over 3 -yr. period based on previous research • Probably adaptable to state-wide evaluation tool • Assess a few performance standards

Ratings Growth Example Hypothetical Contest Ratings for One Ensemble over a Three-year Period Judge 1 Judge 2 Judge 3 Sight. Reading Average Annual Increasea Final Year 1 II II II 2. 25 - 2 Year 2 II II 1. 75 22% 2 Year 3 I II I I 1. 25 29% 1 Note. Roman numerals represent division ratings. a Total increase from year 1 to year 3 = 44%.

Ratings ≠ MEAP or MME Exams • • • MEAP & MME Same for all each yr. Rel. and val. established Many Standards Individual Mostly objective Reflect multiple levels of achievement • • • Ratings Rep. , adj. change Val. & rel. not est. Per. standards only Group Mostly subjective 90%+ earn I or II out of V ratings.

Festival/Contest Ratings: Challenges • • Reliability Curricular limitations Score Inflation Ratings Effectiveness in differentiating quality Influence of non-performance factors Group vs. Individual performance Other factors Role of MSBOA & MSVMA?

Experiences

Describe Your Situation • In roundtables by area? • How are you measuring student growth at your school? • What support are you getting? • What needs or concerns do you have?