Assessment and Evaluation of the Israeli Education System

Assessment and Evaluation of the Israeli Education System Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence Michal Beller Director-General of RAMA International Comparisons Formative Assessments Program Evaluation April 2013 http: //rama. education. gov. il Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 1

Assessment and Evaluation Should Provide Support for an Ongoing Process Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA Baker, 2005 • National Authority for Measurement and Evaluation in Education © 2013# 2

Intended Instruction Hours Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA EAG 2012 National Authority for Measurement and Evaluation in Education © 2013# 3

Teaching Hours Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA EAG 2012 National Authority for Measurement and Evaluation in Education © 2013# 4

Class Size Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA EAG 2012 National Authority for Measurement and Evaluation in Education © 2013# 5

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Ministerial Goals Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 6

Ministerial Goals (2009 -2013) 1. Promote educational values 2. Combat violence and create an optimal educational climate Ministerial Goals 3. Improve achievement Large-Scale Assessment in Israel 4. Best utilize available study time Matriculation exams 5. Narrow the educational gaps 6. Strengthen the status of teachers and improve the quality of instruction 7. Strengthen the status of principals 8. Focus on core subjects 9. Suit curricula to changing educational reality 10. Raise the number of students in professional-technological education 11. Align the education system to the digital age in the 21 st Century 12. Mainstream special needs students into regular education by enhancing the ability to retain them in classes and provide a range of solutions for them Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 7

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams RAMA Meitzav Monitoring School Violence International Comparisons Formative Assessments onal Authority for Measurement and Evaluation in Education Program Evaluation Assessment for Learning Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 8

RAMA An independent statutory unit - the National Authority for Measurement and Evaluation – RAMA - shall be established as the entity that leads and provides professional guidance to the education system with respect to measurement and evaluation. RAMA will conduct periodic evaluations of the education system and evaluations in schools, and will publish its findings in an annual report submitted to the National Council for Education. From the Dovrat Commission (2004) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 9

Improving Student Learning Ultimately, accountability is not only about measuring student learning but actually improving it Ministerial Goals Large-Scale Assessment Consequently, genuine accountability involves in Israel supporting changes in teaching and schooling that can Matriculation exams heighten the probability that students meet standards There at least three major areas where attention is Meitzav Monitoring needed: School Violence Ensuring that teachers have the knowledge and International Comparisons skills they need to teach to the standards Formative Providing school structures that support high Assessments quality teaching and learning Program Evaluation Creating processes for school assessment that Teacher & Principal can evaluate students’ opportunities to learn and Evaluation can leverage continuous change and improvement RAMA Darling Hammond, 2004 National Authority for Measurement and Evaluation in Education © 2013# 10

Campbell's Law "The more any quantitative social indicator is used for social decision making, the more subject it will be to Ministerial Goals corruption pressures and the more apt it will be to Large-Scale Assessment distort and corrupt the social processes it is intended in Israel Matriculation to monitor. ” exams "achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways. ” Campbell, Donald T. , "Assessing the Impact of Planned Social Change“. The Public Affairs Center, Dartmouth College, December, 1976. Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 11

Narrowing the Curriculum Between. Subject Reallocatio n of time The Impact of Setting a Specific Target Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Unintended Negative Effects of External High Stakes Testing Monitoring School Violence International Comparisons Formative Assessments Teaching to the Test; • Test Inflation Assessment is Only a Sample of the Curriculum “Most Children are Left Behind…” Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 12

Real Estate Map (based on Meitzav) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 13

Meitzav League Tables Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 14

Updating the Israeli Assessment System The main goals of the recent update: Implement a culture of “assessment for learning” Mitigate threats of external exams (including gaming the system) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Effective integration of internal and external evaluation Decentralization of the evaluation process along with the use of centrally designed rigorous tools Empowerment of teachers and principals Professional design of assessments and scoring (including equating, alignment to curriculum and standards and more) Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 15

Use of Multiple Indicators Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Internal admin. of external tests Formative Assessments, Assignment Data- Base, “Off-the-shelf” Assessments External Monitoring School Violence International Comparisons Matriculation exams, Meitzav Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 16

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Large Scale Assessment in Israel Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 17

Large Scale Assessment in Israel Matriculation Exams (“Bagrut”) – end of high school Meitzav – National Assessments (Native Language, Math, Science, English) – 5 th and 8 th grades (+2 nd grade in Language) + Climate & Pedagogical Survey to students and teachers (primary and secondary students, teachers and principals) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams International Comparisons: (1995, 1999, 2003, 2007, 2011, 2015) PIRLS - Reading Assessment – 4 th Grade (2001, 2006, 2011, 2016) PISA - Reading, Mathematical & Scientific Literacy – 15+ Years Old (2000/2, 2006, 2009, 2012, 2015) TALIS - Teaching and Learning International Survey - Teachers (2013) PIAAC - Programme for the International Assessment of Adult Competencies – 16 -64 (2013) Information Technology - SITES (1997, 2006); ICILS (2012 pilot) Sample-based Assessments TIMSS - Math and Science Study – 9 th Grade Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 18

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Matriculation Exams Monitoring School Violence International Comparisons (Bagrut) Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 19

Upper Secondary Graduation Rates (2010) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA EAG 2012 National Authority for Measurement and Evaluation in Education © 2013# 20

~52% Do Not Matriculate • Approximately 20% do not study in 12 th grade • Approximately 9% of those studying in 12 th grade do not take the Matriculation exams • Approximately 23% of those who take the Matriculation exams are not eligible for Matriculation Certificate Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 21

Only 48% Matriculate (on Average) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 22

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav National Assessments and School Climate Surveys Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 23

Meitzav ? מהו - המיצ"ב Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments School Climate & Pedagogical Environment Surveys Program Evaluation National Assessments Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 24

Meitzav – School Report Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 25

Achievement on Meitzav Assessments – 2007 -2012 Multi-Year Comparison Across Schools (Grades 5&8) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 26

Achievement on Meitzav Assessments in 5 th Grade according to SES (high-middle-low) amongst Hebrew Speakers Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 27

School Performance on Meitzav by SES Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation SES Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 28

School Climate and Pedagogical Environment Ministerial Goals Teachers Students Large-Scale Assessment in Israel Matriculation exams Meitzav Questionnaires Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Principals Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 29

Meitzav Surveys Ministerial The surveys of school climate and pedagogical Goals environment on the Meitzav are designed to provide a Large-Scale detailed picture of the school climate and pedagogical Assessment in Israel processes that occur within it, as depicted in the Matriculation exams information gathered from questionnaires Meitzav administered to students and teachers The questionnaires provide comprehensive and relevant information on important dimensions in this area, including: • student motivation; Monitoring School Violence International Comparisons Formative Assessments • the relationship between teachers and students; Program Evaluation • team work among faculty; Teacher & Principal Evaluation • Violence • and more RAMA National Authority for Measurement and Evaluation in Education © 2013# 30

Ministerial Goals Large-Scale Assessment in Israel Monitoring School Violence School & National Levels Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 31

Monitoring Level of Violence at the National Level • In order to get a more detailed picture of school violence, RAMA developed the ‘Violence Monitoring Questionnaire’ (VMQ) (based on previous work by Benbenishty & Astor and others) • The VMQ focuses on a range of expressions of violent and dangerous behaviors within schools and on transport to and from school • The goal: Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International • Collect systematic bi-annual comparable information, based on Comparisons student self-response Formative • Identify trends over time with regard the level of violence in the education system • In both 2009 and 2011 -- RAMA administered the VMQ among a representative national sample of students (around 24, 000) in grades 4 to 11 • Next survey - 2013 Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 32

Topics Investigated on the VMQ • Severe violence • Moderate violence • Social violence • Violence using digital media • Verbal violence • Violent gangs and bullying in schools • Sexual violence • Alcohol and drug abuse • Violence by the school staff • Violence towards the school staff • Possession of cold weapons (knives/penknives) • Violence on transport to and from school • School absenteeism due to a fear of harm • Student feelings of safety • Students perception of the school's efforts in preventing violence Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 33

Student Reporting of Violence (1/2) Students reporting (in %) different types of violence according to grade level in 2009 and 2011, and the rate of the change (in %) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 34

Student Reporting of Violence (2/2) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 35

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams International Comparisons Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 36

Reading Assessment – 4 th grade RAMA Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 37

PIRLS 2006: Results for Israel RAMA Large Scale Assessment in Israel Average of the leading country – Russian Federation - 565 Meitzav Monitoring School Climate and Violence "PIRLS Scale Average" International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA Rank from 45 countries 31 11 40 National Authority for Measurement and Evaluation in Education © 2012# 38

Percentage of students reaching the PIRLS 2006 benchmark, by sector RAMA Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 39

Arabic Diglosia • Diglosia refers to a situation in which closely related languages are used by a single language community. Usually it involves a spoken language and a highly codified variety used in formal settings • Diglosia is characteristic of Arabic – • Arab children acquire their Spoken Arabic as first language (L 1) at home • When they attend the Arab school system they begin to learn to speak and to read Modern Standard Arabic (MSA) in first grade, to speak Hebrew (L 2) in second grade, and to read and write in Hebrew and in English in third grade • This in turn led the Ministry of Education to design a radically new Arabic language acquisition program starting from first grade Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 40

PISA Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 41

Trends of Change in Reading Literacy in Israel- PISA Between the Years 2002, 2006, and 2009 +22 Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Mean Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation A significant rise in achievement (35 points) in reading achievement in Israel between 2006 and 2009. This improvement brings Israel closer to the OECD average (493) A rise of 22 points in total from 2002 to 2009 RAMA National Authority for Measurement and Evaluation in Education © 2013# 42

Trends of Change in Reading Literacy - PISA between the years 2006 and 2009 Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA Between 2006 and 2009 Israel rose by 35 points in reading, and is positioned at third place with regard the size of improvement National Authority for Measurement and Evaluation in Education © 2013# 43

Score Variance - PISA 2009 Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA The countries are ranked according to size of the score variance (the gap between the 10 th and 90 th percentile) National Authority for Measurement and Evaluation in Education © 2013# 44

Between and Within Variance of Science Scores on PISA 2006 Between Within Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Finland excels in personalized learning: 30% of instruction hours are spent by teacher outside the classroom with individual students Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 45

Ministerial Goals Large-Scale Assessment in Israel Release of PISA 2012 Results in December 2013 Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 46

PISA - Pushing the Envelope • Literacy of Reading, Math, Science • Financial Literacy • Problem Solving • Digital Assessments Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons • Collaborative Learning • Innovation Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 47

Digital PISA Assessments - March 2012 Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA PISA 2015 – fully computerized National Authority for Measurement and Evaluation in Education © 2013# 48

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 49

PISA IN FOCUS 1: Does participation in pre-primary education translate into better learning outcomes at school? Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 50

General Results at a Glance) Ministerial Goals PISA in Focus 1 reported that significant Large-Scale differences in educational achievements exist Assessment in Israel between students who had attended preprimary education (PPE) ten years earlier, and those who had not (even after accounting for ESCS background) Matriculation exams Meitzav Monitoring School Violence Of all participating countries, this difference was the most prominent in Israel Although a casual conclusion cannot be drawn from these results, they are still of high importance to educators, policy makers and researchers International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 51

Results for Israel for PISA In Focus 1 RAMA Large Scale Assessment in Israel Meitzav Israel manifested the largest difference in reading achievements between students who had attended PPE and those who had not Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 52

Reanalyzing the Data • A cautious analysis revealed that this large difference is mainly explained by a sector variable: • The majority of those who didn’t attend PPE were Arabic-speaking students (mainly Bedouins) • The average score on PISA of Arabic- speaking students is relatively low • Within each language sector, the difference in PISA achievements between students who had and had not attended PPE is about 45 points, and it is stable along the entire ESCS range RAMA Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 53

PPE Attendance in Israel) § In Israel, the attendance rate of PPE for more than one year is very similar to the equivalent rate in OECD countries (74% and 72%, respectively) § However, PPE attendance for more than one year is much more prevalent among Hebrew-speaking students than among Arabic-speaking students (86% and 35%, respectively) § Not attending PPE is rare in the Hebrewspeaking sector (2%), while in the Arabicspeaking sector it is more common and reaches 16% RAMA Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 54

PISA 2009 Achievements and Ethnic Background in Israel RAMA Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation On average, the Arabic-speaking students scored about 100 points less than the Hebrew-speaking students These results are consistent throughout the PISA studies (and other studies) and apply to the three fields of literacy measured in PISA Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 55

Actions Taken RAMA § In the wake of the results presented, Israel's Mo. E decided to take steps to reduce the number of children who do not attend PPE at all, mainly in the Arab sector § The results also shed light upon a possible vulnerable population, i. e. students who did not attend PPE. Teachers and educators should be aware of such students when they enter school Large Scale Assessment in Israel Meitzav Monitoring School Climate and Violence International Assessment s Formative Assessment s Program Evaluation Teacher Evaluation Ministerial Goals RAMA National Authority for Measurement and Evaluation in Education © 2012# 56

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence Students Achievements in Israel International Comparisons Formative Assessments International and National Views Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 57

What do the TIMSS and PIRLS Studies test? TIMSS Study in Mathematics and Science (Grade 8) § Mathematics Content Domains: § Numbers (29%), Algebra (33%), Geometry (19%), Data and probability (19%) § Science Content Domains: Biology (37%), Chemistry (20%), Physics (25%), Earth Science (18%) § Cognitive Domains : § § § Knowing - Mathematics (36%), Science (32%) Applying - Mathematics (39%), Science (44%) Reasoning - Mathematics (25%), Science (24%) PIRLS Study in Reading (Grade 4) § The study includes two types of text (literary and informational) § For each of the text types, four processes of comprehension are assessed: § Retrieving Explicitly Stated Information (22%) § Straightforward Inferences (28%) § Interpreting and Integrating (37%) § Evaluating Text (13%) Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 58

Israel’s Participation in IEA Studies TIMSS 2011 PIRLS 2011 Subject area Mathematics and Science Reading Grade level Grade 8 Grade 4 42 45 Research cycle Once every 4 years Once every 5 years Research years 1999, 2003, 2007, 2011 2001, 2006, 2011 Length of test 90 min. + questionnaire 80 min. + questionnaire Monitoring School Violence Questionnaire participants Students, teachers, principals Students, parents, teachers and principals International Comparisons 4, 699 students in 151 schools 4, 186 students in 152 schools Formative Assessments Participating countries Test participants Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 59

TIMSS and PIRLS 2011: Results and Ranks in Israel Participating Countries - AVG • 600 • Ministerial Goals 541 • 550 • 516 • 500 • 450 • 512 • 467 • Large-Scale Assessment in Israel Matriculation exams Meitzav 477 • Monitoring School Violence 400 • International Comparisons Formative Assessments 350 • TIMSS - • Mathematics • 7 TIMSS - • Science • 13 Rank among 42 participating countries PIRLS - • Reading • 18 Rank among 45 participating countries Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 60

Morocco • Qatar • United Arab • Emirates Oman • Malta • Romania • Saudi Arabia • New Zealand • Trinidad and • Tobago Israel • Iran, Islamic Rep. of • England • Bulgaria • Australia • 2011 , • מתמטיקה Qatar • Macedonia, Rep. of • Ghana • Oman • Palestinian Nat'l Auth. • Jordan • Turkey • Bahrain • Malaysia • United Arab Emirates • Lebanon • Singapore • Armenia • Israel • Georgia • Iran, Islamic Rep. of • Romania • Morocco • New Zealand • England • Syrian Arab Republic • Australia • Ukraine • Chinese Taipei • Saudi Arabia • Malaysia • Armenia • Ukraine • Korea, Rep. of • Hungary • United Arab • Emirates Morocco • Australia • Thailand • Singapore • Ghana • England • Hong Kong SAR • New Zealand • Indonesia • Japan • Russian • Federation Chile • Kazakhstan • Lithuania • United States • Tunisia • Lebanon • Italy • Slovenia • Sweden • Norway • Finland • Inter-percentile range (between • 5 -95 percentiles) Turkey • Qatar • Macedonia, Rep. • of Oman • Chinese Taipei • Georgia • Romania • Palestinian Nat'l • Auth. Jordan • Bahrain • Israel • Iran, Islamic Rep. • of Syrian Arab • Republic 400 • Singapore • Colombia • Hungary • Georgia • Northern Ireland • Indonesia • United States • Poland • Slovenia • Slovak Republic • Canada • Hungary • Saudi Arabia • 350 • Spain • 350 • France • United States • Thailand • Sweden • Indonesia • Kazakhstan • Korea, Rep. of • Japan • Russian Federation • Italy • Lithuania • Slovenia • Hong Kong SAR • Chile • Norway • Tunisia • Finland • Inter-percentile range (between 5 • -95 percentiles) 400 • Azerbaijan • Germany • Chinese Taipei • Lithuania • Portugal • Italy • Sweden • Russian Federation • 400 • Belgium (French) • Denmark • Finland • Austria • Czech Republic • Norway • Hong Kong SAR • Croatia • Netherlands • Inter-percentile range (between • 5 -95 percentiles) TIMSS and PIRLS 2011: Inter-percentile Range of “Participating Countries” TIMSS - Mathematics 350 • 300 • Ministerial Goals 250 • 200 • Large-Scale Assessment in Israel 150 • Matriculation exams 2011 , • מתמטיקה TIMSS - Science 300 • 250 • 200 • 150 • PIRLS - Reading 300 • 250 • 200 • 150 • 2011 , • מתמטיקה Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 61

TIMSS and PIRLS 2011: % of Students Reaching the International Benchmarks in Israel and in the “Participating Countries” 100% • TIMSS - PIRLS - • Mathematics Science Reading • 12% • Advanced • (625 and higher) • High • (625 -550) • 28% • 40% • 34% • 28% • 25% • 31% • 19% • 18% • 22% • 13% • 12% • Israel • Participating • 516 • Countries • - AVG • 467 • Israel • Participating • 516 • Countries • - AVG • 477 • Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav 33% • 24% • 27% • 0% • 29% • 30% • 19% • 8% • 19% • 25% • 60% • 15% • 28% • 20% • 11% • 15% • 80% • Intermediate • (550 -475) • Low • (475 -400) • 8% • 7% • 13% • 7% • 12% • Israel • Participating • 541 • Countries • - AVG • 512 • Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 62

TIMSS and PIRLS: Trends in Achievements 1999 - 2011 The number and composition of participating countries vary across cycles 600 • TIMSS - Mathematics TIMSS - Science PIRLS - Reading +50 +32 +48 516 • 496 • 500 • 466 • Large-Scale Assessment in Israel 541 • 550 • 516 • Ministerial Goals 509 • 512 • Matriculation exams 488 • 468 • 463 • 468 • Meitzav 450 • Monitoring School Violence 400 • International Comparisons 350 • Formative Assessments 2011 2007 2003 1999 • 2011 2007 2003 1999 2011 2006 2001 • • 0 • 28 19 24 7 26 23 25 13 23 31 18 Rank Program Evaluation 38 45 49 42 35 45 No. of participating countries Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 63

TIMSS and PIRLS: Trends in Achievements 2006/2007 - 2011 600 • TIMSS - PIRLS - • Mathematics Science Reading • 550 • +48 +29 +53 516 • 500 • 450 • 516 • Ministerial Goals Large-Scale Assessment in Israel 541 • Matriculation exams 512 • Meitzav 468 • 463 • Monitoring School Violence 400 • International Comparisons Formative Assessments 350 • 2011 2007 • 24 7 2011 2007 • 25 13 2011 2006 • 31 18 Rank No. of participating 49 42 49 42 45 countries Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 64

TIMSS and PIRLS: Trends in % of Students Reaching the International Benchmarks 2006/2007 – 2011 100% • Advanced • (625 and higher) • Low • (475 -400) • 4% • 12% • 15% • 28% • 29% • 10% • 28% • 30% • 15% • 34% • 30% • 60% • Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav 28% • 40% • 11% • 16% • 80% • High • (625 -550) • Intermediate • (550 -475) • TIMSS - PIRLS - • Mathematics Science Reading • 27% • 30% • 24% • 19% • 20% • 25% • 31% • 19% • 12% • 15% • 2011 • 2006 • 0% • 2007 • 2011 • 2007 • International Comparisons 15% • 13% • 25% • 13% • Monitoring School Violence 30% • 7% • 2011 • Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 65

TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011 Among Hebrew Speakers 600 • TIMSS - Mathematics TIMSS - Science PIRLS - Reading +20 550 • +52 Large-Scale Assessment in Israel 568 • +45 548 • 536 • Ministerial Goals Matriculation exams 530 • 500 • Meitzav 485 • 484 • 450 • Monitoring School Violence 400 • International Comparisons Formative Assessments 350 • 2011 2007 • 19 7 2011 2007 • 19 10 2011 2006 • 11 2 Placement Based on Int’l Rank No. of participating countries 49 42 49 42 45 Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 66

TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011 Among Arabic Speakers 600 • TIMSS - Mathematics TIMSS - Science PIRLS - Reading 550 • 500 • +57 400 • Matriculation exams 481 • 465 • 450 • 479 • Meitzav Monitoring School Violence 428 • 422 • 408 • Large-Scale Assessment in Israel +51 +59 Ministerial Goals International Comparisons 350 • 2011 2007 • 34 22 2011 2007 • 37 22 2011 2006 • 40 35 Placement Based on Int’l Rank No. of participating countries 49 42 45 Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 67

TIMSS and PIRLS: Trends in Mathematics Achievements by Socioeconomic Status 2007 - 2011 Hebrew Speakers Arabic Speakers 600 • +36 550 • 565 • +73 521 • 500 • 485 • 497 • 493 • 350 • 51+ 78+ Large-Scale Assessment in Israel Matriculation exams Meitzav 450 • 400 • Ministerial Goals +68 Monitoring School Violence 420 • International Comparisons Formative Assessments 2007 • low • 2011 • middle • high • דוברי עברית - • מתמטיקה Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 68

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Formative Assessments Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 69

Formative Assessments First Grade Assessment in Reading in Native Language Ministerial Assessment of initial reading in first grade – Hebrew assessment Goals Large-Scale for Hebrew speakers and Arabic assessment for Arabic speakers. Assessment The main aim is to assess students’ initial literacy in their native in Israel language. Matriculation exams ABLE - Assessment of Basic Literacy in English Assessment of initial reading ability in EFL at beginning fifth grade. The main aim is to identify students who are struggling with the initial stages of reading. AMIT Kit for Student Mapping Assessment of reading and writing ability at beginning of seventh grade. The main aim is to assess the extent to which students can cope with academic texts (similar to those they encounter in course books). Assessing Speaking Kit in Hebrew Assessment of speaking in English for seventh, eighth and ninth grades. The main aim is to assess the following skills: reading aloud, reporting and discussing. Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 70

Formative Assessments Assessing Speaking Kit (ASK) in English (EFL) Ministerial Assessment of speaking in English for seventh, eighth and ninth Goals Large-Scale grades. The main aim is to asses the speaking ability of students Assessment in Israel studying English as a foreign language (EFL). Language Assessment for New Immigrants Assessment of language skills (reading, listening, speaking, writing) among new immigrant students, designed for students from third grade through to ninth grade. The main aim is to assess the ability of students to function in regular classes with additional support. Banks of Performance Tasks Performance tasks for assessing students in school subjects (Math, Science, History, etc. ) at all grade levels. Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 71

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Program Evaluation Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 72

Types of Projects Evaluated Wide range programs – Systemic reforms and/or wide range projects. Usually imply wide public interest Local programs with high potential – Locally initiated programs that exhibit high potential for systemic expansion Educational projects – Evaluation of Projects of limited scale, requested by initiators of the projects Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Pooled evaluation – A combined evaluation of several programs with shared goals major features Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 73

Evaluations currently in process “New Horizon” Reform in primary and secondary schools “New Horizon” Reform in Kindergartens Ministerial Goals “Personalized Education” in secondary schools Large-Scale Assessment in Israel “The Power to Change (Oz Letmura)” Reform in high schools Matriculation exams “Advancement of 21 st century skills” – integration of ICT in schools The program for advancement of scholastic achievements in secondary schools Evaluation of teachers, principals and other staff members Programs for advancing literacy in kindergartens “Amirim” - a program for advancing gifted children “Social Matriculation Diploma” – advancing community involvement among high school students A unified program for schools’ field trips “Healthy lifestyle” - a program for advancing healthy nutrition and physical activity among students Development of emotional strength among students “AHAM” – improving school climate Interconnections between school climate and scholastic achievements Mapping and assessing Youth movements Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 78

Evaluations currently in process “Nahshon” Project – Distant Learning “Michael” Project – empowerment and Improvement of cognitive skills among students “Masa” program – English speaking volunteers teaching english in primary schools “Feuerstein Program” – cognitive advancement of Ethiopian primary schools students Programs for advancement of Ethiopian - born students A program of Mediation between Immigrant students, their families and school staff Evaluation of “Avney Rosha” institute – an institute for advancing school principals “Ad Halom” – a program for advancing literacy among adults “The transparency project” – connecting inputs and outcomes Autonomous conduct in schools Principals’ leadership Personalized education in high schools Advancement of achievements in mathematics in middle schools Advancement of achievements in geography in middle schools Mapping and analyzing programs for advancement of achievements in matriculation exams Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 79

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Teacher & Principal Evaluations Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 80

Teacher Evaluation in Israel • Teacher evaluation in primary and secondary schools has become a ministry policy and part of the official contracts with the unions. • In primary schools it is required for: § determining tenure § promotion to the highest levels during service § professional development • In high schools it is required for: § individual award for distinguished teachers § accumulation of points for advancement of distinguished specific role holders in schools § professional development Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 81

Formative vs Summative Evaluation Ministerial Goals Large-Scale Assessment in Israel Formative Professional Development Matriculation exams Meitzav Monitoring School Violence International Comparisons Summative High Stakes Tenure Promotion Rewards Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA Different tools? ? Different goals National Authority for Measurement and Evaluation in Education © 2013# 82

Teacher Evaluation in “Ofek Hadash” Teacher evaluation is required for: § Determining teacher tenure § Promotion on the teachers’ rating scale (top ranks): § Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence ú From grade 6 to 7 International Comparisons ú From grade 7 to 8 Formative Assessments ú From grade 8 to 9 Program Evaluation Professional development Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 83

Collaboration with all Stakeholders to Build the Evaluation Process § The Ministry of Education: ú The pedagogic administration: Elementary education and Secondary education ú The training and professional development department ú The pedagogical secretariat ú The religious education department ú The social and youth department ú The Arab, Bedouin, Druse and Circassian departments ú Educational psychological department ú Special education department § District managers and inspectors/supervisors § School principals and teachers § The Israeli Institute for Educational Leadership § Teacher unions Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 84

Basic Assumptions in the Evaluation Process • The process of teacher evaluation, their active participation and involvement in the process, will contribute to the improvement of teaching and promotion of student learning and well-being. • Teacher self-evaluation and the evaluation by the principal of teachers' work will contribute to the image of the fitting teacher and to fostering learning processes and continual improvement • Constant and continual improvement of teachers is based on respect, trust, fairness, reflective dialogue and cooperation, constitutes a central element in constructing a professional community for school improvement and the advancement of the education system • Teacher evaluation is an important component of the role of the school principal and in establishing administrative quality and accountability for the development of human capital in his/her school • Teacher evaluation will strengthen the status of teachers and the teaching profession through the design of professional identity, continued professional development, examining the suitability to teaching and provision of tenure, promotion in terms of rank and roles. Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 85

Central Points in the Process of Teacher Evaluation The principal presents the process of teacher evaluation to Ministerial Goals the teaching staff Large-Scale Assessment in Israel The principal presents the tool for teacher evaluation to the Matriculation exams evaluated teacher at the beginning of the year, including the evaluation process, its stages, and time Meitzav line throughout the year Teacher evaluation will be based on additional evidence that attests to the teacher's work (e. g. , planning a study unit, instructional materials, personal work plan, class file, etc. ) During the year the principal will carry out at least two documented observations of the teacher's lessons and also an observation of an individual instructional session. Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 86

Central Points in the Process of Teacher Evaluation Prior to the lesson observation the principal will hold a discussion with the teacher in order to collect information about the context in which the lesson is taking place and the lesson's goals. Following the lesson observation the principal will hold a feedback discussion with the teacher, as close to the time of the observation as possible. Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons The teacher will also provide a self evaluation based on the Formative four dimensions of evaluation Assessments Together goals will be set for the professional development of the teacher Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 87

Domains for Teacher Performance Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 88

Domains for Teacher Performance Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 89

The Teachers’ Professional Development Scale The evaluation scale indicates the level of expected teacher performance on different development stages and within different domains: Domains for Teacher Performance Unsatisfactory Basic Skilled Distinguished Master Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 90

Tool for Teacher Performance Assessment Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA The tool enable drawing a teacher profile on different domains. National Authority for Measurement and Evaluation in Education © 2013# 91

A Teachers’ Profile Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 92

Domains for Teacher Performance Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 93

RAMA: Assessment in the Service of Learning Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Meitzav Monitoring School Violence International Comparisons Formative Assessments Program Evaluation Teacher & Principal Evaluation RAMA Accountability begins from school self-evaluation National Authority for Measurement and Evaluation in Education © 2013# 94

Trust and Shared Responsibility Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Students Trust and Shared Responsibility Principals Teachers Parents Meitzav Monitoring School Violence Politicians International Comparisons Formative Assessments Program Evaluation Local Education Units Administrators Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 95

Ministerial Goals Large-Scale Assessment in Israel Matriculation exams Thank You Meitzav Monitoring School Violence International Comparisons Formative Assessments mbeller. rama@education. gov. il Comments ? Questions? Program Evaluation Teacher & Principal Evaluation RAMA National Authority for Measurement and Evaluation in Education © 2013# 96