Consider the Evidencedriven decision making for secondary schools

Consider the Evidence-driven decision making for secondary schools A resource to assist schools to review their use of data and other evidence 3 Evidence-driven decision making Page 1 © New Zealand Ministry of Education – copying restricted to use by New Zealand education sector.

Evidence-driven decision making Today we aim to • think about a process for using data and other evidence to improve teaching, learning and student achievement • improve our understanding, confidence and capability in using data to improve practice • think about our needs and our own evidence-based projects Page 2 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven eating You need to buy lunch. Before you decide what to buy you consider a number of factors: • how much money do you have? • what do you feel like eating? • what will you be having for dinner? • how far do you need to go to buy food? • how much time do you have? • where are you going to eat it? Page 3 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven teaching I had a hunch that Ana wasn’t doing as well as she could in her research assignments, a major part of the history course. What made me think this? Ana’s general work (especially her writing) was fine. She made perceptive comments in class, contributed well in groups and had good results overall last year, especially in English. How did I decide what to do about it? Page 4 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven teaching (cont…) I looked more closely at her other work. I watched her working in the library one day to see if it was her reading, her use of resources, her note taking, her planning, or what. At morning tea I asked one of Ana’s other teachers about Ana’s approach to similar tasks. I asked Ana if she knew why her research results weren’t as good as her other results, and what her plans were for the next assignment. I thought about all of this and planned a course of action. I gave her help with using indexes, searching, note taking and planning and linking the various stages of her research. Page 5 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Consider the Evidence A resource to assist schools to review their use of data and other evidence What is meant by ‘data and other evidence’? Page 6 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence Any facts, circumstances or perceptions that can be used as an input for an analysis or decision • how classes are compiled, how classes are allocated to teachers, test results, teachers’ observations, attendance data, portfolios of work, student opinions … Data are one form of evidence Page 7 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Data Known facts or measurements, probably expressed in some systematic or symbolic way (e. g. as numbers) • assessment results, gender, attendance, ethnicity … Data are one form of evidence Page 8 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven decision making We have more evidence about what students know and can do than ever before - their achievements, behaviours, environmental factors that influence learning We should • draw on all our knowledge about the learning environment to improve student achievement • explore what lies behind patterns of achievement • decide what changes will make a difference Page 9 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

What evidence does a school have? • • • Page 10 Demographics Student achievement Perceptions School processes Other practice © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Demographics What data do we have now to provide a profile of our school? What other data could we create? • School • Students • Staff • Parents/caregivers and community Page 11 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Student achievement What evidence do we have now about student achievement? What other evidence could we collect? • National assessment results • Standardised assessment results administered internally • Other in-school assessments • Student work Page 12 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Perceptions What evidence do we have now about what students, staff and others think about the school? Are there other potential sources? • Self appraisal • Formal and informal observations made by teachers • Structured interactions • Externally generated reports • Student voice • Other informal sources Page 13 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

School processes What evidence do we have about how our school is organised and operates? • Timetable • Classes • Resources • Finance • Staffing Page 14 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Other practice How we can find out about what has worked in other schools? • Documented research • Experiences of other schools Page 15 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

The evidence-driven decision making cycle Trigger Explore Question Assemble Analyse Interpret Intervene Evaluate Reflect Page 16 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

The evidence-driven decision making cycle Trigger Explore Question Assemble Analyse Interpret Intervene Evaluate Reflect Page 17 Clues found in data, hunches Is there really an issue? What do you want to know? Get all useful evidence together Process data and other evidence What information do you have? Design and carry out action What was the impact? What will we change? © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

The evidence-driven decision making cycle Trigger Data indicate a possible issue that could impact on student achievement Speculate A teacher has a hunch about a problem or a possible action Explore Check data and evidence to explore the issue Reflect on what has been learned, how practice will change Evaluate the impact on the intervention Question Clarify the issue and ask a question Assemble Decide what data and evidence might be useful Act Carry out the intervention Intervene Plan an action aimed at improving student achievement Page 18 Interpret Insights that answer your question © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector. Analyse data and evidence

The evidence-driven decision making cycle TRIGGER SPECULATE EXPLORE REFLECT QUESTION EVALUATE ASSEMBLE ACT INTERVENE Page 19 INTERPRET © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector. ANALYSE

The evidence-driven decision making cycle. Reflect How will we teach writing in the future? Trigger Some of our students are poor at writing A teacher has a hunch - poor writers might spend little time on homework Explore data Survey of students shows that this is only partially true Evaluate Has writing improved? Intervene Create multiple opportunities for writing; include topics that can use sport as context; connect speaking and writing. PD for staff. Page 20 Interpret information Poor writers likely to play sport, speak well, read less, do little HW Analyse NQF/NCEA results by standard Analyse non NQF/NCEA data and evidence © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector. Question What are the characteristics of students who are poor at writing? Assemble more data & other evidence: as. TTle reading, homework, extracurric, Attendance, etc

Evaluate and reflect • Summative evaluation – assess how successful the intervention was; decide how our practice will change; report to board • Formative evaluation - at every stage in the cycle we reflect and evaluate Are we are on the right track? Do we need to fine-tune? Do we actually need to complete this? Page 21 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Types of analysis We can compare achievement data by subject or across subjects for • an individual student • groups of students • whole cohorts The type of analysis we use depends on the question we want to answer Page 22 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Trigger questions • • • How good/poor is …? What aspects of … are good/poor? Is … actually changing? How is … changing? Is … better than last year? How can … be improved? Why is … good/poor? What targets are reasonable for …? What factors influence the situation for …? What would happen if we …? Formative or summative? Page 23 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Questions about policy We have been running 60 -minute periods for 5 -years now. What effect has the change had? Page 24 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Questions from hunches • I suspect this poor performance is being caused by … Is this true? • We reckon results will improve if we put more effort into. . . Is this likely? • I think we’d get better results from this module if we added … Is there any evidence to support this idea? Page 25 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Questions with purpose What do we know about reported bullying incidents for year 10 students? MAY BE BETTER AS Who has been bullying whom? Where? What are students telling us? What does pastoral care data tell us? Were some interventions more effective with some groups of students than others? Page 26 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Professional decision making We have evidence-based information that we see as reliable and valid What do we do about it? If the information indicates a need for action, we use our collective experience to make a professional decision Page 27 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Professionals making decisions Do any particular groups of year 11 students attend less regularly than average for the whole cohort? The analysis identified two groups – so I need to think about how to deal with irregular attendance for each group. How will I do that? Page 28 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Professionals making decisions You asked what factors are related to poor student performance in formal writing. The analysis suggested that poor homework habits have a significant impact on student writing. You make some professional judgements and decide • Students who do little homework don’t write enough • You could take action to improve homework habits - but you’ve tried that before and the success rate is low • You have more control over other factors - like how much time you give students to write in class So you conclude - the real need is to get students to write more often Page 29 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Deciding on an action Information will often suggest a number of options for action. How do we decide which action to choose? We need to consider • what control we have over the action • the likely impact of the action • the resources needed Page 30 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Planning for evaluation • What evidence do we need to collect before we start? • Do we need to collect evidence along the way, or just at the end? • How can we be sure that any assessment at the end of the process will be comparable with assessment at the outset? • How will we monitor any unintended effects? Don’t forget evidence such as timetables, student opinions, teacher observations … Page 31 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evaluate the impact of our action Did the intervention improve the situation that triggered the process? If the aim was to improve student achievement, did that happen? Page 32 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evaluate the impact of our action Was any change in student achievement significant? What else happened that we didn’t expect? How do our results compare with other similar studies we can find? Does the result give us the confidence to make the change permanent? Page 33 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Future practice • What aspects of the intervention will we build into future practice? • What aspects of the intervention will have the greatest impact? • What aspects of the intervention can we maintain over time? • What changes can we build into the way we do things in our school? • Would there be any side-effects? Page 34 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Future directions • What professional learning is needed? Who would most benefit from it? • Do we have the expertise we need in-house or do we need external help? • What other resources do we need? • What disadvantages could there be? • When will we evaluate this change again? Page 35 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven strategic planning If we use evidence-driven decision making to improve student achievement and enhance teaching practice … … it follows that strategic planning across the school should also be evidence-driven. Page 36 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

Evidence-driven strategic planning INDICATORS FROM DATA . as. TTle scores show a high proportion of Yr 9 achieving below curriculum level NCEA results show high nonachievement in transactional writing STRATEGIC GOAL ANNUAL PLAN YEAR TARGET To raise the levels of writing across the school Develop and implement a plan to raise levels of Writing at year 9 Raise writing as. TTle results Yr 9 boys from 3 B to 3 A Strategic action Development plan to be based on an analysis of all available data and to include a range of shared strategies etc. Develop a writing development plan which addresses writing across subjects and levels , including targets, professional development and other resourcing needs etc. Poor results in other language NCEA standards EVALUATION DATA Appraisal as. TTle writing results improve by … Perception data from year 9 staff indicates … Evaluation of effectiveness of range of shared strategies, barriers and enablers … etc. PD Self review etc School charter Page 37 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.

What now? How can we apply this model in our school? Page 38 © New Zealand Ministry of Education 2009 – copying restricted to use by New Zealand education sector.