Researching Innovation 2 Researching Innovation By definition an

Researching Innovation

2 Researching Innovation Ø By definition, an innovation means that you are testing out something new to see how effective it is. Ø This also means that you will be intervening in some way in your normal practice, therefore the innovation is an intervention. Ø It is also very exploratory, because it is innovative, so you are likely to want to refine and improve it after testing it out. Ø This means that your research will be an intervention study, and it will fit in with a recognised evaluation cycle for an innovation.

3 An Innovation Cycle q Establish the ‘problem’ causing concern q Generate ideas to solve that problem (the innovation) q Formulate a research question q Test out the ideas in a pilot and evaluate it q Refine the innovation q Determine how to evaluate the innovation: what evidence data will be needed? q Implement innovation q Gather evidence data q Analyse data to evaluate the success of the intervention q Disseminate results (which may include how to improve the innovation) q Determine next steps ( abandoning it; refining it; expanding scale; larger scale innovation study)

INTERVENTION STUDIES

Intervention Studies q An intervention study, in a nutshell, examines the effectiveness of an intervention, such as introducing small class sizes or a particular teaching strategy. q The most highly regarded (but contested too) intervention study is a randomised controlled trial (RCT). This has rigorous methods which include randomly allocating students to the intervention, having a well-planned control group, fairly high sample numbers, and complex statistics. q A lot of research in educational psychology uses quasi-experimental designs: this involves different groups receiving different input, usually one group receiving the intervention and another having usual classroom input. Typically there is a pre and a post test measure to analysed the effect of the intervention. q We will be doing a small-scale exploratory intervention study, using pre and post test measures, and qualitative data collection: to build a rounded picture of how the intervention is or isn’t working which takes account of the complexity of teaching and learning.

Your Intervention Study q A small-scale exploratory intervention study cannot produce results which can be generalised to all students and would not be publishable in a research journal, though it would be eminently publishable in a professional journal; BUT: q It takes accounts of the pragmatic context of your study; that you are doing this whilst teaching so both time and access are limited; q It is of high value in developing your own professional practice and understanding of how to look for hard evidence of the success of an intervention; q It is a good starting point for a bigger study: positive evidence gained through an exploratory study can lead to funded research in the form of quasi-experimental design or even an RCT (eg EEF funded);

Your Intervention Study q You will be collecting some form pre and post intervention measures to determine what progress in learning/behaviour has been made q You will be collecting (systematically!) other relevant qualitative data (eg students’ written work; classroom observations; audio recording of group work) to analyse how the intervention is helping them in practice; q You will be keeping field notes or observation notes or audio records of the teaching of the intervention to provide contextual information for the data analysis. You could also: q Collect data about a small group of students and create profiles of how they have responded to the intervention; q Collect data from specific groups: EAL; PP; boys/girls; weak/able to look at different kinds of impact; q Have a control group from another parallel class.

THE INTERVENTION

Designing your Intervention q Having determined what the research ‘problem’ is you are trying to address, and particularly the specific needs of any sub-groups, of learners you should have a sense of what you need to try to improve. q Your innovation is an intervention designed to tackle this problem; q You should have a rationale for why this intervention might work, ideally informed by other research studies; q The intervention should be precise and clear with a precise and clear improvement goal: avoid vague broad goals and interventions with lots of components; q It is much more likely to be effective if the intervention continues over a good period of time allowing you to revisit and consolidate student learning or behaviour change.

Implementing your Intervention q Before beginning full implementation, pilot your intervention – this will give you a chance to refine obvious problems before you do the real thing. q Be clear and systematic about implementing your intervention: Ø what exactly will you be doing? Ø how many times? Ø to which groups? q Be clear and systematic about the data you need to collect to evaluate the intervention: Ø what data will you collect? Ø from whom will you collect it? Ø how many times will you collect it? Ø where will you store it?

PRE AND POST TEST MEASURES

Progress/Impact Measures q To determine whether the intervention improves students’ learning or students’ behaviour (depending on what you want the intervention to change) , you need to know where they are at the start, and how they have improved (or not) after the intervention; q This is usually referred to as a pre and post test design; q You will need to think about what data you can collect which will give you this information. It might be: Ø Precise assessment data; Ø A specific test (off the shelf or designed for your intervention); Ø Structured behaviour observation before and after; Ø Attitude survey before and after q Whatever data you choose it should be precise and well-aligned to the kind of change you want to make.

QUALITATIVE DATA

Qualitative Data The qualitative data is to provide rich and nuanced information on how the intervention is working and for whom. You will need to think about what data will be best to help you understand your intervention. It might include: q students’ written work; q classroom observations; q audio recording of group work/whole class work; q video recording of activities; q student interviews; q colleague interviews; q parent interviews; q focus groups This data needs to be collected systematically, not randomly. So as you plan your intervention you should decide what qualitative data will be collected and when.

Qualitative Data You will also need to capture your own reflections and critical thinking about the implementation of the intervention itself. As an innovation, it is highly likely that having implemented it, you will see many ways in which it could be improved, or particular aspects which worked very well. These field notes can be gathered in a variety of ways: 1. A research journal (or blog!) where you record your observations and reflections after each intervention activity; 2. Audio or video data that allows you to revisit the intervention; 3. Field notes made during the intervention implementation itself; 4. A structured reflection form which you complete after each intervention activity, which focuses your reflections on particular things. This data needs to be collected systematically, not randomly. So as you plan your intervention you should decide what field notes will be collected and when.

There is one word that has recurred repeatedly through this presentation. ? ? ? systematic Remember that research is a systematic process, planned for and recorded carefully throughout