MA in Educational Leadership Teach First School Effectiveness

MA in Educational Leadership (Teach First) School Effectiveness & School Improvement January/February 2016

Session Aims: • Understand the complexities of defining ‘effectiveness’ • Develop a clear perspective about where the school effectiveness and school improvement movements came from (and are going) • Understand the Teach First contribution to the school improvement movement • Grasp the methodological challenges involved in attempting to explore correlation and causality.

Activity 1. 1 a) On your table, quickly reach an agreed definition of the word effective. b) List as many criteria as you can by which you would judge a machine to be effective.

Activity 1. 2 c) Can you use the same criteria for judging the effectiveness of a school as for judging a machine? What is the main problem with this? c) Create a list of characteristics by which you think you could evaluate whether or not a school is effective.

Compare your definitions and ideas with the literature cited in the next few slides.

Definition of school effectiveness (1) School effectiveness refers to all theories and research studies concerning means–ends relationships between educational processes and outcomes, in particular student knowledge and skills … aiming at explanations for differences in student achievement between schools and classrooms (Creemers and Reezigt 1997: 401). How closely does this match your definition of school effectiveness?

Definitions of school effectiveness (2) “We define an effective school as one in which pupils progress further than might be expected from consideration of its intake” (Stoll and Mortimore 1995: 1). How closely does this match your definition for school effectiveness? “The search for effective schools is not a value-free activity. The term ‘effective’ is not a neutral term, however much researchers try and invest it with scientific or technical meaning” (Mac. Beath 1995: 14).

Effectiveness will be defined in different ways by different people at different times. Consider: Student perspective Parent perspective Teacher perspective School leaders’ perspective Government perspective Ø On what might these groups’ definitions of effectiveness agree? Ø How might their definitions differ?

Early studies were sceptical about school effects Early large-scale studies of school effects in general (Coleman et al. , 1966; Jencks et al. , 1972) concluded that achievement is mainly determined by family background, and that pupils’ subsequent careers are little affected by the quality of education they experience (Barker 2007: 24).

Later studies were more positive Later studies (late 1970 s /early 1980 s) emphasized the significance of the school as one important variable in student outcomes. A variety of studies in different settings, different countries and with different age groups showed some school impact. UK examples include Reynolds et al. (1976) in South Wales; Rutter et al. (1979) in London schools; Mortimore et al. (1988) in London primary schools; Smith and Tomlinson (1989) in multi-racial comprehensives.

Characteristics of effective schools Professional leadership Shared vision and goals A learning environment Concentration on teaching & learning ü High expectations ü Positive reinforcement ü Monitoring progress ü ü ü Pupil rights and responsibilities ü Purposeful teaching ü A learning organisation ü Home-school partnership (Sammons et al, 1995: 8)

Activity 2 a) Looking at your own lists of ‘effectiveness’ characteristics, how close is your match to Sammons et al. ’s? b) Is it possible for teachers and/or school leaders to control (all) the characteristics on Sammons et al. ’s list? Is your school/college/academy/trust achieving this? c) Studies which try to develop these kinds of lists are problematic. Why?

School Effectiveness Research – a critique Under-theorised - too much emphasis on vague and imprecise concepts e. g. ‘leadership’, ’ethos’ Ignores context – history, policy, politics, culture Ignores student context – class, race, gender Says little about curriculum content and pedagogy Creates a blame climate - ‘discourse of derision’ Narrow definition of ‘effective’ or ‘good’ – leaves no room for dissent Emphasis on what is measureable Emphasis on ‘quick-fix’ solutions – schools are too complex for this approach.

School Effectiveness Research – a critique The general consensus is that schools usually account for about 10 to 15% of the variation in pupils’ performance (Scheerens 1989; Gray et al. 1990). So why do politicians and researchers concentrate on this 10 - 15% and not the other 85% - 90%? To address some of the problems of School Effectiveness Research, School Improvement Research was developed.

School improvement is … … an approach to educational change that is concerned with process as well as outcomes. School improvement is about raising student achievement through enhancing the teaching-learning process and the conditions which support it. It is about strategies for improving the school’s capacity for providing quality education (Hopkins 1994: 75).

Comparing School Effectiveness and School Improvement (Bush and Coleman 2000: 53) School effectiveness School improvement Focus on schools Focus on individual teachers or groups of teachers Focus on school organisation Focus on school process Data-driven – focus on outcomes Often not empirical – Rare evaluation of effects of change Quantitative Qualitative

School effectiveness School improvement Lack of knowledge about Implementing change strategies Concerned exclusively with change Concerned with pupil outcomes Concerned with process / improvement / progress Focus on specific points in time Concerned with change over time Based on researcher knowledge Based on practitioner knowledge

SI may be an improvement on SER, but issues remain (Coe, 2009) • Overall achievement in USA or England has not improved significantly in thirty years. • Evaluations of SI programmes usually rely on selfreport which is skewed by dissonance reduction. • Studies very rarely investigate impact on student achievement. • Schools might have improved anyway – schools that were comparable with London Challenge schools improved just as much.

My argument so far has been that much of what is claimed as SI is actually not real, or if it is, we don’t really know why it occurred. I have argued that wider and better use of more rigorous evaluation designs would help us to distinguish real from illusory improvement and to understand the strategies and conditions under which real improvement can confidently be predicted to occur as a result of any particular actions (Coe, 2009: 371).

Activity 3: Questions about the Maximum Impact Evaluation into Teach First undertaken by Muijs et al. (2010) 1) What were the main findings? 2) What was the research design? 3) Would Coe consider it rigorous? Why or why not?

Main finding Overall, there is converging evidence that Teach First teachers have a positive impact in schools. While none of the elements of this evaluation in and of themselves can demonstrate conclusively that Teach First teachers have a positive impact, taken together the evidence is compelling (Muijs et al. , 2010: 3).

Broad evidence ü Partnering schools see statistically significant improvement in GCSE results; (BUT note important caveat about correlation v. causality). ü “Teach First teachers consistently rated above the midpoint of the scale for the factors observed, indicating overall high levels of teacher effectiveness” (p 25); ü TF teachers believe they can make a difference and head teacher surveys support this; ü TF teachers see themselves as leaders and so do their colleagues.

Research Design Ø Survey questionnaires of participants and heads Ø Face-to-face interviews with participants, heads, line-managers and colleagues Ø Video-ed classroom observation Ø Multi-level statistical modelling of pupil performance data Ø Content analysis of participants’ learning logs and Personal Achievement Record guides. The design appears thorough, but does it fully meet Coe’s (2009) requirements (see final slide)?

My argument so far has been that much of what is claimed as SI is actually not real, or if it is, we don’t really know why it occurred. I have argued that wider and better use of more rigorous evaluation designs would help us to distinguish real from illusory improvement and to understand the strategies and conditions under which real improvement can confidently be predicted to occur as a result of any particular actions (Coe, 2009: 371).