What do we know about reading outcomes in

What do we know about reading outcomes in SA? -Nic Spaull Stellenbosch University & Allan Gray Orbis Foundation Endowment 6 March 2017 RASA AGM

PSPPD & Zenex Report Launch

What do we know about education in SA? 1. High access: South Africa has relatively high access to education up to Grade 9. Expanding Grade R. Also new progression policies. 2. Low quality: South Africa has very low levels of educational achievement even by low/middle-income standards. OECD ranks SA 74/75 countries beating only Ghana. 3. Egregious inequality: The gap between the better performing 20% of schools and the 80% of dysfunctional schools is one of (or the) largest in the world. Gap between two systems is between 3 & 4 grade levels. Largely determined by ability-to-pay and location. 4. Weak accountability: The alliance between the ANC and COSATU/SADTU means that almost all accountability initiatives never get off the ground. (Volmink Report, ANAs, SACE) 5. Low capacity: Most teachers in quintiles 1 -4 do not know enough about their subjects (content knowledge) or how to teach them (pedagogy) meaning that students rarely grasp the content either.

The centrality of reading “Professional educators and the public at large have long known that reading is an enabling process that spans academic disciplines and translates into meaningful personal, social, and economic outcomes for individuals. Reading is the fulcrum of academics, the pivotal process that stabilizes and leverages children’s opportunities to success and become reflective, independent learners. (Good, Simmons and Smith, 1998: 45). “Reading is, without doubt, the most important linguistic skill that needs to be developed in young children. Reading serves as a building block upon which all other learning takes place…This National Reading Strategy takes as its focus that reading failure begins in early grades, and it is at that level that interventions must be made” (Do. E, 2008: 18).

The curse of inequality

Egregious inequality: 2 systems Spaull, N. (2013). “Poverty & privilege: Primary school inequality in South Africa”. International Journal of Educational Development” 33; 436 -447 6

Do children in English or Afrikaans LOLT schools learn to read? PIRLS 2011 - Proportion of Gr 5 students in English & Afrikaans schools acquiring basic reading skills by school location Note: Proportion reaching low international benchmark in PIRLS 2011. SA tested 3515 grade 5 students in 92 schools where Eng/Afr was LO 95% 100% 90% 80% Colombia all Gr 4 = 72% Honduras all Gr 4 = 74% 81% 84% 64% 70% 57% 55% 60% 50% 40% 30% 26% 28% 20% 10% 0% Remote rural Township Small town/ village Medium-city or large town Urban Suburban National International median (Gr 4)

Province: What % of students do not learn to read in any language? Pre. PIRLS 2011 - Proportion of Grade 4 students that are illiterate and the proportion who cannot read for meaning (in LOLT Gr 1 -3) Using pre. PIRLS 2011 illiterate: cannot reach low benchmark. Read for meaning: reach intermediate benchmark. Note: pre. PIRLS not % Illiterate % Cannot read for meaning 90% 80% 70% 60% 50% 83% 40% 30% 57% 63% 66% 50% 44% 20% 10% 60% 27% 11% 26% 32% 29% 58% 29% 0% Western Cape Gauteng Kwa. Zulu-Natal Eastern Cape Mpumalanga North West Limpopo South Africa

Location: What % of students do not learn to read for meaning in any language? (pre. PIRLS 2011) 100% 90% 35% 80% 70% 69% 33% 29% 40% 69% 60% 50% Can read for meaning 40% Cannot reach 475 points 65% 30% 67% 71% 60% 20% 31% Urban Suburban 10% 0% Township Small Remote rural town/village Total 9

School wealth: What % of students do not learn to read for meaning in any language? pre. PIRLS 2011 ools sch % of 120% st 5 e Poor 100% 80% 2% 7% 19% 1% 7% 16% 0% 3% 19% 0% 5% 25% 0% 4% 1% 9% 18% t es h Ric 1% 5% 0% 1% 16% 22% 0% 4% 20% 1% 6% 1% 0% 11% 13% 23% 19% 33% 0% 4% 1% 4% 0% 10% 3% 12% 10% 16% 26% 31% 33% 30% 28% 47% 33% 60% 40% sc ls Advanced 22% 27% 5% of o ho 24% 72% 77% 78% 70% 83% 77% 72% 30% 77% 74% 57% 73% 69% 55% 57% 31% 39% 60% 56% High 44% 20% 30% 20% 11% Intermediate 3% < Intermediate st ie lth W ea ea lth ie st 5 5 % % sc ho ho ol ol s s 18 17 w d 2 n 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0%

What does this look like in reality? “We found that South African Grade 5 second language learners from rural areas in South Africa had essentially the same distribution as Grade 1 second language learners in Florida, or Grade 2 second language remedial learners who had been removed from normal classes because they “cannot communicate meaning orally in English and demonstrate very little understanding in English”. Draper, K. , and Spaull, S. (2015). Examining oral reading fluency among grade 5 rural English Second Language (ESL) learners in South Africa: Analysis of NEEDU 2013. South African Journal of Childhood Education 5(2) pp. 44 -77.

What does this look like in reality? Many One tree years ago Leopard was a creature with no day, when he was relaxing in the shade of when a thorn Recommended rate at Gr 5 Zebra walked past. years ago Leopard was a creature with no day, spots. he was relaxing in the shade Zebra walked past. of spots. a thorn 40% of SA rural Gr 5 learners (<40 WCPM) WCPM Draper, K. , and Spaull, S. (2015). Examining oral reading fluency among grade 5 rural English Second Language (ESL) learners in South Africa: Analysis of NEEDU 2013. South African Journal of Childhood Education 5(2) pp. 44 -77.

Where to from here? ?

We’ve been on this rodeo before… 1. 2. Policy déjà vu 3. 4. 5. 6. 7. 8. National Reading Strategy (DBE, 2008) – Pandor Teaching Reading in the Early Grades: A Teacher’s Handbooks (DBE, 2008) Western Cape Numeracy & Literacy Strategy 2006 -2016 (WCED, 2006) Foundations for Learning Campaign (DBE, 2008) Gauteng Primary Literacy Strategy 20102014 (GDE, 2010) Systematic Method for Reading Success (Hollingsworth & Gains, 2009) Western Cape Living Labs Schools (WCED, 2015) Drop Everything & Read Campaign (See PSPPD Report (2016) for full discussion of previous reading initiatives)

Where to from here? Interventions need to address one of the following: 1. 2. 3. 4. Improving FP teacher’s knowledge and practice about teaching reading Eliminating excessive class sizes in the FP Improving availability and use of books at school & at home Remedial teaching – helping those who have/are falling behind Research needs to address one of the following: 1. How to teach reading in African languages 2. Setting norms and benchmarks for reading in African languages & EFAL 3. The best method(s) of improving teachers knowledge and practice of teaching reading (Coaches? Training? Materials? )

We know how to teach reading

Course outlined developed…

Teaching Reading (& Writing) in FP

Finding innovative ways to address class sizes?

5 steps to effective interventions

PROBLEM What is the specific problem we are addressing? We can describe the problem and know how widespread this problem is. BINDING CONSTRAINTS THEORY OF CHANGE EVALUATION FEEDBACK LOOPS What are the binding constraints in this problem? How does our intervention address the binding constraint? How do we know if our intervention is working? What are we learning about what is & isn’t working? We can identify the most important causes of this problem & they are solvable. We can describe in comprehensive detail all our assumptions & how and why we think our intervention has an effect (& cost/learner). We have a plan for measurement, & monitoring (internal) and evaluating (external) whether our intervention is working. We have a system of feedback loops from providers & recipients to tell us what is & isn’t working (& maybe why).

PROBLEM What is the specific problem we are addressing? We can describe the problem and know how widespread this problem is. BINDING CONSTRAINTS THEORY OF CHANGE EVALUATION FEEDBACK LOOPS What are the binding constraints in this problem? How does our intervention address the binding constraint? How do we know if our intervention is working? What are we learning about what is & isn’t working? We can identify the most important causes of this problem & they are solvable. We can describe our intervention in detail (+ costs & assumptions), as well as how & why we think it will have an impact. We have a plan for measurement, & monitoring (internal) and evaluating (external) whether our intervention is working. We have a system of feedback loops from providers & recipients to tell us what is & isn’t working (& maybe why).

PROBLEM What is the specific problem we are addressing? We can describe the problem and know how widespread this problem is. BINDING CONSTRAINTS THEORY OF CHANGE EVALUATION FEEDBACK LOOPS What are we learning about what is & isn’t working? What are the binding constraints in this problem? How does our intervention address the binding constraint? How do we know if our intervention is working? We can identify the most important causes of this problem & they are solvable. We can describe in comprehensive detail all our assumptions & how and why we think our intervention has an effect (& cost/learner). We have a plan for measurement, & monitoring (internal) and evaluating (external) whether our intervention is working. We have a system of feedback loops from providers & recipients to tell us what is & isn’t working (& maybe why).

The importance of rigorous evaluation In the absence of a rigorous independent evaluation, how can we reliably know if an intervention is working or not? As a rule of thumb NGOs & interventions should allocate 5 -10% of their overall budget for evaluation.

Lessons for Interventions Design and Policy 24 February 2017 Presentation to AGOFE

Randomised control trial

Description of Interventions 31

Description of Interventions Per learner costs Triple cocktail: Expensive for national roll-out, but R 7 m – R 8 m for one grade in 100 priority schools. 32

EGRS 2

Comments and questions? Presentation available at nicspaull. com

Access: How does all of this affect matric? 3 Get a degree within six years of. . . Get some kind of university. . . 4. 5 Go to university 14 37 Pass matric 60 Write matric 100 Start school 0 20 40 60 80 100 Uses national HEMIS/NSC data for the 2008 matric cohort between 2009 and 2014 (Van Broekhuizen, Van der Berg & Hofmeyr, 2016)

Qualifications by age (birth cohort), Census 2011 (Van der Berg, 2013) 100% 90% Degree Some Matric 80% 70% Some 60% Primary 50% 40% How Some does all of this affect the labour market? 30% 20 (1991) 25 (1986) 30 (1981) 35 (1976) 40 (1971) 45 (1966) 50 (1961) 55 (1956) 60 (1951) 65 (1946) 70 (1941) 0% 80 (1931) 10% 75 (1936) No

Bimodality – indisputable fact PIRLS / TIMSS / SACMEQ / NSES / ANA / Matric… by Wealth / Language / Location / Dept… 37

Hoadley (2016) Descriptive features of Foundation Phase classrooms

Table 13: Oral Reading Fluency scores for English Second Language (ESL/ELL) in Broward County Public Schools (Florida, US) (Broward County, 2012)

Complex language dynamics in SA ANA 2013 Language of Assessment 100% 90% 23% English 24% isi. Zulu 80% 70% 23% 19% isi. Xhosa 19% Sepedi 60% 90% 50% 40% 30% 13% 9% 9% 22% 8% 8% 10% 6% 5% 5% 0% Gr 2 90% 91% Gr 3 Afrikaans Setswana 20% Gr 1 90% Sesotho Xitsonga si. Swati 0% 1% 0% 9% 0% 0% 8% 0% Gr 4 Gr 5 Gr 6 Gr 9 Venda isi. Ndebele

Type Labour Market University/ FET • 15% • • Legislators, managers, assoc professionals • High quality secondary school Type of institution (FET or University) Quality of institution Type of qualification (diploma, degree etc. ) Field of study (Engineering, Arts etc. ) High quality primary school Some motivated, lucky or talented students make the transition Clerks, service workers, shop personnel, skilled agric/fishery workers, plant and machinery operators) Unskilled (18%) QLFS 2014 Minority (20%) Big demand for good schools despite fees Some scholarships/bursaries Unequal society Majority (80%) Low SES background Low quality primary school Attainment (Broad - 35%) Low quality secondary school +ECD Low qual ECD Elementary occupations & domestic workers Unemployed - Quality Semi. Skilled (32%) High SES background 41

What are the “binding constraints" in education? • A binding constraint is one where, if you do not address it first, trying to address other constraints has almost no impact. Uses

PSPPD: Researchers Paula Armstrong Annika Bergbauer Carol Nuga Deliwe (DBE) Jaamia Galant Janet Graaff Martin Gustafsson Ursula Hoadley Dumisani Hompashe Janeli Kotzé Nompumelelo Mohohlwane (DBE) Megan Sager Uses Debra Shepherd Nicholas Spaull Stephen Taylor (DBE) Hendrik van Broekhuizen Servaas van der Berg Surette van Staden Chris van Wyk Ntsizwa Vilakazi* Marisa von Fintel Gabrielle Wills

PSPPD & Zenex Report Launch

Accountability & Capacity Richard Elmore & colleagues

What were the binding constraints?

Main Conclusions 1. The majority of South African children are not learning to read in any language by the end of Grade 3 and this is the over-arching constraint of the whole school system. These students never get a firm hold on the first rung of the academic ladder and stumble forward into higher grades with low self esteem and compromised learning. 2. Focusing on the binding constraints first is the best use of limited human and financial resources 3. Any attempt to solve SA’s root problems must address BOTH accountability and capacity related issues to get traction. 4. You cannot solve a political problem with a technical solution. The lack of accountability is rooted in politics and that is where the solution for that problem must be found.

What Can AG Orbis Foundation do about this? 1. Prioritize Grades 1 -3: Build consensus that solving the problem must start in the early grades & prioritize learning to read for meaning and moving from counting to calculating. Matric starts in Grade 1 1. Capacity: Develop a course to teach Foundation Phase teachers how to teach reading. ITREP study shows total inadequacy of existing HEI’s to teach new teachers how to teach reading. Don’t throw good money after bad. 2. Build an “Allan Gray Teaching Institute” – hire the best and most experienced teachers to be teacher trainers. No fees. Highly practical. Focused on primary school teachers. Trains new teachers and offers short residential courses for existing teachers. Place of excellence for teachers and education (currently these do not exist).

Comments / Questions? These research reports & policy briefs are available online at resep. sun. ac. za Nicholas. Spaull@gmail. com

Brief headline findings from some new research Learning Deficits “Already by Grade 2 more than 50% of students in quintiles 1 -4 are not on track according to the ANAs” (Van der Berg, 2015) Mother-Tongue Instruction “Using ANA data from 9, 180 schools Taylor & Von Fintel (2016) find that additional instruction in mothertongue improves English acquisition by 0. 17 SD. ” District Resources “ 61% of Gr 10 -12 teachers were visited by a curriculum advisor compared to 45% of Grade 1 -3 teachers” (Wills, 2016)

Gender Inequality “ 71% of school teachers are women 36% of school principals are women” (Wills, 2016) Early Childhood Development “The ECD Audit 2013 shows that ECD practitioners earn R 1, 400 - R 2, 000 per month and 70% have no qualifications whatsoever” (Kotze, 2015) Weak Pedagogy “In a review of the literature Hoadley (2016) finds that reading classroom practice is characterized by collective instruction or chorusing with little opportunity for reading & writing”

Grade 5 unit counting Many students do not learn to move beyond a base-1 number system to a base 10 number system etc. (Eric Scholar)

Ref: Eric Scholar

Learning to read for meaning is not ‘easy’ but there is a lot of evidence showing what the components are & how to teach them 55

Course conten t 56

READING FOR MEANING Current state of play… Researcher s • Setting norms/benchmarks in African languages • Figuring out the best way to teach African languages • Publishing & teaching Implement ers • Identifying modalities that ‘work’; coaches? Train-the-trainer? • Recruiting & training African-language coaches/trainers • Dealing with funding cycles and organizational funding • Setting up evaluation designs that get at causality & standardizing instruments Evaluators • Marrying Quant & Qual to see what works and how • Determining cost effectiveness • Dealing with political & budgetary realities Governmen t • Keeping an eye on scalability 57

Egregious Inequality: Bimodality– indisputable fact PIRLS / TIMSS / SACMEQ / NSES / ANA / Matric… by Wealth / Language / Location / Dept… 58

We’ve been on this rodeo before… 1. Policy déjà vu 2. 3. 4. 5. 6. 7. 8. National Reading Strategy (DBE, 2008) – Pandor Teaching Reading in the Early Grades: A Teacher’s Handbooks (DBE, 2008) Western Cape Numeracy & Literacy Strategy 2006 -2016 (WCED, 2006) Foundations for Learning Campaign (DBE, 2008) Gauteng Primary Literacy Strategy 2010 -2014 (GDE, 2010) Systematic Method for Reading Success (Hollingsworth & Gains, 2009) Western Cape Living Labs Schools (WCED, 2015) Drop Everything & Read Campaign (See PSPPD Report (2016) for full discussion of previous reading initiatives)

What do primary school mathematics teachers know about mathematics? (Spaull & Venkat, 2015) Figure 1: Proportion of South African grade 6 mathematics teachers by content knowledge (CK) group SACMEQ 2007 (with 95% confidence interval) [401 Gr 6 maths teachers] 80% 70% 60% 50% 40% 62% 30% 20% 17% 5% 0% CK critically below level taught (pre Gr 4) CK below level taught (Gr 4/5) CK at level taught (Gr 6/7) 16% CK above level taught (Gr 8/9) 61

SADTU membership SADTU % of total (2012) TOTAL Union membership (2012) 66% WP PEU 4% 48% NW 72% NC SAOU 8% 69% MP NATU 7% NAPTOSA 15% 76% LP 82% KZN 63% FS 59% GP SADTU 66% 53% EC 74% 0% 20% 40% 60% 80% 100% 62

“The left hand barrel has horizontal wooden slabs, while the right hand side barrel has vertical slabs. The volume in the first barrel depends on the sum of the width of all slabs. Increasing the width of any slab will increase the volume of the barrel. So a strategy on improving anything you can, when you can, while you can, would be effective. The volume in the second barrel is determined by the length of the shortest slab. Two implications of the second barrel are that the impact of a change in a slab on the volume of the barrel depends on whether it is the binding constraint or not. If not, the impact is zero. If it is the binding constraint, the impact will depend on the distance between the shortest slab and the next shortest slab” (Hausmann, Klinger, & Wagner, 2008, p. 17). 63

Higher education in perspective When speaking about higher education it’s important to remember that this is only a very small proportion of the population Source: DBE (2013) Internal Efficiency of the schooling System 64

Gustafsson, 2011 – When & how WP 10% • “What do the magnitudes from Figure 4 mean in terms of the holding of qualifications? In particular, what widely recognised qualifications do the 60% of youths who do not obtain a Matric hold? …Only around 1% of youths hold no Matric but do hold some other nonschool certificate or diploma issued by, for instance, an FET college” (Gustafsson, 2011: p. 11) 65

Province…SACMEQ III Reading (Gr 6)