1 OECD Programme for International Student Assessment PISA

1 OECD Programme for International Student Assessment (PISA) Improving both quality and equity Insights from PISA 2000 EI Round Table, Florence, 17 November 2003 Andreas Schleicher Head, Indicators and Analysis Division OECD

2 Improving both quality and equity 1. Why we need to worry l The significance of educational outcomes for individuals and societies 2. Where we are today l What PISA shows about student performance and the social distribution of learning opportunities in OECD countries 3. Where we can be l What the best performing countries show can be achieved 4. How we can get there l Levers for policy that emerge from PISA

3 The significance of educational outcomes. Changing contexts for education systems.

4 Changing contexts for education r Living l with an ageing population In some OECD countries, the share of over-65 s in the population is growing twice has fast as the share of under 35 s is shrinking r Unwanted l but needed School systems in most OECD countries will need to integrate unprecedented numbers of immigrants. . . but only few are prepared

5 Changing contexts for education r Knowledge workers the only fast growing share of the workforce l By 2002 - about the time when school reforms put in place today will begin to show effects in labour markets. . . Manufacturing output in OECD area is likely to double… …Manufacturing employment is likely to shrink to 10% of the workforce …And this is affects all jobs…

Traditional jobs are changing too… 6 Literacy skills in the manufacturing sector Pages Total number of pages of repair manuals for Opel cars Source : Georg Spöttl, 2002 from 1933 to 1998 Year 1933 1951 1966 1983 1998

7 Changing contexts for education r Knowledge l workers - the new “capitalists“ Knowledge is becoming the key economic and social resource … and the only scarce one l Collectively, knowledge workers own the modern means of production. . . and through their stakes in pension and mutual funds, they have become majority shareholders and owners of key businesses in the knowledge society l The knowledge society is a society of juniors and seniors. . . rather than of bosses and subordinates

Unique opportunities 8 r Unlimited upward mobility… l l Knowledge changes rapidly Everybody starts from ignorance – Knowledge differs from traditional means of production in that it cannot be inherited or bequeathed l Knowledge is “public” – Knowledge has to be put in a form in which it can be taught and is therefore universally accessible l Every impediment to mobility… …is perceived as a form of discrimination … If our education systems deliver on their promises

9 Where we are today. What PISA shows about student performance and the social distribution of learning opportunities in OECD countries.

PISA - The OECD Programme for International Student Assessment 10 r A regular assessment of the yield of education (2000, 2003, 2006, 2009, …) l r Comparable skill measures l r that can guide policy decisions Insights into the mix of factors which contribute to the development of knowledge and skills l r including and beyond the curriculum and how these factors operate similarly or differently across countries A strong substantive and cross-cultural core for defining performance targets

PISA - The OECD Programme for International Student Assessment 11 r The most comprehensive international assessment to date l Geographic and economic coverage – 340, 000 students randomly sampled – All 30 OECD countries plus a growing number of non. OECD countries l Subject matter coverage – Reading, Mathematics, Science – Cross-curricular competencies l l Variety of task formats Depths – A total of 7 hours of assessment material

12 Three broad categories of key competencies Using “tools” interactively to engage with the world To analyse, compare, contrast, ande. g. evaluate Using language, symbols and texts Toinformation think imaginatively Interacting with Capitalising on the potential PISA 2000: A new concept of literacy of technologies Acting Interacting in Accessing, managing, integrating autonomously diverse groups and evaluating written information e. g. in order to develop andwithin potential, Acting the bigger picture Relating wellones to knowledge others and to participate in, and contribute to, society Co-operating, working in Forming teams and conducting life plans Taking responsibility and Managing and resolving situations conflicts To apply knowledge in real-life understanding rights and limits To communicate thoughts and ideas effectively

13 Using “tools” interactively to engage with the world To analyse, compare, contrast, ande. g. evaluate Using language, symbols and texts Toinformation think imaginatively Interacting with Capitalising on the potential Reading literacy of technologies Acting Interacting in Using, diverse interpreting autonomously groups and reflecting e. g. on written material e. g. Acting within the bigger picture Relating well to others Co-operating, working in Forming teams and conducting life plans Taking responsibility and Managing and resolving situations conflicts To apply knowledge in real-life understanding rights and limits To communicate thoughts and ideas effectively

14 Using “tools” interactively to engage with the world To analyse, compare, contrast, ande. g. evaluate Using language, symbols and texts Toinformation think imaginatively Interacting with Capitalising on the potential Mathematical literacy of technologies Acting Interacting in Emphasis is on mathematical knowledge put into autonomously diverse groups functional use in a multitude of different e. g. situations e. g. Acting within the bigger picture well to others in varied, Relating reflective and insight-based ways Co-operating, working in Forming teams and conducting life plans Taking responsibility and Managing and resolving situations conflicts To apply knowledge in real-life understanding rights and limits To communicate thoughts and ideas effectively

15 Using “tools” interactively to engage with the world To analyse, compare, contrast, ande. g. evaluate Using language, symbols and texts Toinformation think imaginatively Interacting with Capitalising on the potential Scientific literacy of technologies Acting Interacting in Using scientific knowledge, identifying scientific autonomously diverse groups questions, and drawinge. g. evidence-based conclusions to e. g. Acting the within the bigger picture understand make well decisions about natural world Relating to others Co-operating, working in Forming teams and conducting life plans Taking responsibility and Managing and resolving situations conflicts To apply knowledge in real-life understanding rights and limits To communicate thoughts and ideas effectively

16 Using “tools” interactively to engage with the world Under development: Interacting in PISA assessment of diverse groups – Problem-solving skills PISA self-reports on: – Dispositions to learning – Learning strategies – Engagement with school Acting autonomously

17 Using “tools” interactively to engage with the world Interacting in diverse groups Acting autonomously Not yet developed

19 Where we can be. What the best performing countries show can be achieved.

High Performance 20 High performance Low social equity High social equity Moderate impact of social background on performance Strong impact of social background on performance Low social equity High social equity Low Performance

21 High Performance r Quality and equity can be achieved together High performance • ‘dumbing is not an inevitable Low social down’ equity High social equity consequence of the pursuit of equity Moderate impact of Strong impact of social background on performance • ‘levelling up’ is achievable (e. g. Finland, Korea, Canada) social background on performance Low social equity High social equity Low Performance .

Is it all innate ability? 22 Variation in student performance 22 21 -25 21 19 -24 8 3 -10 16 10 -20 7 5 -9 6 4 -9 5 3 -9 19 17 -21 1 10 9 -11

Is it all innate ability? 23 Variation in student performance 22 21 -25 21 19 -24 8 3 -10 16 10 -20 7 5 -9 6 4 -9 5 3 -9 19 17 -21 1 10 9 -11

Is it all innate ability? 24 Variation in student performance Variation of performance within schools Variation of performance between schools 22 21 -25 21 19 -24 8 3 -10 16 10 -20 7 5 -9 6 4 -9 5 3 -9 19 17 -21 1 10 9 -11

25 Policy levers. Overall findings

Policy Levers 26 r Performance in reading l Students from advantaged backgrounds… … have a greater chance of coming to school more engaged in reading and entering into a virtuous circle of increasing reading interest and improved reading performance … but not all engaged students come from privileged homes… … and those from more modest backgrounds who read regularly and feel positive about it are better readers than students with home advantages and weaker reading engagement l Schools can make a significant difference to bring students into the virtuous circle – Seeking mutual reinforcement of cognitive skills and motivation, particularly for boys

Policy Levers 27 r Student approaches to learning l The ability to manage one’s learning is both an important outcome of education and a contributor to student literacy skills at school – l Different aspects of students’ learning approaches are closely related – l Learning strategies, motivation, self-related beliefs, preferred learning styles Well-motivated and self-confident students tend to invest in effective learning strategies and this contributes to their literacy skills Immigrant students tend to be weaker performers … but they do not have weaker characteristics as learners l Boys and girls each have distinctive strengths and weaknesses as learners – – Girls stronger in relation to motivation and self-confidence in reading Boys believing more than girls in their own efficacy as learners and in their mathematical abilities

Policy Levers 28 r Student engagement at school l An important outcome in itself – – l The prevalence of disaffected students varies significantly across schools in each country – – l Only weak link to student’s social background – there is thus scope for school policy/practice to engage students But strong link to school’s social background Students in schools with strong average engagement tend to perform well – – l Disaffection at age 15 can potentially be a precursor to the onset of more serious problems among vulnerable young people Engagement at age 15 is likely to influence students’ choices and educational pathways Engagement and performance seem to work complementary The school climate seems to make more of a difference than resources For individual students, strong performance does not necessarily ensure strong engagement at school – Relationship complex

29 Policy levers. Some characteristics shared by some strongly performing countries Combining the empirical results obtained through PISA with qualitative information on the sociocultural conditions and education policy strategies.

High Performance 30 High performance plus Low social equity Netherlands High social equity Moderate impact of social background on performance Strong impact of social background on performance Low social equity High social equity Low Performance .

31 Sympathy doesn’t raise standards – aspiration does r In the countries studied l l National research teams report a strong “culture of performance” – Which drives students, parents, teachers and the educational administration to high performance standards PISA shows that students perceived a high degree of teacher support – Which should not be simply equated with “achievement press”

Governance of the school system 32 r In the countries studied… l l Decentralised decision-making is combined with devices to ensure a fair distribution of substantive educational opportunities The provision of standards and curricula at national/subnational levels is combined with advanced evaluation systems – That are implemented by professional agencies l Process-oriented assessments and/or centralised final examinations are complimented with individual reports and feed-back mechanisms on student learning progress

E. g. Learning environment and course offering High Performance 33 High degree of school-level autonomy 11% Low degree of school-level autonomy % Variance between schools 20% 9% 76% Strong impact of social background on performance 75% 7% Moderate impact of social background on performance 71% r=. 51 Low Performance .

Organisation of instruction 34 r In the countries studied… l Schools and teachers have explicit strategies and approaches for teaching heterogeneous groups of learners – A high degree of individualised learning processes – Disparities related to socio-economic factors and migration are recognised as major challenges l l Students are offered a variety of extracurricular activities Schools offer differentiated support structures for students – E. g. school psychologists or career counsellors l Institutional differentiation is introduced, if at all, at later stages – Integrated approaches also contributed to reducing the impact of students socio-economic background on outcomes

Early selection and institutional stratification 35 High Performance High degree of integration Early selection and stratification Moderate impact of social background on performance Strong impact of social background on performance Low Performance .

Support systems and professional teacher development 36 r In the countries studied… l l l Effective support systems are located at individual school level or in specialised support institutions Teacher training schemes are selective The training of pre-school personnel is closely integrated with the professional development of teachers Continuing professional development is a constitutive part of the system Special attention is paid to the professional development of school management personnel

Students perceived teacher support 37 High Performance High degree of support Low degree of support Moderate impact of social background on performance Strong impact of social background on performance Low Performance .

39 Summary of common characteristics Uniformity Diversity “hit and miss” Universal high standards “Inputs” Outcomes Bureaucratic Devolved responsibility Look up Look outwards Received wisdom Data and best practice Evaluation to control Prescription Motivating feedback and incentivising success and innovation Informed profession

40 One challenge – different approaches The future of education systems needs to be “knowledge rich” Informed professional judgement, the teacher as a “knowledge worker” Informed prescription National prescription Professional judgement Uninformed prescription, teachers implement curricula Uninformed professional judgement The tradition of education systems has been “knowledge poor”

Further information 41 l www. oecd. org www. pisa. oecd. org email: pisa@oecd. org l Andreas. Schleicher@OECD. org l l … and remember: Without data, you are just another person with an opinion