PISA 2012 1 What is PISA International largescale

PISA 2012 1

What is PISA? § International large-scale assessment organized by the Organization for Economic Cooperation and Development (OECD) § NCES is U. S. coordinating and funding organization § PISA Governing Board (PGB) determines policy § Assessment of 15 -year-old students § Administered every 3 years since 2000 2

What is PISA? § In 2012, 68 participating education systems § Three U. S. states: Connecticut, Florida, and Massachusetts 3

What is PISA? § Assessment subjects: § Mathematics, science, reading literacy § Mathematics was main subject assessed in 2012 § Computer-based mathematics and reading literacy § Optional assessment taken by 32 education systems § Content knowledge, not limited to school-based curricula § PISA assesses applied knowledge/literacy: § “How well can students nearing the end of compulsory schooling apply their knowledge to reallife situations? ” 4

What PISA Reports Average Scores: Scale of 0 -1000 for all domains Proficiency Levels: Percentages of students scoring at 6 levels Trends: Change between average scores in 2012 and the scores in previous assessment years Subgroup scores: International (e. g. , gender, language spoken in home) and U. S. specific variables (e. g. , race/ethnicity) 5

U. S PISA Findings 6

General patterns of U. S. results § Ranked better in reading literacy than in mathematics and science literacy § Below OECD average score in mathematics only § Higher percentage at low proficiency levels than OECD average in mathematics only § Lower percentage of top performers than OECD average in mathematics only § No measurable change in average scores in mathematics, science, or reading literacy 7

Mathematics Literacy 8

PISA 2012 mathematics literacy content categories Quantity (25%) • Are 15 -year-olds able to comprehend multiple representations of numbers, engage in mental calculation, employ estimation, and assess the reasonableness of results? Space and Shape (25%) • Can students understand perspective, create and read maps, and manipulate 3 D objects? Uncertainty and Data (25%) • Can students use probability and statistics and other techniques of data representation and description to mathematically describe, model, and interpret uncertainty? Change and Relationships (25%) • Can students model change and relationships with the appropriate functions and equations? Note: Percentages shown are the approximate percentage of scale score points. 9

PISA 2012 mathematics literacy process categories Formulate (25%) • Can 15 -year-olds recognize and identify opportunities to use mathematics and then provide mathematical structure to a problem presented in some contextualized form in order to formulate situations mathematically? Employ (50%) • Are students able to employ mathematical concepts, facts, procedures, and reasoning to solve mathematically-formulated problems and obtain mathematical conclusions? Interpret (25%) • Can students interpret, apply and evaluate mathematical outcomes in order to determine whether results are reasonable and make sense in the context of the problem? Note: Percentages shown are the approximate percentage of scale score points. 10

PISA mathematics literacy proficiency levels Level 6 Level 5 Level 4 Level 3 Level 2 Level 1 Identify information and carry out routine procedures according to direct instructions in explicit situations. Employ basic algorithms, formulae, procedures, or conventions. Capable of direct reasoning and making literal interpretations of the results. Execute clearly described procedures, including those that require sequential decisions. Select and apply simple problemsolving strategies. Work effectively with explicit models that may involve constraints or making assumptions. Capable of reasoning with some insight in straightforward contexts. Work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representations, symbolic and formal characterizations and insight pertaining to these situations. Apply insight along with a mastery of symbolic and formal mathematical operations and relationships to develop new approaches and strategies for attacking novel situations. 11

Mathematics literacy – Example item Question: CHARTS Proficiency level 1 12

Mathematics literacy – Example item Question: CHARTS - Continued Percentage of students answering correctly in 2012: OECD average: 80% U. S. : 77% 13

Mathematics literacy – Example item Question: DRIP RATE Proficiency level 5 14

Mathematics literacy – Example item Question: DRIP RATE - Continued Percentage of students answering correctly in PISA 2012: OECD average: 26% U. S. : 30% 15

Of 34 OECD Countries, U. S. A. Ranks 27 th in Mathematics Literacy OECD Higher than U. S. average Not measurably different from U. S. average U. S. Lower than U. S. average Source: National Center for Education Statistics, 2013, http: //nces. ed. gov/surveys/pisa 2012/pisa 2012 highlights_3 a. asp. 16

U. S. 15 -year-olds OECD average in mathematics literacy FL 467 (*, **) MA 514 (*, **) CT 506 (*) * = State avg. different from U. S. ** = State avg. different from OECD Average higher than U. S. average Average not measurably different from U. S. average Average lower than U. S. average 17

State results in mathematics literacy Massachusetts Connecticut Florida 9 12 36 12 14 5 44 39 24 Number of education systems higher than state Number of education systems not measurably different from state Number of education systems lower than state Note: Comparisons include 65 education systems. 18

U. S. not measurably different from OECD average in two mathematics content subscales Subscale U. S. average score OECD average score Quantity 478 495 Uncertainty and data 488 493 Change and relationships 488 493 Space and shape 463 490 Not measurably different from U. S. average Significantly higher than U. S. average 19

In mathematics literacy, 9 percent of U. S. 15 -year-old students scored at proficiency level 5 or above FL 6% (*, **) MA 19% (*, **) CT 16% (*, **) * = State percentage different from U. S. ** = State percentage different from OECD Percentage higher than U. S. Percentage not measurably different from U. S. Percentage lower than U. S. 20

No measurable change since 2009 in average mathematics literacy scores in more than half of PISA education systems, including U. S. • Of the 62 education systems in PISA 2009 and 2012: o In 33 (including the U. S. ) no measurable change in average scores o In 18 average scores increased § Russian Federation was below U. S. and OECD averages in 2003; not measurably different in 2012 § Poland was lower than OECD and not measurably different from U. S. averages in 2003; higher than both in 2012 o In 11 average scores declined § § Finland declined between 2006 -2009 and again between 20092012; scored 548 in 2006 and 519 in 2012 Norway was above U. S. average in 2003; not measurably different from U. S. average and below OECD average in 2012 21

Science Literacy 22

PISA science literacy proficiency levels Level 6 Level 5 Level 4 Level 3 Level 2 Level 1 Present scientific explanations that are obvious and follow explicitly from given evidence. Use direct reasoning and make literal interpretations of the results of scientific inquiry or technological problem solving. Identify clearly described scientific issues in a range of contexts. Select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Use welldeveloped inquiry abilities, link knowledge appropriately, and bring critical insights to situations. Construct explanations based on evidence and arguments based on critical analysis. Link different information sources and explanations and use evidence from those sources to justify decisions. Demonstrate advanced scientific thinking and reasoning, and use scientific understanding in support of solutions to unfamiliar scientific situations. 23

Science literacy – Example item Question: MARY MONTAGU Proficiency level 2 24

Science literacy – Example item Question: MARY MONTAGU - Continued Percentage of students answering correctly in 2006: OECD average: 75% U. S. : 73% 25

Science literacy – Example item Question: GREENHOUSE Proficiency level 6 26

Science literacy – Example item Question: GREENHOUSE - Continued Percentage of students answering correctly in 2006: OECD average: 19% U. S. : 18% 27

Of 34 OECD Countries, U. S. A. Ranks 20 th in Science Literacy U. S. Higher than U. S. average Not measurably different from U. S. average Source: National Center for Education Statistics, 2013, http: //nces. ed. gov/surveys/pisa 2012/pisa 2012 highlights_4 a. asp. Lower than U. S. average 28

U. S. 15 -year-old students not measurably different from OECD average in science literacy MA 527 (*, **) FL 485 (**) CT 521 (*, **) * = State avg. different from U. S. ** = State avg. different from OECD Average higher than U. S. average Average not measurably different from U. S. average Average lower than U. S. average 29

State results in science literacy Massachusetts Connecticut Florida Number of education systems higher than state 6 7 26 Number of education systems not measurably different from state 14 15 15 Number of education systems lower than state 45 43 24 Note: Comparisons include 65 education systems. 30

In science literacy, 7 percent of U. S. 15 -year-old students scored at proficiency level 5 or above MA 14% (*, **) CT 13% (*, **) FL 5% (**) * = State percentage different from U. S. ** = State percentage different from OECD Percentage higher than U. S. Percentage not measurably different from U. S. Percentage lower than U. S. 31

No measurable change since 2009 in average science literacy scores in more than half of PISA education systems, including the U. S. • Of the 62 education systems that participated in PISA 2009 and 2012: o In 43 (including the U. S. ) there was no measurable change in average scores o In 13 average scores increased § Poland was not measurably different from U. S. and OECD averages in 2006; above U. S. and OECD averages in 2012 o In 6 average scores declined § Sweden was higher than U. S. and OECD averages in 2006; below U. S. and OECD averages in 2012 32

Reading Literacy 33

PISA reading literacy proficiency levels Level 5 Level 3 Level 2 Level 1 b Locate a single piece of explicitly stated information in a prominent position in a short, simple text. Level 1 a Recognize the main theme or author’s purpose in a text about a familiar topic. Locate one or more pieces of information, which may need to be inferred and may need to meet several conditions. Integrate several parts of a text in order to identify a main idea, understand a relationship, or construe the meaning of a word or phrase. Level 4 Interpret the meaning of nuances of language in a section of text by taking into account the text as a whole. Locate and organize several pieces of deeply embedded information, inferring which information in the text is relevant. Level 6 Make multiple inferences, comparisons, and contrasts that are both detailed and precise. Deal with unfamiliar ideas and generate abstract categories for interpretations. 34

Reading literacy – Example item Question: TELECOMMUTING Proficiency level 3 35

Reading literacy – Example item Question: TELECOMMUTING-Continued Percentage of students answering correctly in 2009: OECD average: 52% U. S. : 55% 36

Reading literacy – Example item Question: THE PLAY’S THE THING Proficiency level 6 37

Reading literacy – Example item Question: THE PLAY’S THE THING- Continued What were the characters in the play doing just before the curtain went up? Percentage of students answering correctly in PISA 2009: OECD average: 13% U. S. : 13% 38

Of 34 OECD Countries, U. S. A. Ranks 17 th in Reading Literacy U. S. Higher than U. S. average Not measurably different from U. S. average Lower than U. S. average Source: National Center for Education Statistics, 2013, http: //nces. ed. gov/surveys/pisa 2012/pisa 2012 highlights_5 a. asp. 39

U. S. 15 -year-olds not measurably different from OECD average in reading literacy MA 527 (*, **) FL 492 CT 521 (*, **) * = State avg. different than U. S. ** = State avg. different than OECD Average is higher than U. S. average Average not measurably different from U. S. average Average is lower than U. S. average 40

State results in reading literacy Massachusetts Connecticut Florida 3 4 21 Number of education systems not measurably different than state 9 16 17 Number of education systems lower than state 53 45 27 Number of education systems higher than state Note: Comparisons include 65 education systems. 41

In reading literacy, 8 percent of U. S. 15 -year-old students scored at proficiency level 5 or above, not measurably different from OECD average CT 15% (*, **) MA 16% (*, **) FL 6% (*, **) * = State percentage different from U. S. ** = State percentage different from OECD Percentage higher than Percentage not measurably different from U. S. Percentage lower than U. S. 42

No measurable change since 2009 in average reading literacy scores in more than half of PISA education systems, including the U. S. • Of the 62 education systems that participated in PISA 2009 and 2012: o In 34 (including the U. S. ) no measurable change in average scores o In 21 average scores increased § Germany was lower than U. S. and OECD averages in 2000; higher than U. S. and OECD averages in 2012 o In 7 average scores declined § Sweden was higher than OECD average and not different from U. S. in 2000; lower than both U. S. and OECD averages in 2012 43

Among OECD Countries, U. S. has the 4 th Largest Achievement Gap Between High-SES and Low-SES Students for 2006 PISA – Science Literacy U. S. Source: PISA 2006 Results, OECD, table 4. 8 b 44

Among OECD Countries, U. S. has the 5 th Largest Achievement Gap Between High-SES and Low-SES Students for 2009 PISA – Reading Literacy U. S. Source: PISA 2009 Results, OECD, Table II. 3. 1 OECD 45

The U. S. ranks 26 th among 34 OECD Countries on the Percentage of Low-SES Students who are High-Performing PISA 2012 Mathematics Literacy OECD U. S. Note: High-performing, low-SES students are those who are in the bottom quarter of the ESCS in their country but perform in the top quarter across students from all countries after accounting for socioeconomic background. Source: PISA 2012 Results, OECD, Annex B 1, Chapter 2, Table II. 2. 7 a 46

The U. S. Achievement Gap Between High-SES and Low-SES Students is Equivalent to Over Two Years of Schooling 2012 PISA – Mathematics Literacy U. S. Source: PISA 2012 Results, OECD, Annex B 1, Chapter 2, Table II. 2. 4 a OECD 47

For more information PISA at NCES: http: //nces. ed. gov/surveys/pisa/ Contact: Dana Kelly NCES 202 -219 -7101 Dana. Kelly@ed. gov 48