Reviewing Using the Instructional Materials Evaluation Tool ELA

Reviewing Using the Instructional Materials Evaluation Tool: ELA Module 101: Text Complexity & Text-Dependent Questions www. achievethecore. org

Essential Questions • How does the Instructional Materials Evaluation Tool (IMET) reflect the major features of the Standards and the Shifts? • What understandings support high-quality, accurate application of the IMET metrics? PAGE 2

Goals • Understand how aligned materials embody the shifts inherent in the Common Core State Standards • Understand the precise meaning of each metric • Recognize examples and non-examples related to each IMET criteria metric PAGE 3

The quality of instructional materials in our classrooms has a large impact on student learning. “There is strong evidence that the choice of instructional materials has large effects on student learning – effects that rival in size those that are associated with differences in teacher effectiveness. ” Chingos & Whitehurst, Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core PAGE 4

Agenda • Introduction to the IMET • Non-Negotiable 1: Text Complexity • Non-Negotiable 2: Text-Dependent Questions & Tasks • Alignment Criterion 1: Range and Quality of Texts • Alignment Criterion 2: Questions & Tasks • Next Steps PAGE 5

Norms for Our Work • Consider two norms you’d like to see us adhere to for the next two days. • Discuss your thoughts with your shoulder partner. • Be ready to share. PAGE 6

History of the IMET • Achieve • Student Achievement Partners • Council of Chief State School Officers PAGE 7

IMET Sources • Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects • Supplement to Appendix A of the Common Core State Standards for ELA/Literacy: New Research on Text Complexity PAGE 8

When to Use the IMET • • Purchasing or evaluating new materials Evaluating materials currently in use Developing materials Understanding key design principles PAGE 9

The IMET: An Overview • Versions for ELA/Literacy: Kindergarten– 2 nd grade, and 3 rd– 12 th grades – For Math: Kindergarten– 8 th grade, and High School • Comprehensive curricula • 3 Levels of Alignment: – Non-Negotiable (NN) Alignment Criteria – Alignment Criteria (AC) – Indicators of Quality • www. achievethecore. org PAGE 10

3 Shifts Regular practice with complex texts and their academic language Reading, writing, and speaking grounded in evidence from text, both literary and informational Building knowledge through content-rich non-fiction PAGE 11

Shift 1: Text Complexity ✓ HS to College Gap Regular practice with complex texts and their academic language ✓ Complexity predicts college success (ACT study) ✓ High rates of remediation ✓ CCSS staircase of complexity: ES → HS ✓ Engaging with complex language and rich vocabulary PAGE 12

Students who can read literally can read inferentially. PAGE 13

Students who can perform one strategy can perform all strategies. PAGE 14

Ability to read complex text differentiates college -ready readers. PAGE 15

Shift 2: Evidence ✓ College and workplace writing Reading, writing and speaking grounded in evidence from text, both literary and informational ✓ Major emphasis of the ELA Standards ✓ Hallmarks of strong readers and writers ✓ Equity PAGE 16

Shift 3: Knowledge ✓Large research base Building knowledge through content -rich non-fiction ✓Knowledge crucial to comprehension ✓Very little nonfiction in school ✓College & workplace focus on nonfiction ✓Informational text harder PAGE 17

Everyone knows knowledge plays a role in comprehension, but how big of a role? “The Baseball Study” Recht & Leslie (1988) PAGE 18

The Baseball Study Recht & Leslie (1988) Compared reading comprehension for four categories of students: High reading ability, High knowledge of baseball High reading ability, Low knowledge of baseball Low reading ability, High knowledge of baseball Low reading ability, Low knowledge of baseball PAGE 19

Knowledge of the topic had a much bigger impact on comprehension than generalized reading ability did. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Measure of Comprehension high reading ability & high knowledge low reading ability & high knowledge high reading ability & low knowledge low reading ability & low knowledge PAGE 20

Activity: Color the Shifts PAGE 21

ELA Shifts 1. Regular practice with complex text and its academic language 2. Reading, writing, and speaking grounded from evidence from text, both literary and informational 3. Building knowledge through content-rich nonfiction PAGE 22

PAGE 23

Structure of the IMET • Front matter – Purpose, Getting Started • 2 Non-Negotiable Alignment Criteria • Text Complexity • Text-Dependent Questions • 7 Alignment Criteria • Summary Worksheet • Indicators of Quality PAGE 24

Navigating the Tool: Begin with Section 1, Non-Negotiable Alignment Criteria Metric to determine if the Non. Negotiable Criteria is met Decide: Meet Non. Negotiable? PAGE 25

Navigating the Tool: End with Evaluation Summary PAGE 26

Non-Negotiable 1: Text Complexity

Text Complexity Non-Negotiable 1: Texts are worthy of student time and attention; they have the appropriate level of complexity for the grade, according to both quantitative measures and qualitative analysis of text complexity. • Metric 1 A: 100% of texts must be accompanied by specific evidence that they have been analyzed with at least one research-based quantitative measure for grade band placement. • Metric 1 B: 100% of texts must be accompanied by specific evidence that they have been analyzed for their qualitative features indicating a specific grade level placement. IMET p. 4 PAGE 28

Text Complexity: What Is It and Why? CCSS. ELA-Literacy. CCRA. R. 10 Read and comprehend complex literary and informational texts independently and proficiently. For example, in 4 th grade: By the end of the year, read and comprehend informational texts, including history/social studies, science, and technical texts in the grades 4 -5 complexity band proficiently, with scaffolding as needed at the high end of the range. PAGE 29

What Makes a Text Complex? • • • Subtle and/or frequent transitions • • • Complex sentences • • Longer paragraphs Multiple and/or subtle themes and purposes Density of information Unfamiliar settings, topics, or events Lack of repetition, overlap, or similarity in words and sentences Uncommon vocabulary Lack of words, sentences, or paragraphs that review or pull things together for the student Any text structure which is less narrative and/or mixes structures PAGE 30

Analyzing Text Quality PAGE 31

Analyzing Text Complexity: Quantitative Verify that each text has been placed within a grade band based on at least one quantitative measure. PAGE 32

Practice: Analyzing Text Quality Read the excerpt from The Great Fire, and, with your table group, analyze the quality of the text: 1 Quantitative Analysis: Study the two measures and determine the appropriate grade band. 2 Qualitative Analysis: Using the rubric, evaluate the qualitative features of the text. Provide evidence for each feature. Place the passage in the appropriate grade. 3 Quality Criteria: Considering the qualitative criteria, is this text worth reading? PAGE 33

Quantitative Text Complexity Grapes of Wrath Lexile Score: 680 L Grade Band Placement: 2 -3? PAGE 34

Qualitative Measures Qualitative measures consider: ✓Text Structures ✓Language Features ✓Meaning/Purpose ✓Knowledge Demands . . PAGE 35

Text Complexity • Text features • Genre • Organization • Layers of meaning • Purpose • Concept complexity • Vocabulary • Sentence length and structure • Figurative language • Regional/ historical usage (dialects) Meaning Structure Language Knowledge • Content knowledge • Disciplinary knowledge • Intertextuality • Background & experiences PAGE 36

Tool: Analyzing Text Quality PAGE 37

Tool: Grade-Band Placement PAGE 38

Practice: Analyzing Text Quality Read the excerpt from The Great Fire and, with your table group, analyze the quality of the text: 1 2 3 Quantitative Analysis: Study the two measures and determine the appropriate grade band. Qualitative Analysis: Using the rubric, evaluate the qualitative features of the text. Provide evidence for each feature. Place the passage in the appropriate grade. Quality Criteria: Considering the qualitative criteria, is this text worth reading? PAGE 39

Reader and Task Considerations Finally, there are factors relating to your readers and the task to consider: ✓ Desired Outcomes/Understandings ✓ Complexity of Content ✓ Specific student needs ✓ Reading Skills ✓ Motivation & Engagement ✓ Prior Knowledge PAGE 40

Practice: Analyzing Text Quality Read the excerpt from The Great Fire, and, with your table group, analyze the quality of the text: 1 Quantitative Analysis: Study the two measures and determine the appropriate grade band. 2 Qualitative Analysis: Using the rubric, evaluate the qualitative features of the text. Provide evidence for each feature. Place the passage in the appropriate grade. 3 Quality Criteria: Considering the qualitative criteria, is this text worth reading? PAGE 41

How Does This Look in the IMET? Metric to determine if the Non. Negotiable Criterion is met Decide: Meet Non-Negotiable? IMET p. 5 PAGE 42

Example or Non-Example? PAGE 43

Example or Non-Example? PAGE 44

Rating PAGE 45

Alignment Criterion 1: Range and Quality of Texts

Range and Quality of Texts Alignment Criterion 1: Materials reflect the distribution of text types and genres required by the Standards. • Metric 1 A: Sequenced to build knowledge • Metric 1 B: Carefully selected anchor (or keystone) texts, with supporting texts varying widely in complexity Example Non-Example • Metric 1 C: Balance of text types and instructional time • Metric 1 D: Reflect text characteristics and genres called for by the standards • Metric 1 E: Opportunity for regular independent reading IMET p. 12 PAGE 47

Range and Quality of Texts Alignment Criterion 1 A: Texts are sequenced to build knowledge, opening the door to increasingly complex texts. “By reading texts in history/social studies, science, and other disciplines, students build a foundation of knowledge in these fields that will also give them the background to be better readers in all content areas. Students can only gain this foundation when the curriculum is intentionally and coherently structured to develop rich content knowledge within and across grades. ” -- CCSS Anchor Reading Standards IMET p. 13 PAGE 48

Range and Quality of Texts Alignment Criterion 1 A: Materials pay careful attention to providing a sequence or collection of texts that build knowledge systematically through reading, writing, listening, and speaking about topics under study. IMET p. 13 PAGE 49

Range and Quality of Texts Example PAGE 50

Range and Quality of Texts Alignment Criterion 1 B: Keystone texts are selected for their quality as being especially worthy of careful reading. • • Consider: Which of these K-2 read aloud texts are worthy of careful reading? Discuss: What would instructions supporting careful reading look like for teachers or students? What actions and directions would materials include? IMET p. 14 PAGE 51

Range and Quality of Texts Example PAGE 52

Range and Quality of Texts Alignment Criterion 1 C: Materials should reflect the balance of texts and instructional time called for in the Standards. Given this shift, what do you think is the percentage of informational texts, including literary nonfiction, the Standards require for both instruction and assessment? (Jot it down on a post-it. ) • In • the elementary school grades? 50% informational and 50% literary (grades K-5) the middle school grades? 55% informational and 45% literary (grades 6 -8) high school? 70% informational and 30% literary (grades 9 -12) IMET p. 15 PAGE 53

Range and Quality of Texts Alignment Criterion 1 D: Materials should reflect the text characteristics and genres called for in the Standards. IMET p. 16 PAGE 54

Range and Quality of Texts Alignment Criterion 1 E: Materials should increase the opportunity for independent reading to develop knowledge and love of reading. Close reading alone is NOT enough! In fact, done alone, it can INCREASE the achievement gap. IMET p. 17 PAGE 55

Range and Quality of Texts A volume of reading should be balanced with the close analytic reading. Close Reading Volume of Reading Fewer pages More pages Grade-level complex text Text at different levels of complexity All students same text Student or teacher choice of text Teaches students to attend to text and to words Rapidly builds knowledge & vocab Heavy support Light support Solely instructional Guided or independent Exposes students to higher level content Builds knowledge of words, and the world Gives all students access Builds love of reading PAGE 56

Range and Quality of Texts Independent Reading PAGE 57

Range and Quality of Texts Practice: AC 1 Range & Quality of Texts • Expeditionary Learning ELA Curriculum Plan, Grades 3 -5 • Louisiana Guidebook Overview, Grades 3 -5 Individually rate the materials against AC 1, noting evidence and questions. Be ready to share evidence you collected for metrics A -E. PAGE 58

Range and Quality of Texts Let’s reach consensus: • What evidence did you find where your materials were in alignment with the criterion? • What other evidence would you want to collect? PAGE 59

Break PAGE 60

Non-Negotiable 2: Text-Dependent Questions

Text-Dependent Questions Non-Negotiable 2: At least 80% of all questions in the submission are high-quality text-dependent and text -specific questions. The overwhelming majority of these questions are text-specific and draw student attention to the text. • Metric 2 A: At least 80% of all questions and tasks should be text-dependent to reflect the requirements of Reading Standard 1. • Metric 2 B: Questions and tasks accurately address the analytical thinking required by the Standards at each grade level. IMET p. 8 PAGE 62

Considering Text Dependency • Does answering the question require that students have read the passage? • Is the question tied to a passage (not standalone)? • Does the question require students to cite or use evidence from the passage to determine the correct answer? • Does the question require students to follow the details of, make inferences from, and/or evaluate what is read? PAGE 63

Practice: Analyzing Question Quality 1. With your partner, analyze the sample questions for The Great Fire excerpt in your handouts. 2. Use the criteria on the next page in your packet to guide your analysis of text dependency. 3. Compare your responses to the Answer Key on the next page. PAGE 64

Defining Text-Specific Questions • Text-specific questions are a subset of textdependent questions. • The questions probe the specifics of the text and avoid “canned” questions that could be asked of any text. • These questions can only be applied to one specific text. PAGE 65

Text-Dependent Text-Specific What is the author’s message in the text? Why does Monk ask this question, “Which ‘We the People’ has ‘troubled the nation’”? What is the main idea of the passage? Why does Monk claim that popular sovereignty is the form of government in America? What details can you find that support the main idea? What evidence is there in paragraph three regarding Marshall’s claim about the “evolving nature of the constitution”? 66 PAGE 66

Example or Non-Example? PAGE 67

Text-Dependent Questions Metric 2 B • Text dependency is not enough. • Questions must reflect the analytical thinking required by the Standards for the grade level. PAGE 68

Text-Dependent Questions Example: Text-Dependent Questions Using Evidence to Answer Text-Dependent Questions: “Order in the Court!” (15 minutes) • Make sure students have their texts “Order in the Court!” (from Lesson 4). Distribute a large index card to each pair. • Ask them to record on the index card their pair’s answer to these questions: 1. 2. 3. In the section “What Was Her Crime? , ” why was it OK with Susan B. Anthony that she was arrested for voting? On page 14, the authors wrote, “The spectators were outraged. ” Why were the spectators outraged? (Underline the evidence using a red pencil. ) Based on her actions in the courtroom, how would you describe the kind of person Susan B. Anthony was? Make sure to use specific evidence from the text to support your response. (Underline the evidence using a green pencil. ) PAGE 69

Text-Dependent Questions Example: Text-Dependent Questions PAGE 70

Text-Dependent Questions Practice: Text-Dependent Questions • Use the passages (Fossil Fish Found! and Earthquakes) and associated question sets to gather evidence for the NN 2 metrics. • Be prepared to share your findings. PAGE 71

Text-Dependent Questions Let’s reach consensus: • What evidence did you find where your materials were in alignment with the criterion? • What other evidence would you want to collect? PAGE 72

Rating: Non Negotiable vs. Alignment Criteria Non- Negotiable Criteria Alignment Criteria Each metric rated Meets/Does Not Meet Each metric rated Meets (2 pts)/ Partially Meets (1 pt)/Does Not Meet (0) Every metric must be met to rate the criterion as Meets Must receive a specific number of points out of the total to rate the criterion as Meets Why? Standard 10 and Standard 1 PAGE 73

Alignment Criterion 2: Questions and Tasks

Questions and Tasks Alignment Criterion 2: Questions and tasks support students in building reading comprehension, in producing the textual evidence to support their responses, and in developing grade level academic language. • Metric 2 A: High-quality sequences of text-dependent questions are prevalent and can address any of the following: sustained attention Example from the text, rereading to Non-Example to making meaning gain evidence and clarity, and the acquisition of foundational skills. • Metric 2 B: Questions and tasks support students in unpacking the academic language (vocabulary and syntax) prevalent in complex texts. • Metric 2 C: Questions build to a deep understanding of the central ideas of the text. IMET p. 19 PAGE 75

Questions and Tasks • Read The Making of a Scientist. PAGE 76

Questions and Tasks Alignment Criteria 2 A: A coherent sequence of questions is designed to deepen understanding as students read and reread the text. • What was Feynman’s father trying to teach his son with the tiles? What sentence is the main point of this scene? • In this section of the text, Feynman put the word “doing” (in the final paragraph) in italics to draw attention to it. Why is he focusing on that word, and how does it connect to the lesson his father is trying to teach him in this example? • Why does Feynman’s father tell him about the lice and the mites on birds? PAGE 77

Questions and Tasks Alignment Criterion 2 B: Questions and tasks support unpacking academic language. Importance of Vocabulary • Nearly a century of research (Whipple 1925, NAEP 2012) • Feature of complex text that likely causes greatest difficulty (Nelson et al 2012) • Having to determine the meaning of too many words slows readers up; problem gets much worse with complex text. • Not knowing words on the page is debilitating. • “ 30 Million Word Gap” • After much research… IMET p. 21 PAGE 78

Questions and Tasks Alignment Criterion 2 B: Questions and tasks support unpacking academic language. Guidelines for Vocabulary • Focus primarily on tier 2 words? • Target words or phrases that are central to understanding the text? • Ask students to use context to determine meaning? • Ask about the meaning of figurative language, not labels of literary terms? • Require use of resources? • Attend to the particulars of the standards at each grade? IMET p. 21 PAGE 79

Questions and Tasks The three vocabulary tiers are vital to comprehension, but Tiers 2 and 3 typically require more deliberate effort. Tier 1 • Words of everyday speech • Ex: chair, book, drive Tier 2 Tier 3 • Appear in a range of texts • Ex: relative, accumulate, formulate, specificity • Specific to a domain or field of study • Ex: lava, amoeba, legislature, aorta, circumference Consider these words found in the Feynman lesson: • • • highchair vertically seconds complicated elementary • • • translate reality magnitude frightened consequence • • • slightest peck lice flakes protein PAGE 80

Questions and Tasks Academic Vocabulary: Examples or Non-Examples? PAGE 81

Questions and Tasks Alignment Criterion 2 C: Questions and tasks should drive students towards a deep understanding of the central ideas of the text. Culminating Task: In the final paragraph Feynman says he “was given something wonderful when he was a child. ” Using two of the examples from the text, explain what he was given and how it influenced his life. IMET p. 22 PAGE 82

Questions and Tasks Practice: Quality Questions and Tasks • With your partner, refer back to the metrics for Alignment Criterion 2. • Return to the passages (Fossil Fish Found! and Earthquakes) and associated question sets to gather evidence for metrics A, B, and C. • You may also use The Making of a Scientist lesson. PAGE 83

Questions and Tasks Let’s reach consensus: • What evidence did you find where your materials were in alignment with the criterion? • What other evidence would you want to collect? PAGE 84

Break PAGE 85

Essential Questions • How does the Instructional Materials Evaluation Tool (IMET) reflect the major features of the Standards and the Shifts? • What understandings support high-quality, accurate application of the IMET metrics? PAGE 86

Goals • Understand how aligned materials embody the shifts inherent in the Common Core State Standards • Understand the precise meaning of each metric • Recognize examples and non-examples related to each IMET criteria metric PAGE 87