The Lexile Framework for Reading Integrating Measurement and

The Lexile Framework for Reading: Integrating Measurement and Instruction COPENHAGEN Sunday, August 14, 2011 A. Jackson Stenner Chairman & CEO, Meta. Metrics Research Professor, UNC-Chapel Hill

The Lexile Framework is NOT: • A reading program • A test or method of assessment • Educational software

What is The Lexile Framework for Reading? • An educational tool – a scale - that links reading material and readers under a common metric – the Lexile. − Measures the complexity of reading material − Measures the reading level of individual readers

The Lexile Framework for Reading A Lexile is a measure of both reading level and text complexity, on the same scale. A student’s Lexile measure is her reading level. an 1170 L reader A book’s Lexile measure is its reading difficulty. an 1170 L book

Why The Lexile Framework? • LINK reader to text under a common measure • FORECAST levels of comprehension • TRACK growth over time

Where are Lexile Measures Found? • All major norm-referenced tests are linked • Over 30 million students received a Lexile measure last school year • Over 450 publishers • Nearly 115, 000 book titles to search at www. lexile. com • Over 400 million Lexiled articles

Where are Lexile Measures Found? Formative and Norm-Referenced Assessments

Where are Lexile Measures Found? Library Resources

The Lexile Scale How high does the Lexile scale go? How low does the Lexile scale go?

The Lexile Scale • Lexile measures typically range from below 200 L for beginning readers to above 1700 L for advanced readers. • Text below 200 L represents beginning-reading material. A student’s Lexile measure of 0 L or below is coded BR, signifying ‘Beginning Reading’.

Lexile Measures & Text Complexity “Quantitative dimensions of text complexity refer to those aspects of text complexity, such as word length or frequency, sentence length, and text cohesion, that are difficult if not impossible for a human reader to evaluate efficiently, especially in long texts, and are thus today typically measured by computer software. ” -from the Common Core State Standards

Determining Text Complexity • Sentence Length (syntactic complexity) Longer sentences are more complex and require more short-term memory to process.

Determining Text Complexity • Vocabulary (semantic difficulty) Less familiar words impede reading fluency and affect comprehension.

Analyzing Text Readability • Read and examine Camping is Fun. • Easy or difficult to read? Why?

Camping is Fun (M. Garcia) We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained and we got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep! On Sunday, we went home. "Did you like camping? " asked Uncle Pete. "Yes!" we said. "Camping is fun!"

Camping is Fun (M. Garcia) We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained and we got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep! On Sunday, we went home. "Did you like camping? " asked Uncle Pete. "Yes!" we said. "Camping is fun!" 200 L

Analyzing Text Readability • Read and examine an excerpt of a speech from Lee Kuan Yew: The Separation of Singapore from Malaysia • Difficult or easy to read? Why?

Speech from Lee Kuan Yew: Separation of Singapore from Malaysia "For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. . Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society. “

Speech from Lee Kuan Yew: Separation of Singapore from Malaysia "For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories. . Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society. “ 1550 L

Why 75% Comprehension? Research suggests that at 75%… • A reader can have a successful reading experience without frustration or boredom • A reader can achieve “functional comprehension” of the text • A reader will be sufficiently challenged (by vocabulary and syntax) to improve 75% is the “right amount of challenge”

Managing Comprehension • Readers may experience frustration when… − Text readability is 100 L+ above their Lexile level. • Readers may experience ease when… − Text readability is 50 -100 L below their Lexile level. • Readers may experience growth when… − Text readability is within their Lexile range. General Reading Recommendation: Targeted text range of 100 L below to 50 L above the student’s Lexile level Note: This range may vary based on text type, reading context and purpose, reading strategies and support, and reader motivation.

Finding targeted text on ecosystems: Books: Inside Ecosystems and Biomes Exploring Ecosystems with Max Axiom, Super Scientist Earth’s Ecosystems Populations and Ecosystems Biomes and Ecosystems The Eastern Forest The Deserts of the Southwest The Prairie Endangered Species Lexile Measure: 700 L 750 L IG 810 L 860 L 880 L 970 L 1010 L 1080 L 1140 L 1160 L

Time Reading and Reading Achievement Fifth-Grade Students Percentile Rank Minutes of Text Reading per Day Estimated Number of Words Read per Year 98 90 70 50 20 10 67. 3 33. 3 16. 9 9. 2 2. 4 1. 0 4, 733, 000 2, 357, 000 1, 168, 000 601, 000 134, 000 51, 000 from Anderson et al. , 1988, Table 3, N = 155. Reading material included books, magazines, and newspapers.

College and Career Readiness Skills Reading Demand of Newspapers • USA Today (1200 L) • Associated Press (1310 L) • Chicago Tribune (1310 L) • Wall Street Journal (1320 L) • Washington Post (1350 L) • NY Times (1380 L) • Reuters (1440 L)

College and Career Readiness Skills Reading Demand of International Newspapers • • • • The Egyptian Gazette (Egypt) 1440 L Oman Daily Observer (Oman) 1430 L Financial Times (Great Britain) 1430 L Straits Times (Singapore) 1410 L Gulf Times (Qatar) 1420 L China Daily (China) 1400 L France Daily (France) 1400 L The Moscow Times (Russia) 1400 L The Australian (Australia) 1390 L German Times (Germany) 1390 L Copenhagen Post (Denmark) 1390 L Irish Times (Ireland) 1380 L Santiago Times (Chile) 1380 L Jerusalem Post (Israel) 1370 L

FREE Lexile Tools www. lexile. com • Lexile Find a Book • Lexile Analyzer

www. Lexile. com

Find-a-Book

Select reading interest:

Search results:

Create a reading list:

Edit reading list to personalize:

Search for books:

Search results:

Link to public library

Link to public library

Tools - Lexile Analyzer

The Lexile Analyzer

The Lexile Analyzer Copy the article

The Lexile Analyzer Paste the text into the Word document

The Lexile Analyzer Remove titles, pictures, graphs, illustrations, captions, sub-headings, and bylines

The Lexile Analyzer Be sure to remove hyper -links

The Lexile Analyzer Be sure to save as ‘Plain Text’

Oasis § A student-centered personalized learning platform – Does not require teacher direction § Differentiated activities § Embedded assessment in the background § Web-enabled with minimal bandwidth requirement § Built on the principles of deliberate practice…

Demo Accounts for Copenhagen § 45 demo accounts have been created inside Oasis for the folks in Copenhagen. § All the usernames are: rasch# where the # is a number from 1 to 45 (eg. , rasch 15, rasch 42). § All the passwords are: jakob § Oasis can be accessed by browsing to: http: //www. alearningoasis. com and clicking on the green “launch” button. If anyone has issues logging in, they can contact Sean Hanlon – shanlon@lexile. com

Reader Ability Temperature

Three well researched constructs § Reader ability § Text Complexity § Comprehension

Reading is a process in which information from the text and the knowledge possessed by the reader act together to produce meaning. Anderson, R. C. , Hiebert, E. H. , Scott, J. A. , & Wilkinson, I. A. G. (1985) Becoming a nation of readers: The report of the Commission on Reading Urbana, IL: University of Illinois

An Equation Conceptual Comprehension = Reader Ability - Text Complexity Statistical Expected Raw Score = i e (RA – TC i ) 1 + e(RA – TC ) i RA = Reading Ability TC = Text Calibrations

The Lexile Framework for Reading § A causal model relating reader ability, text complexity, and comprehension. § Measures reader ability and text complexity, on a common scale—the Lexile scale. § Allows educators to forecast the level of success a reader is likely to experience with a particular text.

Eight Features of the Causal Model Relating Text Complexity, Reader Ability, Task Difficulty and Comprehension 1. The model is individual centered. The focus is on explaining variation within person over time. 2. In this framework the measurement mechanism is well specified and can be manipulated to produce predictable changes in measurement outcomes (e. g. percent correct). 3. Item parameters are supplied by substantive theory and, thus, person parameter estimates are generated without reference to or use of any data on other persons or populations. Therefore, effects of the examinee population have been completely eliminated from consideration in the estimation of person parameters for reader ability.

Eight Features of the Causal Model cont’d. 4. The quantitivity hypothesis can be experimentally tested by evaluating the trade-off property for the individual case. A change in the person parameter can be off-set or traded-off for a compensating change in text complexity to hold comprehension constant. The trade-off is not just about the algebra. 5. When uncertainty in item difficulties is too large to ignore, individual item difficulties may be a poor choice to use as calibration parameters in causal models. As an alternative we recommend, when feasible, averaging over individual item difficulties to produce “ensemble” means. For example empirical text complexities can be excellent dependent variables for testing causal theories.

Eight Features of the Causal Model cont’d. 6. Causal Rasch models are individual centered and are explanatory at both within-subject and between-subject levels. The attribute on which I differ from myself a decade ago is the same attribute on which I differ from my brother today. 7. When data fit a Rasch model, differences between person measures are objective. When data fit a causal Rasch model absolute person measures (reader abilities) are objective (i. e. independent of instrument). 8. Causal Rasch models make possible the construction of generally objective growth trajectories. Each trajectory can be completely separated from the instruments used in its construction and from the performance of any other persons, whatsoever.

Student 1528 May 2007 – April 2011 7 th Grade Male Hispanic Paid Lunch 347 Encounters 138, 695 Words 3, 342 Items 983 Minutes Text Demands for College and Career 1600 1400 Expected: 73. 5% Observed: 71. 7% 1200 1000 May 2016 (12 th Grade)

Theoretical versus Empirical Text Complexity for 719 Articles* Reliability = 0. 997 SEM = 12. 8 L r = 0. 968 r” = 0. 969 R 2” = 0. 938 RMSE” = 89. 6 L * Inclusion criteria: 50 encounters and 1, 000 items

How Temperature and Pressure Relate Under Constant Volume Temperature Volume Pressure 2000°K 20 Liters 20. 0 atm 1000°K 20 Liters 10. 0 atm 500°K 20 Liters 5. 0 atm 250°K 20 Liters 2. 5 atm 125°K 20 Liters 1. 25 atm

Comprehension Rates for Fixed Reader Ability Comprehension Rates for Readers of the Same Ability with Texts of Different Complexity or How Reader Ability and Comprehension Rate Relate Under Varying Text Complexity Reader Ability Text Complexity Text Titles Comprehension Rates 1000 L 500 L The Magic School Bus, Inside the Earth (Cole) 96% 1000 L 750 L The Martian Chronicles (Bradbury) 90% 1000 L The Reader’s Digest 75% 1000 L 1250 L The Call of the Wild (London) 50% 1000 L 1500 L On Equality Among Mankind (Rousseau) 25%

Comprehension Rates for Fixed Text Complexity Comprehension Rates for Readers of Different Ability with Texts of the Same Complexity or How Reader Ability and Comprehension Rate Relate Under Constant Text Complexity Reader Ability Classroom Textbook Comprehension Rates 500 L 750 L 1000 L 1250 L 1500 L 1000 L 1000 L 25% 50% 75% 90% 96%

How Many Ways Can We Say X Causes Y? X “elicited a greater” Y X “impacts” Y X “accounts for” Y X “has been linked to” Y Y “is the result of” X X “didn’t diminish” Y Y “because of” X Y “depends on” X X “has led to” Y X “largely motivates” Y Y “stemmed from” X X “proved critical to” Y X “fosters” Y X “changes” Y X “triggers” Y X “affects” Y

To causally explain a phenomenon [a measurement outcome] is to provide information about the factors [person processes and instrument mechanisms] on which it depends and to exhibit how it depends on those factors. This is exactly what the provision of counterfactual information…accomplishes: we see what factors some explanandum M [measurement outcome, raw score] depends on (and how it depends on those factors) when we have identified one or more variables such that changes in these (when produced by interventions) are associated with changes in M (Woodward, 2003, p. 204).

Ensemble Calibration The Psychometric Engine of Oasis

Oasis Items § Single Instance § No individual item statistics § Theoretical estimates of text complexity become empirical estimates of text complexity § Protocol for determining response illustrations and foils § Machine-generated with no human review

Item Distributions § Items are presumed to be sampled from a distribution with a mean difficulty and an imputed spread § This distribution is called an ensemble

Traditional Rasch Algorithm § Iterative process § Two stages of iteration – Begin with asssumed starting values for person abilities and item difficulties (usually item difficulties) – Estimate person abilities given item difficulties. – Fix these estimated person abilities and estimate new item difficulties given these person abilities – Iterate until a stopping criterion is met

Ensemble Calibration Algorithm § Same iterative technique as traditional method – Fix person abilities to determine ensemble means – Fix resulting ensemble means to determine person abilities – Iterate until stopping criterion is met

What is Different? § Individual item difficulties neither available nor needed § Start iterative procedure with person abilities from Oasis § P-delta-sigma table to account for spread of distributions § Spread of ensemble effects success rate

P-delta-sigma table § A success rate (p) is determined by the difference in the ensemble mean for a distribution and the person ability (delta) and the spread of the ensemble (sigma) § Normal distribution assumption for the ensemble of item difficulties § Table used instead of integration over the random term for programming simplicity – Dimensions: (delta -1000 to +1000 by fives, Sigma 10 to 260 by fives)

Formula for p-delta-sigma = Δ=delta=difference between person ability and ensemble calibration σ=sigma=standard deviation of the ensemble u=variable of integration=random difference of item difficulty from ensemble mean

Spread of Ensemble § Ensembles vary in mean but not in SD § SD is uniform for this implementation § With embellishment a method for unequal variances could be derived

Closing No matter how it is sliced and diced, analyses of joint and conditional probability distributions yield no more than patterns of association. Nothing in the response data nor Rasch analyses of these data exposes the processes (features of the object of measurement) or mechanisms (features of the instrument) that are hypothesized to be conjointly causal on the measurement outcomes.

Contact Info: A. Jackson Stenner CEO, Meta. Metrics jstenner@Lexile. com