Chapter 6 Formal Assessment of Reading Individualized Assessment

  • Slides: 66
Download presentation
Chapter 6 Formal Assessment of Reading: Individualized Assessment

Chapter 6 Formal Assessment of Reading: Individualized Assessment

Introduction • Development of Formal Reading Measures Types of Scores Administration and Scoring of

Introduction • Development of Formal Reading Measures Types of Scores Administration and Scoring of Formal Tests Determining Chronological Age Individualized vs Group Tests What do individualized Norm-referenced Measures of Reading Look Like? • Test Bias • Selecting Formal, Individualized Instruments • Special Considerations for Formal, Individual Assessment of Adult and English Language Learners • • •

Test Development • • • Table of specifications (blueprint) (Figure 5. 2) Development version

Test Development • • • Table of specifications (blueprint) (Figure 5. 2) Development version Pilot or field testing Standardization version Norm sample • Representativeness of sample

Figure 6. 1

Figure 6. 1

Types of Scores

Types of Scores

 • Numerical Scales: Nominal, Ordinal, Interval, Ratio • Raw Score: Actual number correct,

• Numerical Scales: Nominal, Ordinal, Interval, Ratio • Raw Score: Actual number correct, taking into account the basal • Derived Scores • Standard Scores: Z Scores, IQ Scores, T Scores, Scale Scores • Grade Equivalents, Age Equivalents • Percentiles

Assessment Terms • Grade Equivalent • Standard Scorescan Add and • Age Equivalent Subtract-Used

Assessment Terms • Grade Equivalent • Standard Scorescan Add and • Age Equivalent Subtract-Used for • Grade-Based Comparisons Norms • Percentiles • Age-Based Norms Provides A Rank ØAs a Rule, Use Age. Order 1 -99 Based Norms to • Stanines-A Rough Determine Gauge 1 -9 Standard Scores

Text Box 6. 1 Age and Grade Equivalents Age Equivalent 6 -1 6 years,

Text Box 6. 1 Age and Grade Equivalents Age Equivalent 6 -1 6 years, 1 month Grade Equivalent 1. 2 1 st grade, 2 nd month

Text Box 6. 2 Percentile versus Percentage Mastery Percentile Misha 50 Jesse 90 16

Text Box 6. 2 Percentile versus Percentage Mastery Percentile Misha 50 Jesse 90 16 80

Text Box 6. 3 Contrasting Standard Scores for Word Recognition and Reading Comprehension Mean

Text Box 6. 3 Contrasting Standard Scores for Word Recognition and Reading Comprehension Mean = 100; Standard Deviation = 15 Word Recognition Shelby Saimah 85 95 Reading Comprehension 84 70

Text Box 6. 4 Comparing IQ Scores with Reading Achievement Scores (Mean = 100;

Text Box 6. 4 Comparing IQ Scores with Reading Achievement Scores (Mean = 100; Standard Deviation = 15) Saimah Standard Score IQ 94 Reading Comprehension -70 24 Shelby 103 -85 18

Administering and Scoring of Formal Tests

Administering and Scoring of Formal Tests

Test Administration • Test Manual: Administration Instructions, Scoring and Norm Information, Reliability and Validity

Test Administration • Test Manual: Administration Instructions, Scoring and Norm Information, Reliability and Validity Data • Protocol • Calculation of Chronological Age • General Guidelines for Test Administration • Raw Scores • Establishing Basals and Ceilings • Derived Scores: Percentiles, Grade Equivalents, Standard Scores

Test Administration • Determine • Do NOT coach, chronological age prompt, or give feedback

Test Administration • Determine • Do NOT coach, chronological age prompt, or give feedback EXCEPT • Know start and as directed in the stop rules manual • Establish rapport • Obtain raw scoreswith examinee take into account • Administer subtests basal and ceiling according to • Obtain derived directions scores (standard scores, percentiles, etc.

Tips for Administering Standardized Tests • Text Box 6. 5

Tips for Administering Standardized Tests • Text Box 6. 5

Figure 6. 2. Sample Reading Subtests Showing Basal and Ceiling Rules Word Recognition Basal

Figure 6. 2. Sample Reading Subtests Showing Basal and Ceiling Rules Word Recognition Basal Rule: 3 in a row correct. Ceiling rule: 3 in row incorrect. 1. ___ 2. ___ 3. ___ → 4. _1_ 5. _1_ 6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _0_ 11. _0_ 12. _0_ Note: Starting point for this student indicated by the → Vocabulary Basal rule: All items in a set correct. Ceiling rule: All items in a set incorrect. 1. ___ 2. ___ 3. ___ 4. ___ 5. ___ → 6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _1_ 11. _0_ 12. _0_ 13. _0_ 14. _0_ 15. _0_ Note: Starting point for this student indicated by the →

Figure 6. 3. Sample Excerpt from a Norm Table for an Individually Administered Reading

Figure 6. 3. Sample Excerpt from a Norm Table for an Individually Administered Reading Comprehension Test for Students Age 8 years, 9 months. Examiners calculate the raw score for the test, then look up derived scores (standard scores, percentiles, grade- and age-equivalents) based on the examinee’s chronological age. Raw Score Conversion Table for Reading Comprehension Test Age: 8 years, 9 months Raw Stand Percen Grade Age Score ard tile Equivale Score lent nt ↑ 15 95 37 2. 1 7 -6 16 96 39 2. 4 7 -9 17 97 42 2. 7 7 -11 18 98 45 2. 9 8 -3 19 99 47 3. 2 8 -6 20 100 50 3. 5 8 -9 ↓

Determining Chronological Age

Determining Chronological Age

Calculation of Chronological Age Subtract Date of Birth from Current Date: 2016 06 22

Calculation of Chronological Age Subtract Date of Birth from Current Date: 2016 06 22 - 2009 07 14 Months always have 30 days and years (of course) have 12 months). Always double check your work.

Standardized Diagnostic Testing • When to Use Diagnostic Tests • Instructional Level • Informal

Standardized Diagnostic Testing • When to Use Diagnostic Tests • Instructional Level • Informal Instruments • Probes • Direct Measurement • Domains

Individualized vs. Group Tests

Individualized vs. Group Tests

Individual versus Group? Individual Group • Special education eligibility • Determine goals and objectives

Individual versus Group? Individual Group • Special education eligibility • Determine goals and objectives for IFSPs, IEPs • Progress monitoring • May be more reliable for certain students • Determine mastery of curriculum standards • Determine if teachers are effective (value added) • Controversial but happening! • Accountability and general school planning • May be less reliable for certain students

Test Interpretation • Analyze test scores • Is there a pattern of • Are

Test Interpretation • Analyze test scores • Is there a pattern of • Are scores above strengths and average, average or weaknesses? below average? • e. g. Areas of learning • Do the scores yield any disability? basic reading, reading fluency, reading instructional comprehension? information? • Compile with other data • Maintain confidentiality

What do Individualized, Norm. Referenced Measures of Reading Look Like?

What do Individualized, Norm. Referenced Measures of Reading Look Like?

Figure 6. 4

Figure 6. 4

Figure 6. 5

Figure 6. 5

Figure 6. 6

Figure 6. 6

Figure 6. 7

Figure 6. 7

Figure 6. 8

Figure 6. 8

Figure 6. 9

Figure 6. 9

Methods of Assessing Comprehension • KTEA-III • Reading Comprehension subtest • Picture identification, Answering

Methods of Assessing Comprehension • KTEA-III • Reading Comprehension subtest • Picture identification, Answering questions • WIAT-III • Reading Comprehension subtest • Picture identification, Answering questions • WJ-IV • Passage Comprehension subtest • Picture identification, Fill in the blank • Reading Recall subtest • Silent reading followed by oral retelling

Comprehension • Wechsler Individual Achievement Test (WIAT-III) • Reading Comprehension subtest • The examinee

Comprehension • Wechsler Individual Achievement Test (WIAT-III) • Reading Comprehension subtest • The examinee is required to read short sentences or passages and respond to comprehension questions

Reading Comprehension Similar to WIAT-III

Reading Comprehension Similar to WIAT-III

Sample Maze Activity Norm-referenced tests are standardized (without/on/for) a clearly defined group, termed the

Sample Maze Activity Norm-referenced tests are standardized (without/on/for) a clearly defined group, termed the (test/sample/norm) group, and scaled so that each (score/instrument/variance) reflects a rank within the (evaluation/reliability/norm) group. Psychologists have developed norm-(standardized/referenced/rated) tests to assess, for example, (height/attitude/intelligence), reading, mathematics, writing, etc. Although we (aren’t/are/is) fortunate in having a choice of (poorly/haphazardly/well)-standardized and psychometrically sound tests (without/above/with) which to evaluate children, some (numbers/tests/practitioners) do not meet acceptable psychometric standards. (Passage from Sattler, 2001)

Nelson-Denny Reading Test • Comprehension • 7 reading passages • 38 comprehension questions •

Nelson-Denny Reading Test • Comprehension • 7 reading passages • 38 comprehension questions • Literal and interpretive question types • 5 answer choices per question • 20 minutes

Methods of Assessing Vocabulary • WJ-IV: Reading Vocabulary • Synonyms, Antonyms, Analogies • Nelson

Methods of Assessing Vocabulary • WJ-IV: Reading Vocabulary • Synonyms, Antonyms, Analogies • Nelson Denny • Multiple choice

Vocabulary • Woodcock Johnson - IV Tests of Achievement (WJ-IV) • Reading Vocabulary subtest

Vocabulary • Woodcock Johnson - IV Tests of Achievement (WJ-IV) • Reading Vocabulary subtest • The examinee is required to orally state synonyms and antonyms for printed words and orally complete written analogies

Reading Vocabulary Similar to WJ-IV Analogies Antonyms Synonyms

Reading Vocabulary Similar to WJ-IV Analogies Antonyms Synonyms

Nelson-Denny Reading Test • Vocabulary • 80 items • 5 answer choices each •

Nelson-Denny Reading Test • Vocabulary • 80 items • 5 answer choices each • 15 minute time limit • Example • Which word best completes the opening statement? • A chef works with: A. bricks B. music C. clothes D. food E. statues

Methods of Assessing Fluency • KTEA-III • Decoding Fluency subtest • Nonsense word list

Methods of Assessing Fluency • KTEA-III • Decoding Fluency subtest • Nonsense word list (timed) • Word Recognition Fluency subtest • Word list (timed) • WIAT-III • Reading Comprehension subtest • Reading timed passages • WJ-IV • Sentence Reading Fluency subtest • Reading sentences and marking whether statements are true or false (3 minute time limit) • Word Reading Fluency • Choosing two words that go together from an array of choices

Fluency • The Test of Silent Word Reading Fluency-2 (TOSWRF-2) • Examinees are required

Fluency • The Test of Silent Word Reading Fluency-2 (TOSWRF-2) • Examinees are required to identify increasingly difficult words that have no spaces between them by drawing lines between the boundaries of as many words as possible within the time limit (3 minutes) • The Test of Silent Contextual Word Reading Fluency (TOSCRF-2) • Similar to TOSWRF-2 but examinee reads connected text of increasingly difficult graded passages

TOSWRF-2: Example

TOSWRF-2: Example

Nelson-Denny Reading Test • Reading Rate • Part of Comprehension Test • The examiner

Nelson-Denny Reading Test • Reading Rate • Part of Comprehension Test • The examiner calls “Mark” after one minute has elapsed, and the examinees are asked to record the number to the right of their current line of text and continue reading.

Methods of Assessing Phonics • KTEA-III • Nonsense Word Decoding subtest • Nonsense words

Methods of Assessing Phonics • KTEA-III • Nonsense Word Decoding subtest • Nonsense words • WJ-IV • Word Attack subtest • Nonsense words • WIAT-III • Pseudoword Decoding subtest • Nonsense words

Phonics • Wechsler Individual Achievement Test (WIAT-III) • Pseudoword Decoding subtest • Requires the

Phonics • Wechsler Individual Achievement Test (WIAT-III) • Pseudoword Decoding subtest • Requires the examinee to read aloud a list of nonsense words designed to mimic the phonetic structure of words in the English Language.

Pseudoword Decoding Similar to WIAT-III • • heb mib fum bim pon vun dreeb

Pseudoword Decoding Similar to WIAT-III • • heb mib fum bim pon vun dreeb

Methods of Assessing Phonemic Awareness • KTEA-III • Phonological Awareness subtest • Rhyming, Sound

Methods of Assessing Phonemic Awareness • KTEA-III • Phonological Awareness subtest • Rhyming, Sound Matching, Blending, Segmenting, Deletion • WJ-IV • Spelling of Sounds subtest • Rhyming, Deletion, Substitution, Reversal • WIAT-III • Word Reading subtest • Rhyming, Sound Matching, Blending,

Phonemic Awareness • Woodcock Johnson Tests of Achievement (WJ IV) • Spelling of Sounds

Phonemic Awareness • Woodcock Johnson Tests of Achievement (WJ IV) • Spelling of Sounds subtest • Requires the examinee to use rhyming, deletion, substitution, and reversal to manipulate sounds in words.

Spelling of Sounds Similar to WJ-IV • Rhyming: “What rhymes with go? ” •

Spelling of Sounds Similar to WJ-IV • Rhyming: “What rhymes with go? ” • Deletion: “Say cowboy without saying boy” • Substitution: “Change /s/ in sun to /f/. ” • Reversal: “Listen to the sounds in the word pot - /p/ /o/ /t/. Now you say the sounds backward. ”

Other Measures that Assess Phonemic Awareness • CTOPP-2 • Durrell-2 • ERDA-2 • PAL-2

Other Measures that Assess Phonemic Awareness • CTOPP-2 • Durrell-2 • ERDA-2 • PAL-2 RW • TOWRE-2 • TERA-3

Methods of Assessing Sight Word Recognition • WIAT-II • Word Reading subtest • Letter

Methods of Assessing Sight Word Recognition • WIAT-II • Word Reading subtest • Letter and word identification • KTEA-II • Letter and Word Recognition subtest • Letter and word identification • WJ-IV • Letter Word Identification subtest • Letter and word identification

Sight Word Recognition • Kaufman Tests of Educational Achievement (KTEA III) • Letter and

Sight Word Recognition • Kaufman Tests of Educational Achievement (KTEA III) • Letter and Word Recognition subtest • The examinee is asked to point to or name various letters, sounds, and words.

Letter Word Recognition Similar to KTEA III

Letter Word Recognition Similar to KTEA III

Test Bias • Scores from a test should not provide a systematic advantage to

Test Bias • Scores from a test should not provide a systematic advantage to a particular group or subgroup based on irregularities in test content, administration, or interpretation • If a test produces such an unfair advantage it is said to be biased against the group that is disadvantaged (Payne, 2003)

Evidence of Test Bias • a) Does the test assess the same content for

Evidence of Test Bias • a) Does the test assess the same content for all the groups using it? • (b) Are the constructs assessed by the test measured consistently for all groups using it? • (c) Does the test predict (later achievement) as well for all the groups using it? • (d) Does the test provide equally useful instructional information for all groups using it?

Avoiding Bias • Expert judgment • Statistical analyses: Are the constructs assessed the same

Avoiding Bias • Expert judgment • Statistical analyses: Are the constructs assessed the same across different groups (e. g. , gender, race, SES level? ) • Remember tests that are not inherently biased may be misused (e. g. , use of test in English for students with limited English skills)

Selecting Formal, Individualized Instruments • Consider test characteristics • Consider psychometric properties • Consider

Selecting Formal, Individualized Instruments • Consider test characteristics • Consider psychometric properties • Consider needs of children, classrooms, schools, or systems • Determine whether a group or individualized test is needed

Special Considerations for Formal, Individual Assessment of Adult and English Language Learners

Special Considerations for Formal, Individual Assessment of Adult and English Language Learners

Tips for Formal, Individualized Assessment of Adult Learners • Text Box 6. 7 •

Tips for Formal, Individualized Assessment of Adult Learners • Text Box 6. 7 • Assess adult learners’ educational histories, background experiences, and interests as well as specific reading skills. • Multiple assessments will likely be needed to provide complete information about adult learners’ reading strengths and weaknesses. • Ensure that the test used with adults is standardized on that population. • Put adult learners at ease; establish rapport and a purpose before launching into formal testing. • Using grade equivalent scores, and to a lesser extent age equivalent scores, is problematic due to ease of misinterpretation of these score, and their psychometric limitations.

Tips for Adults cont’d • Tests should include item content appropriate for adults, rather

Tips for Adults cont’d • Tests should include item content appropriate for adults, rather than young students, even though the difficulty level of the content may be low in order to achieve a reasonable test floor. • Consider entering your test data and using the instructional skills profile available at the Adult Reading Components Study website: https: //lincs. ed. gov/readingprofiles/resources. ht m

Tips for Formal, Individualized Assessment of English Language Learners • Text Box 6. 8

Tips for Formal, Individualized Assessment of English Language Learners • Text Box 6. 8 • Users should follow standards set for test administration, scoring and interpretation in general and for ELL and other diverse learners as developed by experts. Many of these standards are reproduced in Chapter 6; some of the most relevant are reproduced in this Text Box in abbreviated form. • Be aware that an ELL student’s formal test scores will be compared to students in the U. S. , who may not resemble the ELL in background, culture, knowledge, etc. • Related to Number 2 above, an ELL’s scores on formal tests may be depressed because of cultural, linguistic, and knowledge differences.

Tips for ELLs cont’d • Do not administer a language-loaded test for ELL examinees

Tips for ELLs cont’d • Do not administer a language-loaded test for ELL examinees unless the language content in the test is the focus of assessment. • Related to Number 4 above, if a student’s English language proficiency is the target of assessment English languageloaded tests (i. e. , English reading tests) are appropriate • Ensure that the test does not contain item content that may be offensive in the ELL’s home country or culture. • Ensure that reliability and validity of the test have been established for ELLs. • Ensure that the items are not biased against ELL learners by evaluating recommendations from expert panels.

Tips for ELLs cont’d • Ensure that items are not biased against ELL learners

Tips for ELLs cont’d • Ensure that items are not biased against ELL learners by examining evidence from statistical techniques designed to determine bias. • When possible, use an examiner who can communicate in the ELL’s native language to establish rapport and aid in administering the test. • Be aware that an ELL’s progress in learning a second language and ability to understand test demands may be limited by negative transfer from the native language. • Reduce importance of speeded performance on scores; some cultures do not value speed. • Ensure that the test only measures what you intend to measure (i. e. , specific types of reading skills) and not some irrelevant content.

Tips for ELLs cont’d • If your ELL students score below the mean, ensure

Tips for ELLs cont’d • If your ELL students score below the mean, ensure that if mean differences occur, the differences are not the result of bias. • Use ample teaching items to ensure that ELL learners understand the task demands. • Using grade equivalent scores, and to a lesser extent age equivalent scores, is problematic due to each of misinterpretation of these score, and psychometric limitations.

Assessment at a Glance: Formal, Individualized Assessment • Tables 6. 1 and 6. 2

Assessment at a Glance: Formal, Individualized Assessment • Tables 6. 1 and 6. 2 • Characteristics of Formal, Individualized, Norm. Referenced Assessments of Reading • Psychometric Properties of Formal, Individualized, Norm-Referenced Assessments of Reading

Summary • Development of Formal Reading Measures Types of Scores Administration and Scoring of

Summary • Development of Formal Reading Measures Types of Scores Administration and Scoring of Formal Tests Determining Chronological Age Individualized vs Group Tests What do individualized Norm-referenced Measures of Reading Look Like? • Test Bias • Selecting Formal, Individualized Instruments • Special Considerations for Formal, Individual Assessment of Adult and English Language Learners • • •