Determining Academic Functioning Use of Standardized Tests Leah

Determining Academic Functioning: Use of Standardized Tests Leah Greenwood, Ph. D. , ABPP Diplomate, American Board Of Rehabilitation Psychology The 17 th Annual PA and NJ IARP Conference – June 22 -24, 2016

Overview/Objectives Define key academic skill domains, their cognitive components, and specific disorders. Identify purposes for academic skills testing in educational, clinical, and vocational contexts. Identify types of currently used standardized tests and their strengths and weaknesses. Review best practices for interpreting and reporting test results. Highlight validity concerns when evaluating academic skills in diverse populations.

Why measure academic skills? Program admission Grade advancement Eligibility for services Contexts for academic skills testing Forensic Diagnosis Treatment/Educational planning Vocational – Career/Personnel Research Pre-morbid IQ Job Placement School or job accommodations Career planning Clinical – DSM-5, ICD-10 Educational – NCLB, IDEA, ADA

Types of Standardized Academic Skills Tests U. S. Department of Education Admissions Testing/college, graduate school Individually administered academic Skills Tests Personnel Tests/Job Placement/Career Planning • State-wide, group administered, grades 3 -8, benchmarks • Pennsylvania System of School Assessment (PSSA) • Partnership for Assessment of Readiness for College & Careers (PARCC) – 2014 -15 • Scholastic Aptitude Test (SAT); predictive • ACT, GRE, MCAT, LSAT • Subject matter as well as common academic core; predictive • Brief Screening Tests; Comprehensive Batteries; Domain specific tests • Underachieving students; primary, secondary, postsecondary • Tests used in occupational selection • Multiple Aptitude Batteries (The General Aptitude Test Battery)

Key Academic Skills (DSM-5, 2015) Reading • Reading of single words accurately • Read single words fluently • Reading comprehension Writing • Spelling accuracy • Grammar and punctuation accuracy • Clarity or organization of written expression Mathematics • Number sense • Memorization of arithmetic facts • Accurate or fluent calculation • Accurate mathematical reasoning Observable functional outcomes of underlying neurocognitive processes. Require more than brain maturation, they must be taught and learned explicitly.

Reading Skills The average adult is capable of reading approximately 250 words per minute, or approximately 4 words per second. In the English language, there are more than 1, 100 ways of representing 44 sounds (phonemes) using a series of different letter combinations. Reading skills require direct instruction in 5 areas: • • • Phonics (letter-sound correspondences) Word Identification (syllables, retrieval) Fluency (reading speed and accuracy) Vocabulary (lexicon of known words) Comprehension skills (deriving meaning from print, semantic knowledge, syntactical analysis) Multiple brain regions involved in reading: The supramarginal gyrus, at the juncture of the temporal and parietal lobes responsible for phonological processing The fusiform gyrus key region for automatically recognizing words Anterior cingulate gyrus responsible for organizing incoming information with previously read material General processes: visual processing, auditory processing, working memory, executive skills (organizing, sequencing)

Mathematics Skills Developmental sequence of mathematical abilities with direct instruction: Preschool: Less is understood about neurocognitive substrates of specific mathematical abilities Concepts of more and less, conservation, addition and subtraction occurring within events, rudimentary counting Underlying cognitive abilities that are required: Kindergarten: Number sense, counting knowledge Elementary grades: Mastery of number facts (addition, subtraction), symbol code of integers, Advanced grades: Mastery of computational algorithms, mathematical theory, and applied problem solving • Visual perception skills • Visual motor skills • Sequential reasoning • Verbal comprehension • Abstract concept formation

Writing Skills Writing requires the linking of language, thought and motor skills. A writer must employ and integrate many diverse abilities to write legibly, spell, and translate thoughts into writing. Six separate components of writing learned through direct instruction: • Handwriting quality • Writing fluency (number of words copied within time constraints) • Spelling from dictation • Spelling accuracy within composition • Compositional fluency (number of words produced within time constraints • Computational quality (content, cohesiveness, and organization of written narrative material Neuropsychological components of writing involve: • • • Fine motor skills Visual spatial integration Visual Attention Memory Executive skills (sequencing, self-monitoring, planning and organizational skills)

Academic Skills Testing in Educational Contexts: Elementary and secondary grades (ages 6 -21) State/District wide achievement testing • Grade-wide tests of student achievement • Graduation requirements • Determine benchmarks of proficiency Eligibility for Special Education Services • Individuals with Disabilities Act (IDEA, 2004); • Governs how states identify SLD, and provide interventions • Ensures a free and appropriate public education (FAPE) Individualized Education Plan • Creating individualized educational plans (IEP) • Psychoeducational batteries (school psychologists) • Repeatable testing for tracking progress

Academic Skills Testing in Educational Contexts: Postsecondary Level Skills testing to meet prerequisite standards for college admission Eligibility for Academic Accommodations in college Support for specific academic Accommodations • Admissions testing (SAT, ACT) • Basic skills assessment to place out of general education prerequisites • Identification of disabilities under the ADA; guarantee equal access to programs and services • Use of several types of classification systems (DSM-5, ICD-10) • Specific university standards for assessment of SLD • Weaknesses must substantially interfere with a major life activity • Accommodations must be supported by test results

Diagnostic and Statistical Manual of Mental Disorders, 5 th Ed. (DSM-5, 2013) Specific Learning Disorder, with specifiers International Statistical Classification of Diseases and Related Health Problems – ICD-10, World Health Organization. (U. S. healthcare system – 2015) 315. 00 (F 81. 0) With Impairment in Reading (inc. dyslexia) • Word Reading Accuracy • Reading rate or fluency • Reading Comprehension 315. 2 (F 81. 81)With Impairment in written expression • Spelling accuracy • Grammar and punctuation accuracy • Clarity or organization of written expression 315. 1(F 81. 2) With Impairment in mathematics (inc. dyscalculia) • • Number sense Memorization of arithmetic facts Accurate or fluent calculation Accurate math reasoning

DSM-5 Classification Criteria for Specific Learning Disorder Diagnostic Criteria A. Difficulties learning and using academic skills, as indicated by the presence of at least one of the following symptoms that have persisted for at least 6 months, despite the provision of interventions that target those difficulties. B. The affected academic areas are substantially and quantifiably below those expected for the individual’s chronological age, and cause significant interference with academic or occupational performance, or with activities of daily living. C. The learning difficulties begin during school-age years but may not become fully manifest until the demands for those affected academic skills exceed the individual’s limited capacities. D. The learning difficulties are not better accounted for by intellectual disabilities, uncorrected visual or auditory acuity, other mental or neurological disorders, psychosocial adversity, lack of proficiency in the language of academic instruction, or inadequate educational instruction.

Academic Skills Testing in Clinical Rehabilitation Contexts • Estimation of pre-morbid functioning Part of a comprehensive • New onset due to neurological insults NP assessment • DSM-5 • Remediation of skills Cognitive rehabilitation • Compensatory strategies treatment planning • Obtaining testing accommodations Work re-entry/vocational • Accommodations in school/work settings • Identification of adaptive aids (audiotaped books, calculators, rehabilitation plans open book, spell-check, dictation systems, etc. )

Academic Skills Testing in Vocational Contexts Career Planning Employment Rehabilitation/Forensic • Work-related multi-aptitude measures – capacity for learning • DOL, Occupational Information Network (O*NET), Career Exploration Tools • Achievement measures – proficiency • Hiring, training, job performance, advancement • Neuropsychological evaluations, eligibility for services • Job accommodations • Assessing employability, earning capacity

Testing Instruments used in Educational, Clinical, and Vocational Contexts Individual Screening Batteries • Kaufman Test of Educational Achievement (KTEA-3) -Brief Form • Wide Range Achievement Test-4 (WRAT-4) • Academic Achievement Battery (AAB)- Screening Individual Comprehensive Batteries • Woodcock Johnson IV –Tests of Achievement • Wechsler Individual Achievement Test-III (WIAT-III) • Kaufman Test of Educational Achievement (KTEA-3) -Comprehensive • Diagnostic Achievement Battery 4 th ed. (DAB-4) Specific Domains Tests • Test of Word Reading Efficiency-2 nd Ed. (TOWRE-2) • Diagnostic Assessments of Reading-2 nd (DAR-2) • Test of Written Language-4 th ed. (TOWL-4) • Test of Mathematical Abilities-3 rd (TOMA-3) Vocational Multi-aptitude Batteries • O*Net Ability Profiler, General Aptitude Test Battery (GATB) • Armed Services Vocational Aptitude Battery (ASVAB) • Career Ability Placement Survey (CAPS) • Employee Aptitude Survey (EAS)

Qualification Levels – Pearson Qualification A Level • There are no special qualifications to purchase these products. Qualification B Level • Certification by or full active membership in a professional organzation (ASHA, AOTA, APA, AERA, ACA, AMA, NASP, NAN, INS, CEC, AEA, AAA, EAA) that requires training and experience in a relevant area of assessment. • A master's degree in psychology, education, occupational therapy, speech-language pathology, social work, or in a field closely related to the intended use of the assessment, and formal training in the ethical administration, scoring, andinterpretation of clinical assessments. Qualification C Level • Licensure or certification to practice in your state in a field related to the purchase. • A doctorate degree in psychology, education, or closely related field with formal training in the ethical administration, scoring, and interpretation of clinical assessments related to the intended use of the assessment. Qualification Q Level • Q 1: A degree or license to practice in the healthcare or allied healthcare field. • Q 2: Formal supervised mental health, speech/language, and/or educational training specific to working with parents and assessing children, or formal supervised training in infant and child development, and formal training in the ethical use, administration, and interpretation of standardized assessment tools and psychometrics.

WIAT®-III (Wechsler Individual Achievement Test ®- Third Edition) Ages 4: 0 - 50: 11 years Administration: Paper-and-pencil or online administration Completion Time: Varies by grade level and number of subtests administered Scores/Interpretation: Fall, Winter, and Spring grade-based standard scores, age-based standard scores, percentile ranks, stanines, NCEs, age and grade equivalents. The WIAT-III was nationally standardized on 3, 000 students and adults and features comprehensive normative information. Measure all eight areas of achievement specified by IDEA legislation as important for identifying and classifying learning disabilities Basic Reading Word Reading Pseudoword Decoding Mathematics Math Problem Solving Numerical Operations Math Fluency Oral Language Listening Comprehension Oral Expression Reading Comprehension & Fluency Reading Comprehension Oral Reading Fluency Written Expression Alphabet Writing Fluency Spelling Essay Composition Subtest Sentence Composition Subtest Q-Global - interactive

Feifer Assessment of Reading™ (FAR™) Steven G. Feifer, DEd; Professional Manual by Steven G. Feifer, DEd, and Rebecca Gerhardstein Nader, Ph. D Purpose: Examine the underlying cognitive and linguistic processes that support proficient reading skills Age range: 4 to 21 years Admin: Individual Admin time: 35 minutes for PK; 60 minutes for K-Grade 1; 75 minutes for Grade 2+; 15 -20 minutes for Screening Form Based on the premise that treatments for reading disorders vary by dyslexic subtype, with four specific subtypes addressed: dysphonetic dyslexia, surface dyslexia, mixed dyslexia, and reading comprehension deficits.

Wide Range Achievement Test 4 (WRAT 4) Gary S. Wilkinson, Ph. D, Gary J. Robertson, Ph. D Overview: The WRAT 4 measures the basic academic skills of reading, spelling, and math Qualification Level: B Age Range: Individuals 5 - 94 RTI Tiers: RTI Levels 1, 2 Administration: Paper and pencil Completion Time: Approximately 15 -25 minutes for ages 5 – 7; approximately 35 -45 minutes for ages 8 and up Scoring Options: Hand scoring, computer scoring Reading comprehension evaluation. The WRAT 4 improvements include the addition of a Sentence Comprehension subtest. Extension of norms. To allow for standardized assessment of older adults' literacy skills, grade-based norms increase the usefulness of the test in Grades K 12, and age-based norms extend to age 94 years. Parallel forms make retesting easy. The Blue and Green forms can be used interchangeably with comparable results, permitting retesting within short periods of time. Alternate forms can also be combined into a single examination for a more comprehensive academic skills assessment. Flexible. Use with individuals or small groups to identify those requiring a more comprehensive academic achievement evaluation. Reliable. Standardized on a national sample of over 3, 000 individuals ranging in age from 5 to 94, the WRAT 4 has been shown to be highly reliable.

Academic Achievement Battery™ (AAB™) Comprehensive Form Melissa A. Messer, MHS Purpose: Assesses seven areas of achievement throughout the life span Age range: 4 to 85 years Admin: Individual Admin time: Comprehensive Form: 90 minutes Screening Form: 30 to 45 minutes The AAB composite scores are Basic Reading, Reading Comprehension, Listening Comprehension, Expressive Communication, Written Expression, Mathematical Calculation, Mathematical Reasoning, as well as a total composite—the Academic Achievement Battery Composite. Discrepancy analysis for the AAB will be available for the Reynolds Intellectual Assessment Scale– 2 (RIAS 2); and is currently available for the Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV; 2003) and the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; 2008). The AAB requires only minor training for examiners, and detailed or subjective scoring is kept to a minimum. All scoring criteria can be found in the AAB Item Booklet, allowing for scoring during administration. Overall scoring takes 15 to 20 minutes.

The O*NET Ability Profiler https: //www. onetcenter. org/AP. html The O*NET Ability Profiler measures nine job-relevant abilities: o o o o o Verbal Ability Arithmetic Reasoning Computation Spatial Ability Form Perception Clerical Perception Motor Coordination Finger Dexterity Manual Dexterity The O*NET Ability Profiler must be administered by staff who provide instructions to individuals taking the assessment It can be administered in individual or group settings Computerized scoring Results from the O*NET Ability Profiler: • Are presented on computer-generated customized score reports • Can be linked to the over 800 occupations in O*NET On. Line • Are easily interpreted • Can be used on a stand-alone basis or with other O*NET Career Exploration Tools or with privately developed instruments

The CAPS Web-based immediate scoring Range: Grades 6 -12, college and adult Norms: Middle, high school and college Administration Time: 5 Minutes for each of the eight tests 50 minutes to complete the entire battery with the CAPS Administration CD or online Language(s): English and Spanish versions available Scoring: self-scoring (15 -20 minutes), web-based immediate scoring The Career Ability Placement Survey (CAPS) is a comprehensive, multi-dimensional battery designed to measure 8 vocationally relevant abilities: Mechanical reasoning; spatial relations; verbal reasoning, numerical ability; language usage; word knowledge; perceptual speed and accuracy; manual speed and dexterity Each of the eight ability dimensions is keyed to entry requirements for the majority of occupations in each of the 14 COPSystem Career Clusters. o Help students develop abilities and plan for the future o Use with adults in vocational training programs o Place employees to maximize talent and well-being

Test Score Interpretation Norm-Referenced Age Equivalent Grade Equivalent Percentiles Standard Scores Stanines Criterion-Referenced • Average performance of the normative age group • Behavioral attainment unequal across age groups • Average performance of the normative grade group • Content/quality of instruction varies Knowledge of Content Domains • Measures of achievement • Percentages of correct answers Performance Testing • Mastery testing • Pass/fail or continuum of performance • Percentage of persons in the normative group scoring at or below a certain raw score • Position of score relative to the mean in standard deviation units • Usually age-based, education based • 9 point scale, mean of 5, SD=2

Applicability of a Normative Sample v v v v How large is the normative sample? When was the sample gathered? Where was the sample gathered? How were the individuals identified and selected for the sample? Who tested the sample? How did the examiner or examiners qualify to do the testing? What was the composition of the normative sample : Ø Age? Ø Gender? Ø Ethnicity, race or linguistic background? Ø Education? Ø Socioeconomic status? Ø Geographic distribution? Ø Any other pertinent variables (physical or mental health status, or membership in an atypical group) that may influence test performance?

Reliability and Validity of Test Scores Reliability Concept of true score Sources of error in test scores Interscorer differences Standard error of measurement Test-retest reliability Validity Does a test measure what it purports to measure – construct validity Content validity and face validity Ecological validity

Testing Diverse Populations • The need to separate out cultural and linguistic factors from the measurement of academic skills is basically an issue of validity. • Trust worthy estimates of the individual’s true or actual abilities VERSUS invalid measures of an individual’s degree of English language proficiency or level of acculturation • There have been 4 general methods offered as attempts to address cultural and linguistic differences: • • • Modified or Adapted Testing – eliminating items that are culturally biased Use of a translator/interpreter – does not overcome culturally bound nature of the test Diverse sample representation – including individuals of diverse racial and ethnic background Nonverbal testing – range of abilities is quite limited Native-language testing – scarcity of bilingual professionals • Nondiscriminatory testing requires that the use of standardized tests can be defended as valid indicators of true or actual ability; the methods used rests with the practitioner.