W JAMES POPHAM WHAT WOULD POPHAM CONSIDER TO

W. JAMES POPHAM WHAT WOULD POPHAM CONSIDER TO BE A QUALITY ASSESSMENT OF A SCHOOL’S EDUCATION?

POPHAM’S IDEAS • OBJECTIVES SHOULD BE MEASURABLE AND CLEAR • OBJECTIVES SHOULD BE SPECIFIC • OBJECTIVES SHOULD REFER TO THE POST-INSTRUCTION BEHAVIOR • OBJECTIVES SHOULD REQUIRE CONSTRUCTED RESPONSES FROM LEARNERS • THE USE OF THE WRONG TEST IN EDUCATION LEADS TO SOME NEGATIVE CONSEQUENCES • DISHONESTY, • A REDUCTION IN CURRICULUM • COVERING OF CURRICULUM THAT LACKS DEPTH.

WHY THE STANDARDIZED TEST? • THE PURPOSE OF STANDARDIZED TESTING: • HOLD TEACHERS ACCOUNTABLE FOR WHAT THEY TEACH • MEASUREMENT OF QUALITY OF TEACHER • MEASUREMENT OF QUALITY OF SCHOOL • EASY AND QUICK WAY TO GATHER DATA • MAINTAINS AN EQUAL TEST AND ENVIRONMENT THROUGHOUT THE STATE

TYPES OF STANDARDIZED TESTS • APTITUDE TEST-PREDICTS HOW STUDENTS ARE LIKELY TO PERFORM IN A FUTURE EDUCATIONAL SETTING • EXAMPLES: ACT, SAT • ACHIEVEMENT TEST-WHAT IS RELIED ON TO EVALUATE A SCHOOL’S EFFECTIVENESS • EXAMPLES: IOWA TESTS OF BASIC SKILLS, STANFORD ACHIEVEMENT TEST

WHAT IS THE PROBLEM WITH STANDARDIZED TESTS? • THE TEST CONTAINS TOO FEW ITEMS TO ALLOW MEANINGFUL INFERENCES • SEVERAL FACTORS CAUSE SCORES TO FLOP ABOUT SO IT IS DIFFICULT TO TRACK A STUDENT’S IMPROVEMENT OVER TIME • THE TESTS ACTUALLY DO AN EXCELLENT JOB AT GATHERING EVIDENCE TO MAKE INFERENCES OF A STUDENT’S KNOWLEDGE/SKILLS IN RELATIONSHIP TO STUDENTS NATIONALLY • THE PROBLEM IS THAT THEY ARE NOT DESIGNED TO EVALUATE THE QUALITY OF EDUCATION

THREE REASONS IT IS INVALID TO BASE INFERENCES ABOUT THE QUALITY OF EDUCATION USING STANDARDIZED ACHIEVEMENT TESTS 1. TESTING-TEACHING MISMATCHES 2. A PSYCHOMETRIC TENDENCY TO ELIMINATE IMPORTANT TEST ITEMS 3. CONFOUNDED CAUSATION

TESTING-TEACHING MISMATCHES • EDUCATORS PURSUE SIMILAR GOALS AT A GENERAL LEVEL– ALL SCHOOLS HAVE LANGUAGE ARTS AND MATH, BUT AT THE CLASSROOM LEVEL OBJECTIVES ARE SIGNIFICANTLY DIFFERENT • TEST-DEVELOPERS MUST CREATE A ONE-SIZE-FITS ALL STANDARDIZED ASSESSMENT FOR A NATION THAT HAS SIGNIFICANT CURRICULUM DIVERSITY • A STUDY CONDUCTED BY MICHIGAN STATE UNIVERSITY SHOWED THAT THE CONTENT OF STANDARDIZED TESTS IS NOT USUALLY SATISFACTORILY ADDRESSED BY THE MOST WIDELY USED TEXTBOOKS. • TEST DESCRIPTORS ARE GENERAL TO APPEASE EVERYONE, AS A RESULT, THE TEACHING-TESTING ALIGNMENT IS WAY OFF

A PSYCHOMETRIC TENDENCY TO ELIMINATE IMPORTANT TEST ITEMS • THE TESTS USE A SMALL COLLECTION OF ITEMS TO TRY TO MAKE MEANINGFUL COMPARISONS • THE MAJORITY OF STANDARDIZED TEST ITEMS ARE CONSIDERED “MIDDLE DIFFICULTY” ITEMS • TO ACHIEVE SCORE VARIANCE, ITEMS STUDENTS PERFORM WELL ON ARE EXCLUDED, WHICH BECAUSE OF THEIR IMPORTANCE, IS OFTEN THE CONTENT TEACHERS EMPHASIZE

CONFOUNDED CAUSATION • POPHAM BELIEVES THAT A STUDENTS PERFORMANCE ON THESE TESTS ARE HEAVILY INFLUENCES BY THREE CAUSATIVE FACTORS, AND THAT ONLY ONE OF THEM IS LINKED TO INSTRUCTIONAL QUALITY. • THE THREE CAUSATIVE FACTORS ARE: 1. WHAT’S TAUGHT IN SCHOOL 2. A STUDENT’S NATIVE INTELLECTUAL ABILITY 3. A STUDENT’S OUT-OF-SCHOOL LEARNING

CAUSATIVE FACTOR: WHAT’S TAUGHT IN SCHOOL • SOME OF THE ITEMS IN STANDARDIZED ACHIEVEMENT TESTS MEASURE THE KNOWLEDGE OR SKILLS THAT STUDENTS LEARN IN SCHOOL. IN MATH, FOR EXAMPLE, STUDENTS LEARN MOST OF WHAT THEY KNOW ABOUT THIS SUBJECT IN SCHOOL. HERE’S AN EXAMPLE OF A 3 RD GRADE TEST ITEM THAT MEASURES A STANDARD TAUGHT IN SCHOOL: • SALLY HAD 14 PEARS. THEN SHE GAVE AWAY 6. WHICH OF THE NUMBER SENTENCES BELOW CAN YOU USE TO FIND OUT HOW MANY PEARS SALLY HAS LEFT? • 14 + 6 = ___ • 6 + 14 = ___ • __ − 6 = 14 • 14 − 6 = ___ •

CAUSATIVE FACTOR: A STUDENT’S NATIVE INTELLECTUAL ABILITY • POPHAM DOES NOT BELIEVE THAT ALL CHILDREN WERE BORN WITH IDENTICAL INTELLECTUAL ABILITIES. • THE EXAMPLE BELOW ATTEMPTS TO MEASURE A CHILD'S ABILITY "TO FIGURE OUT" WHAT THE RIGHT ANSWER IS. POPHAM DOESN’T BELIEVE THE ITEM MEASURES WHAT'S TAUGHT IN SCHOOL, BUT WHAT THE STUDENT COMES TO SCHOOL WITH: EXAMPLE TEST ITEM: IF SOMEONE REALLY WANTS TO CONSERVE RESOURCES, ONE GOOD WAY TO DO SO IS TO: a) LEAVE LIGHTS ON EVEN IF THEY ARE NOT NEEDED. b) WASH SMALL LOADS INSTEAD OF LARGE LOADS IN A CLOTHES-WASHING MACHINE. c) WRITE ON BOTH SIDES OF A PIECE OF PAPER. d) PLACE USED NEWSPAPERS IN THE GARBAGE.

CAUSATIVE FACTOR: A STUDENT’S OUT-OFSCHOOL LEARNING • POPHAM FINDS THE BIGGEST PROBLEM WITH THE TEST IS WHEN IT ASSESSES WHAT STUDENTS HAVE LEARNED OUTSIDE OF SCHOOL. CHILDREN WHO COME FROM ADVANTAGED FAMILIES AND STIMULUS-RICH ENVIRONMENTS, ARE MORE LIKELY TO DEMONSTRATE SUCCESS ON THESESTANDARDIZED TEST ITEMS THAN THOSE WHO AREN’T. LOOK AT THE EXAMPLE BELOW AND CONSIDER WHAT IS REALLY BEING MEASURED: TEST ITEM: A PLANT'S FRUIT ALWAYS CONTAINS SEEDS. WHICH OF THE ITEMS BELOW IS NOT A FRUIT? a) ORANGE b) PUMPKIN c) APPLE

RIGHT TASK, WRONG TOOLS • TEACHERS ARE PUT IN CHARGE OF AN EXTREMELY IMPORTANT TASKEDUCATING THE FUTURE • TEACHERS NEED TO BE HELD ACCOUNTABLE • STANDARDIZED ACHIEVEMENT TESTS ARE THE WRONG TOOL TO ASSESS THIS TASK PROPERLY

INSTRUCTIONALLY SUPPORTIVE TESTS • POPHAM HELPED CREATE , AND SERVED AS CHAIR, THE COMMISSION ON INSTRUCTIONALLY SUPPORTIVE ASSESSMENT • THE COMMISSION DEVELOPED NINE REQUIREMENTS TO HELP STATES MEET THE NEEDS OF THE ELEMENTARY AND SECONDARY EDUCATION ACT • IT IS THE COMMISSION’S BELIEF THAT IF STATES DEVELOP THE KIND OF TEST THE COMMISSION ADVOCATES, THEN THE TESTS WILL NOT ONLY MEET THE ESEA NEEDS, BUT ALSO PROMPT IMPROVEMENT IN THE QUALITY OF INSTRUCTION

NINE REQUIREMENTS FOR INSTRUCTIONALLY SUPPORTIVE ASSESSMENTS • PRIORITIZED CONTENT STANDARDS • CLEARLY DESCRIBED CONTENT STANDARDS • STANDARD-BY-STANDARDS TEST RESULTS • OPTIONAL CLASSROOM ASSESSMENTS • MONITOR CURRICULUM BREADTH • APPROPRIATE ASSESSMENTS FOR ALL STUDENTS • TIME & TECHNICAL ADEQUACY • PROFESSIONAL DEVELOPMENT RELATED TO ASSESSMENT • ONGOING EVALUATION TO IMPROVE ASSESSMENTS

WHAT SHOULD BE DONE? • BECOME EXPERTS ON STANDARDIZED ACHIEVEMENT TESTS • SPREAD THE WORD • RESEARCH OTHER EVIDENCE • HTTPS: //WWW. YOUTUBE. COM/WATCH? FEATURE=PLAYER_EMBEDDED&V=OQH V-6 SIKSU