TSL 3112 LANGUAGE ASSESSMENT BASIC TESTING TERMINOLOGY Normreferenced

TSL 3112 – LANGUAGE ASSESSMENT BASIC TESTING TERMINOLOGY • • Norm-referenced & criterion referenced tests Formative and summative tests Objective & subjective tests 1

LECTURE’S OBJECTIVES �Identify the different formats of tests found �Distinguish different types of tests: norm-referenced and criterion-referenced tests �Compare and contrast formative and summative tests �Differentiate between objective and subjective tests (Main reference - Brown, H. Douglas, 2004. Language Assessment: Principles and classroom practices. ) 2

NORM-REFERENCED & CRITERION REFERENCED TESTS 3

NORM-REFERENCED TESTS �To rank each student with respect to the achievement of others in broad areas of knowledge. �Normed using large groups of test takers. Compares one taker to another. Measure achievement, predicts future performance. �Each individual is compared with other examinees and assigned a score--usually expressed as a percentile, a grade or equivalent score. �Student achievement is reported for broad skill areas, although some norm-referenced tests do report student achievement in specific sub-areas. 4

NORM-REFERENCED TEST �Measures broad skill areas sampled from a variety of textbooks, syllabi, and the judgments of curriculum experts. �Each skill is, usually, tested by less than four items. �Items vary in difficulty. Items are selected that discriminate between high and low achievers. • If too many people get a question correct, or too many score well, then test questions are “thrown out” until they achieve a normal curve again. 5

CRITERION-REFERENCED TEST • Criterion-referenced tests, also called mastery tests, compare a person's performance to a set of objectives. Anyone who meets the criterion can get a high score. • Everyone knows what the benchmarks / objectives are and can attain mastery to meet them. • It is possible for ALL the test takers to achieve 100% mastery. • Measure a student against a specific set of knowledge (criterion). 6

CRITERION-REFERENCED TEST �To determine whether each student has achieved specific skills or concepts. �To find out how much students know before instruction begins and after it has finished. �Measures specific skills which make up a designated curriculum. �These skills are identified by teachers and curriculum experts. �Each skill is expressed as an instructional objective. �Each individual is compared with a preset standard for acceptable achievement. �The performance of other examinees is irrelevant. �Each skill is tested by at least four items in order to obtain an adequate sample of student performance and to minimize the effect of guessing. �The items which test any given skill are parallel in difficulty. 7

NORM & CRITERION REFERENCED TESTS The following is adapted from: Popham, J. W. (1975). Educational evaluation. Englewood Cliffs, New Jersey: Prentice-Hall, Inc. Dimension Purpose Criterion-Referenced Tests Ø Ø Content Ø Ø Norm-Referenced Tests To determine whether each student has achieved specific skills or concepts. To find out how much students know before instruction begins and after it has finished. Ø Measures specific skills which make up a designated curriculum. These skills are identified by teachers and curriculum experts. Each skill is expressed as an instructional objective. Ø Ø To rank each student with respect to the achievement of others in broad areas of knowledge. To discriminate between high and low achievers. Measures broad skill areas sampled from a variety of textbooks, syllabi, and the judgments of curriculum experts. 8

NORM & CRITERION REFERENCED TESTS Dimension Item Characteristics Criterion-Referenced Tests Ø Ø Each skill is tested by at least four items in order to obtain an adequate sample of student performance and to minimize the effect of guessing. The items which test any given skill are parallel in difficulty. Norm-Referenced Tests Ø Ø Ø Each skill is usually tested by less than four items. Items vary in difficulty. Items are selected that discriminate between high and low achievers. 9

NORM & CRITERION REFERENCED TESTS Dimension Score Interpretation Criterion-Referenced Tests Ø Ø Ø Each individual is compared with a preset standard for acceptable achievement. The performance of other examinees is irrelevant. A student's score is usually expressed as a percentage. Student achievement is reported for individual skills. Norm-Referenced Tests Ø Ø Each individual is compared with other examinees and assigned a score--usually expressed as a percentile, a grade equivalent score, or a stanine. Student achievement is reported for broad skill areas, although some norm -referenced tests do report student achievement for individual skills. 10

NORM & CRITERION REFERENCED TESTS Uses of Test Results for Teachers Ø Ø Two main ways that test results can be used by teachers: • For revising instruction for entire class. • For developing intervention strategies for individual students. Standardized test results have not typically been used to aid teachers in making instructional decisions. Ø Data-driven decision making takes some practice and experience for classroom teachers. 11

COMPARING NORM & CRITERION-REFERENCED TESTS • Norm-referenced – – General ability Range of ability Large groups Compares people to people-comparison groups – Selecting top candidates • Criterion-referenced – – – Mastery Basic skills Prerequisites Affective Psychomotor Grouping for instruction 12

COMMON CHARACTERISTICS OF NRT & CRT *Require a relevant and representative sample of test items *Require specification of the achievement domain to be measured *Use the same type of test items *Use the same rules for item writing *Judged by the same qualities (validity and reliability) *Useful in educational measurement 13

ADVANTAGES AND DISADVANTAGES OF NRT Advantages： �They easy for instructors to use �They work well in situations requiring rigid differentiation among students �They are generally appropriate in large courses Disadvantages： �An individual's grade is determined not only by his/her achievements, but also by the achievements of others. �no indication of prerequisite knowledge for more advanced material has been mastered �less appropriate for measuring affective and psychomotor objectives �encourages competition and comparison scores 14

ADVANTAGES AND DISADVANTAGES OF CRT Advantages： �Students are not competing with each other �Students are thus more likely to actively help each other learn. �A student's grade is not influenced by the caliber of the class. Disadvantages： �It is difficult to set a reasonable standard for students �Most experienced faculty set criteria based on their knowledge of how students usually perform �Criterion-referenced systems often become fairly similar to normreferenced systems. �absolute standards difficult to set in some areas �standards tend to be arbitrary �not appropriate comparison when others are valuable 15

FORMATIVE & SUMMATIVE TESTS 16

THE GARDEN ANALOGY If we think of our children as plants … § Summative assessment of the plants is the process of simply measuring them. It might be interesting to compare and analyze measurements but, in themselves, these do not affect the growth of the plants. § Formative assessment, on the other hand, is the equivalent of feeding and watering the plants appropriate to their needs - directly affecting their growth. § Formative and summative assessment are interconnected. They seldom stand alone in construction or effect. 17

FORMATIVE ASSESSMENT �Assessment for learning �Taken at varying intervals throughout a course to provide information and feedback that will help improve �the quality of student learning �the quality of the course itself �The purpose is: �To promote further improvement of student learning during the learning process �To involve students in the ongoing assessment of their own achievement �Provides information on what an individual student needs �To practice �To have re-taught �To learn next 18

KEY ELEMENTS OF FORMATIVE ASSESSMENT 1. The identification by teachers & learners of learning goals, intentions or outcomes and criteria for achieving these. 2. Rich conversations between teachers & students that continually build and go deeper. 3. The provision of effective, timely feedback to enable students to advance their learning. 4. The active involvement of students in their own learning. 5. Teachers responding to identified learning needs and strengths by modifying their teaching approach(es). Black & Wiliam, 1998 19

BENEFITS OF FORMATIVE ASSESSMENT FOR TEACHERS (Boston, 2002) �Teachers are able to determine what standards students already know and to what degree. �Teachers can decide what minor modifications or major changes in instruction they need to makes so that all students can succeed in upcoming instruction and on subsequent assessments. �Teachers can create appropriate lessons and activities for groups of learners or individual students. �Teaching can inform students about their current progress in order to help them set goals for improvement. 20

BENEFITS OF FORMATIVE ASSESSMENTS FOR STUDENTS �Students are motivated to learn. �Students take responsibility for their own learning. �Students become users of assessment. �Students learn valuable lifelong skills such as selfevaluation, self-assessment, and goal setting. �Student achievement can improve from 21 -41 percentile points. (marzano 2003; stiggens et. al, 2006) 21

SUMMATIVE ASSESSMENT �Assessment of learning �Generally taken by students at the end of a unit or semester to demonstrate the "sum" of what they have or have not learned. �Summative assessment methods are the most traditional way of evaluating student work. �"Good summative assessments--tests and other graded evaluations--must be demonstrably reliable, valid, and free of bias" (Angelo and Cross, 1993). 22

POSSIBLE ASSESSMENT METHODS Formative Assessment includes �Questions �Classroom Discussions �Learning Activities �Feedback �Conferences �Interviews �Student Self-Assessment Summative Assessment �Selected Response �Multiple Choice �True/False �Matching �Fill-in �Extended Written Response �Performance Assessment 23

BALANCED CLASSROOM ASSESSMENT SYSTEM FORMATIVE ASSESSMENT A process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to help students improve their achievement of intended instructional outcomes. SUMMATIVE ASSESSMENT A tool used after instruction to measure student achievement which provides evidence of student competence or program effectiveness. students are evaluated upon completion of the work and the focus is on the final product. 24

COMPARISON OF ASSESSMENTS FORMATIVE • Occurs During Instruction • Not Graded • Process • Descriptive Feedback • Continuous SUMMATIVE • Occurs at the end • Graded • Product • Evaluative Feedback • Periodic • Sort students in rank order 25

COMPARISON OF ASSESSMENTS A Fine Sieve �Formative assessment informs both teachers and students about student understanding at a point when timely adjustments can be made. �These adjustments help to ensure students achieve targeted standards-based learning goals within a set time frame. A course sieve �Summative assessments happen too far down the learning path to provide information at the classroom level and to make instructional adjustments and interventions during the learning process 26

OBJECTIVES & SUBJECTIVES TESTS 27

OBJECTIVE AND SUBJECTIVE ASSESSMENT Objective assessment is a form of questioning which has a single correct answer. Subjective assessment is a form of questioning which may have more than one current answer (or more than one way of expressing the correct answer). 28

OBJECTIVE TEST �Objective tests include multiple choice, true-false, matching, and fill-in questions. They tend to focus more on specific facts than on general ideas and concepts �Questions on a tests that only have one correct answer �Objective tests require far more careful preparation than subjective tests �Objective examination can be part of formative (diagnostic) and summative (final assessment) exams. �Most popular objective exam is Multiple Choice Questions (MCQ). (the method of scoring is the only factor that distinguishes an objective test from a subjective test)

MCQ �Advantages of multiple choice question: 1. The ability to create a test item bank 2. Quick grading – can be easily computer scored 3. If written well, high reliability - only one possible answer 4. Objective grading 5. Wide coverage of content 6. Can be used for mass testing 7. Precision in providing information regarding specific skills and abilities. 8. Students are familiar with the item type – directions are easy to understand.

MCQ �Weaknesses of multiple choice question: 1. Difficult and time consuming to construct 2. Low validity 3. Mainly tests recognition knowledge and recall of facts. 4. Guessing may have considerable effect 5. Cheating may be facilitated 6. Sometimes skills and areas are tested because they are testable than important 7. Places a high degree of dependence on student’s reading ability and teacher’s writing ability. 8. It may limit beneficial washback. 9. This technique strictly limits what can be tested.

SUBJECTIVE TEST �Subjective tests include essay, short-answer, vocabulary, and take-home tests �Questions on a test that have more than one correct answer. �Each examiner uses his own judgment in evaluating performance and awarding marks.

SUBJECTIVE TEST � Strengths: 1. Easy to set 2. High validity 3. Can assess affective and interpretive aspects of language skills 4. allow a candidate to express originality of thought 5. allow the examiner to assess the candidate's quality of written expression. � Weaknesses: 1. Marking is time consuming 2. Reliability is low 3. Inter-rater as well as intra-rater variability are probable. 4. Dependence on presentation. - good hand writing vs bad handwriting 5. Question evasion - possible for the candidate to avoid questions in areas of the curriculum in which they are weak.

OBJECTIVE VS. SUBJECTIVE TEST Objective �short answer �closed response �mostly recognition, limited production �difficult to write well �quick and easy to grade �reliable �workload “up front” Subjective �long answer �open response �emphasis on production �relatively easy to write �difficult to grade �time-consuming �inter-rater reliability �not as reliable �workload post test 34

REFERENCES � Classroom Assessment: Basic Concepts. Formative vs. Summative Assessments. Retrieved October 20, 2008 from http: //fcit. usf. edu/assessment/basica. html � Formative vs. Summative Evaluation. Retrieved October 20, 2008 from http. jan. ucc. nau. edu/edtech/etc/667/proposal/evaluation/ summative_vs_formative. htm � Formative and Summative Assessment. Retrieved October 20, 2008 from http: //www. krauseinnovationcenter. org/ewyl/modules/module 63. html. � Classroom Assessment: Basic Concepts. Formative vs. Summative Assessments. Retrieved October 24, 2008 from http: //fcit. usf. edu/assessment/basica. html � Pawlas, G. , Oliva, P. (2008) Supervision for Today’s Schools, Sixth Edition. New York: John Wiley and Sons 35

� Arter, Judith, and Jay Mc. Tighe. Scoring Rubrics in the Classroom. Thousand Oaks, CA: Corwin Press, INC. , 2001. � Marzano, Robert J. , Debra Pickering, and Jay Mc. Tighe. Assessing Student Outcomes. Alexandria, VA: Association for Supervision and Curriculum Development, 1993. � Schoenbach, Ruth, et al. Reading for Understanding, A Guide to Improving Reading in Middle and High School Classrooms. San Francisco, CA: Jossey-Bass, Inc. , 1999. 36