Collecting Using Data to Guide Instruction in the

Collecting & Using Data to Guide Instruction in the Science Classroom Wednesday, 11/7/2007 Gary Carlin gcarlin@schools. nyc. gov (718) 828 -4007

Workshop Goals 1. Compare and Contrast Analytic and Holistic rubrics. 2. Create Analytic and Holistic rubrics to evaluate student work and collect data. 3. Create a process by which students will produce their own rubrics. 2

Share Out • Last Assignment: – Give a test and create an Item Analysis. – Revise the test based on the item analysis. – What are the teaching and assessment implications? 3

Item Analysis Reflection What surprised me the most? What were some of the major misconceptions of my students? How would/did I What are the revise my test? implications of the data in terms of my teaching? 4

Physics Regents June 06 25. The energy of a sound wave is most closely related to its (1) period (3) frequency (2) amplitude (4) wavelength Item 25 1 2 3 4 NR 56 645 1045 258 4 5

Become an Item Writer • General Rules for Multiple-Choice Items • In order to become an item writer, teachers must attend a New York State Education Department sponsored item writer training session. § New item-writers are selected from the pool of teachers who complete and submit to the Education Department an on-line application titled Call for Expertise which may be found at http: //www. emsc. nysed. gov/ciai/call. htm. 6

Introductory Activity I May Not Know What It Is, But I Can … 7

Directions • You have 30 seconds to Observe and Describe the unknown object. • You may: - draw a diagram - write your observations - ask questions that would help in the ID - name/guess the object • Then you will exchange for feedback 8

Image #1 9

Pass & Share Respond to your partner’s: – Observations – Description – Drawing – Question(s) – Object identification EGG BEATER 10

Image #2 11

Pass & Share Respond to your partner’s: – Observations – Description – Drawing – Question(s) ORANGE JUICER – Object identification 12

Image #3 13

Pass & Share Respond to your partner’s: – Observations – Description – Drawing – Question(s) CHANNEL LOCK PLIERS – Object identification 14

Image #4 15

Pass & Share Respond to your partner’s: – Observations – Description – Drawing – Question(s) – Object identification BEE STING 16

Task 1. Make a list the of the different types of assessments you currently utilize in your classroom. 2. Separate the list into FORMAL and INFORMAL ASSESSMENT. 17

Formal and Informal Assessment • Are not technical psychometric terms; therefore, there are no uniformly accepted definitions. • "Informal Assessment“ techniques – can easily be incorporated into classroom routines and learning activities. – have results that are indicative of the student's performance on the skill or subject of interest. – can be used at anytime without interfering with instructional time. – are not intended to provide a comparison to a broader group beyond the students in the local project. 18

How is informal and formal assessment data collected? 19

Activity 1 Rubric Quick-Write 20

What is a …. In One Word? 21

Rubrish: heading, section of text, written in red for distinctiveness 22

General Definition ru • bric [‘roobrik] noun A heading on a document. • a direction in a liturgical book as to how a church service should be conducted. • a statement of purpose or function: art of a purpose, not for its own sake, was his rubric. • a category: party policies on matters falling under the rubric of law and order. 23

Recent Educational Definition • Scoring guide that describes criteria for student performance and differentiates among different levels of performance within those criteria. • Scoring tool that lists criterion (“what counts”) with gradations of quality for each criterion. • Describes problems that students encounter (better than identify mistakes). • Authentic assessment tool. 24

The 3 “P’s” for Rubric Uses Processes Performances Products Coop Learn Debate Portfolios Discussions Role Play Research Paper Critical Thinking Article Presentation Museum Exhibit Habits of Mind Mock Trail Investigation Observational Interview Meeting Story Poem Article Production Lesson or Unit 25

The Advantages of Using Rubrics in Assessment are that they … • Allow assessment to be more objective and consistent. • Focus the teacher to clarify his/her criteria in specific terms. • Promote student awareness about how their work will be evaluated and what is expected. • Provide useful feedback regarding the effectiveness of the instruction. • Provide benchmarks against which to measure and document progress. 26

What Do Students Say … • “They help me determine if the work is good or not. ” • “… they help us explain to other people what they need to learn. ” • “… grade other people’s work … so we get an idea how we can improve and know where we messed up. ” • “… they help you know what you left out. What am I doing wrong? ” • “… it is like a list you go down to check off what you’ve done” • “This is a more “fair” way of grading. ” 27

• “You can go back and make your work better … It lets you know exactly what you need to do to get an A. ” • “… if you follow along the rubric you already know your grade. ” • “… it gives me guidelines on what I do. … if rubrics didn’t improve work, why are teachers giving it? ” • “… rubrics can help teachers grade better than just thinking of a grade in their heads. ” • “The best rubrics show you each step to complete your work. Also, they give some examples of how you should do it so you can follow them. ” • “A student can also use it for them if they feel they have been unfairly graded. ” • “… if teachers used rubrics instead of oral directions student would get better grades because they could just look at the rubric instead of keeping asking the teacher. ” 28

But Be Careful, Students Also Say … • “… some of the rules are very strict …” • “Some rubrics don’t fully cover all the situations” • “… the rules can’t be changed. There are no acceptations. ” • “… you have everything the rubrics ask you to put on your paper but it has to be almost perfect and fully explained. ” • “… you really think you got it all and you don’t. Your teacher may disagree. ” • “… if you don’t follow it step by step it could ruin your 29 work. ”

• “Having to follow directions closely until you find what you need. ” • “They’re not always easy to understand. ” • “If you lose your rubric you’re in a great deal of trouble. ” • “… it is hard to translate them into a letter grade. ” • “… if you’re off by just a little bit from the rubric your score goes down. ” • “Most of the time it’s the language. It can be hard to understand …” • “… maybe it makes you feel bad because you feel you have to push yourself harder …” • “It bothers me that most of them require a lot. ” 30

Three Common Features • Focus on measuring a stated objective (performance, behavior, or quality). • Use a range to rate performance. • Contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met. 31

What Do We Both Want? 32

It’s All About Matching Expectations 33

Otherwise, We Get This … 34

When We Wanted … 35

We Need to Speak the Same Language! 36

Activity 2 Today’s Horoscope 37

Activity 3 Contrasting Analytic and Holistic Rubrics 38

Rubric Basics • Rating Scale – Across the top (i. e. , 1 -4) • Dimension – Along the side (Trait being judged, Criteria – conditions must be met to be considered successful) • Descriptors – Within the boxes (Statements that describe each level of performance, Indicators – specific examples) 39

Rubric Scales • Numerical Scale with Verbal Descriptors • Specific Professions • Visuals 40

Rubric Descriptors • • • Understanding Frequency Effectiveness Independence Accuracy Clarity 41

Levels of Quality 4 3 2 1 Yes … No … but … 42

Task • 1. Make a list of 7 -10 important characteristics of Cooperative Learning. • 2. Put them in rank order (1 -most and 10 least important). • 3. Condense the list to 3 -5 important characteristics. 43

Activity 4 Creating an Analytic Rubric: The Perfect Cooperative Learning Group 44

Activity 5 Converting an Holistic Rubric to a Analytic Rubric 45

Directions 1. Select 3 -5 Dimensions for the Rubric. 2. Select a Rubric Scale. 3. Write Descriptors for each level of a Dimension. Hint: Use the Writing Rubric Descriptors handout to help select/write the approriate “Degrees of …” vocabulary to represent the different scale values. 46

Activity 6 Writing a Holistic Rubric Drawing a Line Graph 47

Evaluating Rubrics • Does it reflect what has been emphasized in the class/instruction? • Is it clear what a student needs to do to get a score at each scale point? • It should address all aspects of the outcome being measured – nothing extraneous. • Does the highest point value represent a truly exemplary performance or product? • Is there inter-rater reliability when used by different graders? 48

49

50

51

52

“I’m kind of getting tired of rubrics because we even have a rubric at lunch which is ridiculous. ” 53

Next Session • December 19, 2007 (FDA) • Assignment: – Create 1 Analytic Rubric. – Create 1 Holistic Rubric. – Use the Rubrics to Evaluate Student Work. – Describe how you would modify the Rubric for future use. 54

#44. A volleyball hit into the air has an initial speed of 10. meters per second. Which vector best represents the angle above the horizontal that the ball should be hit to remain in the air for the greatest amount of time? Item 1 2 3 4 NR 44 649 151 902 302 4 55