Mapping Affect Project MAP Promoting links between values

Mapping Affect Project (MAP) Promoting links between values, judgement of learning and cognition in learning chemistry in Taiwan John Oversby, The University of Reading, UK J. P. Oversby@reading. ac. uk Shu-Nu Chang, College of Liberal and General Education, Aletheia University, Taiwan shunu@ms 3. url. com. tw Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 1

My background Chemist (UG and Ph. D): Leeds University ► School teacher (22 years): ► § Africa § UK Secondary Schools § Responsible for ICT ► University teacher (15 years) § Teacher training § Research in modelling and teacher education Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 2

Theme of talk ► Three Taxonomies by Bloom (1956) § Cognition § Motor skills § Affect ► Focus on Affect i. e. Values, Social, Attitude, Motivation, Relationship, Judgment of Learning (Jo. L) Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 3

Two problems… ► Assuring quality learning outcomes § Requires data about learning during learning events rather than at their conclusion ► Developing lifelong learners § Requires that students can assess their own learning without formal summative tasks, relying instead on self-assessment Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 4

Why use student confidence in learning? Informed confidence, or a critical awareness of your own capacity, is one of the primary components of self-efficacy ► Judgement of Learning (Jo. L) is a reliable indicator of test performance ► § Even with gender bias (attribution theory) i. e. males attribute success to hard work and failure to bad luck ► females attribute success to good luck and failure to lack of effort ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 5

Quality Assurance from Jo. L ► Jo. L can be assessed rapidly, and during task performance § Some student training is needed to enable critical self-appraisal ► Frequent evaluation allows timely intervention to maximise learning outcomes ► Records of Jo. L assessment allow a timely and frequent process of assessment for learning to be documented and formalised. Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 6

Development of Self—efficacy …an individual’s estimate of his/her ability to cope with a situation, and outcome expectancy; an individual’s estimate of the likelihood of certain consequences occurring. This combination of assessments of potential threat and coping resources determines how anxious an individual may become in a given situation. § Regular feedback regarding the accuracy of Jo. L should heighten critical capacity and reliability § Appropriate interventions by Teacher or Learner should promote both generic and academic skill development foundations for independent learning Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 7

The Questions • How would you describe your • • understanding of the recent lecture content? ability to use the diagram conventions? participation in the teamwork aspects? ability to communicate ideas and processes? ability with the level of symbols? methods for problem solving? approach to asking questions or seeking help? approach to this subject i. e. personal confidence? Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 9

Measuring Student Confidence ► Fortune Lines § A variety of possible curves confidence Now I know which idea about acids to choose? We didn’t do it like this before Does it work with this new evidence? Ah! Now I know about ways of looking at acids What are these experiments about? Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 10

The Responses ► Based on an extension of Bloom’s taxonomies for the affective (or psychomotor) domains Deep Approach? § Internalised values … ( Characterising self) § Organised values … (ranking values) § Valuing Achieving orientation? § Responding (active) § Responding (passive) § Receiving Surface Approach? Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 11

Defining persistent learning features ► Long term self-assessment is dependent on characterisation of persistent learning features ► These are at odds with most of our learning objectives – short term, fragmented and factual ► Forging new ground in defining higher level generic cognitive aspects ► Perhaps leads to too much innovation and hence problems with self-assessment of the new areas Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 12

Example of persistent feature - diagrams Know types of diagrams and their major features e. g. 2 D, 3 D (and 2. 5 D!) ► Know why many diagrams have accepted conventions e. g. colours of atoms in ball-stick diagrams ► Know how to interconvert diagrams e. g. 3 D into 2 D through contours such as electron density diagrams ► Know how to relate different representations of the same thing e. g. dot cross diagrams with orbital energy diagrams ► Know the strengths and limitations of running dynamic diagrams in the mind ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 13

Example: How would you describe your methods for problem solving? It is Intuitive I don’t really think about the solution method – it usually seems that there is an obvious approach to take, even though most of these problems can be solved in more than one way. Sometimes I take a new approach to a problem to see if it will work more quickly or more accurately. ► I am adept I understand the solution method for all the tutorial and example problems, and know why the different methods were used. I usually start with a general approach based on the type of problem and modify the method as I go along so that I get to the solution. Novel problems are interesting in the challenge because they challenge me to vary my method in some way to get to the answer. ► I am competent The approach for most problems is reasonably straightforward, and depends on the type of problem. I can usually decide what method to use fairly quickly and my choice usually gives me the correct result. If I make errors, they tend to be in the working rather than the choice of method. ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 14

Training of students Provide criteria for initial discussion b) Discuss the snapshots (following) in the light of the criteria c) Discuss the criteria during data collection a) Hindsight suggests that we did not provide strong training in using the criteria. This will be amended in phase 2. Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 15

Snapshots of Individuals Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 16

Research questions ► Can we construct a valid and reliable method for assessing feelings about knowing and values of learning of students in a chemistry course? ► What aspects of learning are persistent features from session to session that enable progress in higher level skills to be monitored? ► How do self-assessments of feelings about knowing change during the course? Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 17

Methodology I 110 freshmen non-major chemistry students in Taiwan ► 6 weeks collecting data ► Attrition rate of 31% - did not attend every session ► Eight features of affect: § Use of diagrams § Use of graphs § Help seeking § Using symbols and equations § Confidence in learning i. e. Jo. L § Commitment § Group work § Explaining ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 18

Methodology II Use of a six point scale of indicators for each attribute, linked to Bloom’s Taxonomy of Values ► Indicators written by John Oversby and Phil Turner (construct validity) ► Collection of named data each week in class ► Two standardised hypothetical Case Studies as reference points. Used in second week and sixth week. ► Case studies written by John Oversby and Phil Turner (construct validity) ► Class test results used as reference point for cognitive ability ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 19

Methodology III ► Data collected by SNC and analysed by JO, anonymously ► Analysis remains work in progress (more to be shared) Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 20

Interpretation I ► Case studies data shows large range over class (standard deviation) ► The range changes little even after six assessments. More experience does not provide greater consistency ► May be an artefact of the referencing system or difficulty in self-assessment Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 21

Are students accurate in Judgment of Learning? ► Apparently not! ► Perhaps we are expecting too much to focus on persistent features of learning e. g. new thinking about metacognitive aspects of diagrams Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 22

Are students accurate in behaviour selfassessment? ► Apparently a little! ► In this case we see that more able students seek help less. ► Perhaps there is a plateau effect, where students aim for good enough. Is there value in doing better than this? Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 23

Are students accurate in self-assessment of confidence (Jo. L)? ► Apparently not! ► No difference between less able and more able students in judgement of confidence in learning. ► May be caused by lack of training Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 24

Are students accurate in self-assessment of commitment? ► Apparently only a little but in the wrong direction! We expected more able students to be more committed! ► May be better analysed in terms of gender split but we do not have this evidence in phase 1 Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 25

Do more able students wish to work in groups? ► Less able students prefer, a little, to work more in groups. ► May be better analysed in terms of gender split (in phase 2) Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 26

Successes and areas for improvement Clarified our own thinking about persistent learning features ► Constructed attribute lists and tested them for validity ► Demonstrated the rapid nature of the self assessments ► Found some evidence that correlated with commitment as evidenced through attendance ► Under-estimated training needs ► Under-estimated challenge of self-assessment of persistent learning features ► Need to look at ways of sharing the information with students ► Need to look more widely at self-regulation ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 27

Where do we go next? Repeat research with more specific training materials based on our pilot study ► Review instructions for defining attribute levels ► Refine Case Studies and check with a variety of colleagues for validity ► Further analysis of data ► Involve more colleagues – Slovenia, Poland, Finland, Austria, US Virgin Islands, UK ► Use different groups of learners – pre-service teachers, undergraduate science students, high school and junior high school pupils ► Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 28

Keeping in touch ► We are constructing a web-site with all the materials: § Background research and other materials § Papers and PP written on the subject § Manuals for doing the project with Beginning Teachers, high school students, teachers § Spreadsheet templates for data collection and analysis – leading to sharing of data § Meetings and conferences on the research ► Write to John Oversby at j. p. oversby@reading. ac. uk after the end of April to be in the next round Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 29

Thanks ► To our respective universities for their support ► To the students in Taiwan for sharing their self-assessments with us ► To Phil Turner (James Cook University) for his ideas ► To ESERA for being the origin of this collaboration ► To Shu-Nu Chang for being so reliable Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 30

Selected references ► ► ► ► Bandura, A. (1991). Self-regulation of motivation through anticipatory and self reactive mechanisms. In R. A. Dienstbier (Ed. ), Perspectives on motivation. Nebraska Symposium on Motivation. Lincoln University of Nebraska Press. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman. Cao L & Nietfield JL Judgment of learning, monitoring accuracy, and student performance in the clssroom context Current Issues in Education 8(4) 2005 Accessed on 26. 8. 05 from http: //cie. asu. edu/vume 8/number 4 Gunstone RF and Mitchell IJ (1998) Metacognition and conceptual change in Mintzes Jr J, Wandersee JH and Novak JD (Eds) (1998) Teaching for understanding. A human constructivist view (pp 133 -165) Academic Press, New York Gunstone RF and Northfield J (1994) Metacognition and learning to teach. International Journal of Science Education 16(5) 523 -537 Gunstone R and White R (1992) Probing understanding Falmer Press, London Krathwohl, D. , Bloom, B. , & Masia, B. (1956). Taxonomy of educational objectives. Handbook II: Affective domain. David Mc. Kay, New York Nordic Helsinki © John Oversby and Shu-Nu Chang 2007 31