OBTL at City U in the Hong Kong

OBTL at City. U in the Hong Kong Context 7 th December 2006 Professor John Biggs Principal Consultant OBTL Project

Outcomes-based Education (OBE) There are three main forms of “OBE”. All focus on educational outcomes but each is based on a different philosophy. It’s easy to confuse them. 1. Outcome-based Education at school level. Originally for disadvantaged children, but later used generally e. g. Target Oriented Curriculum (TOC) for individualising teaching. 2. Outcomes-based Education. Outcomes at institutional level, used for benchmarking, credit-transfer. 3. Outcomes-based Teaching and Learning (OBTL). Defining learning outcomes at course and subject level, to enhance teaching and learning.

Institutional, Progamme and Course Outcomes OBE OBTL Outcomes at the institutional level: Graduate attributes, General competencies for use within and across institutions. Outcomes at the programme level: Academic and professional expertise and values Outcomes at the course level: where the teaching, learning and assessment actually occur

What the UGC said “The UGC’s goal in promoting outcome-based approaches is simple and straightforward— improvement and enhancement in student learning and teaching quality. ” (Alice Lam, May 06) OBTL, in other words. “We think that the curriculum revision under '3 + 4' will be a good opportunity to weave 'outcomes' into the new curriculum. ” The UGC is providing extra funding to promote early adoption of OBTL and a supportive Task Force.

Outcomes-based Teaching and Learning (OBTL): Classroom Based Teaching: To facilitate attaining the ILOs Intended Learning Outcomes: What the student has to do Assessment: How well has the student has attained the ILOs

Implementing OBTL using Constructive Alignment Teaching/Learning Activities: Engaging the Student to perform verb in the ILO: Assessment Task: What the student has to learn, e. g. 'to analyse X' How well the student has met the ILO

Outcomes-based Teaching and Learning through Constructive Alignment There are four steps in designing such teaching: 1. describe intended learning outcomes in the form of verbs in a context, indicating what is to be attained and the desired quality of attainment. 2. create a learning environment likely to bring about the intended outcomes. 3. use assessment tasks enabling you to judge if and how well students’ performances meet the criteria. 4. transform these judgments into standard grading criteria.

Constructive Alignment is a Framework for Reflective Practice Constructive alignment is not a prescriptive “method”: each step provides teachers a focus for reflective practice. Teachers can then design ILOs, select TLAs and assessment and grading procedures that are the most effective and practicable given the particular context and subject matter of each. Several teachers have told us that this simply sharpens what they have been doing already. Absolutely! Good teachers naturally think in terms of what their students have to learn, how they might best learn it, and how they might best confirm that students have indeed learned what was intended.

Constructive Alignment Intended Learning Outcomes (ILOs) expressed as verbs students have to enact Teaching / Learning Activities Designed to elicit desired verbs May be: Large class activities Small class activities Teacher-managed The very best understanding that could be reasonably expected: verbs such as hypothesise, apply to “far” domains, generate, relate to principle, etc. Highly satisfactory understanding: verbs such as explain, solve, understand main ideas, analyze, compare, etc. Quite satisfactory learning, with understanding at a declarative level: verbs such as elaborate, classify, cover topics a to n, Peer-managed Self-managed as best suits context Understanding at a level that would warrant a Pass: low level verbs, also inadequate but salvageable higher level attempts. Assessment Tasks Format such that the target verbs are elicited and deployed in context. Criteria clearly allow judgement as to the quality of the student's performance

Intended Learning Outcomes (ILOs). Statements of what students are expected to be able to do as a result of studying a course or programme. . Expressed in the form of learning verbs needed to achieve the outcome. The teaching/learning activities and assessment tasks also address these verbs.

The Verbs in the ILOs Generic high level verbs include: reflect, hypothesise, solve unseen complex problems, generate new alternatives. Lower level verbs include: describe, identify, memorize. Often these lower level ILOs can be subsumed under higher level. Some ILOs will contain ‘large’ verbs like ‘write’ with focus on a particular task (write a lab report with specifications X, Y, Z …)

The SOLO Taxonomy with sample verbs indicating levels of understanding Competence Identify Name Follow simple procedure Combine Describe Enumerate Perform serial skills List Analyze Apply Argue Compare/ contrast Criticize Explain causes Relate Justify Create Formulate Generate Hypothesize Reflect Theorize . . Fail Incompetent Misses point Incompetence aspect Prestructural one relevant several relevant integrated into generalized to independent aspects a structure new domain Unistructural Multistructural Relational Extended Abstract

Distinguish the kind of knowledge you want Declarative knowledge: . Knowing about things. Knowledge we can declare to someone in writing or telling e. g. ‘Distinguish between OBE and OBTL’ Functioning knowledge: . Knowledge we put to work in solving a physics problem, analysing a case study, designing a building, making an argument e. g. ‘Write an ILO for a subject you are currently teaching’

Designing Teaching/Learning Activities to Align with Intended Learning Outcomes Having designed the ILOs, we now need to activate the verbs or learning activities embedded in the ILOs by designing suitable Teaching/Learning Activities that will facilitate students achieving the ILOs.

Typical ILO Describe Explain Integrate Apply Solve problem Design, create Hypothesise Reflect Possible TLAs set reading, lecture, field trip tutorial, written essay project, assignment project, case study PBL, case study project, creative writing experiment, project reflective diary The point is not how you are going to teach but how and what you want your students to learn. NOTE! Many of these TLAs can be assessments tasks as well. Then you have excellent alignment.

Four common teaching situations and associated teaching and learning activities Situation Teaching activities LECTURE TUTORIAL Learning activities talk, explain, clarify listen, take notes, accept, query, discuss with peers, one-minute paper set/answer questions pre-read, prepare questions, provide feedback learn from peers, critique, analyse PROJECT set brief, ongoing feedback apply, create, self-monitor communicate, teamwork PBL set problems set learning goals, design, apply, accessing desired content, skills integrate, solve problems What teaching /learning activities will best facilitate your ILOs?

Designing Assessment Tasks (ATs) Steps: 1. Selecting a practicable task that embodies the target ILO verb. (Try the TLA first). 2. Making a judgment on how well the ILO has been met by the students' performance on the ATs – developing grading criteria.

Common ILOs Describe Explain Possible Assessment Tasks assignment, essay question exam, oral, letter-to-a-friend Integrate project, assignment Analyse case study, assignment Apply project, case study, experiment Solve problem case study, project, experiment Design, create project, experiment Reflect reflective diary, portfolio, self-assessment Communicate a range of oral, writing or listening tasks addressing the ILOs, e. g. presentation, debate, role play, reporting, assignment, precis, paraphasing, answering questions etc.

Examination is a very commonly used assessment task especially for large classes. We need to consider if 1. examinations involving answering essay type of questions under invigilated conditions is able to assess students' performance in some high level ILOs, e. g. apply, reflect, create etc. 2. there are other alternative assessments tasks which will more appropriately addess those high level ILOs.

Assessments Tasks for Large Classes Task-format Final exam Useful for Ensuring work is student’s own, overview of whole course Multiple choice Recognition, strategy, coverage Ordered outcome Hierarchies of understanding Poster integration, application, creativity Concept maps, Coverage, relationships Flow charts Three minute essay Different levels of understanding, sense of relevance Gobbets Interpret significant detail, explain Short answer Recall units of information, coverage Letter to a friend Integration, application, reflection

Assessment Issues 1. Assess the ILO or the Assessment Task itself? 2. Assess the task quantitatively (marking) or qualitatively (holistically)? 3. How to combine AT performances to obtain a final grade: quantitatively or holistically? The most important thing is that the assessment task is aligned to the ILO. The methods of assessing the task and of combining results into a final grade grading are important but secondary issues.

Example of Grading Criteria for an ILO Grade point/unit Marginal Pass D 1. 00 Satisfactory CC C+ 1. 70 2. 00 2. 30 Good B- B B+ 2. 70 3. 00 3. 30 Excellent A- A A+ 3. 70 4. 00 4. 30 ILO Reflect Able to use available As in “Good”. Able information to selfinformation to selfto generalize selfevaluate and identify evaluation to beyond limited aspects of own more aspects of own the full range of owm existing context. strengths and weaknesses strengthes and weak- Suggest ways of in a general sense. No in a general sense. Little nesses. Self-evaluation improving performevidence of suggestions application of theory in is based on theory. ance to real-life of ways to improve self-evaluation and limited Increasingly able to professional performance. No evidence suggestions of ways to suggest ways to contest. of theory being used in improve performance self-evaluation. in a specific context.

Grading the AT In other courses it may be more relevant to grade the task, which is in effect an ILO. For example, in language courses the ILO is that students can write an effective argument in English/Chinese, or in Science, that they can write a lab report.

Assessing quantitatively For: . . . Used to it. Seems to be the logical way to assess in certain courses. Logistically easy. Against: . Defines quality in terms of accumulating small quantities. . Measurement error also accumulates, thus invalidating fine discriminations. E. g. there is no valid difference between 74 and 75, yet to the student it can make a BIG difference an A or a B! Or worse, the difference between pass or fail. . Sends undesirable messages to students (backwash).

Assessing qualitatively For: • • • Student’s performance is appropriately assessed against what they are intended to learn – criterion-referenced. Backwash is positive. The final grade tells students what they have achieved and what they need for a better grade. Against: • • Requires a different mind set for some teachers. Initially more work in designing ILOs and suitable assessment tasks, but once established is not extra work.

Qualitative assessment involves making judgments against criteria (rubrics), not by counting ‘marks’ If ILOs are to reflect workplace or ‘real world’ standards it is more appropriate to assess them qualitatively. ‘Real world’ outcomes are not stated in terms of marks obtained.

An example of a holistic way of deriving final grade Curriculum and Instruction: A subject in a course for Ed. Psychlsts. ILOs: 1. 2. 3. 4. Apply the principles of good teaching and assessment to chosen contexts. Relate selected aspects of curriculum design and management to the educational system in Hong Kong. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist. Show examples of your reflective decision-making as an educational psychologist. Final grade: A B C F Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice. Awarded when all ILOs have been met very well and effectively. Awarded when the ILOs have been addressed satisfactorily, or where the evidence is strong in some ILOs, weaker but acceptable in others. Less than C, work plagiarised, not submitted.

Assessment: Curriculum and Instruction Show evidence that you have learned according to the criteria in the ILOs. Keep a reflective journal to record useful insights as a data base. A paper explaining how you would like to see the Hong Kong educational system implement a major educational reform. You should have ILO (2) in mind. A report specifically addressing ILOs (3) and (4), a review of those aspects of the course that you think will probably enhance your work as an EP. Your own rationale of your group presentation. You should have ILO (1) in mind. A self-evaluation showing how you have addressed each of the ILOs. Place in a portfolio of about 5, 000 words.

Use of OBTL in Hong Kong based on the Constructive Alignment Model At City U, 240 degree courses are committed and on the way: Faculty of Business: Faculty of Humanities and Soc Sci Faculty of Science and Engineering School of Creative Media School of Law Dept. Building Science and Technology 28 86 105 14 4 3 At Poly. U, well on the way Baptist U starting Hong Kong U, 4 faculties in PBL (a form of constructive alignment) and now moving more generally

Matching OBE and OBTL It's important that the taxonomy used at the institutional level is not only compatible with that used in the classroom but can actually guide the design of programmes and courses. Too many fragmented knowledge/ skill/ value/ societal outcomes make aligning TLAs and ATs to those outcomes difficult if not impossible.

A Suggested Taxonomy for Institutional level outcomes Declarative Outcomes: e. g. content knowledge expert. Functioning Outcomes: e. g. creative/innovative professional, life-long learner, professional expert, good communicator, team-worker. Value outcomes: e. g. ethical professional. Graduate outcomes need to be stated at an appropriate level (extended abstract in SOLO terms): e. g. ‘able to solve unseen problems‘. These graduate outcomes need to be teachable and assessable, usually in the Programme/Course ILOs.

Institutional, Progamme and Course Outcomes OBE OBTL Outcomes at the institutional level: Declarative, Functioning, Value Outcomes at the programme level: Above as appropriate to the programme. Outcomes at the course level: Programme ILOs embedded where appropriate as Course ILOs, to which TLAs and ATs aligned.

Reference Biggs, J. B. Teaching for Quality Learning at University. Buckingham: Open University Press/Mc. Graw Hill, 2003. New edition by Biggs and Tang explicitly addressing OBTL is in process, due late 2007.