Dialog PLUS Innovative distancebased teaching deliverables from the

Dialog. PLUS Innovative distance-based teaching: deliverables from the DIALOGPLUS project University of California Santa Barbara Penn State University of Southampton Leeds University ASAP Seminar – 8 March, 2006 Dialog. PLUS Helen Durham, Louise Mackay, Oliver Duke-Williams, Phil Rees, Katherine Arrell and Riz Nawaz Chair: Dan Vickers ASAP Seminar 8/03/2006

Outline of presentation Dialog. PLUS Phil Rees Project aims, partners, deliverables - nuggets Louise Mackay Teaching and using e-learning materials Helen Durham AIG, Online Atlas and GPS – resources available across whole school Louise Mackay JORUM national repository Oliver Duke. Williams What is Shibboleth and how will it help? [Katherine Arrell] User of Penn State materials and codeveloper of GPS materials [Riz Nawaz] User of Upland Catchment Management materials, which he has enhanced and developed Phil Rees The value added by international collaboration ASAP Seminar 8/03/2006

Project aims • To develop e-learning materials for UG courses (and Masters) • To do this for four Geography Exemplar topics: Human Geography, Geomorphology/Environmental Management, GIS and Earth Observation • To experiment with collaborations with other universities within the UK and in the US • • • Exchanging topic materials within modules Exchanging generic materials Developing materials collaboratively Exchanging whole modules Exchanging students • To link these materials with digital libraries of resources (particularly those funded by JISC) • To preserve these teaching materials in a suitable repository (originally Alexandra Digital Library, now JISC’s JORUM repository) Dialog. PLUS ASAP Seminar 8/03/2006

Access Grid meetings cemented the partnership Soton: Hugh Davies Dave Martin Leeds: John Maber Andrew Booth Phil Rees Helen Durham Penn State: Steve Weaver David Dibiase In the past three months we have switched to Horizon Wimba Dialog. PLUS ASAP Seminar 8/03/2006

Nuggets and their sharing • I proposed the concept of nugget in the project proposal as a set of exchangeable e-learning materials • We spent time at first in trying to understand how this was different from a learning object • Learning object is simply too restrictive and closed a concept • A nugget is elastic: part of a lecture, a practical or a whole module • David Dibiase (Penn State) proposed the following definition Dialog. PLUS ASAP Seminar 8/03/2006

Characteristics of Dialog. PLUS “nuggets” Learning Activity Dialog. PLUS ASAP Seminar 8/03/2006

Characteristics of geography “nuggets” Supporting material Learning Activity Dialog. PLUS ASAP Seminar 8/03/2006

Characteristics of geography “nuggets” Supporting material Learning Activity Self assessment Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS ASAP Seminar 8/03/2006

Developing and teaching e-learning materials The Dialog. Plus (D+) project materials: Ø Full courses from e-learning materials Ø Individual learning activities for embedding - census nuggets Ø Generic learning materials – academic integrity nugget Ø Learning materials developed via Collaborative Learning Activity Design (CLAD) …some of these with our project collaborators (UK and US) What I’ll be looking at: Full courses delivered at SOG using developed e-learning materials Delivery using a mix of face-to-face/distance-based learning styles Why? Provide insight to the introduction and teaching of e-learning courses in the undergraduate curriculum… Dialog. PLUS ASAP Seminar 8/03/2006

What courses have we taught? The courses: Geog 2750 - Earth observation (Earth observation and GIS of the Physical Environment) Geog 3870 - Upland Catchment Management EO: Introduce the physical principles of Earth observation and its role in physical Geography applications UCM: To provide students with appropriate knowledge & skills to enable effective management of UK upland catchments Note: EO and Environmental Management core Geography sub-areas of the D+ project, material useful for Leeds and collaborating partners Dialog. PLUS ASAP Seminar 8/03/2006

Courses breakdown… EO Ø Ø Level 2 module (1 semester) 2003 – delivered f 2 f - 10 lectures and 8 practicals 2004 – online lectures and distance taught maintaining on-campus practicals Students: 2003 – 36, 2004 - 56 UCM Ø Ø Dialog. PLUS Level 3 module (1 semester) Delivered for last three years Combination of face-to-face and online teaching Students: 2004 - 24 , 2005 - 10, 2006 - 29 ASAP Seminar 8/03/2006

Course content and delivery Content & Delivery: EO Ø 10 online lectures and self assessments (Nathan Bodington each Monday) Ø 8 on-campus practicals (demonstrator support) Ø End of term face-to-face sessions Ø Email and discussion room with tutor support UCM Ø 7 online lectures (placed on Nathan Bodington each Monday) Ø 2 face-face sessions (extra optional face-face sessions every 3 weeks) Ø 1 field trip to Wharfedale Ø Email and discussion room used by students Assessment: For both - formative (MCQ’s, worksheets, practical assessments ) and summative (project reports, poster presentation, exam) Dialog. PLUS ASAP Seminar 8/03/2006

Delivering the on-line lectures through the Leeds VLE The on-line lectures (plus additional resources) were delivered using the Leeds Bodington Common VLE. Course “room” resources: Course introduction; Distance learning advice; Weekly lecture; Practical material; Discussion and FAQ room; Lecture MCQ test; Additional Resources; Miscellaneous Resources Dialog. PLUS ASAP Seminar 8/03/2006

How did we structure the materials? What were the drivers for the style and learning objectives? Ø Ø Material driven VLE driven Project driven Availability of suitable staff Lecture structure: On-line Lectures structured in a similar manner as a traditional face to-face lecture with an introduction, aims, signposted sectioned content but come with more detail…. . Dialog. PLUS ASAP Seminar 8/03/2006

Examples of course materials… On-line Lectures - have a distinct “tone” to that of face-to-face lectures in order for students to work unaided. The on-line versions have been adapted to add more to the learning experience by using graphical examples and for example in the EO course external links to image sources and tutorials and interactive tests… with the suggested readings these tools consolidate the introduced topics and direct self-study. Lecture examples Dialog. PLUS MCQ example ASAP Seminar 8/03/2006

Materials may consist of individual learning objects… Learning Activity 1: Click on this video footage to the right and then identify the sources of errors in the stage-discharge method for river flow estimation Dialog. PLUS ASAP Seminar 8/03/2006

Why undertake such an endeavour? Ø Delivery of knowledge and skills to the students on Earth Observation and Catchment Management; Ø Experimentation with a mixture of face-to-face and distance tutoring on an UG module; Ø Preservation of learning materials from year to year/preservation of a course in light of staff changes Why discuss these two courses? Translating traditional courses to on-line courses (with the added challenge of staff changes) Turning a Geography tutor into a Geography e-tutor Is this an easy translation for the tutor, student and the managing department? We shall see…. . Dialog. PLUS ASAP Seminar 8/03/2006

Evaluating the e-learning courses… The courses were both evaluated after 2004/2005 run (with the help of Karen Fill – our Southampton pedagogic expert) A combination of: student questionnaire; group discussion; tutor interview. and as we had the results from the complete f-2 -f version of the EO module run the previous year from the on-line version… statistical analysis of inter-year marks (assessments and examination) What did we learn from this evaluation? . . . Dialog. PLUS ASAP Seminar 8/03/2006

Evaluating the e-learning courses - student questionnaires For both courses None of the students had studied by distance learning before For the questionnaire the students were given statements regarding the content and delivery of the course and asked to score according to their level of agreement (0 – no, 1 – somewhat, 2 - yes, and N/A). A mean agreement score was calculated for each statement. Example of questionnaire statements on style and content Dialog. PLUS ASAP Seminar 8/03/2006

Evaluating the distance-based EO course - summarising student feedback Particularly positive with respect to: - description of content and learning objectives - inclusion of required tools - accessibility of linked resources - appropriateness of the MCQs Particularly negative feedback about: - their own motivation as a result of course delivery; specifically the lack of face-to-face contact (with tutor & fellow students). After some discussion (in groups): Majority Best: being able to work in their own time Majority Worst: lack of face-to-face contact Dialog. PLUS ASAP Seminar 8/03/2006

General questionnaire and discussion comments: s of tion c a r t Dis eb. the W wers s n a l Emai ded ee are n a day. n withi Su – p rpri ho rac sing tim wev tical ly g e. er i se ood n k ssi w ee on ay pin s w of MC g m ere lea de Qs e f vit rnin ad co oc al g lin ul us es d s ed ; h ta /on elp y u ful p a for fte re r lec vis tu ion re. Di for stan ma ce l the ear ma nin tic g – al Di co diffi s c no us nc cu ep lt being face-tot s a Note - The lack of motivation and lack of contact toiothe delivery ex due ts. l w n pla a b y face were the types of comments that were repeated oa students across many in bys many p rd o o ne statements. /em s s se i b lf. le ails to Dialog. PLUS ASAP Seminar 8/03/2006

Student feedback summary… Specific Student requests: UCM 2004: Include Field Trip 2005: More optional face-to-face sessions EO 2004: More face-to-face support For both…. . more interactive activities! Yet some conflicting student feedback… Online discussion proved popular with the students in one class, but completely hated it in another - citing lack of anonymity and distrust of peer’s knowledge Dialog. PLUS ASAP Seminar 8/03/2006

What are some of the issues that have arisen? Student Expectation: Ø Students disconcerted by the lack of on-campus support and tended to undermine their own ability, i. e. , perceived that they hadn’t understood difficult technical topics What could have given rise to this problem? Ø Students not familiar with distance-based learning Ø Lack of suitable advertising (positive reinforcement) of the course and its delivery style – especially for the inexperienced undergraduate! Dialog. PLUS ASAP Seminar 8/03/2006

Evaluating the distance-based EO course - analysis of class marks Following examination (February 2005)… Module marks tested (t-test) by gender and by year (2004 Vs 2005) Statistical analysis of overall marks obtained by students in 2003/04 and 2004/05 revealed: Ø Ø no statistically significant differences year on year for all students; no statistically significant differences by gender within either year or across the years; And in some instances individual assessment marks were better for the distancebased year! Note - there is no statistical anomaly for female students for individual practical marks or overall module marks. Dialog. PLUS ASAP Seminar 8/03/2006

What are the benefits & limitations for the e-tutor? Overall distance-teaching aspects: Main advantages Ø Ø Materials can be delivered and learnt asynchronously Some working efficiency - students problems/queries can be dealt with in “batches” Main disadvantages Ø Ø Dialog. PLUS Number of student queries can be large and time consuming Lack of suitable training/administrative support for the course Labour intensive initial production (For the distance tutor) a feeling of isolation from the rest of the faculty ASAP Seminar 8/03/2006

What lessons can we give to the aspiring e-tutor? 1. Allay student fears with regards to delivery How can we do this? - positive advertisement (e. g. , student flexibility, one on one support with an expert via email) and at a time suitable for student’s module choices (especially for the inexperienced undergraduate) 2. Allay student fears with regards to their ability How? – provide back-up on-campus “technical” support 3. Improve tutor support How? – Faculty level training for on-line teaching; improve administrative/tutor communications – be clear on who does what specifically when distance-based. 4. Mix up delivery styles How? – Have a blended approach – combining f-2 -f and on-line delivery To run successful e-learning courses… advertise, support, provide on-campus backup, blend delivery Dialog. PLUS ASAP Seminar 8/03/2006

Where are they now? On Nathan Bodington Other forms of storage: – Outward facing project website – Book with CD – E-learning in Geography Long term storage: – Benefit to larger HE/FE community. . . talking about this a little later… Dialog. PLUS ASAP Seminar 8/03/2006

But for today… Developed courses can be viewed at our drop-in sessions: Ø See the courses and how to access them Ø Pointers for aspiring e-tutors The full courses - Earth Observation and Upland Catchment Management in Geog. East G 13 – 2 pm to 4 pm today! and now…introducing our other materials: stand-alone learning activities, generic learning materials – some designed using Collaborative Learning Activity Design (and with international collaboration). Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS - Generic resources Resources available to staff and student communities throughout So. G: • Academic Integrity • Global Positioning Systems • Online Census Atlas Dialog. PLUS ASAP Seminar 8/03/2006

Academic Integrity Guidelines for Geographers • Plagiarism: on the rise? • Learning material originated at Penn State University • Adapted by Southampton and Leeds – joint paper on Repurposing a learning activity on academic integrity: the experience of three universities submitted to JIME • School resource available on Nathan Bodington as web-based material or PDF • Includes context-based guidelines, in accordance with the University of Leeds rules and regulations • MCQ available for assessment Dialog. PLUS ASAP Seminar 8/03/2006

Contents of AIG nugget Dialog. PLUS ASAP Seminar 8/03/2006

Examples of Multiple Choice Questions Dialog. PLUS ASAP Seminar 8/03/2006

Global Positioning Systems – Collaborative design case study • Collaborators: David Di. Biase (PSU), Katherine Arrell (Leeds) and Helen Durham (Leeds) • Collaborative Learning Activity Design(CLAD) – Stages of design: Audience Learning outcomes Learning activities Nugget structure Used Concept. Vista to design 4 nuggets. Dialog. PLUS ASAP Seminar 8/03/2006

Visualising nuggets and collaboration Dialog. PLUS ASAP Seminar 8/03/2006

Understanding and using GPS – developed at Leeds Dialog. PLUS ASAP Seminar 8/03/2006

GPS nuggets developed by Penn State Dialog. PLUS ASAP Seminar 8/03/2006

Contents of PSU GPS nuggets Dialog. PLUS ASAP Seminar 8/03/2006

Online Census Atlas • Simple Atlas to allow visualisation and exploration of Census variables from 1971, 1981, 1991 and 2001 • Discrete census years or inter-censal data • 88 variables, common across all Censuses • Data simultaneously displayed in conventional, generalised boundaries and population-based cartogram (Universal Data Map) developed by Danny Dorling and Helen Durham • Available on Manchester server at www. chcc. ac. uk/atlas (requires Athens authentication) • Cartogram-only version available at www. ccg. leeds. ac. uk/teaching/chcc • 3000 -word Observation piece accepted by Area journal Dialog. PLUS ASAP Seminar 8/03/2006

Online Census Atlas – selection of map variables Dialog. PLUS ASAP Seminar 8/03/2006

Percentage change over time in age group 20 -24 year olds by Local Authority, 1981 -2001 Dialog. PLUS ASAP Seminar 8/03/2006

Discussion points • How to make more GPS nuggets accessible when out in the field? Suggestions? • Should the Academic Integrity MCQ be compulsory? Demonstrations of these three nuggets are available today in G. 04 East Building Time: 2 -4 pm Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS ASAP Seminar 8/03/2006

The storage and reuse of e-learning materials What are we doing with these materials long term? Local At the local level – Leeds VLE Project At the project level – website and book with CD National At the national level –storage on JISC JORUM digital repository (http: //www. jorum. ac. uk) International At the international level – possibly the US Alexandria Digital Library or other? For the moment – introducing…JORUM Dialog. PLUS ASAP Seminar 8/03/2006

– what is it and who is it for? What is JORUM? JORUM is a new JISC funded digital repository allowing institutions and project teams to share learning and teaching materials with colleagues in the UK; to collect and share learning and teaching materials, allowing their reuse and repurposing, and standing as a national statement of the importance of creating interoperable, sustainable materials. Who can use JORUM? All HE/FE institution academic and support staff… Where do the materials come from? Centrally funded e-learning projects • JISC Projects (e. g. X 4 L Phase 1 & 2, Distributed e-Learning projects i. e. , DIALOGPLUS) • HE Academy Centres of Excellence in Teaching and Learning (CETLs) UK Further and Higher Education • Individual institutions • Institutions working as part of a Consortium Working with other organisations/programmes • Other JISC services e. g. Digital Curation Centre • HE Academy Subject Centres AND hopefully you! Dialog. PLUS ASAP Seminar 8/03/2006

What type of objects does JORUM support? Whatever is contributed by the community: Learning Resources That teaching staff can use with learners, in blended learning, classroom or online learning activities. Teaching Resources Designed to support teaching staff e. g. , tutor guides, lesson plans, schemes of work and staff development materials. Materials of varying size: Single assets (documents, images, video clips, diagrams); Learning units made up of several elements (illustrations, articles, lectures, assessment exercises, etc). Quality control – produced by teaching staff only – but can be targeted to tutor or student Dialog. PLUS ASAP Seminar 8/03/2006

- how do we use it? The JORUM repository is a searchable online library of learning and teaching resources. The system contains resource metadata allowing users to search or browse for resources using specific educational classification systems and vocabularies. Search by: title, keywords, contributor, resource type, user type & copyright Preview metadata Preview resource Download Who is managing JORUM? JORUM is an online service offered by the JISC-supported national data centres, EDINA and MIMAS. Dialog. PLUS ASAP Seminar 8/03/2006

- how do we use it? Firstly – need to register with JORUM All HE/FE institutions can register for JORUM contributor log-ins – only after institutional licence agreements are signed (thereby allowing your intellectual property to be used by others – for free!) Once registered contributors can start to deposit materials… Points to address prior to depositing: Applicable – remove local context Exchangeable – provide instructions Exportable – remake into an interoperable format Packaged – bundle multiple items together to make one resource Use a content repackager such as RELOAD (http: //www. reload. ac. uk) Dialog. PLUS ASAP Seminar 8/03/2006

- how do we use it? How do we go about depositing? Once logged into JORUM - on contributor service interface Provide metadata and subject classification data - to aid cataloguing Licence and publish to the service – catalogued by JORUM and added to user service Deposit – upload, publish, catalogue – over to the user Jorum Contributor “Putting content in” Jorum User “Getting content out” JORUM - once deposited your learning resource becomes available to registered users Dialog. PLUS ASAP Seminar 8/03/2006

The upshot of using a repository like JORUM What is the long term benefit of storage on such a repository? Use in wider FE/HE community At the moment the service is in its infancy. . . there are further issues to address… What are the current disadvantages of storage on JORUM? Issues of sustainability Materials may become dated Is this is an issue for JISC to address? Long term – of course Lack of access for international collaborating partners We can receive but not give…JISC taking this point onboard for future JORUM development BUT IN THE MEAN TIME – the repository is of benefit to us now Project materials are not lost and are also released to the wider community NOTE - The JORUM URL is: http: //www. jorum. ac. uk Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS ASAP Seminar 8/03/2006

Providing access to material • How it works in theory • How it works in practice – Student at Leeds to take course at PSU Dialog. PLUS ASAP Seminar 8/03/2006

How it works: in theory Student Leeds User 1 Access request WORK 4 User credentials WORK 3 Login request 2 a Origin discovery 8 Resource Course module 2 Authentication request PSU W W A Y F Y F F User’s host Institution VLE Bodington Common Shibboleth Identity Provider 5 Authentication response 6 Attribute request Shibboleth Service Provider 7 Attribute response Dialog. PLUS ASAP Seminar 8/03/2006

Identity Provision and trust • Members of Federations trust each other • But… – Legal frameworks seem to dictate that Federations are constrained by national borders • Therefore… – Service Provider has to trust Identity Provider Dialog. PLUS ASAP Seminar 8/03/2006

Service Provider fears… • Infrastructure – How reliable is the service? – Who can we call if there are problems? – How can we test? • Policy – Who has passwords? – What security policies are imposed? – Whose policies are relevant? Dialog. PLUS ASAP Seminar 8/03/2006

“Is everything ready? ” • Shibboleth an ‘emerging technology’ – Software, terminology and methods are subject to change • Lots of metadata and configuration information to exchange • Biggest hurdle is human aspect – finding the right people to talk to Dialog. PLUS ASAP Seminar 8/03/2006

Oct 21 st – “Clearly you [Leeds] guys trust your ID management system and are currently supporting your environment of higher stakes applications than the one that is represented by this activity for the WUN. … The real risk of malfeasance in this case is almost nonexistent. … I say we accept it. ” – “We … are also not aging now. However we happen to be working in a pilot project with our federal government, and they are insisting that we age passwords…” Dialog. PLUS ASAP Seminar 8/03/2006

Nov 4 th – “we have completed the final configuration of the Shibboleth Production Server and successfully tested with Leeds. We are ready for the Leeds student to participate in Penn State's GEOG 485 via shibboleth. ” Dialog. PLUS ASAP Seminar 8/03/2006

Nov 5 th : Security in the real world – “A wrinkle has arisen, however…The Leeds student … recently reported that his computer was lost in a burglary. ” Dialog. PLUS ASAP Seminar 8/03/2006

Dialog. PLUS ASAP Seminar 8/03/2006

Lessons/Reflections: Students • Students are comfortable in the use of on-line learning in a distance Masters (otherwise they would be on F-to-F programme). • They want reliable, linear materials but with plenty of activity (I’m not sure they are ready for concept maps). • Undergraduates are not so comfortable. “I paid for a face to face course not a distance course. ” (This may simply be a natural reaction to an innovation. ) • David Dibiase: we should justify at least one on-line module to UGs as a preparation for life-long on-line learning after leaving university Dialog. PLUS ASAP Seminar 8/03/2006

Lessons/Reflections: Staff • Very difficult to share nuggets within modules/courses • Easier to share generic material (e. g. Academic Integrity) • We need to learn how to share students (have our students do a course or two at another institution as part of their degree) Dialog. PLUS ASAP Seminar 8/03/2006

Sharing Programmes using SHIBBOLETH • We are engaged at Leeds in experiments with this new software for linking users and resources at different institutions • JISC is funding dozens of such experiments with the software (another Digital Libraries project has already used it to connect LSE students with a Columbia U course) Dialog. PLUS ASAP Seminar 8/03/2006

Conclusions: reflections on international collaboration • US is substantially more advanced than the UK in developing online learning • So we have probably learnt more from the US than the reverse • However, we have delivered far more in terms of learning materials in the UK (Soton and Leeds) than our US partners • We are working on an edited book that will point to the project resources and draw out lessons for the development of e -learning for Geographers Dialog. PLUS ASAP Seminar 8/03/2006