InternshipPh D Opportunity PLA Participatory Learning Approach Michael

Internship/Ph. D. Opportunity PLA: Participatory Learning Approach Michael Bieber Jia Shen, Dezhi Wu, Roxanne Hiltz Information Systems Department College of Computing Sciences New Jersey Institute of Technology http: //web. njit. edu/~bieber - http: //is. njit. edu/pla/ September 2004 Bieber et al. , NJIT © 2004 1

Motivation • To increase learning of course content • Learning through active engagement – involve students as active participants – with the full problem life-cycle – through peer evaluation • Minimize overhead for instructors Bieber et al. , NJIT © 2004 2

PLA Process (e. g. , Participatory Exam) • • Each student creates 2 exam problems All entries Instructor edits the problems if necessary posted on-line Each student solves 2 problems Students evaluate (grade) the solutions to the problems they authored, writing detailed justifications Ph. D. students evaluate each problem a second time Instructor gives a final grade optional: Students can dispute their solution’s grade, by evaluating it themselves and writing detailed justifications Instructor resolves the dispute Bieber et al. , NJIT © 2004 3

Instructor Control Process Course Process. Design Flow: Learning from doing Set up on-line the PLAenvironment activities from Examadditional Processlearning Control reading Assigneverything ID peers write Edit questions Assign who answers questions Assign level-2 graders Determine Final Grades Resolve Disputes Bieber et al. , NJIT © 2004 Student Learning Process Make up problems Read Solve problems - other solutions - grade justifications - disputes Level-1 and Level-2 graders grade solutions Dispute final grade 4

Instructor Control Process Student Learning Process Confirmation Course Design Set up on-line environment ID, understand process Make up problems Read Exam Process Control Assign ID Edit problems Assign who solves problems Assign level-2 graders Determine Final Grades Resolve Disputes Bieber et al. , NJIT © 2004 Solve problems - other solutions - grade justifications - disputes Level-1 and Level-2 graders grade solutions Dispute final grade 5

Enjoyability (2 semesters with CIS 677) Cronbach’s Alpha=0. 68 Questions I enjoyed the flexibility in organizing my resources I was motivated to do my best work I enjoyed the examination process SA A N D SD Mean S. D. # 26. 2% 48. 9% 16. 7% 3. 6% 4. 6% 3. 88 1. 00 221 23. 5% 42. 9% 28. 2% 3. 4% 2. 1% 3. 82 . 92 238 17. 2% 42. 3% 22. 6% 10. 5% 7. 4% 3. 51 1. 13 239 SA - strongly agree (5 points); A - agree (4); N - neutral (3); D - disagree (2); SD - strongly disagree (1); the mean is out of 5 points; S. D. - standard deviation Bieber et al. , NJIT © 2004 6

Perceived Learning Cronbach’s Alpha=0. 88 Questions SA A N D SD Mean S. D. # I learned from making up questions 17. 9% 42. 5% 21. 3% 13. 8% 4. 5% 3. 55 1. 08 240 I learned from grading other students answers 17. 7% 48. 1% 19. 4% 9. 3% 5. 5% 3. 63 1. 06 237 I learned from reading other people’s answers 15. 8% 45. 0% 22. 1% 11. 3% 5. 8% 3. 54 1. 07 240 13. 6% 50. 2% 22. 6% 10. 9% 2. 7% 3. 61 . 95 221 21. 8% 49. 2% 25. 6% 2. 1% 1. 3% 3. 88 . 83 238 I learned to value other points of view 17. 6% 51. 9% 27. 6% 1. 3% 1. 6% 3. 82 . 81 239 I mastered the course materials 7. 4% 6. 9% 2. 7% 3. 54 . 84 188 I demonstrated what I learned in class My ability to integrate facts and develop generalizations improved Bieber et al. , NJIT © 2004 51. 6% 31. 4% 7

Recommendation: Do Again! Question Would you recommend in the future that this exam process used? SA A N D SD 20. 7% 40. 1% 24. 5% 8. 9% 5. 8% Mean S. D. # 3. 60 1. 10 237 Similar results for CIS 365: undergraduate file structures course using short essay questions (Fall 2002) Bieber et al. , NJIT © 2004 8

PLA Environment Software • • Guide the process Form groups Assign problem solvers, evaluators, dispute arbitrators On-line templates to ensure full entries Guide people to post entries in correct place Incorporate group process tools Handle problems as much as possible – Remind people who are late – Reallocate who does what • Based on a workflow management tool… Bieber et al. , NJIT © 2004 9

Extensions: Activity Support • Can we build in tools to support each activity – How to construct a problem – How to research and construct a solution – How to evaluate a solution – How to resolve a disputed evaluation – Technical term: scaffolding (educational literature) – Initial approach: inquiry-based support (ed. literature) Bieber et al. , NJIT © 2004 10

Extensions: Full Collaboration • Groups for: – Problems, solutions, evaluation, dispute arbitration • Requires group process support – Process: work on each activity together or separately, internal review – Grading of individual group members – Process Tools: brainstorming, voting, etc. Bieber et al. , NJIT © 2004 11

Extensions: Scope • Which activities? – so far: exams – what about: quizzes, homeworks, larger projects, in-class projects • Which problem types? – so far: short and long essay questions – what about: multiple choice, short answer, computer programs, semester projects – Sub-problems: • computer program design & implementation • semester project outline & execution Bieber et al. , NJIT © 2004 12

Extensions: Scope, cont. • Course Level – Graduate, undergraduate, secondary school (high school, junior high) • Disciplines – IS/IT, business, science, engineering, humanities, medical, all of secondary school Bieber et al. , NJIT © 2004 13

Extensions: Scope, cont. • Degree of Evaluation (assigning grades) – Currently: solutions – What about: • quality of problems • quality of evaluations/grades – All could be disputed • Degree of Participation – students could evaluate each – students could arbitrate disputes Bieber et al. , NJIT © 2004 14

Internship/Ph. D. Opportunity • Literature review and suggestions for all of the extensions • For more information on PLA see http: //is. njit. edu/pla/ Bieber et al. , NJIT © 2004 15