Computing Education Research Anders Berglund Informationsteknologi Uppsala universitet

Computing Education Research Anders Berglund Informationsteknologi Uppsala universitet Uppsala anders. berglund@it. uu. se

Informationsteknologi Computing education research at Uppsala University n How do our students understand computer science concepts? n How to teach computer science? Anders Berglund, Department of Information Technology

Informationsteknologi Computing Education Research This talk n What can we say about how grading in an internationally distributed project-based course? and n Why and how to do research in computing education? Two parallel stories Anders Berglund, Department of Information Technology My secret agenda: Inspire to Computing Education Research

Informationsteknologi Computing Education Research 1. Background • • 2. Results concerning grading • • • 3. A research approach? Phenomenography The setting The teachers’ grades The students’ peer evaluation The experience of being graded Results concerning grading Summary • • • Computing Education Research Some research results Choosing a research approach Anders Berglund, Department of Information Technology

Informationsteknologi What does it mean to learn something? Unfortunately (? )…. n General case: A “meaningless” question n It all depends on “what you mean by learning” or “how you see things” Anders Berglund, Department of Information Technology

Informationsteknologi Quantitative/Qualitative research n Quantitative research is grounded on • n “… the assumption that features of the social environment constitute an objective reality … collecting numerical data on observable variables” Qualitative research is grounded on • “… the assumption that individuals construct a social reality in the form of meanings and interpretations. … studying … intensively in natural settings” Implications for the role of the researcher, the concept of evidence, interpretation etc. (Gall, Borg & Gall, 1996) Anders Berglund, Department of Information Technology

Informationsteknologi A research approach/methodology/ framework n Offers a way to perform research in learning n Organizes “ways to see things” n A lens with a certain focus n n With a specific research approach: Some issues get clearer, others blurred Offers theoretical stand on learning, ways to see possibilities and limitations, opens to communicate with other researchers etc. Anders Berglund, Department of Information Technology

Research approach/methodology/framework Phenomenography Informationsteknologi (Marton & Booth, 1997) n Takes the learners’ perspective. n Aims at analysing and describing the variation in students’ experience (understanding, learning). n A empirical, qualitative research approach n Data is often collected through interviews n Outcome: A few qualitatively different ways, in which something is understood within a student cohort Examples: TCP, experience of being graded Anders Berglund, Department of Information Technology

Informationsteknologi Phenomenography The students study TCP The researcher studies the different ways in which the students understand TCP Researcher Anders Berglund, Department of Information Technology

Informationsteknologi Why the learners’ perspective? n A teacher’s understanding of how her students understand learn about something (CS concepts, for example TCP) is a good tool for improving teaching. n A change that is not perceived as “good” by the students does not improve learning. · Example: Grades are not the driving force for most students who take (a certain) project course. Anders Berglund, Department of Information Technology

Informationsteknologi Computing Education Research 1. Background • • 2. Results concerning grading • • • 3. A research approach? Phenomenography The setting The teachers’ grades The students’ peer evaluation The experience of being graded Results concerning grading Summary • • • Computing Education Research Some research results Choosing a research approach Anders Berglund, Department of Information Technology

Informationsteknologi Background: The Runestone initiative A project course in computer systems n n A project-based course in distributed systems, real-time programming and computer communication. Third/fourth year students majoring in CS. International collaboration for students who do not go on exchanges. Experience of collaboration over ICT tools. Anders Berglund, Department of Information Technology

Informationsteknologi Project course in computer systems The Runestone project USA Communication by e-mail and chat • 3 + 3 advanced CS students per team • 16 teams in total • No lectures • Tutoring by e-mail and chat Anders Berglund, Department of Information Technology Sweden

Informationsteknologi Student project n n n Student project: Produce a software system to control a (modified) Brio labyrinth from any Web-browser. The task demands computer communication solutions. The task requires collaboration within the team of 6. Anders Berglund, Department of Information Technology

Informationsteknologi Grading in Runestone n n n Both the process and the project are graded, in relation to the teams’ own plan Process grade is based on weekly meetings Components of grade: • • n n Team performance, in relation to the team’s own plan Individual contribution Peer evaluation The instructor’s decision. Team members are graded by “their” instructor Different grading schemes in Sweden and US • • Sweden: pass/fail US: A to E Anders Berglund, Department of Information Technology Problematic? ? ?

Informationsteknologi Analysing the grading in Runestone n n n Teachers’ distribution of grades (quantitative) Peer evaluation The students’ evaluation of each others’ contributions (quantitative) Students’ experienced purpose of being graded (qualitative) Anders Berglund, Department of Information Technology

Informationsteknologi Grades n Grades awarded by the instructors, according to the Runestone scheme (Max = 100, Pass ≈ 60) To all students 83, 61 To Americans 81, 55 To Swedes 85, 05 Anders Berglund, Department of Information Technology

Informationsteknologi Peer evaluation n Each student awarded USD 120. - to his team-mates From Swede to Swede 22, 25 From Swede to American 18, 79 From American to American 20, 07 From American to Swede 20, 07 Then, what is the driving force? Anders Berglund, Department of Information Technology

Informationsteknologi The experienced purpose of being graded Getting a good grade … Focus is on 1. … has a value on its own The grade per se 2. … is a tool to reach other aims The benefits of a good grade Cat 3. … is sub-ordinated to other aims Anders Berglund, Department of Information Technology Me and the team My team and other teams

Informationsteknologi Results on grading n Getting a good grade is not the driving force for most students in this project. n ”Me in the team” or ”My team in front of other teams” is often important. n How generalizable are these results? • n Research in computing education research are normally situationally bound How can we use this in our teaching? • Both ”hard” results and insights gained by doing the research are useful. Anders Berglund, Department of Information Technology

Informationsteknologi Computing Education Research 1. Background • • 2. Results concerning grading • • • 3. A research approach? Phenomenography The setting The teachers’ grades The students’ peer evaluation The experience of being graded Results concerning grading Summary • • • Computing Education Research Some research results Choosing a research approach Anders Berglund, Department of Information Technology

Informationsteknologi On Computing Education Research n n n Theoretically sound research in students’ learning in Computing can serve to improve teaching. Different research approaches offer various contributions to our understanding of students’ learning. The perspective on ”reality”, what can be studied, what can be known, what the researcher’s role is, how research is performed etc. varies. This talk: Some examples of qualitative research Anders Berglund, Department of Information Technology

Informationsteknologi Example, research approach Constructivism n n A family of traditions Jean Piaget, 1896 - 1980 • • n Reality is rejected or irrelevant Knowledge is constructed by each individual No firm methodology Passive learning will fail Extremely influential in school teaching Anders Berglund, Department of Information Technology

Informationsteknologi Empirical results from constructivism in CS Education n Students construct rules for parameters. They are only sometimes successful. (Fleury, 1991) Students construct their own understanding of variables. (Paz, 1996; and others) Software visualization in itself does not help students understanding (Mulholland, 1997) Anders Berglund, Department of Information Technology

Informationsteknologi Applications of constructivism in Computer Science Education n n n Think twice when using visualizations Explicitly teach the model of the computer Don’t start with abstractions Teach planning, teach to avoid “bricolage” Don’t run to the computer Organize “closed labs” (from Ben-Ari, 2001) Anders Berglund, Department of Information Technology

Informationsteknologi Example, research approach: -cultural perspective A socio Thinking/Learning is not influenced by the environment ⇔ Thinking/Learning is an interaction between the individual and the environment n n A family of traditions in research into learning Common “source of inspiration”: Lev Vygotsky (1896 – 1934) Anders Berglund, Department of Information Technology

Informationsteknologi Applications of the soci-cultural tradition n Why do teams of students interpret a programming task so differently? (Holland & Reeves, 1996) Why do our students hand in “incorrect” programs? (Ben-David Kolikant, 2005) The example: Open source community • Linux Anders Berglund, Department of Information Technology

Informationsteknologi Example, research approach: Critical enquiry n Critical enquiry (research with “a mission”, often to address power imbalances) • n n Feminist research Cannot be defined in ontological stand or research methodology Instead, it questions fundamental principals and values and aims to implement changes. Anders Berglund, Department of Information Technology

Informationsteknologi Feminist research, an example of critical research n Are there factors within computer science itself, that preserves the currently dominating gender structure? (Björkman & Trojer, 2002) “We consider it of vital and decisive importance that gender research is done from within computer science. ” “Such research on the core of computer science and its knowledge production would serve to enrich computer science as well as education within computer science. ” Anders Berglund, Department of Information Technology

Informationsteknologi Selecting a research approach Why phenomenography? A personal view n Appropriate for the research questions • • n “Close to” learners • • • n • • Results “talk to” computer scientists Supports deployment of results in teaching Relevant for recognition of CSE research within CS Competence at hand/tradition • n Students’ perspective A way for students to “talk” to teachers/organisers Data stems from individuals Computer science is in focus • n “Complex” “answers” desirable Statistical “answers” hard or impossible to get Shirley Booth I like it Anders Berglund, Department of Information Technology Read! Create a network!

Informationsteknologi Literature Read! n Clancy, M. , Stasko, J. , Guzdial, M. , Fincher, S. , & Dale, N. (2001). Models and Areas for CS Education Research. Computer Science Education, 11(4), 323 -341 n Fincher, S. , & Petre, M. (2004) Computer Science Education Research, London, UK: Taylor & Francis n Berglund, A. , Daniels, M. and Pears, A. (in press). Qualitative Research Projects in Computing Education Research: An Overview. To appear in the Proceedings of the 8 th Australasian Computer Science Education Conference, Hobart, Australia. n Berglund, A. 2005. Learning computer systems in a distributed project course: The what, why, how and where. Acta Upsaliensis Universitatis. Uppsala Dissertations from the Faculty of Science and Technology 62 Anders Berglund, Department of Information Technology

Ce. TUSS Informationsteknologi n n Ce. TUSS ska verka för anpassning av teknikutbildning till samhällets och studenternas behov. Genom att utveckla och sprida information om lärandemiljöer som är • • n Create a network! Personligt meningsfulla Socialt relevanta Gränsöverskridande Baserad på samarbete (lokalt och internationellt) Workshops, kurser etc. Anders Berglund, Department of Information Technology

Informationsteknologi Anders Berglund, Department of Information Technology