What We Knowand What We Need to Know

What We Know…and What We Need to Know Roy D. Pea Center for Innovations in Learning Stanford University NCTET-2002 “A National Summit on Education Technology” January 25, 2002

Structure of the Talk – – Harnessing powers of technology for learning What have we discovered in the field? Need to move from fragmentary knowledge to systemic understanding Redefining the roles of research • • • Focused on issues educators have Stronger designs for research Partnership projects with policy relevance Industry engagement Inventing the future of e-learning NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Megatrends setting the stage – K-12 education • • • – Learning sciences research • • – Higher standards Increasing accountability and ESEA reauthorization Exceptional teacher workforce transformation Extraordinary policy and business attention Tech infrastructure, $10 Bil E-Rate: How to leverage for learning? Research to guide advances in curriculum, pedagogy, teacher learning, assessment Partnership projects bridging theory and practice Industry • Moore’s Law, Metcalfe’s Law • Miniaturization, portability, cost, digital convergence, bandwidth, ASPs New platforms: hand-helds, thin clients, wireless networks Rapid growth of E-learning industry, E-services but no clear K-12 “marketplace” • •

Power of technology to support learning – – – Real-world contexts for learning Connections to experts and communities of learners Visualization and analysis tools Scaffolds for problem solving Opportunities for feedback, reflection & revision Teacher learning How People Learn (Bransford, Brown, & Cocking, Eds. ) National Academy Press, 1999 NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Many exciting and dramatic visions and results from K-12 elearning – – – Virtual fieldtrips and inquiry “quests” teleport students to remote places and engage learning Teacher communities learning online together about best practices Learning environments leveraging scientific understanding to improve access to reading Student-scientist partnerships connect learners to powerful tools and distributed models for doing science and understanding the environment Simulations and dynamic graphing tools bringing calculus understanding to urban middle school students 1: 1 E-Learning models offering teachers better assessment information to guide instructional practices NCTET’ 02 National Summit Stanford University © Professor Roy Pea

What we have discovered in the field – So technology can lead to learning improvements …but does it? – Mixed results and difficult to generalize across thousands of studies (e. g. , PCAST, SIIA, PITAC) • Hardware, software, and their pedagogical uses vary tremendously across schools • Successful uses of technology typically accompanied with other reforms and have their consequences in the complex system of educational practices and organizations • Rigorous longitudinal studies to document isolated technology effects are rare since expensive and difficult to implement NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Why? Technology is but part of a system – Only one element in a coordinated, systemic approach to educational improvement – Standards, curriculum, pedagogy, assessment, teacher development, school culture and school-home connections are fundamentally part of any systemic change – …and instrumental in the roles technology can play and its likely effectiveness NCTET’ 02 National Summit Stanford University © Professor Roy Pea

How learning is organized (Education Systems) Coherence across levels & incentives Content standards Instructional workforce capacity Policy level School/district level Teacher level Why people learn (Socio-cultural context) How people learn (Cognition) Student level What people learn (Content) (From Nora Sabelli, SRI International)

What we don’t know – How to conditionalize our answers to educators’ questions of how to make technology effective in improving student learning, e. g. , • • • – By age and developmental level By subject matter and topic By classroom context, including many ESL learners By school culture By district, community, state contexts Expertise is defined in part by conditionalized knowledge NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Our “research answers” are hard to adapt to questions based in local situations – – – Fragmentary research knowledge today requires the implementers of technologies in educational systems to figure out how to make them work in local conditions So far this is more craft than science We must make progress on the issues of scale for powerful e-learning applications: • • • Not only about numbers But the encompassing of diversity (learners, teachers, schools) And providing an enabling web services infrastructure for ongoing adaptive learning NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Conceptual Framework for Analyzing Education with Multi-Level Modeling From R. Rumberger (2000) SCHOOL LEVEL CLASSROOM LEVEL STUDENT LEVEL School Inputs Structural characteristics Student composition Resources (technology) School Processes Decision-making (using technology) Academic &Social Climate School Outputs Engagement Learning Achievement Classroom Inputs Student composition Teacher background Resources (technology) Classroom Processes Curriculum Instructional strategies (using technology) Classroom Outputs Engagement Learning Achievement Student Background Demographics Family background Academic background Student Experiences Class activities Homework Use of computers Student Outcomes Engagement Learning Achievement

Research and practice connections 20 years on, how did we get to a state of so much research on learning technologies that is so weakly aligned with what we need? – Our problem is the model for how research relates to practice – NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Linear flow model The usual means of knowledge transfer through “dissemination” has rarely worked for bringing research to bear broadly on practice* * Source: Donovan, M. S. , Bransford, J. D. , & Pellegrino, J. (1999, June). (Eds. ). How People Learn: Bridging Research and Practice. Washington, DC: National Academy Press. (Co-author). NCTET’ 02 National Summit Stanford University © Professor Roy Pea

NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Reciprocity-of-influence model * Source: Donovan, M. S. , Bransford, J. D. , & Pellegrino, J. (1999, June). (Eds. ). How People Learn: Bridging Research and Practice. Washington, DC: National Academy Press. (Co-author). NCTET’ 02 National Summit Stanford University © Professor Roy Pea

So—expand “use-relevant” research – Use-driven research vs. Curiosity-driven • • • – Need more focus on issues faced by educators Need greater relevance for policy and educator decision-making Need productive roles for industry partnership and engagement But also recognize values of different roles for research in e-learning technologies “pipeline” • • • Exploratory and proof of concept, can lead to creation of demand Small-scale investigations Large-scale systemic studies in diverse settings NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Need for a strong national “userelevant” (actionable) research program “No information system of database maintained today, including the National Educational Longitudinal Study (NELS), and the National Assessment of Educational Progress (NAEP), has the design and content adequate to answer vital questions about technology’s availability, use and impacts on student learning. NAEP…is flawed as a data source for relating achievement to technology availability and use” (Haertel & Means, 2001). NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Making stronger use-relevant research – Haertel & Means (2000): • – – – Aggregated insights from ten research methodology experts on national ed-tech R&D improvement Many called for arrangements where related studies are conducted in “test-beds”or networks of “sentinel schools” Aim to provide evidence of emerging trends and/or offer sites that have agreed to participate in sustained studies of the effects of technology Called for “intermediary organizations” to review existing research, identify research questions, synthesize results, create templates for data collection instruments, support local researchers NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Missing key infrastructure elements for stronger research designs – – – More meaningful assessment measures for 21 st century skills Mechanisms like consortia, coordinated studies sharing common measures, data collection protocols, tools Improve patterns of learning technology use by “mining” use data from unobtrusive yet meaningful measures, coupled to SIS SRI Technology Evaluation Design Meeting Web Site http: //www. sri. com/policy/designkt/found. html NCTET’ 02 National Summit Stanford University © Professor Roy Pea

NCTM Standards Problem solve Communicate Reason Connect NSES Standards Identify questions Design/conduct investigations Develop descriptions, explanations, predictions Analyze explanations Communicate 21 st Century Skills Seek & organize information Evaluate information Communicate Collaborate NRC Tech Fluency Organize & navigate information structures Evaluate information Collaborate Communicate AAAS Benchmarks Judge quality of claims based on supporting evidence, language used, & logic of argument Communicate ISTE NETS Seek, analyze & evaluate information Solve problems Communicate, collaborate, publish, & produce 21 st Century Skills in Relation to Selected Content and Technology Standards (Haertel & Means, 2000)

(Quellmalz and Haertel, 2000)

Vital need to support teacher learning “There is a national crisis in teacher professional development” -Glenn Commission, 1999 – Huge turnover and new workforce preparation need — 2 Million new teachers needed by 2008 -2009 (3. 1 Mil today) – Increasingly accountable, but unprepared for new standards and assessments, and technology use – Expand deepen PT 3 to support schools of education in helping new teachers learn to use technology effectively in the classroom We need a deepening focus on uses of technology to empower teacher learning • Video case studies of exemplary practice • On-line communities of practice and mentoring – NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Accelerate learning for school leaders – Research indicates strong school leadership is a key success factor in creating a productive school culture for improving learning with technology, including: • • Instructional vision and a rationale linking it to technology use Support for teacher time for planning, collaboration and reporting technology use Critical mass of teachers in technology activities, with a high degree of collaboration Technology access and technical support (Means and Olson, 1995; Becker, 2000) NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Make “the learning enterprise” reach beyond the classroom – – – Ubiquitous computing and communications enables learning anytime anywhere Extend engaged learning beyond the school day and coordinate learning across boundaries Informal contexts • • – Museums Homes Community centers Gardens, parks Need multi-generational designs NCTET’ 02 National Summit Stanford University © Professor Roy Pea

Inventing the Future of E-Learning – – – Aim for 1: 1 e-learning to high standards Portable digital learning portfolios for every learner “E-Learning Workflow Management”: • – – – Need for real-time teacher performance support tools that are coupled to SIS data-driven systems Need ubiquitous broadband for highly interactive visualization-intensive web services, video sharing and services Need new mechanisms for “reciprocity of influence” in inventing the future of learning across communities Concept of LENS partnerships: Learning Expeditions in Networked Systems • Scout the frontiers of what is possible and desirable, e. g. using Internet-2 capabilities, handheld broadband services, NCTET’ 02 National Summit Stanford University © Professor Roy Pea

President’s Information Technology Advisory Committee Report to the President (PITAC: August 1998) Vision of Transforming the Way We Learn “Any individual can participate in on-line education programs regardless of geographic location, age, physical limitation, or personal schedule. Everyone can access repositories of educational materials, easily recalling past lessons, updating skills, or selecting from among different teaching methods in order to discover the most effective style for that individual. Educational programs can be customized to each individual’s needs, so that our information revolution reaches everyone and no one gets left behind. ” NCTET’ 02 National Summit Stanford University © Professor Roy Pea

It’s not enough of a “Grand Challenge” – – Enabling this vision requires re-inventing learning substantively, not only the HOW and WHEN of learning We will do better at re-inventing learning if we heed the PITAC visions of transforming the ways we: • • – – Communicate Deal with information (I/O) Work Design and build things Conduct research Deal with the environment Do commerce Each of these areas in society is spawning new literacies and required skills for an informed and proficient citizen. Keeping education apace of the needed learning curve is the Grand Challenge NCTET’ 02 National Summit Stanford University © Professor Roy Pea

National Research Council: Committee on Improving Learning with Information Technologies – Establish ongoing dialog and interactions among the IT industries, learning sciences, and educational communities toward improving education for all learners through the development and appropriate uses of modern technology. – Find ways to incorporate the knowledge base, research findings, and innovations from each of these communities into more coherent, strategic approaches to developing education technologies and improvements in learning outcomes. – Establish ways to allow the end users of technology in the education community to make more strategic decisions about their purchase, use, and maintenance of education technologies and the kinds of professional development programs that will be required to use education technologies in ways that can effectively transform teaching and learning for the new age. NCTET’ 02 National Summit Stanford University © Professor Roy Pea

What’s possible within 5 years? – Affordable, � Affordable, personal, portable gateways to e-learning content and e -services (using ASPs and wireless handhelds) – Fundamentally better real-time teaching and assessment capabilities in classrooms – Continuous teacher professional development Content (math) Technology (networked Hand helds) Instructional Methodology (co-learning, student feedback)

Thank you for your contributions to improving learning and teaching with technologies! NCTET’ 02 National Summit Stanford University © Professor Roy Pea