Introduction to Design Research a Methodological Background for

Introduction to Design Research: a Methodological Background for Scientific Work Elena Paslaru Bontas Semantic Web Ph. D Network Berlin Brandenburg 30. 09. 2005

Outline l l l Motivation Types of research Design Research Basics Evaluation in Design Research Conclusion

Motivation l Motivation for research: l l pure research: enhance understanding of phenomena instrumentalist research: a problem needs a solution applied research: a solution needs application fields Motivation for research methodology l l (qualitatively) control research process validate research results compare research approaches respect rules of good scientific practice

Research: A Definition l Research: l l an activity that contributes to the understanding of a phenomenon [Kuhn, 1962; Lakatos, 1978] l phenomenon: a set of behaviors of some entity(ies) that is found interesting by a research community l understanding: knowledge that allows prediction of the behavior of some aspect of the phenomenon l activities considered appropriate to the production of understanding (knowledge) are the research methods and techniques of a research community paradigmatic vs multi-paradigmatic communities (agreement on phenomena of interest and research methods)

Scientific Disciplines l Types of research [Simon, 1996]: l l natural sciences: phenomena occurring in the world (nature or society) design sciences ~ sciences of the artificial: l all or part of the phenomena may be created artificially l studies artificial objects or phenomena designed to meet certain goals social sciences: structural level processes of a social system and its impact on social processes and social organization behavioural sciences: the decision processes and communication strategies within and between organisms in a social system

phenomena design sciences Semantic Web (CS) activities [Owen, 1997]

Design research basics l l Process model Artifact types: l l Artifact structure l l result of the research work content of the research approach Evaluation: l l evaluation criteria evaluation approach

Process model l a problem-solving paradigm: l seeks to create innovations that define the ideas, practices, technical capabilities, and products through which the analysis, design, implementation, and use of information systems can be effectively and efficiently accomplished [Tsichritzis 1997; Denning 1997]

Design research process knowledge + operation and goal knowledge flows circumscription process steps Awareness of problem Suggestion Development Evaluation Conclusion logical formalism abduction deduction [Takeda, 1990]

Artifacts l l are not exempt from natural laws or behavioral theories artifacts rely on existing "kernel theories" that are applied, tested, modified, and extended through the experience, creativity, intuition, and problem solving capabilities of the researcher [Walls et al. 1992; Markus et al. 2002]

Design research outputs [March & Smith, 1995] l l l Constructs l conceptual vocabulary of a problem/solution domain Methods l algorithms and practices to perform a specific task Models l a set of propositions or statements expressing relationships among constructs l abstractions and representations Instantiations l constitute the realization of constructs, models and methods in a working system l implemented and prototype systems Better theories l artifact construction s i es l l h T t u p t ou

Design research outputs constructs better theories models emergent theory about embedded phenomena abstraction knowledge as operational principles models methods constructs better theories abstraction artifact as situated implementation [Purao , 2002] instatiations methods constructs

Examples l l l Open up a new area Provide a unifying framework Resolve a long-standing question Thoroughly explore an area Contradict existing knowledge Experimentally validate a theory Produce an ambitious system Provide empirical data Derive superior algorithms Develop new methodology Develop a new tool Produce a negative result

Artifact structure l Structure of the artifact l l l s i s spans the information space the artifact e h t basis for deducing all required information e h t about the artifact f o t determines the configurational characteristics n te the evaluation of the necessary tonenable artifact Co

Evaluation criteria l l the dimensions of the information space which are relevant for determining the utility s e of the artifact s a c can differ on the purpose of the evaluation st Te

Evaluation approach l l l the procedure how to practically test an d o artifact h t e with the defines all roles concerned m g way of handling the assessment and the n i t evaluation s e T result is a decision whether or not the artifact meets the evaluation criteria based on the available information.

Evaluation approach (2) l Quantative evaluation: l l originally developed in the natural sciences to study natural phenomena approaches: l l survey methods laboratory experiments formal methods (e. g. econometrics) numerical methods (e. g. mathematical modeling)

Evaluation approach (3) l Qualitative evaluation: l l l developed in the social sciences to enable researchers to study social and cultural phenomena approaches: l action research l case study research l ethnography l grounded theory qualitative data sources: l observation and participant observation (fieldwork) l interviews and questionnaires l documents and texts l the researcher’s impressions and reactions

Constructs Structure Evaluation criteria Evaluation approach meta-model of the vocabulary lconstruct ontological analysis deficit lconstruct overload lconstruct redundancy lconstruct excess

Methods Structure lprocess-based meta model lintended applications lconditions of applicability lproducts and results of the method application lreference to constructs Evaluation criteria Evaluation approach lappropriateness llaboratory lcompleteness lconsistency research lfield inquiries lsurveys lcase studies laction research lpractice descriptions linterpretative research

Models Structure Evaluation criteria Evaluation approach domain l scope, purpose l syntax and semantics l terminology l intended application lcorrecteness lsyntactical l lcompleteness lclarity lflexibility lsimplicity lapplicability limplementability validation lintegrity checking lsampling using selective matching of data to actual external phenomena or trusted surrogate lintegration tests lrisk and cost analysis luser surveys

Instantiations Structure Evaluation criteria Evaluation approach lexecutable lfunctionality lcode implementation in a programming language lreference to a design model lreference to a requirement specification lreference to the documentation lreference to quality management documents lreference to configuration management documents lreference to project lmanagement documents lusability lreliability lperformance lsupportability inspection ltesting lcode analysis lverification

Conclusion Good research results require a careful design of the research methodology and considerable evaluation efforts

References l l l „DFG Rules of Good Scientific Practice“ available at www. dfg. de, last seen September 2005 Tsichritzis, D. "The Dynamics of Innovation, " Beyond Calculation: The Next Fifty Years of Computing, Copernicus, 1997, pp. 259 -265 Denning, P. J. "A New Social Contract for Research, " Communications of the ACM (40: 2), February 1997, pp. 132 -134 Simon, H. A. The Sciences of the Artificial, 3 rd Edition, MIT Press, Cambridge, MA, 1996 Markus, M. L. , Majchrzak, A. , and Gasser, L. , "A Design Theory for Systems that Support Emergent Knowledge Processes, " MIS Quarterly (26: 3), September, 2002, pp. 179 -212 Walls, J. G. , Widmeyer, G. R. , and El Sawy, O. A. "Building an Information System Design Theory for Vigilant EIS, " Information Systems Research (3: 1), March 1992, pp. 36 -59 Kuhn, T. S. The Structure of Scientific Revolutions, 3 rd Edition, University of Chicago Press, 1996 March, S. T. and Smith, G. “Design and Natural Science Research on Information Technology, ” Decision Support Systems (15: 4), December 1995, pp. 251 -266 Lakatos, I. „The Methodology of Scientific Research Programmes“, John Worral and Gregory Currie, Eds. , Cambridge University Press, 1978 Wikipedia available at www. wikipedia. org, last seen Semptember 2005 Purao, S. “Design Research in the Technology of Information Systems: Truth or Dare. ” GSU Department of CIS Working Paper. Atlanta, 2002

Danke für die Aufmerksamkeit Viel Erfolg für die Promotion paslaru@inf. fu-berlin. de