DELPHI METHO D Dr Rania Albsoul 1 Intended

DELPHI METHO D Dr. Rania Albsoul 1

Intended learning outcomes ■ After this lecture, you will be able to: 1. Understand the meaning of Delphi method. 2. Describe the steps for performing Delphi study. 3. Identify the strengths and weaknesses of performing Delphi study. 4. Identify the criteria of a wise crowd. 2

Introduction ■ There has been much debate about the nature of the Delphi method (Delphi). ■ Some scholars posit that Delphi is a qualitative method, others suggest that it is the purist form of mixed methodology, and still others argue that in the final analysis, literally, Delphi is quantitative. ■ Originally developed in 1950 s by researchers at the Rand Corporation (Research and Development) as a method to forecast the impact of technology on warfare and improve decision making in government settings. ■ Delphi technique is used today in business, education, and the social sciences. 3

Introduction ■ The Delphi method is recommended for use in the healthcare setting as a reliable means of determining consensus for a defined clinical problem. ■ This method is an iterative process that uses a systematic progression of repeated rounds of voting and is an effective process for determining expert group consensus where there is little or no definitive evidence and where opinion is important. ■ The validity of the approach is supported by ‘wisdom of crowds’ research showing that groups can make good judgements under certain conditions. 4

Introduction ■ Crowds can produce better estimates than the best individual experts. However, crowds are not always wise. ■ Conditions that must be met for a crowd to be wise: 1. Diversity of expertise. A heterogeneous crowd of experts will produce better quality decisions than a homogeneous one. 2. Independence. The experts must be able to make their decisions independently, so that they are not influenced by others. 3. Decentralization. Expertise is held by autonomous individuals working in a decentralized way. • The members of the group operate autonomously, but share decisions through the facilitator. 4. Aggregation. There is a mechanism for coordinating and aggregating the crowd’s expertise. • Through the facilitator’s organization of the group and statistical summarization of results. 5

Expertise Criteria ■ The Delphi participants should meet four “expertise” requirements: i) Knowledge and experience with the issues under investigation; ii) Capacity and willingness to participate; iii) Sufficient time to participate in the Delphi; iv) Effective communication skills. 6

DELPHI METHO D 7

What is Delphi Method? ■ The Delphi Method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback. ■ Delphi represents a useful communication device among a group of experts and thus facilitates the formation of a group judgment. ■ It is a forecasting method ■ In a Delphi study, the participants do not interact with one another. ■ This avoids the negative effects of face to face panel discussions and solves the problem of group dynamics. 8

Delphi Method (Continued) ■ The technique allows experts to deal systematically with a complex problem or task. ■ It comprises a series of questionnaires sent either by mail or via computerized systems, to a pre-selected group of experts. These questionnaires are designed to elicit and develop individual responses to the problems posed and to enable the experts to refine their views as the group’s work progresses in accordance with the assigned task. 9

Features of Delphi Method 1. Anonymity of Delphi participants: allows the participants to freely express their opinions without undue social pressures to conform from others in the group. Decisions are evaluated on their merit, rather than who has proposed the idea. q The identity of the participants is not revealed, even after completion of final report. q Minimises Bandwagon effect 2. Iteration: allows the participants to refine their views in light of the progress of the group’s work from round to round. 3. Controlled feedback: informs the participants of the other participant’s perspectives, and provides the opportunity for Delphi participants to clarify or change their views. 4. Statistical aggregation of group response: allows for a quantitative analysis and interpretation of data. 10

When to use Delphi method? ■ When there is no clear cut resolution on a given public policy issues. ■ Time and cost constrains to make frequent face to face meetings. ■ Move a group of people towards consensus. 11

How to perform Delphi? ■ Identify the problem ■ Give the problem to preselected group of experts. ■ Gather the responses. ■ Give everyone the collation. ■ Repeat as necessary. 12

Identify the problem ■ Define the problem you want to investigate. ■ Develop a Delphi round one questionnaire. ■ Care and attention needs to be devoted to developing the initial broad question which is the focus of the Delphi because if respondents do not understand the question, they may provide inappropriate answers and/or become frustrated. ■ Delphi Pilot Study - A pilot study is sometimes conducted with the goals of testing and adjusting the Delphi questionnaire to improve comprehension, and to work out any procedural problems. The researcher may also pre-test each subsequent questionnaire. The Delphi pilot is especially important for inexperienced researchers who may be overly ambitious regarding the scope of their research or underestimate the time it will take a Delphi research participant to fully respond to the Delphi survey. 13

Give the problem to preselected group of experts ■ Recruit experts in the problem under investigation for the Delphi study. ■ Can be small group or hundreds of people ( commonly 20 people). ■ Send the questionnaire to the participants and ask them to respond. 14

Gather the responses ■ Take the responses and collate them into a single anonymous list. ■ Make it as easy as possible for people to read. 15

Give everyone the collation ■ Send the collation back to the participants. ■ Request to score each item on a given scale. ■ You can ask them to add more items. ■ Provide information about the original problem and instructions about how to answer. 16

Repeat as necessary ■ The process may be repeated as many times as it appear appropriate. ■ If there was a wide range of responses, you will need to make several iterations. 17

Steps in Delphi technique ■ Ten steps for the Delphi method: 1. Formation of a team to undertake and monitor a Delphi on a given subject. 2. Selection of one or more panels to participate in the exercise. Customarily, the panelists are experts in the area to be investigated. 3. Development of the first round Delphi questionnaire 4. Testing the questionnaire for proper wording (e. g. , ambiguities, vagueness) 5. Transmission of the first questionnaires to the panelists 6. Analysis of the first round responses 7. Preparation of the second round questionnaires (and possible testing) 8. Transmission of the second round questionnaires to the panellists 9. Analysis of the second round responses (Steps 7 to 9 are reiterated as long as desired or necessary to achieve stability in the results. ) 10. Preparation of a report by the analysis team to present the conclusions of the exercise 18

Key elements of Delphi method ■ There is a facilitator who organizes the Delphi study. ■ The facilitator recruits a group of individuals with some expertise on the topic. ■ The facilitator compiles a questionnaire with a list of statements that the experts rate for agreement. ■ The facilitator gathers responses from the members of the group using the questionnaire. ■ The facilitator gives anonymous feedback to individuals in the group about how their responses compare to the rest of the group. ■ The members of the group are able to revise their responses to the questionnaire after receiving the feedback. ■ Responses converge across rounds of questionnaires, with some statistical criterion being used to define consensus. 19

Strengths ■ Concentrate on ideas not on individuals. ■ Anonymity of participants makes participation a safe activity. ■ Participants can reconsider their opinions. ■ Potential for participation of large number of people. 20

Weakness ■ Analysis of data is complex. ■ Difficult to maintain enthusiasm of the participants. ■ Needs large amount of time to perform several rounds. ■ Self reporting data and respondent bias. ■ Reduced generalizability. 21

22

23