Research Design Types Of Data Data Collection Methods

Research Design, Types Of Data, Data Collection Methods Zulkarnain Lubis

– Research Design and Tactics – Research philosophy – Research approaches – Research strategies – Types of Data – Qualitative vs Quantitative – Primary vs Secondary – Cross Section vs Longitudinal – Data Collection Method – Interview – Questioner – Observation

design of research • The research design is the master plan specifying the methods and procedures for collecting and analyzing the needed information. • A detailed outline of how an investigation will take place. A research design will typically include how data is to be collected, what instruments will be employed, how the instruments will be used and the intended means for analysing data collected. • A master plan that specifies the methods and procedures for collecting and analyzing the needed information

Research Design and Tactics The Research Onion Saunders et al, (2009)

Research philosophy – Positivism – Interpretivism (Phenomenology) – Realism – Pragmatism

Research philosophy: Positivism (1) – Positivism in general refers to philosophical positions that emphasize empirical data and scientific methods – Verified data (positive facts) received from the senses are known as empirical evidence; thus positivism is based on empiricism – Positivism belongs to epistemology which can be specified as philosophy of knowing, whereas methodology is an approach to knowing – Highly structured methodology – Under positivism, the science objects and scientific proposition should meet the requirements: Observable, Repeatable, Measurable, testable, predictable

Research philosophy: Positivism (2) – In positivism studies the role of the researcher is limited to data collection and interpretation through objective approach and the research findings are usually observable and quantifiable. – Quantifiable observations that lend themselves to statistical analysis – Positivist studies usually adopt deductive approach – The researcher is independent and neither affects nor is affected by the subject of the research – Studies with positivist paradigm are based purely on facts and consider the world to be external and objective

Research philosophy: Interpretivism (Phenomenology) (1) – Interpretivism, also known as interpretivist involves researchers to interpret elements of the study – Interpretivism integrates human interest into a study – Interpretivism is associated with the philosophical position of idealism, and is used to group together diverse approaches, including social constructionism, phenomenology and hermeneutics – Interpretivism studies usually focus on meaning and may employ multiple methods in order to reflect different aspects of the issue – Interviews and observations are the most popular primary data collection methods in interpretivism studies. Secondary data research is also popular with interpretivism philosophy.

Research philosophy: Interpretivism (Phenomenology) (2) – Interpretivism may refer to: – interpretivism (social science) an approach to social science that opposes the positivism of natural science – qualitative research, a method of inquiry in social science and related disciplines – interpretivism (legal), a school of thought in contemporary jurisprudence and the philosophy of law – Interpretivists avoid rigid structural frameworks such as in positivist research and adopt a more personal and flexible research structures (Carson et al. , 2001), – Interpretivists receptive to capturing meanings in human interaction (Black, 2006) and make sense of what is perceived as reality (Carson et al. , 2001) – Interpretivists believe the researcher and his informants are interdependent and mutually interactive (Hudson and Ozanne, 1988).

Research philosophy: Interpretivism (Phenomenology) (3) – Business situations are not only complex, they are unique, a particular set of circumstances and individuals – To discover ‘the details of the situation to understand the reality or perhaps a reality working behind them’, associated with “constructionism” or “social constructionism” – reality is socially constructed – People place different interpretations on the situation, in order to make sense of and understand motives, actions and intentions of other people

Research philosophy: Realism (1) – Realism, in philosophy, the viewpoint which accords to things which are known or perceived an existence or nature which is independent of whether anyone is thinking about or perceiving them Based on a belief that reality exists, independent to human thoughts and beliefs – Realism mainly concentrates in the reality and beliefs that are already exist in the environment – Two main approaches (Mc. Murray, Pace and Scott 2004): – Direct; what an individual feels, see, hear, etc – Critical realism; individuals argue about their experiences for a particular situation (Sekaran and Bougie 2010)

Research philosophy: Realism (2) – Scientific realism is the view that theories refer to real features of the world – Its philosophical position is that reality exists independently of the researcher’s mind, that is, there is an external reality – Social objects or phenomena, external to or independent of individuals affect the way people perceive their world, whether the are aware of them or not – Business research is often a mixture between positivism and interpretivism, reflecting the stance of realism

Research philosophy: Pragmatism (1) – Pragmatism, despite many variants, essentially means that we come to know the world through the practicality or usefulness of objects (or concepts) – Pragmatists contend that most philosophical topics—such as the nature of knowledge, language, concepts, meaning, belief, and science—are all best viewed in terms of their practical uses and successes.

pragmatism and it is characterised, in most versions (1) – Pragmatism treats knowledge, concepts and values as true if they are useful – The pragmatists rejected the rationalist view that reality is static and fixed and preferred a view of a changing, dynamic reality – Pragmatism is primarily empiricist and inductive, testing hypotheses, prioritising experience, although not assuming that facts exist ‘out there’ waiting to be discovered – Pragmatism is opposed to doctrines that hold that truth can be reached through deductive reasoning from a priori grounds – Pragmatists were content with probabilistic relationships rather than with deterministic ones. – Pragmatists, though, opposed the notion of passive objectivity.

pragmatism and it is characterised, in most versions (2) – Pragmatism adopts a relative approach: truth is modified as discoveries are made and is relative to the time and place and purpose of inquiry – The function of thought is to guide action not provide timeless abstract truths; pragmatists interpret ideas as instruments and plans of action rather than as images of reality – Thought is simply an instrument for supporting the life aims of the human organism – Thought is grounded in practical reality and has no real metaphysical significance, pragmatists protest against speculation concerning questions that have no application and no verifiable answers. – In its ethical aspect pragmatism holds that knowledge, which contributes to human values, is real.

Paradigms: Language commonly associated with major research paradigms Positivist/ Postpositivist Experimental Quasi-experimental Correlational Reductionism Theory verification Causal comparative Determination Normative Interpretivist/ Constructivist Naturalistic Phenomenological Hermeneutic Interpretivist Ethnographic Multiple participant meanings Social and historical construction Theory generation Symbolic interaction Adapted from Mertens (2005) and Creswell (2003) Transformative Pragmatic Critical theory Neo-marxist Feminist Critical Race Theory Freirean Participatory Emancipatory Advocacy Grand Narrative Empowerment issue oriented Change-oriented Interventionist Queer theory Race specific Political Consequences of actions Problem-centred Pluralistic Real-world practice oriented Mixed models

Paradigms, methods and tools Paradigm Methods (primarily) Positivist/ Postpositivist Quantitative. "Although qualitative Experiments methods can be used within this paradigm, Quasi-experiments quantitative methods tend to be Tests predominant. . . " (Mertens, 2005, p. 12) Scales Interpretivist/ Constructivist Qualitative methods predominate although Interviews quantitative methods may also be utilised. Observations Document reviews Visual data analysis Qualitative methods with quantitative and Diverse range of tools - particular need to mixed methods. Contextual and historical avoid discrimination. Eg: sexism, racism, factors described, especially as they relate and homophobia. to oppression (Mertens, 2005, p. 9) Transformative Pragmatic Data collection tools (examples) Qualitative and/or quantitative methods May include tools from both positivist and may be employed. Methods are matched interpretivist paradigms. Eg Interviews, to the specific questions and purpose of observations and testing and experiments. the research.

Classification of the research purpose – Exploratory research: – Find out what is happening, to clarify your understanding of a problem, – 3 ways for conducting; a search of the literature, interview experts in the subject, conducting focus group interviews – Descriptive studies: – Its object is to portray an accurate profile not persons, events or situations. – Explanatory (causal) studies: – Studies that establish causal relationships between variables

Research Strategies – Experiment – Survey – Case study – Action research – Grounded theory – Ethnography – Archival research

Research Strategies: Experiment – An experiment will involve: – Definition of a theoretical hypothesis – Selection of samples from know populations – Random allocation of samples – Introduction of planned intervention – Measurement on a small number of dependent variables – Control of all other variables

Research Strategies: Survey – Key Features: – Popular in business research – Perceived as authoritative – Allows collection of quantitative data – Data can be analysed quantitatively – Samples need to be representative – Gives the researcher independence – Structured observation and interviews can be used § Survey – To collect a large amount of data from a sizeable population and standardize it to allow easy comparison o Types: Questionnaires, Structured Interviews

Research Strategies: Case study – Key features – Provides a rich understanding of a real life context – Uses and triangulates multiple sources of data – A case study can be categorised in four ways and based on two dimensions: – single case v. multiple case (more ability to generalize) – holistic case (choose 1 organization as a whole) – v. embedded case (some departments or activities) • Case Studies: • The documented history of a particular person, group, organization, or event.

Research Strategies: Action research – Key features – Research IN action - not ON action focusing on the purpose – Involvement of practitioners in the research – The researcher becomes part of the organisation – Promotes change within the organisation – Can have two distinct focus (Schein, 1999) – the aim of the research and the needs of the sponsor

Research Strategies; Grounded theory (Inductive deductive approach) – Key features: – Theory is built through induction and deduction – Helps to predict and explain behaviour – Develops theory from data generated by observations – Is an interpretative process, not a logical-deductive one • Represents an inductive investigation in which the researcher poses questions about information provided by respondents or taken from historical records. • The researcher asks the questions to him or herself and repeatedly questions the responses to derive deeper explanations.

Research Strategies: Ethnography (Inductive approach) – Key features – Aims to describe and explain the social world inhabited by the researcher – Takes place over an extended time period – Is naturalistic – Involves extended participant observation such as studying gorillas in their natural habitat • Ethnography • Represents ways of studying cultures through methods that involve becoming highly active within that culture.

Research Strategies: Archival research – Key features – Uses administrative records and documents as the principal sources of data – Allows research questions focused on the past – Is constrained by the nature of the records and documents – Example: historical research

Research Approach – Deductive approach tests the validity of assumptions (or theories/hypotheses) in hand – Inductive approach contributes to the emergence of new theories and generalizations

• the major differences between deductive and inductive research approaches Deductive methods • Principles based on science • Movement is done from theory to data • Casual relationships between variables need to be explained • Quantitative type of data is mainly collected • Measures of control are applied in order to ensure the validity of data • Concepts are operationalised in order to ensure the clarity of definitions • The approach is highly structured • Researcher is independent from the research process • Samples need to be selected of a sufficient size in order to be able to generalise research conclusions Inductive methods • The meaning of human attachment to events are aimed to be explored • Research context is understood in a deeper manner • Qualitative type of data is collected • More flexible approach to research structure to ensure provisions for changes during the research • Researcher is perceived to be a part of the research process • Research findings do not have to be generalised

Types of Data Primary Data vs Secondary Data • Primary data: Quantitative vs Qualitative directly collected by Discrete: Nominal researcher and his/her team Continuum; Ordinal, Interval, ratio • Secondary: data collected The level of mathematical by others Operations • Nominal : = and cross section vs • Ordinal : time series/ = , , >, < = , longitudinal • Interval : • cross section: = , , > , <, + , = , the study of a phenomenon • Ratio : at a particular time = , , >, <, + , - , , = , • longitudinal: It has the capacity to study change and development

Time Horizons – Cross-sectional studies the study of a phenomenon at a particular time. Because of time restrictions – Longitudinal studies it has the capacity to study change and development

Comparing Qualitative and Quantitative Research

Contrasting Exploratory and Confirmatory Research – Qualitative data – Data that are not characterized by numbers but rather are textual, visual, or oral. – Focus is on stories, visual portrayals, meaningful characterizations, interpretations, and other expressive descriptions. – Quantitative data – Represent phenomena by assigning numbers in an ordered and meaningful way.

Qualitative “versus” Quantitative Research • Quantitative business research • Descriptive and conclusive Addresses research objectives through empirical assessments that involve numerical measurement and statistical analysis. • Qualitative business research • Exploratory – Uses small versus large samples – Asks a broad range of questions versus structured questions – Subjective interpretation versus statistical analysis

What is Qualitative Research? – Qualitative business research • Research that addresses business objectives through techniques that allow the researcher to provide elaborate interpretations of phenomena without depending on numerical measurement – Its focus is on discovering true inner meanings and new insights. – Researcher-dependent • Researcher must extract meaning from unstructured responses such as text from a recorded interview or a collage representing the meaning of some experience.

Uses of Qualitative Research – Qualitative research is useful when: • It is difficult to develop specific and actionable decision statements or research objectives. • The research objective is to develop a detailed and indepth understanding of some phenomena. • The research objective is to learn how a phenomenon occurs in its natural setting or to learn how to express some concept in colloquial terms. • The behavior the researcher is studying is particularly context-dependent. • A fresh approach to studying the problem is needed.

Qualitative Research Orientations • Major Orientations of Qualitative Research – Phenomenology—originating in philosophy and psychology – Ethnography—originating in anthropology – Grounded theory—originating in sociology – Case studies—originating in psychology and in business research

Quantative data analysis: Key Points • Data must be analyzed to produce information • Computer software analysis is normally used for this process • Data should be carefully prepared for analysis • Researchers need to know how to select and use different charting and statistical techniques

Quantative data analysis: Main Concerns • Preparing, inputting and checking data • Choosing the most appropriate statistics to describe the data • Choosing the most appropriate statistics to examine data relationships and trends

Quantative data analysis: Main Considerations • Type of data (scale of measurement) • Data format for input to analysis software • Impact of data coding on subsequent analyses • Case weighting • Methods for error checking

Data Sources • Secondary data – Documentary, survey, or an amalgam of both – Times series for longitudinal studies – Cohort studies (compiling for the same population over time using a series of “snapshots”) – Area-based data sets • Primary data – Experiments and observational study – Questionnaires/tests – Interviews – Focus groups

Secondary Data q. Availability of secondary data sources o References in publications (books, journal articles) o Within organisations (unpublished sources) o Tertiary literature ( indexes and catalogues in archives or online)

Evaluating secondary data: – Limitations – What did you find on the frustrating side as you looked for data on the state’s websites?

Evaluating secondary data: – Limitations – When was it collected? For how long? – May be out of date for what you want to analyze. – May not have been collected long enough for detecting trends. – E. g. Have new anticorruption laws impacted Russia’s government accountability ratings?

Evaluating secondary data: – Limitations – Is the data set complete? – There may be missing information on some observations – Unless such missing information is caught and corrected for, analysis will be biased.

Evaluating Secondary Data – Limitations – Are there confounding problems? – Sample selection bias? – Source choice bias? – In time series, did some observations drop out over time?

Evaluating Secondary Data – Limitations – Are the data consistent/reliable? – Did variables drop out over time? – Did variables change in definition over time? – E. g. number of years of education versus highest degree obtained.

Evaluating Secondary Data – Limitations – Is the information exactly what you need? – In some cases, may have to use “proxy variables” – variables that may approximate something you really wanted to measure. Are they reliable? Is there correlation to what you actually want to measure?

Evaluating Secondary Data – Advantages – No need to reinvent the wheel. – If someone has already found the data, take advantage of it.

Evaluating Secondary Data – Advantages – It will save you money. – Even if you have to pay for access, often it is cheaper in terms of money than collecting your own data.

Evaluating Secondary Data – Advantages – It will save you time. – Primary data collection is very time consuming.

Evaluating Secondary Data – Advantages – It may be very accurate. – When especially a government agency has collected the data, incredible amounts of time and money went into it. It’s probably highly accurate.

Evaluating Secondary Data – Advantages – It has great exploratory value – Exploring research questions and formulating hypothesis to test.

Evaluating secondary data: Advantages – – Fewer resource requirements Unobtrusive Longitudinal studies may be feasible Provision of comparative and contextual data – Unforeseen discoveries may occur – Generally permanent and available

Evaluating secondary data: Disadvantages – Purpose of data collection may not match the research needs – Access may be difficult or costly – Aggregations and definitions may be unsuitable – No real control over data quality – Initial purpose may affect data presentation

Evaluating secondary data: Ensure that data sources – Enable the research question(s) to be answered – Enable research objectives to be met – Have greater benefits than their associated costs – Allow access for research

Suitability of secondary data: Overall suitability: points to consider – Precise suitability, including reliability and validity - assessment of collection methods - clear explanation of collection techniques – Measurement validity – Measurement bias and deliberate distortion

Suitability of secondary data: Overall suitability: points to consider – Coverage and unmeasured variables - ensure exclusion of unwanted data - ensure sufficient data remain for analysis – Costs and benefits

Primary Data – Primary data – data you collect – Primary Data - Examples – Surveys – Focus groups – Questionnaires – Personal interviews – Experiments and observational study

Data collection choice – What you must ask yourself: – Will the data answer my research question?

Data collection choice – To answer that – You much first decide what your research question is – Then you need to decide what data/variables are needed to scientifically answer the question

Data collection choice – If that data exist in secondary form, then use them to the extent you can, keeping in mind limitations. – But if it does not, and you are able to fund primary collection, then it is the method of choice.

Interviews • An interview is a purposeful discussion between two or more people (Kahn and Cannell, 1957) – Structured interviews: Personal (face to face) and Telephone – Semi-structured interviews – Unstructured interviews (in-depth) – Standardized interviews – Non-standardized interviews – Respondent interviews – Informant interviews

Standardized vs Non-standardized interviews

Structured, Semi-structured, and Unstructured Interviews Related to the Research Type

Focus Group Discussion – An unstructured, free-flowing interview with a small group (6 -10 people) led by a moderator who encourages dialogue among respondents. – Advantages: 1. 2. 3. 4. 5. 6. Relatively fast Easy to execute Allow respondents to piggyback off each other’s ideas Provide multiple perspectives Flexibility to allow more detailed descriptions High degree of scrutiny

Focus Group Respondents – Group Composition – 6 to 10 people – Relatively homogeneous – Similar lifestyles and experiences

The Focus Group Moderator – A person who leads a focus group interview and insures that everyone gets a chance to speak and contribute to the discussion. – Qualities of a good moderator: – Develops rapport with the group – Good listener – Tries not to interject his or her own opinions – Controls discussion without being overbearing

Planning a Focus Group Outline – Discussion guide – Includes written introductory comments informing the group about the focus group purpose and rules and then outlines topics or questions to be addressed in the group session.

Focus Group Discussion Guide 1. Welcome and introductions should take place first. 2. Begin the interview with a broad icebreaker that does not reveal too many specifics about the interview. 3. Questions become increasingly more specific as the interview proceeds. 4. If there is a very specific objective to be accomplished, that question should probably be saved for last. 5. A debriefing statement should provide respondents with the actual focus group objectives and answering any questions they may have.

Interactive Media and Online Focus Groups – Online focus group – A qualitative research effort in which a group of individuals provides unstructured comments by entering their remarks into an electronic Internet display board of some type. – Focus blog – A type of informal, “continuous” focus group established as an Internet blog for the purpose of collecting qualitative data from participant comments.

Depth Interviews – Depth interview • A one-on-one interview between a professional researcher and a research respondent conducted about some relevant business or social topic. – Laddering • A particular approach to probing asking respondents to compare differences between brands at different levels. • Produces distinctions at the: – attribute level – benefit level – value or motivation level

Conversations – Conversations • An informal qualitative data-gathering approach in which the researcher engages a respondent in a discussion of the relevant subject matter. – Semi-structured interviews • Written form and ask respondents for short essay responses to specific open-ended questions. • Advantages – An ability to address more specific issues – Responses are easier to interpret – Without the presence of an interviewer, semi-structured interviews can be relatively cost effective

Questionnaire • A set of Questions designed to generate the statistical information from and data necessary for accomplishing a research project's objectives • Definition of Questionnaires: Techniques of data collection in which each person is asked to respond to the same set of questions in a predetermined order (Adapted from de. Vaus. 2002)

When to use questionnaires • For explanatory or descriptive research • Linked with other methods in a multiplemethods research design • To collect responses from a large sample prior to quantitative analysis

Purposes of the Questionnaire § Ensures standardization and comparability of the data across interviews – everyone is asked the same questions § Increases speed and accuracy of recording § Facilitates data processing § Allows the researcher to collect the relevant information necessary to address the management decision problem • To obtain information that cannot be easily observed or is not already available in written or electronic form • Questionnaires enable researchers to measure concepts/constructs

Designing the Questionnaire § Determine survey objectives: Plan what to measure. § § Decide on format. E. g. personal interview, telephone, self. Formulate questions to obtain the needed information Decide on the wording of questions Decide on the question sequence and layout of the questionnaire § Using a sample, test the questionnaire for omissions and ambiguity § Correct the problems (pretest again, if necessary)

Constructing the questionnaire – Main considerations – Order and flow of questions – Questionnaire layout

Choice of questionnaire: Related factors – Characteristics of the respondents and access – Respondents answers not being contaminated or distorted – Size of sample required for analysis – Type and number of questions required – Available resources including use of computer software

Administering the questionnaire – Points to consider – Internet and intranet-mediated responses – Postal questionnaires – Delivery and Collection – Telephone questionnaires – Structured interviews

The Major Decisions in Questionnaire Design – Content - What should be asked? – Wording - How should each question be phrased? – Sequence - In what order should the questions be presented? – Layout - What layout will best serve the research objectives? • The most difficult step is specifying exactly what information is to be collected from each respondent

Steps in Questionnaire Design q Initial Considerations – problem, objectives, target population, sampling, etc. q Clarification of Concepts – select variables, constructs, measurement approach, etc. q Developing the Questionnaire • • Length and sequence. Types of questions. Sources of questions. Wording, coding, layout and instructions. q Pre-testing the Questionnaire. q Questionnaire Administration Planning.

Steps to design a questionnaire: 1. Write out the primary and secondary aims of your study. 2. Write out concepts/information to be collected that relates to these aims. 3. Review the current literature to identify already validated questionnaires that measure your specific area of interest. 4. Compose a draft of your questionnaire. 5. Revise the draft. 6. Assemble the final questionnaire.

Step 1: Define the aims of the study – Write out the problem and primary and secondary aims using one sentence per aim. Formulate a plan for the statistical analysis of each aim. – Make sure to define the target population in your aim(s).

Step 2: Define the variables to be collected – Write a detailed list of the information to be collected and the concepts to be measured in the study. Are you trying to identify: – Attitudes – Needs – Behavior – Demographics – Some combination of these concepts – Translate these concepts into variables that can be measured. – Define the role of each variable in the statistical analysis: – Predictor – Confounder – Outcome

Step 3: Review the literature – Review current literature to identify related surveys and data collection instruments that have measured concepts similar to those related to your study’s aims. – Saves development time and allows for comparison with other studies if used appropriately. – Proceed with caution if using only a subset of an existing questionnaire as this may change the meaning of the scores. Contact the authors of the questionnaire to determine if a smaller version of the instrument exists that has also been validated.

Step 4: Compose a draft [1]: – Determine the mode of survey administration: face-to-face interviews, telephone interviews, self-completed questionnaires, computer-assisted approaches. – Write more questions than will be included in the final draft. – Format the draft as if it were the final version with appropriate white space to get an accurate estimate as to its length – longer questionnaires reduce the response rate. – Place the most important items in the first half of the questionnaire to increase response on the important measures even in partially completed surveys. – Make sure questions flow naturally from one to another.

Compose a draft [2]: – Question: How many cups of coffee or tea do you drink in a day? – Principle: Ask for an answer in only one dimension. – Solution: Separate the question into two – – (1) How many cups of coffee do you drink during a typical day? – (2) How many cups of tea do you drink during a typical day?

Compose a draft [3]: – Question: What brand of computer do you own? – (A) IBM PC – (B) Apple – Principle: Avoid hidden assumptions. Make sure to accommodate all possible answers. – Solution: – (1) Make each response a separate dichotomous item – Do you own an IBM PC? (Circle: Yes or No) – Do you own an Apple computer? (Circle: Yes or No) – (2) Add necessary response categories and allow for multiple responses. – What brand of computer do you own? (Circle all that apply) – Do not own computer – IBM PC – Apple – Others

Compose a draft [4]: – Question: Have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very often – Principle: Make sure question and answer options match. – Solution: Reword either question or answer to match. – How often have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very Often

Compose a draft [5]: – Question: Where did you grow up? – Country – Farm – City – Principle: Avoid questions having non-mutually exclusive answers – Solution: Design the question with mutually exclusive options. – Where did you grow up? – House in the country – Farm in the country – City

Compose a draft [6]: – Question: Are you against drug abuse? (Circle: Yes or No) – Principle: Write questions that will produce variability in the responses. – Solution: Eliminate the question.

Compose a draft [7]: – Question: Which one of the following do you think increases a person’s chance of having a heart attack the most? (Check one) [ ] Smoking[ ] Being overweight [ ] Stress – Principle: Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item. – Solution: Which of the following increases the chance of having a heart attack? – Smoking: [ ] Yes [ ] No [ ] Don’t know – Being overweight: [ ] Yes [ ] No [ ] Don’t know – Stress: [ ] Yes [ ] No [ ] Don’t know

Compose a draft [8]: – Question: – (1) Do you currently have a life insurance policy? (Circle: Yes or No) – If no, go to question 3. – (2) How much is your annual life insurance premium? – Principle: Avoid branching as much as possible to avoid confusing respondents. – Solution: If possible, write as one question. – How much did you spend last year for life insurance? (Write 0 if none).

Step 5: Revise – Shorten the set of questions for the study. If a question does not address one of your aims, discard it. – Refine the questions included and their wording by testing them with a variety of respondents. – Ensure the flow is natural. – Verify that terms and concepts are familiar and easy to understand for your target audience. – Keep recall to a minimum and focus on the recent past.

Step 6: Assemble the final questionnaire [1]: – Decide whether you will format the questionnaire yourself or use computerbased programs for assistance: – Survey. Monkey. com – Adobe Live Cycle Designer 7. 0 – GCRC assistance – At the top, clearly state: – The purpose of the study – How the data will be used – Instructions on how to fill out the questionnaire – Your policy on confidentiality – Include identifying data on each page of a multi-page, paper-based questionnaire such as a respondent ID number in case the pages separate.

Assemble the final questionnaire [2]: – Group questions concerning major subject areas together and introduce them by heading or short descriptive statements. – Order questions in order to stimulate recall. – Order and format questions to ensure unbiased and balanced results.

Assemble the final questionnaire [3]: – Include white space to make answers clear and to help increase response rate. – Space response scales widely enough so that it is easy to circle or check the correct answer without the mark accidentally including the answer above or below. – Open-ended questions: the space for the response should be big enough to allow respondents with large handwriting to write comfortably in the space. – Closed-ended questions: line up answers vertically and precede them with boxes or brackets to check, or by numbers to circle, rather than open blanks. – Use larger font size (e. g. , 14) and high contrast (black on white).

QUESTIONNAIRE DESIGN Two Types of Questions: • Open-ended • Closed-ended Open-ended Questions : place no constraints on respondents; i. e. , they are free to answer in their own words and to give whatever thoughts come to mind. Closed-ended Questions : respondent is given the option of choosing from a number of predetermined answers.

QUESTIONNAIRE DESIGN • Open-ended Questions • • Typically used in exploratory/qualitative studies. • • Allows respondent freedom of response. • Data is in narrative form which can be time consuming and difficult to code and analyze. • • Possible researcher bias in interpretation. Typically used in personal interview surveys involving small samples. Respondent must be articulate and willing to spend time giving a full answer. Narrative is analyzed using of content analysis. Software is available (e. g. , NUD*IST).

Open Ended Questions: key advantages § Wide range of responses and information can be obtained § Answers based on respondent’s not researcher’s frame of reference – consumer’s terms § Lack of influence. Don't channel respondents thinking § Can help interpret closed-ended questions - why § Particularly useful as introduction to survey or topic § When it’s important to measure the salience of an issue § When too many possible responses to be listed or unknown

Open-ended questions: Key disadvantages § Ability and/or willingness of respondent to answer § Interviewer’s ability to record answers quickly or summarize accurately & probe effectively § Interviewer’s attitude influences response § Time consuming (interview sessions, tabulation, classification, assignment, validation) § Difficulty in coding § Require respondents to be articulate § Respondents may miss important points § Non-response

• Examples of Open-ended Questions: • How do you typically decide which restaurant you will eat at? • Which mutual funds have you been investing in for the past year? • How are your investment funds performing • Do you think airport security is better now than it was six months ago?

Examples of Open questions Please list up to three things you like about your job 1…………………… 2…………………… 3……………………

QUESTIONNAIRE DESIGN • Closed-end Questions: • • • Single Answer Multiple Answer Rank Order Numeric Likert-Type Scales Semantic Differential

• QUESTIONNAIRE DESIGN • Closed-end Questions • • Typically used in quantitative studies. Assumption is researcher has knowledge to prespecify response categories. • Data can be pre-coded and therefore in a form amenable for use with statistical packages (e. g. , SPSS, SAS) – data capture therefore easier. • More difficult to design but simplifies analysis. • Used in studies involving large samples. • Limited range of response options.

Closed-ended questions (Fixed-alternative responses); Advantages § Ease of understanding § Requires less effort on part of interviewer and respondent § Ease of tabulation & analysis § Less error prone § Less interviewer bias § Less time consuming § Answers directly comparable from respondent to respondent

Examples of Closed-end Questions: 1. Did you check your email this morning? __ Yes 2. Do you believe Enron senior executives should be put in jail? __ Yes 3. __ No Should the U. K. adopt the Euro or keep the Pound? __ Adopt the Euro __ Keep the Pound 4. Which countries in Europe have you traveled to in the last six months? __ Belgium __ Germany __ France __ Holland __ Italy __ Switzerland __ Spain __ Other (please specify) _______ 5. How often do you eat at Samouel’s Greek Cuisine restaurant? __ Never __ 1 – 4 times per year __ 5 – 8 times per year __ 9 – 12 times per year __ More than 12 times per year __ No

Examples of question types: List questions What is your religion? Please tick the appropriate box Buddhist None Christian Other Hindu Jewish Muslim Sikh

Examples of question types: Category questions How often do you visit the shopping centre? Interviewer: listen to the respondent’s answer and tick as appropriate First visit Once a week Less than fortnightly to once a month 2 or more times a week Less than once a week to fortnightly Less often

Examples of question types: Ranking questions Please number each of the factors listed below in order of importance to you in choosing a new car. Number the most important 1, the next 2 and so on. If a factor has no importance at all, please leave blank. Factor Carbon dioxide emissions Boot size Depreciation Price Importance [ ] [ ]

Examples of question types: Rating questions For the following statement please tick the that matches your view most closely box Agree Tend to agree Tend to disagree Disagree I feel employees’ views have influenced the decisions taken by management

Examples of question types: Quantity questions What is your year of birth? 1 9 (For example, for 1988 write: ) 1 9 8 8

Dichotomous Questions Should the Alberta Government give consumers an energy rebate? 1. Agree 2. Disagree Advantages § Easy to administer and tabulate Disadvantages § Prone to large amounts of error since polarized responses prevent gaining information on the range of variation § Fail to communicate any intensity of feeling

Multiple Choice Questions §Are all possible alternatives included? §Too many alternatives §Position Bias Scaled Response Questions § Closed ended questions where the response choices are designed to capture an intensity of feeling (Likert, Staple, Semantic differential) §Easy to code and more powerful statistical tools §Main problem: Respondent misunderstanding

QUESTIONNAIRE DESIGN Preparing Good Questions: • Use Simple Words. • Be brief. • Avoid Ambiguity. • Avoid Leading Questions. • Avoid Double-Barreled Questions. • Check Questionnaire Layout. • Prepare Clear Instructions. • Watch Question Sequence.

QUESTIONNAIRE DESIGN Recently a survey was conducted by the United Nations using a sample from several different countries. The question asked was: " Would you please give your opinion about the food shortage in the rest of the world? " The survey was a huge failure. Why? • In Africa they did not know what 'food' meant. • In Western Europe, they did not know what 'shortage' meant. • In Eastern Europe they did not know what 'opinion' meant. • In South America they did not know what 'please' meant. • And in the U. S. , they did not know what 'the rest of the world' meant.

QUESTIONNAIRE DESIGN Avoid Position Bias: • “How important are flexible hours in evaluating job alternatives? ” • “What factors are important in evaluating job alternatives? ” No Position Bias: • “What factors are important in evaluating job alternatives? ” • “How important are flexible hours in evaluating job alternatives? ”

QUESTIONNAIRE DESIGN Branching Questions: . . . are used to direct respondents to answer the right questions as well as questions in the proper sequence. • “Have you seen or heard any advertisements for wireless telephone service in the past 30 days? ” • “If ‘No’, go to question #10. • “If ‘Yes’ , were the advertisements on radio or TV or both? ” • “If the advertisements were on TV or on both radio and TV, then go to question #6? • “If the advertisements were on radio, then go to question #8. ” Following questions #6 and #8 the next question would be: • “Were any of the advertisements for ‘Sprint PCS’? ”

QUESTIONNAIRE DESIGN Issues – Self-Completion Instructions: • Introducing and explaining how to answer a series of questions on a particular topic. • Transition statements from one section (topic) of the questionnaire to another. • Which question to go to next (branching or skipping). • How many answers are acceptable, e. g. , “Check only one response” or “Check as many as apply. ” • Whether respondents are supposed to answer the question by themselves, or can consult another person or reference materials. • What to do when the questionnaire is completed, e. g. , “When finished, place this in the postage paid envelope and mail it. ”

QUESTIONNAIRE DESIGN Issues – Interviewer-Assisted Instructions: • How to increase respondent participation. • How to screen out respondents that are not wanted and still keep them happy. • What to say when respondents ask how to answer a particular question. • When concepts may not be easily understood, how to define them. • When answer alternatives are to be read to respondents (aided response) or not to be read (unaided response). • How to follow branching or skip patterns. • When and how to probe. • How to end the interview.

QUESTIONNAIRE DESIGN Identify response bias for below questions: 1. 2. “Do you advocate a lower speed limit to save human lives? ” 3. 4. “About what time do you ordinarily eat dinner? ” 5. “Would you favor increasing taxes to cope with the current fiscal crisis? ” 6. 7. “Don’t you see some danger in the new policy? ” 8. “When you buy ‘fast food, ’ what percentage of the time do you order each of the following types of food? ” 9. “Do you like orange juice? ” “When you visited the museum, how many times did you read the plaques that explain what the exhibit contained? ” “How important is it for stores to carry a large variety of different brands of this product? ” “What small appliance, such as countertop appliances, have you purchased in the past month? ”

QUESTIONNAIRE DESIGN Comments on Questions: 1. A loaded question because everyone wants to save lives. Also, it presumes that lower speed limits saves lives. 2. Too specific because respondents likely cannot remember the exact number of times. 3. Ambiguous because don’t know if dinner is lunch or evening. 4. Not specific enough about types of stores. 5. Overemphasis because refers to crisis. 6. Leading question because uses “danger” in sentence. 7. Answers likely to relate only to countertop appliances and not all small appliances. 8. Over generalization because does not specify time period. 9. Ambiguous because may like orange juice for themselves, or for their kids, but really do not know.

QUESTIONNAIRE DESIGN Pre-testing Questionnaires: • • • Objective: to identify possible shortcomings of questionnaire. Approaches – informal or formal. Can assess: • • clarity of instructions cover letter clarity of questions adequacy of codes and categories for pre-coded questions • quality of responses • likely response rate • No hard and fast rules. • ability to perform meaningful • • • analyses time to complete the questionnaire cost of data collection which questions are relevant whether key questions have been overlooked sources of bias

MEASUREMENT SCALES Types of Scales: • • Metric (interval & ratio) • Likert-type • Summated-Ratings (Likert) • Numerical • Semantic Differential • Graphic-Ratings Nonmetric (nominal & ordinal) • Categorical • Constant Sum Method • Paired Comparisons • Rank Order • Sorting

MEASUREMENT SCALES – Metric Examples of Likert-Type Scales: “When I hear about a new restaurant , I eat there to see what it is like. ” Strongly Agree Neither Agree Disagree Strongly Agree Somewhat or Disagree Somewhat Disagree 1 2 3 4 5 “When I hear about a new restaurant , I eat there to see what it is like. ” Strongly Agree 1 2 3 Strongly Disagree 4 5

MEASUREMENT SCALES – Metric Summated Ratings Scales: A scaling technique in which respondents are asked to indicate their degree of agreement or disagreement with each of a number of statements. A subject’s attitude score (summated rating) is the total obtained by summing over the items in the scale and dividing by the number of items to get the average. Example: “My sales representative is. . “ SD Courteous ___ Friendly ___ Helpful ___ Knowledgeable ___ D ___ ___ N ___ ___ A ___ ___ SA ___ ___

MEASUREMENT SCALES – Metric Alternative Approach to Summated Ratings scales: “When I hear about a new restaurant , I eat there to see what it is like. ” Strongly Agree Neither Agree Disagree Strongly Agree Somewhat or Disagree 1 3 2 Somewhat Disagree 4 5 “I always eat at new restaurants when someone tells me they are good. ” Strongly Agree Neither Agree Disagree Strongly Agree Somewhat or Disagree 1 3 2 Somewhat Disagree 4 5 • This approach includes a separate labeled Likert scale with each item (statement). The summated rating is a total of the responses for all the items divided by the number of items.

MEASUREMENT SCALES – Metric Numerical Scales: Example: “Using a 10 -point scale, where ‘ 1’ is ‘not at all important’ and ’ 10’ is ‘very important, ’ how important is ______ in your decision to do business with a particular vendor. ” Note: you fill in the blank with an attribute, such as reliable delivery, product quality, complaint resolution, and so forth.

MEASUREMENT SCALES – Metric Semantic Differential Scales: A scaling technique in which respondents are asked to check which space between a set of bipolar adjectives or phrases best describes their feelings toward the stimulus object. Example: “My sales representative is. . “ Courteous ___ ___ ___ Discourteous Friendly ___ ___ ___ Unfriendly Helpful ___ ___ ___ Unhelpful Honest ___ ___ ___ Dishonest

MEASUREMENT SCALES – Metric Graphic-Ratings Scales: A scaling technique in which respondents are asked to indicate their ratings of an attribute by placing a check at the appropriate point on a line that runs from one extreme of the attribute to the other. “Please evaluate each attribute in terms of how important the attribute is to you personally (your company) by placing an “X” at the position on the horizontal line that most reflects your feelings. ” Not Importan Very Important Courteousness ___________________ Friendliness ___________________ Helpfulness ___________________ Knowledgeable ___________________

MEASUREMENT SCALES – Nonmetric Categorical scale: Categorical scales are nominally measured opinion scales that have two or more response categories. “How satisfied are you with your current job? ” [ ] Very Satisfied [ ] Somewhat Satisfied [ ] Neither Satisfied nor Dissatisfied [ ] Somewhat Dissatisfied [ ] Very Dissatisfied Note: Some researchers consider this a metric scale when coded 1 – 5.

MEASUREMENT SCALES – Nonmetric Constant-Sum Method: A scaling technique in which respondents are asked to divide some given sum among two or more attributes on the basis of their importance to them. “Please divide 100 points among the following attributes in terms of the relative importance of each attribute to you. ” Courteous Service ____ Friendly Service ____ Helpful Service ____ Knowledgeable Service ____ Total 100

MEASUREMENT SCALES – Nonmetric Paired Comparison Method: A scaling technique in which respondents are given pairs of stimulus objects and asked which object in a pair they prefer most. “Please circle the attribute describing a sales representative which you consider most desirable. ” Courteous versus Knowledgeable Friendly versus Helpful versus Courteous

MEASUREMENT SCALES – Nonmetric Sorting: A scaling technique in which respondents are asked to indicate their beliefs or opinions by arranging objects (items) on the basis of perceived importance, similarity, preference or some other attribute.

MEASUREMENT SCALES – Nonmetric Rank Order Method: A scaling technique in which respondents are resented with several stimulus objects simultaneously and asked to order or rank them with respect to a specific characteristic. “Please rank the following attributes on how important each is to you in relation to a sales representative. Place a “ 1” beside the attribute which is most important, a “ 2” next to the attribute that is second in importance, and so on. ” Courteous Service Friendly Service Helpful Service Knowledgeable Service ___ ___

Scale Development Practical Decisions When Developing Scales: • Number of items (indicators) to measure a • • • concept? Number of scale categories? Odd or even number of categories? (Include neutral point ? ) Balanced or unbalanced scales? Forced or non-forced choice? (Include Don’t Know ? ) Category labels for scales? Scale reliability and validity?

Scale Development Balanced vs. Unbalanced Scales? • “To what extent do you consider TV shows with sex and violence to be acceptable for teenagers to view? ” Balanced: __ Very Acceptable __ Somewhat Acceptable __ Neither Acceptable or Unacceptable __ Somewhat Unacceptable __ Very Unacceptable Unbalanced: __ Very Acceptable __ Somewhat Acceptable __ Unacceptable

Scale Development Forced or Non-Forced? “How likely are you to purchase a laptop PC in the next six months? ” Very Unlikely 1 2 3 4 Very Likely 5 6 __ No Opinion

Scale Development Category Labels for Scales? Verbal Label: • “How important is the size of the hard drive in selecting a laptop PC to purchase? ” Very Somewhat Unimportant 1 2 Neither Important Somewhat Very or Unimportant Important 3 4 5 Numerical Label: • “How likely are you to purchase a laptop PC in the next six months? ” Very Unlikely 1 2 3 4 Very Likely 5 Unlabeled: • “How important is the weight of the laptop PC in deciding which brand to purchase? ” Very Important Unimportant ___ ___ ___

MEASUREMENT SCALES Choosing a Measurement Scale: • Capabilities of Respondents. • Context of Scale Application. • Data Analysis Approach. • Validity and Reliability.

MEASUREMENT SCALES Assessing Measurement Scales: • Validity • Reliability Measurement Error : occurs when the values obtained in a survey (observed values) are not the same as the true values (population values)

Error Due to Bias and Chance – Bias - A systematic tendency to misrepresent the population. – The object of any experimental design is to eliminate bias and reduce chance error as much as possible.

Observation as A Data Collection Method ‘Observation involves the systematic observation , recording, description analysis and interpretation of people’s behaviour’

Observation Considerations: • Methods – human/mechanical/electronic • Useful where respondent cannot or will not articulate the answer. • Cannot be used to measure thoughts, feelings, attitudes, opinions, etc.

Types of observation The two main types – Participant observation – emphasises the discovery of meaning attached to actions (qualitative) – Structured observation – is concerned with frequency of actions (quantitative)

Participant Observation Definition: ‘Where the researcher attempts to participate fully in the lives and actions of subjects, enabling them to not merely observe what is happening but also feeling it’ , Adapted from Gill and Johnson (2002)

Participant observation: Points to consider – Used both as the principle research method and in combination with other methods – Researchers become immersed in the research setting – Researchers try to understand the process by which individual identity is constructed and reconstructed (symbolic interactionism)

Researcher roles in participant observation Typology of participant observation researcher roles Source: Gill and Johnson (2002)

Choice of participant observer role Determining factors – Purpose of the research and time available – Degree of suitability felt by the researcher – Organisational access – Ethical considerations

Data collection and analysis Types of data generated by participant observation – Primary observations – Secondary observations – Experiential data Delbridge and Kirkpatrick (1994)

Data collection and analysis Points to consider – Data may be classed as ‘descriptive observation’ and ‘narrative account’ (Robson, 2002) – Data recording method(s) will depend on the role – Data collection and analysis may be part of the same process – Use of analytic induction leads to redefinition of the original hypothesis

Data collection and analysis Points to consider – Threats to validity – The perspective of the subject - not the researcher – Advantages and disadvantages of participant observation are summarised in Table 9. 1 Saunders et al. (2009)

Structured Observation Points to consider – Structured observation is systematic and aims to establish straightforward facts – Structured observation was an important part of Mintzberg’s (1973) study of managerial work – Proliferation of the Internet potentially widens the scope of participant observation Saunders et al. (2009)

Structured observation Data collection and analysis – Choosing an ‘off the shelf’ coding schedule – Designing your own coding schedule – Combining both types of schedule – Use of simple (manual) or complex (computer) methods of analysis

Structured observation Threats to validity and reliability – Subject error – Time error – Observer effects and strategies to overcome this – habituation and minimal interaction Robson (2002)