Chapter 5 Strategies for Mixing Prior to Analysis

Chapter 5: Strategies for Mixing Prior to Analysis Purpose: To describe and illustrate a variety of strategies for mixing during early phases of a research project, including through the wording of research questions, sampling procedures, and coding during data collection.

Goals of the Chapter 1. To describe and illustrate how mixing at the design phase can be reflected in a purpose statement and research questions. 2. To identify three approaches to writing mixed method research questions. 3. To review quantitative, qualitative, and mixed method sampling strategies. 4. To illustrate five types of mixing in the chapter exemplar. An Introduction to Fully Integrated Mixed Methods Research 2

Example from Popular Media: Blue Zones • Mixing at sampling – census data was used to identify geographical areas with a high concentration of people living past the age of 100. • Initial investigation in Sardiniaan island off Italy An Introduction to Fully Integrated Mixed Methods Research 3

Three Approaches to Writing MM Research Questions (Table 5. 1) 1. Link qualitatively and quantitatively derived variables into a single sentence. 2. Pose two separate, but interlinked research questions: one with the qualitatively derived variable and a second with the quantitatively derived variable. 3. A separate research question that is explicitly labeled as a mixing question. An Introduction to Fully Integrated Mixed Methods Research 4

Example of Strategy #1: QUAL and QUANT Blended in One Sentence • Durksen & Glasson, 2012 • Chapter 2 Exemplar • "The current study examines how pre-service teachers' commitment and engagement develop during high-stakes final teaching practicum, with attention paid to broad patterns (quantitative analysis), and the particular reasons and experiences associated with increasing and decreasing patterns (qualitative case study analysis). " (p. 35) An Introduction to Fully Integrated Mixed Methods Research 5

Example of Strategy # 2: Separate but Linked QUAL and QUANT Research Questions • Elliott et al. , 2014 • From Chapter 6 • 1. "To what extent is perceived neighborhood belonging associated with well-being in three separate cohorts of older adults? " (p. 45) • 2. "How do older adults conceptualize and talk about neighborhood belonging in the context of a semi-structured, biographical interview? " (p. 45) An Introduction to Fully Integrated Mixed Methods Research 6

Example of Strategy # 3: Explicitly Labeled MM Research Question • Creamer & Ghoston, 2013 • From Chapter 6 • 1. (QUAL) "What values, skills, and outcomes are identified in mission statements in colleges/schools of engineering…? " (p. 3) • 2. (MIXED) Do the values endorsed in mission statements differ between institutions with low and higher than average enrollment of women? An Introduction to Fully Integrated Mixed Methods Research 7

QUANT and QUAL Approaches to Sampling • Probability Sampling (QUANT) • Involves random selection • Allows for claims of generalizability that rest on the assumption that the sample represents the population. • Purposeful Sampling (QUAL) • Participants are selected who have experienced the phenomenon. • Does not allow for claims of generalizability. An Introduction to Fully Integrated Mixed Methods Research 8

Differences Between QUAL and QUANT Sampling Approaches (Teddlie and Tashakkori, 2009) Probability Purposeful Sample size – Large enough to be representative Sample size - Typically small (30 or fewer) but expectations vary by method Generally focuses on breadth of information from a wide-range of cases. Generally focuses on depth of information from a relatively small number of cases. Timing of selection – generally established at the design phase Timing of selection – established during the design phase but also during the analysis (i. e. theoretical sampling) Method of selection – formal or procedural, as with the use of a random number generator Method of selection – features judgment and selection for information rich cases. An Introduction to Fully Integrated Mixed Methods Research 9

Similarities between QUAL and QUANT sampling strategies 1. Both are applied systematically. 2. Both are designed for the purpose of answering the research questions. 3. Both are an indication of the priority of a mixed method study. 4. Both are linked to issues of external validity. 1. 2. In QUAL, this is referred to as transferability. In QUANT, this is referred to as generalizability. An Introduction to Fully Integrated Mixed Methods Research 10

Mixed Method Sampling Procedures • Mixed methods sampling procedures use various approaches to combine a traditional QUANT approach to sampling (i. e. , probability) with a qualitative (i. e. purposeful) approach to sampling. 1. 2. 3. Concurrent mixed methods samplinga single sample of participants, with quantitative and qualitative data collected simultaneously. Sequential mixed method sampling –a sampling strategy in a subsequent phase is directly linked to the results of analytical procedures in an earlier strand of the study. Multi-level mixed method sampling – data collection in multiple phases with people at different levels of an organization. An Introduction to Fully Integrated Mixed Methods Research 11

Example of Sequential Mixed Method Sampling Plan 1. PHASE 1 - QUANT 1. A survey distributed to a random sample of students 2. PHASE 2 – QUAL 1. Purposefully select ”typical case” respondents based on mean score on factors or scales. 2. Then use maximum variation sample to get a mix of respondents by gender, age, family status An Introduction to Fully Integrated Mixed Methods Research 12

Example of Multi-Level MM Sampling in a School Effectiveness Study Level Example of Sampling Plan School Districts Could be stratified by size or by urban/rural location Schools Could be stratified by an effectiveness measure or a measure of student performance on a standardized test. Teachers Purposefully selected Students Purposefully selected. An Introduction to Fully Integrated Mixed Methods Research 13

Nested or Identical Samples • Described as an inherently mixed method sampling procedure (Bazeley, 2009) • A nested sample is when the respondents for one phase are a subset of a larger group used in another phase. • Associated with greater opportunities for mixing of QUAL and QUANT data. An Introduction to Fully Integrated Mixed Methods Research 14

All of the Chapter Exemplars Used Either a Nested or Identical Sample Exemplar Topic Timing of Data Analysis Sample Type Catallo et al. , 2012 Intimate partner violence Iterative Nested Cooper, 2014 Teaching practices Sequential Nested Creamer & Ghoston, 2012 Mission statements Sequential Identical Durksen & Klassen, 2012 Pre-service teachers Concurrent Identical Gasson & Waters, 2013 On-line collaboration Sequential Identical Elliott et al. , 2014 Elderly and sense of community Iterative Nested Jang et al. , 2008 School leadership Iterative Nested Mc. Mahon, 2007 Rape myths and college athletes Sequential Nested Young & Jaganath, 2013 HIV education Sequential Identical An Introduction to Fully Integrated Mixed Methods Research 15

Design Features of the Chapter Exemplar by Jang et al. , 2008 Design Feature Rationale/Purpose Complementarity Priority Qualitative Timing of Data Collection Concurrent Timing of Data Analysis Iterative or Multi Phase Mixing Fully Integrated- Mixing at all Stages An Introduction to Fully Integrated Mixed Method Research 16

Types of Mixing in the Chapter Exemplar by Young & Jaganath, 2013 Phase of the Research Process Type of Mixing Design/Research Question One qualitative; one quantitative research question. Sampling Identical sample for both phases with control group Data Collection A time stamp was associated with Facebook posts, enabling the ability to track changes in attitude over time. Data Analysis Facebook posts were open-coded to identify topic; changes in topic were calculated over time. Drawing Conclusions/Inferences A meta-inference was produced. An Introduction to Fully Integrated Mixed Method Research 17

Meta Inference in the Chapter Exemplar by Young & Jaganath, 2013 META INFERENCE: An increase in the amount of conversational space devoted to HIV prevention significantly predicted a request for HIV testing. QUAL Conclusion: There were five conversational themes on the Facebook page, one related to HIV prevention. QUANT Conclusion: There was an increase in prevention-related conversations on the Facebook page. An Introduction to Fully Integrated Mixed Methods Research 18

The Chapter Exemplar is a Fully Integrated Mixed Method Study Mixing occurs at each of the five phases of the research process. An Introduction to Fully Integrated Mixed Methods Research 19

Concluding Comments about Mixing at Data Collection • The sampling plan determines if your results are generalizable to a larger population. • Most mixed method studies combine purposeful and probability sampling. • Creative combinations of sampling strategies can provide an opportunity for a study that has both breadth and depth. • Using either an identical or nested sample is inherently a mixed method sampling procedure. • Research questions that link QUAL and QUANT variables set up the opportunity for mixing during analysis. An Introduction to Fully Integrated Mixed Methods Research 20

Review Quiz Questions 1. What are the differences between QUAL and QUANT sampling procedures? 2. What are the similarities between QUAL and QUANT sampling procedures? 3. What are examples of mixed method sampling procedures? 4. What are three ways to write research questions in a mixed method study? 5. What does it mean to have a fully integrated mixed method study? An Introduction to Fully Integrated Mixed Methods Research 21