MIXED METHODS FOCUS ON DISSERTA TIONS Ryan Rominger

  • Slides: 27
Download presentation
MIXED METHODS: FOCUS ON DISSERTA TIONS Ryan Rominger, Ph. D. , LCPC -PIT Associate

MIXED METHODS: FOCUS ON DISSERTA TIONS Ryan Rominger, Ph. D. , LCPC -PIT Associate University Research Chair Center for Leadership Studies and Organizational Research Adjunct Research Faculty: Sofia University, CIIS Renew Wellness Center, therapist-in-training

MIXED METHODS Agenda for today: Context for today’s discussion Main ‘types’ of MM designs

MIXED METHODS Agenda for today: Context for today’s discussion Main ‘types’ of MM designs Those pesky, yet informative, symbols & boxes Dissertation Examples Questions

http: //journals. sagepub. com/toc/mmra/12/2

http: //journals. sagepub. com/toc/mmra/12/2

MIXED METHODS Assumptions Often, philosophical foundation is pragmatism (use the method that gets the

MIXED METHODS Assumptions Often, philosophical foundation is pragmatism (use the method that gets the data which answers your questions) If not, know what it is and how it influences method strands Combining Qual and Quant help minimize problems within both But if poor quality, can include problems of both Combining helps ‘triangulate’ phenomenon to better understand what is happening If not well coordinated, rather than 2 blind people feeling same elephant, can be 1 feeling elephant, another feeling a lion – i. e. , need to have purposeful, meaningful coordination = ALIGNMENT

Triangulation – to confirm results, look at phenomenon from different angles, seeks convergence and

Triangulation – to confirm results, look at phenomenon from different angles, seeks convergence and corroboration Complementarity – elaborate and clarify complex phenomena, gain information about different facets Development – results from one method helps form the next method of inquiry Initiation – not always intentional, used to look at dissonance between results that leads to generativity Expansion – used to increase scope, different methods/strands do not measure same phenomenon Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease. Doctoral dissertation. 5 PURPO SES OF MIXED METHO D RESEAR CH

MIXED METHODS – MAIN MODELS 1. Sequential mixed methods Researcher “seeks to elaborate on

MIXED METHODS – MAIN MODELS 1. Sequential mixed methods Researcher “seeks to elaborate on or expand on the findings of one method with another method” (C&PC, p. 14) Quant Qual or Qual Quant 2. Concurrent mixed methods Researcher “merges quantitative and qualitative data” (C&PC, p. 14) collection Collects both forms of data at same time Integrates the information during analysis & interpretation

MIXED METHODS – MAIN MODELS 3. Transformative mixed methods Uses overarching theoretical lens (or

MIXED METHODS – MAIN MODELS 3. Transformative mixed methods Uses overarching theoretical lens (or umbrella) which provides framework (C&PC, p. 15) Collect multiple types of data to answer questions about the topic Particularly important in interdisciplinary or transdisciplinary studies Ex: In psychology, lens could be queer theory, investigating the expression of gender among transgender individuals in early adolescence

MIXED METHODS – RESEARCH QUESTIONS 1. Think about order of data collection. If sequential,

MIXED METHODS – RESEARCH QUESTIONS 1. Think about order of data collection. If sequential, ask first RQ first, second If concurrent, ask RQ based on weight or importance – if quant more heavily weighted, start w/quant research hypothesis. If qual more heavily weighted, start with qual research question. Or, use ‘general’ RQ first. 2. Include a specific MM question. Addresses mixing of data; could focus on procedures or content 3. At what point do you merge strands? Data collection, analysis, synthesis, conclusions

MIXED METHODS MM Strategies Important concepts: Timing, Weighting, Mixing, Theory + = concurrent data

MIXED METHODS MM Strategies Important concepts: Timing, Weighting, Mixing, Theory + = concurrent data collection à= sequential data collection Capitalization = weight/priority Quan/Quant = quantitative Qual = qualitative QUANT/qual = qualitative embedded in quantitative design QUANT qual

MIXED METHODS GRAPHICS Sequential Explanatory Designs QUANT qual • Data collection • Data analysis

MIXED METHODS GRAPHICS Sequential Explanatory Designs QUANT qual • Data collection • Data analysis Interpretation of entire analysis Example: Studying “significant life events” during college of recent college graduates. QUANT – 500 college students complete survey. qual – 30 “exemplar” students chosen and interviews about their experience. Interpretation done after all data collected analyzed.

MIXED METHODS GRAPHICS Concurrent Embedded Design (b) #2 #1 qual quant QUANT QUAL Analysis

MIXED METHODS GRAPHICS Concurrent Embedded Design (b) #2 #1 qual quant QUANT QUAL Analysis of Findings Ex #1: Efficacy based RCT experiment on meditation with 6 th graders to help increase focus (QUANT). Interview several of these students to find out what it was like for them (qual).

Decision Based Learning Model for Mixed Methods Desi Would a purely quant study answer

Decision Based Learning Model for Mixed Methods Desi Would a purely quant study answer your RQ? Yes no use Quant would a purely qual study answer your RQ? yes use QUAL no Consider a MM design

Do you need one set of data to inform the collection of the next

Do you need one set of data to inform the collection of the next set? yes no Use Sequential designs Will one data strand be primary? yes use embedded designs no use a concurrent design

Will Qualitative data be primary? yes no Will quantitative data be primary? It is

Will Qualitative data be primary? yes no Will quantitative data be primary? It is likely Explorator y yes It is likely Explanator y no Then quant and qual are equally weighted

DISSERTATION EXAMPLES 1. Burrows, T. J. (2013). A preliminary rubric design to evaluate mixed

DISSERTATION EXAMPLES 1. Burrows, T. J. (2013). A preliminary rubric design to evaluate mixed methods research. Virginia Polytechnic Institute and State University. 2. Method: a. Exploratory multi-phase mixed-method content analysis study. 12 expert participants. Qual then quant. b. RQ: What are the criteria for evaluating mixed-method research? 3. Critique: a. Clearly written, and both method and topic are MM research=alignment. b. Used visual representations of the method and timing of data collection and analysis. c. Methods chapter organized by phase of study, with clear descriptions of analysis. Themes from phase 1 resulted in rubric to be tested in phase 2.

RUBRIC FOR EVALUATING MM RESEARCH

RUBRIC FOR EVALUATING MM RESEARCH

DISSERTATION EXAMPLES 1. 2. Eckhardt, A. L. (2013). Fatigue as a symptom of coronary

DISSERTATION EXAMPLES 1. 2. Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease. University of Illinois at Chicago. Method: a. Series of articles/manuscripts combined into ‘dissertation’. b. c. 3. partially mixed sequential dominant status design (QUAN → qual). QUAN with 102 participants completing several measures (fatigue, mood, quality of life, etc. ), qual with purposive sample subset who were interviewed about fatigue. Critique: a. **Great description of different types of mixed methods, including examples of different types of MM designs, qualities of MM research. b. Provides a unique approach to a dissertation (several manuscripts) with in-depth discussion of MM designs, theory, and paradigms of research especially with regard to mixing of paradigms (positivist and constructivist).

O’Cathain, A. (2010). Assessing the quality of mixed methods research: Toward a comprehensive framework.

O’Cathain, A. (2010). Assessing the quality of mixed methods research: Toward a comprehensive framework. In A. Tashakkori & C. Teddlie (Eds. ), SAGE Handbook of Mixed Methods in Social and Behavioral Research (pp. 531 -555). Los Angeles: SAG As cited in Eckhardt, A. L. (2013). Fatigue as a symptom of coronary heart disease. University of Illinois at Chicago.

DISSERTATION EXAMPLES 1. Wright, J. B. (2018). Physician executives – A mixed-methods examination of

DISSERTATION EXAMPLES 1. Wright, J. B. (2018). Physician executives – A mixed-methods examination of leadership characteristics and how physicians learn to lead. University of Charleston-West Virginia. 2. Method: a. Called a explanatory sequential mixed-methods interpretive phenomenological analysis, with goal of determining physician characteristics connected to good leadership b. Surveyed and interviewed 6 physician executives to discuss “what competencies, thought processes, and techniques are important for future physician executives in healthcare to lead effectively” 3. Critique: a. Quant ‘strand’ is weak with only 6 participants; limits conclusions of group analysis used in dissertation. In MM it is best to have strong quant and qual strands. b. Writes of ‘phenomenological’ interviews, but only a single onehour interview with each participant. Maybe better described as hermeneutical RM, although have seen IPA with only 1 interview.

DISSERTATION EXAMPLES 1. Stephan, K. P. (2017). Does Mathletics, a supplementary digital math tool,

DISSERTATION EXAMPLES 1. Stephan, K. P. (2017). Does Mathletics, a supplementary digital math tool, improve student learning and teaching methods at three private Catholic schools in Florida? – A mixed methods study. Creighton University. 2. Method: a. Called a convergent, parallel mixed methods study. b. Quant strand used comparison groups (6 th grade classes) and Mathletics groups, comparing Iowa Test of Basic Skills (outcome). Qual strand used interviews with teachers. 3. Critique: a. Similar to program evaluation models, as this is essentially a outcome based program evaluation to assess whether implementation would benefit upstream and downstream constituents. b. Clear research questions for each strand, convergent process after separate parallel data collection and analysis, and Qual definitely expanded the Quant results. Member checking, comparison table, etc. to improve validity. c. No graphic representation.

DISSERTATION EXAMPLES 1. 2. Boyle, E. D. (2018). Executive selection process of U. S.

DISSERTATION EXAMPLES 1. 2. Boyle, E. D. (2018). Executive selection process of U. S. defense contractors: A mixed methods study. University of Phoenix. Method: a. Called a convergent, parallel mixed methods study. (No graphic) b. 3. 2 pilot tests and main study. Main study had 37 in quant strand, and qual strand had 3 participants. Researcher did not meet his goals of 159 in quant and 6 -12 in qual after 3 months of participant recruitment. Critique: a. Strong argument for use of mixed method design. Strong, statistical argument for number needed in quant strand. However, did not meet participant number goals, which could have made study stronger. b. While survey, part 2 of which used Likert scales, was created by the author (thus no prior reliability and validity), there was a theoretical base for the questions AND author conducted 2 pilot tests to test content validity and reliability. c. Qual results provided confirmation of quant results.

MIXED METHODS Challenges 1. Can take more time 2. Can take more resources (money,

MIXED METHODS Challenges 1. Can take more time 2. Can take more resources (money, personnel) 3. Need skills for both Qual and Quant methods a) Individual b) Team 4. Trouble determining when to merge/synthesize strands 5. Trouble with unclear research questions – note suggestion to have one overall ‘mixed method’ RQ and within that, method specific RQs 6. Some combinations do not mix well/easily a) Methods – classical phenom & large scale quant b) Theories – positivist (RCT) and constructivist (narrative)

QUESTIONS? EMAIL: RYANROMINGER 440@EMAIL. PHOENIX. EDU Next Webinar: DBL Model & Software April 25

QUESTIONS? EMAIL: RYANROMINGER 440@EMAIL. PHOENIX. EDU Next Webinar: DBL Model & Software April 25 th, @ 4: 00 AZ time