Introducing framework synthesis and thematic synthesis Series of

Introducing framework synthesis and thematic synthesis Series of doctoral lectures on behalf of the Cochrane Qualitative and Implementation Methods Group http: //methods. cochrane. org/qi/ Systematic Reviews in social sciences. Mixed methods and qualitative research reviewed. Doctoral course, Ughent, March 21 -22, 2016 Prof. Dr. Karin Hannes Centre for Sociological Research Social Research Methodology Group, KU Leuven Karin. hannes@kuleuven. be Dr. Chris Carroll and Prof. Dr. Andrew Booth Scharr, University of Sheffield, UK

Methods of qualitative evidence synthesis § (Thematic synthesis); Critical Interpretive Synthesis; Meta-ethnography 1. Only include “good” qualitative studies (? ) 2. Constant comparison; iterative; interpretations generated from the data by reviewers 3. Create a theory Inductive (theory-generating) § Examples: § Thomas J, Harden A. Methods for thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology 2008; 8. § Campbell R et al. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Social Science & Medicine 2003; 65: 671 -684.

Methods of qualitative evidence synthesis § Framework synthesis: 1. Only include “good” qualitative studies (? ) 2. Map data from included studies onto an existing framework to test the framework/theory (a role for theory) 3. Test the “fit” of theory to the question Deductive (theory-testing) § Examples: § Oliver S et al: A multidimensional conceptual framework for analysing public involvement in health services research. Health Expectations 2008, 11: 72 -84. § Brunton G, Oliver S, Oliver K, Lorenc T. A Synthesis of Research Addressing Children’s, Young People’s and Parents’ Views of Walking and Cycling for Transport London, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2006.

Methods of qualitative evidence synthesis § “Best-fit” framework synthesis 1. Identify relevant pre-existing conceptual models or frameworks 2. Identify and extract all relevant qualitative studies satisfying review’s inclusion criteria (no exclusion based on quality) 3. Code data from included studies against framework 4. Use secondary thematic analysis/synthesis to generate completely new themes to supplement the framework’s themes 5. Create new framework and conceptual model or theory § Deductive and Inductive § Framework and Thematic/interpretive synthesis § Carroll C, Booth A, Cooper K. A worked example of “best-fit” framework synthesis: A systematic review of views concerning the taking of potential chemopreventive agents, BMC Medical Research Methodology 2011; 11: 29

Methods of qualitative evidence synthesis § “the output of some methods of synthesis (Thematic Synthesis, textual Narrative Synthesis, Meta-aggregation, Framework Synthesis, and ecological triangulation) is more directly relevant to policymakers and designers of interventions” § Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009; 11; 9: 59. § Framework and thematic synthesis: § Methods for qualitative evidence synthesis recognised to be of particular value to policy-makers; for solving or addressing service problems (rather than simply generating concepts and theories) § Thorne S. Toward Methodological Emancipation in Applied Health Research, QHR 2010; 21(4): 443 -44.

Thematic synthesis Framework synthesis

The case study The question: What are the barriers and facilitators of patient autonomy post stroke? The studies: 1. Proot I et al, Patient autonomy during rehabilitation: the experiences of stroke patients in nursing homes, International Journal of Nursing Studies, 2000; 37: 267 -76. 2. Mangset M et al, ‘We’re just sick people, nothing else’: . . . Factors contributing to elderly stroke patients satisfaction with rehabilitation, Clinical Rehabilitation, 2008; 22: 825 -35.

Data extraction § What is it? § An attempt to reduce a mass of material (your included papers) to a much smaller body of text and numbers, amenable to analysis and the interpretation of findings

Data extraction form § § § Location Setting Sample (n) Age Gender Ethnicity and other relevant variables (e. g. Marital status) Socio-economic status Study design Intervention (if any) Quality assessment criteria Results? Further citations § Format? ? ?

Example form

What results do you extract? § What is your question? § Question: What are the barriers and facilitators of patient autonomy post-stroke? Keep the question in mind as you read: Are the data relevant to this question? Is the question answered by the data?

Extraction of results § Results/findings sections of included studies: § Verbatim quotations and/or § Authors’ statements clearly-supported by data § “. . . It was often difficult to distinguish [Schutz’s] first- from second-order constructs” • Atkins et al. Conducting a meta-ethnography of qualitative literature: Lessons learnt, BMC Med Res Methodol 2008; 8: 21 • Schutz A. Collected Papers 1971; vol. 1: 361.

From: Mangset p. 829

From: Mangset p. 829

From: Mangset p. 829

Extracting data for thematic analysis

Extracting data for thematic analysis

Extracting or “coding” data using framework analysis § Identify or create a thematic framework or conceptual model § Develop own framework or themes or concepts for coding § Kirkevold M et al. Promoting psychosocial wellbeing following a stroke: developing a theoretically and empirically sound complex intervention, International Journal of Nursing Studies, 2012; 49: 386 -97.

Extracting or “coding” data using framework analysis Code number Theme Definition 1 Dignity A self-concept characterised by self-valuing, selfacceptance, usefulness and belief in oneself 2 Well-being A basic mood of joy, pleasure and wellbeing; the absence of feelings of emptiness or sadness 3 Coping Ability to adapt and cope within home and hospital settings 4 Family Support Relations with immediate family 5 Support for patient’s coping efforts Patients take responsibility and are supported by others in this 6 Bodily changes and impairments Difficulties generated by residual or post-stroke conditions Handout 1

Extracting or “coding” data using framework analysis

Extracting or “coding” data using framework analysis

Extracting or “coding” data using framework analysis

Extracting or “coding” data using framework analysis

What now? § You have extracted your data § You may have coded your data against your framework (framework analysis) § You may have only extracted relevant data (without using a framework) (the first stage of secondary thematic analysis) § You may have coded some of your data but not others (“best fit” framework synthesis) § You now need to assign themes to those data without themes, i. e. Using secondary thematic analysis (data reduction, display, conclusion) § How do you do it?

Generating new themes § “Evidence does not speak for itself. It requires interpretation. . on the basis of our personal experience and a range of ideas that we use to make sense of our observations” § Kelly M et al. Evidence-based public health: A review of the experience of NICE developing public health guidance in England, Social Science and Medicine, 2010; 71: 1056 -62.

Developing new themes C. “…“I will take that paper when P. is finished with it. I want to study it. Then I'll take my shoes off, and practise as I see it in the picture” (p 04 b, p. 11), Proot, p. 271 Other / New theme? § Interpret and reflect § Create § Revisit and revise § Discuss § Agree. . . = Learning self-help skills / guided self-determination

Synthesis

Synthesis

A new framework Code number Theme Definition 1 Dignity A self-concept characterised by self-valuing, selfacceptance, usefulness and belief in oneself 2 Well-being A basic mood of joy, pleasure and wellbeing; the absence of feelings of emptiness or sadness 3 Coping Ability to adapt and cope within home and hospital settings 4 Family Support Relations with immediate family 5 Support for patient’s coping efforts Patients take responsibility and are supported by others in this 6 Bodily changes and impairments Difficulties generated by residual or post-stroke conditions 7 Self-determination Define theme as a concept, based on data 8 Need for information Define theme as a concept, based on data 9 Lack of shared decisionmaking Define theme as a concept, based on data 10 Acceptance of therapy Define theme as a concept, based on data

A new framework and synthesis Theme Categories Dignity Well-being Psychological factors Self-determination Coping Bodily changes and impairments Physical factors Need for information Lack of shared decision-making Family Support for patient’s coping efforts Acceptance of therapy Interaction with professionals and family

Synthesis § Alternatively, go beyond the framework to create a new conceptual model or theory. . . § Revisit data to explore the relationships between themes or findings of your framework § The “richer” or “thicker” the data, the deeper the resulting model or theory § Narrative and diagrammatic representations

Do you stop there? § Test your synthesis? § Disconfirming cases § Booth A, Carroll C, Illott I. Desperately Seeking Dissonance: Identifying the “Disconfirming Case”, Qualitative Evidence Synthesis, Qualitative Health Research 2012 (in press). § Morse, J. M. The significance of saturation. Qualitative Health Research 1995; 5, 147149. § Sensitivity analysis § By quality, population, location, setting etc. § By frequency and “thickness” § Carroll C, Booth A, Lloyd-Jones M. Should we Exclude Inadequately-reported Studies from Qualitative Systematic Reviews? An Evaluation of Sensitivity Analyses in Two Case Study Reviews, Qualitative Health Research 2012; 22(10) § Boeije et al (2011) Making a difference: towards a method for weighing the evidence in a qualitative synthesis, Journal of Evaluation in Clinical Practice 2011 17(4): 657 -63

What is the purpose of your review and synthesis? § To illuminate experiences of clients, professionals, policy makers, … and to develop theory § To be useful § To answer questions and to solve problems that cannot be addressed by quantitative research

Conclusions § Different methods for qualitative evidence synthesis § Choose an appropriate method § Extract and interpret data § Synthesis § Test your synthesis § Integrating and writing-up. . .