Qualitative Data Analysis Techniques for Drawing Themes insert

Qualitative Data Analysis Techniques for Drawing Themes [insert date] Qualitative Methods in Evaluation of Public Health Programs Session 6

The evaluation process

Learning objectives By the end of this session, participants will be able to: § Explain qualitative data analysis and its approaches § Describe stages in conducting qualitative analysis § Develop a coding structure for categorizing data § Apply the analytical method for drawing themes

Overview of qualitative analysis § Organizing and interrogating data in ways that allow evaluators to see patterns, identify themes, discover relationships, develop explanations, and make interpretations (Leech & Onwuegbuzie, 2007) § Systematic search for meaning in data

Qualitative data Large volumes of data are reduced or configured. § A process of selecting, focusing, simplifying, abstracting, and transforming data Forms of qualitative data: § Text (written field notes or transcripts) § Image (video/pictures) § Sound (audio recordings)

Approaches for qualitative analysis § Grounded theory § Framework analysis § Discourse analysis § Thematic analysis*** § Others…

Analysis techniques § Grounded theory: Data are analyzed and conceptualized to construct an explanatory theory of basic social processes § Framework analysis: Adapts to evaluation that has specific questions, a limited time frame, a predesigned sample, and a priori issues § Discourse analysis: Concerned with language use and how individuals accomplish personal, social, and political projects through language

Thematic analysis § A method for systematically identifying, organizing, and offering insight into patterns of meaning (themes) across a dataset (Braun & Clarke, 2012) § Entails careful reading and rereading of data (Guest & Macqueen, 2011) § Themes are identified and used to answer a specific evaluation question

Drawing themes

Stages of analysis

Stages of data coding/analysis Developing code book and testing reliability of codes Summarizing data and identifying initial themes Identifying themes Iterative process, not linear Coding processes Connecting the codes and identifying themes Corroborating and legitimizing coded themes

Coding What is a code? § Label for a feature of the data (e. g. , qualitative indicators) that is potentially relevant to the research question—a segment of information that gives a sense of direction What to code? § Field notes, transcripts, observational notes, images, voice recordings, sound, etc. Who? § Somebody who participates in the fieldwork § Somebody who has a deep understanding of the program and evaluation questions When? § Coding must be done after one or two rounds of “reading” raw data

Code identification Thematic analysis Inductive Exploratory Deductive Top-down Bottom-up Data driven Confirmatory Theory-based E. g. , why are women in District Y not reporting cases of intimate partner violence at the facility? Hybrid combination E. g. , women reporting intimate partner violence in District Y has declined owing to increased awareness of IPV as a crime.

Code identification and listing can entail different approaches. A combination, or hybrid, is appropriate for qualitative evaluation.

Hybrid approach for Qualitative evaluation Combining inductive and deductive approaches: § Commonly used § Highly practical for qualitative evaluation purposes For example, once patterns and themes have been established through inductive analysis, the final confirmatory stage may be deductive and affirming the authenticity and appropriateness of the inductive (Patton, 2003).

Identifying codes for qualitative evaluation § Codes are driven by the evaluation questions § Stay focused on primary and specific evaluation questions/issues (program processes, outcomes, and impact) § Qualitative themes: e. g. , processes leading to the outcomes, evolving relationships/ involvement, etc.

Developing a codebook § Begin codebook development once there is agreement on codes— however, definitions are still proposed and reviewed § Serves as data management tool for organizing segments of similar or related texts to assist in interpretation § It provides a clear trail of evidence for credibility of study

Codebook structure Codebook may include the following components: 1. A code 2. A brief definition 3. A full definition 4. Guidelines for when to use the code 5. Guidelines for when not to use the code 6. Examples from the text

Codebook Example of a codebook entry

Testing reliability of codes § Determine the applicability of codes to raw information § Key to select some of the raw documents as test pieces § This should follow the coding process of the documents using the defined codes

Testing the codebook Individual exercise: Using the codebook in the case study, practice applying codes A. 04, A. 05, and/or A. 06. to the transcript. (10 mins. ) Group discussion (10 mins. ): 1. Which codes are relevant? 2. Which areas had no codes?

Applying codes § Apply codes in the codebook to text § Use to identify meaningful units of text § Can be software assisted (practical session is next)

Coding systematically Ask yourself the following questions: § What is this saying? § What does it represent? § What is this an example of? § What do I see is going on here? § What is happening? § What kind of events are at issue here? § What is this trying to convey?

Identifying and reviewing themes

What is a theme? § Shows patterns within the data § Broader than codes § Connects codes across and between data § Captures something about the data in relation to the evaluation question § Tells a “story” from a group of codes

Examples of themes or patterns Patterns in behaviors, experiences, views, knowledge, or emotions in relation to the intervention/program being evaluated. § E. g. , what are women saying about spousal abuse after completing an intimate partner violence educational program?

Theme search Searching for themes is an active process § Evaluator generates or constructs themes rather than discovers them § Refocus the analysis at the broader level of themes, rather than codes § Entails sorting the different codes into potential themes, and collating all the relevant coded data extracts within the identified themes § Analysis of codes

Connecting codes and identifying themes § Review coded data to identify areas of similarity and overlap between codes, and across all sets of data § Identify differences between separate groups

Reviewing themes Two levels of reviewing and refining 1. Reviewing at the level of coded data extracts § Read all the collated extracts for each theme, and consider whether they appear to form a coherent pattern 2. Consider the validity of individual themes in relation to the data set § Ensure your thematic map “accurately” reflects the meanings evident in the data set as a whole

Thematic maps Using visual representation of themes and relationships between codes. These could be: § Mind maps § Charts § Tables Maps, charts, and tables are used to construct a process-outcome matrix for the program (Patton, 2003).

Drawing thematic maps Theme Codes ? ? ? ? Data extract/ quotes ? ? ?

Evaluation question How acceptable is VAW among community members?

Small group activity Connecting codes/extracts to create themes § Split into groups of 3– 4 § Create various piles of the codes/quotes related to each other § Label piles with theme(s); a word or phrase (20 mins. ) § Give a rationale for that theme and discuss in plenary (20 mins. )

Key questions to ask § Is this a theme? § What is the quality of this theme? § What are the boundaries of this theme? § Are there enough data to support this theme? § Are the data too diverse or wide ranging? § Do themes connect with one another to enable the construction of process-outcome matrix for the program?

Corroborating and legitimatizing Coded themes § Process of confirming results § Selected themes must be related, but not overlap, and directly address an evaluation question § Corroborate to avoid fabrication of evidence § This is an iterative process

Tips for conducting analysis 1. Start analysis immediately 2. Involve more than one person 3. Leave enough time and funds for analysis and writing § To be discussed further in upcoming sessions

References Leech, N. l. , & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly; 22(4). Braun, V. , & Clarke, V. (2014). What can “thematic analysis” offer health and wellbeing researchers? International Journal of Qualitative Studies on Health and Wellbeing; 9(26). Braun, V. , & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology; 3(2): 77 – 101. Guest, G. , Macqueen, K. M. , & Namey, E. E. (2012). Applied Thematic Analysis. Thousand Oaks, CA, USA: SAGE Publications, Ltd. Patton, M. Q. (2003). Qualitative evaluation checklist.

This presentation was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14 -00004. MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc. ; Management Sciences for Health; Palladium; and Tulane University. Views expressed are not necessarily those of USAID or the United States government. www. measureevaluation. org