Qualitative Research Using Codebooks and Describing Themes Gery

Qualitative Research Using Codebooks and Describing Themes Gery W. Ryan RAND Corporation & Tracy Van Holt East Carolina University

The Basics of Qualitative Analysis • • • Identifying themes Building and applying codebooks Describing themes Make comparisons Building and testing models

Codes, Codebooks, and Coding • What is the difference? – Codes are a short hand notation for themes – Codebooks are organized lists of themes – Coding is the act of linking themes with chunks of qualitative data • Types of Codes – Structural Codes • Describe characteristics of the data itself • Answers who, what, where, and how the data were collected • Typically, a priori – Thematic Codes • Link themes with particular instances in a set of data (e. g. , referential codes or tags) • Most commonly used type of code • Memos – Used to annotate data with running thoughts of the investigator • Detailed ideas about themes, • Emerging hypotheses or theories • Additional comments on data themselves • Hyperlinks – Link instance of data with another instance of data

Examples of Structural Codes, Thematic Codes and Memos Structural Codes Work Interview Group Topic Thematic Codes & Memos Speaker’s Gender Speaker Moderator If we were going to offer a parenting program, what would you like to hear that would catch your interest or make you want to attend this type of program? How could we market it? 1 I think you’d have a hard time marketing it to men. I don’t know…. I know my wife would like it a lot. She would lead me into it, but …. ♂ 2 Probably stressing the communication, somehow portraying that and the problems communicating because that’s a universal. We were talking yesterday, that’s the biggest challenge, or one of the biggest challenges, so somehow portraying that well and grabbing attention that way. [This is Part II of a 2 -day session. During previous day, participants discussed how they communicated with their adolescents about sex and relationships. ] ♀ 3 I think you’d have to sell the success. What is it that you plan on accomplishing with this. If you can tell me that at the end of this thing, our relationship will be better, my daughter will be a better achiever, you know, I won’t loose my temper as often, that would do it. 4 Money back guarantee. Topic #1 ♂ Worksite #1 ♀ Topic #2 Transcript . . . Moderator If we said, “OK, tonight we want you to have a conversation with your child about sex. ” What would be hard or easy about that? 4 I’ve taken a couple of classes through the years, and my kids get a kick out watching me sit and do homework. And so if I let them know, this is my homework, and gosh mom gets homework too. Ya, and that could be kinda fun for them to know that you’ve gotta sit and read a book and do a report or have a discussion. You know, you’re my homework tonight. ♀ . . . Legend: Italics = Theme #1, Underline = Theme #2, [Bold] = Memo Source: Eastman et al. (n. d. )

Examples of Coding Descriptions Mnemonic Short Description Detailed Description OTC Over-the-Counter Drugs The use of pharmaceutical products purchased over-the-counter without a prescription with the expressed intent of alleviating signs or symptoms. Inclusion Criteria Nonprescription medications purchased over-the-counter for a current illness Exclusion Criteria Medicines that have been recommended by a doctor or other professional for the current illness episode. Medications purchased for the preventative purposes. Typical Exemplars Aspirin, decongestants, laxatives, cough medicine. Atypical Exemplars Close but no Mnemonic Short Description Detailed Description Prescription medication that was left over from a previous illness episode. Vitamins Home. R Home Remedies made at home to alleviate signs & symptoms. Inclusion Criteria Remedy must be administered internally or applied externally. Exclusion Criteria Products sold in pharmacies for specific purpose of stopping illnesses. Behavioral modifications such as going to bed early, avoiding cold water. Preventative treatments such as regular vitamin intake. Typical Examplars Hot teas, chicken soup, compresses, massages, rice water Atypical Exemplars Large doses of vitamins Close but no Behavioral modifications such as bed rest, not going to school, going to sleep early

Example codebook for short responses on international science test (IEA 1994). Problem K 10 Air is colorless, odorless, and tasteless. Describe one way that air can be shown to exist. Correct Response 10 Mentions that you can feel or see effects of air movement. Examples: Wind, flags blowing, waving arms, spreading smell. 11 Mentions that (light) things fall slowly. Example: Air resistance makes things fall slowly. 12 Refers to the fact that air can be weighed. 13 Mentions that balloons or tires, etc. can be filled with air. 14 Refers to air pressure. Example: Barometers show that air exists. 15 Refers to being able to ‘see’ air. Example: You can see air bubbles in water. 19 Other correct. Incorrect Response 70 We can breath air. Refers only to the end of oxygen or air for life and other processes. Examples: All living things need air/oxygen. Candles extinguish without air. We could die if there was no air. 72 Refers to seeing water vapor. Examples: You can see water “vapor” when breathing out [on cold days or on a mirror or glass]. 76 Merely repeats information in the stem. 79 Other incorrect Nonresponse 90 Crossed out/erased, illegible, or impossible to interpret. 99 BLANK

Codebook Development • Development Approaches – Inductive: All themes are induced from the data itself – A priori: All codes come from theory or previous investigator experience – Combination: Starts with a set of a priori codes and builds and modifies as necessary • Organization – Parents & Children • CODE A – Subcode A. 1 » Code Sub-sub A. 1. 2 – Subcode A. 2 • CODE B – Inclusive Codes: When two codes can be assigned to the same chunk of data without being inconsistent. – Exclusive Codes: When only one of the codes in a pair can be assigned to the same data chunk.

Examples of Codebook Modifications Original 1. 0 Illness Characteristics 1. 1 Diagnosis (Diag) 1. 2 Signs/Symptoms (S/S) 1. 3 Duration (Dur) 1. 4 Severity (Sev) 1. 5 Perceived causation (Cause) 2. 0 Treatment Behavior 2. 1 Home remedies (Home. R)2 2. 2 Over-the-counter drugs (OTC) 2. 3 Western medical providers 2. 3. 1 Government Clinic (W-Clin) 2. 3. 2 Catholic Hospital (W-Hosp) 2. 2 Traditional Medical Providers 2. 2. 1 Herbalists (T-Herb) 2. 2. 2 Diviners (T-Div) 3. 0 Patient Characteristics 3. 1 Gender (P-Gend) 3. 1. 1 Male 3. 1. 2 Female 3. 2 Age (P-Age) Expanded 1. 0 Illness Characteristics 1. 1 Diagnosis (Diag) 1. 2 Signs/Symptoms (S/S) 1. 2. 1 Head (S-Head) … 1. 2. 5 Other (S-Other) 1. 3 Duration (Dur) 1. 4 Severity (Sev) 1. 5 Perceived causation (Cause) 2. 0 Treatment Behavior 2. 1 Home remedies (Home. R)2 2. 2 Over-the-counter drugs (OTC) 2. 3 Western medical providers 2. 3. 1 Government Clinic (W-Clin) 2. 3. 2 Catholic Hospital (W-Hosp) 2. 2 Traditional Medical Providers 2. 2. 1 Herbalists (T-Herb) 2. 2. 2 Diviners (T-Div) 2. 3 Behavioral modifications 2. 3. 1 Extra sleep (B-Sleep) 2. 3. 2 Sun avoidance (B-Sun) 2. 3. 3 No bathing (B-Bath) 2. 3. 4 Other (B-Other) 3. 0 Patient Characteristics 3. 1 Gender (P-Gend) 3. 1. 1 Male 3. 1. 2 Female 3. 2 Age (P-Age) 4. 0 Interruption of Daily Routine 4. 1 Work/school related (IDR-Work) 4. 2 Social engagements (IDR-Social)

Examples of Codebook Modifications Original Outline of Cultural Material Codes Shelley’s Adaptations 720 WAR 721 722 723 724 725 726 727 728 729 … 750 SICKNESS 751 Preventative Medicine 752 Bodily Injuries 753 Theory of Disease 754 Sorcery 755 Magical & Mental Therapy 756 Shamans & Psychotherapists 757 Medical Therapy 757. 1 Transplantation 757. 2 Hemodialysis 757. 3 CAPD (peritoneal dialysis) 757. 4 Home dialysis 757. 5 Adjustment to dialysis 757. 6 Compliance with medical regime 757. 7 Machinery involved in dialysis 757. 8 Medicines 757. 9 Medical test results 757. 91 HIV test results 758 Medical Care 759 Medical Personnel Instigation of Wartime Adjustments Strategy Logistics Tactics Warfare Aftermath of Combat Peacemaking War Veterans 750 SICKNESS 751 Preventative Medicine 752 Bodily Injuries 753 Theory of Disease 754 Sorcery 755 Magical & Mental Therapy 756 Shamans & Psychotherapists 757 Medical Therapy 758 Medical Care 759 Medical Personnel 760 DEATH 761 Life & Death 762 Suicide 763 Dying … Source: Shelley's (1992) Adaptation of the OCM Code 757 on Medical Therapy

Codebook Size • How Many Codes? – Codes are data reduction instruments – Dangers of Codebook Proliferation • Everyone thinks his or her codes are important and essential • Designing good codebooks and systematically coding data is extremely labor intensive. Every code added means a lot more work downstream • Coders can’t keep many codes in memory. The larger the codebook the less accurately coders will be in marking texts. – Recommendations • Build codebooks for only what you need and build and add codes incrementally • 30 -50 codes is about right (Boddan & Biklen 1982: 166; Miles & Huberman 1993: 58) • When coding data, work with no more codes than can fit on a single page • How many levels? – Early stages no more than 3 levels deep – In later stages, no more than 5 levels deep

Types of Text Analysis Exploratory Analysis Steps • 1: Select text to examine • 2: Identify themes • 3: Build codebook based on identified themes • 4: Tag text for themes • 5: Search for subthemes • 6: Search for patterns among themes – 6 a: Examine distribution of themes across types of people/cases – 6 b: Examine relationships among themes (potential hypotheses) (E. g. , Grounded Theory, Schema Analysis) Confirmatory Analysis Steps • • 1: Identify hypotheses to test 2: Select units of analysis 3: Select variables to code 4: Operationalize variables in form of a codebook • 5: Apply codes to each unit using either – 5 a: Human coders – 5 b: Computer-based dictionaries • 6: Used coded data to test hypotheses with appropriate statistics (E. g. , Classic Content Analysis)

Code Types • Tags – Indicate where in data themes occur for later retrieval and indexing – Tags can vary in size and can overlap – Most often associated with exploratory analysis • Value Codes – Assign values to fixed, non-overlapping units of analysis • E. g. , Textual units such as paragraphs, pages, documents, episodes, cases, people, pictures, objects, etc. – Codes can be nominal, ordinal, or ratio scale values – Most often associated with confirmatory analysis

Value Codes Qualitative nominal ordinal Quantitative Interval ratio kelvin no value 4 is not necessarily better than 1 or 1 vice versa. Value smaller peppers are less hot < or > Value higher number hotter temp < or > + - possible * / possible Ex. gender, race, electoral votes Ex. Low, medium & high flood areas Ex. Temp Ex. population

Tagging • How to tag – Index: Reference table linking themes (subject headings) with locations in the data (e. g. , index in book) • Typically use general data locations as reference points • Hard to find theme overlap – Point marker: A point embedded in the text that indicates the presence of a theme (e. g. , margin codes • More precise than indexes, but do not indicate where theme begins ends. • No way to identify theme overlap. – Block markers: Beginning and end markers embedded in the text indicating the presence of a theme • Best for precision • Can find theme overlap • What to tag – Big/Small Units – Grammatical Units (Paragraphs, Sentences) – Smallest possible continuous unit • How to start tagging – Ordered vs Random

Linkages Between Texts, Codes and Memos hyperlink Texts The last time I had a cold was back in November, I think. I was tired, crabby, had a sore throat, runny nose, and a bit of a cough. I remember going the Wal Mart to look for the new Cold-Eeze throat lozenges that my mother swears by [Memo #1]. They have zinc in them and are supposed to reduce the length of your cold. I couldn’t find them at Walt Mart because they are a pretty hot item. So I think I just suffered this way throughout the cold with no medication because I’m not a big believer in their benefits (unless, of course, my mother swears by it). I did have some peppermint tea that the midwife at work gave me. I work as an office assistant at a birth center). I tried to get more sleep than usual [Memo #2], but I didn’t take any time off of work or school. I remember trying not to kiss my boyfriend (that’s pretty tough, you know!) so that he wouldn’t get sick, too. My cold lasted probably five days. It was about the fourth time I had been sick that semester which is quite unusual for me [Memo #3]. I usually only get sick only once or twice a year. Codes S/S OTC Treat Signs & Symptoms (S/S) Treatments (Treat) Home Remedy (Home. R) Over-the-Counter (OTC) Western Professional (Prof) Complementary & Alternative Med. (CAM) Interruptions of Daily Routine (IDR) [Memo #2] Social Support (Soc. Sup) Duration (Dur) [Memo #4] … Memos Home. R 1. IDR Dur 2. 3. 4. Can’t tell if the mother suggested Cold. Eaze during this particular episode or if this is just a general recommendation. Is getting more sleep a treatment or an interruption in routine? Hypothesis: People who are sick a lot are less likely to take off from work or school. We might want to expanded “Dur” to include the frequency in which such illnesses occur.

Tagging (Block) ID Gen der Narratives SIGNS/SYMPTOMS ? ? 101 M The illness episode was preceded by a period of ‘running down my system’ which occurred from studying and working to a point where my sleep was being sacrificed. I recall feeling very tired and somewhat ‘achy’ at first, which was later accompanied by difficulty in breathing and swallowing. This was related to sinus congestion and a sore throat. I also recall that the skin around the openings of the nasal passages began to feel raw and irritated from the constant wiping. My throat felt sore as I mentioned earlier. This makes swallowing difficult and somewhat uncomfortable. There were also periods when I had phlegm which necessitated me to have to spit. The phlegm also had a greenish-yellow color to it, which confirmed the significance of this illness. I think I kept swallowing it and it made me queasy and eventually I puked. … 102 F It began with a slight headache behind the eyes and nose, almost stinging and then a dull pain. My head became heavy and I grew a little more irritable. For a while I became chilly, the hallmark that sickness was on the way. I called in the vitamins to combat the initial siege and the orange juice made me feel better (at least in my head. ) It wasn’t too bad, and since it was Christmas vacation and I had little else to do, I got out my Agatha Christi book …. Soar throat Nausea Vomiting Severity

Tagging (Block) ID Gen der Narratives SIGNS/SYMPTOMS 101 M The illness episode was preceded by a period of ‘running down my system’ which occurred from studying and working to a point where my sleep was being sacrificed. I recall feeling very tired and somewhat ‘achy’ at first, which was later accompanied by difficulty in breathing and swallowing. This was related to sinus congestion and a sore throat. I also recall that the skin around the openings of the nasal passages began to feel raw and irritated from the constant wiping. My throat felt sore as I mentioned earlier. This makes swallowing difficult and somewhat uncomfortable. There were also periods when I had phlegm which necessitated me to have to spit. The phlegm also had a greenish-yellow color to it, which confirmed the significance of this illness. I think I kept swallowing it and it made me queasy and eventually I puked. … 102 F It began with a slight headache behind the eyes and nose, almost stinging and then a dull pain. My head became heavy and I grew a little more irritable. For a while I became chilly, the hallmark that sickness was on the way. I called in the vitamins to combat the initial siege and the orange juice made me feel better (at least in my head. ) It wasn’t too bad, and since it was Christmas vacation and I had little else to do, I got out my Agatha Christi book …. Soar throat Nausea Vomiting Severity

Value Coding ID Gen der 101 M 102 F Narratives The illness episode was preceded by a period of ‘running down my system’ which occurred from studying and working to a point where my sleep was being sacrificed. I recall feeling very tired and somewhat ‘achy’ at first, which was later accompanied by difficulty in breathing and swallowing. This was related to sinus congestion and a sore throat. I also recall that the skin around the openings of the nasal passages began to feel raw and irritated from the constant wiping. My throat felt sore as I mentioned earlier. This makes swallowing difficult and somewhat uncomfortable. There were also periods when I had phlegm which necessitated me to have to spit. The phlegm also had a greenish-yellow color to it, which confirmed the significance of this illness. I think I kept swallowing it and it made me queasy and eventually I puked. … It began with a slight headache behind the eyes and nose, almost stinging and then a dull pain. My head became heavy and I grew a little more irritable. For a while I became chilly, the hallmark that sickness was on the way. I called in the vitamins to combat the initial siege and the orange juice made me feel better (at least in my head. ) It wasn’t too bad, and since it was Christmas vacation and I had little else to do, I got out my Agatha Christi book …. Soar throat Nausea Vomiting Severity ? ? ? ?

Value Coding ID Gen der 101 M 102 F Narratives The illness episode was preceded by a period of ‘running down my system’ which occurred from studying and working to a point where my sleep was being sacrificed. I recall feeling very tired and somewhat ‘achy’ at first, which was later accompanied by difficulty in breathing and swallowing. This was related to sinus congestion and a sore throat. I also recall that the skin around the openings of the nasal passages began to feel raw and irritated from the constant wiping. My throat felt sore as I mentioned earlier. This makes swallowing difficult and somewhat uncomfortable. There were also periods when I had phlegm which necessitated me to have to spit. The phlegm also had a greenish-yellow color to it, which confirmed the significance of this illness. I think I kept swallowing it and it made me queasy and eventually I puked. … It began with a slight headache behind the eyes and nose, almost stinging and then a dull pain. My head became heavy and I grew a little more irritable. For a while I became chilly, the hallmark that sickness was on the way. I called in the vitamins to combat the initial siege and the orange juice made me feel better (at least in my head. ) It wasn’t too bad, and since it was Christmas vacation and I had little else to do, I got out my Agatha Christi book …. Soar throat Nausea Vomiting Severity Yes 6 No No 3

The Basics of Qualitative Analysis • • • Identifying themes Building and applying codebooks Describing themes Make comparisons Building and testing models

Characteristics of a Good Description • Range • Central Tendency – Average, Modal, Prototypical exemplars • Distribution – Degree of variation around central point • Details – Examples • Context

Describing Themes, Cases, Groups, and Cultures • Themes – Correspond to a good codebook – general description, central tendency, range, core/periphery features • Cases – Single event or phenomena – Can be descriptive or used to understand bigger phenomenon • Groups – Describe variance among a group of cases – Similarities/differences, range, central tendency, and distribution • Cultures – Describe what people share – Both qualitative (e. g. , cultural models/ideals) and quantitative (e. g. , cultural consensus modeling)

Presentation • General and Particular – – – Universal statements Cross-cultural assertions General statements about society or cultural group Statements about a particular cultural scene Statements about a class of events, objects, or activities discovered in the cultural scene – Specific statements about incidents that take the reader to the actual level of behavior and objects • Core and Periphery Features – – Range Central Tendency Distribution Context • Style – – Direct quotes & paraphrases Exemplary cases Composites and vignettes Quantitative descriptions

A Thematic Description • • • The Survival Theme Bloom’s study of men with AIDS describes four of the most common themes. (1) SURVIVING (overcoming obstacles and enduring hardships) (2) RECIPROCITY (helping and caring for others in need) (3) APPRECIATION (of one’s daily life); and (4) what Bloom calls AVERAGE LIFE (leading an uneventful, simple, or “boring” everyday life. Fifteen of the 20 men expressed SURVIVING. Survival narratives were presented by men who were quite ill, men who were asymptomatic, and men at various stages of illness in between. “Successes in my life? Surviving as long as I have. I only came back here for a year. It was fun, I guess to make the business run. Certainly, the odds were against me. I hated coming back to straight, white America, but it was kind of fun for a couple years; making it work, whatever it took. I guess beating the odds on that was good, and when it worked real well it was real good. ” --Jerry (37 years old) Bloom (2001)

Michael Jackson’s Memorial

Media Bias scale of political preferences (Ho & Quinn 2007)

Are mainstream newspapers biased on memorial coverage? • • • Code 2 articles What are some of themes that we would code? Which concepts would have a negative slant? Which are positive? Could you count up the number of positive/negative mentions in one article?

July 8 th 2009, day after memorial The State Funeral Of a Superstar As a Media Moment Alessandra Stanley, New York Times (Ranked #1 most liberal) Excess rites all wrong for sicko freak! Andrea Peyser, New York Post (Ranked #24 most conservative)