Qualitative Data collection analysis presentation Dr Anne Adams

Qualitative Data (collection, analysis & presentation) Dr. Anne Adams

Overview • Qualitative Data – how to collect it • Qualitative Analysis – how to do it • How to present qualitative data • Tools to support you

Qualitative / Quantitative Approach Research is like fishing Quantitative methods • You find the best river for the fish you want, you have one line, a specific bait for a specific type of fish. Qualitative methods • You may want to catch tuna so you fish in certain parts of the sea BUT on the whole you throw your nets out to sea and catch everything including the things you want and don’t want.

Qualititative methods • In-depth Interviews, • Focus groups, • Observational / ethnographic studies, • Open ended data (system logs) • Data transcribed

Documentation § Course materials, § Standards, § Previous literature (conference / journal, web pages) § Internal reports § External reports (e. g. HEFCE)

Research Diaries (Research Logs / Field Notes) 1) Research conducted & decisions made 2) Thought & creative research process § Faraday’s ‘Field Notes’: "The main interest of the Diary lies quite outside the range of propositions and experimental proofs. It centres round the methods of Faraday's attack, both in thought and in experiment: it depends on the records of the workings of his mind as he mastered each research in turn, and on his attitude not only to his own researches but also to scientific advance in general" (Bragg 1932: v).

Critical Incidents § Used in conjunction with other methods § Used as a data collection & analysis approach § Identify critical moment / factor 1. Ask respondent to recall 2. Observe, record interactions identify incidents 3. Review documentation, logs identify incidents

Observational Diaries 1) Interaction patterns 2) Attitudinal changes (e. g. becoming scared of the computer) 3) Behavioural changes 4) Sequencing (e. g. incremental changes experienced over time)

Critical Incidents (observations) Timeframe Technology interactions Social interactions barriers enablers All verbal nothing printed or electronic provided Social details about the consultant provided Clinician 1 10. 01 Nurse 1 10. 02 Receptionist 1 09. 30 Recorded patient arrival Discussion about information required for appointment Patient 1 9. 15 None Discussion with All verbal no receptionist further information provided Key points about consultant provided

A&E White-boards – Collaboration & unconscious social cues – Interaction roles (e. g. pen holders) & acceptable sharing Penholder Other Staff Total of Whiteboard overall Interactions observations Long looks Glances Entering and Annotating 28% 26% 48% 51% 1% 38% 39% 23% Broome & Adams (’ 05) BCS HC, (’ 06) IMM

Hospital spaces & technology use Neurologist patient interaction pattern: consultant // patient Piece of paper from the diary Paper-based diary Computer Desk Computer Window Nurse A Desk Patient Consultant Patient Window Nurse B Nurse A Desks Wi nd ow Computer Examination bed Patient Consultant Window Patient interactions Examination bed Clinician interactions Adams & Blandford, HCI’ 08 EPSRC / ESRC / NHS funded

Interviews § Structure (semi-structured/ structured) § Style (expert / novice) § Setting (natural, office) § Recording the data (audio – quotes, written notes distract, phone interviews). § Biasing – talking (<15%), your opinion (NO) SEE: http: //oro. open. ac. uk/11909/

Focus Groups § 6 or 7 participants § Moderate to keep focus & obtain all perspectives

Questionnaires (open ended) § Design (quantitative / qualitative) § Purpose (obtain background info & recruit) § Open not closed to allow ‘participants’ accounts of experiences, feelings, observations § E. G. : § Give examples of using mobile devices for learning purposes § Detail how you felt about those experiences § Provide information about where you have seen others using mobile devices for learning.

User Trials – Test current applications with a sample of students. – Experiments where you test one type of course presentation with another. – Video students completing a sample assessment / piece of coursework to see where they actually get stuck rather than where they think they get stuck. – Video and observe interactions between students and / or students and tutors.

Log Analysis § First Class interactions § Eluminate / flash-meeting logs § Analyse (quantitatively and qualitatively) § Interaction and usage patterns § Tracking individuals § Language used

Participant Video § Gaining in popularity as a research method § The OU Participatory Video Group § Give participants video’s they capture data on what is important with regard to your focus of research. § Can be as focused or as open-ended as you want.

Qualitative Analysis Background • Conversational / Discourse Analysis • Thematic Analysis / Grounded Theory • Content Analysis / Critical Incident Analysis • counting • imposing established frameworks • “Both qualitative and quantitative approaches share a common concern with theory as the goal of research” (Henwood & Pidgeon, 1992 p. 101)

Qualitative Analysis Methods • Conversational / Discourse Analysis • Thematic Analysis / Grounded Theory • Content Analysis / Critical Incident Analysis • counting • imposing established frameworks • “Both qualitative and quantitative approaches share a common concern with theory as the goal of research” (Henwood & Pidgeon, 1992 p. 101)

QUANT / QUAL Comparison Quantitative approaches Qualitative approaches 'Simple' numeric data 'Complex' rich data Measurement Meaning Explanation Understanding Prediction Interpretation Generalisable account Contextual account Representative population sample Purposive/ representative perspective sample Hypothesis-testing Exploratory Claims objectivity Accepts subjectivity Closed system (experimental control) Open system (ecological validity)

Qualitative Analysis Methods • Conversational / Discourse Analysis • Thematic Analysis / Grounded Theory • Content Analysis / Critical Incident Analysis • counting • imposing established frameworks “Both qualitative and quantitative approaches share a common concern with theory as the goal of research” (Henwood & Pidgeon, 1992 p. 101)

TELLING STORIES through DATA http: //www. youtube. com/watch? v=jbk. SRLYSojo IBM Data Visualisations at ‘many eyes’: http: //www-958. ibm. com/software/data/cognos/manyeyes/ Details of GT approach used here found at: http: //oro. open. ac. uk/11911/

GT Application • • Data in whatever form is : Broken down, conceptualised, and put back together in new ways. • Analysis Stages – 3 levels of coding : § § § open, axial, selective (with process effects)

Open coding 1. Concepts are identified. 2. Concepts are grouped into categories 3. Properties and dimensions of the category identified

Open coding: detailed • Concepts are: - Conceptual labels placed on discrete happenings, events, and other instances of phenomena • Categories are: - where concepts are classified and grouped together under a higher order – a more abstract concept called a category. • Properties are: - characteristics pertaining to a category • Dimensions are: - Location (values) of properties along a continuum

Open coding: example • “ When I want to have a personal conversation, I encrypt the message. I think that makes the email private. Stops people from listening in”

Open coding: analysis • “ When I want to have a personal conversation (private interaction), I encrypt the message (security measure). I think that makes the email private (Securing privacy). Stops people from listening in (Surveillance). ” • Concepts are: - private interaction, security measures, securing privacy, surveillance Categories are: - Interaction, privacy, security •

Open coding (5) Category Class Property Dimensional Range surveillance frequency Being observed scope often. . . . never more. . . . less intensity high. . low duration long. . short

Axial Coding (1) • High level phenomena identified. • Phenomena conditions identified (causal, context, intervening). • Phenomena action / interaction strategies and consequences identified

Axial Coding (2) • Phenomena are: - central ideas, events. • CONDITIONS • Causal conditions are: - events that lead to occurrence or development of a phenomenon. • Context: - The specific set of properties (and location on a dimensional range) that pertain to a phenomenon. • Intervening conditions: - broader structural context.

Axial Coding (3) • “ When I want to have (context) a personal conversation (phenomenon), I encrypt the message (strategy). I think that makes the email private (consequence). ”

Selective Coding (1) • Define the core category & high-level story line. • Relate subsidiary categories by its properties • Relate categories at the dimensional level • Iterative validation of relationships with data • Identify category gaps 32

Selective Coding (2) • Core category is: The central phenomenon around which all the other categories are integrated. • Story is: A descriptive narrative about the central phenomenon of the study. • Story line is: The conceptualisation of the story the core category. • Ways to represent this ‘STORY LINE’ (Conditional Matrix, Process Effects)

Process Effects • Process is the linking of AI sequences over time Action / Interaction Strategy CHANGE TO CONDITIONS affecting A/I CHANGING CONDITIONS CONSEQUENCES of response (over time) RESPONSE from A/I

Authentication A B Perceived POOR Security L Information O importance W Password disclosure Adams & Sasse (’ 99) ACM Multimedia 35 HIGH L O W Security Perceived threats Increased

Privacy and Process IS Contexts Users Trust Privacy secure (based on assumptions) IR IU Technology makes assumptions inaccurate Emotive Increased perceived privacy invasions Decreased organisational trust 36 Rejection

Initial Problems • Lines between each type of coding are artificial – Data presented at dimensional level – Action / interactions & conditions present. • “ I find computers always break down for me when I have a lot of things to do. So I try not to use them when I have a lot to do. Which slows everything down a bit” 37

Solution • Code both open and axially together • Qualitative analysis tools –NVivo –Atlas TI • Analyse without loosing the detail 38

Problems & Solutions • P: Complex method to apply • S: Ease up on yourself, take the best approach for you …. Paper-based coding with colour pens used by social-scientists (immersing yourself in the data) • P: Focus of research • S: Data collection and analysis tightly interwoven 39

Qualitative Analysis SUMMARY • Powerful for appropriate issues • Application Complex • Rewarding – ‘Convincing Theories’ 40

Good Quality Research • Not Divide but to compliment • Exploratory (discovery) – reductionistic (justification) • Henwood / Pidgeon – good quality research • 7 golden rules of good quality research

TOOLS • Atlas Ti, NVivo

Atlas TI – multiple media Can be used for: • Textual data – transcripts in ASCII or ANSI character code table • Graphical data – BMP, TIFF, Kodak Photo CD • Audio Data – WAV

Hermeneutic unit editor • Primary document Pane – document – line number – margin area • HU editor components – Main menu – main tool bar – drop-down list (prim. doc, quotes, codes, memo) – primary doc tool bar

NVivo

CONCLUSIONS