Data Collection Measurement Data Quality in Quantitative and
- Slides: 80
Data Collection, Measurement, & Data Quality in Quantitative and Qualitative Research 1
Data Collection Methods • Without appropriate data collection methods, the validity of research conclusions is easily challenged 2
Data Collection Methods • Using New Data – Collect own data for the study 3
Data Collection Methods • Using Existing Data – Historical research • Use records and other documents from the past – Secondary analysis • Use of data gathered in a previous study 4
Key Dimensions of Data Collection Methods • Structure – The data collection should be very structured and consistent • Quantifiability – Able to be analyzed statistically • Obtrusiveness – Degree to which people are aware that they are being studied • Objectivity – Try to be as objective as possible 5
Data Collection Quantitative Research 6
Types of Data Collection • Self-Reports • Observation • Biophysiologic Measures 7
Types of Data Collection • Self-Reports – – – Interviews Questionnaires Scales Vignettes Projective techniques Q-sorts 8
Types of Data Collection: Self. Reports Interviews and Questionnaires (Structured) • Participant's responses to questions by researcher • Data is usually collected by means of a formal, written document (instrument) • Uses an interview schedule for questions that are asked orally (face to face or via phone) • Uses a questionnaire when participants complete the instrument themselves 9
Types of Data Collection: Self. Reports Interviews and Questionnaires (Structured) – Closed-ended questions (fixed alternative questions) • • Response alternatives are specified by the researcher Ensures comparability of responses Facilitates analysis Easy to administer More efficient time use Difficult to develop Could lead to overlooking something important 10
Types of Data Collection: Self. Reports Interviews and Questionnaires (Structured) – Open-ended questions • Allows participants to respond to questions in their own words • Allows for richer, fuller information 11
Types of Data Collection: Self. Reports Interviews and Questionnaires (Structured) – Instrument Construction • Develop outline of content of research • Design questions • Pretest – Trial run to determine if instrument is free of biases, errors, etc 12
Types of Data Collection: Self. Reports • Interviews Vs. Questionnaires – Advantages of questionnaires • • Less costly Require less time and effort to administer Can be completely anonymous No biases relating to the researcher being present 13
Types of Data Collection: Self. Reports • Interviews Vs. Questionnaires – Advantages of Interviews • Response rate is higher in face to face interviews • Effective for those that can not complete questionnaires (children, blind, ESL, elderly) • Questions are less likely to be misinterpreted than questionnaires • Interviews can produce additional information through observation 14
Types of Data Collection: Self. Reports • Interviews Vs. Questionnaires – Interviews are considered to be superior to questionnaires 15
Types of Data Collection: Self. Reports Types of Self-Reports (Structured) • • Composite Scales (social - psychological) Vignettes Projective techniques Q sorts 16
Types of Data Collection: Self. Reports Composite Scales (social - psychological) – Scale: assigns a numeric score to people to place them on a continuum with respect to attributes being measured 17
Types of Data Collection: Self. Reports Composite Scales (social - psychological) – Likert scale – Semantic Differential scale – Visual Analog scale 18
Types of Data Collection: Self. Reports Composite Scales (social - psychological) – Likert scale (summated rating scales) • Consists of several declarative statements that express a viewpoint • Participant indicates the degree to which they agree to disagree • Able to summate the scores allowing for discrimination among people with different viewpoints 19
Types of Data Collection: Self. Reports Composite Scales (social - psychological) Example Likert Scale: AU nursing students are very well prepared for working within the current healthcare system Strongly agree Agree Neutral Disagree Strongly disagree 20
Types of Data Collection: Self. Reports Composite Scales (social - psychological) – Semantic Differential • Participants rate a concept on a series of bipolar adjectives • Can measure any concept – Visual Analog Scale • The scale is a straight line with anchors which are the extreme limits of the experience or feeling • Measures subjective experiences 21
Types of Data Collection: Self. Reports – Semantic Differential Example AU nursing graduates are: Competent Incompetent Intelligent Dim – Visual Analog Scale Example On a scale of 0 to 10 how would you rate your pain if 10 was the worst pain you have even experienced and 0 was no pain 22
Advantages of Scales – Scales allow researchers to efficiently quantify the strength and intensities of individual characteristics – Discriminates among people with different attitudes, fears, motives, perceptions, personality traits, needs – Good for group and individual comparisons – Can be implemented either verbally or in writing 23
Disadvantages of Scales Response set biases – Social Desirability Response Set Bias • Participants give answers that are common social views – Extreme Response Set Bias • Participants express attitudes or feelings in the extreme (always, never) – Acquiescence Response Set Bias • Participants agree with all statements (yea-sayers or nay-sayers) 24
Disadvantages of Scales – Ways to Reduce Response Set Biases • Counterbalancing: positively and negatively worded statements • Developing sensitively worded questions • Creating a permissive, nonjudgmental atmosphere • Guaranteeing confidentiality 25
Types of Data Collection: Self. Reports Vignettes – Brief description of events or situations to which participants are asked to react – Information about perceptions, opinions, or knowledge – Questions post vignettes may be open-ended or closeended – Economical to administer – May contain response biases 26
Types of Data Collection: Self. Reports Projective Techniques – Verbal self reports to obtain psychological measurements – Seek minimal participants’ conscious cooperation – Ambiguous or unstructured stimuli elicits participants needs, motives, attitudes, personality traits i. e. Inkblot test, word association, role playing, drawing – Useful in children, hearing or speech impaired 27
Types of Data Collection: Self. Reports Q Sorts – Uses a set of card with words, phrases or statements – Participant sorts cards along a bipolar dimension (agree/disagree) 28
Advantages of Self-Reporting Methods • Most common method of data collection used by nurses • Reveal information that is difficult to obtain by other means • Can gather retrospective and prospective data • Can measure psychological characteristics 29
Disadvantages of Self-Reporting Methods • Questionable validity and accuracy • Biases 30
Types of Data Collection: Observation • Observational Methods – An alternative to self-reports – Can be used to gather information such as characteristics, condition of individuals, verbal communication, nonverbal communication, activities, environmental conditions 31
Types of Data Collection: Observational Methods • Researcher has flexibility in the following areas: – The focus of observation • What events are to be observed – Concealment – Duration of observation – Method of recording observations 32
Types of Data Collection: Observational Methods (structured) – Categories and checklists – Rating Scales 33
Types of Data Collection: Observation Categories and Checklists • Category system: – attempts to designate information in a systematic, quantitative manner – Clear definition of behaviors and characteristics to be observed is necessary – Lists all behaviors or activities the observer wants to observe and records occurrences • Checklist: – instrument to record observations • Rating Scales: – Are tools that require the observer to rate some phenomena along 34 a descriptive continuum
Types of Data Collection: Observation – Observational Sampling • Time sampling – Selection of time periods for observations • Event sampling – Selects behaviors or events for observation 35
Evaluation of Observational Methods • Advantages – Provides depth and variety of information – Some problems are better suited to observation • Disadvantages – – Potential ethical issues Lack of consent to be observed Participants reaction to be observed Biases • Faulty inferences 36
Types of Data Collection Biophysiologic 37
Types of Data Collection: Biophysiologic Types of Biophysiologic Measures – In vivo • Measures performed directly within or on living organisms – i. e. blood pressure, temperature – In vitro • Data gathered from participants by extracting some biophysiologic material from them for lab analysis – i. e. blood work, microbiologic measures, cytology and histological measures 38
Advantages of Biophysiologic Measures – – Are relatively accurate and precise Are objective Provide valid measures of targeted variables Equipment is readily available 39
Disadvantages of Biophysiologic Measures – Measuring tool may affect variables it is attempting to measure – Interferences may create artifact – Energy must often be applied to the organism when taking measurements 40
• Measurement and Assessment of Data 41
Measurement • Involves rules for assigning numeric values to qualities • Determines how much of an attribute is present • Quantification – Communicates the amount in numbers 42
Advantages of Measurement – – Removes guesswork in gathering information Tends to be objective Obtains precise information Can differentiate among people who possess different degrees of an attribute – Common language 43
Errors of Measurement – Always the potential for error in all tools – Extraneous factors affect measurement and distort results • Obtained score – is observed score • True score – true score if no errors • Error of measurement – the different between the true and obtained scores 44
Factors Contributing to Errors of Measurement – Situational contaminants » People’s awareness of observer, environmental factors – Response set biases – Transitory personal factors » Fatigue, mood, hunger (temporary) – Administration variations » Alterations in data collection methods – Item sampling » Errors introduced as a result of sampling 45
Reliability of Measuring Instruments Reliability – Refers to the consistency with which an instrument measures the attribute – The less variation in repeat measures the higher its reliability 46
Reliability of Measuring Instruments Reliability – Aspects of reliability • Stability • Internal consistency • Equivalence 47
Reliability of Measuring Instruments • Stability • The extent to which the same scores are obtained when the instrument is used with the same people on separate occasions • To assess stability: Test-retest reliability – researcher administers the same measure to a sample of people on two occasions and then compares the scores 48
Reliability of Measuring Instruments • Internal Consistency • Reliable to the extent that all its subparts measure the same characteristic • To assess internal consistency: Split-half technique – the items comprising the test or scale are split into two groups and scored, compute reliability coefficient 49
Reliability of Measuring Instruments • Equivalence • Determines the consistency or equivalence of the instrument by different observers or raters • To assess equivalence – interrater (interobserver) reliability – Has two or more trained observers make simultaneous, independent observations, compete reliability coefficient 50
Reliability of Measuring Instruments • Reliability Coefficients – A quantitative statistic that estimates how reliable an instrument is • Determine an instrument’s quality • Low reliability makes it difficult to adequately test research hypothesis • If sample too homogeneous, the lower reliability coefficient will be (instruments are designed to measure differences) 51
Validity of Measuring Instruments Validity – Is the concern whether the measurement tools actually measure what they are supposed to measure 52
Validity of Measuring Instruments Aspects of Validity • • Face validity Content validity Criterion-related validity Construct validity 53
Validity of Measuring Instruments – Face validity • Whether the instrument looks as though it is measuring the appropriate construct 54
Validity of Measuring Instruments – Content Validity • Concerned with adequacy of coverage of the content area being measured – Tests of knowledge – Psychosocial traits – Based on judgment 55
Validity of Measuring Instruments – Criterion-Related Validity • Wants to establish the relationship between scores on an instrument and some external criterion – Compute a validity coefficient – correlates scores on the instrument with scores in the criterion variable 56
Validity of Measuring Instruments – Construct Validity • Concerned with what construct is the instrument actually measuring 57
Validity of Measuring Instruments • To assess construct validity– – known-groups technique » Groups that are expected to differ on certain attributes are administered the instrument then scores are compared – Factor analysis » Statistical procedure – Examination of relationships based on theoretical predictions 58
Reliability of Measuring Instruments • If a measuring device is not reliable, it can not be valid – High reliability of an instrument provides no evidence of its validity – Low reliability is evidence of low validity • An instrument can be reliable without being valid Reliability consistently measures accurately Validity measures what it is supposed to 59
Data Collection Qualitative Research 60
Questions for Thought • What are the systematic rules for analyzing qualitative data? 61
Qualitative Data Collection Types of Self-Reports - Unstructured Self-Reports Methods (Unstructured) • Interviews • Diaries • Observation 62
Qualitative Data Collection Types of Self-Reports - Unstructured Interviews • • • Flexible Not directed by set questions Interviews are conversational in nature Usually interviews are long Can be tape recorded or researcher may take notes 63
Qualitative Data Collection Types of Self-Reports - Unstructured • Completely unstructured interviews • Start with broad (grand tour) questions • Further questions are guided by initial responses – one question's answer leads to the next question • Focused or semi-structured interviews • Researcher lists topics that must be covered in an interview • Uses a topic guide to ensure all question areas are covered 64
Qualitative Data Collection Types of Self-Reports - Unstructured • Focus group interviews • Interviews with groups of 5 to 15 people whose opinions and experiences are solicited simultaneously • Uses topic guide to guide questions 65
Qualitative Data Collection Types of Self-Reports - Unstructured • Life Histories • Narrative self-disclosures about life experiences • Has informants describe experiences in chronological order • Orally or written 66
Qualitative Data Collection Types of Self-Reports - Unstructured Diaries • Have informants maintain daily logs of some aspect of their lives 67
Qualitative Data Collection: Observational Methods - Unstructured Observational Methods • Unstructured observation – Attempt to see the world as the participants see it – Participant observation – data collector actually participates in the group » Participation can be from the role as an observer or totally immersed in the social setting as a participant » Researcher needs to gain entrée into the social group under investigation » Researcher needs to establish rapport and develop trust within the group 68
Qualitative Data Collection: Observational Methods - Unstructured • Observational Data Collection – Physical setting » In what context is the human behaviour occurring – Participants » Information about the participants, what are their roles, characteristics – Activities » What are the participants doing – Frequency and duration » Specific information about the activity 69
Qualitative Data Collection: Observational Methods - Unstructured – Process » How is event occurring – Outcomes » Why is the activity occurring and what are the results – Single positioning » Staying in a single location – Multiple positioning » Involves moving around to observe behaviour from different perspectives – Mobile positioning » Involves following a person throughout a given activity 70
Qualitative Data Collection: Observational Methods - Unstructured • Observational Data Recording • Uses logs and field notes » Log – records daily events » Field notes – observer’s efforts to record information and understand data » Observational notes – descriptions of events and conversations » Theoretical notes – interpretive attempts to attach meaning to observations » Methodologic notes – instructions about what observations that need to be made » Personal notes – comments about researcher’s own feelings 71
Assessment of Qualitative Data – Do the measures used by the researcher yield data reflecting the truth – Qualitative research attempts to do this through establishing the data’s trustworthiness 72
Assessment of Qualitative Data Establish Trustworthiness by assessing: – – 1. Credibility 2. Dependability 3. Confirmability 4. Transferability 73
Assessment of Qualitative Data: Trustworthiness 1. Credibility – Confidence in the truth of the data • Prolonged engagement and persistent observation – Sufficient time to collect data, focus on the phenomena being studied 74
Assessment of Qualitative Data: Trustworthiness Triangulation – Use of multiple referents to draw conclusions, attempts to distinguish true information from errors – Data Source Triangulation » Multiple data sources (interviewing diverse informants on same topic) – Investigator Triangulation » Using more than one person to collect data – Theory Triangulation » Using multiple perspectives to interpret data – Method Triangulation » Using multiple methods (observation and interviews) 75
Assessment of Qualitative Data: Trustworthiness • External checks: Peer debriefing and member checks – Peer debriefing – review and explore various aspects of inquiry with objective peers – Member checks – providing feedback to study participants and assessing their reactions • Searching for Disconfirming evidence – Search for data that challenges the emerging conceptualization or theory • Researcher credibility 76
Assessment of Qualitative Data: Trustworthiness 2. Dependability – Data stability over time and over conditions – Stepwise replication • Having several researchers break into teams and evaluate the data separately and then compare conclusions – Inquiry audit • Scrutiny of the data and supporting documents by an external reviewer 77
Assessment of Qualitative Data: Trustworthiness 3. Confirmability – The objectivity or neutrality of the data, can other independent people agree about data’s relevance – Audit trail – documentation that allows an independent auditor to come to the same conclusions about the data 78
Assessment of Qualitative Data: Trustworthiness 4. Transferability – The extent to which the findings from the data can be transferred to other settings or groups 79
Reference • Loiselle, C. G. , Profetto-Mc. Grath, J. , Polit, D. F. , & Beck, C. T. (2011). Canadian essentials of nursing research. (Third Edition). Philadelphia: Lippincott, Williams & Wilkins. 80
- Data analysis qualitative and quantitative
- Examples of qualitative data
- Landsat collection 1 vs collection 2
- Documentary collection
- Data collection procedure
- Qualitative data analysis coding
- Oceanographic quality measurement
- Quality assurance vs quality control
- Basic concept of quality control and quality assurance pdf
- The terms external secondary data and syndicated
- Data quality and data cleaning an overview
- Data quality and data cleaning an overview
- Data quality and data cleaning an overview
- Project quality management pmp
- What are quality standards in project management
- Quality assurance model in nursing management
- Quality improvement vs quality assurance
- Quality gurus
- Quality is free: the art of making quality certain
- What is tqm
- Instruments in quantitative research
- Qualitative and quantitative data difference
- Interpreting quantitative data
- Chapter 3 exploring quantitative data answers
- Quantitative research design example
- Content analysis is a type of secondary data analysis
- Ib chemistry measurement and data processing worksheets
- Collection organization and presentation of data
- It is a collection of interrelated data
- Collection and analysis of rate data
- Data collection and input in gis
- Data collection management and analysis
- Data raw facts and figures
- What is primary data
- Analysis and findings example
- Qualitative vs quantitative data analysis
- Qualitative data biology definition
- Displaying quantitative data with graphs
- Summarizing quantitative data
- Data types in quantitative research
- Basic concept of quantitative data analysis
- Bandura theory
- What is quantitative data
- Summarizing quantitative data
- Tabular and graphical presentation of data
- Displaying quantitative data with graphs
- Sources of quantitative data
- Describing quantitative data
- It is the part where you indicate the research instrument
- Summarizing quantitative data
- Summarizing quantitative data
- Bivariate measurement data
- Tollgate review example
- Sample of priority improvement areas
- Finding answer through data collection
- Sampoorna students details
- Data collection proposal
- What is research instrument
- Define data collection method
- Phase 2 data
- Encyclopedia
- Pantry audits in research
- Self report method
- Www iria ine es
- Network data collection at slac
- Cawi data collection
- Cra hmda
- Data collection plan
- What is an interview schedule in research
- Is a structured collection of data
- Nominal attribute in data mining
- Discretization and binarization in data mining
- Informal data collection
- A major advantage of primary data collection is that
- Data collection instruments in qualitative research
- A named collection of data on a storage medium
- Interview method of data collection
- A disadvantage of the mall intercept interview method is
- Observational data collection methods
- Statistical treatment of data example
- Collection adt