Validity and Reliability Lesson objectives Explore the concept

Validity and Reliability

Lesson objectives • Explore the concept of validity and reliability in research • Analyse data collection methods • Selecting own disseveration topic

Validity

Internal validity • Refers to the validity of the measurement and test itself External validity • Refers to ability to generalise the findings to the target population

Why validity?

Ensuring validity • Ensuring that there adequate resources for the required research to be undertaken • Selecting an appropriate methodology for answering the research questions • Selecting appropriate instrumentation for gathering the type of data required

Measure of validity • Validity is a requirement of both quantitative and qualitative research. • In quantitative data validity might be improved through careful sampling and appropriate statistical treatment. • Qualitative data validity might be addressed through the honesty, depth, richness and scope of the data achieved, the participants approached, the extent of triangulation and objectivity of the researcher. But validity is a matter of degree and no research can be 100% valid. Cohen, L. , Manion, L. and Morrison, K. (2008) Research Methods in Education 6 th Ed. , London: Routledge (p. 133) Newby, P. (2010) Criteria for validity: Representativeness, Completeness and Transparency, Research Methods for Education, London: Pearson (p. 17)

Reliability

Validity= accuracy, authenticity Reliability = consistency, credibility Ensuring credibility • Clear Descriptions of the process of data collection and analysis • Why and how was the sample collected • How was data coded • How many researchers analysed the data

Authenticity • Findings that come from participants or research context, not researcher’s biases or perspective • Variation of findings • Triangulation

Triangulation • This is a way of assuring that your research is both valid and reliable. • Provides more than one perspective • Describes what you are researching from different perspectives. • Can be: the size of the sample you question, the number of researchers capturing the data, having more than one approach to capture the data, using more than one method to capture data i. e. Interviews and questionnaires. 15

Data Triangulation Using evidence from different types of data sources (primary and secondary research) or interviews, documents, public records and observations Primary data collection methods can be divided into two groups: What are they?

Quantitative and Qualitative. • QUANTITATIVE • Surveys/questionnaires (statistics, data, charts) • QUALITATIVE • Interviews • Focus group • Observation

Primary Data Collection Methods Surveys/ Questionnaires Interviews Observations Focus Groups

Secondary data collection method Is a type of data that has already been published in books, newspapers, magazines, journals, online portals etc. 19

Activity • In your groups, explore the advantages and disadvantages of using … (1) Surveys/Questionnaires (2) Interviews (3) Observations (4) Focus groups 20

Questionnaires / surveys • • ADVANTAGES Broad scope Answers the questions of what, how many, how much, how, and why Ability to control variables Respondents may be more honest in their responses because of the anonymous nature Low marginal costs (reusable) Easy to administer, score, and code Method to test theoretical propositions objectively • • • DISADVANTAGES Provides little depth and understanding of the context and history of a phenomena Research performed via surveys can be too confined because of structured nature of questionnaires Can only provide a snapshot of behaviour at one place and time Structure of questionnaires may cause people to poorly represent their opinion and respondents may not answer some questions Results may differ if using the same questionnaire in a different context or culture Typically low response rates

Interviews • • • ADVANTAGES More time efficient Allow the researcher to establish a rapport with the respondent Allow the acquisition of more indepth information Allow for variation according to response Allow the interviewer to obtain visual cues May be personal or telephone interviews • • DISADVANTAGES Time consuming (setting up, transcribing etc) Can be ‘leading’ and subjective in the transcription Can be costly They rely on the respondents being accurate and honest

Observations ADVANTAGES • Simple (does not require much technical equipment) • Useful for Framing Hypothesis • Common method used in all sciences • Independent of people’s willingness to report DISADVANTAGES • Lack of Reliability • Faulty Perception • Personal Bias of the Observer • Slow Investigation (could be time consuming process)

Focus Groups • • • ADVANTAGES They are useful to obtain detailed information about personal and group feelings, perceptions and opinions They can save time and money compared to individual interviews They can provide a broader range of information They offer the opportunity to seek clarification They provide useful material e. g. quotes for public relations publication and presentations • • • DISADVANTAGES There can be disagreements and irrelevant discussion which distract from the main focus They can be hard to control and manage They can to tricky to analyse They can be difficult to encourage a range of people to participate Some participants may find a focus group situation intimidating or offputting; participants may feel under pressure to agree with the dominant view