BTG 315 Research and Methodology Dr F N

BTG 315 Research and Methodology Dr. F. N. Nwalo 1

Basic steps of a research project • • Find a topic What, When Formulate questions What, Why Define population Who, When Select design & measurement How Gather evidence How Interpret evidence Why Tell about what you did and found out

Topic ideas • Online chat reference – Types of questions • Subject? Type? • # of turnaways* – Difference in discourse • In-person vs. chat – Partnership studies • Similar libraries with same software

Topic Ideas • E-book usage • Usability studies of – Online tutorial(s) – ‘My Library” portals • Analysis of library web sites or library instruction sites or pathfinders by best practices • Student learning outcomes in LI programs

Types of methodologies • Qua. Litative Measures – Descriptive – Numbers not the primary focus – Interpretive, ethnographic, naturalistic • Qua. Ntitative Measures – N for numbers – Statistical – Quantifiable

Qua. Litative measures • Content Analysis – Analyzed course syllabi of library use through discipline and level (Rambler) – Studied online tutorials, applying best practices recommendations (Tancheva)

Qua. Litative Measures • Discourse Analysis – Analyzed student responses in writing and discussions to a short film & compared findings to parallel study with LIS grad Ss (Vandergrift) • Focus Groups – Discussed how participants experience & use the library (Von Seggern & Young) – Studied why students use the Internet and how much time they use it (Wilson)

Qua. Litative Measures • Interviews – Studied 25 HS students’ web use for research assignments (Lorenzen) – Looked at what type of information first year students need and how they go about acquiring it (Seamans) • Observation (obtrusive) – Observed students as they conducted online research & noted their activities (Dunn) • Observation (Unobtrusive) – Retrieval of discarded cheat sheets to analyze academic misconduct (Pullen et. al. )

Qua. Litative Measures • Think Aloud Protocols – Studied how users navigate a library web site (Cockrell & Jayne) • Usability testing – Examined students’ mental models of online tutorials (Veldof & Beavers)

Qua. Ntitative measures • Compare. Things • Count Things • Survey People About Things p< . 05

Qua. Ntitative measures p< . 05 • Comparison studies – Experimental and control groups – Instructional methodologies (Colaric; Cudiner & Harmon) – Program assessment using before/after analysis of research papers(Emmons & Martin)

Qua. Ntitative measures p< . 05 • Pre & Post Tests (Van Scoyoc) • Measures & Scales • Bostick’s Library Anxiety Scale (Onwuegbuzie & Jiao; Van Scoyoc) • Procrastination Assessment Scale (Onwuegbuzie & Jiao)

Qua. Ntitative measures • Numeric Studies – Citation Analysis Bibliometrics (Dellavalle) – Webometrics (Bar-Ilian) p< . 05

Ready Made Data Sets • National Survey of Student Engagement (Whitmire) • College Student Experiences Questionnaire (Kuh and Gonyea) • The Web – Internet Archive (Ryan, Field & Olfman) – Electronic journals (Dellavalle) • Library server logs

Research Defined and Described “Research is the systematic approach to obtaining and confirming new and reliable knowledge” – Systematic and orderly (following a series of steps) – Purpose is new knowledge, which must be reliable This is a general definition which applies to all disciplines 16

Notice that: “… truth was not used in the definition of research” (p 16) “This concept of truth is outside of the productive realm of thinking by researchers” (p 16) 17

Research is not … cont. Data Collection • an intermediate step to gain reliable knowledge • collecting reliable data is part of the research process 18

Research is not … cont. Searching out published research results in libraries (or the internet) • This is an important early step of research • The research process always includes synthesis and analysis • But, just reviewing of literature is not research 19

Research is… 1. Searching for explanation of events, phenomena, relationships and causes – What, how and why things occur – Are there interactions? 2. A process – Planned and managed – to make the information generated credible – The process is creative – It is circular – always leads to more questions 20

• All well designed and conducted research has potential application. • Failure to see applications can be due to: – Users not trained or experienced in the specialized methods of economic research and reasoning – Researchers often do not provide adequate interpretations and guidance on applications of the research • Researchers are responsible to help users understand research implications (How? ) 21

Public good • Public research is a public good – May be more rigorous and objective because it is subject to more scrutiny • Private research may also be rigorous – But research on a company’s product may be questioned as biased. 22

Classification of Research Basic vs Applied Research • Basic – to determine or establish fundamental facts and relationships within a discipline or field of study. Develop theories … (examples in economics? ) • Applied – undertaken specifically for the purpose of obtaining information to help resolve a particular problem • The distinction between them is in the application – Basic has little application to real world policy and management but could be done to guide applied research 23

Disciplinary • designed to improve a discipline • dwells on theories, fundamental relationships and analytical procedures and techniques • In economics, the intended users are other economists • Provides the conceptual and analytical base for other economic research • It is synergistic and complementary with subject matter and problem-solving research 24

Disciplinary… cont. • Provides the foundations for applied research • Circular as applied research reveals the shortcomings of disciplinary research • Examples of some economic theories? (supply & demand, price elasticity, consumer utility …) 25

Subject-matter research • “research on a subject of interest to a set of decision makers “ (p 22) • Tends to follow subject-matter boundaries within a discipline ( eg. resource economics, production economics, labor economics) • Inherently multidisciplinary, drawing information from many disciplines – eg. consumer economic draws from psychology, natural resource economics from biology, economic policy from political science 26

Subject-matter research … cont. • Provides policy makers with general knowledge to make decisions about various problems. • A primary source of policy applications for economics • Subject-matter research is a cornerstone in economics – it involves direct application of economics to contemporary issues. 27

Problem-solving research • Designed to solve a specific problem for a specific decision maker • Often results in recommendations on decisions or actions • Problem-solving research is holistic – uses all information relevant to the specific problem (while disciplinary research tends to be reductionist) • Disciplinary research is generally the most “durable” (long lasting); problem-solving research the least durable 28

Analytic vs Descriptive Research • Descriptive Research – the attempt to determine, describe, or identify something – The intent is often synthesis, which pulls knowledge or information together • Analytic – the attempt to establish why something occurs or how it came to be • All disciplines generally engage in both 29

Methodology Defined & Described Methodology and Method are often (incorrectly) used interchangeable • Methodology – the study of the general approach to inquiry in a given field • Method – the specific techniques, tools or procedures applied to achieve a given objective – Research methods in economics include regression analysis, mathematical analysis, operations research, surveys, data gathering, etc. 30

• Contrast research methodology in economics (the approach to research) to economic methodology (the general approach to economic reasoning and economic concepts) • While these are different they are interdependent ( in the same way as science and research are related) 31

The Process of Research • The process is initiated with a question or problem (step 1) • Next, goals and objectives are formulated to deal with the question or problem (step 2) • Then the research design is developed to achieve the objectives (step 3) • Results are generated by conducting the research (step 4) • Interpretation and analysis of results follow (step 5) 32

Overview of Presentation • Overview of Quantitative/Qualitative Methodologies • Focus on Qualitative Research – Questions, Characteristics, Methods, Data Analysis, Credibility • Research Methodologies / Methods & GILD • Discussion

Overview of Research Methodologies • Qualitative Research – Ethnography, Case Study, Grounded Theory, Autobiography, Participatory Action Research, Phenomenology (each grounded in a specific discipline and philosophical assumptions) • Quantitative Research – Survey methods, Experiments • Mixed Methods – Draw from qualitative and quantitative methods

Quantitative • A quantitative approach is one in which the investigator primarily uses post-positivist claims for developing knowledge (i. e. cause and effect thinking, reduction to specific variables and hypotheses and questions, use of measurement and observation, and the test of theories). (Creswell, 2003, p. 19)

Qualitative - Definition • … qualitative researchers study things in their natural settings, attempting to make sense of or interpret phenomenon in terms of the meanings people bring to them. (Denzin & Lincoln, 2000, p. 3).

Qualitative - Definition § A qualitative approach is one in which the inquirer often makes knowledge claims based primarily on constructivist perspectives (i. e. the multiple meanings of individual experiences, meanings socially and historically constructed, with an intent of developing a theory or pattern) or advocacy/participatory perspectives (i. e. political, issue-oriented, collaborative or change oriented) or both. (Creswell, 2003, p. 18)

Research Questions • Qualitative – In qualitative study inquirers state research questions, not objectives (i. e. specific goals for the research) or hypotheses (i. e. predictions that involve variables and statistical tests). (C. , 2003, p. 105) – Example: How do students use program development tools?

Characteristics of Qualitative Research Takes place in the natural setting Uses multiple methods that are interpretive Is emergent rather than tightly prefigured Fundamentally interpretive (role of researcher as interpreter) Researcher views social phenomena holistically Researcher systematically reflects on who he or she is in the inquiry and is sensitive to hiw or her personal biography and how it shapes the study • Researcher uses complex reasoning that is multifaceted, iterative, and simultaneous • Researcher adopts and uses one or more strategies of inquiry • • •

Research Methods • • • Interviews Focusgroups Participant observation (field notes) Video Text and Image analysis (documents, media data)

Data Analysis (C. , p. 191) Organize and prepare the data for analysis Read all data, get a sense of the whole Begin detailed analysis with coding process Generate a description of the setting/people as well as categories or themes for analysis • Represent themes (writing, visual, etc. ) • Interpret and make meaning out of data • *iterative, non-linear process • •

Credibility • • Use of Triangulation Use of Member Checking Use of rich, thick Description Clarification of Bias Use of Negative or discrepant information Prolonged field time Peer Debriefing • (C. , 2003, p. 196)

Research Methodologies & GILD • Ethnography – An ethnography is a description and interpretation of a cultural or social group or system. The research examines the group’s observable and learned patterns of behaviour, customs, and ways of life. (C. , 1998, p. 58) – Rapid Ethnography

Research Methodologies & GILD • Case Study – … a case study is an exploration of a ‘bounded system’ … over time through detailed, in-depth data collection involving multiple sources of information rich in context. This bounded system is bounded by time and place, and it is the case being studied – a program, an event, an activity, or individuals. (C. , 1999, p. 61)

Useful Methods • Participant observation – Gains insight into understanding cultural patterns to determine what’s necessary and needed in tool development (complementary to interviews) • Interviews/Focusgroups with stakeholders – Explores how tools are used and could be used in a novice programming course – Gains insight into the meaning of tools for students for learning to program

Useful Methods • Data analysis – Themes arising from data would provide insight into current “learning to program” issues and see what is important to students / teachers / administrators • Survey – Useful for verifying results on a larger scale • User Testing – Useful for triangulating results

Choice of Methodology & Methods • Depends on – Research Questions – Research Goals – Researcher Beliefs and Values – Researcher Skills – Time and Funds

Discussion • (How) Can tool improvement, collaboration, ed-tech questions and learning outcomes be addressed in the same study? • What GILD research questions match which research methodologies?

References • Creswell, J. W. (1998). Qualitative inquiry and research design. Choosing among five traditions. Thousand Oaks, CA: Sage. • Creswell, J. W. (2003). Research design. Qualitative, quantitative and mixed methods approaches. Thousand Oaks, CA: Sage. • Denzin, N. K. & Lincoln, Y. (2000). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. Lincoln (Eds. ), Handbook of qualitative research (2 nd ed. , pp. 117). Thousand Oaks, CA: Sage.

Recognising Research: Approaches & Designs Introduction to Study Skills & Research Methods (HL 10040) Dr James Betts FACSM J. Betts@bath. ac. uk @Dr. BSteamjets

Lecture Outline: • The Research Process • The Research Design Continuum • Experimental Designs • Sampling Methods • Scientific Reasoning • Quantitative & Qualitative Research Strategies.

What is Research? • A systematic means of problem solving (Tuckman 1978) • 5 key characteristics:

What is Research? 1. 2. 3. 4. 5. Systematic – research process Logical – induction/deduction Empirical – evidence based Reductive – generalisation Replicable – methodology.

Review the Available Literature Publish Findings Research Process Interpret Findings Formulate a Question Select an Appropriate Research Design Collect Relevant Data

Research Continuum Basic Applied ØTheoretical? ØQuick Answers? ØMore Invasive? ØLess Invasive? ØLaboratory Based? ØField Based? ØTightly Controlled? ØLoosely Controlled? ØLacks External Validity? Internal Validity? > ØExternally Valid? ØFocus on Mechanism ØFocus on Effect ØMore Reductionist ØLess Reductionist.

Research Continuum Basic Applied e. g. ØDoes Caffeine Ingestion Improve Athletic Performance?

Research Continuum Basic Applied e. g. ØDoes Caffeine Ingestion Improve Ca 2+ binding with troponin? ØDoes Caffeine Ingestion Improve Athletic Performance? -Would this Facilitate Acto-Myosin Coupling? -Would this aid contraction?

Research Continuum Basic Applied e. g. ØDoes Caffeine Ingestion Inhibit Glycogen Phosphorylase? ØDoes Caffeine Ingestion Improve Athletic Performance? ØDoes Caffeine Ingestion Increase Lipid Metabolism? -Would this Spare Endogenous Glycogen?

Research Continuum Basic Applied e. g. ØDoes Caffeine Ingestion Stimulate the CNS? ØDoes Caffeine Ingestion Improve Athletic Performance? -Would this Increase Motor Unit Recruitment -Would this Reduce Perceived Effort?

Analytical Research • Reviews – A critical account of present understanding – A meta-analysis is a quantitative method of review • Historical Research – Accessing both primary (e. g. witnesses) or secondary (e. g. literature) sources to document past events • Philosophical Research – Organising existing evidence into a comprehensive theoretical model

Descriptive Research • Case Study – Accrual of detailed information from an individual • Survey Refutable? – Cross-sectional: Status of a various groups at a given point in time – Longitudinal: Status of a given group at various points in time – Correlational: Relationships between variables

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Independent Variable = this variable is the ‘cause’

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Independent Variable = can be manipulated or allowed to vary

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Independent Variable = also known as the predictor variable

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Dependent Variable = this variable is the ‘effect’

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Dependent Variable = should only vary in response to the IV

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Dependent Variable = also known as the criterion variable

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition ofofvariables: Law the single variable: there will always be uncontrollable influences

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Extraneous Variables = must be controlled to isolate the effect of the IV on the DV

Experimental Research • Experimental research involves a direct assessment of how one variable influences another • This allows the establishment of causality • All extraneous variables must be held constant while a single variable is manipulated and the effect measured • Definition of variables: Confounding Variables = extraneous variables which have co-varied with the IV

Sampling Methods • Random- All members of N have an equal chance of selection e. g. • Stage- Randomly select a group, then take sample School e. g. Class • Cluster- Select a natural group to sample from e. g. local community

Sampling Methods • Stratified- identify strata and sample accordingly i. e. Global Pop. Sample (n=100) = 51% = 51 = 49% = 49 • Systematic- e. g. every fourth person but starting at a random point • Opportunity- sample a convenient group Avoid Researchers!

Quantitative versus Qualitative Quantitative Research Strategy Qualitative Research Strategy • Investigation aims to assess a prestated theory (Deductive Reasoning) • Investigation aims to create a novel theory (Inductive Reasoning) • Often involves hypothesis testing • Researcher becomes an inherent part of the study - ethnography • Attempts to minimise the influence of the researcher on the outcome • Quantitative data infers statistics • Data collection therefore requires ‘closed’ responses • Qualitative data infers complex statements or opinions • Data collection therefore permits ‘open’ responses

Why do research? • Validate intuition • Improve methods • Demands of the Job • For publication

Choose a subject • Based on an idea • Based on your experience • Based on your reading • Originality

Choose a study design • • Case report Case series Case controlled study Cross sectional Cohort Retrospective comparison Prospective Comparison

A Case report • Description of one interesting and unusual case • This is anecdotal and may form the basis for further study • This may be the only way to report on something very rare

Case series • Description of several cases in which no attempt is made to answer specific hypotheses or compare results with another group of cases.

Cross sectional study • A survey of the frequency of disease, risk factors or other characteristics in a defined population at one particular point in time.

Cohort study • An observational study of a group of people with a specific characteristic or disease who are followed over a period of time to detect change • Comparison with control group is allowed

Case control study • An observational study where characteristics of people with a disease (cases) are compared with selected people without the disease (controls)

Controlled Trials • An experimental study in which an intervention is applied to one group and the outcome compared with that in a similar group (controls) not receiving the intervention

Adequacy of study • Study sample – must be representative – large enough size to ensure sufficient power • Quality control – Accurate measurements – Compliance of cases and controls

Define Your objectives • • Try to keep these simple The more variables the more difficult However use the opportunity Get help at this stage – Senior colleagues – Experienced researchers

Literature search • • • Check to see if your idea is original Look for a new slant to present Try to get the full article Read all the references Most of these will be vital when writing up