Topics Review Action Research Research designs Data collection

Topics • Review – Action Research – Research designs – Data collection and analysis • Quality of research • Research report • Final project

Topics • Schoolwide action research – Case study videos – Guest speaker • Quiz • Articles relating to common core

Role and Purpose of Action Research • To identify and solve specific classroom or school problems • To improve educational practices • To help to make decisions at a local site • To empower teachers and other educational practitioners • To promote professional growth

What is Action Research? • Systematic inquiry conducted by educators into their own practices with the purpose to gather information about how their school operates, how they teach, and how their students learn • Research done by teachers for themselves • Systematic inquiry into one’s own practices • Research then has increased utility, effectiveness

Review of Terms • Research: a systematic approach to finding answers to questions. • Research Design/Plan: a plan for gathering data for answering specific research questions. • Action Research: a systematic approach to finding answers to questions done by practitioners for themselves to improve educational practices.

Characteristics of Action Research • Practice-oriented – Conducted by practitioners done for themselves • Situation/context specific – Focused on solving problems at a local setting • Reflective – Researcher as participant • Systematic • Cyclical/on-going

Process of Action Research • Identify and define research problem and questions • Design research study to collect data bearing on questions • Conduct the research • Analyze the data • Interpret the data in light of the research questions

Action Research Model… • Stringer’s Interacting Spiral

Action Research Model… • Lewin’s Action Research Spiral Identifying a General or Initial Idea Take First Action Step Reconnaissance or Fact Finding Planning Evaluate Amended Plan Take Second Action Step…

Action Research Model… • Calhoun’s Action Research Cycle 1 Select Area 5 2 Take Action Collect Data 4 Analyze and Interpret Data 3 Organize Data

Action Research Model… • Bachman’s Action Research Spiral Plan Revised Plan

Action Research Model… • Riel’s Action Research Model

Action Research Model… • Piggot. Irvine’s Action Research Model

Action Research Model… • Hendrick’s Action Research Process

Action Research Model • Mertler’s AR four stages:

With a Partner… 1. Analyze the models of Action Research and identify a context (a building situation or issue) you think would be appropriate for each. 1. 2. For example, Stringer’s model could apply to a principal’s ongoing observation (Look) about safety issues when children are dropped off for school. Thinking about the issue and exploring possibilities, several new procedures are created, tried out with models, shared as a possibility with stakeholders, and reviewed with appropriate officials (Think), and finally the new procedure is put into place (Act). Create a personal graphic of an Action Research Model you would implement in your building. Share with group.

Major Research Methods • Quantitative research methods – Require numerical data – Utilize deductive reasoning (‘top-down’ approach) • Qualitative research methods – Require narrative data – Utilize inductive reasoning (‘bottom-up’ approach) • Mixed methods – Studies that combine both quantitative and qualitative data – Many individuals consider action research studies to be most similar to mixed-methods research (than purely quantitative or qualitative research)

Quantiative Research Design • Descriptive designs • purpose is to describe and make interpretations of current status of individuals, objects, conditions, or events. • Survey research • Correlational designs • purpose is to measure and describe statistical relationship between two or more variables • Typically use correlation coefficient • Group comparison designs • attempt to investigate cause-and-effect relationships by comparing two or more groups that differ on some characteristic • Experimental and non-experimental designs • Single-subject designs

Quantitative research designs Group comparison designs • Pre-experimental (no control group or randomization) – One-shot case study – One-group pretest-posttest design • True experimental (control group and randomization) – Posttest-only control-group design – Pretest-posttest control-group design • Quasi-experimental (control group but no randomization) – Non-equivalent control group design – Time-series designs • Causal-comparative designs – Ex post facto = ‘after the fact’ – Presumed cause has already occurred (prior to study)

Qualitative Research Designs • Case study – particular individual, event, or program is studied • Ethnography – in-depth study of a group • Phenomenological study – studies of individual perceptions of a particular situation • Grounded theory – Qualitative studies that attempt to discover a theory

Quantitative Data Collection Techniques • Quantitative data are numerical • Variety of techniques: – Surveys, questionnaires, rating scales—verbal or written administration of set of questions or statements to sample of people • Closed-response • Likert and Likert-type scales • Checklists • Follow guidelines and suggestions for developing instruments – Tests and other formal instruments

Quality of Quantitative Data • Validity—extent to which you actually measured what you intended to measure – Must be appropriate and accurate for your purposes – Seen as a unitary concept, combining content, concurrent, predictive, and construct validity (focus should be on evidence) • Reliability—refers to consistency of collected data – Determined by correlating results with themselves or with another quantitative measure – Three methods: • Test-retest reliability • Equivalent forms reliability • Internal consistency reliability • ‘A valid test is always reliable, but a reliable test is not always valid. ’

Rigor of Quantitative Study • Can you trust the conclusions of the study? • Internal Validity: The extent to which the outcomes of the study result from the variables manipulated, measured or selected rather than from other variables not systematically managed. • External Validity: the extent to which the findings of a particular study can be generalized to people or situations other than those observed in the study.

Best Practices in improving rigor of quantitative studies • Control Group: a group of subjects whose selection and treatment are exactly the same as those of the experimental group except that the control group does not receive the experimental treatment. • Random Assignment: a method for assigning subjects to control and experimental groups. Not to be confused with random selection (a method for selecting a sample of subjects from a population). • Pretests: When random assignment is impossible or undesirable, pretests can be used to examine prior existing differences between groups and to statistically adjust for these differences. • Instrumentation: select valid and reliable instruments to minimize measurement errors. • Sample size: Large sample size from representative sample is desired. • Effect size: report effect size. • Replication: perhaps the best way to ensure internal and external validity

Qualitative Data Collection Techniques • Qualitative data are narrative • Variety of techniques: – Observations—carefully watching and systematically recording what you see and hear • Structured/semi-structured/unstructured observations • Recorded using field notes, videotapes • Should include observer’s comments in the field notes – Interviews—directly asking people questions (as opposed to watching them); conversations between researcher and participants • Prepare an interview guide (may be specific or general) • Several types: – Structured/semi-structured/unstructured interviews – Individual, phone, Focus group

Qualitative Data Collection Techniques – Journals—means of gathering data to provide insight into workings of a classroom • Types of data journals: – Student journals – Teacher journals – Class journals – Existing documents and records— schools are filled with existing sources of data… – Classroom artifacts—as are classrooms!

Quality of Qualitative Data • Validity of research data—extent to which data collected accurately measure what they purport to measure • Validity of qualitative data…concern lies with trustworthiness of data; • Trustworthiness of data is enhanced through: – Triangulation—use of multiple data sources, multiple datacollection methods, and multiple teacher-researchers (if possible) – Member checking—sharing data and analyses with participants to check for accuracy – Prolonged engagement & persistent observation—more time spent ‘in the field, ’ more you get to know participants, culture, behaviors, etc.

Rigor of Qualitative Study • Can we trust the conclusions of the study? • Four trustworthiness criteria (Guba & Lincoln, 1989) • Credibility: related to the “true” picture of the phenomenon. • Transferability: related to whether the findings can be transferred to other situations. (think description of setting/sample) • Dependability: relates to the consistency between the data and the findings. (audit trail) • Confirmability: involves the strategies used to limit bias in the research, specifically the neutrality of the data not the researcher (peer debriefing)

Best Practices in Improving Rigor of Qualitative Studies • Determination of rigor often depends on intended audience for sharing results – Broader dissemination—should be more concerned with generalizability – Narrower dessimination—may be no generalizable results • Methods of providing rigor in action research – – – – Triangulation of data Member checking Peer debriefing Prolonged engagement and persistent observation Thick description of the setting and study Make available an audit trail (a recode of data analyzed in the study) Repetition of the cycle

Research Studies Pair-Share • Review slides on research designs – Which design have you used with your study? – Rigor of designs • Review slides on data collection strategies – What data collections have you used? – Quality of data • Be the critical friend

Final project presentation

Analyze Quantitative Data • Descriptive statistics – Descriptive study • Describe student make-up at your school • Describe student performance at your school – Group comparison • Compare performance on tests by different student subgroups (gender, ethnicity, FRL, grade level, etc. ) – Relationship / correlational • Describe relationship between Plan and ACT • Describe relationship between performance on math and reading

Review • Analyze quantitative data – Descriptive – Group comparison (disaggregation and trend analysis – Correlational • Analyze qualitative data

Group Activity • You are assisting the district supt. to analyze the ISAT data for one building. • Use the data to create a report on – Student demographics (who are we? ) • Interested in Gender, low income by grade level • Please note strengths and challenges in student make-up – Student performance (how did we do? ) • Interested in ISAT reading and math by grade level, gender, low income • Please note strengths and challenges in student learning • Is there any relationship between performance on reading and writing? If there is, describe the relationship. – Develop an action plan based on your analyses

How to analyze Descriptive study Nominal/categorical data – Frequency table – Frequency chart (e. g. bar chart) Interval/ratio data – – Frequency table Frequency chart (e. g. histogram) Measures of central tendency (mean, median, mode) Measures of dispersion (range, standard deviation)

How to analyze Group comparison study - different groups Disaggregation analysis – Break down data by subgroups – Compute averages and standard deviations for each group as well as for the total group To gauge the magnitude of the difference – Compute average performance difference between groups – Divide average performance difference by the standard deviation for the total group – Use Cohen’s (1988, 1992) criterion: |. 2| = small, |. 5| = medium, |. 8| = large Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (second ed. ). Lawrence Erlbaum Associates. Cohen, J (1992). "A power primer". Psychological Bulletin 112: 155– 159. doi: 10. 1037/0033 -2909. 112. 1. 155

How to analyze Group comparison study – same group over time Pre-Post analysis (Trend analysis) – Compute learning gain for each individual – Compute the average gain and the standard deviations of the gains for the group To gauge the magnitude of the gain – Divide the average gain by the standard deviation of the gain for the group – Use Cohen’s (1988, 1992) criterion: |. 2| = small, |. 5| = medium, |. 8| = large

How to analyze Correlational study Correlational analysis – Derive a Scatterplot for two variables – Compute the correlation coefficient To gauge the magnitude of correlation – Use Cohen’s (1988, 1992) criterion: |. 1| - |. 3| small, |. 3| - |. 5| medium, over |. 5| large

How to Analyze Qualitative Data • Transcribing the interviews • Reading the transcripts to tentatively identify categories of responses • Testing the tentative categories by classifying responses in the first hour of the interviews • Using final categories to code all responses • Tallying coded response • Report and display the coded responses • Peer debrief • Member check 40

Group Activity • Analyze qualitative data

Research Ethics • Resect the rights of and protect human subjects involved in a research study – Permissions from students, parents, teachers, others – May require formal permission – Informed consent form (See sample) • Ways to maintain confidentiality and anonymity of study participants – Report aggregate data – Use fictitious names for individual students, schools, etc.

Standard Format of Research Articles • Abstract • Introduction: Context, Research Problem, Review of Literature • Methods • Results • Discussion • References

Introduction • Background - the reasons the author(s) conducted the study; theoretical framework • Statement of Purpose - the goal of the research (the destination); the problem statement • Hypotheses - “educated guesses” about relationships or differences

Methodology • Participants (sample) - who the subjects are, how obtained/selected • Materials (equipment, apparatus, measuring instruments) - what was used, quality of measuring instruments • Procedures - how study was conducted; what subjects did or what was done to them

Results • Summary of the analyses of numerical or narrative data used: • In text • In tables • In figures

Discussion/Conclusions • Non-technical interpretation of results • Linking results to original purposes and research questions • Why the results turned out the way they did • Identifying the study’s limitations • Formulating future plans of action

Descriptive Statistics • Methods used to obtain indices that characterize or summarize data collected • Focus is on the sample(s) at hand • Simple description of: – Individuals – Collection of individuals • Used as basis for inferential statistics

Basic Elements: Hypotheses • Hypothesis: a tentative statement (“educated guess”) about the expected relationship between two or more variables. – State expected relationship or difference – Be worthy of being tested – Be testable – Be brief and clear

Basic Elements: Variables • Variable: what is measured or varied. An attribute or characteristic of a person (or object) that can change from person to person. – Independent – Dependent – Control – Intervening

Classification of Variables • Independent Variable: a variable that is manipulated, measured or selected by the researcher in order to observe its relation to the subject's "response". An antecedent condition. • Dependent Variable: the variable that is observed and measured in response to an independent variable.

Action Research Three approaches • Individual teacher researcher • Collaborative action research • Schoolwide action research • Calhoun, E. (1993). Action research: Three approaches. Educational Leadership, 51(2), p 62 -65.