EDPSY 505 Note Next week meet in Zimmerman

EDPSY 505 Note: Next week meet in Zimmerman Room 254

What Makes Research Scientific?

According to the Dictionary Science: • Is the observation, identification, description, experimental investigation, and theoretical explanation of phenomena.

How does the No Child Left Behind Act of 2001 (NCLB) impact educational research?

Who Cares? You should. • The No Child Left Behind Act of 2001 uses the phrase “scientifically-based research” (SBR) 111 times. • This has spawned an industry of consultants. • It has created a very volatile atmosphere.

What Is Scientific Research? (According to NCLB) • The application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge. • Systematic, empirical methods that draw on observation or experiment. • Involves rigorous data analyses that are adequate to test the hypotheses. • Is evaluated using experimental or quasi-experimental designs. • Is reported in sufficient detail to allow replication. • Has been accepted by a peer-reviewed journal or approved by an independent panel of experts through rigorous, objective, and scientific review.

What Paradigm Appears to Be Influencing NCLB? Postivism? Post-positivism? Critical Theory? Constructivism?

Between a Rock and a Hard Place The rock. • Calling for scientifically based research is good and needed. • “The recent enactment of no child left behind, and its central principle that federal funds should support educational activities backed by “scientifically-based research, ” offers an opportunity to bring rapid, evidence-driven progress – for the first time – to U. S. Elementary and secondary education. ” – Coalition for evidence-based policy. The hard place. • Defining SBR as randomized experimental designs is over-restrictive. • “The requirement that research methods be restricted to group design with a preference for randomized clinical trials will significantly inhibit the development and validation of new scientific knowledge in education. ” – American association on mental retardation (AAMR) board of directors.

“Council recognizes randomized trials among the sound methodologies to be used in the conduct of educational research and commends increased attention to their use as is particularly appropriate to intervention and evaluation studies. ” – AERA council

“However, the council of the association expresses dismay that the department of education through its public statements and programs of funding is devoting singular attention to this one tool of science, jeopardizing a broader range of problems best addressed through other scientific methods. The council urges the department of education to expand its current conception of scientifically-based research. ” -AERA council

What Is Scientific Research? (According to the NRC) • Science poses significant questions that can be investigated empirically. • Science links research to relevant theory. • Science uses methods that permit direct investigation of the question. • Science provides a coherent and explicit chain of reasoning.

What Is Scientific Research? (According to the NRC) • Scientific findings replicate and generalize across studies. • Scientists disclose research and encourage professional scrutiny and critique.

Mayer (2000) • Let’s take a few minutes and read Mayer. • What makes research “scientific”? • How important is it that educational research be respected in “academia and in society in general”? • Should “science” and “research” mean the same things in different disciplines? • What questions/ comments do you have?

Mayer (2000)

The Big Picture • There are many different research processes • Each has its own: – Philosophy of inquiry – Methods of inquiry – Purposes for doing research – Processes and “rules” • Quantitative research has its own • Here is one process:

Scientific Thinking Vs. Everyday Thinking • Everyday thinking – Biased questions • Do you really support the war? – Limited sampling • Your friends and family are different from my friends and family – Selective attention • Confirmation bias – Inaccurate generalization • Stereotypes

Scientific Thinking Vs. Everyday Thinking (Cont. ) • Scientific thinking. – Empirical observations. • Empirical: capable of being confirmed, verified, or disproved by observation or experiment. – Systematic. – Objective. • Less dependent on emotion or personal prejudices. – Replicable.

Purposes of Scientific Research • Exploratory – What is out there? • Descriptive – What does this group look like? • Explanatory – Why and how are these constructs related? • Evaluation – Does this program work? • Prediction – Who will become depressed?

Variables

On the Theory of Scales of Measurement (Stevens, 1946) • Measurement – Is the assignment of numerals to objects. • Nominal – Examples: Gender, party affiliation, and place of birth • Ordinal – Examples: SES, Student rank, and Place in race • Interval – Examples: Test scores, personality and attitude scales. • Ratio – Examples: Weight, length, reaction time, and number of responses

Understanding Variables and Hypotheses (Charters, 1992) • Objects – Things that one does research on. • People, districts, nations, etc. • Properties of objects – Give us a way to talk about how objects are alike and how they differ. • Scores – Values on the property of interest • Must be at least two.

Values • Exhaustive – Must be able to assign a value to all objects. • Mutually Exclusive – Each object can only be assigned one of a set of values. • A variable with only one value is not a variable. – It is a constant.

How variables are used • Two major piles – Descriptive and Verificational (cause-effect) • Descriptive – Describes a population in relation to one or more variables. • Sex bias in textbooks • Trends in dropout rates • Verificational – Does A cause B – Associations between A and B • Is the observed relationship greater than would be expected by chance?

Hypothesis • A proposed explanation for a phenomenon. – Two types • Casual order - 'A causes B' • Correlational – ‘A is related to B’

Sorting Out Variables in a Study • Purpose of most empirical studies in behavioral research is to identify causal relationships. – Independent variables (IV) • Causes, determinants, predictors, factors. – Dependent variables (DV) • Consequences, outcomes, effects – Moderator variables • Variables that change the relationship between the IV and DV. • Aptitude by treatment interactions

Dissecting Hypotheses 1. Identify the two variables and sort them into IV and DV. 2. Describe each variable. • Object, property, mode of variation, elaborate on mode of variation. 3. Specify the relationship expected between the two variables. 4. Note the unit of analysis implied or actually used.

Additional comments • Simple Hypotheses have only two variables--bivariate relations. – H. 1: Authoritarian principals are more effective than non-authoritarian principals • What are the names of the two variables? • How do they vary? • Complex Hypotheses have more than two variables and sometimes contain a moderating variable. – H. 1: Authoritarian principals are more effective than non-authoritarian principals when goals are clear, but non-authoritarian principals are more effective when goals are ambiguous. • What are the names of the three variables? • How do they vary?

Practice • H. 1: The greater the weight of a five-year old, the taller the child. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

Practice • H. 2: Authoritarian principals command more loyalty than non-authoritarian ones when they have influence, but non-authoritarian principals command greater loyalty when principals lack influence. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

• H. 3. Secondary teachers are more custodial in pupil control ideology than elementary teachers. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

H. 4. Academic achievement will be greater among students taught by autocratic teachers than those taught by permissive teachers. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

• H. 5. The larger the size of a community college’s instructional faculty the greater the degree of administrative centralization. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

• H. 6. Democratic supervisors have teachers who demonstrate more creativity in teaching methods than autocratic supervisors. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

• H. 7. When administrators have influence with their superiors, authoritarian administrators command more loyalty from subordinates than non-authoritarian ones, but when administrators have little influence, then nonauthoritarian administrators command more loyalty than authoritarian ones. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

• H. 8. The stronger the collective efficacy of a school, the higher the level of student achievement. – – – – What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable?

Now try it with your proposal topic • • What is the object? What are the variables? What are the names of the variables? How do they vary-categorical or continuous? What is the moderating variable? What is the independent variable? What is the dependent variable? State how you expect them to be related to one another.