Outline Definitions of ResearchEvaluation Purposes of Evaluation Research

Outline • • • Definitions of Research/Evaluation Purposes of Evaluation / Research Science and Scientific Management Research/Evaluation as Process Types of Research / Evaluation

Definitions • Evaluation = Process of judging the merit or worth of something • Research – application of scientific methods to answer questions – controlled inquiry directed at increasing knowledge/establishing truth • Evaluation Research - combine the two

Research Evaluation Research

Why Evaluate - the Academic list • To assess merits of alternative programs • To discover whether & how well objectives are being fulfilled • To determine the reasons for successes & failures. • To uncover the principles underlying a successful program. • To refine, revise, update or track a program

Why Evaluate- "The Real List" 1. Because we are required to 2. To make better decisions. 3. To learn from experience 4. To justify programs 5. To kill programs

Purposes of Research • • Answer management questions - applied research For sake of knowing - pure or basic research Answer research questions - methodological Develop/test management alternatives developmental research • Assess worth or merit of programs - evaluation research

Research Purposes - II • • Exploratory Descriptive Explanatory Predictive

Ways of Knowing or Establishing belief or truth • • Tradition Authority Repetition/Tenacity Science

Science • Body of Knowledge • Method of Inquiry • • • logical systematic abstract general parsimonious – induction – deduction • self-corrective • empirical

Scientific Management • Application of scientific principles to management and decision making – systematic information gathering – empirical, objective, self-corrective

Process -- Steps • • • Research define problem objectives/hypotheses literature review research methods gather data/analysis conclusions • • • Evaluation describe program evaluation criteria program scoping evaluation methods gather data/analysis conclusions

Types of Evaluation by Program Stage • formative (conceptualization/design)] • process (implementation) • summative (outcomes, impacts, efficiency)

Types - By Approach • Standards – norm-based – criterion-referenced • Goals and objectives • Impacts or effects

Types- by method • • • Qualitative - quantitative Secondary data - Primary Survey - Experiment Internal vs external Who - Peers, participants, superiors, subordinates, scientists

Evaluation Criteria • • • Effort - qnty and qlty of inputs Performance - qnty and qlty of outputs Adequacy - meet needs? Efficiency - benefits/costs Equity - distributional issues, fairness

Process Evaluation • Identifies how and why program works – attributes – recipients – conditions – effects • • single or multiple intended or side effects timing & duration, long/short term cognitive, affective or behavioral

Types by program area/subarea • Emphasis: Parks, Tourism, Interp, Program Mgmt, Commercial Rec, . . . • Program : lands, facilities, personnel, budget, marketing, PR, maintenance, design, policies, plans

Types of Research • Basic - Applied • Exploratory, Descriptive, Explanatory, Predictive • In-house / out- house • by discipline - sociological, psych, economics • by topic - boating, wilderness, legal, tourism, . . . • by method - survey, expmt, . . .

Examples of Evaluation Studies • • Community needs assessment Feasibility study, SWOT analysis Program Audits, Market audit Cost/benefit anaysis Impact assessment PPBS, MBO, GPRA, CAPRA, LAC, . . . Peer review Importance-performance analysis