Improving Accountability by Reporting to Multiple Audiences Presentation

Improving Accountability by Reporting to Multiple Audiences Presentation to the Canadian Evaluation Society/ Société Canadienne D’ Évaluation Vancouver, B. C. June 4, 2003 Session 302 1 Presentation by Ormond Hammond, PREL

What is the Strategy? l Reporting 2 to Multiple Audiences

Primary Purpose and Use l 3 Address the diverse accountability needs of multiple audiences by using a core set of indicator data in various reporting formats.

How Was This Strategy Used in REL Performance Indicators Work? 4 Report Primary Audience Other Audiences GPRA Report Annual U. S. Congress Other Labs, Funding Agency, Department, Public REL System Report General Public Education Community, Congress Individual REL Reports Funding Agency Department, Congress

Example of GPRA data: Indicator 1. 1 Number of development sites: an increasing number of local or state sites will be engaged in collaborative development and demonstration of comprehensive reformrelated efforts. Year 5 Sites Students Teachers Administ rators Parents 1997 494 83, 147 5, 899 512 14, 437 1998 615 93, 788 6, 950 749 16, 062 1999 606 538, 865 37, 550 5, 169 13, 697 2000 630 545, 612 34, 923 5, 029 13, 024 2001 359 37, 847 5, 869 1, 801 183 2002 206 NR 4, 316 1, 055 268

GPRA Reports: Required l Targets and Performance Data l l l Assessment of Progress l l 6 Actual Performance Targets Status Explanation

GPRA Reports: Required (Cont. ) l Sources and Data Quality l l l l 7 Additional Source Information Frequency Collection Period Data Available Validated By Limitations Improvements

Example from REL Annual Report - 2001 l Indicator: Number of Development and Implementation Sites l Current Contract: An increasing number of local or state sites engaged in collaborative development and demonstration of a range of reform efforts will show improved practice over time. l Previous Contract: The number of development and demonstration sites increased 27. 5 percent, from 494 in 1997 to 630 in 2000. 8

Example from Individual REL Report: PREL 2001 Indicator 1. 1 Number of development sites: An increasing number of local or state sites will be engaged in collaborative development and demonstration of a range of reform-related efforts . 9 Sites (Schools) 10 Students 1276 Teachers 77 Administrators 10 Parents 0 Other 0

Benefits and Limitations l Pluses: – – – 10 Helps us see ourselves and function as a system Ensures that we have some common measures Increases our relevance and usefulness to different audiences

Benefits and Limitations l Concerns and Challenges: – – – 11 Cannot report on the work in depth Cannot report on strengths of individual labs Difficult to report on data limitations (e. g. , some types of data simply cannot be collected in the same way everywhere)

Lessons Learned l l l 12 Primary audience is not the only audience Work towards consistency in data collection Use reporting to improve accountability at all levels