Predicament and Promise The Internal Evaluator as Ethical

Predicament and Promise: The Internal Evaluator as Ethical Leader November 4, 2011 American Evaluation Association Anaheim, CA Francis J. Schweigert, Ph. D. College of Management Metropolitan State University Minneapolis, MN © November 2011 1

Overview I. Putting Ethics in Its Place II. The Predicament of the Internal Evaluator III. The Promise of the Internal Evaluator: The View from Nowhere IV. The Promise of the Internal Evaluator: Public Ethics and Public Deliberation V. Conclusion: New Directions in Public Ethics 2

I. Putting Ethics in Its Place Morals The regulation of interaction through affect (stimulus magnification) socialization (norms, values, roles, rank) conscience (individual freedom). Ethics The regulation of interaction through public reason deliberation 3

Mechanisms Regulating the Public Order Individual Conscience Morals Public Deliberation Ethics The Public Order Reciprocity— Quid pro Quo and Generalized Markets Enforced Mandates, Prohibitions, Plans Law Culture Common Sense 4

Ethics Questions or Claims of Value Determination of Value Systematic Inquiry Standard of Public Scrutiny 5

Ethics Which claims are valid? Which claims are binding as public duties? How just, for whom? Questions or Claims of Value Method: Public deliberation Determination of Value Systematic Practice Standard: Inquiry Public reason Value Standard: Justice Standard of Public Scrutiny 6

Evaluation and Ethics: Standard Categories Evaluator’s personal values, beliefs, commitments, priorities Evaluation standards, human subjects protections Funder, employer, or client values, mission, resources, commitments 7

Evaluation, Ethics, and Public Ethics Evaluator’s personal values, beliefs, commitments, priorities Funder , employer, or client values, mission, resources, commitments Public ethics, justice Evaluation standards, human subjects protections 8

Evaluation Questions or Claims of Value Determination of Value Systematic Inquiry Standard of Public Scrutiny 9

Evaluation What evidence matters? Which efforts qualify for public resources? Questions or Claims of Value Method: Evaluation research Determination of Value Standards: Fidelity, efficiency, effectiveness, accountability Systematic Practice Standards: Standard of Inquiry Utility, propriety, Public Scrutiny accuracy, feasibility. 10

Evaluation — and Method: Evaluation Research — Ethics Method: Public Deliberation Questions or Claims of Value Determination of Value Systematic Inquiry Standard of Public Scrutiny 11

II. The Predicament of the Internal Evaluator A. The moral setting. B. Challenges and risks. 12

Moralnet Theory: Traditional Societies Tribe (Language, ritual, ideology, hierarchies, overarching values) Band (Food, shelter, protection, leadership, ceremonies, face-to-face recognition, moral accountability) Family (Naroll, 1983) Family 13

Moralnet Theory: Industrial Societies Society, Nation (Language, ritual, ideology, hierarchies, overarching values) Workplace, Community Band (Food, shelter, protection, leadership, ceremonies, face-to-face recognition, moral accountability) Family (Naroll, 1983) Family 14

Workplace as Normative Reference Group Workplace identity Mutual protection Mutual respect Team membership and loyalty Earning power Learning, motivation to improve 15

Workplace as Normative Reference Group Workplace identity Mutual protection Mutual respect Moral approval and disapproval Team membership and loyalty Earning power Learning, motivation to improve 16

Workplace as Normative Reference Group Workplace identity Mutual protection Mutual respect Evaluator Team membership and loyalty Earning power Learning, motivation to improve 17

Moral Predicament of the Evaluator Professional identity and loyalty Workplace identity Mutual protection Value standards of fidelity, efficiency, effectiveness Mutual respect Evaluator Team membership and loyalty Earning power Research standards of objectivity and evidence Practice standards of propriety, accuracy, utility, feasibility Learning, motivation to improve Judgments under public scrutiny 18

Moral Predicament of the Evaluator All workers – sensitive to workplace dynamics of moral approval and disapproval. – sense of self-worth and self-identity in work and organizational membership. Professions – can require stepping outside the world of moral approval and disapproval. – judged against ideals and norms of their profession – “Do no harm” – “Fair administration of justice and conflict resolution” Evaluators – profession is regulative, not productive – no contribution without evaluation standards – “Judgment of merit, worth, significance” 19

III. The Promise of the Internal Evaluator: The View from Nowhere A. Evaluation capacity building q Regulative role of evaluation q Standards of objectivity q Value in the public view B. Evaluative thinking 20

Regulative Role of Evaluation l a n o i at s z i n ces a g Or Suc Mission and direction Operations and production ts , n e rth m g Wo d Ju rit, nce e fica M of igni S Evaluation of fidelity, efficiency, effectiveness, accountability 21

Regulative Role of Evaluation l a n o i t a s z i n ces a g Or Suc Adam Smith’s “Impartial Spectator” Evaluation of fidelity, efficiency, effectiveness, accountability Mission and direction Operations and production The View from Inside: Invested in success ts , n e rth m g Wo d Ju rit, nce e fica Thomas Nagel’s M i f n o ig “View from Nowhere” S (Nagel, 1986; Smith, 1790/1984) 22

“The View from Nowhere” Standards of Objectivity 1) a public framework of thought, with judgments and conclusions based on reasons and evidence; 2) the use of norms of correct judgment, such as logical argument and rules of evidence; 3) an order of reasons based on principles and criteria; 4) a viewpoint distinguishable from a particular point of view; 5) an account of agreement upon methods of evidence and argument; and 6) the ability to explain disagreements (Rawls, 1996, pp. 110 -112) 23

Ethical and Evaluative Space “ In doing so, evaluation creates an ethical space—that is, a space defined by a temporary suspension of moral ethical [sic. ] assumptions. We will assume in this space a vacuum of intentions, a sense of anticipation, that its boundaries are given by abutting edges of varying expectations and perspectives. “. . This is a space in which some people are invited to make novel judgements about the work of others, and in which the nature of those novel judgements can be regulated and scrutinized. It is in making relatively transparent the basis of judgement, that is, providing the data which people can interpret in different ways, that we add the ethical dimension. “ Ethical space invites regulation, the re-imposition of ethical order…” (Kushner, 2000, p. 150) 24

Building Evaluation Capacity “ Building the evaluation capacity of an organization to support staff in thinking evaluatively means integrating evaluation into the organization’s culture. This goes well beyond a focus on using the results of isolated studies. “ It takes us into the arena of organizational culture, looking at how decision makers and staff incorporate evaluative thinking into everything they do as part of ongoing attention to mission fulfillment and continuous improvement. ” (Patton, 2008, p. 157) 25

IV. The Promise of the Internal Evaluator: Public Ethics and Public Deliberation A. Reporting and Using Findings q Compliance and interpretation q Performance and policy formation B. Organization’s Public Reality 26

Evaluation Interpretation, Compliance, and Rule-Making • Policy expectations and purposes: • ——What are realistic terms of behavior? INTERPRET • ——What is required for program fidelity? COMPLY • Program implementation and results: • ——What must be identified, measured, and tracked? • ——What analysis and synthesis is meaningful? INFORM • Policy revision or development: • ——What policy conclusions might follow? • ——What further study is required? 27

Organization’s Public Reality “ It lies within the internal evaluator’s purview to ask if programs are carried out in a manner that affirms human dignity, human community, and fair terms of cooperation. “ Similarly, evaluators can examine results achieved for evidence of social benefit and just allocation of goods and services. ” (Schweigert, 2011, p. 16) 28

V. Conclusion: New Directions in Public Ethics A. Evaluation in service to justice B. Evaluation in service to the public order 29

Extending Evaluative Thinking to the Public Arena Evaluation of organizational performance, results, and significance Examination of influence and impact on the public good Procedural Justice: Processes in accord with norms of respect and fair cooperation? Distributive Justice: Allocations in accord with norms of need and merit? 30

Evaluation in Service to Justice In the person of the citizen: The General Will Evaluative thinking As the good of another: Effects of programs on the public The arrangement of basic institutions: Human rights Fair terms of cooperation Public reason in deliberation Justice 31

Evaluative Assessment and the Public Order 4. 3. 2. 1. What are the objective bases for claims? What are the responsibilities and commitments of office and are they carried out, as a matter of public justice? What are the conditions and requirements of participation, and how well do these satisfy the requirements of procedural fairness and human dignity? How are goods and services being allocated, and can this distribution be justified in reasonable terms of merit or need? 32

Evaluation in Service to the Public Order “For administrators and legislators to take responsibility for justice in public policy formation and implementation requires that they take the objective view embodied in evaluation, holding their implementation and results publicly accountable to evidence of performance and impact on the human condition. “In support of this commitment to justice, internal evaluators can provide the crucial support of evaluative thinking—the view from nowhere. This includes…the assessment of the ethical implications of programs that are being evaluated, informing not only internal government operations but public deliberations as well. ” (Schweigert, 2011, p. 20) 33

Evaluation as the “Alpha Discipline” “ The alpha discipline is the one that has the power, in this case the keys to the kingdom of the disciplines. Evaluation has that power because all disciplines are completely dependent for their legitimacy on the quality of their intradisciplinary evaluation, i. e. , their ability to identify good vs. bad theories, data, hypotheses, explanations, etc. , the tools of every discipline’s expertise. ” (Scriven, 2011, January 9) 34

List of References Cited Kushner, S. (2000). Personalizing evaluation. Thousand Oaks, CA: Sage Publications. Nagel, T. (1986). The view from nowhere. New York: Oxford University Press. Naroll, R. (1983). The Moral Order: An Introduction to the Human Situation. Beverly Hills, CA: Sage Publications. Patton. M. Q. (2008). Utilization-focused evaluation (4 th ed. ). Thousand Oaks, CA: Sage Publications. Rawls, J. (1971). A Theory of Justice. Cambridge, MA: Harvard University Press. Rawls, J. (1996). Political liberalism. New York: Columbia University Press. Schweigert, F. J. (2011). Predicament and promise: The internal evaluator as ethical leader. Manuscript submitted to New Directions for Evaluation. Scriven, M. (2011, January 9). Thought Leaders Forum, RE: Michael Scriven, one who needs very little introduction. Retrieved from the American Evaluation Association Discussions at http: //comm. eval. org/EVAL/Discussions/Message/Default. aspx? MI D=817 Smith, A. (1984). The theory of moral sentiments (6 th ed. ). Indianapolis, IN: Liberty Fund. Original work published 1759, rev. 6 th edition 1790. 35