Transformative Mixed Methods in Evaluation Donna M Mertens

Transformative Mixed Methods in Evaluation Donna M. Mertens Gallaudet University Katrina L. Bledsoe Walter R. Mc. Donald & Associates, Inc. AEA/CDC 2010 Summer Evaluation Institute Atlanta GA, June 15 & 16, 2010 © No part of this presentation can be duplicated without consent from the author.

Advanced Organizer n n n Role of the evaluation in an unequal world Transformative paradigm places priority on social justice & human rights Rationale Paradigm as driving force Transformative Paradigm Assumptions n Dimensions of diversity n Methodological framework n Criteria for rigor n

Introduction/Expectationsn n n Name, position, organization (no acronyms please) Where do you come from? Where are you now with regard to evaluation in communities as multicultural settings? What do you want to get out of this workshop? Why did you decide to come here? Describe your experiences in research with communities

Exercise-What is the role of the Evaluator You know the process called “free association”? Imagine someone asks you – what is the role of the evaluator? What would be the first few thoughts that would come to mind?

Discussion of Some Possible Evaluator Roles n n n Advocate Social Change Agent Representative of Community Values Critical Friend Resident Truth-teller

Steps in Transformative Paradigm 1. 2. 3. 4. 5. 6. 7. 8. Identify your worldview (paradigm) If transformative, identify community members to involve Develop mechanisms for working together to identify research focus & research questions Develop a rationale and write a mixed methods purpose statement Develop a design for mixed methods List your qual and quant data to be collected Review your qual and quant data analysis Draw a diagram of procedures in the design

Cyclical model for Transformative Research (Mertens, in press) Community Participation

Paradigms: Your world view Basic beliefs about n ethical action (axiology) n reality (ontology) n relation between knower and would-beknown (epistemology) n the appropriate approach to systematic inquiry (methodology) With acknowledgement to the ground breaking work of Guba & Lincoln

Transformative Paradigm n n Axiology: Respect for cultural norms; support for human rights and social justice Ontology: Rejects cultural relativism. Recognizes privilege given to what is perceived to be real based on: social, political, cultural, economic, ethnic, gender, and disability positionality Epistemology: Interactive link; knowledge is socially and historically located; trusting relationship Methodology: Qualitative/dialogic; Quantitative mix; Context

Ethics (Axiology): Ethnic Categorization n Prenatal Program in New Jersey Groups collapsed into one, although it was noted that there were several distinct cultural groups. n Refusal to speak with groups to determine what was needed for program n n Services then discontinued when not used

Understanding Privilege n Knowing one’s self (sense of self) n Knowing one’s status in the world n Acknowledging one’s privilege

Social Justice & Dimensions of Diversity n n n Gertrude Stein: Rose is a rose… Myth of homogeneity (Mertens, 2005) Sampling with dimensions of diversity in mind: Who needs to be included? How can people be included in the most appropriate way? But, is a pregnant African American woman? Is a person with a disability? (Mertens & Mc. Laughlin, 2004)

Diversity and Sampling Strategies Example: Deaf/HH court access n Communication - languages and modes: n n n n American Sign Language; highly educated ASL; limited education Gesture/pantomime/limited signing/low literacy Deaf/blind Hard of hearing people with assistive listening devices Oral deaf adults Mexican sign language Other dimensions of diversity: gender, race/ethnicity; status with court

Case study n n What dimensions of diversity are relevant in the case study? How would you know? How would you invite people to participate? How would you support them in culturally appropriate ways?

We…need enhanced understandings of related systemic processes of asymmetric power relations and privilege, not simply awareness and knowledge of difference and diversity… Hazel Symonette (2004). How and to what extent is sociocultural diversity associated with patterned differences in access, resource opportunities, and life chances? ” (p. 108)

Example of Mixed Method Transformative Approach n Trenton Obesity Prevention Study 1. Identified world view of evaluators and partners 2. Identified community members to involve 3. Developed process to work with one another 4. 5. Redesigned original research proposal to reflect mixed-methods approach. Provided new model of research design and program

Power issues & Methodological Assumptions n n Determination of the research focus, planning, implementation and use from a transformative ethical stance Power can be reframed in transformative research

Power Issues in the TOPS Project n n n Articulated power structure Articulated standards of that power structure Articulated the assumptions of what quality research would look like within that power structure n Versus the reality of what would be most appropriate for the situation

Methodology Applied to Case Study n n What kind of methodological approach would you use in conducting a transformative evaluation? How would you address issues of power in the evaluation?

Transformative Criteria n n n n Experiential Consequential Interpersonal Ontological Catalytic Critical reflexivity Reciprocity (first three criteria are elaborated on in Kirkhart, 2005; others are found in Mertens, 2005; and Lincoln, 2009)

Agenda for Action How will you modify your own evaluation to incorporate ideas from this workshop?

Resources n n n Mertens, D. M. (2010). Research and evaluation in education and psychology: Integrating diversity with qual, quant and mixed methods. 3 rd ed. Thousand Oaks, CA: Sage. Mertens, D. M. (2009). Transformative research & evaluation. NY: Guilford Mertens, D. M. & Ginsberg, P. (2009). (Eds. ) Handbook of Social Research Ethics. Thousand Oaks, CA: Sage.

Contact information n Donna M. Mertens, Gallaudet University n n Donna. Mertens@Gallaudet. edu Katrina L. Bledsoe, Walter R. Mc. Donald & Associates, Inc. n KBledsoe@wrma. com