Character Study A Formative Evaluation Plan for Character

Character Study A Formative Evaluation Plan for Character Education Curriculum (Grades 6 -12) Aaron Case M. Ed. – Instructional Design

Harmony School of Innovation is a 6 -12 Public Charter and College Preparatory School and teaches with a primary focus on STEM. HSI – Fort Worth has recently acquired the title of Texas State School of Character from Character. org In the process, HSI has adopted the “PRIDE Initiative” which stands for Professionalism, Respect, Involvement, Determination, and Excellence. Introduction to Evaluation Plan In 30 -minute homeroom advisories, teachers are expected to hold Student Success Conferences with students regarding their individual progress of being a student with PRIDE. Once a week in these homeroom advisory periods, teachers present a lesson from the Character Education Advisory Curriculum. Students in their homeroom advisory class are graded on a three point scale (E = 100, S = 85, N = 70) on students’ demonstrations of PRIDE. Students receive two grades per the five components of PRIDE totaling 10 grades for their advisory homeroom class.

Evaluation Questions 1. Is the current Character Education Curriculum beneficial to students and teachers across the 6 -12 grades? 2. Has the adoption of the PRIDE initiative contributed positively to the overall campus culture?

SME Phase – Interview Data Instrument and Collection Phase #1 consisted of an interview with an SME – the Assistant Dean of Academics for HSI – Fort Worth, also head of the Character Committee who shaped the PRIDE Initiative for HSI. This interview was conducted June 2 nd, 2019 in the Assistant Dean’s office. The interview was open-ended and semi-structured. The reason I chose to use this kind of data-collecting instrument, over a survey or a structured interview, was because the questions that I had regarding the Character Education curriculum as well as PRIDE, dealt with an understanding of the timeline of how HSI – Fort Worth became a School of Character, as well as the decision-making process for how the advisory class was structured. As I learned more information, my questions and inquiries changed; hence, why the interview was semi-structured, and not structured.

Small Group Phase – Survey Data Instrument and Collection Phase #2 consisted of an survey with a small group – ten teachers from HSI – Fort Worth, who also have a responsibility to conduct Student Success Conferences regarding students’ PRIDE as well as deliver the pride curriculum. Five teachers have advisory classes in 6 -8 th grades, and five have advisory classes in 9– 12 th grades. This survey was open between May 27 th through June 3 rd 2019, through Google Forms. The survey consisted of ten Likert Scale selected responses. Each question focuses on their perceptions of school culture in regards to PRIDE, as well as the effectiveness and usefulness of the Character Education curriculum in the advisory classroom. The reason I used this data collection process was to receive focused and honest feedback on how the PRIDE Initiative has impacted the school culture and their views of how effective the character education curriculum has been in the homeroom advisory classes. The Likert scale responses will provide a more objective pool of results than holding a focus-group interview or perhaps a questionnaire with open-ended constructed responses.

Field Group Phase – Survey Data Instrument and Collection Phase #3 consisted of a survey with a student population – 877 students ranging from 6 -12 th grade who have participated in Student Success Conferences as well as the Character Education curriculum. Five teachers have advisory classes in 6 th-8 th grades, and five have advisory classes in 9 th – 12 th grades. This survey was open between January 16 th – May 24 th , through Google Forms. Student completed the survey during advisory via their school assigned Chromebooks. The data was interpreted on June 1 st, 2019. The survey consisted of fifteen questions: Five Likert scale questions and five open-ended questions. The Likert Scale questions were a scale between three choices (100 = Evident; 85 = Practitioner; 70 = Novice) to describe their level of each of the five components of PRIDE that they exhibit. The open-ended question that followed each Likert scale served as a rationale for their own self-reflection, as well as how they will work to improve that element of PRIDE in their everyday behavior. This data was retrieved from the school database, created by the Dean of Academics mentioned in the SME Phase of the collection. The reason I’m using this data in my evaluation plan is because it serves a purpose to see how students view the PRIDE initiative, as well as how they see their attitudes and motivations to showcase PRIDE in their everyday behavior.

Table #1 Phase 2 – Small Group Data

Table #2 Phase 2 – Small Group Data

Table #3 Phase 3: Field Group Data

Interpretation of Results SME - Interview Key Takeaways from the Interview: Entering 3 rd year of program. Teacher consistency and buying-in to the Character Education program and PRIDE initiative has been essential to the successes of the program. Current Character Education Curriculum has issues with relating to all grade levels. Example: A junior in high school reacts differently to a lesson about bullying than a sixth grader; likewise, a sixth grader may not see the value in a lesson about job interview dress-code the way a junior in high school entering the age of getting a job. Allowing students to have homeroom advisory teachers that actually teach them in the mainstream classroom will help the authenticity of the Student Success Conference; however, the program can work regardless if teachers continue to make establishing relationships a priority.

Interpretation of Results Small Group – Teacher Survey When it comes to the PRIDE Initiative (Table #1): Teachers recognized Involvement and Determination as evident components of PRIDE in their classrooms. Teachers did not recognize Excellence and Respect as evident components of PRIDE in their classrooms. For both results, this may be due to the objectivity and subjectivity of these two domains. For Involvement and Determination there is concrete evidence that teachers can use to check students Involvement and Determination, such as participation in clubs and activities, bringing a planner to each class, and academic progress in class. For Respect and Excellence, the subjectivity of what these domains entail make it difficult for teachers to know where to locate concrete evidence that their students have embodied PRIDE, other than behavior, rule-following, and general “good student” behavior.

Interpretation of Results Small Group – Teacher Survey When it comes to the Character Education curriculum implemented in the advisory classes (Table #2): Teachers generally see the Professionalism and Involvement domains of the Character Education curriculum as moderately effective. Teachers generally do not see the Respect and Determination domains of the curriculum as effective. When examining the grade levels of the teachers who took the survey, the results that leaned toward “ineffective” were grades (9 -12), and teachers who leaned toward “effective” were grades (6 -8). With this, I can conclude that teachers in middle school find more value in the advisory curriculum than those in high school. These results further add to the SME’s thoughts that the curriculum needs to be restructured to meet the needs of individual grade levels, rather than expecting all grade levels to use the same curriculum.

Interpretation of Results Field Group – Student Survey When it comes to how students evaluate their own embodiment of PRIDE (Table #3): Most students see themselves as professional and respectful. Most students do not see themselves as involved or excellent. Students are seeing themselves as moderately determined. When examining their open-ended responses further: It seems clear that students responded to the rubric items when they made their answers. Items like “professionalism” and “involvement” refer directly to professional dress code and being involved in after-school clubs or activities. If the student is in dress code, they responded as saying they were professional. If the student was not involved in afterschool clubs or activities, they responded as saying they were not involved, despite both domains going beyond just dress code or after-school clubs.

Strengths in the Program PRIDE as a Culture From the open-ended questions found in the student reflections on their own embodiment of PRIDE, it is clear that there is very little student confusion as to what the campus expects from its students when it comes to having good character. Students that scored themselves high, generally had a rationale that supported their feelings, while students that scored themselves low, understood why they were getting a lower score for that element of PRIDE. According to the SME, teachers have reported witnessing the vocabulary of PRIDE entering the conversations of their students, whether this be during an English lesson where a student noticed a character was acting “determined”, or in the hallway accounting for many peer-to-peer interactions showcasing “respect. ”

Strengths in the Program Cross-Campus Relationships From the interview conducted with the SME, students have been taking a PRIDE-ful ownership of their actions and progress. Students begin to share the strides that they are making in class or at home, often using the PRIDE language. According to the SME interview, PRIDE Student of the Month nominations from teachers offers a moment for students from each grade level to be recognized for exhibiting the components of PRIDE. According to the teachers that took part in the small-group survey, staff generally see their students as “determined. ”

Weaknesses in the Program Character Education Curriculum Effectiveness According to both the SME and the teacher survey results, teachers do not find that the advisory curriculum in its current state is an effective means of supporting PRIDE in the classroom. Both the SME (who is a Dean for grades 6 -8) and teachers who teach grades 6 -8 found that the curriculum was better suited for middle school than high school. There is no evidence currently supporting that there is a plan for a revised or altered curriculum, which means students who already participated in the advisory classes with the current curriculum will be participating in the same program two years in a row, which could add to the already perceived ineffectiveness.

Weaknesses in the Program PRIDE Measurability According to all three phases of data collection, the SME interview, the teacher survey, and the student self-reflections, the ability to measure one’s level of PRIDE is unclear and inconsistent. When assessing student “Respect”, teachers found that students were generally leaning toward moderate to low levels of respect, while students scored themselves higher in this domain of PRIDE. Teachers generally saw more evidence of “Excellence” than students did of themselves. This clash of consistency is likely due to the subjective natures of some domains of PRIDE and the objective natures of others, meaning that there is not evidence to accurately measure all five domains of PRIDE. This leads to issues when teachers who have advisory classes are expected to enter 10 grades into the gradebook that are measured by student’s evidence of PRIDE, the measurements will shift depending on the teacher.

Revisions After interpreting the data, assessing the strengths and weaknesses that come with that interpretation, I can conclude that the following revisions addressing the two original evaluation questions: Is the current Character Education Curriculum beneficial to students and teachers across 6 -12 grades? Revision 1: The Character Education curriculum skews in its benefits toward middle school grades (6 -8), and does not translate the same way to students in high school (9 -12); therefore, the curriculum needs to be specialized toward the character needs that each grade level presents, providing a curriculum that is vertically aligned, builds on itself, and does not repeat material and information that has already been taught.

Revisions After interpreting the data, assessing the strengths and weaknesses that come with that interpretation, I can conclude that the following revisions addressing the two original evaluation questions: Has the adoption of the PRIDE initiative contributed positively to the overall campus culture? Revision 2: The element of grading students based on their PRIDE presents problems for both students and teachers. Currently, advisory teachers grade students on their PRIDE based on the student success conferences they have with them during advisory, as well as their interpretations of their behavior in this 30 -minute time period. If the advisory class continues to have students be graded on PRIDE, then grades could stem from an objective assessment from the Character Education lessons, rather than the intrapersonal relationships between students and teachers that occur in the advisory class, which is more subjective. In summary, include more objective measures for PRIDE, allowing for students to have more opportunities to be recognized for their embodiment of PRIDE.

Changes to Future Evaluations After conducting this data collection process, I’ve considered the following changes to how I will pursue future evaluations: Conduct follow-up surveys – After interpreting the data, I found that it would have been useful to have a second set of data from a different point in the year. For example, having students in advisory reflect on their PRIDE at the very beginning of the school year, and seeing how it changed by the end to see if the PRIDE initiative is indeed working. Similarly, having teachers respond to the same survey at different points of the year can support whether the curriculum improved over time, or if the perceived quality remained consistent. Survey Structure Widen Interview Scope and Pool of Interviewees

Changes to Future Evaluations After conducting this data collection process, I’ve considered the following changes to how I will pursue future evaluations: Conduct follow-up surveys Survey Structure – After reviewing the results of the two surveys, I found that not including any open-ended questions from the teachers left some questions unanswered that I still had about teacher’s thoughts on the advisory curriculum. With the Likert scales and selected responses, I was only able to get a sense of the effectiveness of the curriculum and the presence of PRIDE in a narrow lens that would have been helpful in the interpretation process. In contrast, having students provide open-ended constructed responses to each of the domains of PRIDE in their self-reflection might have been helpful for advisory teachers to use when holding student success conferences, but for the purposes of this evaluation provided almost too much information. In addition, the Likert scale of only three scoring options limited the variety and range of results that having five would have had. Widen Interview Scope and Pool of Interviewees

Changes to Future Evaluations After conducting this data collection process, I’ve considered the following changes to how I will pursue future evaluations: Conduct follow-up surveys Survey Structure Widen Interview Scope and Pool of Interviewees – Considering the ideas of following-up and changing the structure of the survey, the evaluation would have benefitted from interviews from teachers who participated in the small-group survey. While the SME interview clarified the purpose for why the PRIDE Initiative and Character Education curriculum was put into place, the SME’s perceived strengths and weaknesses of the program would have been helpful to pair against an advisory teacher’s perceived strengths and weakness of the program. As mentioned previously, the teacher survey allowed me as the evaluator to solidify the need for revisions to the Character Education curriculum, but if I was wanting to determine specifics, having follow-up interviews that hosted those advisory teachers who participated in the survey would have been important.