ACT Complete A Holistic Framework for Education and

ACT Complete: A Holistic Framework for Education and Workplace Success Ryan O’Connor, Ph. D Director, Assessment Design 1

Overview • Broadening the Definition • Mapping a Multidimensional Solution • The Four Broad Domains – Core academic skills – Cross-cutting capabilities – Behavioral skills – Education and career navigation skills • Models & Implications 2

Limitations of Current Conceptions of College and Career Readiness Education • Focus exclusively on academic preparation – particularly mathematics and English • Empirical research indicates that success in school and at work is multidimensional • Readiness should similarly be focused on a broad and diverse set of personal characteristics Success Workplace Success Enrollment Employment Course Grades Job Performance GPA Organizational Citizenship Behavior Retention Tenure Satisfaction Job Satisfaction Graduation Promotion 3

Evidence Supporting Multidimensional College and Career Readiness School Outcomes Work Outcomes Each of the “Big Five” dimensions of personality predict academic success in school, and conscientiousness predicts academic performance in college just as well as cognitive skills (Poropat, 2009). Both conscientiousness and integrity provide incremental validity in the prediction of job performance beyond cognitive ability (Schmidt & Hunter, 1998). Academic goals (e. g. , commitment to Interests predict job performance, especially goals), grade goals, and achievement when they are congruent with jobs (Nye et motivation each predict college GPA al. , 2012). incrementally over other predictors such as high school GPA and ACT/SAT scores (Richardson et al. , 2012). 4

5

ACT Product Ecosystem 6

Holistic Framework: Dimensions of Readiness • Problem: Separate products in each domain, inconsistent features, no integrated picture. • Solution: Develop a holistic portrait of what individuals need to know and be able to do throughout the K-Career Continuum to be successful in education and work. • Result: Identified four broad domains important for education and work success. 7

Holistic Model of Education and Work Success Core Academic Skills Cross-Cutting Capabilities Behavioral Skills Education & Career Navigation Education and Work Success

Methodology © Tom Fishburne 2011 9

Research-based Rationale for the Broad Domains Core Academic Skills Cross-Cutting Capabilities Behavioral Skills Education & Career Navigation Empirical Research �� �� Theory/Models �� �� Standards �� � � �� Expert Input �� �� Surveys/Focus groups � �� � � O*NET Data � � �� = Significantly informed framework; � = Moderately informed framework 10

Core Academic Skills • Math • ELA • Science • Mathematics: Relating and transforming numeric and symbolic quantities, including applications to data sets, patterns, space, and change. • English Language Arts: Literacy skills related to comprehension and capacity for use of written and oral language. • Science: Gathering and interpreting observations, experimental data, and disciplinary content knowledge in order to predict and explain phenomena. Current ACT Assessments: K-12: ACT Aspire, Explore, Plan, ACT Postsecondary: Compass Workforce: Work. Keys, NCRC 11

Core Academic Skills Framework: Math Example 12

The Core Academic Skills Framework • Identifies key cognitive knowledge and skills that are critical for more advanced learning (e. g. , known barriers, limiting factors) • Emphasizes the connected, progressive nature of learning • Provides rich context by integrating research on LPs, misconceptions, and strategies • Expands current ACT CCRS by highlighting the importance of STEM and oral communication • Continues to anchor the ACT holistic model of education and work success • Together with the Cross-cutting Capabilities, maps the cognitive learning outcomes necessary for the 21 st century 13

Cross-Cutting Capabilities • Technology and Information Literacy • Collaborative Problem Solving • Studying and Learning • Thinking and Metacognition • Technology and Information Literacy: Using technology to acquire, evaluate, transform, and share information. • Collaborative Problem Solving: Contributing to group problem solving processes. • Studying and Learning: Using learning strategies to enhance comprehension, integration, and retention of learning. • Thinking and Metacognition: Mastering modes of thinking that apply to broad classes of problems. v Critical Thinking v Problem Solving v Decision Making v v Current ACT Assessments: K-12: ACT Aspire, Explore, Plan, ACT Postsecondary: Compass Workforce: Work. Keys Computational Thinking Metacognition 14

Cross-Cutting Capabilities Framework Illustration

The Cross-cutting Capabilities Framework • Extends ACT’s broadened definition of readiness, highlighting skills not typically the focus of academic instruction (e. g. , study strategies, technology, information literacy, and collaborative problem solving) • Comprised of transferrable knowledge and skills that: – Complement growth in the core academics and support advanced learning through application – Promote the ability “to operate with confidence in a complex, information-rich, globalized world” (ACARA) 16

Behavioral Skills • • • Acting Honestly • Keeping and Open Mind • Socializing with Others • Getting Along Well with Others • Maintaining Composure • Sustaining Effort Acting Honestly: extent to which a person values and adheres to ethical and moral standards of behavior, as well as personal level of humility. Maintaining Composure: extent to which a person is relatively calm, serene, and able to manage emotions effectively. • Socializing with Others: a person’s preferred level of social interaction, behavior in interpersonal situations, and optimism. • Getting Along Well with Others: extent to which a person interacts positively and cooperates with others, and is generally kind, friendly, and tactful. • Sustaining Effort: a person’s level of diligence, effort, organization, self-control, and compliance. • Keeping an Open Mind: a person’s level of open-mindedness and curiosity about a variety of ideas, beliefs, people, and experiences. Current ACT Assessments: K-12: Engage, Engage Teacher Edition Postsecondary: Engage Workforce: Talent 17

Behavioral Framework Illustration 18

The Behavioral Skills Framework • Incorporates research across multiples areas of psychology (e. g. , developmental, personality, and I/O psychology) • Includes general behavior areas as organizers and more specific behavioral skills components (e. g. , actual descriptors of behaviors, PLDs) • Applies to a broad range of ages, settings, and outcomes along the K – Career Continuum • States what individuals need to know and be able—and willing—to do in order to thrive in education and work 19

Education & Career Navigation • Self Knowledge • Environmental Factors • Integration • Managing Career & Education Actions • Self-knowledge: Perceptions of one’s own abilities, interests, skills, values, attitudes, beliefs, etc. that contribute to understanding the self. • Environmental Factors: Information, conditions, and experiences related to education and work that are acquired primarily from external sources and surroundings. • Integration: Ongoing process of combining self-knowledge and environmental factors to form personally-relevant knowledge structures used to evaluate information and to plan courses of action pertaining to education and work. • Managing Career & Education Actions: Ongoing process of implementing plans and enacting purposive behaviors that facilitate education and occupation progress. Current ACT Assessments: K-12: Interest Inventory, Work Values, Self. Rated Abilities Postsecondary: Interest Inventory, Work Values, Self. Rated Abilities Workforce: Fit 20

Education & Career Navigation Framework Illustration 21

Education & Career Navigation Framework • Focuses on acquiring, combining, and using knowledge about the self and the environment to purposefully and actively achieve goals • Is grounded in theoretical, empirical and intervention research support (e. g. , school counseling) • Includes components and subcomponents that are important throughout one’s education and career • Tracks the relative importance of these constructs on a variety of outcomes to better help individuals chart their education and work journeys 22

Bringing the Broad Domains Together • All four domains are clearly important for education and work success • Different constructs/domains relatively more important for different outcomes/transitions • Relationships among these constructs influence how they work together to affect success – Some constructs are complementary across broad domains – Other constructs have potential for incremental validity

Transitions Across the K-Career Continuum K – 12 K Core Academic Skills Cross-Cutting Capabilities Behavior Skills Education & Career Navigation 1 2 3 4 5 6 7 Postsecondary 8 9 10 11 Workforce 12 Early ES (K-2) Later ES (3 -5) MS (6 -8) Early HS (9 -10) Later HS (11 -12) Foundational Specialized Workforce ES (K-5) MS (6 -8) HS (9 -12) Early HS (9 -10) Later HS (11 -12) Postsecondary Entry/ Reentry Workforce Entry/Reentry Focus Graduation Progression/Shift Unemployment 24

Success Across the K-Career Continuum • We have identified relevant constructs for each of the 4 domains as they relate to major transitions. • A wide range of constructs contribute to success for each transition and outcome. • On the following slides, two models highlight example constructs important for – College Success (College GPA) – Work Success (Job Performance) 25

Holistic Model of College Success Science ELA Math Critical Thinking Technology and Information Literacy Studying and Learning Persistence Dependability Self-Confidence College GPA Socialization Academic Self-Efficacy Goals 26

Holistic Model of Work Success ELA Technology and Information Literacy Collaborative Problem Solving Decision Making Critical Thinking Goal Striving Persistence Cooperation Flexibility Job Performance Fit (Congruence) Supports Job Self-Efficacy 27

Implications for Research and Test Development • Quickly identify the constructs shown by research to be important for a given purpose and transition • Guide decisions about solution development • Inform prototyping of promising constructs/items • Detail the evidence needed for accurate, actionable assessment • Provide a framework for score interpretation and reporting • Support programmatic research for understanding growth and success across the K-Career continuum 28

Implications for States, Schools and Employers ü ESSA continues to require accountability systems that include academic achievement CCR indicators ü ESSA allows states to include additional measures of education quality (e. g. , school climate, engagement, attendance, safety) ü Integrated reporting on multiple measures helps students with decision-making at critical transitions (e. g. , Interest/Major Fit, Person/Institution Fit) ü Increased alignment of institutional resources and interventions improves retention and attainment ü Expanded readiness ensures a successful workforce is prepared for the future 29

The ACT Multidimensional Holistic Framework 30