Assessing Professional and Personal Development in Contemporary Graduate

  • Slides: 44
Download presentation
Assessing Professional and Personal Development in Contemporary Graduate Education Graham D. Rowles, Monica M.

Assessing Professional and Personal Development in Contemporary Graduate Education Graham D. Rowles, Monica M. Underwood, John F. Watkins and Heidi H. Ewen University of Kentucky Graduate Center for Gerontology Mary T. Johnson Indiana University School of Medicine

Our Sector of Education is Undergoing Major Change • “Business as Usual” doesn’t satisfy

Our Sector of Education is Undergoing Major Change • “Business as Usual” doesn’t satisfy the public expectation for compassionate professionals any longer • Learners are challenged to incorporate defined guiding principles • Graduate programs are codifying these principles within formal student assessment processes

Old Social Contract • Professionals were perceived as motivated by their service to others

Old Social Contract • Professionals were perceived as motivated by their service to others • Society relied upon professional associations to set ethical and performance standards • Colleagues can effectively monitor compliance to standards for practicing professionals

New Perceptions • The public is suspicious of professionals of all types • Society

New Perceptions • The public is suspicious of professionals of all types • Society relies on formal administrative rules and government regulations for control over professional behaviors • Litigation, lobbying, and attention to special interest groups have altered relationships with professionals

Emerging Paradigm • Peer awareness & assessment of self in relationship to others •

Emerging Paradigm • Peer awareness & assessment of self in relationship to others • Continuing development at all levels to enhance professional viability • Programs that strive to inculcate learners with professional standards during training

Professional Characteristics UK Graduate Center for Gerontology • 7 Dimensions of Intellectual & Professional

Professional Characteristics UK Graduate Center for Gerontology • 7 Dimensions of Intellectual & Professional Growth • 4 Dimensions of Personal Growth Graduate Center for Gerontology IU School of Medicine • 5 Competencies Defined by Medical Expertise • 4 Competencies Related to Personal Growth & Role Recognition

From Admission to Graduation and Beyond Assessing Outcomes in Gerontology Doctoral Education Graham D.

From Admission to Graduation and Beyond Assessing Outcomes in Gerontology Doctoral Education Graham D. Rowles, Ph. D. Monica M. Underwood, B. S. John F. Watkins, Ph. D. Heidi H. Ewen, Ph. D. University of Kentucky Supported by University of Kentucky Quality Enhancement Program grant CIP 30. 1101; PCS 046 Graduate Center for Gerontology

Doctoral Program Development 1989 1991 1994 1997 1998 2003 2004 2005 University of Southern

Doctoral Program Development 1989 1991 1994 1997 1998 2003 2004 2005 University of Southern California University of Massachusetts, Boston University of South Florida University of Kentucky University of Kansas University of Maryland Baltimore/ University of Maryland Baltimore County University of North Texas Miami University of Ohio Purdue University (dual discipline degree) Graduate Center for Gerontology

Diversity of Emphasis Interdisciplinary: theoretical and applied (University of Southern California) Public Policy (University

Diversity of Emphasis Interdisciplinary: theoretical and applied (University of Southern California) Public Policy (University of Massachusetts, Boston) Health/Mental Health/Cognitive Aging/Public Policy (University of South Florida) Interdisciplinary: Aging and Health (University of Kentucky) Applied Gerontology (University of North Texas) Social and Behavioral Gerontology (University of Kansas) (Miami University of Ohio) Epidemiology/Policy/Psychosocial (University of Maryland Baltimore/ University of Maryland Baltimore County) Dual Degree with Discipline (Purdue University) Graduate Center for Gerontology

Context 1. Lack of evaluation strategies and tools for the dynamic assessment of student

Context 1. Lack of evaluation strategies and tools for the dynamic assessment of student progress through doctoral programs in gerontology 2. Lack of information on gerontology doctoral program outcomes 3. Lack of mechanisms for assessing overall doctoral program effectiveness Graduate Center for Gerontology

The Quality Enhancement Project Objective To develop a comprehensive tracking system and evaluation tool

The Quality Enhancement Project Objective To develop a comprehensive tracking system and evaluation tool to provide ongoing assessment of student progress and doctoral program effectiveness in moving toward specified programmatic and personal goals. Graduate Center for Gerontology

The Quality Enhancement Project Elements 1. Comprehensive program data base and resource system. 2.

The Quality Enhancement Project Elements 1. Comprehensive program data base and resource system. 2. Self rating assessment tool for students to dynamically monitor their own progress toward accomplishing specific professional and personal goals. 3. Protocol for faculty assessment of students. 4. System of interactive feedback among students and faculty. Graduate Center for Gerontology

The Data System Characteristics 1. On-line system 24 hour access 2. Student profiles and

The Data System Characteristics 1. On-line system 24 hour access 2. Student profiles and portfolio (controlled by student) 3. Assessment tool (completed by students with selective faculty access) 4. Controlled Accessibility 1. and privacy Graduate Center for Gerontology

Screen Sample: Profile and Portfolio Pages Graduate Center for Gerontology

Screen Sample: Profile and Portfolio Pages Graduate Center for Gerontology

Dimensions of Assessment Professional Growth Scholarly and Professional Independence Critical and Integrative Thinking Methodological

Dimensions of Assessment Professional Growth Scholarly and Professional Independence Critical and Integrative Thinking Methodological Competence Intellectual Risk Taking/Creativity Gerontological Knowledge Base Scholarly Communication Identification with Gerontology Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 1) Scholarly and Professional Independence • I am able

Dimensions of Assessment (Professional Growth) 1) Scholarly and Professional Independence • I am able to express my own thoughts and ideas. • I am able to conduct myself effectively in a professional environment. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 2) Critical and Integrative Thinking • I am confident

Dimensions of Assessment (Professional Growth) 2) Critical and Integrative Thinking • I am confident in my ability to critically assess research design and methods. • I feel confident integrating insights and ideas from diverse areas of research and scholarship. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 3) Methodological Competence • I am confident in my

Dimensions of Assessment (Professional Growth) 3) Methodological Competence • I am confident in my ability to formulate useful research questions. • I am competent working in a clinical environment (i. e. patient contact). • I am confident in my ability to analyze and interpret data. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 4) Intellectual Risk Taking/Creativity • I am confident expressing

Dimensions of Assessment (Professional Growth) 4) Intellectual Risk Taking/Creativity • I am confident expressing alternative points of view in scholarly and professional settings. • I am confident in extending theory through my own research activities. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 5) Gerontological Knowledge Base • I feel well-grounded in

Dimensions of Assessment (Professional Growth) 5) Gerontological Knowledge Base • I feel well-grounded in theory. • I am well-versed in the current issues and problems of aging. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 6) Scholarly Communication • I am confident in my

Dimensions of Assessment (Professional Growth) 6) Scholarly Communication • I am confident in my ability to orally present my research ideas and findings. • I am confident in my ability to express my research ideas or findings in writing. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Professional Growth) 7) Identification with Gerontology On the following scale, indicate

Dimensions of Assessment (Professional Growth) 7) Identification with Gerontology On the following scale, indicate the extent to which you identify yourself as a gerontologist. 10 – Identify myself as a gerontologist 9 8 7 6 5 – Not sure of my scholarly identity 4 3 2 1 0 – I identify myself as a disciplinarian (biologist, sociologist, psychologist) Graduate Center for Gerontology

Dimensions of Assessment Personal Growth Involvement/Commitment Emotional Wellbeing/Stress Management Physical Health Time Management Graduate

Dimensions of Assessment Personal Growth Involvement/Commitment Emotional Wellbeing/Stress Management Physical Health Time Management Graduate Center for Gerontology

Dimensions of Assessment (Personal Growth) 1) Involvement/Commitment • Engagement/involvement in departmental seminars, colloquiums, meetings

Dimensions of Assessment (Personal Growth) 1) Involvement/Commitment • Engagement/involvement in departmental seminars, colloquiums, meetings and events is important to my development. • I have opportunities to participate in the development of the program. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Personal Growth) 2) Emotional Well-Being/Stress Management • I can adjust to

Dimensions of Assessment (Personal Growth) 2) Emotional Well-Being/Stress Management • I can adjust to the rigors of the doctoral program. • I can maintain a sense of self as a participant in this program. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Dimensions of Assessment (Personal Growth) 3) Physical Health • How is participating in the

Dimensions of Assessment (Personal Growth) 3) Physical Health • How is participating in the doctoral program affecting your health, if at all? (1=very negatively, 2=negatively, 3=no effect, 4=positively 5=very positively, 9=don’t know) • How often do you experience symptoms of depression or anxiety? (1=always, 2=often/frequently, 3=occasionally/ infrequently, 4=never) Graduate Center for

Dimensions of Assessment (Personal Growth) 4) Time Management • I maintain adequate levels of

Dimensions of Assessment (Personal Growth) 4) Time Management • I maintain adequate levels of leisure and recreation. • I effectively manage the many responsibilities and competing demands of the program. Scale: 1=Strongly Disagree, 2=Disagree, 3=Neither Agree Nor Disagree, 4=Agree, 5=Strongly Agree, 9=Don’t Know Graduate Center for Gerontology

Self Assessment Schedule 1. Entry into program 2. End of first semester (completion of

Self Assessment Schedule 1. Entry into program 2. End of first semester (completion of initial core) 3. End of first year 4. Completion of Integrative Seminar (usually following fourth semester) 5. Following qualifying examinations 6. Following doctoral dissertation defense 7. Two years post graduation Graduate Center for Gerontology

Current Status 1. Pilot tested with four students (a Sigma Phi Omega Initiative) 2.

Current Status 1. Pilot tested with four students (a Sigma Phi Omega Initiative) 2. Students excited about potential for file sharing and collaborative work on projects 3. Students enthusiastic about vita building potential and flexibility. Graduate Center for Gerontology

Current Status (cont. d) 4. Concerns about open faculty access to personal growth measures

Current Status (cont. d) 4. Concerns about open faculty access to personal growth measures (particularly emotional wellbeing/stress management). Resolution: only advisor and Director of Graduate Studies (on a “need to know” basis) have access to personal growth measures. 5. System now fully on line as of September, 2006 6. Research using the system has begun Graduate Center for Gerontology

Future Directions 1. Continued research on student progress, productivity and outcomes (IRB approved, initial

Future Directions 1. Continued research on student progress, productivity and outcomes (IRB approved, initial analyses completed), and curriculum evaluation and refinement of program 2. Evaluation and refinement of system 3. Sharing of system with other doctoral programs to develop comprehensive system of evaluation of progress in gerontological education. Graduate Center for Gerontology

Assessing Student Progress in a New Medical Curriculum Using Electronic Portfolios Mary T. Johnson,

Assessing Student Progress in a New Medical Curriculum Using Electronic Portfolios Mary T. Johnson, Ph. D. Indiana University School of Medicine Director of Problem Solving Competency

Medical Competencies • Problem Solving • Lifelong learning • Basic Clinical Skills • Effective

Medical Competencies • Problem Solving • Lifelong learning • Basic Clinical Skills • Effective Communication • Scientific Basis of Medicine • Social and Community Contexts of Health Care • Self-Care, Self-Awareness, & Personal Growth • Moral Reasoning and Ethical Judgment • Professionalism & Role Recognition

Portfolios in Medical Education • The time is ripe for Electronic Portfolios in U.

Portfolios in Medical Education • The time is ripe for Electronic Portfolios in U. S. Medical Education • European medical portfolio model • ACGME Toolbox – Documentation of learning for Residency Programs • US medical schools: educational needs similar to other professional disciplines

School-Wide Concerns • Medical schools should share responsibility with medical students for their learning

School-Wide Concerns • Medical schools should share responsibility with medical students for their learning outcomes • A better, more granular picture of how emerging professionals understand assume accountability for their own learning is needed • A vision should develop to facilitate student planning of learning choices and assembly of evidence for their learning

The Indiana Statewide System for Medical Education Electronic portfolios will be used soon to

The Indiana Statewide System for Medical Education Electronic portfolios will be used soon to assess student competency accomplishments at all sites STATEWIDE STANDARDS Integration of separate electronic systems for competency assessment, course grading, and verification of student clinical encounters

Student Perception: Where Is Competency Content?

Student Perception: Where Is Competency Content?

Competency Content: Self Selected Examples

Competency Content: Self Selected Examples

Faculty Must Engage in Discussion About Best Practices What are tensions between portfolio uses,

Faculty Must Engage in Discussion About Best Practices What are tensions between portfolio uses, and how can they be resolved? – Metaphor: Narrative medicine - “our stories” vs Traditional medicine - “just the facts” – Locus of control: Curriculum-Driven vs Learner-Defined Content – Purpose: accountability vs wonder

Potential Portfolio Applications • Course Level Student Assessment • Clerkship Level Student Assessment •

Potential Portfolio Applications • Course Level Student Assessment • Clerkship Level Student Assessment • Competency Level Student Assessment • Program Evaluation • Individual Residency Advisement & Dean’s Letter Preparation • Liason Committee on Medical Education (LCME) Accreditation Documentation

Graduating Competent Physicians • Assurances – licensure examinations, traditional checkpoints for specific accountability (USMLE:

Graduating Competent Physicians • Assurances – licensure examinations, traditional checkpoints for specific accountability (USMLE: Step 1 before Year 3, Step 2 prior to graduation) • Certifications – BLS, ACLS, CME, Match to Residency Program & USMLE Step 3 • Balance – Accomplishment at Three Levels of Medical Competency; Integration across Disciplines • Reflection Connections – an additional demonstration of the compassionate, caring dimension of medical training

Concluding Observations • Evolving Central Assessment Goal • Balancing Benefits and Burdens • Compliance

Concluding Observations • Evolving Central Assessment Goal • Balancing Benefits and Burdens • Compliance Issues • Confidentiality and Control Issues • Relating Academic and Professional Growth to Life as a Whole Graduate Center for Gerontology

For further information contact: • Mary T. Johnson, Ph. D. • Associate Professor of

For further information contact: • Mary T. Johnson, Ph. D. • Associate Professor of Microbiology and Immunology John F. Watkins, Ph. D. Associate Professor Graduate Center for Gerontology Indiana University School of Medicine University of Kentucky Statewide Competency Director for Problem Solving geg [email protected] edu [email protected] edu • Graham D. Rowles, Ph. D. • Heidi H. Ewen, Ph. D. Director and Professor Post-doctoral Scholar Graduate Center for Gerontology University of Kentucky growl [email protected] edu heidi. [email protected] edu