Training Cost Analysts A Cohesive Pedagogical Framework for

Training Cost Analysts A Cohesive Pedagogical Framework for Success Kammy Mann Herren Associates Washington Capital ICEAA Chapter Luncheon July 27, 2016

The purpose of this effort was to identify: the current state of the Cost Analyst training program, the best practices in the industry, and training gaps. Training Available Best Practices (Industry and Research) Understand Required Skills Gaps in Training Assess Recommendation Combine information to produce a proposed pedagogical framework & recommendations 2 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

How do you Learn Cost Analysis? Field of Education Percent Business/Management 20% Engineering 28% Math/Statistics 11% Economics 11% Accounting/Finance 15% Operations Research 5% Computer Science/Information Systems 1% Physical Sciences 2% Other 7% ICEAA Member Survey, 2013 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION 3

Current State/Challenges • Cost Analysis is not an easy profession! • Training employees cost time, money and materials • There are currently no common or consistent training requirements for cost analysts • Mistrust between the “cost shop” and the rest of the organization – “Weather Man” analogy 4 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Importance of Training • Training employees costs time, money, and materials • Research has shown that not training employees leads to discontentment and turn over • Time and Money can be lost when untrained workers make mistakes, rework occurs, and inadequate products are delivered 5 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Challenges Cost Estimators Typically Face GAO Cost Estimating and Assessment Guide, 2009 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION 6

Academic Literature Review • Research was taken from the field of Engineering Education – Students learn by doing design/hands-on activities vs. traditional lectures (“Sage on the Stage”) – Problem/Project based learning shows higher knowledge retention rates – Constructivism – knowledge is learned through experience – Continuous feedback and assessment, tailor to individual student’s learning style – Cooperative/cross-functional team learning is integral to prepare students for the work force 7 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Pedagogical Frameworks A Pedagogical Framework is a structured method for teaching that guides the instructor in a systematic way to instruct the students. Step 1: Identify Desired Results Step 2: Determine Acceptable Evidence Step 3: Plan learning experience and instruction Bloom’s Taxonomy Understanding by Design 8 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

An understanding of what drives effective training is knowledge of positions, anticipated schedule, and data availability Essential Question: Does current training meet training needs? Survey Feedback Training Analysis Best Practices Analysis What we did: Conducted a Survey of ICEAA members What we did: Reviewed CEBo. K, DAU, CCEA and contractor training programs What we did: Studied policies and documents to understand best practices Purpose: • Provide quantitative story on training drivers Purpose: • Identify available training and certification programs • Tie the qualitative story to quantifiable events Purpose: • Determine framework and best practices • Determine if policy roadmap is consistent with survey feedback Themes: • Continuing education lacking • Training to become future leaders/mentors lacking Themes: • Framework provides applicable building blocks • Few opportunities for continuing education Themes: • On the job mentoring most profitable • Computer Science/math and Statistics desired further training • Recurring drivers stem from lack of on the job mentoring 9 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Initial Outputs Phase 1 Data Collection Survey Ø Ø 73 ICEAA members Top Issues Pain Points Roles/Responsibilities Interviews Ø 15 cost analyst from 05 C and Herren Ø Top Issues Ø Pain Points Round Two Documentation Ø GAO Cost Guide Ø CEBo. K Assessment Feedback 10 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Disciplines and Concepts in Cost Analysis GAO Cost Estimating and Assessment Guide, 2009, p. 51 11 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

An questionnaire was developed as a guide to drive discussion about training for Washington Area Chapter ICEAA members Key Questions Most Profitable Training? Gaps in Training? Applicability of DAU? Areas for Improvement? Job Satisfaction? 12 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Survey Feedback: The goal of the survey process was to be able to tie anecdotal evidence to specific observed training gaps Preferences for 20+ years Output § Formal classrooms have the least effect on overall learning § Mentorship has the greatest effect on overall learning § There is a desire for a collaborative/team environment between departments and organizations Preferences for 0 -5 years Years of Experience: 0 -5 the job mentoring On. Onthe job mentoring 7% 7% Years of Experience: 20+ On the jobjob mentoring On the mentoring 7% Certifications 11% 39% 14% In-house training (peer reviews, In-house training working groups, brown bags, etc. ) Educational background 29% DAU Courses DAU courses Area for Improvement Greatest Area for Improvement 2% 2% Collaboration and understanding between cross-functional departments/agencies Educational background 14% Certifications 71% Certifications Collaboration Mentorship and on-the-job training 30% 60% Mentorship In-house training (peer reviews, In-house training working groups, brown bags, etc. ) 13 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Common Themes - Interviews Pressures that Limit Progress ØAbsence of continuing education § Sources limited exclusively to ICEAA ØAbsence of leadership track § Need future and leaders mentors ØLack of collaboration and understanding § Cross-functional departments and agencies ØLack of on the job mentoring 14 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Common Themes - Interviews Advice for Training Ø Two-week intensive course § Create road-map with accompanied training Ø Increase job-specific training ØContinued training after basic is completed Ø Offer mentorship programs 15 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Proposed Framework Understanding by Design Find out Student’s Learning Style Step 1: Identify Desired Results Step 2: Determine Acceptable Evidence Acquire (Classroom) Meaning (Constructivism) Transfer (Mentorship) Apprentice Discovery Step 3: Plan learning experience and instruction Methods of Learning Cost Analysis Inductive Incidental Reflection & Assessment Deductive Bloom’s Taxonomy 16 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Recommendations • Establish a systematic methodology for training • Leverage current trainings such as DAU, GAO, ICEAA CEBo. K. However, don’t rely on “sage on a stage” lecturing! • Consider utilizing a hands-on design/project based approach to training (case studies, team projects, etc. ) • Mentoring is very important! • Assessment and continuous improvement is key 17 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Proposed Training Schedule for New Analysts 1. Assign new analyst a mentor to guide them through the learning process 2. Have analysts take a self assessment of learning style and discuss how they can get the most out of the training process 3. 1 -2 week intensive classroom setting to acquire basic cost analysis knowledge, applying principles of Bloom’s Taxonomy 4. At the end of the classroom session, to create meaning, have the analysts participate in a case study/design project in a team environment 5. Throughout the first 2 years of working in the field have the analyst meet frequently with their mentor and cohort for self reflection and additional hands-on training 6. Have the analysts act as mentors and teachers for the new group of analysts who enter the program in subsequent years 18 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Future Work/Research • Test the proposed pedagogical framework and document in a case study • Consider other fields of educational research • Workforce development research and best practices • Let’s start a conversation about training our next generation of analysts! 19 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Questions? 20 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Contact Info Kammy Mann: Kammy. mann@jlha. com 21 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Backup 22 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Citations List • • • • • • Adams, R. S. , & Felder, R. M. (2008). Reframing Professional Development: A Systems Approach to Preparing Engineering Educators to Education Tomorrow’s Engineer. Journal of Engineering Education, 239 -240. Berry, J. , & Asiello, D. (2011). Department of Defense Sustainability Planning: The View from the Second Year. New Orleans, LA: Noblis. Bloom, B. S. ; Engelhart, M. D. ; Furst, E. J. ; Hill, W. H. ; Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David Mc. Kay Company. Bloom, B. S. & Krathwohl, D. R. Taxonomy of Educational Objectives. Handbook 1: Cognitive Domain. Addison-Wesley, New York, 1984. Bordogna, J. "Engineering—The Integrative Profession, " NSF Directions, May/June, Vol. 5, No. 2, (1992). Brereton, M. F. , J. Greeno, J. Lewis, C. Linde and L. Leifer, "An Exploration of Engineering Learning, " Proceedings of the 5 th International Conference on Design Theory and Methodology. Held: Albuquerque, NM, USA. American Society of Mechanical Engineers, Design Engineering Division (Publication) DE v 53, Published by ASME, New York, NY, USA. p. 195 -206, (1993). Cost Estimating Body of Knowledge (CEBo. K) (1. 2 version). (2013). Vienna, VA: International Cost Estimating and Analysis Association. Estimated Impacts of Sequestration-Level Funding. (2014). Washington, D. C. : Department of Defense. <http: //www. defense. gov/pubs/2014_Estimated_Impacts_of_Sequestration-Level_Funding_April. pdf>. Felder, R. M. , Woods, D. R. , Stice, J. E. , & Rugarcia, A. (2000). The Future of Engineering Education, Teaching Methods that Work. Chemical Engineering Education, 34(1), 26 -39. Forehand, M. (2010, July 6). Bloom's Taxonomy - Emerging Perspectives on Learning, Teaching and Technology. Retrieved March 23, 2016, from http: //www. roe 11. k 12. il. us/GES%20 Stuff/Day%204/Process/Blooms/Mary%20 Forehand%20 discussion. Bloom%27 s%20 Taxonomy. pdf Hein. G. E. (1991). Constructivist Learning Theory. Proceedings from: International Committee of Museum Educators Conference, Jerusalem, Israel. ICEAA. (2013). Survey of Members. Vienna, VA: International Cost Estimating and Analysis Association. Jonassen, D. H. & Ronrer-Murphy, L. (1999). Activity Theory as a Framework for Designing Constructivist Learning Environments. ETR&D, 47(1), 61 -79. Leifer, L. remarks made at the International Workshop on Project-Based-Learning, Stanford University, Aug. 7 -10, (1995). Lipman, V. (2013). “Why Employee Development is Important, Neglect and Can Cost you Talent. ” Forbes. Retrieved from: http: //www. forbes. com/sites/victorlipman/2013/01/29/why-development-planning-is-important-neglected-and-can-cost-youngtalent/#60 dd 33 e 62 ce 0 Mills, J. E. & Treagust, D. F. (2003). Engineering Education – Is Problem-Based or Project-Based Learning the Answer? Austrailasian Journal of Engineering Education, 2 -16. National Science Foundation (NSF). (1995). Restructuring Engineering Education: A Focus on Change. Report of an NSF Workshop on Engineering Education, June 6 -9, 1994, Arlington, VA. Perkins, D. N. "What Constructivism Demands of the Learner, " Constructivism and the Technology of Instruction: a conversation, LEA Associates, Hillsdale, NY, p. 161 -165, (1992). Richey, Karen. (2015) GAO Cost Estimating & Scheduling, Best Practices and Audit Findings. Washington, D. C. Sanders, M. (2009). STEMmania. The Technology Teacher, 20 -28. Sewell, M. & Marczak, M. (2016). “Using Cost Analysis in Evaluation”, The University of Arizona, retrieved from 23 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION

Citations List (Part 2) • • • • Sheppard, S. & Jenison, R. “Freshman Engineering Design Experiences: an Organizational Framework” International Journal of Engineering Education. 1996. Soloman, B. A. & Felder, R. M. (2016). Index of Learning Styles Questionnaire. NC State University. https: //www. engr. ncsu. edu/learningstyles/ilsweb. html Sonwalkar N. ‘‘Changing the Interface of Education with Revolutionary Learning Technologies: An Effective Guide for Infusing Technology Enabled Education for Universities and Corporations. ’’ Presentation at i. Universe, New York, September 2004. Sonwalkar N. The sharp edge of the cube: pedagogically driven instructional design for online education. Syllabus December 2001, pp. 12– 16(www. campus-technology. com/article. asp? id = 5858); Sonwalkar N. Changing the Interface of Education with Revolutionary Learning Technologies. Syllabus November 2001, pp. 10– 13, (http: //www. campus-technology. com/article. asp? id = 5663). Sonwalker, N. (2013). The First Adaptive MOOC: A Case Study on Pedagogy Framework and Scalable Cloud architecture – Part 1. MOOCs Forum, 22 -29. Spurlin, J. E. , Rajala, S. A. , & Lavelle, J. P. (2008). Designing better engineering education through assessment: A practical resource for faculty and department chairs on using assessment and ABET criteria to improve student learning. Sterling, VA: Stylus Pub. Teslow, J. L. , L. E. Carlson and R. L. Miller, "Constructivism in Colorado: Applications of Recent Trends in Cognitive Science, " ASEE Annual Conference Proceedings, Session 1602, (1994). The Compass. (2016). Understanding by Design: A Framework to Support Effective Teaching and Learning. Newton-Conover’s Online Newsletter for Educators. Retrieved from: http: //nccscurriculum. org/2014/01/22/understanding-by-design-a-framework-tosupport-effective-teaching-and-learning/ Weimer, M. (2002). Learner-centered teaching: Five key changes to practice. San Francisco: Jossey-Bass. Wiggins, G. , & Mc. Tighe, J. (2005). Understanding by Design (Expanded 2 nd ed. ). Alexandria, Virginia: Association for Supervision and Curriculum Development. Woods, D. R. Problem-based Learning: How to Gain the Most from PBL. Woods Publishing, Waterdown, 1994. Distributed by Mc. Master University Bookstore, Hamilton, ON. Wulf, W. (1998). The Urgency of Engineering Education Reform. Proceedings from: Realizing the New Paradigm for Engineering Education, Baltimore, MD. VARK Questionnaire. (2016). Retrieved on 29 March, 2016 from: https: //www. engr. ncsu. edu/learningstyles/ilsweb. html U. S Government Accountability Office. (2009) GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. Washington, D. C. : Applied Research and Methods. 24 STRATEGY | MEASUREMENT | ANALYSIS | REDESIGN | TRANSFORMATION