1 Identifying A Psychometric Profile For Vulnerability Assessment

1 Identifying A Psychometric Profile For Vulnerability Assessment Professionals: Talent Identification To Support Career Assessment CLEAR Cyber Leaders Conference Dakota State University, University of South Dakota School of Law and the Better Business Bureau Martha Crosby, Ph. D. University of Hawaii – Manoa Curtis Ikehara, Ph. D. Applied Computer Electronics Custom Design Gregory P. M. Neidert, Ph. D. Professor Emeritus, Arizona State University Morgan A. Zantua, M. A. University of Washington CIAC, Director Professional Workforce Development 13 November 2019 10: 30 -11: 50 am Sioux Falls, SD

2 ABSTRACT Paper 2: Title: Identifying a Psychometric Profile for Vulnerability Assessment Professionals, Talent Identification to Support Career Assessment An inter-collegiate research team completed initial research analysis of 119 professional cybersecurity workers from government and industry to identify talent profiles aligned with four roles within the Protect and Defend (PD) NICE Workforce Framework: Cybersecurity Defense Analyst, Cybersecurity Defense Infrastructure Responder, Cybersecurity Incident Responder and Cybersecurity Vulnerability Assessment Analyst. Anonymized data collected from multiple organizations with performance assessments to build statistically validated psychometric profiles of high potential PD cybersecurity candidates. Data collection was obtained using the World of Work Inventory (WOWI) a multidimensional on-line career assessment which measures six aptitude and achievement dimensions in the Career Training Potentials, twelve work-style preferences in the Job Satisfaction Indicators and task-relevant preferences related to seventeen career families in the Career Interest Activities. Anonymized, aggregated ranked data described profiles of existing high performing candidates working in the field. Utilization of a methodology to identify cybersecurity talent at different phases of an individual’s career life cycle supports recruitment of high potential talent from diverse backgrounds to increase the numbers of candidates entering cybersecurity education and training programs.

3 Demand For Cybersecurity Professionals https: //www. cyberseek. org/heatmap. html

4 Nice Cybersecurity Workforce Framework (NCWF)

5 7 Specialty Areas Operate And Maintain Securely Provision Analyze 35 Work Roles Oversee And Govern Collect and Operate Investigate Protect And Defend 56 Job Titles

6 Professionalization Of Cybersecurity Benchmark against Medical Profession – 100 years in the making Differentiation by specialties Hospital Surgical Residency Program – 20% attrition of surgical residents Cost of attrition Patient ramifications Implications to cybersecurity career profiling

Surgical Resident Study https: //www. wowi. com/about/Psych_Profile_of_Surgeons_and_Surgical_Residents. pdf 7

8 Resident Rating/Ranking Grid

9 Person-job Fit Predicting Good Person-job Fit Content Good Person-job Fit Performance Process

10 Person-job Fit Multi-dimensional WOWI is the first empirically-based, fully integrated, multi-dimensional career assessment. It is three tests integrated into one comprehensive assessment. Content Process Content 6 12 17 Career Training Potentials (CTPs) Job Satisfaction Indicators (JSIs) Career Interests Activities (CIAs)

WOWI Career Interest Activities 17 Task-relevant Interest Measures Table 5 CIA Subscale Characteristics of Career Interest Activities Subscale CIA Subscale Helping others by providing specialized information and services. Includes occupations in medicine, law, education, religion, library work, counseling, the social sciences, etc. The Sciences Applying research methods and statistics to solve theoretical and applied problems in the physical, life and social sciences. Using the principles of engineering and physics for the design of machines, materials, instruments, structures, processes and services. The Sciences Engineering and Related Using the principles of engineering and physics for the design of machines, materials, instruments, structures, processes and services. Business Relations Collecting, analyzing, advising, and making decision based on a variety of data sources. Includes occupations in accounting, insurance, finance, purchasing personnel, human resources, etc. Managerial Controlling, directing, and organizing the work of others in a wide range of settings. Requires knowledge of business principles, business operations, and human behavior. The Arts Integrating personal expression and art concepts, techniques, and processes to develop works which elicit an emotional or esthetic response. Includes acting, sculpting, painting, etc. Clerical Compiling, recording, communicating, computing, copying, and otherwise organizing information for others. Interacting with and influencing others in favor of certain products, services, or ideas. Sales Interacting with and influencing others in favor of certain products, services, or ideas. Service Helping individuals with their personal wants and needs. Includes occupations in cosmetology, day care, recreation, hospitality, food-and-beverage preparation and service, etc. Processing Machine Work Bench Work Structural Work Mechanical Work Graphic Arts Mining Working with tools, equipment, materials, products, structures, structural parts, or operating machinery. Includes mechanical, electrical, masonry, and tool and die work, carpentry, plumbing, mining, drafting, factory work, etc. 11

WOWI Job Satisfaction Indicators 12 Work Styles/Job-related Temperaments Table 4 Characteristics of Job Satisfaction Indicators (JSI) subsets Versatility + - Likes variety and change; several things going on at once. Likes to concentrate on 1 task at a time; a linear approach to work. Adaptable to Repetitive Work + - Enjoys predictability; activities in a set order. Likes spontaneity; uncomfortable with tasks that repeat in a short time frame. Adaptable to Performing Under Specific Instructions + - Adjusts to being monitored; likes to follow set policies, procedures, recipes, instructions, blueprints, etc. Likes general direction/instruction; uncomfortable with close supervision Dominant +/+ +/-/+ -/- Likes to lead; be responsible for decisions; is self-directed. Prefers to be in a support role; dislikes being responsible for others. Gregarious Isolative + - Likes to work with others, but also likes to be alone to focus on work and get it done. Likes people and does not like being alone; likes being a team player. Dislikes spontaneous interruption and likes being in control of when others see them. Not motivated by a traditional work environment. Influencing + - Likes to sell products, services or ideas; enjoys persuading, impressing, and motivating others. Likes to be in situations where there is no conflict; is uncomfortable having to persuade, motivate, or sell to others. Self-controlled + - Likes to work under pressure, deadlines, and in crisis situations; tends to procrastinate. Prefers calm atmosphere; dislikes constantly working under pressure, against tight deadlines, and meeting demanding quotas. Values + - Likes to make value judgments; has an intuitive approach to making decisions and solving problems. Dislikes making decisions using intuition or hunches. Objective + - Likes to problem-solve in a rational way; relies on facts and data. Does not like to rely on facts to make decisions. Subjective + - Enjoys being self expressive; likes to be creative; is uncomfortable conforming to others' standards of style. Does not have a strong drive to be self-expressive or creative. Rigorous + - Has integrity of workmanship; tendency toward perfectionism; likes to be detail-oriented. No need for exacting results, likes the big picture, the bottom line; dislikes focusing on details 12

WOWI Career Training Potentials 6 Aptitude/Achievement Levels Table 3 Measurement of Career Training Potential. S (CTP) Verbal Ability to read and comprehend words. Predictor along with the numerical score of ability to do school work. Numerical Ability to manipulate the language of numbers, indicates understanding, and skill in performing basic mathematical functions. Abstraction Potential it the area of figurer out problems through a logical procedure_ Ability to solve problems by means of size, position, shape or quantity without assistance from words or numbers Spatial-Form Ability to visualize and think in three dimensions. Ability to formulate a finished product from seeing the visual plans. Potential to sense forms and position of things in space. Mechanical/ Electrical Potential to construct, operate and repair machinery and understand Electrical physical forces. Also includes the influence of prior knowledge and understanding of electricity, electronics and electromagnetic. Organizing Ability to organize information, including the ability to perform sequential reasoning using letters, abbreviations, words and numbers. 13

WOWI Psychometric Profile Report - Sample 14

WOWI Psychometric Profile Report for Attending Surgeon 15

O*Net On. Line https: //www. onetonline. org/link/details/15 -1122. 00 16

Nice Cybersecurity Workforce Framework (NCWF) … not so NICE 17

18 Industries Approached & Engaged Military Government Federal State City Transportation Telecommunications Retail

19 Parameters of Cybersecurity Study Data Collection Participants 60+ participants [planned on getting 200] Teams Selected Job Role within Specialty Materials World of Work Inventory (WOWI) Online Assessment All 3 scales were used Leadership Ratings/Rankings Institutional Review Board (IRB) standards

20 Data Collection Procedures – Planned Engagement across industries & organizations Time allotment Cultural compatibility with research model Engagement by leadership Ranking of employees Research Questions Differences between Hi, Medium & Low Performers Differences between government & industries & organizations

21 Data Collection Challenges Procedures – Reality Variations in Team size Job description Limitation in getting only the High Performers Other points: Cultural acceptance by organizations Union issues

22 Results – What We Can Say: Career Training Potential Data Organization 1 n=55 Organization 2 n=17 Organization 3 n=15 Organization 4 n=13 Organization 5 n=12 Organization 6 n=4 Organization 7 n=3 Verbal 47. 05 Verbal 47. 76 Verbal 47. 07 Verbal 43. 69 Verbal 46. 17 Verbal 47. 25 Verbal 45. 33 Numerical 41. 67 Numerical 41. 88 Abstract 40. 00 Numerical 43. 08 Numerical 40. 67 Numerical 44. 00 Mech. Elec 44. 33 Abstract 40. 44 Abstract 41. 65 Numerical 36. 53 Abstract 42. 46 Mec/Elec 36. 67 Abstract 44. 00 Numerical 42. 67 Mech/Elec 37. 89 Mec/Elec 37. 65 Mech. Elec 35. 73 Manage 40. 38 Abstract 36. 33 Engin 43. 75 Self. Cnt 42. 00 Spatial 33. 31 Organi 32. 47 Object 32. 67 Mech. Elec 37. 54 Spatial 34. 00 Rigor 42. 00 Abstract 40. 00 Organi 32. 80 Spatial 31. 29 Organi 32. 00 Organi 36. 0 Organi 31. 33 41. 50 Mech. Elec 40. 00 Verbal

Results – What We Can Say: High Scoring Areas Across Organizations Variable Label WOWI Scale # Verbal 7 Numerical 7 Abstract 7 Mech/Elec (aptitude) 6 Organi 5 Spatial 3 Mech/Elec (interest) 2 Engineering 1 Managerial 1 Objective 1 Rigorous 1 Self-Controlled 1 23

24 Small sample sizes within and across organizations and types Did not allow high range scale enough statistical power to Results – What We Can Say Limitations emerge as significant Did not allow us to detect differences between industries and organizations Only restricted range of high performers Did not allow us to find differences between high, medium and low performing cybersecurity professionals Wide range of job descriptions represented exacerbated problems with small N

25 Lessons Learned Identification of research participants Team based model – application to other job roles Transferring methodology to other cyber security career specialty areas

26 Applications & Current Work Office of Personnel Management [OPM] challenge Incumbent worker training Ongoing work with cyber security testing and training organization

NICE Cybersecurity Workforce Framework (NCWF) … Quantitatively not so Nice 27

28 INITIAL A PRIORI WOWI PROFILE OF CYBER DEFENSE ANALYST DEVELOPED FOR CLIENT Initial a Priori WOWI Profile Developed by Gregory P. M. Neidert, Ph. D. & Morgan Zantua, M. A. Developed 7 -10 May 2019 CIAs CTPs Scale Verbal Numerical Abstractions Spatial-Form Mechanical. Electrical Organizing Skill Required or Optional R R R N/A R R Min Score 44 44 40 0 36 40 Scale Required or Optional Min Score Max Score Public Service O -14 56 The Sciences R -21 56 56 Engineering R -21 56 56 Business Relations N/A 56 Managerial N/A The Arts N/A Media Design N/A Office & Admin Support N/A Sales N/A Service N/A Primary Outdoor Max Score 56 56 56 Best-Liked Subjects JSIs Scale Required or Optional Min Score Max Score Note Requires 1 of 3 Mathematics, Science, Space, Technology Engineering, Electronics, Drafting, Surveying Versatile R 7 42 Adapt Repetitive Wk O -21 35 Adapt Specific Inst R -28 35 Occupational Areas Dominant R -21 35 Requires 1 of 3 Gregarious O -28 28 G<I Isolative R -21 42 I>G N/A Influencing O -28 35 Processing N/A Self-Controlled R 28 49 Machine Work N/A Valuative O -28 28 V<O Bench Work N/A Structural Work N/A Objective R 7 56 O>V Subjective N/A 35 56 Mechanical. Electrical Extraction Work R N/A -28 56 Rigorous R Mechanical, Machines, Vehicles, Electricity, Assembly Mathematics Science (General, Biology, Chemistry or Physics) Technical Education

WOWI Psychometric Profile Report - “CYBER MIN” Sample 29

WOWI Psychometric Profile Report - “CYBER MAX” Sample 30

WOWI Psychometric Profile Report – “Real” Sample 31

32 Next Steps - Opportunities Application to Oversee and Govern Continue systematic research Replication to legal profession

33 Thank You for Your Time and Attention Any questions?