Design Knowledge Coordination Enhancing Novice Aerospace Engineers Design

Design Knowledge Coordination: Enhancing Novice Aerospace Engineers’ Design Skills Through Coordinated Decision-Making Scottie-Beth Fleming May 24, 2016 Committee Members: Dr. Amy Pritchett (Chair), Dr. Brian German, Dr. Neil Weston, Dr. Jennifer Turns, Dr. Ute Fischer, Dr. Wendy Newstetter This work was sponsored by the NSF, grant number DGE 0644493 1

Background and Research Questions Framework Development Application of Framework to Authoritative Example of Aircraft Design Case Study Examining Indicators of Design Knowledge Coordination within Student Design Processes Characterization of Educational Interventions that Support Design Knowledge Coordination Conclusions and Future Work 22

Aerospace Engineering Design Coordination: 1) connects two things together and makes them part of a larger system of dependencies, 2) it does so in specific ways, and 3) it also holds them apart and keeps them distinct Gerson, 2008 Images from http: //wonderfulengineering. com/ http: //www. geaviation. com/ http: //www. airbus. com/ http: //www. boeing. com/

Aerospace Engineering Design Industry has expressed a need for engineering graduates to use effective approaches to system design, integration, and synthesis Dutson et al. , 1997 Images from http: //wonderfulengineering. com/ http: //www. geaviation. com/ http: //www. airbus. com/ http: //www. boeing. com/

Aerospace Engineering Curriculum Aerodynamics Materials Propulsion Capstone Design Grinter, 1994; Dym et al. , 2005 Performance Flight Dynamics Students must interpret how these disciplines contribute to and integrate within the design of an aerospace system

Aerospace Engineering Design Novice aerospace engineers are often unable to + integrate multidisciplinary design considerations + translate preferences, constraints, and decisions to others also engaging in the design process Ahmed et al, 2004; Daly et al. , 2001; Fleming & Coso, 2014; Gertler, 2014; Griffin, 2005; NTSB, 2013; Oakley et al, 2004; Richey, 2005 Images from http: //wonderfulengineering. com/ http: //www. geaviation. com/ http: //www. airbus. com/ http: //www. boeing. com/

Aerospace Engineering Design Novice aerospace engineers are often unable to + integrate multidisciplinary design considerations + translate preferences, constraints, and decisions to others also engaging in the design process Design Knowledge Coordination a structured approach to integrating design considerations across the different disciplines in engineering design through use of goals, tasks, metrics, and decisions Ahmed et al, 2004; Daly et al. , 2001; Fleming & Coso, 2014; Gertler, 2014; Griffin, 2005; NTSB, 2013; Oakley et al, 2004; Richey, 2005 Images from http: //wonderfulengineering. com/ http: //www. geaviation. com/ http: //www. airbus. com/ http: //www. boeing. com/

Research Questions Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Research Question (2) How do novice aerospace engineers coordinate and integrate knowledge about a design? Research Question (3) What educational interventions can better support novice aerospace engineers’ design knowledge coordination? 8

Research Questions Contribution (1) Design Knowledge Framework of Design Knowledge Coordination Contribution (2) Authoritative description of design knowledge coordination in AE design Contribution (3) Observable indicators of design knowledge coordination in student design processes Contribution (4 a) Learning goals for educational resources Contribution (4 b) Examples of concepts that support student development of design knowledge coordination 9

Design Knowledge Coordination Framework Development 10 10

Creating the Design Knowledge Coordination Framework Frame the Context: Engineering Design Literature Case Studies Depicting Decision-Making in Aerospace Engineering Design Strategic Literature Search: Complex Systems Decision-Making and Engineering Design Reasoning Observable Patterns: Indicators of Design Knowledge Coordination Boyer, 1990; Fleming & Pritchett, 2015

Frame the Context: Constructs of Engineering Design Decision-Making Construct Description Literature Area Goals Define the design problem’s objectives within the given context and constraints Direct outcomes of the design process Goal Alignment Tasks Dictate the direction and future content of overall work. Break the process into actionable work items Engineering Design Context Metrics Represent information about the design that is available to or needed by the designer. Pass information between tasks Shared Knowledge, Information Sharing Decision Outcomes of a task where a design has to specify new or update previously decided knowledge about a design Outcome of Process Pahl, Beitz, Feldhudsen & Grote, 2007; Nicolai & Carichner, 2010 ; Cross, 2006; DAU, 2001; Steward, 1981; Goel & Pirolli, 1992; Browning, 2001; Prasad, Morenc & Rangan, 1993; Chattopadhyay et al. , 2011; Chen, 2005; Clarkson & Hamilton, 2000; Marks, Mathieu & Zaccaro, 2001; Mathieu, Marks & Zaccaro, 2001; Mesmer-Magnus & Dechurch, 2009; Richey, 2005; Gertler, 2014; Lorell et al. , 2013; Griffin, Kinnu & Colombi, 2005; Clark & Schaefer, 1989; Cramton, 2001; Keysar, Barr, Balin & Paek, 1998; Cannon-Bowers & Salas, 2001; Mathieu, Heffner, Goodwin, Salas & Cannon-Bowers, 2000; Baird, Circus, Moore & Jagodzinski, 2000; Fussel & Krauss, 1992; Mathieu, Maynard, Rapp & Gilson, 2008; Bunderson & Sutcliff, 2002; Jehn & Shah, 1997; Stasser & Titus, 1985; Miranda & Saunders, 2003; van Ginkel & van Knippenberg, 2009; Daly, Augustine, Davis, Covert & Gray, 2001; Hinz, Tindale & Vollrath, 1997; Aurisicchio, Bracewell & Wallace 2012

Strategic Literature Search Reviewed literature on approaches to design and decision-making that decomposes systems into distinct components while also enabling interdependencies between these components. + + Articulation Work Coordination Theory Distributed Cognition Knowledge Management Processes Strauss, 1988; Corbin & Strauss, 1993; Gerson & Star, 1986; Malone & Crowston, 1990; Malone & Crowston, 1994; Gerson, 2008; Strauss, 1985; Hollan, Hutchins & Kirsh, 2000; Perry, 1998; Hutchins, 2001; Hutchins, 1995; Liu, Nersessian & Stasko, 2007; Mc. Mahon, Low & Culley, 2004; Kirsh, 2006; Hansen, Nohria & Tierney, 1999 13

Strategic Literature Search: Summary of Findings Construct of Engineering Design Goals Tasks Metrics Decisions Coordination Mechanism Identify tradeoffs between disciplineoriented design goals Recognize the links between tasks and subtasks Use cross-disciplinary metrics to support iteration across the design and consistently update information about the design Maintain consistent values for metrics across the different disciplines Incorporate a variety of metrics from other disciplines to justify decisions Describes the overall impact of a particular decision Literature Area Coordination Theory Articulation Work, Coordination Theory Knowledge Management Distributed Cognition Articulation Work Strauss, 1988; Corbin & Strauss, 1993; Gerson & Star, 1986; Malone & Crowston, 1990; Malone & Crowston, 1994; Gerson, 2008; Strauss, 1985; Hollan, Hutchins & Kirsh, 2000; Perry, 1998; Hutchins, 2001; Hutchins, 1995; Liu, Nersessian & Stasko, 2007; Mc. Mahon, Low & Culley, 2004; Kirsh, 2006; Hansen, Nohria & Tierney, 1999 14

Observable Patterns Mapped the findings from the strategic literature search to the engineering design context Design Knowledge Coordination Framework Basic Approach Discipline-Oriented Approach Coordinated Approach Goals Task Metrics Decisions 15

Basic Approach to Engineering Design Goal Tasks Metrics Decision Defines a general design goal Defines tasks and subtasks Incorporates an order and hierarchy to the tasks and subtasks Uses metrics to complete tasks and subtasks Justifies decisions through completion of design tasks

Discipline-Oriented Approach to Engineering Design Goal Tasks Metrics Decision Design goals are decomposed into more specific goals for each discipline The tasks and subtasks have an order and hierarchy that align with engineering disciplines Within each discipline, there are critical metrics guiding the decisions and tasks Decisions are justified through disciplineoriented metrics

Coordinated Approach to Design Goal Identifies tradeoffs between discipline-oriented design goals Tasks Recognizes the links between tasks and subtasks Iterates across the tasks to update information about the design Maintains consistent values for metrics across the different disciplines Describes the overall impact of a particular decision Decision Incorporates a variety of metrics from other disciplines to justify decisions Metrics

Design Knowledge Coordination Framework Goals Tasks Metrics Basic Approach Defines a general design goal Defines tasks & subtasks Incorporates an order and hierarchy to the tasks and subtasks Uses metrics to complete tasks and subtasks Discipline-Oriented Approach Design goals are decomposed into more specific goals for each discipline The tasks and subtasks have an order and hierarchy that align with engineering disciplines Within each discipline, there are critical metrics guiding the decisions and tasks Justifies decisions through Decisions are justified through discipline. Decisions completion of design oriented metrics tasks Coordinated Approach Identifies tradeoffs between disciplineoriented design goals Recognizes the links between tasks and subtasks Iterates across the tasks to update information about the design Maintains consistent values for metrics across the different disciplines Describes the overall impact of a particular decision Incorporates a variety of metrics from other disciplines to justify decisions

Design Knowledge Coordination Contribution (1) Framework of Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Contribution (2) Authoritative description of design knowledge coordination in AE design 2020

Authoritative Text of Aircraft Design + Three Primary Texts § Roskam (1990) § Raymer (2006) § Nicolai & Carichner (2010) + Focus on the Conceptual Design Process § § § Raymer’s Design Wheel Overall (general) shape, size, weight, performance of airplane configuration Will it work? What does it look like? What requirements drive the design? What tradeoffs should be considered? What should it weigh and cost? Roskam’s General Conceptual Design Process Coso, 2014; Design Course Syllabi; Roskam, 1990; Raymer, 2006; Nicolai & Carichner, 2010

Design Goals Minimize weight and cost Maximize performance capabilities Task 1 Define Rqmts and Outline Mission Task 2 Perform 1 st Estimation of Weight Task 3 Determine Critical Performance Parameters Task 4 Determine Config and Layout of Aircraft Task 5 Improve Aircraft Weight Estimation Task 6 Conduct a Performance Analysis Task 7 Optimize the Design Mission Requirements Takeoff Weight, Empty Weight, Fuel Weight Coefficient of Drag, Wing Area #Engines, Type of Landing Gear, High-Lift Devices Empty Weight, Structural Arrangement Control System Augmentation Technology, Updated Weight Sub. Task 3. 1 Estimate Drag Polar Max Lift Coefficient Zero-Lift Drag Sub. Task 3. 2 Prepare Constraint Diagram and Select Optimal Wing Loading Wing Area Metrics: Coefficient of Drag, Coefficient of Lift, Aspect Ratio, Wing Area, Takeoff Weight, Wing Loading, Thrust to Weight Ratio, Approach Speed, …

Design Goals Minimize weight and cost Maximize performance capabilities Task 1 Define Rqmts and Outline Mission Task 2 Perform 1 st Estimation of Weight Task 3 Determine Critical Performance Parameters Task 4 Determine Config and Layout of Aircraft Task 5 Improve Aircraft Weight Estimation Task 6 Conduct a Performance Analysis Task 7 Optimize the Design Mission Requirements Takeoff Weight, Empty Weight, Fuel Weight Coefficient of Drag, Wing Area #Engines, Type of Landing Gear, High-Lift Devices Empty Weight, Structural Arrangement Control System Augmentation Technology, Updated Weight Sub. Task 3. 1 Estimate Drag Polar Max Lift Coefficient Zero-Lift Drag Sub. Task 3. 2 Prepare Constraint Diagram and Select Optimal Wing Loading Wing Area Metrics: Coefficient of Drag, Coefficient of Lift, Aspect Ratio, Wing Area, Takeoff Weight, Wing Loading, Thrust to Weight Ratio, Approach Speed, …

Links between tasks and subtasks are maintained through metrics + 44 metrics connect tasks and subtasks between disciplines (i. e. systems and/or functional groups) + Metrics are generally performance -based § Weight, Range, Mach, Altitude

Weight Calculation & Performance Estimation Aerodynamics Structures Stability Cost Analysis Critical metrics Weight (Takeoff, Payload, Fuel, Crew, Empty) Wing Area Velocity (Approach, Cruise, Dive, Maneuver, Max, Stall) #Crew per Aircraft Aspect Ratio of the Wing CD (CD, CD 0) CL (CL, CLAlpha, CLMax) Air Density Load Factor (n, nult) Range (R, R Cruise) Mission/Requirements Outline Abbreviated List of Critical Metrics to Link Tasks X X X X X X X X X X

Subtasks are inherently cross-disciplinary Design Goals Task 1 Define Rqmts and Outline Mission Requirements Task 2 Perform 1 st Estimation of Weight Takeoff Weight, Empty Weight, Fuel Weight Task 4. 1 Airfoil Selection and Planform Shape Airfoil Thickness Task 3 Determine Critical Performance Parameters Task 4 Determine Config and Layout of Aircraft Task 5 Improve Aircraft Weight Estimation Task 6 Conduct a Performance Analysis Coefficient of Drag, Wing Area #Engines, Type of Landing Gear, High-Lift Devices Empty Weight, Structural Arrangement Control System Augmentation Task 4. 2 High-Lift Devices Selection Flap Type & Size Task 4. 3 Fuselage Layout Fuselage Diameter & Length Task 7 Optimize the Design Technology, Updated Weight Task 4. 4 Empennage Design Tail Position Elevator Size Rudder Size 26

Iteration is embedded within the design process & is used to maintain consistent metrics Roskam’s General Conceptual Design Process Roskam, 1990 27

Design Knowledge Coordination Contribution (1) Framework of Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Research Question (2) How do novice aerospace engineers coordinate and integrate knowledge about a design? Contribution (2) Authoritative description of design knowledge coordination in AE design Contribution (3) Observable indicators of design knowledge coordination in student design processes 2828

Research Design + Multiple Case Study § “systematic inquiry into an event or a set of related events which aims to describe and explain the phenomenon of interest” (Bromley, 1990) § Characterized design knowledge coordination using a detailed, qualitative analysis of student presentations within a design course + Data Triangulation § 2 nd data source from semi-structured focus group with same participants § Asked student teams to negotiate a representation of the design process the team used to design their aircraft § Confirmed results of case study Yin, 1994; Bromley, 1990; Miles & Huberman, 1994; Cohen, Manion & Morrison, 2000; Creswell & Miller, 2000; Krueger, 2002

Site and Sample + 2 -semester course § 1 st semester (Fall 2014): Individual Projects § 2 nd Semester (Spring 2015): Team Projects + Projects based on AIAA design competitions § Spring project based on a strategic military transport aircraft + 51 students + 2 Co-Instructors + 3 Teams Selected for Detailed Analysis § Selected using 3 metrics § 2 performance-based metrics (i. e. aggregate student team course grade in the fall, percentage of high and low performers per team) § 1 motivation-based metrics (i. e. AIAA self-selection for competition participation)

Timeline and Data Sources +Focus Group §Audio recording §Field Notes §Artifacts t r a St ct e j ro P f o Weekly Informal Design Reviews Spring 2015 Fall 2014 Week 1 Lectures, Lab sessions, Week 1 5 Individual projects Weekly Informal Design Reviews Lectures, Lab sessions, Team-based design project Final Design Review Focus Group Analysis Focus Group +Midterm Design Review §Observation (Video + Transcript) §Slides +Final Design Review §Observation (Video + Transcript) §Slides Midterm Design Review Case Study End of Spring 2015

Qualitative Coding Scheme: Design Knowledge Coordination + High-level Task “Then we’re starting the weight sizing process. So, we know what we want our plane to do, we know what it’s going to look like more or less, but now we need to know how big it’s going to be. We took about twenty different aircraft and we input them into this Excel sheet by looking at their empty weight and maximum takeoff gross weight, and from this data we were able to outline a trend. This trend would then help us calculate, from estimated takeoff gross weight, our empty weight. Rocco, Team 1 § Weight Sizing Process + Goal and Embedded Decision § Aircraft Size (gets specific using other quotes) + Detailed Task § Estimate Weights Using Regression + Metrics § § Takeoff Gross Weight of Similar Aircraft Empty Weight of Similar Aircraft Takeoff Gross Weight Empty Weight

Interrater Reliability + Examines consistency of applied coding scheme between multiple research coders § Controls for reliability of codes + Conducted 2 interrater reliability analyses + 1 st round § 37% agreement § Differences due to level of detail with application of code (e. g. adding multiple codes for weight) § 91% of missed codes were due to omissions from 2 nd researcher’s analysis + Gave third researcher the authoritative example as a guideline of level of desired detail + 2 nd round § 85% agreement (required >70% for acceptance) § Discussed differences until agreement Moskal & Leydens, 2000; Jonsson & Svingby, 2007; Brown et al. , 2004; Stemler, 2004

Case Study Outline Team 1: + + + Team 2: Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes + + + Team 1 Design Knowledge Coordination Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes Team 2 Design Knowledge Coordination Team 3: + + + Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes Team 3 Design Knowledge Coordination Cross-Case Comparison, and Comparison to Authoritative Text 34

Case Study Outline Team 1: + + + Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes Team 2: Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes Team 3: Qualitative Coding Midterm Design Review Final Design Review Weekly Observer Notes + + For time considerations, only discussing + + + aspects of Team 1 & Team 3 + Team 1 Design Knowledge Coordination Team 2 Design Knowledge Coordination Team 3 Design Knowledge Coordination Cross-Case Comparison, and Comparison to Authoritative Text 35

Framing the Results Goals Tasks Metrics Basic Approach Defines a general design goal Defines tasks & subtasks Incorporates an order and hierarchy to the tasks and subtasks Uses metrics to complete tasks and subtasks Discipline-Oriented Approach Design goals are decomposed into more specific goals for each discipline The tasks and subtasks have an order and hierarchy that align with engineering disciplines Within each discipline, there are critical metrics guiding the decisions and tasks Justifies decisions through Decisions are justified through discipline. Decisions completion of design oriented metrics tasks Coordinated Approach Identifies tradeoffs between disciplineoriented design goals Recognizes the links between tasks and subtasks Iterates across the tasks to update information about the design Maintains consistent values for metrics across the different disciplines Describes the overall impact of a particular decision Incorporates a variety of metrics from other disciplines to justify decisions

Goals Team 1 Defined goals from multiple sources, and decomposed the goals into detailed components Team 3 Defined a generic design goal, but did not perform a detailed requirements analysis or integrate goals into decision-making process Team 1 Requirements Summary (Midterm) Team 3 Requirements Summary (Midterm) 37

Tasks Team 1: Followed a cross-disciplinary, hierarchical task breakdown Team 1 (Midterm) High-Level Task Define Requirements Select Aircraft Configuration Outline Mission Initial Aircraft Sizing Detailed Task Outline Aircraft Requirements Identify Stakeholders and Needs Preliminary Configuration Selection Outline Mission Profile Sensitivity Studies Historical Weight Regression Select technologies First Weight Estimate Class I Drag Polar Class II Drag Polar Constraint Sizing Team 3: Task breakdown lacked clear organization and hierarchy Team 3 (Midterm) Requirements Identify Stakeholders Baseline Aircraft Configuration Selection Mission Profile Specific Range Plots Weight Sizing Technology Selection Drag polar Select Range Constraint Analysis 38

Tasks Team 1: Aligned tasks in the authoritative text Team 3: Tasks did not match the authoritative text Team 1 (Midterm) Authoritative Text Team 3 (Final) High-Level Task Detailed Task High-level task Detailed Task Outline Aircraft Define Requirements Outline RFP Requirements Identify Stakeholders Stakeholder and Needs Identification Define Requirements & Select Aircraft Define Requirements & Overall Configuration Outline Mission Configuration Preliminary Outline Mission Selection Configuration Selection Outline Mission Requirements Outline Mission Profile Outline Mission Sensitivity Studies Figure of Merit Analysis Historical Weight Historical Regression of Configuration Selection Regression Weight Initial Aircraft Sizing Technology Select technologies Technology Selection Identification First Weight Estimate Weight Sizing Initial Aircraft Sizing First Weight Estimate Class I Drag Polar Estimate Drag Polar Sizing Determine Critical Class II Drag Polar Performance-Based Evaluate Aircraft Metrics Constraint Sizing Constraints 39

Tasks Team 1 Incorporated an integrated approach to task completion through metric use to link disciplines together • 23 metrics shared across disciplines/system tasks • 17 are used in the authoritative text • 13 shared between >2 disciplines/systems Critical metrics Weight (Empty, Payload, Takeoff, etc) Altitude Cost (O&S, Maintenance) Mach Area of Wing Cargo Center of Gravity Coefficient of LIft (CL, CLMax, CL Landing) Efficiency of Fuel Burn Range Specific Fuel Consumption Takeoff Field Length Technology Factor # Shared Disciplines 6 5 4 4 3 3 3 Team 3 Did not show a strong connection between disciplines • 17 metrics shared across disciplines/system tasks • 9 are used in the authoritative text • 3 shared between >2 disciplines/systems Critical metrics Weight Altitude Mach # Shared Disciplines 6 3 3 3 3 40

Metrics Team 1 Used a variety of metrics to explain their design process and explained their reasoning for including each metric “As far as landing gear, we also went ahead and selected the tires. We first found the static modes Pn and Pm… we have the weights obtained per strike. The tires selected were from Goodyear… the rated speed 180 miles per hour is well over the speed that we need to approach or take off. So, the tread we selected was the aircraft which is more preferred for…” Damien (Final) Team 3 Referenced metrics as an indicator aircraft performance and capabilities within tasks, but did not fully explain metrics in the context of the design problem “We have three struts at the main landing gear. These are just the weight per nose gear is 6, 923 pounds and the weight per main gear is 44, 577 pounds. ” Alexandra (Final) 41

Metrics Team 1 Used a variety of metrics to explain their design process and explained their reasoning for including each metric Team 3 Referenced metrics as an indicator aircraft performance and capabilities within tasks, but did not fully explain metrics in the context of the design problem References 341 metrics in Midterm and Final 45 are acknowledges on the slide but not stated verbally References 318 metrics in Midterm and Final 119 are acknowledges on the slide but not stated verbally 42

Decisions Team 1 Justified decisions using both quantitative mathematical modeling and qualitative consideration of tradeoffs between design options Team 3 Did not justify decisions or explain their decisions in context of the design problem "Other solutions that we looked at and that we were considering was having a pitch feedback control system for the elevator, but that would then add complexity and cost that we thought would just be unnecessary, since the time to double was so big, " Rocco (Final) “We looked at the wing area that we obtained from our constraint sizing and then from that, we were able to get a wing span based on hanger sizes that we looked at, and then get an aspect ratio. ” Damien (Final) “This is our final spreadsheet we came up with a takeoff weight of 648 000 pounds. The L/D’s, cruise will be on the next slide, Climb L/D and all the other are hovering around 20. Which we thought was pretty good. ” Asa (Midterm) 43

Team 1 Basic Approach Discipline-Oriented Coordinated Approach Design goals are decomposed Identifies tradeoffs between Goals Defines a general design goal into more specific goals for discipline-oriented design each discipline goals Recognizes the links between Defines tasks & subtasks The tasks and subtasks have tasks and subtasks Incorporates an order and hierarchy that Tasks Iterates across the tasks to hierarchy to the tasks and align with engineering update information about the subtasks disciplines design Within each discipline, there Maintains consistent values Uses metrics to complete Metrics are critical metrics guiding for metrics across the tasks and subtasks the decisions and tasks different disciplines Describes the overall impact Decisions are justified of a particular decision Justifies decisions through Decisions through discipline- oriented Incorporates a variety of completion of design tasks metrics from other disciplines to justify decisions

Team 3 Basic Approach Discipline-Oriented Coordinated Approach Design goals are decomposed Identifies tradeoffs between Goals Defines a general design goal into more specific goals for discipline-oriented design each discipline goals Recognizes the links between Defines tasks & subtasks The tasks and subtasks have tasks and subtasks Incorporates an order and hierarchy that Tasks Iterates across the tasks to hierarchy to the tasks and align with engineering update information about the subtasks disciplines design Within each discipline, there Maintains consistent values Uses metrics to complete Metrics are critical metrics guiding for metrics across the tasks and subtasks the decisions and tasks different disciplines Describes the overall impact Decisions are justified of a particular decision Justifies decisions through Decisions through discipline- oriented Incorporates a variety of completion of design tasks metrics from other disciplines to justify decisions

Indicators of Low and High Coordination Indicators of Low-Coordinated Process Goals Tasks • Defines generic goals usingle reference • • Integrates goals within tasks/disciplines • Does not connect tasks • Does not assign team member responsibilities or roles References metrics as an indicator of aircraft performance and capabilities within tasks Does not fully explain critical metrics included in the design process • Metrics Decisions • • Does not use a clear order or hierarchy to structure tasks Does not justify decisions or explain their decisions in context of the aircraft design Indicators of High-Coordinated Process • Defines goals imposed by primary requirements document as well as defines goals related to the context and stakeholders of the design problem • • Integrates goals across disciplines • • • Uses hierarchy and order to decompose tasks across and within disciplines Integrates tasks across-disciplines using cross-disciplinary metrics Assigns team member responsibilities according to task definitions Uses a variety of metrics to explain decision reasoning in a cross -disciplinary context Explains the reasoning and use of critical metrics included in the design process Justifies decisions using both quantitative mathematical modeling and qualitative supporting arguments, across disciplines 46

Design Knowledge Coordination Contribution (1) Framework of Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Contribution (2) Authoritative description of design knowledge coordination in AE design Research Question (2) How do novice aerospace engineers coordinate and integrate knowledge about a design? Contribution (3) Observable indicators of design knowledge coordination in student design processes Research Question (3) What educational interventions can better support novice aerospace engineers’ design knowledge coordination? Contribution (4 a) Learning goals for educational resources Contribution (4 b) Examples of concepts that support student development of design knowledge coordination 4747

Aerospace Engineering Curriculum Aerodynamics Materials Propulsion Capstone Design Grinter, 1994; Dym et al. , 2005; ABET, 2016; Crismond & Adams, 2012 Performance Flight Dynamics ABET Criteria (c) Design a system with constraints (d) Function on multidisciplinary team (f) Have professional and ethical responsibility (g) Communicate (h)Understand impact of engineering

Aerospace Engineering Curriculum Aerodynamics Materials Propulsion Performance Flight Dynamics Use design knowledge coordination framework and literature to inform teaching activities • Course has limited time for new material • Topics and teaching methods should complement the design process taught within aerospace engineering capstone courses. Capstone Design Grinter, 1994; Dym et al. , 2005; ABET, 2016; Crismond & Adams, 2012 ABET Criteria (c) Design a system with constraints (d) Function on multidisciplinary team (f) Have professional and ethical responsibility (g) Communicate (h)Understand impact of engineering

Design Knowledge Coordination Framework Goals Tasks Metrics Basic Approach Defines a general design goal Defines tasks & subtasks Incorporates an order and hierarchy to the tasks and subtasks Uses metrics to complete tasks and subtasks Discipline-Oriented Approach Design goals are decomposed into more specific goals for each discipline The tasks and subtasks have an order and hierarchy that align with engineering disciplines Within each discipline, there are critical metrics guiding the decisions and tasks Justifies decisions through Decisions are justified through discipline. Decisions completion of design oriented metrics tasks Crismond & Adams, 2012 Coordinated Approach Identifies tradeoffs between disciplineoriented design goals Recognizes the links between tasks and subtasks Iterates across the tasks to update information about the design Maintains consistent values for metrics across the different disciplines Describes the overall impact of a particular decision Incorporates a variety of metrics from other disciplines to justify decisions

Learning Goals for Design Knowledge Coordination Goals Tasks Metrics Decisions Learning Goals where students… Use multiple reference points to define design goals (e. g. primary requirements, design context, and stakeholders) Apply design goals consistently and iteratively throughout the design process to make and validate design decisions Define tasks and subtasks with an order and hierarchy Recognize the links between tasks and subtasks Iterate across the tasks to update information about the design Recognize critical discipline-oriented metrics to required to complete tasks and subtasks Maintain consistent values for metrics across the different disciplines Justify decisions through a completion of tasks and consideration of evidence Describe the overall impact of a particular decision Design Activities Requirements Analysis Stakeholder Analysis Concept Maps Design Structure Matrix Systems Engineering Decomposition Tools 51

Requirements Analysis Goals Learning Goals where students… Use multiple reference points to define design goals (e. g. primary requirements, design context, and stakeholders) Apply design goals consistently and iteratively throughout the design process to make and validate design decisions Design Activities Requirements Analysis Stakeholder Analysis + Gather requirements from multiple references, such as from the customer, mission objectives, mission environment, and key performance-based metrics + Creates a verifiable and complete list of requirements for the design + Evaluate the level of detail included in the requirements and their integration into design decisions DAU, 2001; INCOSE, 2004

Concept Maps Tasks + + Learning Goals where students… Define tasks and subtasks with an order and hierarchy Recognize the links between tasks and subtasks Iterate across the tasks to update information about the design Represents the relationships between concepts Map the conceptual design at the start of the design process Reflect on the design process at various points in the semester Evaluate the completeness of the concept map Turns, Atman & Adams, 2000; Ellis, Rudnitski, Silverstein, 2004; Watson, Pelkey, Noyes, Rogers, 2016 Design Activities Concept Maps

Design Structure Matrix Metrics Learning Goals where students… Recognize critical discipline-oriented metrics to required to complete tasks and subtasks Maintain consistent values for metrics across the different disciplines Design Activities Design Structure Matrix + Reaches across disciplines and explicates the interdependencies within the task environment + Creates a method for externalizing reasoning of decisions and documenting process Image from Browning, 2001, p. 292 Goel & Pirolli, 1992; Steward, 1981; English, 2009; Browning, 2001

Functional Decomposition Decisions Learning Goals where students… Justify decisions through a completion of tasks and consideration of evidence Describe the overall impact of a particular decision Design Activities Systems Engineering Decomposition Tools + Decomposes the design by function and identifies what each function needs to do (e. g. the engine must be capable of providing XX pounds of thrust, as determined by a previous analysis). + Represents the impact of decisions as a waterfall effect, where one decision might impact several other future decisions + Can perform trade studies and evaluate alternate solutions + Evaluate the completeness and detail of the decomposition DAU, 2001; INCOSE, 2004

Design Knowledge Coordination Contribution (1) Framework of Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Contribution (2) Authoritative description of design knowledge coordination in AE design Research Question (2) How do novice aerospace engineers coordinate and integrate knowledge about a design? Contribution (3) Observable indicators of design knowledge coordination in student design processes Research Question (3) What educational interventions can better support novice aerospace engineers’ design knowledge coordination? Contribution (4 a) Learning goals for educational resources Contribution (4 b) Examples of concepts that support student development of design knowledge coordination 56

Conclusions and Future Work 57 57

Design Knowledge Coordination Contribution (1) Framework of Design Knowledge Coordination Research Question (1) What design knowledge coordination does aerospace engineering require? Contribution (2) Authoritative description of design knowledge coordination in AE design Research Question (2) How do novice aerospace engineers coordinate and integrate knowledge about a design? Contribution (3) Observable indicators of design knowledge coordination in student design processes Research Question (3) What educational interventions can better support novice aerospace engineers’ design knowledge coordination? Contribution (4 a) Learning goals for educational resources Contribution (4 b) Examples of concepts that support student development of design knowledge coordination 58

Future Directions Engineering Education Research and Application + Develop and evaluate educational activities using design knowledge coordination framework + Evaluate and design comprehensive curriculum programs across technical areas Evaluating human decision-making in complex, real-world systems + Modify and apply coordination constructs to industry practices (engineering design as well as other complex decision contexts) + Develop protocols training programs for rapidly changing workforce

Thank you! + + + Dr. Amy Prichett Dr. Brian German Dr. Ute Fischer Dr. Wendy Newstetter Dr. Neil Weston Dr. Jennifer Turns + + + Dr. Karen Feigh Dr. Leslie De. Church Dr. Julia Melkers Carl Johnson Study Participants Matthew Miller + Dr. Alexandra Strong + CEC Lab Members + Center for the Enhancement of Teaching & Learning + AE Department (11 years!) This work was sponsored by the NSF, grant number DGE-0644493

61 61