2020 Additive Manufacturing Workshop 23 25 June Virtual

2020 Additive Manufacturing Workshop 23 – 25 June Virtual Greg Kilchenstein (ODASD Materiel Readiness) Kelly Visconti (OUSD Research & Engineering) Marilyn Gaska (America Makes / Lockheed Martin) 1

June 25 (Day 3) AGENDA 1: 00 – 1: 05 PM Welcome Back - Greg Kilchenstein / Kelly Visconti / Marilyn Gaska 1: 05 – 3: 30 PM Work Group Co-Leads Present Outbriefs • Steps / Actions Completed • Open Actions / Post-AM Workshop Actions • Issues / Challenges / Gaps Identified • Key Takeaways 3: 30 – 4: 00 PM Closing Remarks – Mr. Rob Gold and Mr. Steve Morani 4: 00 PM Wrap-Up - Greg Kilchenstein / Kelly Visconti, / Marilyn Gaska 2

2020 AM Working Group Outbriefs • Framing the Do. D-level AM Guidebook • Addressing AM Cyber Challenges • AM Data Management • Workforce Development for AM • AM Metrics – Measures of Effectiveness and Measures of Performance • AM Part Risk Categorization and Relation to Part Criticality • AM TDP for Procurement in Sustainment 3

AM Workshop Protocol • Sign in / Dial-in prior to start time. • Log-on with your Full Name • Please keep your phones on mute unless you are presenting. Do NOT put your phone on hold. Should you have to temporarily drop off please hang up and call back. • Presenters - slides will be advanced by NCMS / LMI • This is an open forum • For technical difficulties contact Jennifer Khoury at jenniferk@ncms. org or 734 -995 -7989

2020 Additive Manufacturing Workshop Framing the Do. D-level AM Guidebook Co-Leads: Greg Kilchenstein Kelly Visconti 5

Framing the AM Guidebook Objectives: • Create a Resource to Assist with Implementing AM Across the Do. D AM Lifecycle • Develop a Framework for a Do. D AM Guidebook • Complete a Storyboard of Contents to Include in the AM Guidebook

Framing the AM Guidebook Sub-Working Groups: • Acquisition and Contracting • Co-Leads: Ray Langlais; Mike Acosta; Eleanor Shelton • Research & Engineering / Specs & Standards • Co-Leads: Greg Kilchenstein; Kelly Visconti • Collaboration, Communication & Policy Implementation • Co-Leads: Debbie Lilu; Chris Fick

Framing the AM Guidebook Steps / Actions Completed: • Categorized and Prioritized 19 Key Topics • Detailed 60+ Topic Statements that Illustrate the Impact of the Topic on AM in the Do. D • Placed Topics in a Framework • Developed a Strawman Outline for the Guidebook

Acquisition and Contracting • • Intro General AM Info – AM Acquisition Strategy • Acquisition and Contracting Considerations • Acquisition of Data – AM Sustainment Strategy – Risk Management • • • AM and Weapon Design AM Data Rights Tech Data and Data Rights Access – – – • • • TDP IP Unavailable Tech Data Security Data Repository (Overlap with Collaboration and R&E group) Providing R&D Data – FAIR principles (Data Management Group input) ID of AM Parts Contract Administration Appendix: Sample Contract Schedules 9

Research & Engineering / Specs & Standards • Qualification and Certification Approaches – – • Common Procedures – – – • • • Risk based qualification and certification approach Acceptability/Verification Procedures and Practices Qualification by similarity (equivalency) and reciprocity for qual/cert Assure repeatability and reproducibility Engineering and Design Materials Process Final Part Testing Reverse and Re-Engineering Environmental, Health, Safety and Sustainability Considerations Configuration and Change Control Parts Management Anti-Counterfeit Technology Protection 10

Research & Engineering / Specs & Standards • Standards and Specifications – Use of Common S&S – Development of Common Standards and Specification • Basic and Applied Research Considerations – Fundamental R&D of materials – R&D for new materials and improved existing materials – designed for AM specifically, to drive lower costs and improve outcomes. – Process variability and characterization – In-situ process monitoring and control – Material characterization - reliable and trusted databases – Business case for continued AM R&D. • References for all Q/C and SS section • Economic and Affordability Considerations • New Topic: Physical Infrastructure Management 11

Collaboration, Communication & Policy Implementation • Collaboration – – Tactical and Strategy approach Process for sharing info and community engagement Stakeholder plans and communication plans Precompetitive collaboration • Public database – (resources) • Training – Common Do. D (CUI) database • Implementation – Need to collaborate – Policy Plan from Users and Acquisition • Recorded procedures • Training across all levels on the policy and implementation (resources) – – – Engaged industry in the maintenance of AM policy. Wargame/table-top for implementation Measure effectiveness of Policy (Metrics) Dynamic implementation plans/maintaining best practices Share implementation plans Assess implementation plans (service level – metrics) 12

Framing the AM Guidebook Next Steps: • Incorporate Input from other AM Working Groups • Finalize the Outline for the Guidebook • Draft Do. D AM Guidebook Language • Establish a Review Process § Include AM Workshop WG Members

Framing the AM Guidebook Issues / Challenges / Gaps Identified: • Establish Ownership of Guidebook • Securing Resources • Buy-in for the Guidebook (Coordination) • Sync Completion with Do. DI Publication

Framing the AM Guidebook Key Takeaways: • Establish an Office of Responsibility in OSD for the Document • Need Training and Resources to Implement the Policy • Broad Collaboration with Industry and Across the Services Needed to Share Data and Best Practices and Input to Guidebook • Need for a Common Portal – Dynamic / Actively Maintained • AM Guidebook • Data Repository

Framing the AM Guidebook Questions?

2020 Additive Manufacturing Workshop Final Outbrief Addressing AM Cyber Challenges Co Leads: Al Lowas Dana Ellis 17

Addressing AM Cyber Challenges Objectives: • Review progress to date • Review planned contracts for studies • Review state of the industry • Discuss methods to proliferate findings

Addressing AM Cyber Challenges Sub-Working Groups: • Industrial Control System Overlay • Co-Leads: Steven Richard, Kevin Winters • How to Encourage AM Machine Manufacturers to Improve Cybersecurity • Co-Leads: Dana Ellis, Adwoa Amofa

Addressing AM Cyber Challenges Steps / Actions Completed: • • Reviewed objectives to strengthen cyber-physical security posture Reviewed NIST Risk Management Framework Updated on SAE G-32 Cyber Physical Systems Security for Additive Manufacturing Systems committee progress • JA 7496 – Cyber Physical Systems Security Engineering Plan Reviewed Industrial Control System overlay Updated on Mx. D Cyber & the National Center for Cybersecurity in Manufacturing efforts Reviewed Markforged’s security approach for AM Discussed assumptions and best practices for security Discussed multiple companies’ perspectives on securing AM

Addressing AM Cyber Challenges Open Actions / Post AM Workshop Actions: • • Develop a white paper for Authorizing Officials (AOs) to differentiate OT/Io. T for AM and traditional business IT Develop a white paper for awareness in applying the NIST SP 800 -82 Rev 2 ICS Overlay on AM to begin cross-training SMEs Work with the SAE G-32 CSSP subcommittee to develop language that encourages AM vendors to incorporate security controls earlier in product development and define critical, maturity and resilience standards • Leverage language for contractual risk transfer within supply chain Investigate network approaches to securing (Do. D) AM enclaves with goal of ATO

Addressing AM Cyber Challenges Key Takeaways: • • Do. D Advanced Manufacturing Network for industrial control systems is necessary to separate AM (and other OT) from enterprise business IT Consider physical and process monitoring as a substitute for continuous monitoring of AM Investigate novel AM Software as a Service (Saa. S) operating on Fed. RAMP to control AM vendor hardware Cybersecurity assessors and AM leads must have awareness of respective mission requirements (secure infrastructure and produce components, respectively) and select security controls on risk evaluation, not by a checklist

Addressing AM Cyber Challenges Questions?

2020 Additive Manufacturing Workshop Final Outbrief AM Data Management Co Leads: Mark Benedict Alex Kitt Jennifer Wolk 24

AM Data Management Objectives: • Continue to mature AM data standards and definitions • Explore how the Do. D could share pedigreed materials and engineering data

AM Data Management Sub-Working Groups: • Common Data Dictionary • Lead: Alex Kitt • Common AM Database Experiment (CAMDEN) • Lead: Mark Benedict

AM Data Management Steps to Achieve Objectives: • • Discuss recent progress in the development of a common data dictionary. • Efforts within ASTM and recent success stories in applying the common data dictionary. Explore the use of the common data dictionary and prioritize specific elements within key user personas. Provide an overview on common databases. Utilize a live instance of CAMDEN to explore populated data within this experiment.

AM Part Realization Part Selection Design & Analysis Part Certification Post Fab Process-ing AM Process Criticality, Alloy, & Complexity Mechanical Properties Input Material Removal from Plate Qualified & Frozen AM Process Eng. Knowns & Unknowns Operational Environment Build Parameters Thermal: Stress Relief, HIP, HT Know Material Properties Extant Manufacturing Process Tolerances Orientation Machining Subscale/Full Scale Testing Certification Requirements Post AM Processing Requirements Build Plate Density Surface Treatments NDI BCA / A 0 Orientation & layout Qualified, Frozen Process Assembly, e. g. , bushing & bearings Digital Thread / Data Management & Configuration Plan EDRAP / Qualification-Certification Manufacturing Plan Quality Assurance Plan Ref: Frazier & Mc. Michael, AM for Def. & Gov. , Dec 8, 2015 28

Buckets 29

Bucket Material Sci Designer 1 X 2 X 3 X M&P Certifier Structures Certifier X Data Adm. X x X x X X 4 X Manufacturing 5 X 6 X X X x 7 X X X x 8 x X 9 X X X x 10 x 11 X x 12 X x 13 X x 14 x X X x 30

Where the data is stored • • Local storage On AM machines Engineers desktop computers, hard drives, etc Mainframe & cloud. Enterprise structured databases (e. g. , MSC, Granta) Hard copy, i. e. , paper Libraries: ASTM, ASM, MMPDS PDM – product data management system What are the characteristics of an ideal data storage system • • • FAIR – Findable, Accessible, Interoperable, reusable IOT Connected. Seamless flow of data from the machine without human intervention. Open data structure and data model. 31

Query 6 (Very Specific Query) Description: (Query data in the following order across all available data) 1. AM process type 2. Alloy 3. Spec 4. Processing conditions (parameter set) 5. Post-processing conditions 6. Property of interest (surface roughness) Value of performing query (with examples): • Return the data of interest to perform statistical analysis, and form design allowables which can be used for design or FEA • Make graphical representations and use filters to control ranges, etc. – Form a processing window – See how the query results compare to a requirement Authoritative data source: In a database or in test result files (pdfs, spreadsheets) Key Contributors: Kyle Cobb 32

Query 7 (Specific Result Format with Comparisons) Description: Find all Ti 64 specimens that have SEM data [characterization data] and return all images at X magnification” And/or Find any characterization data for “sibling”/”similar” Ti 64 specimens Value of performing query (with examples): Compare a data set in a specific format with other/legacy data sets Authoritative data source: Databases containing files with metadata Key Contributors: Matt Jacobsen 33

Query 8 (What data do we even have for this part? !) Description: Search by a part identification Return: • Any data that is available for that part or a particular region/location of the part Value of performing query (with examples): • Helping people to remember what a part had done to it and what data is available Authoritative data source: Build documentation sheets (electronic/hand written), databases Key Contributors: Matt Jacobsen, Kyle Cobb 34

Query 5 (Queries that are not currently possible) Description: How does part geometry correlate to surface roughness? e. g. ) Show me the surface roughness of downward facing surfaces at different angles Value of performing query (with examples): Understanding relationships between geometry and part quality Authoritative data source: CAD or slice file (which natively is not broken into searchable feature types and would need some level of further interrogation in order to query in this way), test results Key Contributors: Jason Thiese 35

Query Activity Findings • Example queries need to be general, not specific – E. g. : Mechanical test data instead of Ultimate Tensile Strength • What we do with the queried data is KEY to learning from it – E. g. : graphs, statistical analysis, comparisons • Data in its native format may not be good enough – The format of the data being queried matters because it will limit how we can most easily use it. This then informs how we should be collecting/transferring data sets (via CDEF) – Further data synthesis/formatting may be necessary prior to making certain queries – E. g. : CAD files can’t be searched for radii below 25 degrees 36

Data for Sustainment 1. 2. 3. 4. 5. – 6. – – Risk of using AM: Relates to part criticality, understanding of requirements Based on TDP am I qualified to print the part: Is my process qualified, can I validate my additive build Do we have sufficient information to build/purchase part based on standard/policy/etc. ex. : ASTM 52901: Requirements for purchasing parts created with AM. Guidelines for information exchange. It is cost/schedule effective to AM part Current State: Know if we temp/permanent repair with AM or replace with new AM part (this is usually pre-decided in the maintenance plan, assumes AM has already been considered) Future State: Decide if we temp/permanent repair… How to translate/modify legacy part (ex. cast) requirements to for additive replace/repair? Is requirement applicable for an additive product? Is there a risk in modifying because we don’t understand the total system (ex. part was big b/c someone needed to step on) 37

Data for New Acquisition Program Purchasing 1. Is the cost right? 2. Is the part qualified? 3. Is the process/operator/organization/feedstock certified/qualified 4. Characterization builds to meet requirements specs: historical results from that company 5. Do we have proof that the organization is qualified/certified and associated data. May include reference parts 6. Does the part fit the mechanical etc requirements 38

Data to Reduce Risk of Using AM Description: Relates to part criticality, understanding of requirements, understanding of capabilities. Note that: How to translate/modify legacy part (ex. cast) requirements to additive replace/repair is subtle: – – Is requirement applicable for an additive product? Is there a risk in modifying because we don’t understand the total system (ex. part was big b/c someone needed to step on) Value of performing query (with examples): Without understanding risk, you either assume which can be deadly, or act overly conservative which removes potential value from AM Authoritative data source: No obvious answer (ideally on a DOD PLM) Key Contributors: 39

AM Data Management Steps / Actions Completed: • • • Reviewed current progress of CDD; including activities for standardization Reviewed current state of the art with database and database sharing ASTM Work Item Wk 72172 moving to ballot Steps / Actions Ongoing: • • User persona mapping Credentialing of Hyperthought users and live CAMDEN views (Distro D)

AM Data Management Issues / Challenges / Key Takeaways: • • • Need for workforce development in data science Policy emphasis for data management at the start of acquisition • FAIR (Findable, Accessible, Interoperable, Reusable) principles • How do we keep the end in mind to organize from the start? • Capture best practices • Build a culture of data across communities Automation of data acquisition; industry participation Use case of interoperability to translate information to build data Data structure and data language (ex. CDD) will allow faster ingestion and automation of data Significant progress in using CDD led by NIST; including ASTM work ballot • Need for wider adoption CAMDEN intersection with CDD via MOBI Expand CAMDEN user base and exploration; validation of data contents against schemas How to measure progress with respect to data management?

Questions?

2020 Additive Manufacturing Workshop Final Outbrief Workforce Development Co Leads: Josh Cramer Michael Britt-Crane 43

Workforce Development Objectives: • Identify and define roles specific to additive manufacturing across the Do. D enterprise and industrial defense base • Assess skills and knowledge of those roles against the additive manufacturing body of knowledge • Review of additive manufacturing body of knowledge

Workforce Development Sub-Working Groups: • Acquisition Coded Professionals • • Do. D Technicians • • Co-Leads: Josh Heller, Dave Heckman, Ashley Totin Do. D Management/Leadership • • Co-Leads: Dave Ludwa, Whitney Tharp, Shawn Nesmith Do. D Engineers • • Co-Leads: Ron Edmunds, Christian Grethlein, Brian Yoo, Alex Steeb Co-Leads: Brad Ruprecht, Christina Maranto, Tiffany Westbay Industry • Co-Leads: Lawrence Binek, Travis Mayberry

Workforce Development Steps/Actions Completed: • • Workgroup was assembled in 5 breakout teams specific to focus roles Each breakout session and subgroup team focused on a specific goal: • 1 st Session: Identify roles specific to additive manufacturing and define those roles – add job codes where applicable. Teams also ranked these roles in magnitude. • 2 nd Session: Building from the first session, teams took those defined roles and aligned them in both knowledge and performance-based skillsets to the additive manufacturing body of knowledge. Teams also noted specific roles where there needs to be awareness of additive manufacturing. • 3 rd Session: Assessed the additive manufacturing body of knowledge for gaps, updates and missing content across the Do. D and industrial defense base at large

Workforce Development Open Actions / Post AM Workshop Actions: • • Continue to have detailed collaboration between OSD Man. Tech EWD and America Makes to jointly drive this work force Continue the work through joint efforts of JAMWG EWD Working Group and America Makes and embed into the EWD Roadmap at America Makes to potentially house the AM BOK for all groups to leverage and utilize as they need Establish a cadence or communication to key stakeholders in both the Do. D and Industry

Workforce Development Issues / Challenges Identified: • • • Some of the information gathered will be was narrower focused than intended – future to expand to wider audience Starting (not new to many) to see consensus in the need of awareness education/training Uncertainty of “how far” to drill down into some roles and entities – still a need to define and align Will continue to be a need for very “specialized” training in certain roles Integration into legacy systems, definitions and roles Need validation beyond the workshop

Workforce Development Key Takeaways: • • • All groups felt a need to standardize as much as we can in roles, language and definitions Increase collaboration across the teams and stakeholders to better leverage resources and opportunities Better communication of both assets as well as who, what where, when and how around trainings and credentials More integration across the larger AMMO Workshop teams Need for more diverse and expanding stakeholder group • Let’s keep going!

Workforce Development What questions do you have?

2020 Additive Manufacturing Workshop Final Outbrief AM Metrics Co Leads: Anne Flannigan Stephen Kuhn-Hendricks William Peterson Matthew Sloane Ernesto Ureta Timothy Vorakoumane 51

AM Metrics Objectives: • Conceptually identify the inputs and process for each metric Ø How would the metrics be calculated? • Identify the required pieces of data Ø Does the required data already exist in a database? Ø Where would the data be collected? • Comment on the feasibility for each metric Ø How difficult would it be to collect the required data? • Comment on value of the metric Ø How useful is the metric to decision makers

AM Metrics Sub-Working Groups: • Measures of Effectiveness • Co-Leads: Stephen Kuhn-Hendricks; Matthew Sloane • Measures of Capability Maturity • Co-Leads: Anne Flannigan; Ernesto Ureta • Measures of Performance • Co-Leads: William Peterson; Timothy Vorakoumane

AM Metrics Steps / Actions Completed: • • Divided into subgroups and identified critical inputs for AM metrics: cost, time, yield on investment (YOI), system maturity, performance Presentation of metrics from academia, commercial, and government and demonstration of tools to capture metrics (UDRI, GE, Navy). Critical review of Day 1 metrics discussions with full working group Provided inputs to AM Guidebook Ø Developing common metrics Ø Developing methods for tracking metrics progress

AM Metrics Issues / Challenges / Gaps Identified: • • • Full potential of AM will not be realized without an appropriate policy for AM contracting Establish consistent and standardized guidance/policy for qualification/certification of AM facilities, machines, materials, equipment, and personnel Establish a uniform criticality hierarchy (risk matrix v. green, blue, red box) across all potential AM platforms Platform readiness is not an AM metric, but our metrics on cost and schedule MUST consider the readiness of the platform The threshold for a “mature AM system” needs to be defined. Alternatively steps in the progress toward a fully mature system Subjectivity of decisions to pursue AM solutions/investment in AM

AM Metrics Key Takeaways (Effectiveness): Definition: Effectiveness refers to the value of AM production to system readiness with respect to cost, time, and yield • Configuration management will be key for seeing the greatest benefits of AM • Part use is critical in determining cost and time Ø A “good enough” solution (stop-gap) can lower cost and shorten schedule (series of stop gap parts? ) • Cost and schedule assessments will dynamically change with increased learning in parallel paths Ø Same part, similar part, equipment, feedstock experience, process experience

AM Metrics Key Takeaways (Capability Maturity): Definition: Capability maturity refers to the “readiness” of your AM system • Create a tiered criteria for ensuring AM systems are mature enough to handle a predefined criticality, complexity, and output Ø Requires definitions/thresholds of a mature system which could differ by organization Ø Potential checklist for defining maturity • Maturity is not a single direction, once mature a system can also regress • The state of maturity will continue to evolve and our strict definition of maturity may be akin to a moving target Ø Based upon technology advancement

AM Metrics Key Takeaways (Performance): Definition: Performance refers to the quality and quantity of the output/status of the AM system • AM parts produced are impacted by complexity, criticality, material, and level of decision maker Ø Number of parts produced is based on the level of the organization (field/organic, intermediate/depot, or the industry equivalent) • AM parts produced are dependent on replacement/redesign versus novel designs Ø Prototyping, obsolescence, tooling, and sustainment • Qualification and certification is dependent upon academic, commercial, or government (DAWIA, apprenticeship, licensing) Ø Differing considerations for operator and support personnel (engineers, technician, administrative)

AM Metrics Open Actions / Post AM Workshop Actions: • • Continual collaboration of co-leads to refine metrics definitions and begin defining calculations (white paper) Identify existing databases that currently capture the metrics data and identify those pieces of information that need to be captured Ø • • • Where necessary build or append databases to capture additional information Develop methodology for weighting metrics based upon impact to system readiness Collaboration of metrics co-leads with data management, policy groups Make Metrics a working group at AM workshop next year

AM Metrics Questions?

2020 Additive Manufacturing Workshop Final Outbrief AM Part Risk Categorization AM TDP for Procurement in Sustainment Co Leads: Edilia Correa Andy Meighan Tony Delgado 61

AM Part Risk Categorization Steps / Actions Completed: • Shared methods Military Services briefed in JAMA on risk categorization • Gathered methods of determining criticality and broke down the components of criticality • Discussed common themes across part selection processes • Obtained inputs from participants that should be considered as part of the selection process AM TDP for Procurement in Sustainment Steps / Actions Completed: • Briefers provided status of respective initiatives: commercial standards, individual Mil. Svc approaches, and DLA R&D data strategy • Identified situations where 3 D and 3 D AM presents a need for additional information than conveyed in subtractive manufacturing • Collected information on TDP format that will inform the JAMA Task-2 62

AM Part Risk Categorization AM TDP for Procurement in Sustainment Open Actions / Post AM Workshop Actions: • • • Download data and generate graphics that display survey tool (pollev) results o Criticality, Themes (risk categorization), AM Guidebook (recommend 30 days) Validate observations with JAMA core team Continue to participate in workshop and collaborative opportunities 63

AM Part Risk Categorization AM TDP for Procurement in Sustainment Issues / Challenges / Gaps Identified: • Common term definitions (IP, Proprietary Info) • • Various subjects being covered by other groups…cross pollinate • • Cybersecurity in systematic AM data sharing Number of elements in the Military Services’ risk methodologies Both disparity and commonality in packaging of the data and data integrity respectively o AM TDP may extend beyond design to manufacturing • Commercial standards still lagging Mil. Svc standards • Testing and part certification were common topics for potential inclusion in a TDP Digital Model Base Enterprise is the foundation for all advanced data sharing 64

AM Part Risk Categorization AM TDP for Procurement in Sustainment Key Takeaways: • • Focused on the use of AM for urgent warfighter needs and to improve readiness Need to level-set AM methodology across Do. D Not just about policy and processes, it is about making the information useful • • • Configurable packaging of TDP’s (modularity, flexibility) TDP is a legal document Part criticality may change how much prescriptive data is needed in AM TDP Current TDP focus in ensuring accuracy of design data will evolve to ensure accuracy of contracted deliverables TRL/MRL tool type criticality-based TDP guidance sheet 65

AM Part Risk Categorization AM TDP for Procurement in Sustainment Questions? 66