Optimizing NASA IVV Benefits Using Simulation Grant Number

Optimizing NASA IV&V Benefits Using Simulation Grant Number: NAG 5 -12736 David M. Raffo, Ph. D College of Engineering and Computer Science School of Business Administration Portland State University davidr@sba. pdx. edu

Problem • There is a critical need allocate scarce IV&V resources in the most effective manner possible • Questions • – How can the economic benefit of IV&V technologies be optimized? – What is the best strategy for applying IV&V technologies? – What is the benefit of applying one combination of IV&V techniques to a given process vs another? The goal of this project is to provide NASA with a flexible decision support tool that can be applied to NASA projects and NASA IV&V to assess the benefits of various quality assurance and IV&V policies on a given NASA project.

Relevance to NASA • Contributes to mission assurance & improved project management • More cost effective management of IV&V • Potential to be applied to multiple NASA projects at multiple NASA Centers (started project with KSC) – Develop method for rapidly tailoring simulation models in the NASA environment – Comparison of effectiveness of DES models with COCOMO & SD models – Augment DES models with SD factors

Importance/ Benefits • What if a NASA project canceled due to software delays and cost overruns had this tool: – They could have assessed the benefits vs. costs of IV&V early on. They could have let IV&V in earlier or had a broader scope. – They could have had better estimates, validated those estimates, and quantified the impacts of scope and budget changes. – They could have validated processes they had or designed better processes. – They could have assessed the impacts of developer and project manager experience on productivity and its impact on the project

Approach • This tool is based on Software Process Simulation techniques and the Process Tradeoff Analysis Method (PTAM) based on work originally done at the SEI. • Key Advantages of using a Simulation Based Approach: – Compare alternative process scenarios and make tradeoffs – Forward looking at what is “possible” not just what has been – Extensive “What if” analyses to explore alternatives desired by management – Compute a ROI and develop a quantitative business case – Update initial plans; choose best corrective action

Approach Software Development Process Better Process Decisions Process Performance Cost, Quality, Schedule SW Process Simulation Model Project Data Process and Product

Process Analysis Tradeoff Tool • Large-Scale software project planning support • Rapidly deployable models dramatically reduce time to develop and modify • Platform for analyzing process tradeoffs and impact of QA, IV&V or other process technologies (tools) • NASA Project studying the impact of IV&V • IV&V can be applied at one or more phases over the entire software development lifecycle • Alternative IV&V application configurations can be compared (ROI)

Basis for Project Management Original Plan Current Trajectory Actual Data Corrective Action X

Accomplishments • Delivered prototype version of simulation tool to NASA. This prototype contains: – IEEE 12207 Software Development Process Lifecycle Model that can be adapted to a variety of NASA projects – Nine potential IV&V techniques (using new IV&V WBS) • Increased PF to 7 - Researchers using NASA data (2 projects) • Reported work at Pro. Sim’ 04

Next Steps • Creating new software process life cycle models • Adding more IV&V Techniques • Continuing NASA data analysis with goal of developing business case scenarios • Continuing tool development • Joint research with KSC • Continuing research in Optimization Theory – Assessing variety of techniques including: Treatment Learning, DEA, and Financial Measures

The End