DAO Strategic Management Richard B Rood 20170420 revision

DAO Strategic Management Richard B. Rood 20170420 (revision) 20020820 (original presentation)

Introduction-1 This is a set of notes derived from a series of meetings to provide continuity in management when author (R. B. Rood) stepped down from management of the Data Assimilation Office (DAO). The focus of these meetings was management of the DAO, and a set of issues raised in light of the transference of the responsibilities of the Principal Investigator and shortfalls in the computational budget. At these meetings it was decided that the DAO would generate a strategic plan, which would undergo external review approximately one year after the planning process was initiated. This external review would essentially restart the DAO Advisory Panel. In addition, it was decided that the Joint Center for Satellite Data Assimilation would set out its strategic direction, and in particular, the Joint Center, the DAO, and NCEP would commit to planning of a next generation data assimilation system with a target development time of five years once the planning was initiated. This has been submitted for discussion to the Joint Center’s Oversight Board.

Introduction-2 (conclusions) The viewgraphs that follow are a written version of the ideas discussed with regard to management of the DAO. The primary conclusions are 1) DAO requires a formalized management function that balances all activities in the organization in light of external factors and product delivery. 2) Formal management of systems development, validation, and computational systems is required. A point of particular note, the lead of the validation function cannot also have responsibility for system development. 3) The DAO needs to provide a stable, slowly evolving software interface to internal and external customers. This requires that the DAO Data Assimilation System (DAS) be ‘owned’ by the organization and have a vetting process for combining developments into the DAO DAS. 4) The current number of projects are too extensive for the computational and human resources. There needs to be prioritization and scheduling to map the projects to computational resources from a organizational perspective.

DAO Functions: Fig 1/Caption -1 The DAO functions can be divided into four basic groups. Science Research, Date Assimilation (DA) System Development, Validation, Computational Systems. These functions require different strategies for management. The Science Research function is not conducive to tight management; however, Science Research in the DAO needs to be applied research focused towards meeting the customer needs of the DAO. These needs should be expressed in the Validation Plan. The DA System Development should be the conduit from research to validation to customers.

DAO Functions: Fig 1/Caption -2 The Computer Systems function needs to assure the viability of the validated data assimilation system (DAS) to provide the products to the customer as well as to provide the computational platforms to support all the other functions of the DAO. (This description of DAO functions implicitly assumes a number of robust external functions; for instance, data service. Figure 1 shows the DAO functions, but not the implied external functions. ) Going from left to right across the figure, those functions to the right require increasingly more rigorous management. Balance is needed between the functions, and it is explicitly noted that the unstructured nature of research is in conflict with the ordered structure

DAO Functions: Fig 1/Caption -3 needed to maintain an optimally configured computational system. In addition, the fact that there is significant complexity and uncertainty in the products available to provide computational platforms provides a second source of unstructured activity that the Computational Systems function must face. Therefore, even this group needs to participate in experimentation and prototyping that has uncertainties similar to science-research uncertainties. (Implying a sub-structure similar to the DAO as a whole. ) The arrows from Computational Systems to all of the other functions explicitly highlight that the computational resources are limited and impact the ability of all functions to perform their jobs. A major aspect of DAO management is prioritization of projects, allocation, and monitoring of computational resources.

DAO Functions: Fig 1/Caption -4 The dashed vertical lines between the functions make it clear that formal separation is required between these functions. Checks and balances are required. At the same time, the solid double arrows connecting DAS and Validation represent that these two functions are very closely related and need to work in concert, essentially, as customers of each other. It is CRITICAL that the manager responsible for Validation is not also responsible for development in the DA System.

DAO Functions: Figure 1 Science Less Managed DA System Validation Comp Sys. More Managed

DAO Management Function: Fig 2/Caption -1 The DAO requires direct management of its resources to focus activities on systems development to meet customer requirements. At least two levels of formal management are required - management at the function level and management at the organizational level. It is noted that management responsibility comes into conflict with both the scientific research interests of individuals and the reward structure of the Center. The Figure 2 shows the Management Function. The figure represents the need to have different levels of management. The dashed line to Science represents the need for less structured management for Science.

DAO Management Function: Fig 2/Caption -2 It is the responsibility of the top-level management to balance the tensions between the different functions in the DAO and the requirements from the external environment to meet, as best as possible, the needs of the customers.

DAO Management Function: Figure 2 External Environment Budget Initiatives Competition Center Environment DAO Management Function Management Science Customers Deliverables DA System Function Management Validation Comp Sys.

DAO Management: Fig 3/Caption -1 Figure 3 defines the management function more concretely and is based in DAO history. Traditionally the management function required at the top is more than the DAO Head can do individually. Further, a broad range of skills and techniques is required; therefore, more than one person may be needed. Experience shows that it is best to develop, internally, disciplined management with science experience; however, if necessary, external formally trained managers can be partnered with the DAO head. The need for disciplined, formal management cannot be ignored.

DAO Management: Fig 3/Caption -2 The box marked Validation represents, perhaps, the most important concept in DAO management. Namely, a robust validation process is central to the success of the DAO. It defines scientific development, schedule, and resource allocation. There is a need for checks and balances between development and validation. It is ill-posed to assign validation to developers. Given that ‘organizational ownership’ requires that the validation balance a variety of scientific problems, the validation needs to be managed by a team. This team will include developers, but, again, the ultimate validation authority cannot lie in the DAS group. This is, de facto, a teaming of the two groups, and should be the basis of organizational ownership of the DAO DAS.

DAO Management: Fig 3/Caption -3 DAO has used a Tiger Team approach functioning as a Validation function. This activity needs to be a sustained and permanent part of the DAO activity. The top level management function then balances the tension between systems development and validation. In addition the top level management must assure that customer needs are being addressed, including delivery of software and products, as well as external factors are being considered. Of special concern is the management of the computational resource. (Currently the DAO is attempting to support too many projects given its computational (and human) resources. These need to be prioritized and scheduled, based on customer need and the validation plan. )

DAO Management: Fig 3/Caption -4 Since the DAO generally draws on senior scientific staff for both leadership and management, certain other tensions should be explicitly noted. First, the current promotion process best rewards people for their independent science. Second, there is a natural conflict of interest between organizational science and personal research. A process to control this conflict is needed. This could be attained by having the management functions rotate on term, with the personal research put on hold or detailed to some sort of ‘escrow’ managed by a colleague.

DAO Management: Figure 3 DAO Head + Management Validation Science Development Board Science DA System Validation Comp Sys.

DAO DAS system: Fig 4/Caption – 1 In order to allow the Validation and Development function to work, as well as to accommodate the needs of both the Science and Computational Systems functions, it is necessary to manage a “DAO DAS. ” That is, a DAS that is managed by the DAO, and is supported for both internal and external customers. The configuration and management of a DAO system by the DAO management function is critical to DAO success.

DAO DAS system: Fig 4/Caption – 2 The DAO maintains a configured system and users have access to and the ability to contribute to the system. The current process appears to require customers and partners to keep up with the developers, and presents an unstable interface to customers and users. In addition it appears to place on separate paths the development system and the validated system. The validated system, presumably a candidate for an operational system, then becomes a path of operational implementation, maintenance, and optimization. This distances the validation path from the intellectual energy of the development path.

DAO DAS system: Fig 4/Caption – 3 The DAO needs to develop a software management structure that allows the developers of science, computational enhancements, etc. to contribute, while at the same time bringing these developments back to the branch in the software repository that houses the validated system, i. e. , the “DAO- owned” branch. These changes need to be vetted through the validation board, and introduced at regular intervals, with public documentation of the changes and projection of the next set of changes. This vetting process will stabilize the system presented to customers as well as assure an end-to-end consideration of the DAS. DAO management and developers need to assume the responsibility of having new developments brought to a stable “slowly” evolving DAS, rather than requiring all customers to bring their changes to a rapidly evolving DAS.

DAO DAS system: Figure 4 DAO Head + Management Validation Science Development Board Science DA System Validation DAO-DAS SYSTEM Comp Sys.

Next viewgraphs • The next two viewgraphs come from many discussions with software developers. They might be useful for discussion.

Software Management – 1 (validated systems dropped like eggs) ? Validation Validated system becomes separated from development path and dead end. Science development continues on blue path and validation path loses contact and intellectual commitment.

Software Management – 2 (How to bring developments to validated systems? ) ? Computational Improvements Joint Center etc ? Validation Process under the authority of Validation and development board integrates system. Target to have validated “DAO DAS” as primary DAO software.