The Architecture of Coordination James Herbsleb Carnegie Mellon

The Architecture of Coordination James Herbsleb Carnegie Mellon University jdh@cs. cmu. edu http: //conway. isri. cmu. edu/~jdh/ The author gratefully acknowledge support by the National Science Foundation under Grants IIS-11 0414698, IIS-0534656, and IGERT 9972762 1

Architectures and Organizations § Level of an industry − § Level of an organization − § E. g. , modular clusters E. g. , matching hypothesis Level of a practicing architect, manager, or engineer Architectural decisions define coordination problems − Understanding and managing impact of decisions − 2

Today. . . § § 3 How can you measure the match of an organization with the structure of the software it is producing? Coordination theory about the “modularity mismatches” – the failures or limits of modularity

Product Structure: Conway’s Law • “Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure. ” M. E. Conway, “How Do Committees Invent? ” Datamation, Vol. 14, No. 4, Apr. 1968, pp. 28– 31. 4

Conway’s Law Components Teams Homomorphism Software 5 Organization

Designing Coordination Problems Components Software 6 Teams Organization

Designing Coordination Problems Teams Components ? What kind of coordination is required? Software 7 Organization

Socio-Technical Congruence and Productivity Measuring Coordination Requirements (CR) Concept 8 Task Assignments Task Dependencies (A) (D) a 11 … a 1 k an 1 … ank X Coordination Requirements (AT) d 11 … d 1 k dk 1 … dkk X (CR) a 11 … a 1 n ak 1 … akn = cr 11 … cr 1 n crn 1 … crnn

Coordination in a Distributed Project • • Software firm, 1 product, 114 developers 8 teams, 3 locations, all in US Product separated into components Each component assigned to 1 team Cataldo, M. , Wagstrom, P. , Herbsleb, J. D. , Carley, K. (2006). Identification of coordination requirements: Implications for the design of collaboration and awareness tools. In Proceedings, ACM Conference on Computer-Supported Cooperative Work, Banff Canada, pp. 353 -362. 9

A Word About Tools and Data • Use archival data from large software development project • Modification request (MR) system – Users, testers, developers request changes – Bug fixes, new functionality • Version control system – Maintains all changes to all files – Some set of changes correspond to each MR – Has data about who made change when • Communication data – IRC chat – Discussions within MR system 10

Socio-Technical Congruence and Productivity Measuring Coordination Requirements (CR) Concept Task Assignments Task Dependencies (A) (D) a 11 … a 1 k an 1 … ank Developer modified files X Coordination Requirements (AT) d 11 … d 1 k dk 1 … dkk X Files changed together a 11 … a 1 n ak 1 … akn Transpose of developer modified files Data 11 (CR) = cr 11 … cr 1 n crn 1 … crnn Who needs to coordinate with whom

Volatility in Coordination Requirements Members of other teams Proportion Change in coordination group Week 12

Socio-Technical Congruence and Productivity Measuring Congruence Coordination Requirements (CR) cr 11 … cr 1 n crn 1 … crnn Actual Coordination (CA) ca 11 ca 1 n … can 1 cann … • Team structure • Geographic location • Use of chat • On-line discussion Diff (CR, CA) = card { diffij | crij > 0 & caij > 0 } Congruence (CR, CA) = Diff (CR, CA) / |CR| 1 13

Predicting Resolution Time Table 2: Results from OLS Regression of Effects on Task Performance (+ p < 0. 10, * p < 0. 05, ** p < 0. 01). Model**II Model* III Model* IV (Intercept) 2. 987 3. 631 1. 572 1. 751 * * * Dependency 0. 897 0. 653 0. 784 0. 712* Priority -0. 741* -0. 681* -0. 702* -0. 712* Re-assignment 0. 423* 0. 487* 0. 304* 0. 324* Customer MR -0. 730 -0. 821 -0. 932 -0. 903 * * * Release -0. 154 -0. 137 -0. 109 -0. 098* Change Size (log) 1. 542* 1. 591* 1. 428* 1. 692* Team Load 0. 307* 0. 317* 0. 356* 0. 374* Programming Experience -0. 062* -0. 117* -0. 103* Tenure -0. 269* -0. 265* -0. 239* -0. 248* Component Experience (log) -0. 143* -0. 195* -0. 213* Structural Congruence -0. 526* -0. 483* Geographical Congruence -0. 317* -0. 312* * MR Congruence -0. 189 -0. 129* * IRC Congruence -0. 196 -Interaction: Release. X Structural Congruence 0. 007 0. 009 Interaction: Release. XGeographical Congruence -0. 013 -0. 017 + Interaction: Release X MR Congruence -0. 009 -0. 011+ Interaction: Release X IRC Congruence -0. 017* -N 809 1983 2 Adjusted R 0. 787 0. 872 0. 756 0. 854 (* p < 0. 05, ** p < 0. 01) 14

Average Level of Congruence for Top 18 Contributors 16

Average Level of Congruence for the Other 94 Developers 17

The Story So Far. . . • Coordination requirements are highly volatile. • Congruent coordination activities are associated with performing the work faster. • The top developers behave much more congruently than the rest. • Have replicated congruence finding on GNOME projects • Have experimented with many different ways of computing dependencies – Logical dependencies the most predictive by far 18

Theory • Build on modularity/architecture literature – Product structure and task structure • Build on network analysis – Network attributes matter – Relations among people, decisions, components • Methodology: build on idea of logical dependencies 19

Coordination: Five Propositions • P 1: Design progresses by making decisions. • P 2: Decisions are linked by constraints in a potentially large and complex network. – The “decision network” • P 3: The need for coordination among individuals arises from – Properties of the decision network – Assignment of decisions to people • P 4: Effective coordination is the result of coordination actions, moderated by coordination capacity. • P 5: Coordination breakdowns occur when effective coordination is insufficient for coordination needs 20

Example Domain: Field Robotics 21

Example Network: Field Robotics 22

Coding Method • Meeting segments Avionics 23 Optics Software

Coding Method • Meeting segments • Divide into decision discussions Avionics 24 Optics Software

Coding Method • Meeting segments • Divide into decision discussions • Code according to components involved in the decision discussion – Co-occurrence much like logical dependency Avionics Optics Camera 25 Mobility Software

Example Network: Field Robotics 26

Hardware Discussions Newman Modularity: . 39 27

Software Discussions Newman Modularity: . 03 28

Network Theory of Coordination Decision Network Task Assignment Coordination Requirements Coordination Actions Coordination Capacity 29 Coordination Effectiveness Outcomes

Challenges • Planning – Predicting coordination requirements – Assessing coordination capacity – Choosing optimal coordination actions • Coordinating – Monitoring for unexpected mismatches – Organizational tactics, architectural tactics • Theorizing – Principles, patterns for network dependencies – Prior theories as special cases 30