The ATAM A Comprehensive Method for rchitecture Evaluation

The ATAM A Comprehensive Method for rchitecture Evaluation & The CBAM A Quantitative Approach to Architecture Design Deci$ion Making CSSE 377 Software Architecture & Design 2 Week 6, Day 1, including Ch 11 - 12 (opening parts) in Bass’s book 1

Today – Feedback on Project 4 so far Describe to implementers how you are improving the testing for tomorrow’s retests. Part 3 Intro on Analyzing Archs Main topic: Chapter 11 & 12 in SA (Opening parts of each chapter) Tomorrow – Background: Red sky at morning Sailors take warning! Red sky at night Sailors’ delight. Have your implementers try the new testing process in class. Vote on Friday for final Project 4 deliverable Biweekly Quiz # 3 Then… Fall Break! 2

Intro Analyzing architectures • Need to know why, when, and where to do this activity • Tougher than design review of a program – who is left, who’s both “knowledgeable” and “objective” about the overall design? • We’ll describe Bass’s methods, and how AT&T did this, for comparison 3

SA Ch 11 – The ATAM - What it is • “A Comprehensive Method for Architecture Evaluation” • One of several review methods to “guess” the success of a design ahead of time • This one is more in depth than some, because it also tries to help do the design, not just critique it ATAM = Architecture Tradeoff Analysis Method 4

SA Ch 11 – The ATAM Stolen from older disciplines! Here’s an “architecture review” going on in architecture. Other engineers do similar things. From www. unrealstockholm. org/drupal/? q=node/100. 5

SA Ch 11 – The ATAM Participants in the ATAM • The Evaluation Team • Project Decision Makers • Architecture Stakeholders – You have to decide carefully on how to include stakeholders! – What will their inclusion do to the review? • Clients • Users • Others affected by the system 6

SA Ch 11 – The ATAM Outputs of the ATAM • A concise presentation of the architecture • Articulation of the business goals • Quality requirements in terms of collection of scenarios • Mapping of architectural decisions to quality requirements • A set of identified sensitivity and tradeoff points • A set of risks and non-risks • A set of risk themes 7

SA Ch 11 – The ATAM Phases of the ATAM • Phase 0 – Partnership and presentation • Phase 1 – Evaluation • Phase 2 – Evaluation (continued, with project stakeholders!) • Phase 3 – Follow-up See book for details on each of these 8

SA Ch 11 – The ATAM Phase 1 - Strategic role of QA’s • Phase 1, step 5 – The QA’s are a key part of generating a Utility Tree, ranking by both importance and difficulty of implementation! H, M, or L for each, as shown: 9

SA Ch 11 – The ATAM By the way, what’s “Utility”? • It’s an economics concept: – Utility is a measure of the relative satisfaction from, or desirability of, consumption of various goods and services. – Utility is an abstract concept rather than a concrete, observable quantity (like $). – Utility often suggests a “value” more than cost. E. g. , in the “importance” variable on the previous slide. • Probably you know this, if you had: – SL 151 Principles of Economics • For more info, see, for example, http: //www. investopedia. com/university/economics 5. asp. 10

SA Ch 11 – The ATAM Next… • Analyze architectural approaches for achieving the stand-out items in the tree. • This takes a lot of time! An external team sticks around to participate in high-level design work. 11

SA Ch 11 – The ATAM In Phase 2… • Create “scenarios” by brainstorming with stakeholders. – Like the ones you’ve been doing for QA’s – Only not as formal – Like, “At peak load, the system is able to complete 150 normalized transactions per second. ” • Prioritize scenarios by voting! • Compare to utility tree done by development team’s decision makers . • Analyze arch approaches to these scenarios & present results. • Top scenarios (and ways to achieve them) become the design. 12

SA Ch 11 – The ATAM Case study – the Nightingale Project -- Read it yourself. -- I’ll put one question about it on the biweekly quiz! -- It’s a real system, but they changed the company info for confidentiality. -- It was a system that got the company into a big market, but it was hard to adapt to new customers. -- Alternative, tabular form of “Utility Tree, ” – pp. 296 -7. 13

SA Ch 11 – The ATAM The AT&T Arch Review Process • For comparison with ATAM: – Usually a 2 -day event, with a review team of outside experts and a trained leader. – A project team presented the problem and their architectural solution. • The review team asked questions. • Everyone wrote issues on “snow cards. ” – The review team huddled, organized the cards, and decided on criticality. – They presented their findings back to the project team. – In a couple weeks, they sent a full report, with recommendations. – Projects reported savings of about 10% from this process. Ref – p. 263 of Bass’s book. 14

SA Ch 12 – The CBAM = “Cost Benefit Analysis Method” An SEI reference on this: http: //www. tarrani. net/mike/docs/architect-4 q 01. pdf Architectural goals: P = Performance A = Availability S = Security M = Modifiability etc. 15

SA Ch 12 – The CBAM Here’s the problem in a nutshell… This of course is the Wing. Tite In software development, we’re almost always confused… Why? Basically, our confusion is mostly about both “What to do” and also “How to do it. ” How do you measure the “goodness” of any one feature you might develop or basic design you might try or tool you might buy? The answer… 16

SA Ch 12 – The CBAM Which brings up… It’s ok to be confused in science & engineering. Who says so? Richard Feynman, for example. http: //www. youtube. com/watch? v=lytxaf. TXg 6 c For more on him, see for example, www. its. caltech. edu/~feynman/plenty. html Here he is, in 1959, explaining why we should probably spend a lot of money to develop microelectromechanical devices. 17

SA Ch 12 – The CBAM The most common case! Convert everything to Utility ! The issue shown here is “how much” of each quality attribute we need. Extends the Utility Tree’s H, M, L ratings into quantities. Question – What’s the x-axis here? 18

SA Ch 12 – The CBAM The most common case! Answer – The xaxis here is adding successive scenarios, in order of priority. But, when the book gets to p. 313, it tends to interpret the x-axis more traditionally, as “cost” of creating these (vs. the y-axis of value). This scenario (a) could then be interpreted to mean that as investment cost increases, the resulting benefits taper off. What you’d expect most of the time. 19

SA Ch 12 – The CBAM Besides utility, the other side is cost. The formulas – See p. 313! Your quality attribute scenarios lead to decisions about “utility. ” They assume you can really guess closely enough to apply them with any confidence. Calculating ROI… Implementing it 20

SA Ch 12 – The CBAM From our discussion of reliability (CSSE 377 2009 Wk 2 Day 4 Availability-2. ppt, slide 8) – An example of combining economics and technical expertise: Deciding “how reliable” different parts of your system should be: Gold code – Has to be really reliable. The stuff that has to work right no matter what. Examples? Silver code – Typical “high use” code. Examples? Bronze code – We want it to test out as working correctly, but if it has problems, they will be less severe than for other parts of the system. Examples? 21

SA Ch 12 – The CBAM Case study – NASA ECS Project -- Read it yourself -- I’ll put one question about it on the biweekly quiz! -- It’s a real system – Google for it. 22

Today’s daily quiz… • Put together a quality attribute “tree” for your project – Like Slide 9 • If you were your project’s “clients, ” what changes would you recommend!? • You’ll get it back (to save) – Use it in next week’s enhancement to your arch doc. 23