Your speaker David Ziffer www Project Pro com

Your speaker David Ziffer www. Project. Pro. com 630 -879 -0620 Connect w/me on Linked. In

The coming revolution in software productivity: Eliminating useless variation in software development

Part 0: A vision of a not-too-distant future

Imagine we are cruising along on our current course. As software managers we spend our days thinking about …. Methodology where we hope to gain maybe a 5% gain in productivity

and …. Libraries and Frameworks where we hope to gain maybe another 5% gain in productivity

and …. Tools where we hope to gain maybe another 5% gain in productivity

Then one amazing day …. Someone comes up with a fundamentally different way of building applications. Instead of gradual, fractional changes in productivity, people are now able to write apps in half the time, or maybe even one third the time. A revolution takes place. Everything we were doing before becomes irrelevant in the face of this change.

The only problem is …. This new system and all its documentation are written in Hindi. Or maybe Mandarin.

But don’t worry. Your future is bright. Orange, that is.

Part 1: The past

Hardware: a billion-to-one improvement in fifty years 1960: 16 bytes per cc 2010: 16 GB per cc

Hardware gains were made by active management of products & processes 1950 1990 1960 1970 2000 1980 2010

In comparison, software progress is pretty lackluster 1975: C 2010: C#

Software management focuses on three types of changes • Methodology (Waterfall, Agile, etc. ) – Waterfall & Agile are 40 years apart – but unlike core planes vs. LSI, people still make a case for using either – suggesting that the efficacy is something less than spectacular • Libraries (and frameworks) – completely eliminate the task of coding, but only for particular functions – they do not fundamentally change the production efficiency of the code that still must be written • Tools (particularly compilers) – These are the only software innovations that have fundamentally changed the efficiency of writing code

Arguably there have been only three significant gains in software productivity, and all are in the area of languages • 1950 s: Assembly language replaces machine language – (1000 to 1) • 1960 s: Structured languages replace assembly language – (100 to 1) • 1980 s: Object-oriented enhancements improve structured languages – (10 to 1)

Language changes are getting fewer, further apart, and less effective • 1950 s: machine language -> assembly – 1000 to 1 improvement at the dawn of computing • 1960 s: assembly -> structured languages – 100 to 1 in ten years • 1980 s: structured -> object oriented – 10 to 1 in twenty years • 2010: it’s been 30 years and nothing much new yet

Part 2: The present

Software today is produced in the manner of preassembly-line cars • Small groups of craftsmen build apps one at a time from the ground up. • Every app is improvised to some great degree. • Different construction methods are used at various times even within the same app, according to the tastes of the individuals working on it, i. e. there is plenty of useless variation.

Small groups of craftsmen build apps 1910 1980

There’s plenty of improvisation 1910 2010

And there’s plenty of useless variation, leading to expensive development, poor reliability & high cost of ownership

Useless design variation is the rule. Even within the same app we are likely to find multiple implementations of: – database table structure (keys, status fields, etc. ) – table-access functions in the database (insert, update, delete, read) – database-access functions in the application (ditto) – business logic structure including • data transformation (between database and UI) • validation • error handling – user interface structure

Enforcement of coding standards tends to fail: – Training in the use of standards costs money. – Enforcement of standards costs money. – Use of standards is generally at the option of the developer, who may not subscribe to them. – The value of standards is apparent only to those who understand long-term profitability. – Most importantly: use of standards imposes a guaranteed extra short-term effort that must be tolerated in order to potentially benefit in the possibly distant future.

Since we don’t tend to have standards, our concept of quality control tends to center around postmortem inspection: – A code review performed after a lot of bad code has been written is expensive and ineffective: • Why didn’t we help the programmer up front to produce what was expected? • We probably can’t afford to rewrite the code and even if we can, we’re late. – A testing regime can only tell us whether the code happens to pass a limited set of tests. Testing does not tell us whether the code is well constructed, i. e. : • whether we can impute from the design whether the code will pass tests that we can’t afford to run • whether new staff can be easily assimilated • whether the code can be changed easily without fear of breakage.

Something has to change • Most programmers do not have the training or experience to efficiently or correctly build most of the components of most apps. • Unless we stop assigning them such tasks we cannot improve productivity significantly. • Giving programmers frameworks and libraries is not enough; this does not fundamentally change the way applications are built. • Changing & refining methodologies is at best an exercise in low expectations.

Part 3: The future

Meaningful productivity gains are always implemented through technical management • Non-technical management cannot possibly create or enforce innovations in programmer behavior. • Since productivity-enhancing revolutions are always highly technical in nature, non-technical managers cannot implement them. • Management by the “feature list and schedule” actually impedes productivity improvement by forcing technical managers & programmers to improve products and processes at their own risk.

Henry Ford was technical manager No non-technical or non-automotive person could ever have conceived of the automotive assembly line "1913 - Trying out the new assembly line" By an unknown photographer, Detroit, Michigan, 1913 National Archives and Records Administration, Records of the Bureau of Public Roads

Steve Jobs is a technical manager No non-technical or non-computer person could ever have conceived of (or stolen) Apple’s many innovations.

Apple’s least memorable years were under the direction of a non-technical CEO You cannot substantially improve a complex technical product or product line by merely applying veneer and promotion

What is the potential here? Consider just one aspect of the change from assembly language to structured languages • Prior to structured languages, every programmer had the latitude to design his own parameter-passing conventions. • Consequently, conventions were typically different from app to app or even within the same app from function to function. • Program writers consumed a lot of time creating such conventions. • Program maintainers spent a lot of time decoding such conventions.

Let’s shoot for the ideal … In an ideal world programmers would have to think about only three things that are fundamentally different across applications: • the data model • the business rules • and the user interface How do we get there?

by getting rid of the “real world” … In the real world programmers are mostly consumed with reinventing: • • data modeling database access in the application business logic technology error handling UI component management UI design So how do we get rid of that?

First we must reconcile ourselves to an unpleasant reality Significant software productivity gains always come at the cost of developer freedoms • Assemblers prevented programmers from diddling with address bits, but rewarded them by letting them modify programs without rewriting every instruction containing an address. • Structured languages hid the vast majority of underlying code from the programmer, but improved productivity by vastly reducing the amount of code written by hand. • OO language enhancements are largely voluntarily and depend heavily upon programmer competence; consequently they have had less impact.

Innovations that do not restrict programmer freedoms have relatively little impact • Libraries create tremendous economies by eliminating the writing of some code entirely, but only for the functionality that is implemented by the library. • Methodologies arguably have little impact on overall productivity.

We get to the ideal by eliminating useless variation in software development: • A new paradigm must define a standard program architecture to be used as the basis of the vast majority of common applications (i. e. databased business apps). • A complete prototype of the standardized architecture must be the basis of application development. It is not enough to tell or show programmers how to do things; rather they need templates. • To conform the prototype to the specific needs of each application (different schemas, business rules, user interfaces), the prototype must contain code generators that generate customized code in a standardized way.

continuing … • The prototype and its generators must support standard conventions for constructing: – database tables – database code – data-layer tables & related components – business rule structure – user interface structure (especially, keeping business rules out of the UI) • The prototype should contain complete implementations of standard components that perform generalized functions (such as an objectrelational-mapper [ORM]).

Keep in mind … The objective here is not to build a system that survives for all time and defines application structure forever: • Modern car factories do not look exactly like Henry Ford’s original assembly line. His specific implementation was not the point. • The point was that the assembly line specified a very detailed plan for building a car, far more specific than anyone had previously imagined. • The assembly line demonstrated the fantastic economies that could be obtained through detailed standardization.

Recommended Reading

RAP Standard Four-Tier Architecture

Generating a table -- Person create table TBcrm. Person ( --#include "Primary. Key. sql" -- non-indexed fields Date. Of. Birth Gender Govt. Id. Number DTDate. Time, char(1) null, varchar(20) null, --#include "Common. Fields. sql" )

Describing a table row For every table, RAP generates: const string Description = DESCRIPTION You could change this to: const string Description = "person '[Govt. Id. Number]”; But we actually do this: const string Description = "person '[" + Data. Tables. TBcrm. Person. Column. Names. Govt. Id. Number + "]'";

Using a foreign key in a description RAP generates: const string Description = DESCRIPTION You could change this to: const string Description = "name '[First] [Middle] [Last]’ of ‘{Person. Id}’”; And actually we write: const string Description = "person name '[" + Data. Tables. TBcrm. Person. Name. Column. Names. First + "] [" + Data. Tables. TBcrm. Person. Name. Column. Names. Middle + "] [" + Data. Tables. TBcrm. Person. Name. Column. Names. Last + "]' of {" + Data. Tables. TBcrm. Person. Name. Column. Names. Person. Id + "}";

Describing cascading behavior For every relationship, you describe what happens to the children when the parent is deleted: and what happens to the parent when the children are deleted:

Get RAP at www. Project. Pro. com

RAP is a Linked. In group

Your speaker David Ziffer www. Project. Pro. com 630 -879 -0620 Connect w/me on Linked. In “You can’t inspect quality into a product. ” – W. Edwards Deming