Why Programming Matters Eric Roberts Professor of Computer

Why Programming Matters Eric Roberts Professor of Computer Science, Stanford University Past Chair of the ACM Education Board NACCQ Napier, New Zealand 11 July 2009

Outline • In recent years, there have been calls to reduce or eliminate programming from the computer science curriculum. The reasons offered in support of such proposals include: – Students increasingly seem to find programming both distasteful and difficult, which leads them to pursue other subjects. – There are more jobs in the computing field that do not require programming, allowing students to succeed without this skill. – Economic considerations have led to increased offshoring of software, reducing job opportunities in the developed countries. • While there is some truth in each of these arguments, I am convinced that reducing curricular emphasis on programming would be a serious mistake with enormous economic costs. • What we need to do instead is improve both the pedagogy and the practice of programming to restore the “passion, beauty, joy and awe” that Grady Booch described at SIGCSE 2007.

The Importance of Computing: A Story FINANCIAL TIMES Friday, November 12, 1999 CLUSTERS: The science of sizing up the opposition The clusters have grown but silicon fen still lacks the large companies of big rivals writes Brian Groom. Hermann Hauser, venture capitalist and co-founder of 20 Cambridge companies, recently attended a conference of European high technology executives. The last question was: where is the silicon valley of Europe? . . . Cambridge has made great strides but is not yet in the silicon valley league. “Stanford alumni have produced companies worth a trillion dollars. Massachusetts Institute of Technology half a trillion dollars. If Cambridge is getting to $20 bn we will be lucky”.

And Parliament Responds • In 2000, a delegation of nine Members of the British Parliament visited Stanford to talk about how they might transform Cambridge into the sort of economic engine Stanford has been for many years. • In my meeting with the MPs, I focused on Stanford’s “broad audience” approach to teaching computer science, in which we try to give as many students as we can a serious introduction to the field, in the hopes that at least some of them go on. • At the time, Stanford was doing well: – 75% of undergraduates took a CS module – 12% of all first degrees were in CS – CS attracted many of Stanford’s best students Barry Sheerman (chairman) Huddersfield, Labour Charlotte Atkins Staffordshire Moorlands, Labour Valerie Davey Bristol West, Labour Michael Foster Worcester, Labour Helen Jones Warrington North, Labour Gordon Marsden Blackpool South, Labour Dr Evan Harris Oxford West and Abingdon, Liberal Democrat Stephen O’Brien Eddisbury, Conservative Nick St Aubyn Guildford, Conservative

But the Timing Was Less than Perfect • The dot-com collapse hit computing enrolments very hard. – At Stanford, CS enrolments fell by 37% from 2000 to 2006. – The Computing Research Association (CRA) estimates that computing enrolment in the United States fell by almost 50% over that same period. – Similar declines have been reported in most developed countries. • In the last few years, enrolments have again been rising. – Stanford’s enrolment has risen by almost 20% in each of the last two years. – The CRA recently reported that the numbers for the United States grew slightly in the last year. • Despite this good news, the rate of degree production remains too slow to meet industrial demand.

The Need for Greater Understanding I have not seen a compelling narrative for the decline in student interest. — Bill Gates, Stanford University, February 19, 2008 • The failure of universities to produce a sufficient number of graduates with the necessary computing skills is now widely recognized as a crisis in both academia and industry. • Unfortunately, the underlying causes for the decline in student interest are not well understood. Although several theories seek to explain the decline in student interest, they do not provide a comprehensive explanation of student behavior. • For the last several years, I’ve been working to develop a “compelling narrative” of the sort Bill Gates described during his visit to Stanford. That narrative is as yet incomplete, and I welcome any comments and criticisms.

Possible Reasons for the Decline 1. Fears about job security in the wake of redundancies and offshoring continue to have a profound effect on student interest in our discipline.

But Industry Demand Remains High • The computing industry offers some of the best employment opportunities for university graduates. The following data are from the United States, but the situation is similar elsewhere: – The number of jobs in the domestic software industry are at an all-time high and are projected to grow dramatically over the next decade. – Salaries for newly minted B. S. graduates in Computer Employment Science are high, sometimes exceeding the $100, 000 mark. (thousands) Growth Top 10 job growth categories (2006 -2016) 2006 2016 – In 2005, Money magazine rated software engineer as the number one 1. Network systems and data communications analysts 262 402 53. 4 job in America. • 2. Personal and home care aides 767 1, 156 50. 6 3. Home healthenough aides 787 1, 171 skills 48. 7 is Producing students with strong computing 4. Computer software engineers, that applications 507 competitiveness 733 44. 6 essential for any nation wants to maintain Veterinary technologists and technicians 71 100 41. 0 Google and Facebook economy. are fighting hard to hire this year’s in 5. today’s global crop of computer graduates, we’ve heard, and 6. Personal financialscience advisors 176 248 41. 0 ground zero is Stanford. Most of the class of 2008 already 7. Makeup theatrical and performance 2 3 39. 8 have job artists, offers even though graduation is months away. Last year, salaries of up to $70, 000 were common for the 8. Medical assistants 465 148 35. 4 best students. This year, Facebook is said to be offering $92, 000, and Google has increased some offers to $95, 000 62 84 35. 0 9. Veterinarians to get their share of graduates. Students with a Master’s degree in Computer Science are being offered as much as 10. Substance abuse and behavioral disorder counselors 83 112 34. 3 $130, 000 for associate product manager jobs at Google. Source: U. S. Department of Labor, Bureau of Labor Statistics, Employment Projections: 2006 -16, December 2007.

Degree Production vs. Job Openings 160, 000 140, 000 120, 000 100, 000 Ph. D. Master’s Bachelor’s Projected job openings 80, 000 60, 000 40, 000 20, 000 Engineering Physical Sciences Biological Sciences Computer Science Sources: Adapted from a presentation by John Sargent, Senior Policy Analyst, Department of Commerce, at the CRA Computing Research Summit, February 23, 2004. Original sources listed as National Science Foundation/Division of Science Resources Statistics; degree data from Department of Education/National Center for Education Statistics: Integrated Postsecondary Education Data System Completions Survey; and NSF/SRS; Survey of Earned Doctorates; and Projected Annual Average Job Openings derived from Department of Commerce (Office of Technology Policy) analysis of Bureau of Labor Statistics 2002 -2012 projections. See http: //www. cra. org/govaffairs/content. php? cid=22.

A Thought Experiment about Offshoring • Suppose that you are Microsoft and that you can hire a software developer from Stanford whose loaded costs will be $200, 000 per year. Over in Bangalore, however, you can hire a software developer for $75, 000 per year. Both are equally talented and will create $1, 000 annually in value. What do you do? • Although the developer in Bangalore has a higher return, the optimal strategy is to hire them both. After all, why throw away $800, 000 a year? • Any elementary economics textbook will explain that one hires as long as the marginal value of the new employee is greater than the marginal cost. The essential point is that companies seek to maximize return, and not simply to minimize cost.

Possible Reasons for the Decline 1. Fears about job security in the wake of redundancies and offshoring continue to have a profound effect on student interest in our discipline. 2. The kind of exposure students get to computing at the elementary and secondary level tends to push people away from the discipline long before they reach the university.

The Problem Starts Early The UCLA HERI study shows that students have already made their decisions before they reach university. Source: Higher Education Research Institute at UCLA, 2005

Computing Faces Challenges in Schools • People who have software development skills command high salaries and tend not to teach in schools. • In many schools, computing courses are seen as vocational rather than academic. • Students who are heading toward top universities are advised to take non-CS courses to bolster their admissions chances. • Because schools are evaluated on how well their students perform in math and science, schools are shifting teachers away from computer science toward these disciplines. Those disciplines, moreover, often actively oppose expanding high-school computer science. • Administrators find tools like Power. Point more sexy and exciting. J • Computing skills in general—and programming in particular—have become much harder to teach. • Teachers have few resources to keep abreast of changes in the field.

Possible Reasons for the Decline 1. Fears about job security in the wake of redundancies and offshoring continue to have a profound effect on student interest in our discipline. 2. The kind of exposure students get to computing at the elementary and secondary level tends to push people away from the discipline long before they reach the university. 3. The image of work in the field—and, more importantly, all too much of the reality of work in the field—is unattractive to most students and no longer seems fun, particularly in comparison to other opportunities that bright students might pursue.

Possible Reasons for the Decline 1. Fears about job security in the wake of redundancies and offshoring continue to have a profound effect on student interest in our discipline. 2. The kind of exposure students get to computing at the elementary and secondary level tends to push people away from the discipline long before they reach the university. 3. The image of work in the field—and, more importantly, all too much of the reality of work in the field—is unattractive to most students and no longer seems fun, particularly in comparison to other opportunities that bright students might pursue. 4. The university curriculum is badly broken and needs a radical overhaul.

The Curriculum Has a Second-Order Effect • The computing curriculum as traditionally implemented has deficiencies and can always be improved. • As an explanation for declining enrolments, however, the “curriculum is broken” theory has serious shortcomings: 1. It cannot explain why enrolments have varied so much over time. 2. It fails to account for the fact that institutions saw a similar loss of enrolment even when their curricula were different. Most of the proposed curriculum improvements were in place somewhere in 2000 -01, but declines occurred everywhere. The resurgence of enrolment in the last year also seems independent of curriculum. 3. Students decide to avoid computing long before they have any idea what the university curriculum is. 4. Students who take our courses tend to like them but still shy away from the computer science major.

Students Like Our Courses But Go Elsewhere Gateway Course for Course Management for Computer Science & Engineering Computer Science Management Science & Engineering student flow

Possible Reasons for the Decline 1. Fears about job security in the wake of redundancies and offshoring continue to have a profound effect on student interest in our discipline. 2. The kind of exposure students get to computing at the elementary and secondary level tends to push people away from the discipline long before they reach the university. 3. The image of work in the field—and, more importantly, all too much of the reality of work in the field—is unattractive to most students and no longer seems fun, particularly in comparison to other opportunities that bright students might pursue.

The Real Image Problem Dilbert “The Knack” http: //www. youtube. com/watch? v=Cm. YDgnc. Mh. Xw

The Reality Is Also a Problem Has anyone considered the possibility that it’s just not fun anymore? —Don Knuth, October 11, 2006 • Students at Stanford have expressed the following concerns: – Long hours with little chance for a balanced life – A less pleasant social milieu than other occupations

The Code-Rush Syndrome From March 1998 to April 1999 an independent documentary film crew followed a team of software engineers at Netscape Communications as they lived through a watershed moment in the brief history of their company and the Internet. Code Rush, WETA television, 2001

The Reality Is Also a Problem Has anyone considered the possibility that it’s just not fun anymore? —Don Knuth, October 11, 2006 • Students at Stanford have expressed the following concerns: – Long hours with little chance for a balanced life – A less pleasant social milieu than other occupations – A sense that success in programming is possible only for those who are much brighter than they see themselves to be – Work that is often repetitive and unchallenging, particularly when it involves maintaining legacy technology – Work that has become more difficult than it used to be – No chance for a lasting impact because of rapid obsolescence – Fears that employment with an individual company is dicey even though opportunities are good in the industry as a whole – Frustration at being managed by nontechnical people who make more money but are not as bright (Dilbert’s boss)

Dilbert vs. the Boss Capital Labour Interestingly, during the bubble of 2001 to 2007, students with strong technical skills opted for careers in management or finance, giving them a seemingly secure place on the capital side of the capital/labour divide.

For Some, Programming Is the Enemy • Because students respond more negatively to programming than they once did, some have argued for reducing its emphasis in the university curriculum. • As an illustration of this trend, consider the following post that appeared on SIGCSE-MEMBERS on August 14, 2006: I have an idea for a panel that I’d like to organize for SIGCSE’ 07. I’m asking for volunteers (or nominations of others) to serve on the panel. The panel I’d like to organize would have a title something like: “Alternative Models for a Programming-lite Computer Science Curriculum” The theme of the panel would be to share ideas and thoughts on how we might reduce (or eliminate) the emphasis on programming within a computer science curriculum. The basic idea is to cause discussion centered on the knowledge and skills students of tomorrow will need in the global economic workspace and the implications for the CS curriculum. As more and more aspects of software development of “offshored”, what kind of curriculum would allow a student to be successful in the IT field? We have met the enemy and he is us. — Walt Kelly

Industry Is Not Amused • Every technical person in the industry with whom I’ve spoken is horrified by the prospect of reducing the emphasis on programming in the undergraduate curriculum. • Employers in developed countries with high-tech sectors are desperate for more people with programming talent. In his keynote at ITi. CSE 2007 in Dundee, Scottish entrepreneur Chris van der Kuyl said that the lack of programming talent was the greatest limiting factor in the industry. He called it coding. • At meetings of the ACM Education Council in recent years, industry representatives have been united in their concern about the scarcity of competent software developers. • In a new graduate-level course taught at Stanford by Monica Lam and Martin Rinard, the course description begins like this: “The key to delivering a successful software project is to hire top notch programmers. ”

Industry Reports a Labor Shortage — April 28, 2005 Gates Cites Hiring Woes, Criticizes Visa Restrictions By David A. Vise Microsoft Corp. Chairman Bill Gates said yesterday the software giant is having enormous difficulty filling computer jobs in the United States as a result of tight visa restrictions on foreign workers and a declining interest among U. _S. students in computer science. <tab>Speaking on a technology panel at the Library of Congress, Gates said a decline in the number of U. _S. students pursuing careers in science and technology is hurting Microsoft in the short run, and could have serious long-term consequences for the U. _S. economy if the problem is not addressed. <tab>“We are very concerned that the U. _S. will lose its competitive position. For Microsoft, it means we are having a tougher time hiring, ” Gates said. “The jobs are there, and they are good-paying jobs, but we don’t have the same pipeline. ” http: //www. washingtonpost. com/wp-dyn/content/article/2005/04/27/AR 2005042702241. html

But There Are Contrary Arguments In the July 2008 Communications of the ACM, Villanova professor Stephen Andriole and I debated the relevance of programming in the computing curriculum. Steve argues that the commodification of enterprise software means that there is no longer as much need for programming expertise in that sector. I respond that this sector is only one of many, and that the demand for employees with good programming skills is both large and growing.

Programming Remains Central • Calls to “reduce or eliminate” programming from computing curricula arise from some undeniable assumptions: – There are more jobs in IT that don’t require programming. – Programming is not particularly popular with students today. – Offshoring of programming jobs has increased. • Unfortunately, this analysis ignores the following equally valid propositions: – – There are more jobs in IT that do require programming. Programming has historically been what attracts students the most. Globalization has created more IT jobs in India/China and the U. S. Offshoring exists largely because of a shortfall of skilled employees.

Critical Observations about Software 1. Software development is an extraordinarily difficult task, exceeding in complexity most other engineering work. That difficulty, moreover, is intrinsic to the discipline and is not likely to change in the foreseeable future. 2. Despite advances in tools and methodologies (and because of those advances to some extent), software development and software education have become more difficult over the history of the field. 3. Software development requires people with an unusual combination of skills. Those people are in short supply, but their economic value is enormous. Experienced programmers differ in productivity by several orders of magnitude.

Essential and Accidental Complexity To see what rate of progress one can expect in software technology, let us examine the difficulties of that technology. Following Aristotle, I divide them into essence, the difficulties inherent in the nature of software, and accidents, those difficulties that today attend its production but are not inherent. . The complexity of software is an essential property not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence. —Fred Brooks “No Silver Bullet” IEEE Computer, April 1987

Critical Observations about Software 1. Software development is an extraordinarily difficult task, exceeding in complexity most other engineering work. That difficulty, moreover, is intrinsic to the discipline and is not likely to change in the foreseeable future. 2. Despite advances in tools and methodologies (and because of those advances to some extent), software development and software education have become more difficult over the history of the field. 3. Software development requires people with an unusual combination of skills. Those people are in short supply, but their economic value is enormous. Experienced programmers differ in productivity by several orders of magnitude.

The March of Progress 1536 pages 911 pages 266 pages 274 pages

Sobering Thoughts • There are more public methods in the java and javax package hierarchies than there are words in Jensen and Wirth’s Pascal User Manual and Report. The amount of explanation once deemed sufficient to teach the standard introductory programming language is thus no longer sufficient for an index of the operations available today. • Typical software today exists at a level of scale and complexity that would have been unthinkable a generation ago. The most common operating system used in mobile phones, for example, contains approximately 100 million lines of code. • Given the scale of modern software systems, it is typically impossible for students to develop projects as extensions to existing code frameworks. An academic term is now barely sufficient to understand what is already there, leaving no time for further development?

An Illustrative Example In their book Java Puzzler, Josh Bloch and Neal Gafter pose a variety of puzzles to test the reader’s understanding of Java, including the following: Students are regularly taught that the statements x++; and x += 1; are equivalent. In Java, describe a situation in which the first statement is legal, but not the second. How about the other way around?

Java Scoping If you execute the program shown at the right in Java, what output will it generate? Why? public class Scope. Test { public static void main(String[] args) { C 1 obj = new C 2(); System. out. println(obj. fn(obj)); } } class C 1 { public static final int N = 1; public int fn(C 1 obj) { return 10 + obj. N; } } class C 2 extends C 1 { public static final int N = 2; public int fn(C 1 obj) { return 20 + obj. N; } }

Critical Observations about Software 1. Software development is an extraordinarily difficult task, exceeding in complexity most other engineering work. That difficulty, moreover, is intrinsic to the discipline and is not likely to change in the foreseeable future. 2. Despite advances in tools and methodologies (and because of those advances to some extent), software development and software education have become more difficult over the history of the field. 3. Software development requires people with an unusual combination of skills. Those people are in short supply, but their economic value is enormous. Experienced programmers differ in productivity by several orders of magnitude.

Variations in Programmer Productivity • In 1968, a study by Sackman, Erikson, and Grant revealed that programmers with the same level of experience exhibit variations of more than 20 to 1 in the time required to solve particular programming problems. • More recent studies [Curtis 1981, De. Marco and Lister 1985, Brian 1997] confirm this high variability. • Most industry insiders believe that the productivity variance is even higher today. In 2005, Google’s VP for Engineering, Alan Eustace, told The Wall Street Journal that one top-notch engineer is worth 300 times or more than the average.

The Microsoft Programming Personae Microsoft’s cultural lore defines three types of programmers: Mort is your most common developer, who doesn’t have a CS background, may even be a recent newcomer, and doesn’t quite understand what the computer is doing under the covers, but who writes the dinky IT programs that make businesses run. Elvis, more knowledgeable, cares about code quality, but has a life too. Einstein writes some serious-ass piece of code like device drivers, wants to get things done, needs to be able to go low level and high level, needs a language without restrictions to get his job done. — Wesner Moise, “Who are you? Mort, Elvis or Einstein, ” September 25, 2003 http: //wesnerm. blogs. com/net_undocumented/2003/09/who_are_you_mor. html For the most part, Microsoft (along with Google and other first-rank companies) are seeking to hire the Einsteins, which explains the low hiring ratio.

Productivity Variations Are Common • The idea that individuals might differ in productivity by two or more orders of magnitude initially seems hard to believe. • In fact, such differences in effectiveness occur across a range of occupational categories: – – – Mathematicians Creative artists (writers, composers, poets, painters) Entertainment (musicians, actors, motion-picture directors) Financial wizards, CEOs Professional athletes • No one achieves mastery in these fields through talent alone. Training and practice are essential. As Malcolm Gladwell notes in Outliers, mastery of any field typically requires 10, 000 hours.

Paul Graham’s Hackers and Painters When I finished grad school in computer science I went to art school to study painting. A lot of people seemed surprised that someone interested in computers would also be interested in painting. They seemed to think that hacking and painting were very different kinds of work—that hacking was cold, precise, and methodical, and that painting was the frenzied expression of some primal urge. <tab>Both of these images are wrong. Hacking and painting have a lot in common. In fact, of all the different types of people I’ve known, hackers and painters are among the most alike. <tab>What hackers and painters have in common is that they’re both makers. Along with composers, architects, and writers, what hackers and painters are trying to do is make good things. http: //www. paulgraham. com/hp. html Paul Graham (photo by Niall Kennedy)

Alternative Models of Software Education If software development is really an art, it might make sense to create “conservatories” for the teaching of software arts, similar to music conservatories. One possibility might be some sort of New England Conservatory of Coding. (Or perhaps a Hogwarts School for Software Wizardry. ) SEMESTER AT C++ SEA Another model might be to create intensive programs that encourage students to focus on the art of software development, in much the same way that programs like the University of Virginia’s Semester at Sea program offers a concentrated immersion in oceanography, geography, and cultural anthropology.

Three Student Personae Taking a cue from Microsoft’s classification of programmers, it is interesting to think about different students that computer science attracts: • In the boom years, we tended to attract the entrepreneur, lured by the dream of start-up riches. That student now heads for finance. • Most academic programs tend to focus on creating the engineer, but that lifestyle strikes today’s students as boring. • We need to attract the artist—someone who can appreciate the passion, beauty, joy, and awe that make our field exciting. And consider not only how much easier it is to find female artists to serve as role models but also how much easier it is to convince women that the lifestyle of an artist might be attractive. Georgie Dora Mary Frida Emily Carrington Cassatt O’Keeffe Kahlo Carr

Knuth’s Turing Award Lecture (1974) <tab>Implicit in these remarks is began the notion that there is When Communications of the ACM publication in 1959, something undesirable an area of human that is the members of ACM’sabout Editorial Board made activity the following classified andescribed “art”; it has be a Science before it has any remark as as they the to purposes of ACM’s periodicals: real stature. On the other hand, I have been working for more is called to become an of important than“If 12 computer years on aprogramming series of books “The Art Computer part of computer research and development, transition Programming. ” People frequently ask me why I apicked such a of and programming from an art to a disciplined science title; in fact some people apparently don’t believe that I mustdid be effected. ” really so, since I’ve seen at least one bibliographic reference to some books called “The Act of Computer <tab>Such a goal has been a continually recurring theme during Programming. ” the ensuing years; for example, we read in 1970 of the “first <tab>In this talk I shall try to explain why I think “Art” is the steps towardword. transforming the art into a appropriate I will discuss whatofit programming means for something science. ” Meanwhile have actually inexamine making to be an art, in contrastwe to being a science; succeeded I will try to our discipline science, andorinbad a remarkably way: whether arts area good things; and simple I will try to merely by deciding to call it “computer science. ” show that a proper viewpoint of the subject will help us all to improve the quality of what we are now doing. http: //doi. acm. org/10. 1145/361604. 361612 Don Knuth ACM Turing Award 1974

What We Need To Do • Recognize that the problems extend well beyond the university. • Press government and industry to improve computing education in the schools, possibly through public-private partnerships. • Take creative steps to bolster both the image and the reality of work in the profession. • Make it clear to students (as well as faculty) that programming remains essential to much of the work in the field. • Emphasize the “beauty” of programming by focusing more attention on software as an art. • Explore new styles of pedagogy that are more finely attuned to artistic domains. Beauty is truth, truth beauty, that is all Ye know on earth, and all ye need to know. — John Keats, “Ode on a Grecian Urn, ” 1819

The End