The Economics and Psychology of Security Ross Anderson

The Economics and Psychology of Security Ross Anderson Cambridge University

Social Science and Security n n n The link between economics and security atrophied after WW 2 Since 2000, we have started to apply economic analysis to IT security and dependability Economic analysis often explains failure better then technical analysis! Infosec mechanisms are used increasingly to support business models (DRM, accessory control) rather than to manage risk Economic analysis is also vital for the public policy aspects of security Sociology and psychology are now engaged too

Traditional View of Infosec n n n People used to think that the Internet was insecure because of lack of features – crypto, authentication, filtering So engineers worked on providing better, cheaper security features – AES, PKI, firewalls … About 1999, we started to realize that this is not enough

Incentives and Infosec n n Electronic banking: UK banks were less liable for fraud, so ended up suffering more internal fraud and more errors Distributed denial of service: viruses now don’t attack the infected machine so much as using it to attack others Health records: hospitals, not patients, buy IT systems, so they protect hospitals’ interests rather than patient privacy Why is Microsoftware so insecure, despite market dominance?

New View of Infosec n Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives n n Bank customers suffer when poorly-designed bank systems make fraud and phishing easier Patients suffer when hospital systems break privacy Casino websites suffer when infected PCs run DDo. S attacks on them Insecurity is often what economists call an ‘externality’ – a side-effect, like environmental pollution

New Uses of Infosec n n Xerox started using authentication in ink cartridges to tie them to the printer – and its competitors soon followed Motorola then started authenticating mobile phone batteries to the phone Carmakers make ‘chipping’ harder, and plan to authenticate major components DRM: Apple grabs control of music download, MS trying to do the same for HD video content

IT Economics (1) n n n The first distinguishing characteristic of many IT product and service markets is network effects Metcalfe’s law – the value of a network is the square of the number of users Real networks – phones, fax, email Virtual networks – PC architecture versus MAC, or Symbian versus Win. CE Network effects tend to lead to dominant firm markets where the winner takes all

IT Economics (2) n n Second common feature of IT product and service markets is high fixed costs and low marginal costs Competition can drive down prices to marginal cost of production This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility … These effects can also lead to dominant-firm market structures

IT Economics (3) n n Third common feature of IT markets is that switching from one product or service to another is expensive E. g. switching from Windows to Linux means retraining staff, rewriting apps Shapiro-Varian theorem: the net present value of a software company is the total switching costs This is why so much effort goes into managing switching costs – once you have $3000 worth of songs on a $300 i. Pod, you’re locked into i. Pods

IT Economics and Security n n High fixed/low marginal costs, network effects and switching costs all tend to lead to dominantfirm markets with big first-mover advantage So time-to-market is critical Microsoft philosophy of ‘we’ll ship it Tuesday and get it right by version 3’ is not perverse behaviour by Bill Gates but quite rational Whichever company had won in the PC OS business would have done the same

IT Economics and Security (2) n n n When building a network monopoly, you must appeal to vendors of complementary products That’s application software developers in the case of PC versus Apple, or now of Symbian versus Win. CE, or Win. MP versus Real Lack of security in earlier versions of Windows made it easier to develop applications So did the choice of security technologies that dump most costs on the user (SSL, PKI, …) Once you’re a monopolist, lock it all down!

Why are so many security products ineffective? n n Akerlof’s Nobel-prizewinning paper, ‘The Market for Lemons’ introduced asymmetric information Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000 What is the equilibrium price of used cars in this town? If $1500, no good cars will be offered for sale …

Security and Liability n n Why did digital signatures not take off? Industry thought: legal uncertainty. So EU passed electronic signature law But: customers and merchants resist transfer of liability by bankers for disputed transactions If you’re a customer, best stick with credit cards, so fraud remains largely the bank’s problem

Privacy n n n Most people say they value privacy, but act otherwise. Most privacy technology firms failed Acquisti – people care about privacy when buying clothes, but not cameras (data relating to body or image are more privacy sensitive) Issue for mobile phone industry – phone viruses worse for image than PC viruses Varian – you can maybe fix privacy by giving people property rights in personal information Odlyzko – technology makes price discrimination both easier and more attractive

Why Bill wasn’t interested in security n n While Microsoft was growing, the two critical factors were speed, and appeal to application developers Security markets were over-hyped and driven by artificial factors Issues like privacy and liability were more complex than they seemed The public couldn’t tell good security from bad anyway

Why is Bill now changing his mind? n n Security can help lock customers in, and extend power from one market to another Information Rights Management changes ownership of a file from the machine owner to the file creator Remember: value of software company = total switching costs. And once documents can’t be converted without creators’ permission, the switching cost is much higher And: will WMP/Vista let Microsoft do to high definition movies what Apple did for music?

Open versus Closed? n n Are open-source systems more dependable? It’s easier for the attackers to find vulnerabilities, but also easier for the defenders to find and fix them Theory: openness helps both equally if bugs are random and standard dependability model assumptions apply Statistics: bugs are correlated in a number of real systems (‘Milk or Wine? ’) Trade-off: the gains from this, versus the risks to systems whose owners don’t patch

How Much to Spend? n n n How much should the average company spend on information security? Governments, vendors say: much more than at present! But they’ve been saying this for 20 years! Measurements of security return-oninvestment suggest about 20% p. a. overall So the total expenditure may be about right

Skewed Incentives n n n Why do large companies spend too much on security and small companies too little? Research shows there’s an adverse selection effect Corporate security managers tend to be riskaverse people, often from accounting / finance More risk-loving people may become sales or engineering staff, or small-firm entrepreneurs There’s also due-diligence, government regulation, and insurance to think of

Skewed Incentives (2) n n n If you are Dir. NSA and have a nice new hack on XP and Vista, do you tell Bill? Tell – protect 300 m Americans Don’t tell – be able to hack 400 m Europeans, 1000 m Chinese, … If the Chinese hack US systems, they keep quiet. If you hack their systems, you can brag about it to the President So offence can be favoured over defence

Large Project Failure n n Maybe 30% of large projects fail But we build much bigger failures nowadays than 30 years ago so… Why do more public-sector projects fail? Consider what the incentives are on project managers versus ministers – and what sort of people will become successful project managers versus ministers!

Security and Sociology n n n There’s a lot of interest recently in using social networks to analyse interactions and systems Barabási and Albert showed that a scale-free network could be attacked efficiently by targeting its high-order nodes Think: rulers target Saxon landlords / Ukrainian kulaks / Tutsi schoolteachers /… Can we use evolutionary game theory ideas to figure out how networks evolve? Idea: run many simulations between different attack / defence strategies

Security and Sociology (2) Vertex-order attacks with: n Black – normal (scalefree) node replenishment n Green – defenders replace high-order nodes with rings n Cyan – they use cliques (c. f. system biology …)

Psychology and Security n n n Fastest growing online crime is phishing – it only started in 2004, but by 2006 it cost the UK £ 35 m and the USA perhaps $200 m ‘Pretexting’ always existed (see Mitnick’s book), but phishing industrializes it In a company you can train the staff in operational security (though many don’t). It’s harder when the target is your users! Maybe more secure machines would inevitably drive the bad guys to target the people instead What can security folks learn from psychology?

Psychology and Security (2) n n n Security usability research is fairly new and the results are pessimistic: most security products don’t work well or at all Over half of all SSL certificates are ‘wrong’ No problem – we train people to keep on clicking ‘OK’ until they can get their work done Banks react to phishing by ‘blame and train’ efforts towards customers – but we know from the safety-critical world that this doesn’t work Systems designed by geeks discriminate against women, the elderly and the less educated

Psychology and Security (3) n Social psychology has long been relevant to us! n n n n Solomon Asch showed most people would deny the evidence of their eyes to conform to a group Stanley Milgram showed that 60% of people will do downright immoral things if ordered to Philip Zimbardo’s Stanford Prisoner Experiment showed roles and group dynamics were enough The disturbing case of ‘Officer Scott’ How can systems resist abuse of authority? Why do people need enemies? Why does terrorism work?

Psychology and Security (4) n n n Evolutionary psychology may eventually explain cognitive biases. It is based on the massive modularity hypothesis and the use of FMRI to track brain function Simon Baron-Cohen’s work on autism suggests a ‘theory of mind’ module central to empathy for others’ mental states This is how we differ from the great apes It helps us lie, and to detect lies told by others So are we really homo sapiens – or homo sapiens deceptor?

The Information Society n n n More and more goods contain software More and more industries are starting to become like the software industry The good: flexibility, rapid response The bad: frustration, poor service The ugly: monopolies How will society evolve to cope?

The Research Agenda n n We need to figure out how to balance competing social goals, as we have in the physical world Security economics gives us tools to understand what’s going on and to analyse policy options Sociology also gives some useful insights And security psychology is not just a side discipline relevant to usability and phishing – it has the potential to bring us fundamental insights, just as security economics has

More … n n n Economics and Security Resource Page – www. cl. cam. ac. uk/~rja 14/econsec. html (or follow link from www. ross-anderson. com) WEIS – Annual Workshop on Economics and Information Security – next at CMU, June 7– 8 2006 Foundation for Information Policy Research – www. fipr. org