Lecture 21 Countering Malicious Code CS 588 Cryptography

Lecture 21: Countering Malicious Code CS 588: Cryptography University of Virginia 23 April 2005 Computer Science University of Virginia CS 588 David Evans http: //www. cs. virginia. edu/evans

Menu • Reference Monitors • Java/. NET Security 23 April 2005 University of Virginia CS 588 2

Malcode Defenses • Constrain program behavior – Reference Monitors • In-line Reference Monitors • Prevent possibly harmful code from running – Safe Languages – Proof-Carrying Code 23 April 2005 University of Virginia CS 588 3

Program Execution Monitor Program Speakers Network Disk 23 April 2005 Memory University of Virginia CS 588 Super. Soaker 2000 4

Program Execution Reference Monitor Program Speakers Network Disk 23 April 2005 Memory University of Virginia CS 588 Super. Soaker 2000 5

Ideal Reference Monitor 1. Sees everything a program is about to do before it does it 2. Can instantly and completely stop program execution (or prevent action) 3. Has no other effect on the program or system Can we build this? Probably not unless we can build a time machine. . . 23 April 2005 University of Virginia CS 588 6

Real Ideal Reference Monitor most things 1. Sees everything a program is about to do before it does it 2. Can instantly and completely stop program execution (or prevent action) limited 3. Has no other effect on the program or system 23 April 2005 University of Virginia CS 588 7

Operating Systems • Provide reference monitors for most security-critical resources – When a program opens a file in Unix or Windows, the OS checks that the principal running the program can open that file • Doesn’t allow different policies for different programs • No flexibility over what is monitored – OS decides for everyone – Hence, can’t monitor inexpensive operations 23 April 2005 University of Virginia CS 588 8

Reference Monitor as Finite State Automaton [Schneider 99] All other instructions 0 All other instructions Aim 1 Aim Fire 2 All other instructions Fire STOP Policy Violation 23 April 2005 University of Virginia CS 588 9

What policies can be enforced? • Assume: – Security Automaton can see entire state of world, everything about instruction about to execute – Security Automaton has unlimited memory, can do unlimited computation • Are there interesting policies that still can’t be enforced? 23 April 2005 University of Virginia CS 588 10

What’s a Security Policy? • What’s a program? – A set of possible executions • What’s an execution? – A sequence of states • What’s a security policy? – A predicate on a set of executions 23 April 2005 University of Virginia CS 588 11

More Formally. . . • : set of all possible executions (can be infinite) • S: set of executions possible by target program S • P: security policy set of executions Boolean S is safe iff P ( S ) is true. 23 April 2005 University of Virginia CS 588 12

Reference Monitors cannot enforce all Security Policies • Some policies depend on: – Knowing about the future • If the program charges the credit card, it must eventually ship the goods – Knowing about all possible executions • Information flow – can’t tell if a program reveals secret information without knowing about other possible executions • Reference Monitors can only know about past of this particular execution 23 April 2005 University of Virginia CS 588 13

Safety Policies • Reference monitors can only enforce safety policies • Safety policy is a predicate on a prefix of states (see Schneider 98 for more formal definition) – Cannot depend on future: prefix means once it is false, it is always false – Cannot depend on other possible executions 23 April 2005 University of Virginia CS 588 14

Java Security Real or Decaf? 23 April 2005 University of Virginia CS 588 15

What is Java? A. B. C. D. E. Island in Indonesia A Programming Language (Java ) A Portable Low-Level Language (JVML) A Platform (Java. VM) A (semi-)successful marketing strategy – Java. Script is not related to Java or Java F. Work on your projects G. All of the above 23 April 2005 University of Virginia CS 588 16

Java : Programming Language “A simple, object-oriented, distributed, interpreted, robust, secure, architecture neutral, portable, high-performance, multithreaded, and dynamic language. ” [Sun 95] 23 April 2005 University of Virginia CS 588 17

What is a secure language? 1. Language is designed so it cannot express certain computations considered insecure. A few attempt to do this: PLAN, packet filters 2. Language is designed so that (accidental) program bugs are likely to be caught by the compiler or runtime environment instead of leading to security vulnerabilities. 23 April 2005 University of Virginia CS 588 18

Safe Programming Languages • Type Safety – Compiler and run-time environment ensure that bits are treated as the type they represent • Memory Safety – Compiler and run-time environment ensure that program cannot access memory outside defined storage • Control Flow Safety – Can’t jump to arbitrary addresses Which of these does C++ have? Not a new idea: LISP had these in 1960! 23 April 2005 University of Virginia CS 588 19

Java Safety • Type Safety – Most types checked statically – Coercions, array assignments type checked at run time • Memory Safety – No direct memory access (e. g. , pointers) – Primitive array type with mandatory run-time bounds checking • Control Flow Safety – Structured control flow, no arbitrary jumps 23 April 2005 University of Virginia CS 588 20

Malicious Code Can a safe programming language protect you from malcode? 1. Code your servers in it to protect from buffer overflow bugs 2. Only allow programs from untrustworthy origins to run if the are programmed in the safe language 23 April 2005 University of Virginia CS 588 21

Safe Languages? • But how can you tell program was written in the safe language? – Get the source code and compile it (most vendors, and all malicious attackers refuse to provide source code) – Special compilation service signs object files generated from the safe language (SPIN, [Bershad 96]) – Verify object files preserve safety properties of source language (Java) 23 April 2005 University of Virginia CS 588 22

JVML malcode. java Java Source Code javac Compiler malcode. class JVML Object Code Java. VM Joe User 23 April 2005 Joe wants to know JVML code satisfies Java ’s safety properties. University of Virginia CS 588 23

Does JVML satisfy Java ’s safety properties? iconst_2 istore_0 aload_0 arraylength push integer constant 2 on stack store top of stack in variable 0 as int load object reference from variable 0 replace array on top of stack with its length No! This code violates Java ’s type rules. 23 April 2005 University of Virginia CS 588 24

malcode. class JVML Object Code Bytecode Verifier Trusted Computing Base Java Bytecode Verifier Invalid “Okay” STOP Java. VM Joe User 23 April 2005 University of Virginia CS 588 25

Bytecode Verifier • Checks JVML code satisfies Java ’s safety properties • Type safe – stack and variable slots must store and load as same type • Memory safe (guaranteed by instruction set) • Control flow safe: jumps must be within function, or call/return 23 April 2005 University of Virginia CS 588 26

Are Java Bytecode Verifiers Complicated? • ~700 rules to enforce, JVML specification is (not all clearly specified) • Emin Gün Sirer found > 100 bugs in commercial bytecode verifiers (using automatic test generation) – At least 15 of them were security vulnerabilities • JVML includes jsr instruction (jump to subroutine), can be called with different types in variables and on stack 23 April 2005 University of Virginia CS 588 27

Java malcode. javac Compiler malcode. class JVML Trusted Computing Base Java Bytecode Verifier Invalid “Okay” STOP Joe User 23 April 2005 Java. VM University of Virginia CS 588 28

Java. VM • Virtual machine – interpreter for JVML programs • Has complete access to host machine • Bytecode verifier ensures some safety properties, Java. VM must ensure rest: – Type safety of run-time casts, array assignments – Memory safety: array bounds checking – Resource use policy 23 April 2005 University of Virginia CS 588 29

Java. VM Policy Enforcment [JDK 1. 0 – JDK 1. 1] From java. io. File: public boolean delete() { Security. Manager security = System. get. Security. Manager(); if (security != null) { security. check. Delete(path); } if (is. Directory()) return rmdir 0(); else return delete 0(); } 23 April 2005 University of Virginia CS 588 30

java. lang. Security. Manager /** Throws a Security. Exception if the calling thread is not allowed to delete the specified file. This method is invoked for the current security manager by the delete method of class File. */ (Some other comments deleted. ) public void check. Delete(String file) { throw new Security. Exception(); } 23 April 2005 University of Virginia CS 588 31

Security Manager • Reference monitor – How well does it satisfy the requirements? • Complete mediation • Can stop execution/prevent action • Limited effect on execution until policy violation • User/host application creates a subclass of Security. Manager to define a policy 23 April 2005 University of Virginia CS 588 32

Hot. Java’s Policy (JDK 1. 1. 7) public class Applet. Security extends Security. Manager {. . . public synchronized void check. Delete(String file) { check. Write(file); }. . . } 23 April 2005 University of Virginia CS 588 33

Applet. Security. check. Write (some exception handling code removed) public synchronized void check. Write(String file) { if (in. Applet()) { if (!init. ACL) initialize. ACLs(); String real. Path = (new File(file)). get. Canonical. Path(); for (int i = write. ACL. length ; i-- > 0 ; ) { if (real. Path. starts. With(write. ACL[i])) return; } throw new Applet. Security. Exception ("checkwrite", file, real. Path); } } Note: no checking if not in. Applet! Very important this does the right thing. 23 April 2005 University of Virginia CS 588 34

JDK 1. 0 Trust Model • When Java. VM loads a class from the CLASSPATH, it has no associated Class. Loader (can do anything) • When Java. VM loads a class from elsewhere (e. g. , the web), it has an associated Class. Loader 23 April 2005 University of Virginia CS 588 35

JDK Evolution • JDK 1. 1: Signed classes from elsewhere and have no associated Class. Loader • JDK 1. 2: – Different classes can have different policies based on Class. Loader – Explict enable/disable/check privileges – Security. Manager is now Access. Controller 23 April 2005 University of Virginia CS 588 36

What can go wrong? • Java API doesn’t call right Security. Manager checks (63 calls in java. *) – Font loading bug, synchronization • Class. Loader is tricked into loading external class as internal • Bug in Bytecode Verifier can be exploited to circumvent Security. Manager • Policy is too weak and allows damaging behavior 23 April 2005 University of Virginia CS 588 37

Java malcode. javac Compiler malcode. class JVML Trusted Computing Base Java Bytecode Verifier Invalid “Okay” STOP Joe User 23 April 2005 Java. VM University of Virginia CS 588 38

Bytecode Verifier • Checks JVML code satisfies safety properties – Simulates program execution to know types are correct, but doesn’t need to examine any instruction more than once – After code is verified, it is trusted: is not checked for type safety at run time (except for casts, array stores) Key assumption: when a value is written to a memory location, the value in that memory location is the same value when it is read. 23 April 2005 University of Virginia CS 588 39

Violating the Assumption … // The object on top of the stack is a Sim. Object astore_0 // There is a Sim. Object in location 0 aload_0 // The value on top of the stack is a Sim. Object If a cosmic ray hits the right bit of memory, between the store and load, the assumption might be wrong. 23 April 2005 University of Virginia CS 588 40

Improving the Odds • Set up memory so that a single bit error is likely to be exploitable • Mistreat the hardware memory to increase the odds that bits will flip • One bit flip allows you to violate type safety, compromise all security Sudhakar Govindavajhala and Andrew W. Appel, Using Memory Errors to Attack a Virtual Machine, July 2003. 23 April 2005 University of Virginia CS 588 41

Getting a Bit Flip • Wait for a Cosmic Ray – You have to be really, really patient… (or move machine out of Earth’s atmosphere) • X-Rays – Expensive, not enough power to generate bit-flip • High energy protons and neutrons – Work great - but, you need a particle accelerator • Hmm…. 23 April 2005 University of Virginia CS 588 42

n n 50 -watt spotlight bulb Between 80° 100°C, memory starts to have a few failures Attack applet is successful (at least half the time)! Hairdryer works too, but it frys too many bits at once Using Heat Picture from Sudhakar Govindavajhala 23 April 2005 University of Virginia CS 588 43

Should Anyone be Worried? Java virtual machine 23 April 2005 University of Virginia CS 588 44

Recap • Verifier assumes the value you write is the same value when you read it • By flipping bits, we can violate this assumption • By violating this assumption, we can violate type safety: get two references to the same storage that have inconsistent types • By violating type safety, we can get around all other security measures 23 April 2005 University of Virginia CS 588 45

. NET 23 April 2005 University of Virginia CS 588 46

Major Security Vulnerabilities (Cumulative) 50 45 http: //www. cs. virginia. edu/evans/pubs/acsac 2004. html 40 35 30 25 Java VM 20 15 10 . NET VM 5 0 23 April 2005 1996 1997 1998 University of 2000 Virginia CS 588 1999 2001 2002 2003 2004

Course Evaluations 23 April 2005 University of Virginia CS 588 48

CS 588 Pledge I will provide useful feedback. I realize this is an evolving course and it is important that I let the course staff know what they need to improve the course. I will not wait until the end of the course to make the course staff aware of any problems. I will provide feedback either anonymously … or by contacting the course staff directly. I will fill out all course evaluation surveys honestly and thoroughly. 23 April 2005 University of Virginia CS 588 49

Course Evaluations • Fill out the SEAS Evaluation on-line – It may not be anonymous, but I promise not to peek – Respond based on whether you want me to get fired or promoted • (optional) Fill out a Rate. My. Professors. com survey – Provide useful information to future students • Fill out my course-specific survey (handed out last day) – Help improve future versions of the course for later students 23 April 2005 University of Virginia CS 588 50

Charge • Links to papers on the web site: – Hair dryer attacks – Comparing Java and. NET • Thursday: – Quantum cryptography – Cryptographic hashing attacks (Boyd and Isabelle) • Tuesday: – Project presentations, final out, Sneakers 23 April 2005 University of Virginia CS 588 51