Lecture 18 Malcode Countermeasures CS 588 Security and

Lecture 18: Malcode Countermeasures CS 588: Security and Privacy David Evans University of Virginia 19 Nov 2001 University of Virginia CS 588 http: //www. cs. virginia. edu/~evans Computer Science

Menu • Reference Monitors • Java Security • Proof-Carrying Code (time permitting) • Monday: Firewalls, Intrusion Detection Movie – make sure I stop by 3: 00! 19 Nov 2001 University of Virginia CS 588 2

Malcode Defenses • Constrain program behavior – Reference Monitors • In-line Reference Monitors • Prevent possibly harmful code from running – Safe Languages – Proof-Carrying Code 19 Nov 2001 University of Virginia CS 588 3

Program Execution Monitor Program Speakers Network Disk 19 Nov 2001 Memory University of Virginia CS 588 Super. Soaker 2000 4

Program Execution Reference Monitor Program Speakers Network Disk 19 Nov 2001 Memory University of Virginia CS 588 Super. Soaker 2000 5

Ideal Reference Monitor 1. Sees everything a program is about to do before it does it 2. Can instantly and completely stop program execution (or prevent action) 3. Has no other effect on the program or system Can we build this? Probably not unless we can build a time machine. . . 19 Nov 2001 University of Virginia CS 588 6

Real Ideal Reference Monitor most things 1. Sees everything a program is about to do before it does it 2. Can instantly and completely stop program execution (or prevent action) limited 3. Has no other effect on the program or system 19 Nov 2001 University of Virginia CS 588 7

Operating Systems • Provide reference monitors for most security-critical resources – When a program opens a file in Unix or Windows, the OS checks that the principal running the program can open that file • Doesn’t allow different policies for different programs • No flexibility over what is monitored – OS decides for everyone – Hence, can’t monitor inexpensive operations 19 Nov 2001 University of Virginia CS 588 8

Reference Monitor as Finite State Automaton [Schneider 99] All other instructions 0 All other instructions Aim 1 Aim Fire 2 All other instructions Fire STOP Policy Violation 19 Nov 2001 University of Virginia CS 588 9

What policies can be enforced? • Assume: – Security Automaton can see entire state of world, everything about instruction about to execute – Security Automaton has unlimited memory, can do unlimited computation • Are there interesting policies that still can’t be enforced? 19 Nov 2001 University of Virginia CS 588 10

What’s a Security Policy? • What’s a program? – A set of possible executions • What’s an execution? – A sequence of states • What’s a security policy? – A predicate on a set of executions 19 Nov 2001 University of Virginia CS 588 11

More Formally. . . • : set of all possible executions (can be infinite) • S: set of executions possible by target program S • P: security policy set of executions Boolean S is safe iff P ( S ) is true. 19 Nov 2001 University of Virginia CS 588 12

Reference Monitors cannot enforce all Security Policies • Some policies depend on: – Knowing about the future • If the program charges the credit card, it must eventually ship the goods – Knowing about all possible executions • Information flow – can’t tell if a program reveals secret information without knowing about other possible executions • Reference Monitors can only know about past of this particular execution 19 Nov 2001 University of Virginia CS 588 13

Safety Policies • Reference monitors can only enforce safety policies • Safety policy is a predicate on a prefix of states (see Schneider 98 for more formal definition) – Cannot depend on future: prefix means once it is false, it is always false – Cannot depend on other possible executions 19 Nov 2001 University of Virginia CS 588 14

Java Security Real or Decaf? 19 Nov 2001 University of Virginia CS 588 15

What is Java? A. B. C. D. E. Island in Indonesia A Programming Language (Java ) A Portable Low-Level Language (JVML) A Platform (Java. VM) A (semi-)successful marketing strategy – Java. Script is not related to Java or Java F. Work on your projects G. All of the above 19 Nov 2001 University of Virginia CS 588 16

Java : Programming Language “A simple, object-oriented, distributed, interpreted, robust, secure, architecture neutral, portable, high-performance, multithreaded, and dynamic language. ” [Sun 95] 19 Nov 2001 University of Virginia CS 588 17

What is a secure language? 1. Language is designed so it cannot express certain computations considered insecure. A few attempt to do this: PLAN, packet filters 2. Language is designed so that (accidental) program bugs are likely to be caught by the compiler or runtime environment instead of leading to security vulnerabilities. 19 Nov 2001 University of Virginia CS 588 18

Safe Programming Languages • Type Safety – Compiler and run-time environment ensure that bits are treated as the type they represent • Memory Safety – Compiler and run-time environment ensure that program cannot access memory outside defined storage • Control Flow Safety – Can’t jump to arbitrary addresses Which of these does C++ have? Not a new idea: LISP had these in 1960! 19 Nov 2001 University of Virginia CS 588 19

Java Safety • Type Safety – Most types checked statically – Coercions, array assignments type checked at run time • Memory Safety – No direct memory access (e. g. , pointers) – Primitive array type with mandatory run-time bounds checking • Control Flow Safety – Structured control flow, no arbitrary jumps 19 Nov 2001 University of Virginia CS 588 20

Malicious Code Can a safe programming language protect you from malcode? 1. Code your servers in it to protect from buffer overflow bugs 2. Only allow programs from untrustworthy origins to run if the are programmed in the safe language 19 Nov 2001 University of Virginia CS 588 21

Safe Languages? • But how can you tell program was written in the safe language? – Get the source code and compile it (most vendors, and all malicious attackers refuse to provide source code) – Special compilation service signs object files generated from the safe language (SPIN, [Bershad 96]) – Verify object files preserve safety properties of source language (Java) 19 Nov 2001 University of Virginia CS 588 22

JVML malcode. java Java Source Code javac Compiler malcode. class JVML Object Code Java. VM Joe User 19 Nov 2001 Joe wants to know JVML code satisfies Java ’s safety properties. University of Virginia CS 588 23

Does JVML satisfy Java ’s safety properties? iconst_2 istore_0 aload_0 arraylength push integer constant 2 on stack store top of stack in variable 0 as int load object reference from variable 0 replace array on top of stack with its length No! This code violates Java ’s type rules. 19 Nov 2001 University of Virginia CS 588 24

malcode. class JVML Object Code Bytecode Verifier Trusted Computing Base Java Bytecode Verifier Invalid “Okay” STOP Java. VM Joe User 19 Nov 2001 University of Virginia CS 588 25

Bytecode Verifier • Checks JVML code satisfies Java ’s safety properties • Type safe – stack and variable slots must store and load as same type • Memory safe (guaranteed by instruction set) • Control flow safe: jumps must be within function, or call/return 19 Nov 2001 University of Virginia CS 588 26

Are Java Bytecode Verifiers Complicated? • ~700 rules to enforce, JVML specification is (not all clearly specified) • Emin Gün Sirer found > 100 bugs in commercial bytecode verifiers (using automatic test generation) – At least 15 of them were security vulnerabilities • JVML includes jsr instruction (jump to subroutine), can be called with different types in variables and on stack 19 Nov 2001 University of Virginia CS 588 27

Java malcode. javac Compiler malcode. class JVML Trusted Computing Base Java Bytecode Verifier Invalid “Okay” STOP Joe User 19 Nov 2001 Java. VM University of Virginia CS 588 28

Java. VM • Virtual machine – interpreter for JVML programs • Has complete access to host machine • Bytecode verifier ensures some safety properties, Java. VM must ensure rest: – Type safety of run-time casts, array assignments – Memory safety: array bounds checking – Resource use policy 19 Nov 2001 University of Virginia CS 588 29

Java. VM Policy Enforcment [JDK 1. 0 – JDK 1. 1] From java. io. File: public boolean delete() { Security. Manager security = System. get. Security. Manager(); if (security != null) { security. check. Delete(path); } if (is. Directory()) return rmdir 0(); else return delete 0(); } 19 Nov 2001 University of Virginia CS 588 30

java. lang. Security. Manager /** Throws a Security. Exception if the calling thread is not allowed to delete the specified file. This method is invoked for the current security manager by the delete method of class File. */ (Some other comments deleted. ) public void check. Delete(String file) { throw new Security. Exception(); } 19 Nov 2001 University of Virginia CS 588 31

Security Manager • Reference monitor – How well does it satisfy the requirements? • Complete mediation • Can stop execution/prevent action • Limited effect on execution until policy violation • User/host application creates a subclass of Security. Manager to define a policy 19 Nov 2001 University of Virginia CS 588 32

Hot. Java’s Policy (JDK 1. 1. 7) public class Applet. Security extends Security. Manager {. . . public synchronized void check. Delete(String file) { check. Write(file); }. . . } 19 Nov 2001 University of Virginia CS 588 33

Applet. Security. check. Write (some exception handling code removed) public synchronized void check. Write(String file) { if (in. Applet()) { if (!init. ACL) initialize. ACLs(); String real. Path = (new File(file)). get. Canonical. Path(); for (int i = write. ACL. length ; i-- > 0 ; ) { if (real. Path. starts. With(write. ACL[i])) return; } throw new Applet. Security. Exception ("checkwrite", file, real. Path); } } Note: no checking if not in. Applet! Very important this does the right thing. 19 Nov 2001 University of Virginia CS 588 34

in. Applet boolean in. Applet() { return in. Class. Loader(); } Inherited from java. lang. Security. Manager: protected boolean in. Class. Loader() { return current. Class. Loader() != null; } 19 Nov 2001 University of Virginia CS 588 35

current. Class. Loader /** Returns an object describing the most recent class loader executing on the stack. Returns the class loader of the most recent occurrence on the stack of a method from a class defined using a class loader; returns null if there is no occurrence on the stack of a method from a class defined using a class loader. */ protected native Class. Loader current. Class. Loader(); 19 Nov 2001 University of Virginia CS 588 36

Recap • java. io. File. delete calls before deleting • Hot. Java overrides Security. Manager with Applet. Security to set policy • Applet. Security. check. Delete calls Security. Manager. check. Delete Applet. Security. check. Write • Applet. Security. check. Write checks if any method on stack has a Class. Loader • If not no checks; if it does, checks ACL list 19 Nov 2001 University of Virginia CS 588 37

JDK 1. 0 Trust Model • When Java. VM loads a class from the CLASSPATH, it has no associated Class. Loader (can do anything) • When Java. VM loads a class from elsewhere (e. g. , the web), it has an associated Class. Loader 19 Nov 2001 University of Virginia CS 588 38

JDK Evolution • JDK 1. 1: Signed classes from elsewhere and have no associated Class. Loader • JDK 1. 2: – Different classes can have different policies based on Class. Loader – Explict enable/disable/check privileges – Security. Manager is now Access. Controller 19 Nov 2001 University of Virginia CS 588 39

What can go wrong? • Java API doesn’t call right Security. Manager checks (63 calls in java. *) – Font loading bug, synchronization • Class. Loader is tricked into loading external class as internal • Bug in Bytecode Verifier can be exploited to circumvent Security. Manager • Policy is too weak and allows damaging behavior 19 Nov 2001 University of Virginia CS 588 40

Hostile Applets • See http: //java. sun. com/sfaq/chronology. html (about 1 new vulnerability/month) • Easy to write “annoying” applets (policy is too imprecise; no way to constrain many resource operations) • http: //www. cigital. com/hostileapplets/index. html 19 Nov 2001 University of Virginia CS 588 41

Proof-Carrying Code 19 Nov 2001 University of Virginia CS 588 42

Maze (From http: //www. cedillasys. com/pages/technotes/) 19 Nov 2001 University of Virginia CS 588 43

Proof-Carrying Maze (From http: //www. cedillasys. com/pages/technotes/) 19 Nov 2001 University of Virginia CS 588 44

Proof-Carrying Code • Guarantee properties of untrustworthy code by checking a proof provided by code producer • Creating a proof is hard – Have to make up invariants, choose cases, pick strategies, etc. • Checking a proof is easy – Simple mechanical application of rules 19 Nov 2001 University of Virginia CS 588 45

Have we seen anything like this? • Java Bytecode Verifier is a simple instance of PCC: – Bytecodes include extra information on typing, stack use, etc. – Bytecode verifier checks it to enforce lowlevel code safety properties 19 Nov 2001 University of Virginia CS 588 46

Fortune Cookie “That which must can be proved cannot be worth much. ” Fortune cookie quoted on Peter’s web page • True for all users • True for all executions • Exception: Low-level code safety 19 Nov 2001 University of Virginia CS 588 47

Reasons you might prefer PCC • Run-time performance? – Amortizes additional download and verification time only rarely – SFI Performance penalty: ~5% • If you care, pay $20 more for a better processor or wait 5 weeks • Smaller Trusted Computing Base? – Not really smaller: twice as big as SFI (Touchstone VCGen+checker – 8300 lines / Mis. Fi. T x 86 SFI implementation – 4500 lines) • You are a vendor who cares more about quality than time to market – Not really PCC (not across a trust boundary) 19 Nov 2001 University of Virginia CS 588 48

Feedback Slips 19 Nov 2001 University of Virginia CS 588 49

You are getting a strip of paper with your recorded grades so far (check they are correct), and my recommendation for PS 5/project: You’re doing fine (but keep working on your project). Enjoy your Thanksgiving. You’re doing okay, but go easy on the Cranberry Sauce. You should either do PS 5 or work a lot on your project. You need to get cracking to avoid a mediocre grade (or worse) in this course. You should do PS 5 and work on your project. No one got this. Still plenty of time to do well in the course. 19 Nov 2001 University of Virginia CS 588 50

Movie 19 Nov 2001 University of Virginia CS 588 51