490 dp Part I Challenges Intermezzo Applications Part

490 dp Part I: Challenges Intermezzo: Applications Part II: Java Object Serialization Robert Grimm

Challenges • Pervasive computing – Vision: Focus on users and their tasks – Enabled by ubiquitous smart devices • Central question – How can devices get users’ tasks done? • They need to work together!

Distributed State “Information retained in one place that describes something, or is determined by something, somewhere else in the system” • Examples – Association between addresses and names – Sequence number to identify most recent data – File block cached in memory of client – List of clients caching a file

Why is Distributed State Good? • Performance – Not going over the network saves time – Example: Local cache of files • Coherency – Easier to coordinate based on knowledge – Example: Server notification when cache expires • Reliability – Replication makes it possible to tolerate failures – Example: Same files stored on two servers

Why is Distributed State Bad? • • Consistency Crash sensitivity Time and space overheads Complexity

Consistency • Problem: Keep copies consistent • Approaches – Detect stale data on use • Treat copy as hint • Example: name-to-address map – Prevent inconsistency • Require exclusive ownership before modifying • Example: all operations go through one node – Tolerate inconsistency • Make window of inconsistency small • Example: delays in network games

Crash Sensitivity • Problem: Mask failures • Approaches – Reconstruct state • Example: reopening files in Sprite FS – Limit degree of distribution / affected state • Example: partition files according to usage – Fully replicate state • Example: Coda file system

Time and Space Overheads • Time – Go across the network • Space – Distributed copies – Tracking distributed copies • Overheads depend on – Degree of sharing – Degree of modification

Complexity • Distributed state requires – Maintaining consistency – Masking of failures • Distributed state makes it harder to – Debug – Tune

Trade-offs • • Consistency Availability Scalability Complexity

No Perfect Solution NFS Sprite FS Consistency Limited Ö Availability Limited Scalability Limited (Baseline OK) Complexity Low High • Solution needs to be “good enough”

What Means “Good Enough” • Depends on application domain – Make an informed trade-off • Examples – Cluster-based services • Porcupine • Distributed Data Structures – Disconnected storage services • Epidemic replication • Two-tier replication

Porcupine • Cluster-based email server • Assumptions – Email typically doesn’t get modified – Deleted emails may reappear (temporarily) • Eventual consistency • But, availability and scalability [Saito et al. 99]

Distributed Data Structures • Cluster-based hash table • Assumptions – Network is fast and doesn’t partition – Nodes fail infrequently – OK to return failure at storage layer • Consistency, availability, and scalability [Gribble et al. 00]

Bayou • Epidemic replication [Demers et al. 87] – Two nodes periodically synchronize state – Only pair-wise connectivity • Structured storage (database) • Eventually consistent • But, always available [Petersen et al. 97]

Coda • First-tier nodes – Fully connected – Store all data • Second-tier nodes – Often disconnected – Store subset of data • Limited consistency, but greater availability [Kistler & Satya 92, Mummert et al. 95]

Conflicts • Caused by competing updates • Detected “after the fact” • Need to be resolved automatically

Conflict Resolution Techniques • Based on data – Timestamps – Heuristics – Programs [Kumar & Satya 95, Reiher et al. 94] • Part of update: Bayou [Terry et al. 95] – Dependency check – Merge procedure

Morals • No perfect solution – Need to exploit application domain • Complexity grows very quickly – Beware of special case code (recovery)

Intermezzo: Applications • Team 1: Cluster-based application – Scalable Napster / Gnutella repository – Scalable document repository • Leased storage • Customizable actions when leases expire • Team 2: Roving application – Personal jukebox – PIM on steroids – Universal inbox

Break

Java Object Serialization • Problem – Turn graph of objects into byte string – Turn byte string back into graph of objects A B A C D B C D

The Basic Idea • Write a description of each object • Keep track of each written object A A B C D D B C D • <1: A <2: B <3: D>> <4: C ref(3)>>

All Things Serializable • Not everything is serializable – java. lang. Object – java. lang. Thread • Serializable objects implement java. io. Serializable – An empty marker interface

Default Serialization • Writes out all fields – Independent of their access controls (private, package private, protected, public) • Good style to document invariants – Use @serial tag @serial Must not be <code>null</code>

Default Deserialization • Allocates memory for new object – No constructor invoked – Fields initialized to their default values • Reads in all fields – Independent of their access controls

Transient Fields • Some fields shouldn’t or can’t be serialized private Object lock; • How to prevent default serialization from trying to write them out? – Declare such fields as “transient” private transient Object lock; – Restored to defaults during deserialization • null in above example

Overriding Serialization • Customize serialization by implementing private void write. Object(Object. Output. Stream) throws IOException; • Good style to document customization – Use @serial. Data tag

Overriding Serialization • Example: Thread-safe serialization private void write. Object(Object. Output. Stream out) throws IOException { synchronized (lock) { out. default. Write. Object(); } }

Overriding Serialization • Example: Filter elements from a list – Declare list to be transient – In write. Object() • Invoke default serialization • Iterate over list, writing filtered elements out. write. Object(el); • Write end-of-list marker out. write. Object(Boolean. FALSE); • Alternatively, write length & elements

Overriding Deserialization • Customize deserialization by implementing private void read. Object(Object. Input. Stream) throws IOException, Class. Not. Found. Exception;

Overriding Deserialization • Example: Restore lock private void read. Object(Object. Input. Stream in) throws IOException, Class. Not. Found. Exception { in. default. Read. Object(); lock = new Object(); }

Overriding Deserialization • Example: Restore list • In read. Object() – Invoke default deserialization – Read filtered elements until end-of-list marker – Alternatively, read length & elements

Notes on Customization • Don’t perform operations that take a long time – No I/O besides accessing object stream • Swing UI elements are serializable – But are not designed for long-term storage – Declare them transient – Restore UI in application logic

The Replacements • Example: Symbols — there can only be one private Object read. Resolve() throws Object. Stream. Exception { return intern(name); } • Done after object graph has been restored – Embedded self references are not replaced!

Inheritance • If a superclass implements Serializable, all subclasses are also serializable – Each class in such a hierarchy serializes only its own state – Classes can control all state by implementing java. io. Externalizable • If superclass is not Serializable, a serializable subclass must handle the superclass’s state

Inheritance • To make a subclass of a serializable class not serializable private void write. Object( Object. Output. Stream o) throws IOException { throw new Not. Serializable. Exception( get. Class(). get. Name()); } • This indicates a semantic problem!

Versioning • Problem – Classes can change – While instance is in serialized form • Solution – Let classes declare their version – Define what are compatible changes

Stream Unique Identifier (SUID) • Hash of the class – Determined by serialver tool – Accessible in Java through Object. Stream. Class. get. Serial. Version. UID() • Modified version declares same SUID as original version private static final long serial. Version. UID = …;

Incompatible Changes • • • Deleting fields Moving classes up or down in the hierarchy Changing non-static fields to static Changing non-transient fields to transient Changing the declared type of a field Adding / removing access to default fields from write. Object() / read. Object() • See specification!

Compatible Changes • • Adding fields Adding / removing classes Adding Serializable Adding / removing write. Object() / read. Object() • Changing static fields to non-static • Changing transient fields to non-transient

Security • Serialized objects expose their internal state • If that state is sensitive it must be protected – Don’t serialize sensitive state – Encrypt serialized objects

There’s More • serial. Persistent. Fields to declare the serialized format – Useful for backwards compatibility • Object. Input. Validation to validate deserialized objects • Class descriptors – Serialized form of a class • Two versions of serialization protocol