RTDroid toward building dynamic realtime systems Lukasz Ziarek

RTDroid: toward building dynamic real-time systems Lukasz Ziarek University at Buffalo Dagstuhl Seminar: Resource Bound Analysis http: //rtdroid. cse. buffalo. edu

Collaborators • Ethan Blanton – Fiji • Yin Yan (Ph. D. Student) Systems Inc • Many masters students • Jan Vitek – Northeastern • Karthik Dantu – UB • Steve Ko - UB http: //rtdroid. cse. buffalo. edu

Motivations • Open source project • Well-defined APIs • Rich on-device peripherals • How can we write a real-time application on Android, while still leveraging Android’s existing functionality? http: //rtdroid. cse. buffalo. edu

Cochlear Implant Device • Cochlear implant as a motivating example • An external device for audio recording and processing • Implanted electrodes stimulate the cochlea http: //rtdroid. cse. buffalo. edu

Architecture of Cochlear Implant Application on Android UI Activity Volume Receiver Button Listener UI Handler Audio Recorder (Service) Periodic Audio Recording Task Audio Processor (Service) Msg. Handler Periodic Audio Processing Task Output Simulator (Broadcast Receiver) 8 ms

Sound Processing Latency Duration of 12% of audio processing releases is longer than 8 ms. Cochlear Implant: Audio Processing Duration

What Is Missing in Android? • Lack of real-time expressiveness • Can’t prioritize the execution of application components • Can’t specify real-time configuration • Lack of real-time communication • No priority awareness for message/intent delivery • No protection mechanism for cross-context communication • Lack of memory guarantees Non Real-time components UI Activity Audio Recorder (Service) Volume Receiver Button Listener UI Handler Periodic Audio Recording Task Audio Processor (Service) Msg. Handler Periodic Audio Processing Task Output Simulator (Broadcast Receiver)

A View Of Android Applications Constructs System Application Framework And APIs Services Runtime Bionic Dalvik/ART Linux with Android Modifications Android Stack No Real-time Expressiveness FIFO Message Passing Fair System Services Unpredictable Garbage Collection No Support for RT POSIX Extension Fair Scheduling Out of Memory Killer Kernel Forced GC Power Management

Proposed Architectures for Real-time Android Applications Application Framework Core Libraries Dalvik VM Linux Real-time RT Apps Libraries Core RT Apps Dalvik VM Linux Kernel Real-time Hypervisor Completely Isolated

Proposed Architectures for Real-time Android Applications Application Framework Core Libraries Dalvik VM RT JVM Core Libraries Extended Dalvik VM Linux Real-time Interactions Unclear Linux Real-time ? Lower Latencies Best Effort

RTDroid Architecture Applications Redesigned Application Framework Components Runtime Extended Bionic Dalvik/ART Fiji MVM Linux with Android RT Linux or RTEMS Modifications RTDroid Stack http: //rtdroid. cse. buffalo. edu Ø Programing Model Ø Validation Mechanisms Ø Redesigned Message Passing Ø Redesigned System Services Ø Real-time Garbage Collection Ø POSIX RT Extension Ø RTSJ Libraries Ø Priority Based Scheduling Ø PIP Locks

COMPONENTS AND SYSTEM SERVICES http: //rtdroid. cse. buffalo. edu

Redesigned Components Application Framework Constructs and APIs Runtime System Services RT Handler RT Alarm. Manager RTSJ RT Sensor. Manager RTDroid RT Sound. Manager

Why Looper and Handler? • One of the central message-passing mechanisms. • All Android apps unknowingly use Looper-Handler. • Most of the system services use Looper-Handler. http: //rtdroid. cse. buffalo. edu

Looper and Handler • Looper is a per-thread message loop. • Handler processes messages. • A thread uses Handler. send() to send a message. Sender Thread Receiver Thread Looper Message Queue Message … Message Handler Ref (provides send())

What’s Wrong with Looper/Handler in a Realtime Context? Message Queue RT msg N … Handler RT client thread Non-RT client thread http: //rtdroid. cse. buffalo. edu msg 1 Looper Messages are processed FIFO or by client-specified time Non-RT client thread No guarantee on message delivery

RTLooper and RTHandler Msg Queue msg 1 … RT Looper (RT Thread) RT client thread http: //rtdroid. cse. buffalo. edu … msg n Handler instance Looper (Non-RT Thread) msg 1 RT Handler instance Non-RT client thread … Non-RT client thread

Android Sensor Manager APIs Application Sensor Event Listener Sensor Manager Enable/disable hardware Configure sampling rate Subscribe to sensor events Accelerometer Gyroscope … Magnetometer http: //rtdroid. cse. buffalo. edu

How Does Android’s Sensor Manager Work? Input Event Sensor Service H A L Sensor Thread Sensor Fusion Sensor Manager Native Sensor. Manager Sensor Event Listener Sensor Thread Event Queue Kernel http: //rtdroid. cse. buffalo. edu System Runtime Framework Application

What Happens In a Real-Time Context? • One application is listening on two sensors: Accelerometer with higher priority Gyroscope with lower priority Application Android Framework Kernel Accelerometer Data Gyroscope Data Shared delivery path and unbound delivery time http: //rtdroid. cse. buffalo. edu

RT Sensor Manager • Event-driven architecture • Prioritize the delivery procedure of sensor events • Multiple listeners with different sampling rates http: //rtdroid. cse. buffalo. edu

RT Sensor Manager Listener 3 Listener 1 Listener 2 Delivery Thread 1 Delivery Listener’s rate and priority Thread 4 Thread 3 Thread 2 highest priority Accel The Events Gyro Events Accel Polling Thread Gyro Polling Thread /dev/input/ event 6 /dev/input/ event 1 Listener 4 Orientation Event of its listeners Sensor Fusion Orientation Polling Thread

Remaining Questions • Multi-app execution • Real-time Android Programming model • Static Memory Guarantees http: //rtdroid. cse. buffalo. edu

MULTI-APP EXECUTION http: //rtdroid. cse. buffalo. edu

Android: Application Isolation & Communication • One VM per application • Virtual Memory • Each application has its own PID, UID for isolation App A App B System Server App Components Activity. Manager Activity Thread ART Binder Driver Linux Kernel Service. Manager

Multi-Tasking VM (MVM) All applications run in a single VM. The VM runs each application in an isolation unit called a partition, but all applications run as a single process (the same memory space). How does MVM provide isolation? • Separated Class loading (Fiji uses lightweight VMs) • Dynamically manage Memory boundaries • Statically Configured Heap Memory http: //rtdroid. cse. buffalo. edu

Time Partitions in MVM slice manager: • Fixed-length time slice for each partition • Partition initiation, suspension, resume, termination • Two approaches for thread scheduling: • Direct OS scheduling vs partition dispatcher with a thread pool

Communication in MVM • Shared address space provides flexibility • Wait free pair transactions • One way communication channel over primitives • CAS and replicas • Priority Rollback Protocol • Inspired by transactional memory • Similar tracking structures • write buffers, undo log • The lock itself provides concurrency control http: //rtdroid. cse. buffalo. edu

PROGRAMMING MODEL http: //rtdroid. cse. buffalo. edu

Can we write an App with RTDroid? Applications Application Framework Extended Bionic Runtime Fiji VM Linux with Android RT Linux or RTEMS Modifications http: //rtdroid. cse. buffalo. edu Ø Ø RT-Looper and RT-Handler RT Alarm Manager RT Sensor Manager What else is missing?

Programming Model • Declarative real-time components • Real-time communication channel • Memory guarantees • region-based allocation • Design principles: – – – Android Applications RTDroid Applications Android Framework RTDroid Framework Dalvik / ART Android Apps Fiji Java VM RT app Android Linux Kernel with RT Retains a similar style of Android Patch Minimizes code complexity in application Enables application validation with real-time configuration prior to execution

Declarative Real-Time Components • • Real-time components • RT service • RT broadcast receiver • Periodic task Application manifest • Priority • Timing parameters • Memory bounds <service name="pkg. Processing. Service” priority="79"> Service Broad. Receiver + + on. Receive(…) <mem. Sizes persistent="1 M” release="1 M" /> on. Start. Command() <periodic-task + on. Bind()name="proc. Task“ priority="79"> + on. Create() Realtime. Receiver <release start="0 ms" periodic="8 ms" /> + on. Destroy() + on. Clean() </periodic-task> … Realtime. Service </service> + on. Pause() Periodic. Task + on. Release()

Real-Time Communication Channels Four types of channels – RT message passing channel – RT intent broadcasting channel – RT bulk data transfer channel – Cross-context channel • Communication policies • Message delivery order – Component invocation – Message constraints – <channel name=“msghandler” type=“msgpassing”> <!-- message delivery and processing policies --> </ channel> <service name="pkg. Recording. Service" priority=“ 78"> <intent-filter count="2" role=“publisher"> <action name="msghandler"/> </intent-filter> </service> <service name=" pkg. Processing. Service " priority=“ 79"> <intent-filter count="2" role=“subscriber"> <action name="msghandler"/> </intent-filter> </service>

Memory Guarantees in RTDroid • Region-based memory management with constant cost for object allocation and reclamation. • Based on the concept of the scoped memory in RTSJ • The complexity of region management is hidden in our program model • The lifetime management of scope is associated with the lifetime of each application component. http: //rtdroid. cse. buffalo. edu

Memory Guarantees in RTDroid Immortal scope Persistent scope Release scope Objects that have the same lifetime as the application Objects that have the same lifetime as its associated component Temporary objects for the execution of callback functions http: //rtdroid. cse. buffalo. edu

Region-Based Memory Management in Cochlear Implant Processing Service Output Receiver Release Persistent RT Channels Immortal Scope Real-time Runtime in RTDroid Heap Memory (Fiji. VM) Heap Memory (ART/Dalvik) UI Activities Android Adaptor RTDroid Proxy Android Runtime

A FULLY DYNAMIC SYSTEM http: //rtdroid. cse. buffalo. edu

Dynamic Real-Time Systems • Ability to install / remove / restart applications • Non real-time • Real-time • JIT as a service … for validation? • Validation of system wide constraints • Modification of an existing application • Adding / removing capabilities http: //rtdroid. cse. buffalo. edu

System wide memory management • Can we manage memory more efficiently / predictably if we take a whole system view? • Hussein et al. showed for non RT http: //rtdroid. cse. buffalo. edu

Validating Interactions • Session Types • Mechanisms for validating interactions • Communication protocols “match” • Participants are duals of one another No notion of rate / resources • Partial failures • • Error handling without restarting a session http: //rtdroid. cse. buffalo. edu

Reasoning about power? • Power management is incredibly important for mobile • We currently do not do anything interesting • Lots of interesting work to explore • Energy types • Power modeling + power management http: //rtdroid. cse. buffalo. edu

Validating Timeliness? • What is feasible to validate at runtime? • Can package static results with the app • Bytecode based WCET (Bates et al. ) • Potential starting point? • JIT based specialization / validation? • Online Schedulability analysis? http: //rtdroid. cse. buffalo. edu

EVALUATION http: //rtdroid. cse. buffalo. edu

Platform Categories RTEMS v Soft Real-time Smartphone http: //rtdroid. cse. buffalo. edu v Soft Real-time Desktop v Soft Real-time Embedded

Cochlear Implant Sensing Processing Non-Real-Time UI Real-Time Processing http: //rtdroid. cse. buffalo. edu …

Cochlear Implant http: //rtdroid. cse. buffalo. edu

Evaluation on j. Papabench http: //rtdroid. cse. buffalo. edu

j. Papa. Bench (UAV flight control) http: //rtdroid. cse. buffalo. edu

Wind Farm Blade Monitoring http: //rtdroid. cse. buffalo. edu

Wind Farm Blade Monitoring http: //rtdroid. cse. buffalo. edu

Conclusion • RTDroid A real-time mobile system with Android-like programming interfaces. • RT kernel + RT JVM + RT framework layer • • RTDroid runs multi-apps in a single Multitasking VM with isolation. • RTDroid provides a real-time programming model with memory guarantees. http: //rtdroid. cse. buffalo. edu

Porting j. Papa. Bench into RTDroid Fly-By-Wired (FBW) Simulation Autopilot Test. PPMTask Handler Simulator. Flight Model. Task. Handler Navigation Task. Handler Send. Data. To Autopolit Simulator. IR Task. Handler Altitude. Control Task. Handler Data injection Check. Failsafe Task. Handler Check. Mega 128 Values. Task. Handler Simulator. GPS Task. Handler Sensor. Manager … Data subscription Stablization Task. Handler

Evaluation for Java Autopilot RTEMS Base line performance on Nexus S Base line performance on LEON 3

Evaluation for Java Autopilot RTEMS RT Sensor. Manager stress tests on LEON 3

RTService Code Example <service name=“. Autopilot. Service” persistent=“ 1 M” release=“ 1 M”> <rt-android: priority=90> Class Autopilot. Service extents RTService{ Flight. Model f. Model; Autopilot. Model apt. Model; public final Sub. Receiver sf. Recvr = new Sub. Receiver(){ pubic void on. Reciver(Content t, Intent i){ read intent msg; f. Model. process(i); } }; public void on. Create(){ //flight. Model, autopilot. Model instantiation stab. Task = init. Sub. Service( new Stab. Runnable(apt. Model, f. Model)); register. RTSub. Receiver(sf. Recvr, ”sf-recvr”); } <Sub. Service id=“stabilization-task” release= “ 1 M”> <rt-android: priority=29> <rt-android: periodic ms=“ 50”, ns=“ 0”> </Sub. Service> <Sub. Receiver id=“sf-recvr” release=“ 1 M”> <intent-filter count=4> <action name=“sim-status”> <data size=“ 50 K” mimetype=“app/octet-stream”> </itent-filter> </Sub. Receiver> </service> }

RTService Code Example Class Autopilot. Service extents RTService{ Flight. Model f. Model; Autopilot. Model apt. Model; public final Sub. Receiver sf. Recvr = new Sub. Receiver(){ pubic void on. Reciver(Content t, Intent i){ read intent msg; f. Model. process(i); } }; public void on. Create(){ //flight. Model, autopilot. Model instantiation stab. Task = init. Sub. Service( new Stab. Runnable(apt. Model, f. Model)); register. RTSub. Receiver(sf. Recvr, ”sf-recvr”); } } Autopilot. Service Persistent Scope (1 M) Autopilot. Service Release (1 M) stab. Task Release (1 M) sf. Recvr Release (1 M)