Reachability Analysis for Discrete Time Stochastic Hybrid Systems
Reachability Analysis for Discrete Time Stochastic Hybrid Systems Saurabh Amin Alessandro Abate Shankar Sastry UCBerkeley http: //chess. eecs. berkeley. edu/ Introduction Computational Results Stochastic hybrid systems (SHS) can model uncertain dynamics and stochastic interactions that arise in many systems. An important problem in SHS theory is that of probabilistic reachability Probabilistic Reachability Problem • What is the probability with which the system can reach a set during some time horizon? • (If possible), select a control input to ensure that the system remains outside the set with sufficiently high probability • When the set is unsafe, the problem becomes a quantitative safety verification problem. In this case, find the maximal safe sets corresponding to different safety levels Discrete Time Stochastic Hybrid System (DTSHS) DTSHS definition: Where is the set of modes, the map defines the dimension of the continuous state space of these modes, and are the transition and reset control spaces, and , , and are continuous, discrete, and reset stochastic kernels respectively Dynamics of thermostat system Continuous dynamics Off On Continuous and reset transition kernels Off On Optimal control policy for three different safe sets Executions generated by optimal policy for the three safe sets Discrete transition kernel “Switch” action “Don’t switch” action Stochastic Reachability , Consider Markov polices Assume complete observability and finite time horizon Reach probability is the probability that the execution of associated with policy and initial distribution will enter set during time Probabilistic safe set is the set that guarantees safety probability : for policy Maximal probabilistic safe Sets for different safety levels Backward Reachability Computations Since we have DTSHS as controlled Markov process For computing reach probability for fixed Markov policy define the functions by State space Control space , Controlled transition kernel which is if and if Motivational example For initial mode “OFF” and so, Maximal Probabilistic Safe Set Computation Thermostat ON Then, For safety level OFF the maximal safe set Trivial switching control law by and so, Then, Existence of optimal policy is shown and February, 23 2005 Conclusions and Future Work For controlled DTSHS Dynamic programming recursion Define the functions For initial mode “ON” • Proposed a model for controlled discrete time SHS suitable for optimal control and reachability analysis • Interpreted the safety verification problem in terms of stochastic reachability notion • Developed a dynamic programming based approach for computing probabilistic maximal safe sets and the optimal feedback policy • Applied the proposed methodology on a simple example and presented computational results Future work • To address the problem of reachability analysis for continuous time SHS • Apply the research to probabilistic safety verification problem in more challenging applications such as air traffic control systems 2006 Berkeley EECS Annual Research Symposium (BEARS)
- Slides: 1