Why Probability and Nondeterminism Concurrency Theory Nondeterminism Scheduling

Why Probability and Nondeterminism? Concurrency Theory • Nondeterminism – Scheduling within parallel composition – Unknown behavior of the environment – Underspecification • Probability – Environment may be stochastic – Processes may flip coins MPRI 3 Dec 2007 Catuscia Palamidessi 1

Automata A = (Q , q 0 , E , H , D) Transition relation D Q ´ (EÈH) ´ Q Internal (hidden) actions External actions: EÇH = Æ Initial state: q 0 Q States MPRI 3 Dec 2007 Catuscia Palamidessi 2

Example: Automata A = (Q , q 0 , E , H , D) d q 0 n Execution: Trace: MPRI 3 Dec 2007 n q 1 n q 2 choc q 4 ch q 3 coffee q 5 q 0 n q 1 n q 2 ch q 3 coffee q 5 n n coffee Catuscia Palamidessi 3

Probabilistic Automata PA = (Q , q 0 , E , H , D) Transition relation D Q ´ (EÈH) ´ Disc(Q) Internal (hidden) actions External actions: EÇH = Æ Initial state: q 0 Q States MPRI 3 Dec 2007 Catuscia Palamidessi 4

Example: Probabilistic Automata fair q 1 flip 1/2 q 3 beep q 5 2/3 q 0 unfair MPRI 3 Dec 2007 1/2 q 2 flip 1/3 q 4 Catuscia Palamidessi 5

Example: Probabilistic Automata flip 1/2 qh beep qp q 0 flip MPRI 3 Dec 2007 2/3 1/3 qt Catuscia Palamidessi 6

Example: Probabilistic Automata fair q 1 flip 1/2 q 3 beep q 5 2/3 q 0 unfair 1/2 q 2 flip 1/3 q 4 What is the probability of beeping? MPRI 3 Dec 2007 Catuscia Palamidessi 7

Example: Probabilistic Executions q 0 fair flip q 1 1/2 q 3 beep q 5 1/2 (beep) = 1/2 q 4 (beep) = 2/3 q 0 MPRI 3 Dec 2007 unfair 2/3 q 2 flip 1/3 q 3 beep q 5 2/3 q 4 Catuscia Palamidessi 8

Example: Probabilistic Executions flip fair q 0 unfair q 1 1/2 1/2 q 4 1/2 q 3 2/3 q 2 flip MPRI 3 Dec 2007 q 3 beep q 5 1/4 7/12 beep q 5 2/6 1/3 q 4 Catuscia Palamidessi 9

Measure Theory • Sample set – Set of objects W • Sigma-field (s-field) – Subset F of 2 W satisfying • • Inclusion of W Closure under complement Closure under countable union Closure under countable intersection Example: set of executions Study probabilities of sets of executions which sets can I measure? • Measure on (W, F) – Function from F to ³ 0 • For each countable collection {Xi}I of pairwise disjoint sets of F, (ÈIXi) = SI (Xi) • (Sub-)probability measure – Measure such that (W) = 1 ( (W) £ 1) • Sigma-field generated by C 2 W – Smallest s-field that includes C MPRI 3 Dec 2007 Catuscia Palamidessi 10

Measure Theory Why not F = 2 W ? Flip a fair coin infinitely many times W = {h, t}¥ (w) = 0 for each w W (first coin h) = 1/2 Theorem: there is no probability measure on 2 W such that (w) = 0 for each w W. MPRI 3 Dec 2007 Catuscia Palamidessi 11

Cones and Measures • Cone of – Set of executions with prefix – Represent event “ occurs” • Measure of a cone – Product edges of flip q fair 1/2 1 q. Theorem 0 q 1/2 3 beep 1/2 C q 5 q 4 q 3 cones qextends 5 unfair A measure 2/3 on beep uniquely q 2 1/3 to a measure flip q 4 on the s-field generated by cones 1/2 MPRI 3 Dec 2007 Catuscia Palamidessi 12

Examples of Events • Eventually action a occurs – Union of cones where action a occurs once • Action a occurs at least n times – Union of cones where action a occurs n times • Action a occurs at most n times – Complement of action a occurs at least n+1 times • Action a occurs exactly n times – Intersection of previous two events • Action a occurs infinitely many times – Intersection of action a occurs at least n times for all n • Execution occurs and nothing is scheduled after – Set consisting of only – C intersected complement of cones that extend MPRI 3 Dec 2007 Catuscia Palamidessi 13

Schedulers - Resolution of nondeterminism Scheduler Function s : exec*(A) ® Q x (E H) x Disc(Q) if s( ) = (q, a, ) then q = lstate( ) Probabilistic execution Measure s, r(Cs) = 0 generated by s from state r if r s s, r s, r(Cr) = 1 MPRI 3 Dec 2007 s, r(C aq) = s, r(C ) q) if s( ) = (q, a, ) Catuscia Palamidessi 14

Probabilistic CCS P : : = 0 | P|P | P + P | ( ) P | X | let X = P in X | PÅp. P Nondeterministic process Prefix P ¾® (P) P + Q ¾® Probabilistic processes P 1Åp. P 2 ¾® p 1+ (1 -p) 2 MPRI 3 Dec 2007 Catuscia Palamidessi 15

Probabilistic CCS Interleaving Hiding P ¾® P|Q ¾® |Q a, â ( a) P ¾® ( a) Communication a P 1 ¾® (P 2) â P 2 ¾® (P 2) P 1 P 2 ¾® (P 2 | P 2) Recursion P[ let X = P in X / X ] ¾® let X = P in X ¾® MPRI 3 Dec 2007 Catuscia Palamidessi 16

Bisimulation Relations We have the following objectives • They should extend the corresponding relations in the non probabilistic case • Keep definitions simple • Where are the key differences? MPRI 3 Dec 2007 Catuscia Palamidessi 17

Strong Bisimulation on Automata Strong bisimulation between A 1 and A 2 q, s, a, q $ s Relation R Q x Q, a q Q=Q 1È+Q 2, such that a q 1 b q 3 q 0 s 0 a a q 2 b q 4 MPRI 3 Dec 2007 R q R s a s s 1 b s 3 Catuscia Palamidessi 18

Strong Bisimulation on Probabilistic Automata Strong bisimulation between A 1 and A 2 q, s, a, $ Relation R Q x Q, a q Q=Q 1È+Q 2, such that a q 0 s 0 a 1 q 2 b 1 q 3 q 4 MPRI 3 Dec 2007 s 1 b 1 s 3 R R a s R [LS 89] C Q/R. (C ) = (C ) Catuscia Palamidessi 19

Probabilistic Bisimulations • These two Probabilistic Automata are not bisimilar q 1 a. 2 . 8 a a. 3. 7. 4. 6 q 2 q 3 b c b c s 1 a ~ ~p b . 2 . 8 s 2 s 3 a c . 4. 6 s 2 b s 3 c • Yet they satisfy the same formulas of a logic PCTL • Bisimilar if we match transitions with convex combinations of transitions – The logic observes probability bounds on reachability properties MPRI 3 Dec 2007 Catuscia Palamidessi 20

Weak Bisimulation on Automata Weak bisimulation between A 1 and A 2 q, s, a, q $ s Relation R Q x Q, a q Q=Q 1È+Q 2, such that t q 1 b q 3 q 0 R R a s t q 2 b q 4 MPRI 3 Dec 2007 s 1 b s 3 q s a s Þ s $ : trace( )=a, fstate( )=s, lstate( )=s Catuscia Palamidessi 21

Weak bisimulation on Probabilistic Automata Weak bisimulation between A 1 and A 2 q, s, a, $ Relation R Q x Q, a q Q=Q 1È+Q 2, such that t R q 0 q 2 b 1 q 3 R a s q 1 q 4 MPRI 3 Dec 2007 s 1 b 1 s 3 R [LS 89] C Q/R. (C ) = (C ) Catuscia Palamidessi 22

Weak Transition q a r There is a probabilistic execution such that – (exec*) = 1 (it is finite) – trace( ) = (a) (its trace is a) – fstate( ) = (q) (it starts from q) – lstate( ) = r (it leads to r) a q Þ s iff $ : trace( )=a, fstate( )=q, lstate( )=s MPRI 3 Dec 2007 Catuscia Palamidessi 23

Exercises • Prove that the probabilistic CCS is an extension of CCS (to define what this means is part of the exercise) • Prove that probabilistic bisimulation is an extension of bisimulation • Write the Lehmann-Rabin algorithm in probabilistic CCS (without using guarded choice) MPRI 3 Dec 2007 Catuscia Palamidessi 24