Stochastic Processes Nancy Griffeth January 10 2014 Funding
- Slides: 17
Stochastic Processes Nancy Griffeth January 10, 2014 Funding for this workshop was provided by the program “Computational Modeling and Analysis of Complex Systems, ” an NSF Expedition in Computing (Award Number 0926200).
Motivation n n Continuous solutions depend on large numbers (e. g. , ODE’s) Stochastic solutions are better for small numbers n Strange and surprising things happen!
Game n n Two players each have a pile of pennies Taking turns: n n Player flips a penny Heads, he gets a penny from the other player Tails, he gives the penny to the other player Until one player runs out of pennies
Questions n n What each player’s chance of winning? What is the expected value of the game to a player? How long will the game go on before a player runs out of pennies? What is the probability after m moves that player A has a pennies and player B has b pennies? n Or the probability distribution over (a, b)?
Chance of winning n n If A and B have the same number of coins, they’re equally likely to win. With m coins to player A and n coins to player B n n n A wins with probability m/(m+n) B wins with probability n/(m+n) The player with more coins is more likely to win.
Expected Value n Definition: n n Exp(winnings)=Prob(winning)*value of winning +Prob(losing)*value of losing Value of winning is m+n Value of losing is 0 So, if A starts with m coins and B with n: n n E(A winnings) = m/(m+n)*(m+n) + n/(m+n)*0 =m E(B winnings) = n/(m+n)*(m+n) + m/(m+n)*0 =n
Markov Chain 3, 3 2, 4 1, 5 0, 6 The probability of each transition is ½, except 0, 6 ->0, 6 and 6, 0 ->6, 0, where the probability is 1
How long n n n Relatively hard from the UCLA tutorial Sum over n of n*[p((5, 1); n)*0. 5+p((1, 5); n)*0. 5] = Sum over n of n*p((5, 1; n) Starting from (3, 3), about 8 turns
Probability Matrix 0, 6 1, 5 2, 4 3, 3 4, 2 5, 1 6, 0 0, 6 1 0 0 0 1, 5 0 0 0 0 2, 4 0 0. 5 0 0 0 3, 3 0 0 0. 5 0 0 4, 2 0 0. 5 0 5, 1 0 0 0. 5 6, 0 0 0 0 1
Two Steps 0, 6 1, 5 2, 4 3, 3 4, 2 5, 1 6, 0 0, 6 1 0 0 0 1, 5 0. 25 0 0 0 2, 4 0. 25 0 0 3, 3 0 0. 25 0 4, 2 0 0 0. 25 5, 1 0 0. 25 0. 5 6, 0 0 0 0 1
Three Steps 0, 6 1, 5 2, 4 3, 3 4, 2 5, 1 6, 0 0, 6 1 0 0 0 1, 5 0. 625 0 0. 125 0 0 2, 4 0. 25 0 0. 375 0 0. 125 0 3, 3 0. 125 0 0. 375 0 0. 125 4, 2 0 0. 125 0 0. 375 0 0. 25 5, 1 0 0 0. 125 0 0. 625 6, 0 0 0 0 1
64 Steps 0, 6 1, 5 2, 4 3, 3 4, 2 5, 1 6, 0 0, 6 1 0 0 0 1, 5 0. 833 0 0 0. 167 2, 4 0. 667 0 0 0. 333 3, 3 0. 5 0 0 0. 5 4, 2 0. 333 0 0 0. 667 5, 1 0. 167 0 0 0. 833 6, 0 0 0 0 1
Steady States n n n Sooner or later, somebody wins (with probability=1) Either party can win The system has two stable steady states (bistable)
The “Game” with Molecules L 2 L 1 L 0 R 10 R 9 R 8 L. R 0 L. R 1 L. R 2 L. R 0 L. R. R. L 1
Chemical Master Equation n n A set of first-order differential equations describing the change in the probability of states with time t. With time-independent reaction rates, the process is Markovian.
The Chemical Master Equation We are interested in p(x; t), the probability that the chemical system will be in state x at time t. The time evolution of p(x; t) is described by the Chemical Master Equation: Where sμ is the stoichiometric vector for reaction μ, giving the changes in each type of molecule as a result of μ, and aμ is the propensity for reaction μ
Bistability n n n A steady state is a state that a system tends to stay in once it reaches it A steady state can be stable or unstable A system that has two stable steady states is called bistable.
