On the generality of binary treelike Markov chains

On the generality of binary tree-like Markov chains K. Spaey - B. Van Houdt - C. Blondia Performance Analysis of Telecommunication Systems (PATS) Research Group University of Antwerp - IBBT MAM 2006 - June 12 -14, 2006 - Charleston, S. C.

On the generality of binary tree-like Markov chains Aim of the paper: Show that an arbitrary tree-like Markov chain can be embedded in a binary tree-like Markov chain with a special structure MAM 2006 On the generality of binary tree-like Markov chains 2

Tree-like QBD Markov chains • • States are grouped into sets of m states: “nodes” The nodes form a d-ary tree MAM 2006 • • Transitions: from a node to itself, to its parent node, to its child nodes Characterized by the matrices B and F, Dk, Uk, k = 1, . . . , d. On the generality of binary tree-like Markov chains 3

Tree-like QBD Markov chains • Key equations: § § for k = 1, . . . , d • Stability condition: § needs to be stochastic for all k • Steady state probabilities: § for all J, for all k § § MAM 2006 On the generality of binary tree-like Markov chains 4

Tree-like QBD Markov chains • Tree-like Markov chains were introduced as a special case of the tree-structured Markov chains (Bini, Latouche & Meini, Solving nonlinear matrix equations arising in tree-like stochastic processes, Linear Algebra Appl. 366, 2003) • Any tree-structured Markov chain can be reduced to a tree-like Markov chain (Van Houdt & Blondia, Tree structured QBD Markov chains and tree-like QBD processes, Stochastic Models 19(4), 2003) • Any tree-like Markov chain can be embedded in a binary (d=2) tree-like Markov chain MAM 2006 On the generality of binary tree-like Markov chains 5

Constructing the binary tree-like MC • Tree-like MC (Xt, Nt) • Binary tree-like MC Xt: nodes d-ary tree § Nt: auxiliary variable § § § • Nodes are denoted by strings (symbols 1, . . . , d) J = j 1 j 2. . . jn-1 jn Root node: ø • Auxiliary variable § i = 1, . . . , m : nodes binary tree : 2 D auxiliary variable • Nodes are denoted by binary strings (symbols 0, ) starting with a “ ” Root node: ø • Auxiliary variable (0, i) corresponding to node ø § (a, i) for other nodes § i = 1, . . . , m § a = -(d-1), . . . , -1, 0, 1, . . . , d-1 § MAM 2006 On the generality of binary tree-like Markov chains 6

Constructing the binary tree-like MC ø ø 1 0 00 000 4 • 2 11 3 00 31 21 0 0 0 22 0 211 111 0 12. . . 00 13 0 121 0 112 1111 Binary notation ψ of the nodes of the d-ary tree: § and ψ(ø) = ø • 1 -1 correspondence between states (J, i) of (Xt, Nt) and states (ψ(J), (0, i)) of • Every possible transition in (Xt, Nt) between (J, i) and (J’, i’) will be mimicked by a path of transitions in between (ψ(J), (0, i)) and (ψ(J’), (0, i’)) MAM 2006 On the generality of binary tree-like Markov chains 7

Constructing the binary tree-like MC • d-ary tree: transition from a node to its k-th child § (J, i) (J+k, j) with prob. (Uk)i, j ø ø 1 0 00 000 4 2 3 00 31 11 21 0 0 0 22 0 211 111 0 12. . . 00 13 0 121 0 112 1111 • binary tree: (ψ(J), (0, i)) (ψ(J) , (k-1, j)) with prob. (U )(0, i), (k-1, j) = (Uk)i, j § (ψ(J) 0, (k-2, j)) with prob. (U 0)(k-1, j), (k-2, j) = 1 § . . . (ψ(J) 0. . . 0, (1, j)) with prob. (U 0)(k-2, j), (k-3, j). . . (U 0)(2, j), (1, j) = 1 § (ψ(J) 0. . . 00, (0, j)) = (ψ(J+k), (0, j)) with prob. (U 0)(1, j), (0, j) = 1 § MAM 2006 On the generality of binary tree-like Markov chains 8

Constructing the binary tree-like MC • d-ary tree: transition from a child k to its parent § (J+k, i) (J, j) with prob. (Dk)i, j ø ø 1 0 00 000 4 2 3 00 31 11 21 0 0 0 22 0 211 111 0 12. . . 00 13 0 121 0 112 1111 • binary tree: (ψ(J+k), (0, i)) = (ψ(J) 0. . . 00, (0, i)) (ψ(J) 0. . . 0, (-1, i)) with prob. (D 0)(0, i), (-1, i) = i, j, = diag(D 1 e) § . . . (ψ(J) , (-(k-1), i)) with prob. (D 0)(-1, i), (-2, i). . . (D 0)(-(k-2), i), (-(k-1), i) = 1 § (ψ(J), (0, j)) with prob. (D )(-(k-1), i), (0, j) = ( -1 Dk)i, j § MAM 2006 On the generality of binary tree-like Markov chains 9

Constructing the binary tree-like MC • d-ary tree: transition from a node to itself root node: (ø, i) (ø, j) with prob. Fi, j § other node: (J, i) (J, j) with prob. Bi, j § • binary tree: root node: (ø, (0, i)) (ø, (0, j)) with prob. § other node: (J, (0, i)) (J, (0, j)) with prob. § MAM 2006 On the generality of binary tree-like Markov chains 10

Calculating the steady state probabilities d-ary tree like MC • • binary tree-like MC for k = 1, . . . , d stability condition: needs to be stochastic for all k • • stability condition: and need to be stochastic All Gk stochastic G 0 and G stochastic Algorithms for calculating the steady state probabilities: Fixed point iteration (FPI) § Reduction to quadratic equations (RQE) § Newton’s iteration (NI) § MAM 2006 On the generality of binary tree-like Markov chains 11

Calculating the steady state probabilities • The matrices corresponding to the constructed binary tree-like MC, e. g. , § U 0 , U , D 0 , D , § have a structure that is related to the matrices that correspond to the original d-ary tree-like MC • Example (d=4) MAM 2006 On the generality of binary tree-like Markov chains 12

Calculating the steady state probabilities • Fixed point iteration (FPI) § iterative algorithm: § V[N] monotonically converges to V • Applied to binary tree: more iterations needed § taking the structure of into account identical to applying FPI to d-ary tree § MAM 2006 On the generality of binary tree-like Markov chains 13

Calculating the steady state probabilities • Reduction to quadratic equations (RQE) § iterative algorithm: • Gi[0] = 0, i = 1, . . . , d • d quadratic matrix equations solve for Gi[N+1] • Gi[N] converges to Gi, i = 1, . . . , d • Applied to binary tree: • G 0[0] = G [0] = 0 • slower convergence • taking the structure of the matrices into account d quadratic matrix equations as when applying RQE to d-ary tree MAM 2006 On the generality of binary tree-like Markov chains 14

Calculating the steady state probabilities • Newton’s iteration (NI) iterative algorithm § computes the matrices Gi, i=1, . . . , d § converges quadratically § each step requires solving an equation of the form § large linear system of equations Ax=b (inefficient) • Applied to binary tree: § each step requires solving an equation of the form ≈ Sylvester equation § linear system of equations § reduction to binary tree can result in computational gain ? ? ? MAM 2006 On the generality of binary tree-like Markov chains 15

Conclusions • Any tree-like Markov chain can be embedded in a binary treelike Markov chain with a special structure any tree-structured Markov chain can be embedded in a binary tree-like Markov chain with a special structure • Mainly of theoretical interest: FPI and RQE algorithms applied to binary tree do not speed up calculations of the steady state probabilities § NI algorithm: currently unclear § MAM 2006 On the generality of binary tree-like Markov chains 16