Bayesian networks and their application in circuit reliability

Bayesian networks and their application in circuit reliability estimation Erin Taylor

What is a Bayesian Network? • An example We want to describe the causal relationship between the following events: 1) The season 2) Whether it is raining outside 3) The sprinkler is on 4) The sidewalk is wet 5) The sidewalk is slippery • We can construct a graph to represent the causal link between these 5 events.

What is a Bayesian Network? Assumptions: • “Sprinkler on” and “Rain” are determined by “Season” • “Sidewalk wet” is determined by “Sprinkler on” and “Rain” • “Sidewalk slippery” is determined by “Sidewalk wet” Bayesian network Season Sprinkler Rain Wet Slippery • Each node represents a random variable, in this case the probability of a particular event.

Properties of Bayesian Networks • A Bayesian network is – A Directed Acyclic Graph (DAG) – A model of probabilistic events • In a Bayesian network – Nodes represent random variables of interest – Links represent causal dependencies among variables • Bayesian networks are direct representations of the world. Arrows indicate real causal connections, not the flow of information, as in neural networks.

Properties of Bayesian Networks Season Sprinkler Rain Wet Slippery • Links are not absolute – If the sprinkler is on, this does not always mean that the sidewalk is wet – For example, the sprinkler may be aimed away from the sidewalk

Properties of Bayesian Networks • Given that the sidewalk is wet, we can calculate the probability that the sprinkler is on: P(sprinkler on | sidewalk wet) Season Sprinkler Rain Wet Slippery • Bayesian networks allow us to calculate such values from a small set of probabilities in a process called reasoning or Bayesian Inference

Reasoning in Bayesian Networks • Reasoning in Bayesian networks operates by propagating information in any direction 1) If the sprinkler is on, the sidewalk is probably wet (prediction) 2) If the sidewalk is wet, it is more likely that the sprinkler is on or it is raining (abduction) 3) If the sidewalk is wet and the sprinkler is on, the likelihood that it is raining is reduced (explaining away) Season Sprinkler Rain Wet Slippery • Explaining away is a special type of reasoning that is especially difficult to model in other network models

Specifying a Bayesian Network • A new example: 1) 2) 3) When a family leaves their house, they often turn the front light on and let the dog out If the dog is dirty, the family often puts him outside If the dog is out, you can sometimes hear him bark Family out Light on Dog dirty Dog out Hear bark

Specifying a Bayesian Network • To specify the probability distribution of a Bayesian network we need – The prior probability of all root nodes – The conditional probabilities of all nonroot nodes given all possible combinations of their direct predecessors P(fo) = 0. 15 Total specified values: 10 Family out Light on P(lo | fo) = 0. 6 P(lo | ~fo) = 0. 05 P(dd) = 0. 01 Dog dirty Dog out Hear bark P(do | fo dd) = 0. 99 P(do | fo ~dd) = 0. 90 P(do | ~fo dd) = 0. 97 P(do | ~fo ~dd) = 0. 3 P(hb | do) = 0. 7 P(hb | ~do) = 0. 01

Bayesian Networks and Probability Theory • In traditional probability theory, specifying the previous example would require the joint distribution of all 5 variables: Family out Light on Dog dirty Dog out Hear bark P(fo, dd, lo, do, hb) • The joint distribution of 5 variables requires 25 -1 or 31 values

Bayesian Networks and Probability Theory • To see where 25 -1 comes from, consider the set of Boolean variables (a, b) • To specify the joint probability distribution we need the following values: P(a b) P(a ~b) P(~a ~b) • In the general case, this yields a total of 2 n values for a system of n variables • Since the sum of all possible outcomes must be 1, we can reduce the number of values to 2 n -1

Bayesian Networks and Joint Probabilities • Using Bayesian networks for this example, we can reduce the number of values that need to be specified from 31 to 10. Family out Light on Dog dirty Dog out P(fo, dd, lo, do, hb) Hear bark Total specified values: 31 Total specified values: 10 How is this possible?

Simplifying Joint Distributions • Bayesian networks reduce the complexity of joint distributions by introducing several independence assumptions • Conditional Independence: – If we know whether the dog is out, then the probability of hearing him bark is completely independent from all other events – Other events only serve to indicate the probability that the dog is out P(hb | fo dd lo do) = P(hb | do) – Also P(dd | fo lo do hb) = P(dd) Prior probability Family out Light on Dog dirty Dog out Hear bark

Simplifying Joint Distributions • Only the five terms on the right side are needed to specify the joint distribution of our example: P(fo, dd, lo, do, hb) = P(fo) P(dd) P(lo|fo) P(do|fo dd) P(hb|do) Prior probabilities of root nodes Conditional probabilities of nonroot nodes given all combinations of predecessors • The number of values that need to be specified in a Bayesian network grows linearly with the number of variables whereas the joint distribution grows exponentially

Evaluating Probabilities Using BN’s • Basic computation on Bayesian networks is the computation of every node’s belief (conditional probability) given the evidence observed • For example – Evidence: The dog is heard barking – Compute: The probability that the family is out – Compute: The probability that Family out the light is on Light on Dog dirty Dog out Hear bark

Evaluating Probabilities Using BN’s • • • Solving Bayesian networks involves Bayesian inference Exact Solution – Involves enumerating all possible probability combinations – Generally NP-Hard Simple Query: P(fo = true | hb = true)

Evaluating Probabilities Using BN’s • Approximate Solutions – Logic sampling – Markov chain Monte Carlo algorithm – Likelihood weighting method • General approach to approximate solutions – Select values for a subset of nodes – Use this ‘evidence’ to pick values for remaining nodes – Keep statistics on all the nodes’ values

Logic Sampling • Logic Sampling Algorithm 1) Guess values for all roots nodes according to prior probability P(fo=0. 15) -> 15% of time fo=true 2) Work down network guessing values for next lower node based on parent values P(fo) = 0. 15 Family out Light on Previous values: fo=true and dd=false P(do|fo ~dd)=0. 90 -> 90% of time do=true 3) Repeat many times for entire network and keep track of how often each node is assigned each value Dog dirty Dog out Hear bark P(do | fo ~dd) = 0. 90 4) To determine a conditional probability, P(fo=true | hb=true), consider cases when hb=true and count the number of times fo=true.

Applications • Bayesian networks were popularized by the Artificial Intelligence community who used them as a learning algorithm OR “If I see a red object, what is the probability that I should stop? ” • Used to model trust in a P 2 P network “Bayesian network-based trust model in P 2 P networks” by Wang and Vassileva • Used to evaluate circuit reliability “Scalable probabilistic computing models using Bayesian networks” by Rejimon and Bhanja

BN’s for Circuit Reliability • Circuit Example Z 1 Z 2 X 1 X 2 Z 3 Inputs: Z 1, Z 2, Z 3 Internal Signals: X 1, X 2 Outputs: Y 1, Y 2 Y 1 Y 2

BN’s for Circuit Reliability • Goal: Analyze circuit reliability in the face of dynamic errors • Procedure: – Construct an error prone version of the circuit where each gate has a probability of failure = p Z 1 Z 2 Z 3 X e 1 X e 2 p Y e 1 p Y e 2 – Analyze this circuit in relation to the fault-free circuit

BN’s for Circuit Reliability • Error at the ith output can be represented mathematically: Ei = Y e i Å Y i P(Ei = 1) = P(Yei Å Yi =1) • Equivalent circuit representation Z 1 Z 2 Y 1 X 2 E 1 Y 2 Z 3 Xe p 1 X e 2 p Y e 1 Y e 2 E 2

BN’s for Circuit Reliability • In a circuit, each gate output has a causal relationship with its input Þ circuits can be represented as Bayesian networks • In the Bayesian network representation of a circuit – – Inputs are root nodes Outputs are leaf nodes Internal signals are internal nodes Each node’s conditional probability is determined by the gate type and probability of error, p

BN’s for Circuit Reliability Z 1 Z 2 Y 1 X 2 E 1 Y 2 Z 3 X e 1 X e 2 p p Y e 1 Bayesian network E 2 Z 1 Z 2 Y e 2 Circuit Xe X 1 X e 2 1 Y e 2 E 1 Z 3 X 2 Y 1 Y 2 E 2

BN’s for Circuit Reliability • Specifying the Bayesian network – Prior probabilities for all root nodes Circuit inputs – Conditional probabilities for all nonroot nodes given all possible combinations of parents Z 1 Ye 1 is the output of NAND gate with inputs Z 1 and Xe 1 P(Ye 1=1| Z 1=0, Xe 1=0) = (1 -p) Z 2 X 2 E 1 Y 2 Z 3 X e 1 Probability of no gate error Y 1 X e 2 p p Y e 1 Y e 2 E 2

BN’s for Circuit Reliability • Solving for the error probabilities, E 1 and E 2 – Error probability is fixed (p = 0. 005, 0. 1) – Logic sampling algorithm is used – Determines the probability of output error given each input combination • Results – Circuits with 2, 000 -3, 000 gates took on average 18 s to simulate – Average accuracy: 0. 7% – Worst-case accuracy: 3. 34% Z 2 Z 1 X e 1 – Compare this to an accurate method which takes ~1, 000 s to simulate simple circuits with only 10’s of gates Z 3 Y e 1 Y e 2 E 1 X 2 X 1 X e 2 Y 1 Y 2 E 2

Advanced Subjects in BN’s • Dynamic Bayesian networks – Models variables whose values change over time – Captures this process by representing each state variable as multiple copies, one for each time step • Learning in Bayesian networks – The conditional probabilities of each node can be updated continuously – Similar to the way weights are adjusted in neural networks

Conclusions • Bayesian networks are a powerful tool for modeling any probabilistic system • Applications are diverse – – Medical field Image processing Speech recognition Computer networking • Used for efficient evaluation of nanoscale circuit reliability