The Synthesis of Combinational Logic to Generate Probabilities

The Synthesis of Combinational Logic to Generate Probabilities Weikang Qian Marc Riedel Kia Bazargan David Lilja Electrical & Computer Engineering University of Minnesota

Motivation • Randomness and noise in nanoscale circuits o Traditional view: impediment o A novel view: resources for probabilistic computation 2

Motivation • Random sources: Generate probabilities needed by probabilistic algorithms Stochastic Bit Streams: Probability 3/6 0, 1, 1, 0 … Random Sources Deterministic Hardware 1, 1, 0, 1 Probability 4/6 Issue: Costly to generate many different probabilities directly from random sources! Solution: Synthesize combinational logic to generate probabilities! 3

Basic Problem Other Probabilities Needed Set S of Input Probabilities (|S| small) p 1 Random Sources p 2 p 3 Choose Set S ? Logic Circuit ? q 1 q 2 q 3 q 4 4

Assumptions • All signals in the circuit are random Boolean variables. • Elements of set S can be used many times. • All inputs are independent. Independent p 1 p 2 p 3 p 3 logic circuit q 5

Conventions • “Probability”: Probability of a signal being logical one • “Synthesize probability q”: Synthesize the circuit generating probability q 6

Example 0. 4 0. 5 0. 2 Logic Circuit 0. 3 0. 6 7

Generating Decimal Probabilities Choose Set S = {p 1, p 2, p 3} p 1 p 2 p 3 Large Cost! |S| Small! Logic Circuit • Found Set S for o |S| = 2 o |S| = 1 q 2 q 3 q 4 8

Generating Decimal Probabilities Theorem 1: With S = {0. 4, 0. 5}, we can synthesize arbitrary decimal output probabilities. Example: Synthesize q = 0. 757 (Black dots are inverters) 9

Proof of Theorem 1 • Constrain design to AND gates and inverters o AND gates: x ∙ y o Inverters: 1 − x • Constructive proof o Induction on the number of digits n • Base case 0. 1 = 0. 4 × 0. 5, 0. 2 = 0. 4 × 0. 5 0. 3 = (1 − 0. 4) × 0. 5 0. 6 / 0. 7 / 0. 8 / 0. 9 = 1 − 0. 4 / 0. 3 / 0. 2 / 0. 1 • Inductive step o Assume true for (n − 1). Show true for n. 10

Proof of Inductive Step q: n digits; w: (n− 1) digits; u = 10 nq, numerator of q 0 ≤ q ≤ 0. 2, u = 2 k conversions w = 5 q q = 0. 4 × 0. 5 × w 0 ≤ q ≤ 0. 1, u = 2 k+1 w = 10 q q = 0. 4× 0. 5×w 0. 1 < q ≤ 0. 2, u = 2 k+1 w = 2 − 10 q q = (1− 0. 5 w)× 0. 4× 0. 5 0. 2 < q ≤ 0. 4, u = 4 k w = 2. 5 q q = 0. 4 × w cases q … 11

Algorithm from Theorem 1 Example: Synthesize q = 0. 757 from S = {0. 4, 0. 5} 0. 757 1− 0. 243 × 0. 4 0. 6075 1 − 0. 3925 × 0. 5 0. 785 1 − 0. 215 × 0. 5 0. 43 0. 4 1− 0. 6 × 0. 5 0. 3 1− 0. 7 × 0. 5 Base cases 0 ≤ q ≤ 0. 2, u = 2 k 0 ≤ q ≤ 0. 1, u = 2 k+1 … 0. 2 < q ≤ 0. 3, u = 2 k+1 0. 3 < q ≤ 0. 4, u = 2 k+1 0. 4 < q ≤ 0. 5 < q ≤ 1 × 0. 4 0. 14 1 − 0. 86 × 0. 5 0. 35 conversions w = 5 q w = 10 q … w = 10 q − 2 w = 4 − 10 q w = 1 − 2 q w = 1− q 12

Circuit Synthesized 0. 4 0. 757 1− 1− 0. 6 0. 243 × 0. 5 × 0. 4 0. 3 1− 0. 6075 0. 7 1− × 0. 5 × 0. 4 0. 14 1 − 0. 86 × 0. 5 0. 3925 × 0. 5 0. 785 1− 0. 215 × 0. 5 0. 43 (Black dots are inverters) 13

Generating Decimal Probabilities Question: Does there exist a set S of size one which can generate arbitrary decimal probabilities? p p p combinational circuit q p Yes! 14

Proof • There exists a p (≈ 0. 1295) in the unit interval such that • Boolean function with , has output probability • Thus, 0. 4 and 0. 5 can be generated from p. (Use Theorem 1. ) 15

Implementation • Focus on generating decimal probability form S = {0. 4, 0. 5}. • Goal: Reduce circuit depth. 16

Balancing • Logic Level Optimization: Balancing Before Balancing After Balancing (a and b are primary inputs) 17

Factorization of Fractions • High Level Optimization: Factorization of Fractions Example: Synthesize q = 0. 49 from S = {0. 4, 0. 5} Basic Factor 18

Factorization of Fractions • Quality of factor pair a = al × ar – Heuristic: Max{Digits(al), Digits (ar)} – Digits(x) ↑ Depth of the circuit for fraction x ↑ Example: 0. 1764 = 0. 252× 0. 7 = 0. 63× 0. 28 better! 19

Factorization of Fractions • Consider both q and 1 − q – Example: q = 0. 37, then 1 − q = 0. 63 has better factor pair 0. 7× 0. 9 • When both q and 1 − q cannot be factorized – Use conversions from Theorem 1 to get w with one less digit than q. – Factorize w. 20

Experimental Results • ‘Basic’: Algorithm based on Theorem 1 + Balancing • ‘Factor’: Algorithm based on Factorization + Balancing • Compare number of AND gates and depth of synthesized circuits • Present average results for different numbers of digits n 21

Experimental Results Average number of AND gates vs. number of digits n 22

Experimental Results Average depth vs. number of digits n 23

Summary • Considered the problem of generating required decimal probabilities. • Gave a pair of probabilities / a single probability for generating arbitrary decimal probabilities. • Proposed implementation based on balancing and factorization of fractions to reduce the depth of the circuit. • Also considered synthesis with different assumptions: – Whether the probabilities are duplicable or not. – Whether we are free to choose set S. 24

Future Work • Tradeoff between circuit area and approximation error – Synthesize q = 0. 251: an AND gate realizing 0. 25 but with error 0. 001? • Correlation on input probabilities • Accuracy of input probabilities – p 1 = 0. 4 becomes 0. 4 + δ 25

Thank You! 26

Problem Flavors • Set S: pre-determined / free to choose • Elements: duplicable / non-duplicable Independent p 1 p 2 p 3 p 3 Independent p 1 logic circuit q p 2 p 3 Duplicable logic circuit q Non-Duplicable 27

Problem Variations Generate Decimal Probability Set S Elements Determined? Duplicable? Case One No Yes Case Two Yes No Case Three No No Case Four Yes 28

Case Two • Set S given, elements non-duplicable • Essentially to fill in the output column of the truth table • We know the probability of input combination – e. g. , r 2 = (1 − p 1) p 2 (1 − p 3) p 1 x 1 p x 2 2 p 3 x 3 logic circuit unknown z q 29

Case Two: Solution • If z 0 = z 3 = z 6 = 1, the output probability is po = r 0 + r 3 + r 6 • In general, • Generally, a target probability q cannot be realized exactly. Find least error approximation Given q, we try to find zi’s to be either 0 or 1 to minimize the objective function --- solved by linear programming 30

Case Three • Set S not Given and Elements non-Duplicable p 1 x 1 p x 2 2 p 3 x 3 logic circuit z q Question: What is the optimal set S = {p 1, p 2, p 3}? An optimal set S is 31