Introduction to decision analysis modeling Alice Zwerling Postdoctoral

Introduction to decision analysis modeling Alice Zwerling, Postdoctoral fellow, JHSPH azwerli 1@jhu. edu Mc. Gill TB Research Methods Course July 7, 2015

Overview of Decision Analysis models • Understand why we use decision analysis models & what these models can and cannot do • Examples of decision analysis and simple decision trees • Understand key decision analysis concepts and terminology 2

Decision Analysis Models • Some decisions are easier than others 3

Decision Analysis • Provides a systematic approach to decision-making under conditions of uncertainty • Is employed across many diverse fields: healthcare, business, economics, law, engineering…. . • Helps decision makers think clearly about many elements of complex decisions 4

Decision Analysis Modeling • Requires defining events in terms of their logical and temporal significance • Disaggregating a complex problem into smaller steps or events simpler to understand 5

What is a model? • Representation of real world phenomena, object or behaviours • Theoretical construct • Simplification of real world • Model needs to be simple enough to be understood but complex enough to capture essential elements of problem or question to be addressed • Many simplifying assumptions are needed… 6

Decision Trees • Branching structure used to represent different kinds of events including decisions and uncertainties, and the associated outcomes or alternatives 7

Decision Trees Chance Node Decision Node Branches (alternative pathways) A B Time (sequence of events) moves from left to right 8 Terminal Node (Outcomes or pay-offs)

Decision Trees Pathway probabilities (probability of going down each branch) p =0. 1 p =0. 9 • 9 User defined probabilities are entered at each chance node, need to add to 1

Decision Trees Outcomes: Costs & Effectiveness (DALYs/QALYs or cases 0/1) 10

Decide what sequences/events are important to include/ model 11

12 Vassall et al Plos. Med, 2011

Steps in Decision Analysis • Define the problem • Define alternatives to be assessed • Identify clinical outcomes of interest • • Build the tree structure Assign probabilities to all chance events Assign costs and outcomes Estimate expected average values of each strategy (Folding back) • Assess uncertainty and variability (sensitivity analyses) 13

A simple problem… Evaluate a new treatment Cure Standard Tx Failure Patients with TB diagnosis Cure New Tx Failure • Create structure 14

A simple problem… Evaluate a new treatment Cure p=0. 6 Standard Tx Failure p=0. 4 Patients with TB diagnosis Cure p=0. 8 New Tx Failure p= 0. 2 • Assign probabilities 15

A simple problem… Evaluate a new treatment Cure p=0. 6 Standard Tx $100 Saves 10 life years Failure p=0. 4 $100 Saves 1 life year Cure p=0. 8 $500 Saves 10 life years Patients with TB diagnosis New Tx Failure p= 0. 2 • Assign values and outcomes 16 $500 Saves 1 life year

Calculations Cure p=0. 6 $100 Saves 10 life years Standard Tx Failure p=0. 4 $200 Saves 1 life year Cure p=0. 8 $500 Saves 10 life years Patients with TB diagnosis New Tx Failure p= 0. 2 $600 Saves 1 life year Outcome Standard Treatment New Treatment Expected cost (0. 6 x $100) + (0. 4 x $200) =$60 + $80 = $140 (0. 8 x $500) + (0. 2 x $600) =$400 + $120 = $520 Expected life years saved (0. 6 x 10) + (0. 4 x 1) = 6. 4 (0. 8 x 10) + (0. 2 x 1 ) = 8. 1 17

Calculations Expected Outcomes Standard Treatment New Treatment Expected cost (0. 6 x $100) + (0. 4 x $200) =$60 + $80 = $140 (0. 8 x $500) + (0. 2 x $600) =$400 + $120 = $520 Expected life years saved (0. 6 x 10) + (0. 4 x 1) = 6. 4 (0. 8 x 10) + (0. 2 x 1 ) = 8. 1 • Incremental cost: $520 - $140 = $380 • Incremental life years saved: 8. 1 – 6. 4 = 1. 7 • Cost per life year saved = $380/1. 7 = $223. 53 18

Decision analysis • Final model outcomes are calculated based on the probability of entering into a particular node and the price tag or effectiveness measure associated with that node • Individuals move through the decision trees for a specified amount of time • Costs and rewards accrue over the simulation • At end of simulation get a tally of specified outcomes (eg. TB related costs person, number of TB cases, number of TB deaths, etc for each intervention considered (outcomes)

Comparing scenarios!!! 20

21

22

Brief note on sensitivity analyses • Systematically asking how would decision results change if parameters/probabilities were different • Determines how robust model result may be • Vary probabilities and/or outcomes across possible ranges • Can produce tornado diagrams 23

Decision Analysis Models • Not designed to capture transmission • Static, not dynamic models • Typically deterministic (can be stochastic) 24

Decision Analysis Models • STATIC models: (as opposed to dynamic transmission models) • The annual risk of infection (ARI) is not sensitive to the changing number of infectious cases in the population • Does not account for ongoing transmission in a population 25

Population vs. Individual models Population based Individual based • Follows populations • Divides population into mutually exclusive groups • Homogeneity within groups, but can be subdivided • Individual level factors are averaged together, model shows changes in average characteristics of whole population • Follows individuals • Characteristics of each individual are tracked through time • Can explore complex relationships, social/spatial interactions, heterogeneity • May include approaches such as agent-based models, or queue model Decision analysis are most commonly population based, but can track individuals 26

Population based Individual based • 25% of population has TB HIV co-infection • Each individual tracked • 45% of cohort is female • Pt 1 of 10, 000 simulated pts Is a female 45 years old, HIV+ • Cohort comprised of persons on average 45 years old 27

Deterministic models Stochastic models • All parameters are fixed, no random element • Incorporate chance into the model • Model predictions are fixed, same answer with every run • Results vary with every run • Describes what happens on average to whole population • More frequent in literature • Important in small populations or where chance might play a role • Require many simulations (more computing power) Decision analysis can use either approach, but are most commonly deterministic models 28

Decision Analysis: Advantages • • • Easy to learn & user friendly Intuitive, visual representation of problem Can take advantage of average data (e. g. meta-analyses) Low cost and faster (relative to empirical studies) Can consider hypothetical situations or specific populations/scenarios for whom a trial is not ethical or feasible • Can capture more complex pathways • Can be used to generalize/extrapolate trial findings (time/pop’n) • Integrated costing capability and can be easily modified for cost-effectiveness (common form of modeling methods in economic evaluation)

Decision analysis: Disadvantages • Decision analysis describes what happens to a cohort of selected individuals – By design does not incorporate pop’n outside of cohort • Population level impact not accounted for • Transmission effects not typically incorporated in decision analysis models • The annual risk of infection is not sensitive to changing number of infectious cases in the population • Can be partially addressed using Markov models and introducing assumptions around transmission parameters 30

Decision Analysis Models CAN…. • Compare relative impact and cost of two different well defined interventions • Understand problems in a logical and transparent fashion • Identify weakness in our conceptualization of problem • Make assumptions explicit 31

Summary: Models are not so good for… • • Predicting the future Giving precise estimates Working magic with bad/limited data Can only work to level of complexity that we understand/ have data to support • Models rely on assumptions, may be limited by poor data (can test these…) • Struggle to capture heterogeneity/phenomenon that we are not aware of or do not understand

Summary… • Different types of models are available Choice depends on : • The research question • The data that we have to work with • The assumptions that we are willing to make • How quickly we need the results • The expertise of the modelling “team”

Reproduced from xkcd. com 34