Dangerous Omissions the Cost of Ignoring Decision Uncertainty

  • Slides: 19
Download presentation
Dangerous Omissions – the Cost of Ignoring Decision Uncertainty Mark Sculpher Susan Griffin Karl

Dangerous Omissions – the Cost of Ignoring Decision Uncertainty Mark Sculpher Susan Griffin Karl Claxton Steve Palmer Centre for Health Economics, University of York,

Background • • • Increasing demands to assess new drugs closer to launch Inevitable

Background • • • Increasing demands to assess new drugs closer to launch Inevitable uncertainty in evidence Hence a series of linked questions – Should a technology be adopted? – How uncertain is this decision? – Is more evidence needed? • • But decisions may be fragmented Need a methods framework to handle these questions

What are the decisions? • Should a technology be adopted given existing information? –

What are the decisions? • Should a technology be adopted given existing information? – Which clinical strategies are worthwhile? – For which patient groups? • Is current evidence sufficient to support use in NHS? – Do we need more evidence? – What type of evidence is required? – What additional research should be conducted to provide this evidence?

What are the decisions? • Should a technology be adopted given existing information? –

What are the decisions? • Should a technology be adopted given existing information? – Which clinical strategies are worthwhile? – For which patient groups? • Is current evidence sufficient to support use in NHS? – Do we need more evidence? – What type of evidence is required? – What additional research should be conducted to provide this evidence?

Is it costeffective? Is it worthwhile? Is the ICER less than the cost-effectiveness threshold?

Is it costeffective? Is it worthwhile? Is the ICER less than the cost-effectiveness threshold? ICER = Additional cost QALYs gained = £ 20, 000 2 QALYs = £ 10, 000 per QALY If the cost-effectiveness threshold is £ 20, 000 per QALY, B is cost-effective Is net benefit positive? Net health benefit = QALYs gained – QALYs lost £ 20, 000 = 2 – 1 = 1 QALY = 2 – £ 20, 000 Net money benefit = £ value of QALYs gained – additional costs = 2 x £ 20, 000 – £ 20, 000 = 1 QALY

Should a technology be adopted? Treatment A QAL Y 1 Cost £ 10, 000

Should a technology be adopted? Treatment A QAL Y 1 Cost £ 10, 000 0 £ 5, 000 2 £ 15, 000 1 £ 10, 000 ICER = Additional cost QALYs gained = £ 20, 000 2 QALYs = £ 10, 000 per QALY Is the ICER less than the cost-effectiveness threshold? £ 10, 000 per QALY < £ 20, 000 per QALY, B is cost-effective Is net benefit positive? Treatment B QAL Y 2 Cost £ 30, 000 3 £ 20, 000 4 £ 40, 000 3 £ 30, 000 Net health benefit = QALYs gained – QALYs lost £ 20, 000 = 2 – 1 = 1 QALY = 2– £ 20, 000 Net money benefit = £ value of QALYs gained – additional costs = 2 x £ 20, 000 – £ 20, 000 = 1 QALY

What are the decisions? • Should a technology be adopted given existing information? –

What are the decisions? • Should a technology be adopted given existing information? – Which clinical strategies are cost-effective? – For which patient groups? • Is current evidence sufficient to support use in NHS? – Do we need more evidence? – What type of evidence is required? – What additional research should be conducted to provide this evidence?

How uncertain is a decision? How things could turn out Net Health Benefit Treatment

How uncertain is a decision? How things could turn out Net Health Benefit Treatment A Treatment B Treatment C Best choice Possibility 1 9 12 8 B Possibility 2 12 10 9 A Possibility 3 14 17 11 B Possibility 4 11 10 10 A Possibility 5 14 16 12 B Average 12 13 10 What’s the best we can do now? But we are not always right Choose B and expect 13 QALYs Chance that B is the best = 3/5 = 0. 6 Chance that A is the best = 2/5 = 0. 4 Chance that C is the best = 0/5 = 0 So if we adopt B the probability of error = 0. 4

How uncertain is the decision? Choose A Choose B B ICER = £ 25,

How uncertain is the decision? Choose A Choose B B ICER = £ 25, 000 per QALY A C

Why does uncertainty matter? How things could turn out Net Health Benefit Treatment A

Why does uncertainty matter? How things could turn out Net Health Benefit Treatment A Treatment B Best we Best choice could do if we knew What we could lose Possibility 1 9 12 B 12 0 Possibility 2 12 10 A 12 2 Possibility 3 14 17 B 17 0 Possibility 4 11 10 A 11 1 Possibility 5 14 16 B 16 0 Average 12 13 13. 6 0. 6 What’s the best we can do now? Could we do better? Choose B and expect 13 QALYs If we knew we get 13. 6 QALYs Maximum benefit of more evidence is 0. 6 QALYs But is it worth it?

Do we need more evidence? Cost of research Choose A Choose B

Do we need more evidence? Cost of research Choose A Choose B

Do we need more evidence?

Do we need more evidence?

Decisions in a joined up world? • Adopt technologies if we expect them to

Decisions in a joined up world? • Adopt technologies if we expect them to be cost effective based on existing evidence • But only if we simultaneously address question: Is the evidence sufficient? • Demand or commission further research to inform this choice in the future

A fragmented world • Separation of adoption and research decisions – Adoption decisions without

A fragmented world • Separation of adoption and research decisions – Adoption decisions without accountability for impact on future research – Research decisions without accountability for relevance to adoption decisions • Dangers – Adoption decisions undermine evidence base for practice • Incentives and ethics – Commissioned research does not inform decisions • Adoption becomes the only policy instrument

Account for the cost of uncertainty What we loose if we reject a technology

Account for the cost of uncertainty What we loose if we reject a technology What we loose if we accept technology

Clear signals and incentives Provide more evidence!

Clear signals and incentives Provide more evidence!

Clear signals and incentives Reduce price

Clear signals and incentives Reduce price

Why say no (or only in research)? • Clear signals – No because it

Why say no (or only in research)? • Clear signals – No because it is not a cost-effective use of resources – No because there is currently insufficient evidence to justify NHS use – Spell out the key evidence needed (not the research) • Clear incentives – If and when additional evidence is made available then considered for early review – Incentives to sponsors (evidence and price) – Incentives for others stakeholders to lobby for publicly funded research – Clear signals to research commissioners

Conclusions • All decisions about new technologies involve uncertainty • Is uncertainty being used

Conclusions • All decisions about new technologies involve uncertainty • Is uncertainty being used in decision making? • Need to address adoption decision and need for further research simultaneously • But adoption decision may be only policy lever