NC 3 Rs resources to improve the design

  • Slides: 26
Download presentation
NC 3 Rs resources to improve the design of animal experiments Dr Nathalie Percie

NC 3 Rs resources to improve the design of animal experiments Dr Nathalie Percie du Sert BBSRC STARS Course Tuesday 11 April 2017

The NC 3 Rs (UK National Centre for the 3 Rs) § Lead on

The NC 3 Rs (UK National Centre for the 3 Rs) § Lead on the discovery and application of new technologies and approaches to replace, reduce and refine the use of animals for scientific purposes (the 3 Rs) Visit our website: www. nc 3 rs. org. uk @NC 3 Rs National Centre for the 3 Rs § Work with research funders, journals, academia, industry and regulators § Activities divided between: - Research funding - Centre-led science programmes

NC 3 Rs resources The ARRIVE guidelines and the Experimental Design Assistant

NC 3 Rs resources The ARRIVE guidelines and the Experimental Design Assistant

Background Quality of published animal research Experimental design Statistical analysis Reporting of studies Only

Background Quality of published animal research Experimental design Statistical analysis Reporting of studies Only 12% of publications report randomisation and 14% report blinding Only 70% of publications fully described the statistical methods and presented the results with a measure of variability Animal characteristics – missing in 25% Sample size justification – missing in 100% Only 59% stated the study hypothesis, number and characteristics of animals used Survey reviewed 271 publications and identified key areas for improvement Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al. (2009). Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLo. S One 4(11): e 7824.

Experimental design – internal validity Consider threats which might compromise the validity of the

Experimental design – internal validity Consider threats which might compromise the validity of the experiment, any opportunities for the investigator to influence: § animal selection § conduct of the experiment § assessment of outcome § which outcomes are reported Measures used to reduce validity threats include: § Random allocation to treatment groups § Allocation concealment § Blinding during outcome assessment § Inclusion/exclusion criteria

Experimental design – randomisation Method is important – haphazard is not random Use a

Experimental design – randomisation Method is important – haphazard is not random Use a validated procedure (e. g. computer throw a dice, flip a coin) generated, Randomisation is crucial for two reasons: 1. Minimise selection bias e. g. haphazard selection may results in slowest mice allocated to the same group 2. Key assumption of the statistical analysis Different groups should be drawn from the same background population using random sampling

Experimental design – randomisation Pick a number between 1 and 20 347 responses Is

Experimental design – randomisation Pick a number between 1 and 20 347 responses Is 17 the “most random” number? http: //scienceblogs. com/cognitivedaily/2007/02/05/is-17 -the-most-random-number/

Experimental design – blinding § 12 students § Maze-bright and maze-dull rats § Elevated

Experimental design – blinding § 12 students § Maze-bright and maze-dull rats § Elevated T-maze, dark arm reinforced bright rats 3 Number of correct responses 2 dull rats 1 0 135 Days § Rats had been labelled bright or dull randomly § Only difference was in the minds of the investigators! Rosenthal R, Fode KL (1963). The effect of experimenter bias on the performance of the albino rat. Behavioral Science 8(3): 183 -189.

Improving the reporting of in vivo research The ARRIVE guidelines were developed to improve

Improving the reporting of in vivo research The ARRIVE guidelines were developed to improve the reporting of biomedical research using animals. § Checklist of 20 items, containing key information necessary to describe a study comprehensively and transparently. The guidelines include: § Information which relates to internal validity § Information which would allow a study to be repeated § Information about the context and scientific relevance of the study https: //www. nc 3 rs. org. uk/arrive-guidelines

The EDA Experimental Design Assistant The EDA was developed to improve the design of

The EDA Experimental Design Assistant The EDA was developed to improve the design of animal experiments § Web-based tool § Aimed at in vivo researchers § Developed as a collaboration between: § In vivo researchers § Statisticians § Academia and industry § Software designers specialised in artificial intelligence § Road tested by researchers and statisticians https: //eda. nc 3 rs. org. uk

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram Effect of drug A on plasma glucose levels Animals characteristics: diabetic mice Experimental unit: : mouse Group 1 Independent variable of interest : Drug A Pharmacological intervention 1 Vehicle Drug Vehicle Pool of animals Measurement: Plasma glucose Allocation: Complete randomisation Group 2 Pharmacological intervention 2 Drug Outcome Measure: Glucose levels Analysis

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram Experiment Effect of drug A on plasma glucose levels Animals characteristics: diabetic mice Experimental unit: : mouse Group 1 Independent variable of interest : Drug A Pharmacological intervention 1 Vehicle Drug Vehicle Pool of animals Measurement: Plasma glucose Allocation: Complete randomisation Group 2 Pharmacological intervention 2 Drug Practical steps Outcome Measure: Glucose levels Analysis

The EDA diagram § Examples § Templates § Feedback from the critique

The EDA diagram § Examples § Templates § Feedback from the critique

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram § Feedback and advice on your experimental plan § Dedicated support for randomisation, blinding and sample size calculation § Practical information to improve knowledge of experimental design § Improved transparency of the experimental plan, allowing more efficient communication

Feedback and advice from the EDA § Dataset of rules triggers prompts based on

Feedback and advice from the EDA § Dataset of rules triggers prompts based on the EDA diagram § Feedback provided: § Diagram structure § Ask to provide more information § Point out inconsistencies § Prompt you to consider things that are not on the diagram § Highlight implications of some of the choices made § Provide recommendation for analysis § Rule set to be expanded over time § Provide more feedback § Enable the system to detect more subtle issues § Increasingly specific feedback

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram § Feedback and advice on your experimental plan § Dedicated support for randomisation, blinding and sample size calculation § Practical information to improve knowledge of experimental design § Improves transparency of the experimental plan and helps communication

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram § Feedback and advice on your experimental plan § Dedicated support for randomisation, blinding and sample size calculation § Practical information to improve knowledge of experimental design § Improved transparency of the experimental plan, allowing more efficient communication

The EDA offers: § The ability to build a stepwise visual representation of an

The EDA offers: § The ability to build a stepwise visual representation of an experiment – the EDA diagram § Feedback and advice on your experimental plan § Dedicated support for randomisation, blinding and sample size calculation § Practical information to improve knowledge of experimental design § Improved transparency of the experimental plan, allowing more efficient communication

The EDA workflow

The EDA workflow

Objectives § Improve the reliability of published results § Promote better understanding of experimental

Objectives § Improve the reliability of published results § Promote better understanding of experimental design, raise awareness about issues § Facilitate peer review/assessment of the experimental plans with an explicit description – Transparency – Pre-registration § Promote more careful consideration of the experimental plans – Spend time planning – Diagram facilitate discussion

EDA uptake ~2500 accounts on the system with 30 diagrams created per week Recommended

EDA uptake ~2500 accounts on the system with 30 diagrams created per week Recommended by UK funders

Acknowledgments NC 3 RS working group Certus Technology Prof Clare Stanford (Chair), UCL Dr

Acknowledgments NC 3 RS working group Certus Technology Prof Clare Stanford (Chair), UCL Dr Brian Lings Dr Simon Bate, Glaxo. Smith. Kline Mr Ian Bamsey Dr Manuel Berdoy, University of Oxford Dr Robin Clark, Huntingdon Life Sciences Alpha testers Prof Innes Cuthill, University of Bristol Beta testers Dr Derek Fry, University of Manchester Dr Natasha Karp, Wellcome Trust Sanger Institute Prof Malcolm Macleod, University of Edinburgh Dr Lawrence Moon, King’s College London Dr Richard Preziosi, University of Manchester https: //eda. nc 3 rs. org. uk www. nc 3 rs. org. uk/ARRIVE