Breaking Bad How not to do analysis context

Breaking Bad – How not to do analysis: context, interpretation & exposure rates RSA Team

Does autumn sun ‘kill 28 drivers per year’ • Used CF 706 ‘Dazzling Sun’ • Headline was that the percentage of collisions were increasing (1. 2% to 1. 31%) • No other information was presented, speculated that ‘Autumn’ and ‘Evenings’ were the problem

Are men twice as likely to speed as women? • Story in The Telegraph – November 2014 • One man in 15 has points • Compared to one woman in 30 • Based on stats from DVLA on total numbers of men and women caught speeding

Speeding points by exposure • Didn’t account for Speed Awareness Course attendance • More importantly, didn’t account for exposure! • We used RAC Foundation ‘On the Move’ report from Dec 2012 for • DID show that men had twice as many points per licence holders • BUT we calculated: Points per miles driven=(Licence*Mileage)/(Drivers with Points/3 years) • Women = 140, 420 and Men = 141, 506 – Almost the same!

Where do Britain’s Most Dangerous Drivers Live? • Published by Brake. • Using DVLA data from 2013 to show numbers of offending drivers by home postcode district e. g. OX 17. • ‘Top 10’ table created showing highest offender numbers. • SL 6 highest numbers of offenders with 1, 847, followed by NG 5 (1, 543), SL 1 (1, 537), ST 5 (1, 516) and BH 23 (1, 503).

Offenders by population • Nearly 3, 000 postcode districts in Great Britain of varying population. • Average of around 22, 000 people, ranging from 12 to over 150, 000. • Population data for postcode districts taken from the data behind MAST to create offence rate (offenders per 1, 000 population). Brake ‘Top 10’ Our ‘Top 10’ Postcode District Population (over 17) Offences Offence Rate SL 6 61, 592 1, 847 30. 0 OX 49 3, 361 202 60. 1 NG 5 69, 330 1, 543 22. 3 WR 6 9, 930 585 58. 9 SL 1 53, 874 1, 537 28. 5 NE 41 1, 680 98 58. 3 ST 5 67, 336 1, 516 22. 5 SL 0 8, 842 509 57. 6 BH 23 44, 057 1, 503 34. 1 B 96 4, 165 233 55. 9 LE 2 94, 153 1, 459 15. 5 SA 3 21, 773 1, 166 53. 6 NG 16 52, 077 1, 411 27. 1 RM 19 4, 925 256 52. 0 NE 3 42, 181 1, 371 32. 5 RG 9 18, 431 943 51. 2 LE 3 78, 838 1, 341 17. 0 ST 12 3, 016 151 50. 1 KY 11 47, 256 1, 331 28. 2 NE 20 9, 600 475 49. 5

Where are Britain’s Most Dangerous Roads – Swiftcover – February 2015

AXA “Schools Road Safety Index” • The Proposal • Compare road safety trends for communities located near schools • The Analysis • Identify crashes in 500 m radius of each school, on non-motorway roads • Scope of six years, allowing identification of changes between three year periods • Express as rates per km of each road class (A, B, unclassified) • Identify broad comparability by network density and rurality • The Caveats • Most urban roads are within 500 m of a school (and often more than one) - so the data cannot be aggregated • Most collisions are not related to the presence of schools - so the data measures road safety trends in the community, not the school itself

Example – a school radius

Distortion upon distortion • This school radius happened to have the highest absolute number of crashes involving child casualties in its region • To a journalist, this apparently proves that it is “the worst area for road accidents” • Adding up separate numbers of crashes within 500 metres of each school inevitably results in ‘double counting’ • A journalist adds them together anyway, then inaccurately claims that “More than 1, 100 crashes happened outside [local] secondary schools over five [sic] years” • What’s in a name? • “Schools Road Safety Index” is not principally relevant to schools, and is not indexed • There is substantial disconnect between proposal and outcome