Exitpoll analysis and prediction Stephen Fisher Following Curtice

Exit-poll analysis and prediction Stephen Fisher Following: Curtice, John and David Firth (2008) Exit polling in a cold climate: the BBC-ITV experience in Britain 2005, J. R. Stat. Soc. A, 171, 509 -539. Presentation to the NCRM-BPC Opinion Polls Conference, British Academy, 20 th January 2010

Broad research design principles: 1 • Model the pattern of change across constituencies in the share of the vote since the last election – i. e. not directly estimating the results in 2010 but change since 2005, since more variance between constituencies in shares than in changes. – Also, assessing and allowing for different swings in different seats

Broad research design principles: 2 • Estimate probabilities for each party winning each seat – i. e. allow for the random/unexplained variation between constituencies in the prediction • Predicted number of seats for each party is the sum of the probabilities across constituencies • Primary aim is to predict seat totals, not share of the vote

Infrastructure • 2010 exit poll will be a joint BBC/ITN/Sky project • Fieldwork by MORI and NOP, as in 2005 • People contributing to the analysis: Jouni Kuha (LSE), John Curtice (Strathclyde), Clive Payne (Oxford), Rob Ford (Manchester) • Debt of gratitude and computer code to David Firth

Selection of constituencies • Revisit all 107 viable locations from 2005 • Top up to 130 by sampling the kinds of constituencies thought to be useful and currently under-represented. – e. g. new decision to explicitly attempt to have a group of Lab-LD seats • Pick the most representative polling station in the constituency for the new locations

Exit poll locations in 2005

Statistical analysis • Model the change in the share of the vote since the last election • Consider data with and without interviewer guesses for those who refused – In 2005 ignoring the guesses and refusals worked best • Consider lots of different predictor variables – E. g. census data, market research data, strategic situation, incumbency – Expectations of geographical variation informed by practice with pre-election day polls • But keep the final model simple (N=130)

Producing predictions • Generate predicted probabilities for each party winning each seat from the statistical regression models of the data. – Using estimates of both explained and unexplained variance. • Sum the probabilities for each party across constituencies to estimate the total number of seats for the party. • In 2005 the exit-poll data suggested a Lab majority of 100 under uniform change, but the method accurately predicted 66 – Introduction of explained (regression) and unexplained (probabilistic prediction) variation both equally account for the difference between uniform change and final prediction.

Probabilistic prediction compared with the swingometer • What would be the effect of allowing for unexplained variation in a swingometer estimate of the result? – Depends on the distribution of seats according to marginality. – Smoothes the relationship between predicted swing and predicted seats.

Need for probabilistic prediction in 2005

Marginal Lab-Con seats for 2010 E. g. probabilistic method would predict fewer Con seats from a 7% swing than the swingometer because it would allow for the seats immediately either side of the 7% point to split between Con and Lab, and there are more to the left than right. -But not much difference.

Simulation 1: Stability of notional 2005 results • Rerun 2005 notional results but adding noise to allow probabilistic results – Seats with 05 margin <1% become 50: 50 – Seats with 05 margin c. 4% become 90: 10 • Changes the seat totals from Con 210, Lab 348 to Con 212, Lab 346 with LD unchanged. – i. e. not much change

Simulation 2: Poll projection • ukpollingreport. co. uk average of polls: – Con 41, Lab 29, LD 18 – i. e. +8, -7, -5 since 2005 • Uniform swing: Con majority of 44 • Probabilistic projection: Con majority of 48 • Very little difference

Simulation 2: Seats in the balance for the Tories under the simulation constituency wp 05 sp 05 LAB LAB LAB LD LD LAB LAB LD CON CON CON SNP CON CON LD CON CON CON pctmaj 05 conprw labprw 0. 41 0. 42 0. 43 0. 44 0. 45 0. 47 0. 48 0. 49 0. 50 0. 53 0. 54 0. 57 0. 59 0. 58 0. 57 0. 00 0. 56 0. 00 0. 53 0. 21 0. 52 0. 00 0. 50 0. 00 0. 46 0. 26 0. 00 ldprw Dagenham & Rainham Erewash Norwich South Bath Leeds North East Crewe & Nantwich Ochil & South Perthshire Oxford West & Abingdon Newport West Warwickshire North Hampstead & Kilburn Coventry South Dorset Mid & Poole North Argyll & Bute Telford Berwickshire, Roxburgh & Selkirk Winchester Luton South Brighton Pavilion St. Austell & Newquay 15. 7 7. 4 13. 6 15. 5 13. 4 15. 3 1. 1 15. 2 13. 1 13. 0 15. 0 13. 0 12. 7 14. 7 13. 1 12. 4 0. 00 0. 57 0. 00 0. 55 0. 00 0. 32 0. 00 0. 51 0. 50 0. 00 0. 50 0. 47 0. 00 0. 43

Simulation 2 – Distribution of Predicted Probabilities

Particular difficulties for exit poll prediction in 2010 • Boundary changes – Possible errors in both dependent and explanatory variables • Expenses scandal – More MPs stepping down – Potentially more variance between constituencies • Census data old (2001) • High expectations from 2005!