Report of the Auditability Working Group David Flater

Report of the Auditability Working Group David Flater National Institute of Standards and Technology http: //vote. nist. gov TGDC Meeting, July 2010

Outline n Presentation n n n n Charge to the working group The goal of software independence (SI) What was actually required in the 2007 TGDC draft Alternatives to SI and their consequences Paper, voter-verification, and accessibility Effectivity concerns 3 options Debate n n TGDC and EAC discussion Resolutions (choose an option) TGDC Meeting, July 2010 Page 2

Charge to the working group Alternatives to Software Independence (SI) – EAC directs the TGDC to develop draft requirements for audit methods to achieve the goal of Software Independence (SI). The goal is to develop requirements for the auditability of the election system without requiring a specific technology. The starting point for these requirements should be the work already completed by NIST on alternatives to SI. TGDC Meeting, July 2010 Page 3

The SI rationale (abridged) n n The following is not the entire SI rationale, but it is the acid test that distinguishes SI from other forms of auditability Accept as plausible that there could be one rogue or coerced software engineer in each independent supplier of voting equipment to the jurisdiction n n Alternately, that each supplier relies on insecure COTS software that a third party can exploit, or that common mode failures exist, or etc. All electronic records potentially compromised If there are no other records, then it is not possible to compare records to audit the result The goal of SI, as abridged: mitigate this threat (and others that are easier) TGDC Meeting, July 2010 Page 4

Mitigation: independent voter-verifiable records n n Independent records enable a meaningful audit Voter-verification establishes independent validity Validated records must be protected from modification Paper records suffice n n Direct and indirect verification Ballots dropped into ballot box More difficult to achieve wholesale compromise of paper records without detection Alternatives that mitigated the threat without using paper were not prohibited in the 2007 TGDC draft TGDC Meeting, July 2010 Page 5

IVVR versus paper n What the 2007 TGDC draft actually required n n n Either independent voter-verifiable records (IVVR), or "Innovation class submission" Intent: the term IVVR was introduced specifically to avoid mandating paper Extent: paperless solutions are still researchy From absence of example, cannot conclude: n That the requirements are more restrictive than necessary to achieve the goal That no conforming paperless solution can possibly exist n Do not have working group consensus on these assertions n TGDC Meeting, July 2010 Page 6

Alternatives and consequences n Electronic Independent Verification Devices (e. g. Vote. Guard) n n Parallel testing n n n Arms race between complexity of testing and complexity of evading detection Cannot be required in the VVSG Punts the problem to poll workers Software assurance n n At best incomplete response to the rogue programmers threat Would require invasive, expensive changes to the development process and all-new systems End-to-end crypto still a research topic Unknown unknowns ("innovation class") TGDC Meeting, July 2010 Page 7

Tried a different approach n n Previous state: auditability = SI Suggested new state: auditability = ability to do an automated, independent recount n n Falls short of the SI goal if voter-verifiability is not included n n Automated, because manual counting is inaccurate Independent, so that it is a meaningful audit Want something comparable to shipping opscan ballots to neighboring county Paperless approaches = IVD At best incomplete response to the rogue programmers threat How much does "independent" entail Taking verification off the critical path or making it "random" TGDC Meeting, July 2010 Page 8

Paper, voter-verification, and accessibility n There have been misunderstandings about what exactly the 2007 TGDC draft required and did not require n n n 2007 TGDC draft reflected a difficult compromise n n Identical experience for every voter is infeasible Prohibiting or limiting voter-verification would not be a win Absence of conforming implementations raised objections n n Paper record accessibility requirements were intended to be more general (i. e. stronger) than in VVSG 1. 0 "Software independence" maybe conveyed that not allowed to use software for audio readback; that was not the intent Rejecting paper entirely versus requiring paper record accessibility If there is agreement to pursue some alternative to SI, then agreement on reasons for rejecting SI is not required If there is not agreement to pursue some alternative to SI, then a better compromise has not yet been identified TGDC Meeting, July 2010 Page 9

Effectivity concerns n n n n There have been misunderstandings about the impact of VVSG 2. 0 on already-deployed systems VVSG 2. 0 intended to be "forward-looking" for new certifications after some date EAC determines the date once new guidelines are approved Certificates issued under previous versions of the guidelines will not be revoked automatically when new guidelines are approved No mandate to retrofit or replace already-deployed systems "Worst" case: Assuming that a jurisdiction has [voluntarily adopted] a law that deployed systems must comply with latest EAC guidelines to be used in an election—approval of VVSG 2. 0 is not imminent V = Voluntary TGDC Meeting, July 2010 Page 10

Option #1 n n n Endorse one or more of the existing paperless alternatives Different alternatives have different implications and consequences Implied policy decisions n n IVD: reject the rogue programmers threat or accept an incomplete mitigation Parallel testing: accept difficult and/or incomplete procedural mitigation that is outside the scope of EAC certification Software assurance: commit to invasive, expensive changes to the development process and all-new systems Defining a higher-level auditability concept requires relaxing one of the constraints n Otherwise auditability = SI TGDC Meeting, July 2010 Page 11

Option #2 n Conclude that it was all a big misunderstanding n n Goal of SI + no mandate for paper is what we had in 2007 Paper ballot accessibility requirements—as intended n n n Accept that an example of a conforming system need not exist yet n n n No manual paper ballot handling (Acc-VS) Alternative format verification of complete paper ballot print content No mandate to retrofit or replace Engage Standards Board, Board of Advisors during the process Refocus on communication, first impressions TGDC Meeting, July 2010 Page 12

Option #3 n No misunderstanding—confirm the previous result n n n Accept the SI argument; accept the SI conclusion Market shifting to opscan Paper ballot accessibility requirements—as intended n n Manufacturers reportedly are responding to paper handling and readback requirements Fighting the previous battle? TGDC Meeting, July 2010 Page 13