MN Post Election Audit Summit (Day 1)
October 26th, 2007 (217 views )
I’m here at the Post Election Audit Summit in Minneapolis, Minnesota. There are about 100 election officials, technical types and advocates in the same room talking about the very difficult subject of auditing elections — essentially “checking the work” of elections systems, people and processes. I’ll keep running notes below the jump… We’ve already heard from Deforest B. Soaries (the first chair of the EAC) and Mark Ritchie (Minnesota Secretary of State) and our final keynote tomorrow will be given by Debra Bowen (California Secretary of State).
I’m keeping a flickr photoset of the summit; click the image at right to see more (yes those are M&M’s with “Yay Audits!” and “Audits Rock!” on them).
(PS: Don’t forget to check out my notes on Day 2 of the Summit.)
Panel 1: Why Audit?
The first panel (here’s the summit schedule) is called “Why Audit?” and is moderated by Kim Alexander (president of the California Voter Foundation) and includes Mary Batcher, Lesley Mara (deputy SoS for CT), Doug Jones (U Iowa, ACCURATE) and Lillie Coney (EPIC).
Mary Batcher talked about the VVSG II and the notion of software independence (that election software should not be a critical component that, if it fails, can threaten the results of an election).
Lesley Mara spoke about how audits are in the interest of election officials and, specifically, how CT used post-election audits to ease the transition from lever machines to computerized voting equipment. As is the case with Minnesota’s audits that Mark Ritchie spoke about, election officials thought that the audits were easier that they thought they were going to be and that the benefits far outweigh the costs.
Doug Jones talked about designing voting systems for auditability. He showed evidence that auditable voting machines date to the late 1800’s (patent 415,548 (1889)). Elections have been audited, inconsistently — in terms of methods and people responsible for the audit, for years. What if the audit shows a problem? Wouldn’t it be better to just not know? Doug says no. This is valuing a short-term benefit at tremendous long-term risk. When we do audit, we hobble auditors. E.g., the Sarasota FL-13 problem… while the complaints and issue seemed to cluster around vote-flipping (where a voter chooses one candidate and the machine records it for another candidate) but the audit activities concentrated on tabulation instead of human factors issues. Is there a role for automation in manual counts of ballots (I’m going to make the same point in my talk). Audits could do much more. We should include event log auditing.
Lillie Coney started talking about EPIC’s FOIA/Public Records work to get access to fundamental voting documents. Lillie then moved on to talk about lessons learned from financial auditing in terms of transparency, resource trade-offs. She then talked about citizen involvement in enforcing transparency requirements. What transparency isn’t: finding agreement, seeking approval, getting consciences and admissions of guilt. Challenges: turning elections activities into data, adequate resources, using auditing as a bridge for communication to the public about auditing.
Q&A: Did CT university replicate conditions under which elections are used to evaluate the technology? No, we wanted technical evaluation of the equipment. We wanted to know if we should pull the plug.
Ron Rivest clarified that the VVSG II doesn’t contemplate using purely electronic records, even if signed, until those types of systems can have standards written for them. You still need the paper, for now.
Panel 2: How & What to Audit I — Audit Types Compared, Which Contests/Ballot Categories to Audit
The second panel is the panel I’m sitting on… so I won’t be able to liveblog it. The panel went very well and we had tons of questions. I fully expect to be responding to questions for a long time. That is a very good sign… we need as much collaboration in this as we can get. I’ll put up the notes for my talk here: 20071025-PEAS-jhall.pdf.
Panel 3: Panel 3: How & What to Audit — Selection Process, Triggers, Escalation, Teams & Materials, Transparency
This panel was moderated by Mark Halvorson who talked briefly about Minnesota’s wonderful audit law.
Mark Lindeman (Bard) talked a bit more about transparency and used a definition from Transparency, International. He talked about how impediments to transparency can cause a “creation of enemies” amongst election officials and voters.
Arel Cordero (UC Berkeley) talked about transparency in selection of audit units. He talked about the 10-sided dice proposal (Cordero, Wagner and Dill 2006). As long as it is possible for public observers to convince themselves that it is truly random.
Patty O’Connor (Auditor, Blue Earth County, Minnesota) talked procedures associated with actually counting ballots. Separation, sorted by orientation, etc. They matched 100% of all votes in the two precincts that they had to audit.
Victor Addona (Mathematics & Computer Science, Macalester College) talked about using statistics in the entire audit process, from soup to nuts. He then went on to talk about statistically-triggered escalation procedures. He’s currently thinking about how large errors need to be to say, “we can’t certify this election”.
Phil Stark (UC Berkeley) talked about the properties of a statistical audit with a focus on what “properties” one might want to put into legislation. He mentioned that all the legislation he has seen doesn’t talk about what to do based on the outcome of an election audit. There needs to be a “complete” audit paradigm that tells you when to stop with a “certify/don’t certify” decision. Phil talked about his new paper which includes an upper-bound for the amount of error within a precinct that is possible for a given race. Short story: the maximum error is huge… and we don’t know where the psychology kicks in… that is, we don’t know at what point a certain error/fraud is noticeable.
Break for lunch 12:30-1:30
Panel #4: Resources — Time, People & Money
This panel was moderated by Pam Smith of the Verified Voting Foundation.
Bob Kibrick (VVF) presented work done with Gail Perelin of the Santa Cruz Department of Elections on resource constraints. Manual audit requirement was instituted in 1965. We need more data to understand the resource issues, we need data in the right form and standardized across counties.
Dean Logan (LA County) talked about LA County’s experience. The amount of time to conduct the audits and the space available are big issues. Who do you recruit? Small counties use staff… some use pollworkers (as long as they don’t audit the precinct in which they worked)… can recruit through parties… temporary workers. Need tight security in close races… need training staff… need people to mind observers. Advanced preparation is critical… there are a number of steps that have to be configured: ballot sorting and prep, work schedules, formal training for staff and observers, public notice requirements, facilities (space is large). Pre-certification vs. post-certification audit… need to think more about this; it’s promising. Litigation is another factor… you can expect in emotional and hotly contested races that there will be litigation. Money: personnel costs, supplies, space, security (very expensive) and litigation (very expensive). Observer guidelines, good documentation, voter intent guidelines. Dean showed some great numbers that I didn’t catch. Dang!
Matt Damschroder (Director, Franklin County Board of Elections, Ohio) talked about doing the first VVPAT audit in Ohio. Ohio law is not up-to-speed with modern elections… recount statutes don’t even reach punch-card or lever machines; focused on paper ballots. Needed to think about what we do if we come across a blank paper tape or tapes that have been jammed. Because of the margin in one particular race, they went with 10% recount instead of 3%. Spent about 800 man hours on that audit. It was largely successful because it gave us information about how to do the audit and what the costs would be… which is especially important if we need to do a county-wide recount during the Presidential election next year. We’d estimate that at about $80,000, at least. Challenges as incorporating audits: we need to stop shoehorning in one more thing into the process. The laws look like a bulletin board at a grocery store; anyone who had a great idea has something up there. Matt talked about using a power drill and two plastic rolls to advance the paper records.
Freddie Oakley (Yolo Co., CA Clerk-Recorder) talked about costs. She estimated, using only the essential activities for the Gubernatorial elections, would cost $3.38 per voter in Yolo County. Their audit cost 8 center per voter. She doesn’t think that’s a lot… she’d pay a quarter. If you value the audit at all, do you value it at 50 cents out of $3.38. Freddie talked about how Conny McCormack (“Dean’s boss”) would oppose any audit increase legislation… what she doesn’t understand is why costs (and their slightness) aren’t presented more fairly and clearly to the voter. “God bless Ron Rivest.” We need to be careful with how legislators use statistics. We need objective evidence on how much audits cost. She wants better audits.
Panel 5: Roundtable — Advantages & Pitfalls: Lessons learned from creating and implementing auditing regulations and legislation
Doug Chapin (Electionline) moderated this panel. This panel is about taking the theory and making it concrete for legislators and implementable by elections officials.
Douglas Kellner (NY State BoE) spoke about specific issues with New York audits and how they’re very much more concerned at the NY Board of Elections with Ballot Security.
William Kalsbeek (NC Statistician) talked about conducting NC’s recent audit. One interesting wrinkle wrt to NC is that they do their random selection non-publicly and keep the results in escrow until unofficial counting is completed. This is, of course, so that precincts/etc. not chosen cannot be targeted by maldoers. I wonder if the proceedings are recorded on video or something and released with the results?
Bill Hilty (State Representative, Minnesota) talked about Minnesota’s model audit law (CEIMN has a pointers in their audit report).
Jenny Flanagan (Executive Director, Common Cause, Colorado) talks about moving legislation for audits at the state level. You need a legislator that “get’s it” because auditing can be very complicated. It’s important to have county officials, “boots on the ground”, in the room and bought in to the value of the process. Of course, advocates need to be involved in order to integrate with the vocal public and ensure that their concerns are addressed.
That’s it for today… the rest is break out sessions which will be hard to blog… I’ll return later tonight to give any good updates.