http://www.blackboxvoting.org/audit-methods.htm(This was updated in January from a previous posting. As to the plan, well, I can tell you that Bev is about as busy as one individual can be. Do we need that specific plan from Bev? Nope. Most of what needs to happen is right here)
What is "proper auditing" and why isn't it included in pending legislation?
— Bev Harris
Old news, still needed: You cannot audit properly without an independent record which has been verified by the voter. The system does not meet the "by the people" test unless citizens can see their vote for themselves — a receipt with a code number to check against an encrypted file on the Internet fails the "by the people" test. We need a voter-verified paper ballot (not "receipt," not "trail" — only a BALLOT has standing as a legal document).
But after we get a voter-verified paper ballot, what do we do with it? The prevailing (and incorrect) view is that all we need to do is use it for random spot-checks against the machine. Like the Logic & Accuracy tests, indeed we must do this, but (like the L&A tests) this method will not reliably catch fraud. Random spot-checks will help catch programming errors and fraud perpetrated by a moron. So what should we do?
I must tell you, these answers are evolving more slowly than I would like, partly because we are still asking computer scientists to give us answers for which they have little expertise. Just recently, following a Washington legislative session, legislators asked computer scientists to tell them what a statistically acceptable random spot-check percentage should be. That's nice, but it's only part of the answer. As I've been saying, we need to bring in the right kind of expertise. No one has yet done that, and now we're making things up as we go.
Here it is in a nutshell: We need to interview experienced elections officials to map out the life of a vote, in each of the voting methods. Vote-counting is a form of bookkeeping. We need to share this information on the life of a vote, including chain of custody and current safeguards, with forensic auditors and reformed embezzlers, and ask them to identify attack points and suggest preventive procedures.
Interviewing elections officials: Several of us at BlackBoxVoting.org have already begun citizen audits. We need detailed information from elections officials from both large urban areas and small rural areas, because the logistics of running elections vary depending on county population. We also need to interview officials who handle a variety of voting styles: touch-screen, optical-scan, punch-card, absentee, and mail-in.
Why reformed embezzlers and forensic accountants? If you want to prevent hacking, you want to interview some hackers. If you want to prevent the embezzling of votes, you need to talk to embezzlers. The "law & order" side of embezzling is forensic accounting. Note that you cannot bring these experts in until you have first mapped out the life of a vote in each voting method, because they need to analyze the specific system, not a theoretical one. Also, they will want to ask questions of elections officials after they identify potential attack points.
- Map the life of a vote in each system.
- Identify attack points with experts who have experience with attacking counting and record-keeping systems.
- Obtain suggestions from elections officials and reformed embezzlers/forensic accountants to plug the holes.
Then, we need to put some teeth into legislation to require proper auditing procedures and penalize noncompliance. Most locations have some auditing procedures in place.
- Almost all locations use a canvassing technique where the number of voters who sign in is compared with the number of votes cast. This is just one of several auditing methods needed.
- Some locations already require a random spot-check of paper ballots against machine tallies, but the techniques for "random" selection are not prescribed, and sometimes the selections aren't really randomly chosen.
- In California, a very important audit procedure is required — but is not used consistently, and there is no noncompliance penalty! This procedure involves posting a tally at the polling place which supposedly is compared with the tally at the county. Still, this needs more teeth: It needs a requirement that this posting be done before any communication with other locations about number of voters or vote results has been transmitted. What you want is comparison of independent data, never data that could be adjusted before running the report to match it up with something else. Also, these reports should be run in detail and for each voting machine, in addition to the polling place as a whole.
- Because several fairly serious attack points currently have no safeguard procedures set up, we need to add a few new auditing procedures.
Auditing procedures are often surprisingly easy and can cost almost nothing. They do require discipline and enforcement, however.
For example, running a report at the polling place costs virtually nothing, and takes very little time. You simply push the "print" button -- and yes, even the touch-screens have this function already built in.
The basic idea in auditing is that data (votes) should not change at all while passing through the system. Also, the number of votes needs to be matched in various ways — for example, number of voters who sign in matched with number of votes cast; number of absentee ballots received by the post office compared with number of absentee ballots transferred from post office to elections division. Most of the time, these are simple reports already available with current technology.
Strengthening audit procedures means setting up additional check-points as the vote data moves through the election. The data from each checkpoint must match the next. Any discrepancies mean we must pull the voter-verified paper ballots and count them by hand.
Digital data should change not a whit as it moves through the system. If you have a memory card or a cartridge at the polling place and it gives you the following report: "Jones, 104 votes, Smith, 221 votes" when you transfer that memory card or cartridge to a new location, it must still say exactly "Jones, 104 votes, Smith, 221 votes." Not a single vote must change.
So, one of our audit methods, a very simple one, will be to run a few more reports, compare them more rigorously, and insist on pulling the paper ballots if the digital data, or the report run from the digital data, becomes mismatched.
We need several additional checkpoints on our absentee ballot system. Even if we were to go back to an all-paper, all hand-counted system, we'd need to strengthen some of the safeguards (like requiring the U.S. Post Office to provide a report of the number of ballots received, and comparing that with a report of the number of ballots actually received by the elections division).
If we are to use a hybrid system (paper ballots and computer tallies), with voter-verified paper ballots, we need several more checkpoints. When memory cards, or cartridges, travel from place to place -- either by modem or in person -- we need a before and after report. Example: Report taken at polling place #116 before any results are phone in, modemed in, or driven in to a regional or county location. When the cards arrive, run another report, verify that the data is the same. After the data is tabulated into the overall county results, run a detail report on each of the precincts (yes, all of them) and compare to make sure all of them still match the original.
Observe how this reduces attack points: If you get a report from polling place #116 and before communicating with any other area, print it; and if this report says Smith 102, Jones 211; and then you transmit the votes by whatever method to the county and run another report after transmission, which should still say Smith 102, Jones 211 (thus showing that no one hacked in and inserted changes during upload), upon receipt at the county server, the county runs a report that still says Smith 102, Jones 211; and then the county merges vote data from all its polling places and gets a grand total, which it submits as the election results, you run another detail report on each polling place and #116 still shows Smith 102, Jones 211 — you have just removed most of the attack points between the initial report and the final tabulation.
Think of vote auditing procedures as checkpoints. Votes go from here to here to here. They must pass a checkpoint (print a report) at each stage. All the checkpoints must match. Simple!
Objections for the above: I have heard election officials give the inappropriate answer that comparing results as they travel through the system to make sure they still match won't work. They say it won't work because during the course of the election, absentee votes, provisional votes, challenge votes and mail-in votes are added to the polling place votes. In some places, mail-in votes are not differentiated from the polling place votes. Well now, here's the beauty of computers: Simply account for each type of vote on a separate line item. This is not difficult. It is not time-consuming. It's called "proper accounting."
For absentee voting, we also need to add some reports. The data we want to track here includes the number of ballots sent and recieved, and the information we require needs to include the detail of how many were sent, and received, from each 13-digit zip code, along with the weight of each piece.
The checkpoints for these absentee reports should include the number of ballots sent out from the elections division, from any presort facility or middleman, and from the post office. Incoming, compare the number of ballots (and weight of ballots) received by post office, presort facility, and elections division. Basically, you want a before-and-after number, and it should match.
We also need to formalize what happens when the audit turns up discrepancies. In a nutshell, when you discover a discrepancy, you pull the paper ballots and also expand the spot-check audit. If another discrepancy is discovered, you expand the audit further.
We have used some dreadful procedures for handling audit discrepancies up to now. Some elections officials have told me they need the ability to change vote totals in order to reconcile them. (Yes, really!). Diebold spokesman David Bear recently told a reporter that the reason the GEMS program allows entry of negative votes is in case the elections officials need to use such a function for some reason. News flash: There is NO legitimate reason, if you are doing proper bookkeeping, to have an election official override votes cast by citizens with "negative" votes.
The proper way to handle a situation where an adjustment is needed is to retain the original data in a pristine condition, add a "correction entry" and journal the reason why with a very clear and sufficiently detailed explanation. You never just go in and alter the original data.
When the audit reveals discrepancies, two things need to happen: First, we need to pull the paper ballots for that machine or polling place and examine them, and next, we need to review the other data to see if we find any more flawed data. In an audit, if you find a discrepancy, you expand the review.
For example, suppose you have 500 machines in the county, and five machines in polling place #116. You spot-check polling place #116 and discover that you are 100 votes short, and further examination shows that Machine D4 miscounted and dropped the 100 votes. If you do not expand the audit upon finding this discrepancy, you do not know if your county had 20 percent or one-quarter of a percent of its machines miscount. Before the expanded audit, you have a 20 percent failure rate (one out of five machines). You cannot assume it was one out of 500 machines (county-wide) — actually, you have no idea until you expand the audit.
A common auditing error that I am hearing from county elections officials is this: "If it wouldn't change the election, there is no need to expand the audit." This is incorrect. Let's look at the above example: You have 500 machines in the county, you spot-check 1 percent, or 5 machines. You find that one of these machines miscounts by 100 votes. The election spread was 1000 votes, so 100 votes couldn't make a difference, right?
Wrong — because, without expanding the audit, you may have a 20 percent machine failure (one in five machines). If one in 500 machines loses 100 votes and the spread is 1,000, it wouldn't change the election. If one in five machines loses 100 votes — that is, 100 of 500 machines — you've got a 10,000-vote discrepancy, which is ten times as much as you need to swing that election.
But is it practical to expand the auditing just because one machine miscounts? How much should we expand it? Here is where the statisticians should weigh in, but the concept is really quite simple: When the data is discrepant, you must expand the audit to a larger pool. If, in that larger pool, you find another discrepancy, you must expand to a still larger pool.
For example: You examine 1 percent and find that one machine of five did not count right. You then examine 10 percent and find one more machine that miscounted. At first, you were sitting at a 20 percent machine or programming malfunction (1 in 5 machines). After expanding the audit, you were sitting at a four percent problem (2 in 50 machines). Still too high. So you expand to a 50 percent audit and find two more machines that miscount. Now you are at 4 machines out of 250, a 1.6 percent failure rate.
And, of course, where one machine might drop 100 votes, another might drop 1,000. In Allamakee County, Iowa a machine miscounted by 3.9 million votes, and in Boone County, Indiana, another system miscounted by 139,000 votes. That's the trouble with computers. When they screw up, they can do so in a spectacular fashion. We can't assume that a 100-vote discrepancy in one computer means all other computers will miscount by only that amount.
It's simple: If you find a discrepancy, you expand the audit. Find another, expand it more. And folks, I really don't care about the explanation for the error. I will tell you this: embezzlers always have an explanation when you find a discrepancy. Doesn't matter what it is. If you find a discrepancy, you expand the audit.
I am working on developing a 10-point audit plan so that we can come to a meeting of the minds on what is needed. This is not a five-minute process, and cannot be done by computer scientists working in a university office. It requires getting out in the field to talk with elections officials who can reliably and accurately define the life of a vote in each of the various ways it is cast. It then involves meeting with auditors and, yes, I have it in mind to talk with an embezzler or two who have gone straight, to identify attack points and checkpoint procedures.
I'm hoping that we will begin to take auditing much more seriously. Our vote is not an experiment, and our democracy is too important to be making it up as we go. I am, frankly, a little horrified that people with few qualifications are telling legislators their supposedly authoritative opinions, and that, alternatively, people are out there writing legislation while talking to no auditing experts at all. Therefore, I'm gonna find experts myself, document what they have to say, write it up and present a 10-point set of procedures that tell us WHAT TO DO WITH THE PAPER BALLOTS!