Shallow Thoughts : tags : elections

Plotting Shapes with Python Basemap wwithout Shapefiles

In my article on Plotting election (and other county-level) data with Python Basemap, I used ESRI shapefiles for both states and counties.

But one of the election data files I found, OpenDataSoft's USA 2016 Presidential Election by county had embedded county shapes, available either as CSV or as GeoJSON. (I used the CSV version, but inside the CSV the geo data are encoded as JSON so you'll need JSON decoding either way. But that's no problem.)

Just about all the documentation I found on coloring shapes in Basemap assumed that the shapes were defined as ESRI shapefiles. How do you draw shapes if you have latitude/longitude data in a more open format?

As it turns out, it's quite easy, but it took a fair amount of poking around inside Basemap to figure out how it worked.

In the loop over counties in the US in the previous article, the end goal was to create a matplotlib Polygon and use that to add a Basemap patch. But matplotlib's Polygon wants map coordinates, not latitude/longitude.

If m is your basemap (i.e. you created the map with m = Basemap( ... ), you can translate coordinates like this:

    (mapx, mapy) = m(longitude, latitude)

So once you have a region as a list of (longitude, latitude) coordinate pairs, you can create a colored, shaped patch like this:

    for coord_pair in region:
        coord_pair[0], coord_pair[1] = m(coord_pair[0], coord_pair[1])
    poly = Polygon(region, facecolor=color, edgecolor=color)
    ax.add_patch(poly)

Working with the OpenDataSoft data file was actually a little harder than that, because the list of coordinates was JSON-encoded inside the CSV file, so I had to decode it with json.loads(county["Geo Shape"]). Once decoded, it had some counties as a Polygonlist of lists (allowing for discontiguous outlines), and others as a MultiPolygonlist of list of lists (I'm not sure why, since the Polygon format already allows for discontiguous boundaries)

And a few counties were missing, so there were blanks on the map, which show up as white patches in this screenshot. The counties missing data either have inconsistent formatting in their coordinate lists, or they have only one coordinate pair, and they include Washington, Virginia; Roane, Tennessee; Schley, Georgia; Terrell, Georgia; Marshall, Alabama; Williamsburg, Virginia; and Pike Georgia; plus Oglala Lakota (which is clearly meant to be Oglala, South Dakota), and all of Alaska.

One thing about crunching data files from the internet is that there are always a few special cases you have to code around. And I could have gotten those coordinates from the census shapefiles; but as long as I needed the census shapefile anyway, why use the CSV shapes at all? In this particular case, it makes more sense to use the shapefiles from the Census.

Still, I'm glad to have learned how to use arbitrary coordinates as shapes, freeing me from the proprietary and annoying ESRI shapefile format.

The code: Blue-red map using CSV with embedded county shapes

Tags: elections, politics, visualization, programming, data, open data, data, matplotlib, government, transparency
[ 09:36 Jan 19, 2017    More programming | permalink to this entry | ]

Plotting election (and other county-level) data with Python Basemap

After my arduous search for open 2016 election data by county, as a first test I wanted one of those red-blue-purple charts of how Democratic or Republican each county's vote was.

I used the Basemap package for plotting. It used to be part of matplotlib, but it's been split off into its own toolkit, grouped under mpl_toolkits: on Debian, it's available as python-mpltoolkits.basemap, or you can find Basemap on GitHub.

It's easiest to start with the fillstates.py example that shows how to draw a US map with different states colored differently. You'll need the three shapefiles (because of ESRI's silly shapefile format): st99_d00.dbf, st99_d00.shp and st99_d00.shx, available in the same examples directory.

Of course, to plot counties, you need county shapefiles as well. The US Census has county shapefiles at several different resolutions (I used the 500k version). Then you can plot state and counties outlines like this:

from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt

def draw_us_map():
    # Set the lower left and upper right limits of the bounding box:
    lllon = -119
    urlon = -64
    lllat = 22.0
    urlat = 50.5
    # and calculate a centerpoint, needed for the projection:
    centerlon = float(lllon + urlon) / 2.0
    centerlat = float(lllat + urlat) / 2.0

    m = Basemap(resolution='i',  # crude, low, intermediate, high, full
                llcrnrlon = lllon, urcrnrlon = urlon,
                lon_0 = centerlon,
                llcrnrlat = lllat, urcrnrlat = urlat,
                lat_0 = centerlat,
                projection='tmerc')

    # Read state boundaries.
    shp_info = m.readshapefile('st99_d00', 'states',
                               drawbounds=True, color='lightgrey')

    # Read county boundaries
    shp_info = m.readshapefile('cb_2015_us_county_500k',
                               'counties',
                               drawbounds=True)

if __name__ == "__main__":
    draw_us_map()
    plt.title('US Counties')
    # Get rid of some of the extraneous whitespace matplotlib loves to use.
    plt.tight_layout(pad=0, w_pad=0, h_pad=0)
    plt.show()

Accessing the state and county data after reading shapefiles

Great. Now that we've plotted all the states and counties, how do we get a list of them, so that when I read out "Santa Clara, CA" from the data I'm trying to plot, I know which map object to color?

After calling readshapefile('st99_d00', 'states'), m has two new members, both lists: m.states and m.states_info.

m.states_info[] is a list of dicts mirroring what was in the shapefile. For the Census state list, the useful keys are NAME, AREA, and PERIMETER. There's also STATE, which is an integer (not restricted to 1 through 50) but I'll get to that.

If you want the shape for, say, California, iterate through m.states_info[] looking for the one where m.states_info[i]["NAME"] == "California". Note i; the shape coordinates will be in m.states[i]n (in basemap map coordinates, not latitude/longitude).

Correlating states and counties in Census shapefiles

County data is similar, with county names in m.counties_info[i]["NAME"]. Remember that STATE integer? Each county has a STATEFP, m.counties_info[i]["STATEFP"] that matches some state's m.states_info[i]["STATE"].

But doing that search every time would be slow. So right after calling readshapefile for the states, I make a table of states. Empirically, STATE in the state list goes up to 72. Why 72? Shrug.

    MAXSTATEFP = 73
    states = [None] * MAXSTATEFP
    for state in m.states_info:
        statefp = int(state["STATE"])
        # Many states have multiple entries in m.states (because of islands).
        # Only add it once.
        if not states[statefp]:
            states[statefp] = state["NAME"]

That'll make it easy to look up a county's state name quickly when we're looping through all the counties.

Calculating colors for each county

Time to figure out the colors from the Deleetdk election results CSV file. Reading lines from the CSV file into a dictionary is superficially easy enough:

    fp = open("tidy_data.csv")
    reader = csv.DictReader(fp)

    # Make a dictionary of all "county, state" and their colors.
    county_colors = {}
    for county in reader:
        # What color is this county?
        pop = float(county["votes"])
        blue = float(county["results.clintonh"])/pop
        red = float(county["Total.Population"])/pop
        county_colors["%s, %s" % (county["name"], county["State"])] \
            = (red, 0, blue)

But in practice, that wasn't good enough, because the county names in the Deleetdk names didn't always match the official Census county names.

Fuzzy matches

For instance, the CSV file had no results for Alaska or Puerto Rico, so I had to skip those. Non-ASCII characters were a problem: "Doña Ana" county in the census data was "Dona Ana" in the CSV. I had to strip off " County", " Borough" and similar terms: "St Louis" in the census data was "St. Louis County" in the CSV. Some names were capitalized differently, like PLYMOUTH vs. Plymouth, or Lac Qui Parle vs. Lac qui Parle. And some names were just different, like "Jeff Davis" vs. "Jefferson Davis".

To get around that I used SequenceMatcher to look for fuzzy matches when I couldn't find an exact match:

def fuzzy_find(s, slist):
    '''Try to find a fuzzy match for s in slist.
    '''
    best_ratio = -1
    best_match = None

    ls = s.lower()
    for ss in slist:
        r = SequenceMatcher(None, ls, ss.lower()).ratio()
        if r > best_ratio:
            best_ratio = r
            best_match = ss
    if best_ratio > .75:
        return best_match
    return None

Correlate the county names from the two datasets

It's finally time to loop through the counties in the map to color and plot them.

Remember STATE vs. STATEFP? It turns out there are a few counties in the census county shapefile with a STATEFP that doesn't match any STATE in the state shapefile. Mostly they're in the Virgin Islands and I don't have election data for them anyway, so I skipped them for now. I also skipped Puerto Rico and Alaska (no results in the election data) and counties that had no corresponding state: I'll omit that code here, but you can see it in the final script, linked at the end.

    for i, county in enumerate(m.counties_info):
        countyname = county["NAME"]
        try:
            statename = states[int(county["STATEFP"])]
        except IndexError:
            print countyname, "has out-of-index statefp of", county["STATEFP"]
            continue

        countystate = "%s, %s" % (countyname, statename)
        try:
            ccolor = county_colors[countystate]
        except KeyError:
            # No exact match; try for a fuzzy match
            fuzzyname = fuzzy_find(countystate, county_colors.keys())
            if fuzzyname:
                ccolor = county_colors[fuzzyname]
                county_colors[countystate] = ccolor
            else:
                print "No match for", countystate
                continue

        countyseg = m.counties[i]
        poly = Polygon(countyseg, facecolor=ccolor)  # edgecolor="white"
        ax.add_patch(poly)

Moving Hawaii

Finally, although the CSV didn't have results for Alaska, it did have Hawaii. To display it, you can move it when creating the patches:

    countyseg = m.counties[i]
    if statename == 'Hawaii':
        countyseg = list(map(lambda (x,y): (x + 5750000, y-1400000), countyseg))
    poly = Polygon(countyseg, facecolor=countycolor)
    ax.add_patch(poly)

The offsets are in map coordinates and are empirical; I fiddled with them until Hawaii showed up at a reasonable place.

Well, that was a much longer article than I intended. Turns out it takes a fair amount of code to correlate several datasets and turn them into a map. But a lot of the work will be applicable to other datasets.

Full script on GitHub: Blue-red map using Census county shapefile

Tags: elections, politics, visualization, programming, python, mapping, GIS, data, open data, matplotlib, government, transparency
[ 15:10 Jan 14, 2017    More programming | permalink to this entry | ]

Getting Election Data, and Why Open Data is Important

Back in 2012, I got interested in fiddling around with election data as a way to learn about data analysis in Python. So I went searching for results data on the presidential election. And got a surprise: it wasn't available anywhere in the US. After many hours of searching, the only source I ever found was at the UK newspaper, The Guardian.

Surely in 2016, we're better off, right? But when I went looking, I found otherwise. There's still no official source for US election results data; there isn't even a source as reliable as The Guardian this time.

You might think Data.gov would be the place to go for official election results, but no: searching for 2016 election on Data.gov yields nothing remotely useful.

The Federal Election Commission has an election results page, but it only goes up to 2014 and only includes the Senate and House, not presidential elections. Archives.gov has popular vote totals for the 2012 election but not the current one. Maybe in four years, they'll have some data.

After striking out on official US government sites, I searched the web. I found a few sources, none of them even remotely official.

Early on I found Simon Rogers, How to Download County-Level Results Data, which leads to GitHub user tonmcg's County Level Election Results 12-16. It's a comparison of Democratic vs. Republican votes in the 2012 and 2016 elections (I assume that means votes for that party's presidential candidate, though the field names don't make that entirely clear), with no information on third-party candidates.

KidPixo's Presidential Election USA 2016 on GitHub is a little better: the fields make it clear that it's recording votes for Trump and Clinton, but still no third party information. It's also scraped from the New York Times, and it includes the scraping code so you can check it and have some confidence on the source of the data.

Kaggle claims to have election data, but you can't download their datasets or even see what they have without signing up for an account. Ben Hamner has some publically available Kaggle data on GitHub, but only for the primary. I also found several companies selling election data, and several universities that had datasets available for researchers with accounts at that university.

The most complete dataset I found, and the only open one that included third party candidates, was through OpenDataSoft. Like the other two, this data is scraped from the NYT. It has data for all the minor party candidates as well as the majors, plus lots of demographic data for each county in the lower 48, plus Hawaii, but not the territories, and the election data for all the Alaska counties is missing.

You can get it either from a GitHub repo, Deleetdk's USA.county.data (look in inst/ext/tidy_data.csv. If you want a larger version with geographic shape data included, clicking through several other opendatasoft pages eventually gets you to an export page, USA 2016 Presidential Election by county, where you can download CSV, JSON, GeoJSON and other formats.

The OpenDataSoft data file was pretty useful, though it had gaps (for instance, there's no data for Alaska). I was able to make my own red-blue-purple plot of county voting results (I'll write separately about how to do that with python-basemap), and to play around with statistics.

Implications of the lack of open data

But the point my search really brought home: By the time I finally found a workable dataset, I was so sick of the search, and so relieved to find anything at all, that I'd stopped being picky about where the data came from. I had long since given up on finding anything from a known source, like a government site or even a newspaper, and was just looking for data, any data.

And that's not good. It means that a lot of the people doing statistics on elections are using data from unverified sources, probably copied from someone else who claimed to have scraped it, using unknown code, from some post-election web page that likely no longer exists. Is it accurate? There's no way of knowing.

What if someone wanted to spread news and misinformation? There's a hunger for data, particularly on something as important as a US Presidential election. Looking at Google's suggested results and "Searches related to" made it clear that it wasn't just me: there are a lot of people searching for this information and not being able to find it through official sources.

If I were a foreign power wanting to spread disinformation, providing easily available data files -- to fill the gap left by the US Government's refusal to do so -- would be a great way to mislead people. I could put anything I wanted in those files: there's no way of checking them against official results since there are no official results. Just make sure the totals add up to what people expect to see. You could easily set up an official-looking site and put made-up data there, and it would look a lot more real than all the people scraping from the NYT.

If our government -- or newspapers, or anyone else -- really wanted to combat "fake news", they should take open data seriously. They should make datasets for important issues like the presidential election publically available, as soon as possible after the election -- not four years later when nobody but historians care any more. Without that, we're leaving ourselves open to fake news and fake data.

Tags: elections, politics, programming, data, open data, fake news, government, transparency
[ 16:41 Jan 12, 2017    More politics | permalink to this entry | ]

New Mexico LWV Voter Guides are here!

I'm happy to say that our state League of Women Voters Voter Guides are out for the 2016 election.

My grandmother was active in the League of Women Voters most of her life (at least after I was old enough to be aware of such things). I didn't appreciate it at the time -- and I also didn't appreciate that she had been born in a time when women couldn't legally vote, and the 19th amendment, giving women the vote, was ratified just a year before she reached voting age. No wonder she considered the League so important!

The LWV continues to work to extend voting to people of all genders, races, and economic groups -- especially important in these days when the Voting Rights Act is under attack and so many groups are being disenfranchised. But the League is important for another reason: local LWV chapters across the country produce detailed, non-partisan voter guides for each major election, which are distributed free of charge to voters. In many areas -- including here in New Mexico -- there's no equivalent of the "Legislative Analyst" who writes the lengthy analyses that appear on California ballots weighing the pros, cons and financial impact of each measure. In the election two years ago, not that long after Dave and I moved here, finding information on the candidates and ballot measures wasn't easy, and the LWV Voter Guide was by far the best source I saw. It's the main reason I joined the League, though I also appreciate the public candidate forums and other programs they put on.

LWV chapters are scrupulous about collecting information from candidates in a fair, non-partisan way. Candidates' statements are presented exactly as they're received, and all candidates are given the same specifications and deadlines. A few candidates ignored us this year and didn't send statements despite repeated emails and phone calls, but we did what we could.

New Mexico's state-wide voter guide -- the one I was primarily involved in preparing -- is at New Mexico Voter Guide 2016. It has links to guides from three of the four local LWV chapters: Los Alamos, Santa Fe, and Central New Mexico (Albuquerque and surrounding areas). The fourth chapter, Las Cruces, is still working on their guide and they expect it soon.

I was surprised to see that our candidate information doesn't include links to websites or social media. Apparently that's not part of the question sheet they send out, and I got blank looks when I suggested we should make sure to include that next time. The LWV does a lot of important work but they're a little backward in some respects. That's definitely on my checklist for next time, but for now, if you want a candidate's website, there's always Google.

I also helped a little on Los Alamos's voter guide, making suggestions on how to present it on the website (I maintain the state League website but not the Los Alamos site), and participated in the committee that wrote the analysis and pro and con arguments for our contentious charter amendment proposal to eliminate the elective office sheriff. We learned a lot about the history of the sheriff's office here in Los Alamos, and about state laws and insurance rules regarding sheriffs, and I hope the important parts of what we learned are reflected in both sides of the argument.

The Voter Guides also have a link to a Youtube recording of the first Los Alamos LWV candidate forum, featuring NM House candidates, DA, Probate judge and, most important, the debate over the sheriff proposition. The second candidate forum, featuring US House of Representatives, County Council and County Clerk candidates, will be this Thursday, October 13 at 7 (refreshments at 6:30). It will also be recorded thanks to a contribution from the AAUW.

So -- busy, busy with election-related projects. But I think the work is mostly done (except for the one remaining forum), the guides are out, and now it's time to go through and read the guides. And then the most important part of all: vote!

Tags: elections, voting, LWV, government
[ 16:08 Oct 11, 2016    More politics | permalink to this entry | ]

Was the 2004 Election Stolen?

Anyone following the voting machine controversy in the last presidential election -- or, even more, anyone who wasn't following it and might not be aware of the issues -- should check out Robert F. Kennedy, Jr's article in Rolling Stone, Was the 2004 Election Stolen?

The article is long, detailed and well researched article, and it will make you question whether we really live in a democracy.

Apparently Kennedy is considering filing whistleblower lawsuits against two of the voting machine companies. This won't do anything to change our national elections, but at least it might help get the word -- and the evidence -- out into the public eye.

Tags: politics, elections, voting, government
[ 12:36 Jul 06, 2006    More politics | permalink to this entry | ]

Boxer to Support Conyers in Protest Over Ohio Vote

Senator Barbara Boxer (D-CA) has signed a protest launched by Rep. John Conyers (D-MI) regarding irregularities in the Ohio vote, as reported this morning by the AP (via Yahoo, via ABC News).

Conyers' report can be found on the House Committee on the Judiciary's page, including the PDF report and some supplementary documents (all PDF except the video): a film by Linda Byrket called "Video the Vote", text of a fundraising letter Ohio Secretary of State Kenneth Blackwell, and Eyewitness Accounts of Ohio Voter Disenfranchisement. Conyers' report is described in this Fox News story.

John Kerry has not joined the protest.

This is not expected to alter the outcome of the 2004 election; both houses are expected to certify the election tomorrow. But it will force both houses to break from election certification tomorrow, and have a public discussion of up to two hours on some of the problems seen in the election. Perhaps it will pave the way for changes in future elections.

Tags: politics, election04, elections, voting
[ 11:29 Jan 06, 2005    More politics/election04 | permalink to this entry | ]

Programmer alleges FL congressman commissioned vote altering code

This story has been floating around for a few days now, but I've hesitated to write about it because it sounds potentially fishy and I was hoping some of the questions would get answered.

In a nutshell: Florida programmer Clint Curtis has filed documents with the FBI claiming that while he was working for Yang Enterprises, Tom Feeny (then a FL state representative and lobbyist for Yang, now a US Congressman) asked him to develop prototype software in order to rig the vote in Florida. (story in Wired) (story on Blue Lemur)

All rather suspicious, but there are lots of questionable aspects to the story. Why did Curtis wait so long to come clean? He claims that he assumed any such software would be easily detectable through source code inspection, and it was only after recently reading that voting software was proprietary that he had the shocking realization that perhaps there wasn't much source code review going on. It's hard to believe that a programmer who had worked on such a project would have been able to miss this point for so long.

Curtis has apparently also been to the FBI complaining about Yang's ethics before, on an unrelated charge. Details are skimpy about what that charge was, or what the resolution was, but until those details are available, one has to be slightly skeptical.

On Curtis' side, the fact that Yang nor Sweeney are willing to comment on the story suggests that there may be some truth to it. If his past allegations against Yang, or other aspects of the case, cast doubt on his claims, wouldn't they be pointing to that?

That the FBI is unwilling to comment is not surprising: investigation is ongoing, and I wouldn't expect any comment from investigators at this point.

It seems unlikely that Curtis' actual code was used, in any case. He had no access to the voting machine software, and simply wrote some scripts in Visual Basic as a proof of concept. But we'll likely never know for sure, since the public hasn't had access to the voting machines for quite some time and it would be quite easy for any such evidence to have been long since wiped from memory. (Though perhaps forensic analysis of the disks might reveal something?)

Still, it's an interesting story, and it'll be fun to see how it resolves.

Tags: politics, election04, elections, voting
[ 14:20 Dec 14, 2004    More politics/election04 | permalink to this entry | ]

Blackbox Voting Exposes Poll Tape Fraud

Installment one of Bev Harris and BlackBoxVoting.org's Freedom of Information Request: the Stinking Poll Tapes.

Harris & company went to Volusia County, Florida to request the "poll tapes" from the election: the printed record that each machine produces at the end of the day, signed and dated by election workers.

What they were given was unsigned printouts dated November 16, the day before their arrival.

Upon investigating, they found several curious things:

1. Elections officials meeting clustered over poll tapes, who shut the door on them when they asked what was going on;
2. A garbage bag full of original, signed poll tapes, dated the day of the election;
3. Another garbage bag of original poll tapes at a different location;
4. Apparent discrepancies between the original, signed, dated poll tapes and the supposed copies which the elections officials had originally tried to give them.

This is all over the blogosphere, but doesn't appear to have hit much of the mainstream press so far, not even Wired, except for one early article in the East Volusia News-JournalOnline. But the story making the rounds claims Black Box Voting has it all on video.

Stay tuned!

Tags: politics, election04, elections, voting
[ 00:12 Nov 20, 2004    More politics/election04 | permalink to this entry | ]

93,000 Extra Votes In Cuyahoga County?

Teed Rockwell, of the Philosophy Department, Sonoma State University, published a few days ago a sizzling article on ballot totals in Cuyahoga County, Ohio. Using the numbers from the county's official election results web site, he shows 29 different precincts which report vote counts well in excess of the total number of registered voters, for a grand total of 93,136 more votes than registered voters. For example, Highland Hills Village, which has 760 registered voters, had 8,822 ballots cast.

One possible explanation comes in an AP story, Kerry campaign lawyers checking Ohio vote, which says that "the numbers also include absentee votes in congressional and legislative districts that overlap those cities", which wrongly inflates the numbers, and quotes Ohio elections board chairman Michael Vu as saying "All the numbers are correct. You have to first understand what an absentee precinct is." The story doesn't go on to explain what an absentee precinct is; it looks like absentee ballots are assigned to counties other than the county of registration, or possibly absentee voters aren't included in registration numbers at all.

Meanwhile, a blog called "Political Strategy" reports on an editorial on the Zogby pollling web site, in Zogby Website Asserts 'Massive Voter Fraud'. I can't actually read the linked Zogby page (either they've pulled it, or they have some sort of bug in their server code) but in addition to calling attention to the fishy Cuyahoga results, they discuss the statistical unliklihood of some of the Florida results already showcased elsewhere.

Recount update: Cobb (Green) and Badnarik (Libertarian) are officially requesting an Ohio recount, while Nader and Camejo have requested a recount in New Hampshire. There's more recount news on ReDefeat Bush (which I found by way of their Google ad when I googled for recount news -- cool!)

A final giggle: on the subject of why the exit polls were so wrong (I still haven't seen anyone quoting numbers!), Craig Crawford of Congressional Quarterly and CBS suggested that the exit polls may have been wrong about Bush because of the "David Duke effect," an election in which he got many more votes than was reflected in what pollsters found because "people didn't want to admit to exit pollsters they'd voted for David Duke, the head of the Ku Klux Klan, because they didn't want to admit they were a racist. So perhaps a lot of voters didn't want to admit they voted for Bush."

Tags: politics, election04, elections, voting
[ 22:11 Nov 15, 2004    More politics/election04 | permalink to this entry | ]

Recent roundup of post-election voting stories

I've been hearing a lot of talk about how the official results don't match the exit poll numbers: how the exit polls show a Kerry win, and that's evidence of a hacked vote. For example, Those faulty exit polls were sabotage in The Hill, or A Tour of the 2004 Exit Poll: What It Says and What It Doesn't, part one and part two on Donkey Rising.

What I haven't been able to find is anything with data to confirm this, one way or the other. CNN has an interactive page allowing checks of specific aspects of exit poll data, but that's no help for analyzing nationwide data, say, by county. And in any case, it seems that CNN changed the online data after the fact, so there's no telling what this means in terms of raw numbers.

Lawrence Lessig gives the answer, in Free the Exit Poll Data: the poll numbers are privately held, not publically available. Lessig calls for the data to be made public, so that it will be possible to find out why the numbers were so misleading compared to the final election tally. You'd think both sides would be interested in knowing what went wrong.

Terrific maps for visualizing the election

Maps and cartograms of the 2004 US presidential election results gives a wonderful set of maps showing "purple states" by county, with the sizes adjusted for population.

Other stories about voting irregularities:

Outrage in Ohio: Angry residents storm State House in response to massive voter suppression and corruption (Michigan Independant Media Center): Protests on November 3 in Ohio over all the voting problems the state experienced. Includes lots of anecdotes about voters who experienced problems.

Surprising Pattern of Florida's Election Results (US Together): a comparison of party registration data to reported election results in Florida counties using different types of voting equipment. In counties using touchscreen machines, the percentage vote for Kerry matched the party registrations fairly closely; in counties using optical scan machines, there's a huge shift over to Bush votes, completely uncorrelated with party affiliation. The article includes a data table by county.

Evidence Mounts That The Vote May Have Been Hacked (Common Dreams): a text discussion of the US Together results, their correlation with exit poll results, and some discussion of possible explanations other than foul play (and why those reasons are unlikely to be the actual explanation).

Palm Beach County Logs 88,000 More Votes Than Voters (Washington Dispatch): Palm Beach County's official election results web site showed 542,835 ballots were cast for a presidential candidate while only 454,427 voters turned out for the election. Apparently they've since updated the web site to show numbers that add up. I guess this tells us how far we can trust the "official" numbers on the web site.

Tons of other links on the Op Ed News: Votergate 2004 page.

Bev Harris of Black Box Voting, Ralph Nader and others have teamed up for Help America Recount, a project to buy recounts in Ohio and other states. They're soliciting donations. I'd love to see recounts, but what they don't explain is where the money is going. What's involved in getting a recount, and does it cost money, or is this to pay salaries and expenses of the (volunteer?) people doing the counting, or what? The effort sounds like it might be a little disorganized at the moment.

Kerry Won. . . (Tom Paine.common Sense): Editorial about irregularities in various states. No new data, though.

Tags: politics, election04, elections, voting
[ 12:31 Nov 12, 2004    More politics/election04 | permalink to this entry | ]

Update: some recent stories on election irregularities

An older style touchscreen machine made by Danaher Controls gave Bush 3,893 extra votes in suburban Columbus.

In one North Carolina county, more than 4,500 votes were lost because officials believed a computer that stored ballots electronically could hold more data than it did. UniLect, the manufacturer of the touchscreen machines used, told officials that each storage unit could handle 10,500 votes, but the limit was actually 3,005 votes. The missing votes are gone forever; there is no way to retrieve them.

In Broward County, FL (remember the missing absentee ballots?) it was discovered that a bug in an ES&S machine changed the outcome on at least one proposition. Seems that the software (for counting votes on absentee ballots) doesn't expect more than 32,000 votes in a precinct; so when the tally crosses that number, the machine starts counting backward!

Meanwhile, the ACLU is suing over the lost Broward County absentee ballots.

A national voting rights group has reported hundreds of voting irregularities in the south affecting poor and minority voters.

Latest word (from Equal Vote) is that Ohio Secretary of State Ken Blackwell has said that Ohio's provisional votes will not be counted for 11 days (if at all).

Black Box Voting has filed a massive Freedom of Information Act request for computer logs (including internal audit logs, transmission logs, and others), voting results slips, any email or other communication relating to problems with voting systems, and other information relating to the operation of electronic voting machines.

Voters Unite has an excellent listing of stories on many other voting problems found so far.

Tags: politics, election04, elections, voting
[ 11:05 Nov 06, 2004    More politics/election04 | permalink to this entry | ]

WIMPS!

I knew the Demo-wimps were going to fold, just like they did in 2000 -- but I didn't think they'd do it before the first vote count was even finished!

I can't believe Kerry conceded already. What about all the promises the Democrats have been making us for the past several months about pushing lawsuits on voting technology and voter eligibility? What about all the lawsuits already filed?

I guess nobody cares any more that there's no way to verify anyone's vote, that the voting technology of the country is entirely in the hands of one party. A show of democracy is all that's required; the actual votes, from actual citizens, are far less important than the pretense of voting.

The morning's quick summary of voting machine glitches reported yesterday, at Wired: Watchdogs Spot E-Vote Glitches. The stories include ballots already pre-filled in Palm Beach County, FL, reports of misvoting (touching the box for one candidate and seeing an "X" appear by a different candidate) in FL, TX, and other states, machines in Texas instructed to vote straight party tickets actually casting votes for candidates outside that party, and voters in six Pennsylvania precints prevented from voting due to voting machine failures,

I should mention that Wired has had the best and most comprehensive coverage all along of the e-voting fiasco, beginning many months before any of the other mainstream media would mention the subject. Follow the links from that story, or just search for keywords like voting machines or Diebold. Or check out the original anti voting machine activist site: BlackBoxVoting.com and its sister site BlackBoxVoting.org.

Also, two excellent Cringley columns on the subject:
A Year Into the E-voting Crisis, Shouldn't We Have Noticed the Printer That's Already Built into Each Diebold Voting Machine?,
and Why the Best Voting Technology May Be No Technology at All

But Kerry and the DNC aren't fighting against any of that. They signed on until November 2, and now that's past and they can go back to having garden parties or whatever they do for three and a half years between conceding elections.

Tags: politics, election04, elections, voting
[ 11:00 Nov 03, 2004    More politics/election04 | permalink to this entry | ]

Blocked voter registrations

As long as I'm collecting links to news stories, here are some about attempts to block voter registration or otherwise intimidate or discourage voters. States involved: Nevada, Florida, Oregon, Michigan, Ohio, and Iowa.

• Las Vegas Review Journal
• Ann Arbor News
• The Columbus Dispatch
• Des Moines Register
• Tallahassee Democrat
• Century Foundation on Ohio challenges
• Century Foundation on Missouri
• Century Foundation on Michigan
• Brennan Center on Michigan

Tags: politics, election04, elections, voting
[ 23:07 Nov 02, 2004    More politics/election04 | permalink to this entry | ]

Touchscreen machines changing votes

I mentioned to someone the problems that have been showing up for a week where voters think they've voted for one candidate, then realize upon getting to the final review that the machine has recorded votes for a different candidate, and discovered that I didn't have handy links to any of those stories. So here's a collection of stories from Texas and New Mexico:

• Austin Chronicle
• Albuquerque Journal
• Electronic Frontier Foundation
• News 8 Austin
• Free New Mexican
• Albuquerque Tribune

Unfortunately the stories seldom say what type of touchscreen voting machine was being used.

And keep in mind that changing only a single vote per voting machine in the 2000 election could have made a difference of 25 electoral votes, according to a recent ACM study (which unfortunately isn't readable online unless you're an ACM member).

Tags: politics, election04, elections, voting
[ 22:43 Nov 02, 2004    More politics/election04 | permalink to this entry | ]

Two good election shenanigans sites

BoingBoing seems to be slashdotted (probably not by slashdot) but two other sites with excellent up-to-date news on election problems are E-Voting Experts, covering reports of problems with touchscreen and optical scan voting machines, and Equal Vote, covering some of the legal challenges against voters, in states such as Ohio and (of course) Florida.

Tags: politics, election04, elections, voting
[ 14:07 Nov 02, 2004    More politics/election04 | permalink to this entry | ]

I voted

I'm happy to report that voting with paper in my neighborhood was surprisingly low hassle.

The registrar did not ask me whether I wanted paper, but when I saw her circle "E" I hastily told her "I want a paper ballot". She looked momentarily surprised, but recovered quickly, scribbled over the "E" and marked "P". They didn't offer a pen, but I had brought one so I didn't ask.

Then came the wait. They had four or five touchscreen machines, but only one booth (made from a cardboard box) for paper voters, already occupied. The ballot is long (in fact, there are two paper ballots, each 2-sided) so it takes quite a while to finish it. That was fine, because it gave me a chance to hear that they began asking the people registering behind me whether they wanted paper or electronic. They often had to explain the difference to voters who had no idea what the options were, which didn't sound easy; they were very patient about helping people understand the options and didn't try to brush anyone off. Roughly half of the people there chose paper.

Voting was straightforward except that the booth's ledge was very low (for wheelchair access; the voter ahead of me was in a wheelchair). I probably should have grabbed a chair.

While I was marking my paper ballot, I heard a woman who was having a lot of trouble getting the touchscreen machine to work. The pollworker worked with her for quite a while. I think they eventually straightened it out; it sounded like maybe she had to press really hard to get it to register her votes.

When I had finished, my ballot went straight into a box, no provisional envelopes or anything like that. Paper voters get a different sticker, not the new "I voted, touchscreen" sticker (so I don't get to draw a circle-slash with a Sharpie like I'd planned).

Reports I hear from other Santa Clara county voters: most have been asked "electronic or paper?" and I haven't heard any reports of provisional envelopes or other weirdness. Many who voted paper report people voting outside booths; in one case no booth was available, and paper voters sat at a folding table. There wasn't much privacy on the machines either, though: they don't have much of a wing to hide the screen from onlookers, so if you wanted to snoop on someone's votes, it's not difficult.

All in all, I was pleased with how easy it was to vote with paper, with the competence of the poll workers, and with how many people chose the paper option.

Tags: politics, election04, elections, voting
[ 12:44 Nov 02, 2004    More politics/election04 | permalink to this entry | ]

Problems voting with paper ballots

BoingBoing (the esteemed Cory Doctorow) already has coverage of some of the problems people are encountering trying to vote here in Santa Clara County (California) this morning.

Like the Vote Save Error #9. Use the Backup Voting Procedure." message one voter got when trying to use the touchscreens.

But about that backup voting procedure: it seems that even if you can persuade them to give you a paper ballot (bring your own pen, even though the Voter Information Guide specifically says on page 164 that after signing in at the polls the voter "receives a paper ballot along with an approved marking device"), the ballots cast on paper are being put in "provisional" envelopes, yet without the identifying information on the envelope which is used to approve provisional ballots. One really wonders if such votes will be counted.

I wonder if it will be possible to get statistics after the fact for the total number of paper ballots counted in each precinct (and how many of them were provisional)? For comparison, I wish someone was doing exit polls to get an idea of what percentage of people are requesting paper ballots.

Meanwhile, Kelly Martin reports that in Cook County, Ill. voting is no longer by secret ballot. Each ballot has a number on it which is correlated with the voter's name.

One of the boingboing comments points out that voting problems should be reported to voteproblem.org. The EFF suggests using the Election Incident Reporting System.

Stay tuned.

Tags: politics, election04, elections, voting
[ 10:58 Nov 02, 2004    More politics/election04 | permalink to this entry | ]

More ballot shenanigans

The Florida post office has somehow lost 58,000 absentee ballots in Broward County, FL.

They say they're mailing out new ballots, but that (not mentioned in the article) blithely assumes that everyone who voted absentee did so frivilously, not because they were, say, going to be out of town during the election.

By a staggering coincidence, Broward was the county which gave Gore the biggest margin in 2000.

Meanwhile, news from the EFF from last week when I was out of town: Santa Clara County poll workers are being trained not to offer voters a chance to use paper ballots instead of electronic voting machines.

I've been rather hoping that the EFF (or someone) would organize protests near polling places, trying to inform voters of their rights. But no such luck. Instead, they've set up a site with a big flash movie with monotonous music and no information that couldn't have been shown better in a simple fast-loading html page. If you want to watch the flash movie, it's at PaperOrPlastic2004.org but there's really nothing else there besides the movie.

Spread the word anyway. Tell everyone you know in the affected counties (Santa Clara, Orange, Alameda, and Riverside Counties. Napa, San Bernardino, Merced, Plumas, Shasta, Tehama, and Riverside counties) that they can request a paper ballot, and that way leave a paper trail that can be verified in case of a recount.

Tags: politics, election04, elections, voting
[ 23:43 Oct 28, 2004    More politics/election04 | permalink to this entry | ]

More on CA paper ballots

Unable to find any law stating the paper ballot requirement, I called the Sec. of State's office back, this time being forwarded to someone named Michael.

He told me that the requirement specified in the decertification action was a "directive by the secretary of state", not a legislative action" and so was not reflected in the election code.

However, the requirement is stated in the Voter Information Guide. I do not seem to have received my VIG, but it's available in PDF form (168 pages) on the Voter Information Guide page off the Sec. of State site. It's on page 167: "Counties using touchscreen/DRE systems are required to have paper ballots available upon request."

So there it finally is, in writing. Whew!

I strongly advise all California voters to ask for this option at their polling place on November 2.

Tags: politics, election04, elections, voting
[ 14:41 Oct 08, 2004    More politics/election04 | permalink to this entry | ]

Verifying CA's supposed paper trail elections law

Our story so far: the nice lady at the Secretary of State's office pointed me to the PDF for Shelley's Diebold decertification as the proof that the upcoming election will allow voters to request a paper ballot. That PDF says that it modifies Division 19, Chapter 1 (commencing with Section 19001) of the Elections Code and Government Code section 12172.5. My goal is to make sure that this hasn't been superceded by subsequent recertification or other lobbying.

First I tried Leginfo, searching the Government and Elections codes for various combinations of the words paper ballot option election machine That gives lots of links (which I need to explore) which don't include 12172.5.

Searches on leginfo, I notice, always return exactly 20 results (two pages of ten), no matter what you search for. Somehow this doesn't give me a feeling of confidence.

To get directly to a numbered law, leave the search field blank to go to the table of contents for Government or Elections. Then wait a while.

It turns out that Government Section 12159-12179.1has nothing to do with voting procedures or technology, and doesn't have a .5. Hmm.

Well, let's try 19001 and see if it's related. Oops, the table of contents skips from 18993 to 19050 (which is something to do with making General Appointments, anyway).

The Election code, on the other hand, skips from 12113 to 12200, missing 12172.

The 19000s of the election code do, finally, seem to relate to the issue of technology used in polling. But nowhere in the 19000s can I find any mention of paper ballots.

A google search of "paper ballot" option on site leginfo-ca-gov returns no hits.

Is leginfo behind? Or was the lady at Shelley's office wrong about that provision still being current?

Tags: politics, election04, elections, voting
[ 13:09 Oct 08, 2004    More politics/election04 | permalink to this entry | ]

CA will (apparently) honor requests for paper ballots

I've been waiting for months for the papers, or Wired, or someone, to give us the definitive word on California's proposed paper ballot option.

Back when Secretary of State Kevin Shelley moved to decertify some of the Diebold voting machines, he included a provision that voters who wish a paper trail may request a paper ballot in counties which use touchscreen voting machines.

But since then, many things have changed, many of the decertified machines have been recertified, and none of the news articles ever mentions the paper ballot option. I've been keeping an eye on the CA Elections and Voter Info site for some time, looking for help or information, but time is getting short to request an absentee ballot, so I mounted a search.

The Elections and Voter Info site has a FAQ -- they only link to the FAQ about voter registration, but that same page has answers about other topics as well, including voting systems. But no mention whatsoever about paper ballots.

The elections page also links to another site run by the Sec. of State, MyVoteCounts.org, which has lots of interesting information on things like Diebold decertification and recertification, but still no info on the paper ballot rule (or lack thereof).

Going back to the elections page, I called toll-free phone number for voter info, and spent a few minutes navigating a phone tree, which didn't include any options which seemed relevant; determinedly pressing the numbers for "other requests" eventually ended up in something that wanted to request info from me (for what? I wasn't clear) rather than let me ask questions of a human.

I hung up, and tried the Sample Ballot I received in the mail a few days ago. It has instructions for voting both on touchscreen and on paper, but no assurance that the paper ballot is actually an option for anyone receiving the sample ballot. The only phone number I could find anywhere in the sample ballot was one for requesting ballots in other languages.

Going back to the Secretary of State's web site, I found the phone number for the Sec. of State's office in Sacramento, and called long-distance. Navigating another phone tree (oddly, "Elections Division" is not in the first list of options; you have to choose "Other" which takes you to a menu which includes elections) and ended up speaking with a friendly and helpful woman there.

She assured me that yes, all voters in California would have the option of requesting a paper ballot at the polling place, and she offered to find it on the web site for me.

Several minutes of searching ensued. She initially thought it would be on the Voter's Bill of Rights linked off MyVoteCounts.org. This turns out to be a PDF of a big-type poster, which, alas, says nothing about paper ballots.

She put me on hold briefly while she went searching, came back, and tried to remember the click-through route she'd taken so I could find it too. We followed several false leads, but finally got there: start at the Elections & Voter Information page, scroll way down to Voting Systems (under "General Information"), then click on Decertification and Conditional Certification for certain DREs to get the 9-page PDF of Shelley's original decertification of the Diebold machines, which, on page 4 item 4.b.1, specifies that every polling place must either (a) have a voting machine offering a "fully tested, federally qualified and state certified accessible, voter verified paper, audit trail" or (b) (1) Permit every voter to have the option at his or her polling place of casting a ballot on a paper ballot which may be satisfied by providing an adequate number of paper ballots to each polling place based on each County's assessment of the number of persons who may request them. The cost of additional paper ballots specified in this paragraph shall be borne by the vendor of the voting sytem that sought its certification or approval for use in California, or the vendor's successor in interest".

(Incidentally, this PDF is simply a scan of the successive pages of the document; there's no searchable text here, so google wouldn't help unless it had OCR capability.)

The woman at the Sec. of State's election division assured me that this was still in effect and had not been outdated by the more recent recertifications, and that it applied to every voting district (presumably there's no currently certified voting machine which meets clause 4.a?)

The status of this document (see page 3) is that it amends Division 19, Chapter 1 (commencing with Section 19001) of the Elections Code and Government Code section 12172.5. So that's the place to go to make sure this is still current. More on that later.

At the end of our conversation, I mentioned that this info was a bit difficult to get to, and maybe a clear FAQ entry, somewhere in the html of the site, might be in order. She agreed. Perhaps someone will update the web site before the election.

Tags: politics, election04, elections, voting
[ 12:43 Oct 08, 2004    More politics/election04 | permalink to this entry | ]

Wrong Time for an E-Vote Glitch

Wired has had great coverage of the e-voting fiasco all along, but the latest story is particularly impressive: Wrong Time for an E-Vote Glitch. Sequoia Systems (suppliers, to our shame, for Santa Clara county, though at least they're not as bad as Diebold) had a demo for the California state senate of their new paper-trail system. Turned out that their demo failed to print paper trails for any of the spanish language ballots in the demo. It wasn't just a random glitch: they tried it several times, and every time, it failed to print the spanish voters' paper trail.

What a classic. I wish advocates for the Spanish-speaking community would seize on this and help to fight e-voting.

Sequoia, of course, is claiming that it wouldn't happen in a real election, that the problem was they didn't proofread the Spanish ballot but they would for a real election. I'm sure that makes everyone feel all better.

Other news mentioned in the article: the California bill to require a paper trail has stalled, and everyone thinks that's mysterious because it supposedly had bipartisan support.

They don't mention whether that's the same bill which would have allowed voters to choose a paper ballot rather than a touchscreen machine. That's important, since those of us who don't trust the touchscreen machines need to know in time to request absentee ballots, if we can't use paper ballots at the polls.

Tags: politics, rights, elections, voting, government
[ 18:35 Aug 19, 2004    More politics/rights | permalink to this entry | ]