VoxPopuLII

The Civic Need

Civic morale in the U.S. is punishingly low and bleeding out. When it comes to recent public approval of the U.S. Congress, we’re talking imminent negative territory, if such were possible. Gallows chuckles were shared over an October 2011 NYT/CBS poll that found approval of the U.S. Congress down to 9% — lower than, yes, communism, the British Petroleum company during the oil spill, and King George III at the time of the American Revolution. The trends are beyond grim: Gallup in November tracked Congress falling to 13% approval, tying an all-time low. For posterity, this is indeed the first branch of the federal government in America’s constitutional republic, the one with “the power of the purse“, our mostly-millionaire law-makers. Also: the branch whose leadership recently attempted to hole up in an anti-democratic, unaccountable “SuperCommittee” to make historic decisions affecting public policy in secret. Members of Congress are the most fallible, despised elected officials in our representative democracy.

OpenCongress: Responding with open technology

Such was the visceral distrust of government (and apathy about the wider political process, in all its messy necessity) that our non-profit organization, the Participatory Politics Foundation (PPF), sought to combat with our flagship Web application, OpenCongress.org. Launched in 2007, its original motto was: “Bringing you the real story about what’s happening in Congress.” Our premise, then as today, is that radical transparency in government will increase public accountability, reduce systemic corruption in government, and result in better legislative outcomes. We believe free and open-source technology can push forward and serve a growing role in a much more deliberative democratic process — with an eye towards comprehensive electoral reform and increased voter participation. The technology buffet includes, in part, the following: software (in the code that powers OpenCongress); Web applications (like the user-friendly OpenCongress pages and engagement tools); mobile (booming, of course, globally); libre data and open standards; copyleft licensing; and more. One articulation of our goal is to encourage government, as the primary source, to comply exhaustively with the community-generated Principles of Open Government Data (which, it should be noted, are continually being revised and amended by #opengov advocates, as one would expect in a healthy, dynamic, and responsive community of watchdogs with itchy social sharing fingers). Another articulation of our goal, put reductively: we’ll know we’re doing better when voter participation rates rise in the U.S. from our current ballpark of 48% to levels comparable to those of other advanced democracies. Indeed, there has been a very strong and positive public demand for user-friendly Web interfaces and open data access to official government information. Since its launch, OpenCongress has grown to become the most-visited not-for-profit government transparency site in the U.S. (and possibly the world), with over one million visits per month, hundreds of thousands of users, and millions of automated data requests filled every week.

OpenGovernment.org: Opening up state legislatures

The U.S. Congress, unfortunately, remains insistently closed-off from the taxpaying public — living, breathing people and interested constituent communities — in its data inputs and outputs, while public approval keeps falling (for a variety of reasons, more than can be gestured towards here). This discouraging sentiment might be familiar to you — even cliché — if you’re an avid consumer of political news media, political blogs, and social media. But what’s happening in your state legislature? What bills in your state House or Senate chambers are affecting issues you care about? What are special interests saying about them, and how are campaign contributions influencing them? Even political junkies might not have conversational knowledge of key votes in state legislatures, which — if I may be reductive — take all the legislative arcane-ness of the federal Congress and boil it down to an even more restrictive group of state capitol “insiders” who really know the landscape. A June 2011 study by the University of Buffalo PoliSci Department found that, as summarized on Ballotpedia :

First, the American mass public seems to know little about their state governments. In a survey of Ohio, Patterson, Ripley, and Quinlan (1992) found that 72 percent of respondents could not name their state legislator. More recently, an NCSL-sponsored survey found that only 33 percent of respondents over 26 years old could correctly identify even the party that controlled their state legislature.

Further, state legislative elections are rarely competitive, and frequently feature only one major party candidate on the ballot. In the 2010 elections, 32.7 percent of districts had only one major party candidate running. (Ballotpedia 2010) In 18 of the 46 states holding legislative elections in 2010, over 40 percent of seats faced no major-party challenge, and in only ten states was the proportion of uncontested seats lower than 20 percent. In such an environment, the ability to shirk with limited consequences seems clear.”^[1]

To open up state government, PPF created OpenGovernment.org as a joint project with the non-profit Sunlight Foundation and the community-driven Open States Project (of Sunlight Labs). Based on the proven OpenCongress model of transparency, OpenGovernment combines official government information with news and blog coverage, social media mentions, campaign contribution data, public discussion forums, and a suite of free engagement tools. The result, in short, is the most user-friendly page anywhere on the Web for accessing bill information at the state level. The site, launched in a public beta on January 18th, 2011, currently contains information for six U.S. state legislatures: California, Louisiana, Maryland, Minnesota, Texas, and Wisconsin. In March 2011, OpenGovernment was named a semi-finalist in the Accelerator Contest at South by Southwest Interactive conference.

Skimming a state homepage — for example, California — gives a good overview of the site’s offerings: every bill, legislator, vote, and committee, with as much full bill text as is technically available; plus issues, campaign contributions, key vote analysis, special interest group positions, and a raft of social wisdom. A bill page — for example, Wisconsin’s major freedom of association bill, SB 11 — shows how it all comes together in a highly user-friendly interface and, we hope, the best-online user experience. Users can track, share, and comment on legislation, and then contact their elected officials over email directly from OpenGovernment pages. OpenGovernment remains in active open-source development. Our developer hub has more information. See also our wish-list and how anyone can help us grow, as we seek to roll out to all 50 U.S. state legislatures before the November 2012 elections.

Opening up state legislative data: The benefits

To make the value proposition for researchers explicit, I believe fundamentally there is clear benefit in having a go-to Web resource to access official, cited information about any and all legislative objects in a given state legislature (as there is with OpenCongress and the U.S. Congress). It’s desirable for researchers to know they have a permalink of easy-to-skim info for bills, votes, and more on OpenGovernment — as opposed to clunky, outmoded official state legislative websites (screenshots of which can be found in our launch blog post, if you’re brave enough to bear them). Full bill text is, of course vital for citing, as is someday having fully-transparent version-control by legislative assistants and lobbyists and members themselves. For now, the site’s simple abilities to search legislation, sort by “most-viewed,” sort by date, sort by “most-in-the-news,” etc., all offer a highly contemporary user-experience, like those found by citizens elsewhere on the Web (e.g., as online consumers or on social media services). Our open API and code and data repositories ensure that researchers and outside developers (e.g., data specialists) have bulk access to the data we aggregate, in order to remix and sift through for discoveries and insights. Bloggers and journalists can use OpenGovernment (OG) in their political coverage, just as OpenCongress (OC) continues to be frequently cited by major media sites and blog communities. Issue advocates and citizen watchdogs can use OG to find, track, and contact their state legislators, soon with free online organizing features like Contact-Congress on OC. OpenGovernment‘s launch was covered by Alex Howard of O’Reilly Radar, the National Council of State Legislatures (The Thicket blog), and Governing, with notes as well from many of PPF and Sunlight’s #opengov #nonprofit allies, and later on by Knight Foundation, Unmatched Style, and dozens of smaller state-based political blogs.

The technology that powers OpenGovernment.org

The technology behind OpenGovernment was assembled by PPF’s former Director of Technology (and still good friend-of-PPF, following his amicable transition to personal projects) Carl Tashian. In designing it, Carl and I were driven first by a desire to ensure the code was not only relatively-remixable but also as modular as possible. Remixable, because we hoped and expect that other open-source versions of OpenGovernment will spring up, creating (apologies for the cliché, but it’s one I am loathe to relinquish, as it’s really the richest, most apt description of a desirable state of affairs) a diverse ecosystem of government watchdog sites for state legislatures. Open data and user-focused Web design can bring meaningful public accountability not only to state legislatures, but also to the executive and judicial branches of state government as well. PPF seeks non-profit funding support to bring OpenGovernment down to the municipal level — county, city, and local town councils, as hyper-local and close to the neighborhood block as possible — and up to foreign countries and international institutions like the United Nations. In theory, any government entity with official documents and elected official roles is a candidate for a custom version of OpenGovernment facing the public on the open Web — even those without fully-open data sets, which of course, most countries don’t have. But by making OpenGovernment as modular as possible, we aimed to ensure that the site could work with a variety of data inputs and formats. The software is designed to handle a best-case data stream — an API of legislative info — or less-than-best, such as XML feeds, HTML scraping, or even a static set of uploaded documents and spreadsheets.

Speaking of software, OpenGovernment is powered by GovKit, an open-source Ruby gem for aggregating and displaying open government APIs from around the Web. Diagrammed here, they are summarized here with a few notes:

• Open States – a RESTful API of official government data, e.g. bills, votes, legislators, committees, and more. This data stream forms the backbone of OpenGovernment. A significantly volunteer effort coordinated by the talented and dedicated team at Sunlight Labs, Open States fulfills a gigantic public need for standardized data about state legislation — largely by the time-intensive process of scraping HTML from unstandardized existing official government websites. Really remarkable and precedent-setting public-interest work, the updates are by James Turk on the Labs Blog. Data received daily in .json format, and wherever possible, bill text is displayed in the smooth open-source DocumentCloud text viewer (e.g., WI SB11).
• OpenCongress – API for federal bills, votes, people, and news and blog coverage. OpenGovernment is primarily focused on finding and tracking state bills and legislators, but one of our premises in designing the public resource was that the vast majority of users would first need to look up their elected officials by street address. (Can you name your state legislators with confidence offhand? I couldn’t before developing OpenCongress in 2007.) So since users were likely to take that action, we used our sibling site OpenCongress to find and display federal legislators underneath state ones (e.g., CA zip 94110).
• Google News, Google Blog Search, Bing API – we use these methods to aggregate news and blog coverage of bills and members, as on OpenCongress: searching for specific search terms and thereby assembling pages that allow a user to skim down recent mentions of a bill (along with headlines and sources) without straying far from OpenGovernment. One key insight of OpenCongress was that lists of bills “most in the news” and “most-on-blogs” can point users towards what’s likely most-pressing or most-discussed or most-interesting to them, as search engine or even intra-site keyword searches on, say, “climate change bill” don’t always return most-relevant results, even when lightly editorially curated for SEO. On pages of news results for individual bills (e.g., CA SB 9) or members (e.g., WI Sen. Tim Carpenter), it’s certainly useful to get a sense of the latest news by skimming down aggregated headlines, even given known issues with bringing in similarly titled bills (e.g., SB 9 in Texas, not California) or sports statistics or spam. Future enhancements to OpenGovernment will do more to highlight trusted news sources from open data standards — a variety of services like NewsTrust exist on this front, and there’s no shortage of commercial partnerships possible (or via Facebook Connect and other closed social media), but PPF’s focus is on mitigating the “filter bubble” and staying in play on the open Web.
• Transparency Data API (by Sunlight Labs) to bring in campaign contribution data from FollowTheMoney. If Open States data is the backbone of OpenGovernment, this money-in-politics data is its heart. PPF’s work is first and foremost motivated by a desire to work in the public interest to mitigate the harmful effects of systemic corruption at every level of government, from the U.S. Congress on down. (See, e.g., Lessig, Rootstrikers, innumerable academic studies and news investigations into the biased outcomes of a system where, for example, federal members of Congress spend between 30 and 70 percent of their time fundraising instead of connecting with constituents.) Part of this is vocally endorsing comprehensive electoral reforms such as non-partisan redistricting, right-to-vote legal frameworks, score voting, parliamentary representation, and the Fair Elections Now Act for full public financing of elections. But the necessary first step is radical transparency of campaign contributions by special interests to elected officials, accompanied by real-time financial disclosure, stronger ethics laws, aggressive oversight, and regulation to stop the revolving door with lobbyists and corporations that results in oligarchical elites and a captured government. Hence “The Money Trail” on OpenGovernment, e.g., for Texas, is a vital resource for connecting bills, votes, and donations. The primary source for money figures is our much-appreciated and detail-oriented non-profit partners at the National Institute on Money in State Politics, who receive data in either electronic or paper files from the state disclosure agencies with which candidates must file their campaign finance reports. Future enhancements to OG will integrate with MAPLight‘s unique analysis of industries supporting and opposing individual bills with their donations. MAPLight has data for CA and WI we’re looking to bring in, with more to come.
• Project VoteSmart’s API brings in special-interest group ratings for state government and allows OpenGovernment to highlight the most-impactful legislation in each state, marking their non-partisan “key vote” bills (e.g., for TX). VoteSmart does remarkable legislative analysis that neatly ties in bills to issue areas, but VoteSmart doesn’t have a built-in money-in-politics tie-in on their pages, or tools to track and share legislation. (This is just another way in which OpenGovernment, by aggregating the best available data in a more user-focused design, adds value, we hope, in an open-source Web app, about which more below.) Project VoteSmart’s work is hugely valuable, but the data is again ornery — special interest group ratings are frequently sparse and vary in scale, and are therefore difficult to accurately summarize or average — so for members, where applicable, we show a total of the number of ratings in each category (e.g., for TX Sen. Dan Patrick) and link to a fuller description.
• Wikipedia – OG first attempts to match on a legislator’s full name to a bio page on Wikipedia, with largely good but occasionally false-positive results. Of course many politicians go by nicknames, so this is a straightforward enhancement we’ll make once we can prioritize it with our available resources. See, e.g., TX Sen. Joan Huffman on OG, and her bio on Wikipedia.
• Twitter – OG has first-pass implementation of bringing in mentions of a state hashtag and bill number, e.g., #txbill #sb7, and for members, state name and legislator name, e.g., Texas Joan Huffman. This is another relatively straightforward engineering enhancement that we can make more responsive and more accurate with additional resources — for example, bringing in more accurate mentions and highlighting ones made by influential publishers on social media. Spending our time working within walled gardens to capture mentions of key votes isn’t inherently pleasant, but bringing out vital chatter onto the open Web and making it available via our open API will be worth the time and investment.
• Miro Community, free and open-source software from PPF’s sibling non-profit the Participatory Culture Foundation (PCF), makes it possible to crowdsource streaming online video about state legislatures (e.g., CA).

The OpenGovernment.org Web app is free, libre, and written in open-source Ruby on Rails code (developer hub). Like OpenCongress, the site is not-for-profit, non-commercial, promotes #opengovdata, open standards, and offers an open API, with volunteer contributions and remixes welcome and encouraged. Two features: most pages on the site are available for query via JSON and JSONP; and we offer free lookup of federal and state elected officials by latitude / longitude by URL. PostgreSQL and PostGIS power the back-end — we’ve seen with OpenCongress that the database of aggregated info can become huge, so laying a solid foundation for this was relevant in our early steps. The app uses the terrific open-source GeoServer to display vote maps — many enhancements possible there — and Jammit for asset packaging. For more technical details, see this enjoyable Changelog podcast w/ Carl from February 2011.

Web design on this beta OG Web app is by PPF and PCF’s former designer (and still good friend after an amicable parting) Morgan Knutson, now a designer with Google. As product manager, my goal was creating a user interface that — like the code base — would be as modular as possible. Lots of placeholder notes remain throughout the beta version pointing to areas of future enhancement that we can pursue with more resources and open-source volunteer help. Many of the engagement features of the site — from tracking to commenting to social sharing — were summarized brilliantly by Rob Richards in this Slaw.ca interview with me from July 29th, 2011 — viz., walking users up the “chain of engagement.” It’s a terrific, much-appreciated introduction to the civic-engagement goals of our organization and our beliefs regarding how well-designed web pages can do more than one might think to improve a real-life community in the near-term.

More on open government data and online civic engagement

To briefly run through more academic or data-driven research on the public benefits of #opengovdata and open-source Web tools for civic engagement (not intended to be comprehensive, of course, and with more caveats than I could fit here) :

• Tiago Peixoto of the World Bank — @participatory on micropublishing — lots of academic studies
• David Eaves, open data activist based in Canada — see also Open Gov Data camp (summary, official)
• “Open Data is Civic Capital: Best Practices for ‘Open Government Data'”, essay (rev. Jan. 2011) by Joshua Tauberer, our longtime data partner behind GovTrack and part of the new POPVOX team in the #gov20 startup landscape
• Wiki Government — book by Prof. Beth Noveck — and Open Government — book by O’Reilly Media, edited by Daniel Lathrop & Laurel Ruma.
• My #opengov curated list on Twitter (@ppolitics).
• Civic Software Index, crowdsourced spreadsheet coordinated by our terrific allies abroad, OKFN and MySociety (primarily UK — and PPF’s primary inspiration to create OpenCongress in the U.S.)
• Various allied non-profit organizations and data partners listed in the footer of PPF’s homepage, especially Sunlight in D.C. and CivicCommons (also based in NYC — with a deeply rich wiki of #opengov resources)

OpenGovernment.org: Some metrics

To wrap up this summary of OpenGovernment in 2011, then, I’ll summarize some of the metrics we’ve seen on Google Analytics — with limited outreach and no paid advertising or commercial partnerships, OpenGovernment beta with its six states will have received over half a million pageviews in its first year of existence. As with OpenCongress, by far the most-viewed content as of now is bills, found via search engines by their official number, which send approximately two-thirds of all traffic (and of that, Google alone sends over half). Hot bills in Texas and the WI organizing bill constitute three of OG’s top ten most-viewed pages sitewide. After hearing about a firearms bill in the news or from a neighbor, for example, users type “texas bill 321” or “sb 321” into Google and end up on OG, where they’re able to skim the bill’s news coverage, view the campaign contributions (for example) and interest group ratings (for example) of its authors and sponsors, and notify their legislators of their opinions by finding and writing their elected officials.

OpenGovernment.org: Next steps, and How you can help

In addition to rolling out to all 50 U.S. states and launching pilot projects in municipal areas, one of our main goals for OpenGovernment is integration with the free organizing features we launched this past summer on OpenCongress version 3. Enabling OG users to email-their-state-reps directly from bill pages will significantly increase the amount of publicly transparent, linkable, query-able constituent communication on the open Web. Allowing issue-advocacy organizations and political blog communities to create campaigns as part of future MyOG Groups will coordinate whipping of state legislators for a more continually-connected civic experience. And as always, tweaks to the beta site’s user interface will allow us to highlight the best-available information about how money affects politics and votes in state legislatures, to fight systemic corruption, and to bring about a cleaner and more trustworthy democratic process. Help us grow and contact us anytime with questions or feedback. As a public charity, PPF aspires to be grow to become more akin to the Wikimedia Foundation (behind Wikipedia), Mozilla (behind Firefox), and MySociety (behind TheyWorkForYou, for the UK Parliament, and other projects). We’re working towards a future where staying in touch with what’s happening in state capitols is just as easy and as immediately rewarding as, for example, seeing photos from friends on Facebook, sharing a joke on Twitter, or loading a movie on Netflix.com.

David Moore is the Executive Director of the Participatory Politics Foundation, a non-profit organization using technology for civic engagement. He lives in Brooklyn, NY.