skip navigation
search

There have been a series of efforts to create a national legislative data standard - one master XML format to which all states will adhere for bills, laws, and regulations.Those efforts have gone poorly.

Few states provide bulk downloads of their laws. None provide APIs. Although nearly all states provide websites for people to read state laws, they are all objectively terrible, in ways that demonstrate that they were probably pretty impressive in 1995. Despite the clear need for improved online display of laws, the lack of a standard data format and the general lack of bulk data has enabled precious few efforts in the private sector. (Notably, there is Robb Schecter's WebLaws.org, which provides vastly improved experiences for the laws of California, Oregon, and New York. There was also a site built experimentally by Ari Hershowitz that was used as a platform for last year's California Laws Hackathon.)

A significant obstacle to prior efforts has been the perceived need to create a single standard, one that will accommodate the various textual legal structures that are employed throughout government. This is a significant practical hurdle on its own, but failure is all but guaranteed by also engaging major stakeholders and governments to establish a standard that will enjoy wide support and adoption.

What if we could stop letting the perfect be the enemy of the good? What if we ignore the needs of the outliers, and establish a "good enough" system, one that will at first simply work for most governments? And what if we completely skip the step of establishing a standard XML format? Wouldn't that get us something, a thing superior to the nothing that we currently have?

The State Decoded
This is the philosophy behind The State Decoded. Funded by the John S. and James L. Knight Foundation, The State Decoded is a free, open source program to put legal codes online, and it does so by simply skipping over the problems that have hampered prior efforts. The project does not aspire to create any state law websites on its own but, instead, to provide the software to enable others to do so.

Still in its development (it's at version 0.4), The State Decoded leaves it to each implementer to gather up the contents of the legal code in question and interface it with the program's internal API. This could be done via screen-scraping off of an existing state code website, modifying the parser to deal with a bulk XML file, converting input data into the program's simple XML import format, or by a few other methods. While a non-trivial task, it's something that can be knocked out in an afternoon, thus avoiding the need to create a universal data format and to persuade Wexis to provide their data in that format.

The magic happens after the initial data import. The State Decoded takes that raw legal text and uses it to populate a complete, fully functional website for end-users to search and browse those laws. By packaging the Solr search engine and employing some basic textual analysis, every law is cross-referenced with other laws that cite it and laws that are textually similar. If there exists a repository of legal decisions for the jurisdiction in question, that can be incorporated, too, displaying a list of the court cases that cite each section. Definitions are detected, loaded into a dictionary, and make the laws self-documenting. End users can post comments to each law. Bulk downloads are created, letting people get a copy of the entire legal code, its structural elements, or the automatically assembled dictionary. And there's a REST-ful, JSON-based API, ready to be used by third parties. All of this is done automatically, quickly, and seamlessly. The time elapsed varies, depending on server power and the length of the legal code, but it generally takes about twenty minutes from start to finish.

The State Decoded is a free program, released under the GNU Public License. Anybody can use it to make legal codes more accessible online. There are no strings attached.

It has already been deployed in two states, Virginia and Florida, despite not actually being a finished project yet.

State Variations
The striking variations in the structures of legal codes within the U.S. required the establishment of an appropriately flexible system to store and render those codes. Some legal codes are broad and shallow (e.g., Louisiana, Oklahoma), while others are narrow and deep (e.g., Connecticut, Delaware). Some list their sections by natural sort order, some in decimal, a few arbitrarily switch between the two. Many have quirks that will require further work to accommodate.

For example, California does not provide a catch line for their laws, but just a section number. One must read through a law to know what it actually does, rather than being able to glance at the title and get the general idea. Because this is a wildly impractical approach for a state code, the private sector has picked up the slack - Westlaw and LexisNexis each write their own titles for those laws, neatly solving the problem for those with the financial resources to pay for those companies' offerings. To handle a problem like this, The State Decoded either needs to be able to display legal codes that lack section titles, or pointedly not support this inferior approach, and instead support the incorporation of third-party sources of title. In California, this might mean mining the section titles used internally by the California Law Revision Commission, and populating the section titles with those. (And then providing a bulk download of that data, allowing it to become a common standard for California's section titles.)

Many state codes have oddities like this. The State Decoded combines flexibility with open source code to make it possible to deal with these quirks on a case-by-case basis. The alternative approach is too convoluted and quixotic to consider.

Regulations
There is strong interest in seeing this software adapted to handle regulations, especially from cash-strapped state governments looking to modernize their regulatory delivery process. Although this process is still in an early stage, it looks like rather few modifications will be required to support the storage and display of regulations within The State Decoded.

More significant modifications would be needed to integrate registers of regulations, but the substantial public benefits that would provide make it an obvious and necessary enhancement. The present process required to identify the latest version of a regulation is the stuff of parody. To select a state at random, here are the instructions provided on Kansas's website:

To find the latest version of a regulation online, a person should first check the table of contents in the most current Kansas Register, then the Index to Regulations in the most current Kansas Register, then the current K.A.R. Supplement, then the Kansas Administrative Regulations. If the regulation is found at any of these sequential steps, stop and consider that version the most recent.

If Kansas has electronic versions of all this data, it seems almost punitive not to put it all in one place, rather than forcing people to look in four places. It seems self-evident that the current Kansas Register, the Index to Regulations, the K.A.R. Supplement, and the Kansas Administrative Regulations should have APIs, with a common API atop all four, which would make it trivial to present somebody with the current version of a regulation with a single request. By indexing registers of regulations in the manner that The State Decoded indexes court opinions, it would at least be possible to show people all activity around a given regulation, if not simply show them the present version of it, since surely that is all that most people want.

A Tapestry of Data
In a way, what makes The State Decoded interesting is not anything that it actually does, but instead what others might do with the data that it emits. By capitalizing on the program's API and healthy collection of bulk downloads, clever individuals will surely devise uses for state legal data that cannot presently be envisioned.

The structural value of state laws is evident when considered within the context of other open government data.

Major open government efforts are confined largely to the upper-right quadrant of this diagram - those matters concerned with elections and legislation. There is also some excellent work being done in opening up access to court rulings, indexing scholarly publications, and nascent work in indexing the official opinions of attorneys general. But the latter group cannot be connected to the former group without opening up access to state laws. Courts do not make rulings about bills, of course - it is laws with which they concern themselves. Law journals cite far more laws than they do bills. To weave a seamless tapestry of data that connects court decisions to state laws to legislation to election results to campaign contributions, it is necessary to have a source of rich data about state laws. The State Decoded aims to provide that data.

Next Steps
The most important next step for The State Decoded is to complete it, releasing a version 1.0 of the software. It has dozens of outstanding issues - both bug fixes and new features - so this process will require some months. In that period, the project will continue to work with individuals and organizations in states throughout the nation who are interested in deploying The State Decoded to help them get started.

Ideally, The State Decoded will be obviated by states providing both bulk data and better websites for their codes and regulations. But in the current economic climate, neither are likely to be prioritized within state budgets, so unfortunately there's liable to remain a need for the data provided by The State Decoded for some years to come. The day when it is rendered useless will be a good day.

Waldo Jaquith is a website developer with the Miller Center at the University of Virginia in Charlottesville, Virginia. He is a News Challenge Fellow with the John S. and James L. Knight Foundation and runs Richmond Sunlight, an open legislative service for Virginia. Jaquith previously worked for the White House Office of Science and Technology Policy, for which he developed Ethics.gov, and is now a member of the White House Open Data Working Group.
[Editor's Note: For topic-related VoxPopuLII posts please see: Ari Hershowitz & Grant Vergottini, Standardizing the World's Legal Information - One Hackathon At a Time; Courtney Minick, Universal Citation for State Codes; John Sheridan, Legislation.gov.uk; and Robb Schecter, The Recipe for Better Legal Information Services. ]

VoxPopuLII is edited by Judith Pratt. Editors-in-Chief are Stephanie Davidson and Christine Kirchberger, to whom queries should be directed.

Raise your hand if you’ve heard (or said) a variation of one of these tired truisms: "Politics is dominated by lobbyists and spending." "Policy making has degenerated into a glorified yelling match." "Our country has never been more polarized." "Today’s online communities foster echo chambers of the like-minded rather than fora for discussion."

Is your hand raised? Because ours certainly are.

The only thing anyone can seem to agree on today is that the current U.S. political system is broken. We’re mired in a confluence of corporate spending, ugly discourse, and voter voicelessness.

LexPop provides an open public platform for tackling these problems.

Meet LexPop

LexPop allows participants to collaborate in the creation of legislative bills -- bills that are later introduced by actual legislators. At its most basic, LexPop is a Wikipedia for creating public policy. (There’s a lot more to it than that, as we’ll explain below.) In our first project, Massachusetts Representative Tom Sannicandro (D-Ashland) -- one of those actual legislators we’re talking about -- has agreed to introduce a net neutrality bill created on LexPop.

LexPop has two primary goals. Our first goal is to give the public a voice. We hope to provide a space for ordinary people (i.e., people who can’t afford to hire lobbyists) to contribute substantively to public policy -- to give their best ideas a fair hearing.

As you know, lobbyists write the bulk of the legislation coming out of our various legislatures. LexPop provides a voxlobbylane.jpgcounterpoint to the current model -- a way for the public to provide legislators with voter-created model legislation. A legitimate, 21st-century democracy will invite the public into meaningful collaboration, and LexPop is part of the march in that direction.

Our second goal is to determine the best way to achieve the first. That is, a compelling movement is attempting to take governance into the 21st century, and organizations like PopVox and OpenCongress are doing great work. Several organizations and initiatives, including a government-sponsored effort in Brazil, are trying to make it possible for citizens to help write legislation. But at this point, nobody knows the best way to make the co-creation of laws a reality. Our work will contribute to figuring out what’s possible, what works, and what doesn’t.

How LexPop works

There are two ways to use LexPop. Our primary focus is on Policy Drives -- where legislators pledge to introduce bills written on the site. Policy Drives are somewhat analogous to what goes on at Wikipedia, but LexPop provides more structure through the use of three specific phases:

  • Phase 1: Initial discussion, debate, argument, and research;
  • Phase 2: Outlining the bill in plain English (for those who aren’t regular readers of Vox PopuLII); and
  • Phase 3: Transforming the ‘plain English’ outline into legislative text.

voxnet-neutrality.jpgWe’re currently in the discussion phase of our first Policy Drive, devoted to the net neutrality bill Rep. Sannicandro has agreed to introduce.

A second option on LexPop is working on a “WikiBill.” WikiBills are written via the familiar, wide-open wiki model, and they offer a spot for the public to create model legislation on their own, without the three-phase structure of Policy Drives, and without a legislator-sponsor. WikiBill creators collaborate through a free-for-all process, very similar to Wikipedia -- start from scratch and cobble the bill together. There’s no end to the WikiBill process, so participants can create a bill, submit it to their representatives, modify it, and submit it again.

Yeah, sounds great. But can this really work?

It’s usually at this point in the conversation that questions start coming up. LexPop, and similar projects, are largely operating in uncharted waters, and so there’s good reason to think the project sounds ambitious, perhaps even crazy. Below are a few of the questions we’re asked most often, along with our preliminary answers.

Will anyone contribute to this sort of effort?
We think so. (Obviously.)

Here’s why: Ordinary people collaborate on difficult projects online -- especially online -- often with great success. Take Linux, the open source operating system. The vast majority of people who work on Linux aren’t paid; they’ve incrementally created it in their spare time.

Are you reading this blog on Firefox? Well, guess what? Your browser was built almost entirely by volunteers.

At LexPop, we’re asking people who are passionate about certain issues to give some of their free time to developing better policy, in the same way engineers have asked them to help develop software. Sure, it will be complicated, but people are smart, and given the right opportunity and tools, they’ll be able to (once again) create something extraordinary.

Politics is too controversial -- How can you expect people to come to consensus on one answer?
To answer this question, we like to look to Jesus -- the "Jesus" page on Wikipedia, that is.

There are plenty of controversial topics addressed on Wikipedia, but it’s the pages for these topics that are often the most accurate. Wikipedians who edit the Jesus page know the topic is controversial, so they back up what they say with facts -- otherwise, the crowd of users won’t allow it. Over time, the Jesus page has turned into something that most users are pretty happy about. And this is the similarity between LexPop and Wikipedia: They’re both about collaboratively writing something that isn’t perfect in the eyes of any one participant, but is better than the alternative.

Fine, but isn’t there a better model than a wiki?
This is one of the things we’re trying to figure out, and one of the things with which we need your help. We’re starting with a modified wiki (the three phases), but as we learn, we’ll adapt. A wiki allows a certain type of collaboration (the kind found on Wikipedia), but it may not be the best way to collaborate. Is the three-step process we’re using the right model, or should the phases be combined? With your help, we’ll find out -- and we promise to share our findings.

Will legislation created on LexPop be representative?
We don’t claim that bills made on LexPop will be perfectly representative, and we’re not trying to make representative democracy obsolete. After a bill is written on the site, it will still have to go through the same bill-into-law process as every other piece of legislation.

voxexperts.jpgBut LexPop will certainly be more representative than the system we have now. With LexPop, non-profit organizations with valuable knowledge of an issue, passionate experts well-versed on a topic, and regular voters (Joes the Plumber, if you will) will no longer be shut out of the process. Right now, we live in a world where participation too often means a voter pours out her heart in a letter and receives a form response that the intended recipient didn’t write, read, or even sign. Our system for adding more voices to lawmaking may not be perfect, but it will be less imperfect than the current political system.

LexPop provides a first draft of legislation that’s written by people, not by lobbyists. This is our value-add; we’re opening a new channel for public participation, and taking a step toward a more legitimate and deliberative democracy.

But we need your helpvoxmeeting_brains.jpg

And we need it big time. For a project like this to work, we need participants.

If you’re interested in collaborative democracy, please get involved in the conversation. You’ll be helping even if you post only one comment. Even if you aren’t particularly interested in net neutrality, we encourage you to learn more about it on the site, and then make sure you come back when we have a Policy Drive on your favorite issue.

Also, we’d be grateful if you spread the word about our site. Like us on Facebook, Tweet about LexPop (@LexPopOrg), blog about us, or, even better, let us write a guest blog post on your site (Thanks, VoxPopuLII !).

We’d also love for you to tell us what we’re doing wrong. LexPop is perfect in neither theory nor practice. So please help us make LexPop and, ultimately, deliberative democracy better with your feedback. We have a Google Group for discussion about LexPop, or you can contact us through the website.

Coda

LexPop is a platform for public engagement and empowerment. LexPop provides a space for discussion-driven public policy and a stronger, more agile democracy. LexPop is about more voices. Add yours.

Matt_BacaMatt Baca is a joint J.D./M.P.A. student at New York University School of Law and the Harvard Kennedy School. He's interested law, public policy, government 2.0, and the Rockies (team and mountains).

Olin_Grant_ParkerOlin Parker is a Master's in Public Policy student at the Harvard Kennedy School. His interests include disability policy, education reform, the states of Kansas and Louisiana, and his 17 month-old daughter.

VoxPopuLII is edited by Judith Pratt. Editor in chief is Robert Richards.