Running with the Big Docs

Along with LII Editorial Boss Sara Frug, I spent yesterday morning with the folks at the Sunlight Foundation — an organization with a compelling mission and a growing set of activities that reflect it. Founded with the idea of using Web 2.0 techniques to bring transparency to Congress, Sunlight is now becoming a rallying point for a diverse community of folks who share the idea of making government better by making the information it generates and consumes more accessible.

That’s an idea we find really attractive. We’ve been amazed — shocked, really — at how little access government has to its own work product (never mind the public). We recently learned that some branches of the Federal courts limit access to the commercial legal information services based on seniority; we understand that the same is true of Federal agencies, where junior people don’t have access to Lexis and Westlaw [note to legal research teachers: “junior” would pretty much describe our recent graduates, wouldn’t it? Think we should be teaching them more about free online sources? ] Our e-mail is chock-full of questions and testimonials from government attorneys who rely on our edition of the US Code and our Federal rules collections. Our most successful projects over the last fifteen years have involved improving or re-mixing the presentation of Federal data to make it more easily used and understood by a broad audience.

So our question for John Wonderlich at Sunlight was “how can we help?”.

Turns out there are a number of ways. We have a lot of expertise in the arcana of Federal data online, experience with data standards, software tools that have remained in-house because we didn’t think anyone else had any use for them, and so on. There are a lot of ways that the LII can and will participate in the growing community of technologists who want to “hack government”. One of the best ways we thought of involves some help from you… particularly if you are a law librarian, legal scholar, or anyone else with experience working with government documents.

We know from experience that some online documents are especially useful to people building new services on top of government information. Here are some examples of these “linchpin” documents:

• The US Code classification tables, which describe pending codification of public laws
• The Table of Popular Names, which ties the names used by journalists, the public, and politicians to refer to legislation to the sections of the US Code where it ends up
• The Parallel Table of Authorities, which relates sections of the Code of Federal Regulations to the statutes that enable them
  
• The Unified Regulatory Agenda, which tells the public what agencies will be regulating in the near future
• The US Government Manual, which describes the organization of the government.

These are documents that provide important information about the context and structure of government, or that link isolated pools of legal information together. For example, the classification tables form the basis of our US Code updating features; we parse them into a database that is then used to power both clickable update links and RSS feeds. As published by the government, they are difficult to use; they would be near the top of our list of documents the government should be publishing in easily processed XML form. Put another way, they are the documents that are most useful in building online services that make legal information more transparent. They would be a good focus for our efforts here at the LII, for the growing community of government-transparency hackers, and for lobbying of (and cooperation with) the GPO’s FDSys effort.

We think that it should be possible to build a list of (let’s say) 100 such documents — the Big Docs that those who develop these kinds of services would most like to see placed online in a form that is both easily processed by machines (ie. XML) and continuously updated by the official body that creates them. What would your suggestions be? Put ’em in the comments, please.

[editorial note: this week’s post is a bit later than usual because of my stay at the 2008 CALI conference, about which more next time. The posting schedule will no doubt continue to be spotty throughout the summer — I’m travelling and talking more than usual — Tb.]