This August, Craig Newton ‘07, (re)joined the LII as our new Associate Director for Content Development. While a student at Cornell Law School, Craig was the Editor-in-Chief of the LII Supreme Court Bulletin.

He spent the last six years at the law firm Cooley LLP litigating a broad range of commercial disputes for companies such as Adobe, Facebook, and Qualcomm before returning to Ithaca.  Having left a career as first a naval flight officer and then an intelligence analyst to attend law school, Craig calls upon all facets of his prior experience as he mentors teams of law students, manages our network of volunteers and legal professionals, monitors the website to discern usage trends and patterns, and pursues new partnership initiatives–all with an eye toward improving a range of products, including the Supreme Court Bulletin and case collection, the WEX legal encyclopedia/dictionary, the constellation of materials surrounding the US Constitution, and our ever-more capable editions of the US Code and the Code of Federal Regulations.

He and Associate Director for Technology Sara Frug are working together at the intersection of legal content and engineering to improve search and navigation features, as well as on a number of experiments intended to make our content more discoverable and more usable.

We are enthusiastic about his presence and the capabilities he adds to our team, and we look forward to your continued suggestions and feedback on our editorial content.

With the addition of Craig — and of consulting designer Manolo Bevia — the LII crew now numbers 9 1/4, delivering services to over 24 million unique visitors each year.

The Semantic Technology and Business Conference showcases business applications from the leading semantic web companies and researchers: RPI, Stanford, MIT, and PARC, as well as Hoffmann-La Roche, Walmart, Yahoo, and Google. At the conference, the LII’s semantic web engineer, Mohammad AL-Asswad, Ph.D., presented approaches to improving access to import/export regulations in agriculture. We had observed that off the shelf, a search engine doesn’t know how to establish the context of a section of the CFR. Mohammad showed the results of his work combining a set of natural language processing techniques with a government-supplied subject vocabulary and our own metadata in order to improve search results.

In July, the International Association of Agricultural Information Specialists brought together 125 delegates from 28 countries for its World Congress, hosted this year by Cornell University’s Mann Library. Agriculture librarians presented the results of research on subjects ranging from innovation proliferation among small scale pineapple farmers in Ghana to digital librarianship in Indian agricultural libraries to open access in institutional repositories. Mohammad presented at the poster session, showing the connections between agricultural regulations and scientific literature.

Why? Regulators need to know about the science related to the regulations they are making and enforcing. Scientists need to know about regulations that affect their research; they often also want the opportunity to know about new rulemakings that affect areas in which they have expertise. Using natural language processing techniques, Mohammad was able to find topics in the regulations that map to keywords in the agricultural ontology AGRIS. You can see preliminary results of this feature using the “Find the Science” feature here.

The morning of October 2, the United States Government shut down for the first time in nearly 18 years, as a congressional stalemate froze funding and furloughed some government workers.

Aside from more overt casualties — closed federal offices, and shuttered national monuments and parks — some of the government’s websites were not updated or taken completely offline. Behind the scenes, some large data sets were made unavailable, as in the case of the the Library of Congress whose machine-readable schemas became unavailable to applciations around the world that depend on them. Our engineering team made copies of the MODS, MADS and PREMIS schemas available on LII servers to help other applications survive the shutdown.

On the front end, many affected sites posted notices on their home page or on splash screens that attributed the outages to “a lapse in federal government funding,” and those that remained open, like the National Weather Service, continued to point blame at funding while stating that they were remaining available in the “interest of public safety and security.” But who exactly decides what’s in the public interest? It seems to us that the laws of the United States are pretty important, but if the government were to suddenly take them offline completely (as they did with other public sites like the United States Census and NASA) we would still be expected to know and to follow government laws and regulations, wouldn’t we?

Of course, we see this as yet another reason why LII is so important– making the laws available without cost and protecting our right to know and understand these laws without being subjected to the whims of political in-fighting. But this shutdown also produced out some interesting traffic data, which demonstrates just how vital the LII remains to those working in the government sector.

In the two weeks of the shutdown, the LII saw a modest, overall three percent increase in traffic to our site, but when we looked at the locations people were accessing from, those statistics told a different story. Visits from several large cities were up significantly, 14% in New York, San Francisco and San Diego, for example. But traffic from cities with a high degree of federal offices fell drastically– 22% in Boston and 20% in Washington DC. Conversely, the state of Maryland, home to many Washington DC workers, demonstrated a 25% increase in traffic during the same time period.

These numbers might suggest that people who needed access to law continued to receive it at the official government websites, or that they came to the LII thinking that the government sites would be shut down. But it also suggests that a significant number of government workers on furlough from their jobs in DC and Boston visit the LII each day. And that says something even more important. It suggests that thousands of federal workers prefer the LII to the official government primary legal collections, which would confirm anecdotal evidence we hear from government employees all the time. So even though the official sites remained open, the LII is the site of choice for many who rely on accurate current legal information to perform the government’s work. And that’s a recommendation we’re proud to have.

LII Associate Director Sara Frug is having a bad day. It’s twelve in the afternoon, East Coast time, and the LII web site has slowed to a crawl. It’s taking 16 seconds to get a page of legal information that you could normally get in 6. The LII’s West Coast audience is just getting to the office. Between now and 5 o’clock both East and West Coast people will make heavy use of the site. Things are going from bad to worse, and they will get even worse than that before they get better.

How bad is bad? The typical load average on the LII’s production server is supposed to hover somewhere around 4; that’s what you see on a fully-utilized machine that isn’t straining. Dan Nagy, the LII’s systems administrator, is seeing 39. Dan is not eager to tell you how he knows, but a bad decision made in the course of a tune-up could take it to 300 in about 45 seconds. If that happens, the LII is about as useful as a wet brick. And the LII crew knows: the slow get slower. Anybody who has ever driven on the Boston freeways during the morning rush can see how that works. I-93’s running at capacity, some bozo drops a soda can out the window, and hey-presto, traffic jams solid all the way south to New Bedford.

Sara runs the LII’s crew of engineers. Right now, she is talking to Dan and to Wayne Weibel, the LII’s Drupal developer. They’re looking at the output from WebPageTest.org, a free service that gives a lot of web-performance diagnostics (see below). A couple of things are obvious. The server isn’t clearing connections fast enough. They try a couple of quick alterations to the server configuration. The first shoots the load numbers through the roof, and locks up the server. Oops. Definitely not an improvement. A second is successful. Now the server’s clearing connections fast enough to not bog down completely, so the site’s visitors are getting what they want. But it’s still slow. Time to go under the hood.

A single web page at the LII is made up of a lot of pieces. The home page has 65 separate components. Around 20 of them come from the LII, and the remainder are things like that little Facebook logo with the number of “likes” on the page, which has to be fetched from Facebook every time a visitor shows up. Each LII component can involve multiple database queries that look up text, provide additional links to related information, and so on. Every component comes with a “set-up charge” — the cost in time of setting up a connection from the user’s browser so that the component can be retrieved. There are a lot of moving parts. The LII uses a variety of strategies that reduce the need to regenerate pages, either holding preassembled pages in a cache to be served up directly, or telling the user’s browser that it’s OK to hang on to pieces that get used repetitively and not reload them. Either approach cuts down on the work the server has to do. Right now, it’s not enough.

Over the next three days, Sara, Wayne, and Dan will spend a lot of time staring at complex “waterfall” diagrams that give a timeline for the loading of each component that goes into the page. The waterfall also shows them what’s waiting for what; some components need others before they can load. They’ll identify bottlenecks and either eliminate them or find a way to make the component smaller so it takes less time to load. The server will be tweaked so that it doesn’t clog up as quickly. Components will be combined so that they can be loaded from a single file, reducing the number of data connections needed and the setup time involved. They’ll get the average page loading time from 7 seconds down to under 4, even during the afternoon rush. Three seconds doesn’t sound like much, but for users, it’s an eternity. During the rush, it’s critical. An average load time of 7 seconds quickly ramps up to 16 seconds or more, because, well, the slow get slower. An average load time of 4 seconds is stable at 4 seconds. The crew think they can get it down to 3. The Director wants 2. The LII audience wants something instantaneous, or even better, telepathy. Sara and the crew will turn this crisis into an opportunity. Suddenly, everybody’s got a lot of good ideas about how to make things even faster and they’re starting to compete with the clock. The team is drag-racing now.

There’s more to all this than speed. Because the server is now on the right side of the loading curve, it can serve around 150 more simultaneous users during rush hour, when there are typically around 1350 simultaneous visitors on the site. That’s an 11% increase. There are also implications for our donors. Throwing hardware at a problem like this can be expensive. None of the changes they’ve made involved upgrading hardware or buying more computing cycles. That’s a good thing; the next biggest server ( a “double extra large”, in cloud-computing-speak) would cost about twice what we’re paying now.

So, more people in the LII audience getting more legal information twice as fast at no increase in cost. What’s not to like about that?

Our friends at CALI / eLangdell have just released “Introduction to Basic Legal Citation,” a free eBook version of Peter W. Martin’s popular online resource.

The new version was recently revised in the fall of 2012 to take account of changes in the citation rules of a small number of U.S. jurisdictions and the format of currency information furnished for statutes by LexisNexis and Westlaw. It is indexed to the fourth edition of the ALWD Citation Manual and the nineteenth edition of The Bluebook, both published in 2010.

Professor Martin requests that you donate to the Legal Information Institute if you would like to show your support for this free work. We happen to think that’s a fine idea.