An ambitious website that will open up government data to the public will launch in beta, or pilot, form in December.

Reams of anonymous data about schools, crime and health could all be included.

Data.gov.uk has been developed by Sir Tim Berners-Lee, founder of the web, and Professor Nigel Shadbolt at the University of Southampton.

It is designed to be similar to the Obama administration’s data.gov project, run by Vivek Kundra.

Today the Electronic Frontier Foundation (EFF) posted thousands of pages of records detailing behind-the-scenes negotiations between government agencies and Congress about providing immunity for telecoms involved in illegal government surveillance.

The documents include drafts of legislation and communications between Congress and the Department of Justice (DOJ) and the Office of the Director of National Intelligence (ODNI) about amendments to the Foreign Intelligence Surveillance Act (FISA). They were released as a result of litigation that started back in 2007, when Congress first debated granting immunity to the telecommunications companies for taking part in massive, unchecked surveillance of Americans’ telephone and Internet communications. EFF used the Freedom of Information Act (FOIA) to request information about communications between the DOJ, ODNI, Congress, and telecom lobbyists.

FOIA Release: Draft legislation to amend FISA

FOIA Release: Correspondence about amending FISA, bill summaries, and other documents

Here at the White House, those of us on team Recovery Act put a very high premium on transparency and accountability.  Like they say, “sunlight’s the best disinfectant,” and when it comes to tracking the $787 billion economic stimulus plan, our view is that we should let the sunshine in. 

Well, today, we’re happy to report that the independent Recovery Accountability and Transparency Board—they’re the folks who bring you www.Recovery.gov—has taken yet another step toward showing the American people their money at work.  The board released its very first report on the small portion of Recovery Act spending that recipients have reported on so far, and these preliminary data show that the Act is doing just what it’s supposed to do: helping to put Americans back to work, while partially offsetting the ongoing job market impact of the worst recession in decades.

A new House bill would open Congressional Research Service (CRS) reports to the public. The measure, H.R. 3762, closely mirrors a Senate bill introduced by Sen. Joseph Lieberman (I-CT) in the spring.

Federal agencies shouldn’t be the only ones to open their data for the public — states and local governments should also be ramping up efforts to become more transparent, the National Association of State Chief Information Officers (NASCIO) contends in a newly published report.

The first thing a state should do is create a one-stop portal, or data catalog, for all its publicly-accessible data, along the lines of the White House’s Data.Gov , the report states.

A Call to Action for State Government: Guidance for Opening the Doors to State Data (18 pages; PDF)

On Monday, the White House announced the official launch of the XML version of the Federal Register (FR), now available from 2000 to the present through GPO’s Federal Digital System (FDsys), the Federal Register Web site and Data.gov. . .

. . . There are important distinctions between the XML versions available on FDsys and Data.gov. Data.gov is limited to full-year XML downloads, while FDsys offers the ability to quickly and easily download the Federal Register by day, month or year. By providing these options, FDsys may better fit user needs.

In addition, FDsys also provides access to the official, digitally signed PDF version of the Federal Register, in addition to the XML.

For more on this, read Carl Malamud’s interview with Raymond Mosley, Director of the Office of the Federal Register and Michael L. Wash, the Chief Information Officer of the Government Printing Office (O’Reilly Radar, Oct 6, 2009).

Lawyers, lobbyists, librarians and concerned citizens, rejoice: As of Monday, it is much easier to access the Federal Register. The de facto daily newspaper of the executive branch publishes approximately 80,000 pages of documents each year, including presidential disaster declarations, Medicare reimbursement rates, and thousands of agency rulings on policies ranging from banking to fishing to food. It’s a must-read for anyone with business before the federal government or concerned about inside-the-Beltway decisions, including academics, good-government advocates and Register junkies (yes, they do exist).

Starting Monday, issues dating back to 2000 will be available at Data.gov [also GPO's Federal Digital System and The Federal Register Web site] in a form known in the Web world as XML, which allows users to transport data from a Web site and store it, reorganize it or customize it elsewhere. Officials suggested that the move puts readers, rather than the government, in charge of deciding how to access the Register’s reams of information. 

Read the White House announcement.

Post from the Sunlight Foundation on why this is so important.

Today, U.S. District Court Judge Emmet G. Sullivan ordered the release of records of former Vice President Cheney’s interview with the FBI in the Valerie Plame Wilson leak investigation. Under the Freedom of Information Act (FOIA), CREW had sued the Department of Justice (DOJ) seeking release of the records. Judge Sullivan agreed with CREW that because the investigation is now over the Department of Justice (DOJ) cannot withhold all documents based on an exemption that protects law enforcement records from disclosure. Judge Sullivan rejected DOJ’s argument that future White House officials would be unwilling to cooperate with law enforcement inquiries if these records were released.

The Court agreed, however, with the DOJ’s view that portions of the records are privileged, meaning they do not have to be released, because some discuss internal agency deliberations, while others reflect communications between the president and vice-president. Paragraphs 6 and 8 of the Declaration of David J. Barron detail the information DOJ can withhold.

Read Judge Sullivan’s memorandum opinion here and order here.

Read David J. Barron’s declaration here.

Two back-to-back announcements by the Environmental Protection Agency (EPA) this week extend the agency’s admirable record on transparency since the beginning of the Obama administration. EPA announced two policy changes that increase the transparency of the agency’s pesticides programs: opening up the registration process for pesticides to public scrutiny and moving to require all pesticide ingredients be listed on product labels.

“No information may remain classified indefinitely,” according to a draft of an Obama Administration executive order on national security classification policy.

As a statement of principle, this may seem tame and self-evident.  But until now, no Administration has been willing to make such a categorical statement about the temporal limits of national security secrecy, and it may have significant policy consequences.

See this side-by-side comparison between the August 2009 draft and the current executive order.