Information Commissioner: PM Must Take a Stand on Open Data

Suzanne Legault, Canada's Interim Information Commissioner, gave a blockbuster speech yesterday to the standing Parliamentary Committee on Access to Information, Privacy, and Ethics. In it, she lays out 5 principles for open government - the first of which calls for leadership from the Prime Minister.

The five principles are:

  • There must be commitment at the top to lead a cultural change conducive to open government. At a minimum, this involves issuing a declaration on open government with clear objectives. The commitment also entails assigning responsibility and accountability for coordination, guidance and deliverables. It requires prescribing specific timeframes.
  • There should be ongoing and broad public consultations. Citizens should be encouraged to participate using electronic means. It is critical to determine what government information the public wants and how they want to receive it.
  • Information should be made accessible in open standard formats and rendered re-usable. Information should be derived from various sources and integrated to reduce the silos inherent in bureaucratic structures.
  • Privacy, confidentiality, security, Crown copyright and official languages issues need to be addressed and resolved.
  • Open government principles should be anchored in statutory and policy instruments.

The speech ends with a quote from 'Unlocking Government: How Data Transform Democracy', a recent Deloitte Canada report:

Government leaders have before them an opportunity to combine the resourcefulness of online citizens and entrepreneurs with the power of factual data to more effectively achieve their mission. In an information-driven age, the ability of governments to seize this opportunity may ultimately determine whether a government fails or succeeds.

It's wonderful to see a Canadian official at this level make a strong public statement in support of open government and open data. This speech may mark a turning point in the reported culture of strict information control at the federal level.

Read the Ottawa Citizen's coverage of the speech here.

Canadian Open Data Action

There's been some inspiring action on open data lately in Canada, at both the federal and municipal levels.  I've been under the weather lately, so haven't been able to cover these events to the extent they deserve, but here's a quick update:
  • The OpenDataOttawa hackfest last Saturday looks like it was a great success.  The event attracted 120 people for a full day of learning and coding.  16 apps and prototypes were demonstrated, some of which are listed on this Ottawa open-data app directory.  Once again, the organizers ran a superior media outreach campaign, getting stories into the Citizen, the Sun, and CBC Radio Ottawa.
  • As part of the hackfest, City of Ottawa officials publicized upcoming plans for a $50K open data application contest as part of their data catalog launch (pending a council vote).
  • openparliament.ca, one of the most lovely and functional Canadian federal open government sites yet, launched two weeks ago.  openparliament.ca combines Parliamentary hansard and House voting records, news stories, and twitter feeds to provide an easily browse-able window into what our federal representatives are up to.  Some sample views:

    It's worth noting that openparliament.ca is partly enabled by the work of HowdTheyVote.ca founder, Cory Horner, who created a clever API for looking up MPs and ridings by postal code.  Horner's API, which is a geographic lookup based on a combination of 1) electronics maps purchased cheaply from Canada post, and 2) freely available maps from Elections Canada, allows sites like openparliament.ca to bypass the $2500 fee usually charged by StatsCan for this information.

  • datadotgov.ca, a citizen-led federal open data directory, also launched two weeks ago.  Led by Canadian open government advocate and guru David Eaves, the site aims to highlight which federal departments are and are not sharing their information.  Currently, Natural Resources is in the lead.
  • David Eaves also shared on his blog that parliamentary IT will start sharing both parliamentary bios and the Hansard in XML, making it easier for sites like openparliament.ca to get started.

Here are some links to other open data apps, news, and resources, collected via our site OpenDataLinks.ca:

Recently Added Apps:

  • EatSure.ca

    London, Ontario restaurant inspection scores, mapped.

  • OttawaTrash

    Allows Ottawa residents to subscribe to their garbage shedule via ICal notification.

  • IBM ManyBills: A Visual Bill Explorer

    A web based visualization of 2009 U.S. congressional legislation.

  • RepresentMe.ca

    Want to know how your elected representatives are acting on your behalf? RepresentMe helps you find out who's representing you and what they've done lately.

  • Canadian Government Expenses

    Since 2003, this project has assessed a total of 67821 Canadian Travel and Hospitality expenses.

  • UK MP Expenses

    An application visualizing UK MP expenses, sourced from the Guardian's MP expense data set.

  • Technology for Transparency Network

    Mapping and evaluating technology projects that promote transparency, accountability, & civic engagement around the world.

Recently Added News:

Recently Added References:

Over $90K of Prizes in Canadian OpenData Contests

Contests:
  • Apps4ClimateAction: The Government of BC has announced Apps4ClimateAction, a contest for applications that raise awareness about climate change, with a prize pool of $40,000. Apps vying for prizes can tap into a surprising number of contributed data sets from provincial, municipal, and federal governments – as well as data sets contributed by non-profit partners. Negotiating the release of this flood of information from so many diverse sources is a tremendous achievement by contest organizers.
  • City of Edmonton: The City of Edmonton released a timeline for their Apps4Edmonton contest, which will offer up to $50,000 of prizes. Apps4Edmonton follows an innovative contest structure: In the first phase of the contest, citizens – even non-techies – can submit and vote on ideas. The ranked idea list is then published, and coders can start submitting their apps. Winners will be showcased by the city at GTEC, Canada's conference on Technology for Government.
  • ForTheWeb: MakeWebNotWar, a series of Canadian web design conferences and coding competitions, has included an open data prize category at their ForTheWeb event happening this May in Montreal. Apps built on a number of listed Canadian municipal data catalogues can be entered in pursuit of fame and a Dell Compact Desktop.
Canadian Open Data Catalogues & Announcements
  • Missasauga quietly launched a data catalogue on March 8th.
  • Calgary passed a motion to launch an open data catalogue in 2nd quarter 2010. You can read full text of the March 22 motion on p26 here (pdf), as well as a first-hand account of the citizen action behind it.

Is your city thinking of an open data catalogue? Need some inspiration for getting started? OpenMuniWiki, by the Open Planning Project, compares and contrasts a number of open data policy models from cities around the world.

Informal Government of Canada Open Data Strategy Jam

An update to yesterday's post on Canadian Open Data events: An informal meet-up of Government of Canada employees for drafting Canada's Open Data Strategy was announced yesterday on twitter. The group will be meeting Friday at Darcy McGee's, not far from Parliament Hill. The announcement, which warms my heart, reads:

Informal group getting together to build a draft strategy in an insanely short period of time, off the corner of our desks because we believe in it.

The meet-up is a first response to Chuck Shawcross, CIO of Environment Canada, who challenged GoC employees at a recent conference to write an open data strategy on GCPEDIA, the federal government's internal wiki. I see that a number of heroes within GoC on this topic (as well as senior staff) have already signed on for the meet-up.

Not that this group needs recommendations, but some good starting points would be:

Good luck, you guys.

Canadian Open Data Events -- Including a $50K Contest

  • Edmonton: The city of Edmonton hosted an Open Data Workshop on March 6th, where Edmonton CIO Chris Moore announced they will be hosting an upcoming contest called Apps4Edmonton, the first ever Canadian open data apps contest. The contest will include up to $50,000 in prize money. More links from this event as they arrive.
  • Ottawa: Ottawa activists are organizing an Open Data Hackfest for April 24th. The Ottawa group (which includes members of CivicAccess) has been running an impressive media outreach campaign, landing two interviews in the Citizen, one in the Sun, a spot on CBC Radio Ottawa, and – in what’s likely a Canadian first – an open data story en français in le Droit.
  • Vancouver: Nik Garkusha, Microsoft’s open source strategy lead in Vancouver, and local company Nitobi, will be hosting an “Open Data Drinks and Apps” night on March 18th where they’ll be talking about building applications using open data catalogues, and showing some application demos. More details here.
  • Quebec City: On March 18th there will be a Web Education Day for government employees (also open to the public) which includes a panel session on Canadian open data initiatives.

News From Elsewhere:
  • As part of president Obama’s Open Government Directive, each US federal department is collecting feedback from citizens on top ideas for improving government services. The list of US federal department feedback sites is here. Government 2.0 commentators on GovLoop are tracking which departments are seeing the most action.

Open Data Feels Good

Last week I spoke with City of Edmonton CIO Chris Moore about the history of the city’s open data initiative. The roots of Edmonton’s open data catalogue, launched last month, stretch back to February 2009, when the city’s IT group began to re-assess the way they deliver services, their internal culture, and their role in the city. It turns out that the catalogue, and city’s open data strategy, are the publicly visible face of a full revitalization campaign for the city’s IT group.

When Chris joined the city’s IT staff in October 2008, he found himself at the head of an unhappy group of people. Much of IT was dependant on external consultants. Morale was low, people felt locked into existing patterns, and staff were reluctant to speak their minds. As Chris started discussions to get at the heart of the issues in the group, he received an anonymous email saying that what he was trying to do was nice, but “I really don’t think any of this is going to change.”

Chris led a series of 22 town halls among the 300 city IT staff, which started with open talks about ‘the way things are’. Here’s a sketch drawn during one of those discussions. I think it speaks to anyone who’s ever worked in a demoralized environment:

edmonton_it_before_doodle

Starting from those conversations, the IT group formulated a series of ten core values, which include collaboration, innovation, and open communication. They also formulated a group of desired outcomes for improved service delivery, and for city staff as creative IT professionals. The challenge was then to turn the values into concrete actions that could deliver these outcomes for IT staff and the city. The ideas raised in the discussions seemed in lock-step with the example set by Washington’s Digital Public Square, North America’s first open data initiative, and reports on its potential to spur that region’s social and economic development. Chris says:

“I realized we had all the right elements for implementing open data /open government here.”

Following initial discussions with citizens at Changecamp Edmonton, the city hosted a dedicated open data workshop that brought together software developers and open data activists to talk about what an open data initiative should look like. City staff then solicited the help of 39 people in collaborating on a report on open data, which culminated in the launch of the catalogue.

The next phase of the strategy, which will be announced at an Open City Workshop on March 6th, involves reaching out to a broader community of citizens, non-profits, grass-roots activists, and high school and university students, to get them motivated to work with the city to build a better community. According to Chris:

“It’s up to us, at the city, to be the social consciousness of the city, encouraging people to get involved.”

While the overall IT transformation campaign is still a work in progress, Chris sees a new energy in city IT staff. Internal communication has skyrocketed, as new ideas and solutions are proposed and debated on an internal bulletin board which sees approximately 200 hits a day. On a recent anonymous employee survey, the majority of IT managers strongly agreed (7-9 on a 9 point scale) that the transformation plan met their needs for personal fulfillment.

As to the benefits that Edmonton citizens will receive from the open data portion of the initiative, via applications that will be able to tap into the newly-available information, Councillor Don Iveson was quoted last month as saying:

“There’s a prospect for innovation that we haven’t even imagined yet.”

Canadian Streets Get Fixed

We launched fixmystreet.ca 10 months ago to provide citizens with an easy way to publicly track municipal 311 requests.   In that time, 220 requests have been filed in 6 different cities.  We passed a new milestone last month, when city councilors in Charlottetown PEI began recommending that citizens report problems through fixmystreet.ca.   Councillor Rob Lantz wrote recently about how the site is helping to unify the city’s 311 tracking, saying:

“It’s simple, but provides a level of transparency and accountability that people appreciate.”

Joseph Biggley, a Charlottetown resident*, posted one of the most creative entries we’ve received yet.  His report is a fun example of how the site can work.  Frustrated with the timing of how a walk-light operated, Joseph made video and posted it on YouTube.  He then reported the problem to fixmystreet.ca with a link to the video, and shared it with friends on twitter.  7 people who watched the video subscribed to receive updates on the issue, making it the top-rated issue on the site for Charlottetown.   Joseph received a response from the city in two days.

Some of my other favorite reports from the last year:

FixMyStreet.ca is free to use for both cities and citizens, and is an open source project.  We’re looking for municipal co-ordinators to help us bring fixmystreet.ca to other cities in Canada.  If you’re interested, see here for more information, or contact us directly.

Also, like any crowd-sourced website, fixmystreet.ca is only as useful as the number of people who know about.  If you live in a city listed on fixmystreet.ca, we’d be delighted if you’d help us out by hanging this poster in an appropriate place, such as a university, coffee shop, or grocery store bulletin board.

More statistics on issues reported on FixMyStreet.ca to be posted next week.

* Joseph also gave us a kind mention in a recent article on Open Government in Spacing Atlantic

Open Government Data Roundup

In Canada
  • Edmonton launched its open data catalogue, becoming the fourth city in Canada to do so. The catalogue is the first one by a North American city to use a new Microsoft product, the Open Government Data Initiative platform. Read news coverage by the Edmonton Journal, and/or an account from the point of view of the volunteers and city workers who put it together. A report on open data prepared by the city, which includes a timeline for the site, is here (pdf).
  • Vancouver released an upgraded version of its catalogue with more data sets, and a particular emphasis on geospatial information. Kevin Bowers, Manager of IT Technology Planning at the City of Vancouver posted here on how updating information on the website is now part of the city’s routine procedures.
  • K. Bower’s note reminded me of a comment at WiredCamp from Phillip Scott, an IT manager involved with the City of Toronto's cataloguethat “the city looks at open data as part of routine disclosure”. If only all Canadian governments saw things that way.
  • Note:  There are rumours of an open data initiative underway at the city of Ottawa, though no formal announcement as of yet.    See comment #2 here. Good job in advance, you folks.

Finding the Light Switch in a Dark Country

In ‘The Dark Country’, an article in the January/February Walrus, Gil Sochat describes the frustration of journalists whose work is blocked by systemic delays in Canada’s Access to Information process. In the article, Mary Agnes Welch, president of the Canadian Journalist’s Association -- a group that recently co-authored a study ranking this country’s Access to Information laws behind that of India, Mexico, and Pakistan -- blasts our current government:

“[Prime Minister Stephen Harper has] gone beyond merely gagging cabinet ministers and professional civil servants, stalling access to information requests and blackballing reporters who ask tough questions. He has built a pervasive government apparatus whose sole purpose is to strangle the flow of public information.” 

The article details the current problems with the Access to Information process, such as an ‘Amber Light’ flag, where half of incoming Access to Information requests are labeled as 'sensitive', requiring delays of four to nine months. Further, if the government refuses to release a document, the arbitration process for appealing the decision can take up to two years. This systemic reluctance to release information has a very real impact on our democracy.

These days, journalists rarely have the luxury of chasing a story that takes a year or more to write. Increasingly, journalists are giving up on filing requests; stories that break in other countries via Access to Information requests, such as the UK’s recent parliamentarian expense scandal**, now must be discovered in Canada by other means, if they are to reach the light of day at all.

Sochat asks: “Is it possible for our political culture to shift away from its traditional emphasis on discretion and toward transparency?” But, after posing the question, Sochat stops short of suggesting any solutions. A look at how other countries are handling the problem may provide some answers.

In the UK, the non-profit MySociety has built a website, WhatDoTheyKnow.com as a citizen gateway for filing Freedom of Information requests (that country’s equivalent to Access to Information requests). The site helps citizens target what department to send their request to, and provides an online form for filing the request. The site then sends the request on their behalf. Departmental answers are sent back to WhatDoTheyKnow.com, which posts them for all to see. The site also tracks how long requests have been outstanding, highlighting those that have remained unanswered. In October, MySociety estimated that over 10% percent of all UK FOI requests in the 2nd quarter of 2009 were made through the site.

Does a website making it easier to file FOI requests mean that more requests get filed? It stands to reason that it would. But according to BBC Journalist Martin Rosenbaum, while the number of UK requests has increased since WhatDoTheyKnow began operating, the increase has been primarily in requests to departments that are not particularly popular on WhatDoTheyKnow, so the site is not directly to blame. In fact, by providing searchable results of past inquiries, WhatDoTheyKnow may well decrease the total number of FOI requests, as citizens don’t have to ask the same question twice.

FOI administrators in the city of Dallas, Texas recently discovered a similar principle: that publishing more information can lead to less requests. They found that once they implemented a virtual reading room of their city archives, the number of FOI requests dropped by 83%, as ‘problem’ citizens – those that had filed the most requests -- were reassured their government had nothing to hide.

Interestingly, Canada’s Access to Information law may have already paved the way for providing this type of virtual reading room. A little-known stipulation of the Access to Information Act compels federal departments to maintain offline reading rooms – such as the one described here (search on Reading Room) – which include archives of the answers to filled ATIP requests, going back two to three years. These dusty, neglected archives are maintained by the Information Commissioners for each department, and any citizen or journalist can make an appointment to browse their contents in person. It takes only a baby-step of imagination to go from an offline reading room to an online one, with all the benefits that would bring. The amount this would save in freeing up the office space used for maintaining the archives alone seems like it should be enough to justify the cost of the online archiving system.

I've heard rumors that explorations regarding virtual reading rooms may be underway at the Office of the Information Commissioner as we speak. In the meantime, journalists can do more to not just publicize the issue and its impact on our democracy in articles like ‘The Dark Country’, but to also characterize the problem using concrete statistics. Tellingly, none of the journalists I’ve interviewed for VisibleGovernment’s stab last year at cloning WhatDoTheyKnow for Canada -- which, as discussed here, is made more complicated by our government’s insistence on communicating via paper -- were able to tell me the average delay for an Access to Information request in Canada, or provide general statistics on the process. They just knew that their own requests were taking a long time. The journalists I spoke to, with only one exception, were using a paper-based process to file and track requests, meaning old, unanswered requests went in a drawer, and were usually forgotten.

It's an old management standard that what’s not measured doesn’t get fixed. The Information Commissioner, whose job it is to address systemic issues with the Access to Information process, could help matters by starting to publish the Access to Information response time statistics collected under section 70 c-1 of the act in a central location where they can be easily browsed and compared. (While the office currently publishes compliance report cards, these reports seem to be only done for selected departments, and are not a comprehensive review.) Statistics published for each department would highlight where the issues are, and provide a baseline for improvement.

The answers are out there, we need only the political will to implement them.

** For an entertaining first-hand account from Ben Leapmen, the UK journalist who filed the FOI requests for parliamentarian expenses that ended several MP’s political careers, including the speaker of the house, see this video, starting at about minute 21. My favourite quote, from an MP who was criticised for using public funds to build a house for his pet ducks: “The ducks never liked it anyway.”

US, UK, and AUS Make Open Data Announcements. Where's Canada?

Last week saw three big open data announcements, from governments on three continents:

In the US, the Obama administration released an 11-page Open Government Directive, the culmination of 2009's Open Government Initiative public consultation process. The directive's language is concrete and actionable, even including specific dates for implementation milestones. Here's a sample:

  • To the extent practicable and subject to valid restrictions, agencies should publish information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications. An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.
  • To the extent practical and subject to valid restrictions, agencies should proactively use modern technology to disseminate useful information, rather than waiting for specific requests under FOIA.
  • Within 45 days, each agency shall identify and publish online in an open format at least three high-value data sets (see attachment section 3.a.i) and register those data sets via Data.gov. These must be data sets not previously available online or in a downloadable format.
  • Within 60 days, each agency shall create an Open Government Webpage located at http://www.[agency].gov/open to serve as the gateway for agency activities related to the Open Government Directive and shall maintain and update that webpage in a timely fashion.

In the UK, Her Majesty's Government released its Smarter Government Action Plan, which sets out "how Government will improve public service outcomes while achieving the fiscal consolidation that is vital to helping the economy grow". The first part of the plan, "Strengthen the role of citizens and civic society", includes a set of actions on "Radically opening up data and promoting transparency". Items here include:

  • Releasing public datasets and making them free for reuse, with several data sets listed as a first priority for the next year.
  • Making government data accessible through a single access point at http://www.data.gov.uk/ by January 2010
  • Creating new ways for citizens to interact with public services and public policy -- specifically mentioning a program for public comment on all aspects of the UK's National Heath Service by December 2010.

Australia's Government 2.0 Task Force published a set of draft recommendations for engaging citizens in the business of government.

Recommendation #6 is "Make Public Sector Information open, accessible and reusable". Here's a sample of what they're saying down under:
By default, Public Sector Information (PSI) (..) should be:
  • free (provided at no cost in the absence of substantial marginal costs);
  • based on open standards;
  • easily discoverable;
  • understandable (supported by metadata that will aid in the understanding the quality and interpretability of the information);
  • machine-readable; and
  • freely reusable (not having limitation on derivative uses).

In a move relevant to Canada, and other former colonies, the Australia task force has chosen to follow New Zealand’s lead in recommending that Crown Copyright be abandoned, and replaced with CC-BY as the default licence for all government data.

Note: It's terrific to see the thinking of Canadian advocate David Eaves included in the Australian Task Force recommendations.

Implications for Canada:

With co-ordinated releases from three major western governments, it’s safe to say that the demand for accessible, searchable, and useable government information is no longer a fringe issue. Yet the Canadian federal government stays silent. As Michael Geist said this morning in the Toronto Star:

“These new initiatives herald a dramatic shift in the way governments use the Internet to make themselves more transparent and useful to their citizens. They may also leave Canadians asking if their government is not prepared to lead, then why not at least follow?”

Lack of awareness is not the issue: representatives from Canada’s Treasury Board meet regularly with CIOs from the US, UK, Australia, and New Zealand in what’s called the 'Five Nations CIO Council’ to exchange plans and information. What’s missing is political support from either those who set national policy in Canada or their critics in parliament.

With the government in the middle of a stimulus program which is at present mostly electronically un-traceable, and whose results will be largely un-knowable, it’s not surprising that the powers that be would shy away from adopting policies promoting greater transparency. What’s puzzling is that the opposition parties have not yet found a way to leverage this international movement to their benefit.

Canadian Cities, Citizens Build Community with Open Data

TORONTO: Earlier this month, Toronto city officials met with local web designers, developers, and community activists at the city’s first ever “Open Data Lab” at Toronto Innovation Showcase.  The event followed the launch of TO’s open data site, toronto.ca/open, that morning.  Here’s a picture of the action, taken from Mark Kuznicki’s summary of the event:

Toronto's Open Data Lab


The event had a particularly novel structure, featuring 10 minute speed-dating sessions between city IT staff and local developers, followed by parallel half-hour brainstorming sessions.  The Mayor of Toronto, David Miller, later met with open data volunteers for beer.

 

ITWorld quotes Mayor Miller describing the city’s open data initiative:

“Anyone can download, analyze and mash up our data or write applications to make it more accessible and useful. It is an invitation to Torontonians to do what they do best: create, innovate and build a better city,”

The Open Data Lab spawned a vigorous discussion group, here, where ideas are debated and formats are suggested to the city.  City officials have been doing an admirable job of keeping on top of the discussion.  To help with the task, a Toronto citizen created a web app, datato.org to organize and prioritize requests for the city.

Note: A panel discussion on Open Government from earlier in the day at Toronto Innovation Showcase is available as a webcast.

EDMONTON: City of Edmonton IT staff hosted an Open Data workshop at City Hall this weekend.  The focus of the event, attended by 45 local developers, was brainstorming around the city’s upcoming open data catalogue.  Read coverage of the event by the Edmonton Journal.

VANCOUVER: The City of Vancouver will be hosting another Open Data Hackathon event at City Archives December 9th.

As a side note, there are indications that Vancouver will start publishing road repair advisories next month in GeoRSS, a format which includes encoded geographic co-ordinates.  The change in format will enable applications that, for instance, allow users to subscribe to upcoming repairs to roads on their commute, or in their neighbourhood.

Open Government Data Roundup

In Canada:
  • Vancover’s Open Data site launched last month to wide acclaim. The city recently completed a survey to help prioritize which data sets to open up next.
  • The City of Toronto will be hosting an Open Data Lab to engage the developer community around the city's soon-to-be-launched data portal at http://toronto.ca/open. The event, featuring a talk by Peter Corbett of Apps for Democracy, will be part of ‘Toronto Innovation Showcase’ on November 2. See details here.
  • I was happy to participate as a panelist at the National Town Hall on 'Citizens' Engagement and State Accountability’ hosted by the Office of the Information Commissioner in Ottawa on Sept. 28th. A video webcast of the panel is here.
Elsewhere:
  • The National Association of State Chief Information Officers has released “A Call to Action for State Government – Guidance for Opening the Doors to State Data”. The report gives recommendations to help state governments get started with data transparency portals.
  • The cities of Portland and Seattle have announced open data initiatives.
  • The w3C has released a draft guide to "Publishing Open Government Data", which features a list of "Staightforward Steps to Publish Government Data".
  • The UK’s Cabinet Office has asked for help from citizens in designing its open data portal. From the Cabinet’s Digital Engagement Blog: “With over 1000 existing data sets, from 7 departments (brought together in re-useable form for the first time) and community resources, we want developers to work with us to use the data to create great applications; give us feedback on the early operational community; and tell us how to develop what we have into a single point of access for government-held public data”. Read the full announcement here.
  • In a move diametrically opposed to open government initiatives elsewhere in the UK government, the UK’s national postal service has threatened legal action against a citizen website providing free postal code lookups. The website, called Ernestmarples.com after the British postmaster general who introduced the postal code system, allowed other websites to circumvent the 4000 pound fee the postal service charges for this information. In the short time Ernestmarples.com was available an ecosystem of non-profit and other websites providing location-based lookups flourished around it – all of whom are scrambling to find other options. As many of the websites are volunteer-based public services, the 4000 pound fee is not affordable. The BBC quotes Jim Killock, a digital rights activist: "It is easy to see that large numbers of small business ideas and not for profit services are being blocked by these licence fees," he said. "It is in effect a tax on innovation." Canada’s postal code boundary definitions are locked down by similarly prohibitive access fees.

How Crown Copyright Hurts Canadians

This week, VisibleGovernment.ca launched a last-minute campaign to solicit submissions to Canada’s copyright consultation process asking for Crown Copyright reform.

This may seem like an esoteric issue: but crown copyright is a constant legal threat to citizen projects that use and share government information. It’s an issue that we run up against regularly here at VisibleGovernment.ca, and one that, ultimately, hurts all Canadians. Here are some examples of how:

1. First: Send the Lawyers.

Crown Copyright includes ‘Fair Use’ provisions that are meant to protect Canadian’s ability use information in a way that is consistent with public good. However, these provisions are routinely interpreted by government lawyers in only the most narrow sense.

As described by Michael Geist, in 2007, the group Friends of Canadian Broadcasters were sent a legal notice to cease and desist publishing videos of parliamentary committee proceedings on their website. According to government lawyers, re-publishing unaltered videos put the group in contempt of Parliament.

FoCB challenged the finding, and managed to extract a special, ground-breaking exemption from a Parliamentary committee which allows all forms of recordings of parliamentary proceedings to be re-used and re-distributed – as long as they are unaltered, and not used for commercial purposes.

FoCB is to be commended. Many groups, when faced with a potential lawsuit, would have backed down. Instead, it stood up for what should have been obvious to even government lawyers – that this information belongs to the public. Unfortunately, the new exemption allowing re-distribution of Crown Copyright material only applies for this one type of government information.

2. Crown Copyright Promotes Information Monopolies.

In the US, the watchdog website fedspending.org (whose influence on the US government is described in more detail here) posts statistics on the government contract tendering process. For instance, users can look at a particular company, such as Boeing, to see how many of its contracts were awarded without a bid process. The website helps citizens, and other companies, spot unfair practices in contract tendering.

In Canada, this same information is distributed via Merx, who holds exclusive agreements with many departments and agencies. The information is only viewable before a contract is awarded – after which it disappears. After a delay of several months, the awarded contract may show up on a federal disclosure sites, but not in a way where it can be easily traced to its tendering process.

As described by a contributor to our discussion group, the company behind Merx is prepared to vigorously defend its contract agreements as the exclusive provider of this information. Any group that uses Merx’s information – information which originates in our public institutions -- is opening the door to an expensive lawsuit. This application of Crown Copyright, which provides Merx with an apparent information monopoly, is a barrier to replicating fedspending.org in Canada.

3. Open Source? Not if our Government Touches It.

This spring, at an Ottawa conference where governments share their web 2.0 innovations, I watched a demo of custom software for sharing internal information that a department of the Government of Ontario had cobbled together from a combination of open source components.

The conference was attended by representatives from different municipal, provincial, and federal governments, and it seemed likely that some of them could also make use of the software improvements done by this team. Indeed, the gist of the presentation was ‘We did this – you can do it too.’ I asked: since the software was constructed on an open source base, would they be releasing their modifications back to the community so that they could be adopted by other governments directly? The answer was that they wanted to, but it was ‘with the lawyers’. Because the work was produced by provincial employees, it was protected by Ontario Crown Copyright and could not be released.

This was actually the second project I heard of at the Government of Ontario that wanted to be open source, but was held up at the legal stage. That project had been ‘with the lawyers’ for over a year. Meanwhile, the software lingers, unshared, and unused by other governments. The lawyers apparently have other priorities.

Earlier that day, a visitor from a US government department had shown some custom software his agency had built to visualize the dynamics and movement of fires. Their software was open source, and hosted on Google Code. He estimated that about 40% of their contributions came from members of the public. When asked what the software license was, he said ‘Public Domain’ – as all works by US government employees are not copyrightable. He seemed surprised by the question, as if it were inconceivable that things could be any other way.

4. Models for Change

Other countries are reforming Crown Copyright to promote access to and sharing of government information. In February, 2009, the UK government’s Power of Information Task Force final report found that Crown Copyright was a major barrier to the re-use of Public Sector Information, and recommended that Crown Copyright be changed to a ‘Crown Commons’ license to encourage re-use.

The UK government is following up on this recommendation: Tim Berners-Lee, the inventor of the word-wide web, is one of the people tasked with creating the new Crown Commons license.

New Zealand has gone one step further. In a recent published draft, the state has proposed getting rid of Crown Copyright entirely. Wherever Crown Copyright would be used, Creative Commons Attribution (CC-BY) would be used instead. The proposal argues:

“Now more than ever is there a very present need to bring information the Government holds on behalf of its people into the public domain so that it may be used in ways that stimulate innovation, generate cultural creativity, social interaction and dialogue, while also kick starting economic growth.”

Reading this recently released New Zealand proposal has inspired us to action. Already, some Canadian departments are bravely inventing their own licenses for sharing information. In other situations, like parliamentary proceedings described above, exceptions are being made on an expensive, case-by-case basis. The system should be changed to favour allowing the re-use of government information by default, rather than by exception.

It so happens there is a copyright consultation going on right now in Canada. While the public focus so far has been almost entirely on digital rights and its impact on the music industry, this is a unique opportunity to submit your views on Crown Copyright.

We’ve prepared a draft submission, modeled directly on Vancouver’s recent Open Data, Open Standards, Open Source motion (Open3), which you can use to either send as is, or modify to express your own views. The important thing is that you speak up -- we may not get another chance for some time.

Open Government Data Roundup

News and ideas on Open Government Data from around the web.
  • Nanaimo, BC and San Francisco join the list of cities with Open Data initiatives.  Nanaimo beat out Toronto and Vancouver to launch the first municipal open data site in Canada.   David Eaves traces the timeline of open municipal data initiatives in his blog post 'Rise of the Open City'.  Also of note from Nanaimo is an innovative council minutes site, created on the cheap by city employees.
  • With the excitement around the 'open city' movement, stories on open government have been breaking into Canadian mainstream media -- see recent stories in the Toronto Star and Edmonton Journal.
  • Elsewhere, the Sunlight Foundation announced the finalists in its 'Apps for America 2' contest.   Finalists include an app described as an Everyblock for state data.
  • The government of New Zealand has released a draft Open Access and Liscensing Framework to promote changes to state copyright licensing that allow the freedom to re-mix and re-use.
    From the paper:
    “Now more than ever is there a very present need to bring information the Government holds on behalf of its people into the public domain so that it may be used in ways that stimulate innovation, generate cultural creativity, social interaction and dialogue, while also kick starting economic growth.”
    (hat tip to Tracey at CivicAccess)
  • Australia's Government 2.0 Task Force has been pushing out some valuable content in the last month, see especially their of Government 2.0 Issues Paper.   The government task force has allocated $2.45 Million (AU) for web projects, creating a public consultation site to:
    1. approve or criticise the projects we’ve set out
    2. propose improvements to those ideas
    3. propose alternative ideas
    4. suggest people or firms/agencies that might do a good job of these projects
    Will we see a Canadian Government 2.0 Task Force, equipped with a similar budget?

Creating Government Websites from the Outside-In

At a panel discussion on redesigning government websites at PDF09, a question was raised about regulating US agencies to publish standards compliant information. A suggestion from the floor came up: “Don’t worry about regulation. Create software that makes it cheaper and easier for governments to publish standards compliant data, and agencies will use it.”

It’s a great point. Showing governments that something can be done by doing it first can inspire/shame the powers that be into action. It also creates a path of least resistance for agencies to follow. As Clay Johnson, director of Sunlight Labs argues: “We can get government to change by writing code and writing websites faster than we can get members of congress to agree how to do it. So let's do it for them." (from video linked here)

The US government website USASpending.gov shows how this process can work. Three years ago, a group of senators including Barack Obama and John McCain introduced a bill to US congress called the ‘Federal Funding Accountability and Transparency Act’. The act stipulated that the office of Management and Budget create a searchable website of nearly all government spending by 2008. According to a report by the US non-profit OMBWatch:

“At the time, the Bush administration and others believed that such a website could not be created, and if it could, it would be very costly and take many years to put into place.” (p. 19)

Regardless, the act passed. Faced with having to implement the website, the US Office of Management and Budget found that the easiest and most cost effective thing to do was to copy a non-profit site already in use that was doing the same thing. fedspending.org, built by the non-profit OMBWatch, had for some time been collecting and sharing information on federal spending obtained from Freedom of Information requests and other sources. The Office of Management and Budget licensed the software behind fedspending.org, and launched it as USASpending.gov, supplying the same information directly from the government.

As described in the Washington Post, the OMB was initially reluctant to participate with the non-profit, quoting OMB associate director Robert Shea as saying:

"OMB Watch spends a great deal of its efforts criticizing what I do the rest of my day, trying to improve program performance, [so] my level of interest in cooperating with them was very low,"

But finally Shea decided:

"OMB Watch had already proven it could be done, so why do it from scratch?"

Exactly. The OMB delivered the site ahead of schedule, and for exponentially less than the originally anticipated cost. Further, USASpending.gov kept many of the innovative features implemented by OMBWatch, including an API for programmers to access the data – a groundbreaking move at the time.

Creating websites that push the envelope on visualizing, sharing, and publishing government information drives innovation within government by showing what’s possible. As in the example of USASpending.gov, these websites can even become the cheapest and best option for governments to adopt internally. Governments are, in general, terrible places for creativity and experimentation – the Outside-In model may well be the most viable way of getting innovation into government websites.

Stay tuned for an in-depth comparison of the federal contract data available on USASpending.gov, and the data that’s available in Canada under proactive disclosure laws.

Open Government Data Roundup

News and Ideas on Open Government Data from Around the Web
  • US Open Government Initiative Collaboration Policy Drive Complete

    Approximately 20,000 people participated in the US Federal Government's Open Government Initiative, providing ideas and input on "How to Make Government More Open".   The initiative, run in three stages on three separate tools, was itself an experiment in using open, collaborative tools to collect and distill  feedback.  While reviews of the effort were mixed -- with many columnists focusing on early efforts by a fringe groups to derail the process -- governments everywhere have something to learn from the experience.

  • Sunlight Labs to One-Up data.gov

    Another open government effort that has received mixed reviews is data.gov.  While off to a promising (and precedent-setting) start, the website's collection of data sets has not grown as quickly as hoped, and  lacks some key features, such as the ability to provide feedback on the data.  Sunlight Labs has announced its intention to help the government, and users of data.gov, by doing one better.

  • Groups launch citizen-created data.gov's

    Inspired by data.gov, citizen groups in Russia and New Zealand are first off the mark in setting up their own government data listings.  Watch this space for Canadian updates.

  • Open311 Initiative Underway

    The Open Planning Project is facilitating an Open311 Initiative, "A collaborative effort to create an open standard for 311 services".

  • UK Guardian Crowd-sources Analysis of MP Expenses.

    In a great example of crowd-sourced journalism, Guardian readers are mining thousands of pages of MP expenses. Sample the discussion here.

  • UK ‘Power in the People's Hands’ Report Released.

    The UK Cabinet Office has published a concise but wide-ranging survey called 'Power in the People's Hands: Learning from the World's Best Public Services' that includes several open data case studies.  While not strictly open-data related, I found the 'Entitlements' case study, where Sweden was able to cut wait times in half, profound.  A good read.

In Canada:

Introducing Disclosed.ca

I’d like to introduce you to the newest VisibleGovernment.ca project: Disclosed.ca. Disclosed.ca is a tool for searching federal contract disclosures across government departments. These contract disclosures have been available online since 2004, when the current Proactive Disclosure laws came into effect. However, until now the records have been spread across ~100 department websites. Further, the disclosure websites are set up such that the records are effectively unsearchable.

Using well-honed coding skills and determination, Disclosed.ca’s founder Ilia Lobsanov has scraped these records from this multitude of department websites. For the first time, Disclosed.ca makes the bulk of federal contract disclosures available on the web as a single searchable database. Lobsanov estimates he has completed scraping about 70% of the federal disclosure websites.

Collecting this information is no easy task. Under the Canadian Westminster system of government, each department is responsible for its own IT. As a result, every department has implemented their own unique disclosure system, publishing the same information in widely varying formats.

Even partially complete, Disclosed.ca has attracted a following of users. The most traffic so far has come from investment websites, who are using Disclosed.ca to do due diligence on companies receiving government contracts and to research trends. This was an unexpected use of the site that took Lobsanov by surprise.

The fact that one of the first uses of Disclosed.ca is investment research is a perfect example of the economic value that is so often locked up in in-accessible government records. Making a few simple changes in how information is published -- making it machine-readable and re-usable -- can free it up to be used in new ecosystems of websites that create value, be it financial or social.

An interview with Disclosed.ca’s founder, Ilia Lobsanov, follows.

What motivated you to start disclosed.ca?

I stumbled upon the Proactive Disclosure web pages by accident, by way of the Environment Canada weather pages. Then I saw every agency was publishing contracts and grants and travel expenses. It was insightful to see the spending amounts. But it was impossible to search any ofthis data. So I got to work writing scrapers for each agency. The first iteration of the site went live in April 2008 hosted on Google App Engine.

How do you think disclosed.ca will help Canadians?

It will raise awareness of the fact that their Government is in fact transparent to an extent. Having applications like disclosed.ca will steer the Government to be even more transparent. This is good for Canadians.

What was the hardest part of creating the site?

Dealing with the Google App Engine API and its limited Search feature was sometimes like fumbling for a light switch in the dark.

On the other hand, acquiring all the contract data was a difficulty I had expected and welcomed. It forced me to write the scraper as generically as possible with the format variations between agencies defined in a configuration file. At the moment however the scraper is not as generic as I would like: there's some code that is specific to certain agencies which should be factored out.

How do you see disclosed.ca, and sites like it, evolving in the future?

Definitely more visualizations. More community involvement at the source code development end. More citizenship participation. For example: letting the public comment on and discuss specific instances of Government data.

'Beers for Canada' Campaign Raises $1005

Thanks, Canada! You pitched in to help us raise $1005 during our Beers for Canada campaign between June 30th and July 2nd.  Now that the dust has settled, the final donation tallies are:
  • $1005 raised.
  • 15 beers at $7
  • 10 pitchers at $20
  • 7 rounds at $100
Special thanks go out to these blogs for the great coverage that helped us get the word out: As well as Tim O'Reilly, Austin Hill, Tara Hunt, and other twitter power-houses who kept the campaign's energy up.

Stay tuned for an announcement on how this money will be spent.

Notes from Personal Democracy Forum

Top Stories
  • Michael Bloomberg, Mayor of New York City, announced a BigApps contest for applications based on open NYC data.
  • Vivek Kundra, US CIO, unveiled a dashboard showing metrics for US federal IT expenditures, including per-contract timelines and performance indicators. While there is no stated plan for this sort of metrics-based interface for the rest of US government expenditures, it can only be a matter of time.
  • Sunlight Labs launched Transparency Corps, a mechanical-Turk application for breaking up transparency tasks that only humans can do – such as tabulating the over 9000 ways the US govt. refers to Walmart in its contract listings -- into small, bite sized chunks so that anyone can participate.
Favourite Quotes
  • “I have to say I’m feeling pretty good right now.” – Ellen Miller, Sunlight Foundation E.D. after Vivek Kundra’s demo of an IT spending dashboard .
  • “They [the govt.] actually do look at these things when they exist. “ -- Mike Mathieu, FrontSeat founder, talking about how many of the points voted up on ObamaCTO ended up on the president’s implementation list.
  • “Technology brings power to the edges.” – Ellen Miller, Sunlight Foundation
  • “We’ve got to make government by default searchable and linkable.” – Jeff Jarvis
  • Alec Ross speaking to how a Columbian rally organized by four unknown young men on Facebook did more to damage the govt. than 10 years of military action.
  • “Does the average person believe they can have an effect? If not, you don’t have a representative government.” – Tom Crowl, at the ‘Hacking the City’ BOF
  • “You actually don’t need to regulate that much. Create software that makes it cheaper and easier for governments to publish standards compliant data, and agencies will use it. “ -- suggestion from the floor at “Redesigning .gov for Transparency and Participation” panel session
  • “One of our problems is just that we have too much content. Some of our sites have over a million pages, and a team of maybe four people.” -- bracing honesty from Sheila Campbell at USA.gov, the agency that oversees US federal websites, at the “Redesigning .gov for Transparency and Participation” panel session. An EPA webmaster added during Q&A that his organization had over 500,000 pages of hand-coded HTML to manage.
  • “It’s not about ‘embracing technology’, it’s about embracing people.” – Joe Raspers, on why the Obama08 internet campaign was successful.
  • The hoots of applause during Jim Gilliam’s presentation on ‘Imagining White House 2.0’.
  • Esther Dyson somehow ending up with a ‘Hello My Name Is’ Meetup sticker on her back.

This Canada Day, buy your country a beer.

Happy Canada Day! This Canada Day, treat your country to a beer. VisibleGovernment.ca is launching its first ever fundraising drive, ’Beers for Canada’. For the price of a beer, you can help VisibleGovernment.ca build tools to promote transparency and encourage our leaders to share more information openly.

Where will your beer money go?

The funds raised by the Beers for Canada effort will be used for:

  • Creating new tools and websites that encourage more open communication between government and citizens.
  • Launching the Code For Canada application design competition that awards prizes to people who build web, facebook, and iPhone apps that provide visualization, analysis, and access to federal government data sets.
  • Working with other open government organizations like The Sunlight Foundation in the U.S. and MySociety in the U.K. to bring tools they’ve created to Canadian screens, and to share Canadian-made applications with others.
  • Encouraging government openness in public forums, helping government organizations to share their data, showcasing examples of open government, and promoting the benefits of transparency in public office.

All it takes is a visit to www.beersforcanada.com this Canada Day holiday to help connect public officials to the general public. Have a great holiday and enjoy your beverages responsibly.