After a quick pass through Paul Miller’s draft Linked Data report for JISC, I looked out the notes I had made when we talked in the Black Medicine cafe. There were unusually few notes, for quite a long conversation.

I don’t think we really discussed anything that featured in the draft Linked Data report; not the implementation issues. We talked about the broader implications of linked open data for JISC services, about business models for support of open data, about the upcoming effort on data.gov.uk …

One topic I did take notes on was that of long-term URL preservation – what kind of institution to approach to make a commitment to keep a URL around for 30+ years for the use of, say, a library special collection georeferencing project (and hopefully many others).

Here is an edit to a set of notes I wrote for #simonjbains and others at the Digital Library in Edinburgh. It’s likely this requirement is not unique to geo-services, but bibliography and media archive projects would surely face similar needs to make sure that references really stick around.

RCAHMS was another interesting choice given their involvement in digital gazetteer reference already with
Scotland’s Places.

A big pile of city crime reports is not all that useful. But what if you could combine that data with information on bars, sidewalks and subway stations to find the safest route home after a night out?

In Washington, a Web site called Stumble Safely makes that possible. It is one example of the kind of creativity that cities are hoping to mobilize by turning over big chunks of data to programmers and the public.

Many local governments are figuring out how to use the Internet to make government data more accessible. The goal is to spawn useful Web sites and mobile applications — and perhaps even have people think differently about their city and its government.

“It will change the way citizens and government interact, but perhaps most important, it’s going to change the way elected officials and civil servants deliver programs, services and promises,” said Gavin Newsom, the mayor of San Francisco, which is one of the cities leading the way in releasing government data to Web developers. “I can’t wait until it challenges and infuriates the bureaucracy.”

Advocates of these open-data efforts say they can help citizens figure out what is going on in their backyards and judge how their government is performing.

But programmers have had trouble getting their hands on some data. And some activists and software developers wonder whether historically reticent governments will release data that exposes problems or only information that makes them look good.

It is too early to say whether releasing city data will actually make civil servants more accountable, but it can clearly be useful. Even data about mundane things like public transit and traffic can improve people’s lives when it is packaged and customized in an accessible way — a situation that governments themselves may not be equipped to realize.

A Web site called CleanScores, for instance, tracks restaurant inspection scores in various cities and explains each violation. After School Special combines data from San Francisco schools, libraries and restaurants so parents can plan after-school activities and see how children’s nutritional options compare by neighborhood. And Trees Near You, available for the iPhone, lets people identify trees on New York streets.

By releasing data in easy-to-use formats, cities and states hope that people will create sites or applications that use it in ways City Hall never would have considered.

San Francisco recently unveiled DataSF, a Web clearinghouse of raw government data that the public can download. The data sets include seismic hazard zones, street sweeping schedules and campaign finance filings. New York City’s Data Mine includes directories of sidewalk cafes, property values, horseback riding trails and historic houses.

Washington was a leader in releasing its data, and the federal government is doing it too at Data.gov.

Much of this data has always been publicly available, but until recently it has been almost impossible to find. Getting hold of it might have required tenacity, drive and endless phone calls.

The push to publicize government data goes as far back as the 1960s, but technology has made it possible for people to use the data in ways that would not have been possible even a year ago, said Eric Gundersen, president of Development Seed, the Washington company that created Stumble Safely. The company builds data and map applications for international development programs.

“The timing now with the open data movement is really critical because there are a lot of open-source tools that really make that data usable,” Mr. Gundersen said. These include the mapping tool he used to build Stumble Safely and also a site for the United States Agency for International Development that maps public health clinics.

Some activists are skeptical that governments will release politically risky data that could show that people are not doing their jobs. Mayor Newsom said he wants to release all kinds of data, and said he would not be surprised if “people who love to hate their mayor” create an application that maps his public schedule, to bolster their cases about which parts of town he neglects.

There is also the concern that people might misinterpret what the data is telling them.

“In the most basic of forms, with regard to crime stats and unemployment numbers, these kinds of bulletin boards are very useful,” said Saul Bloom, executive director of Arc Ecology, an environmental nonprofit group, who has been an activist in San Francisco for three decades. “But on detailed data dealing with very complicated material, you really have to know what you’re looking for in order to distinguish between good data and junk data.”

Mr. Bloom also worries that cities could manipulate data to gloss over things like unemployment rates by neighborhood.

Governments are trying to make data openness a more open process itself by asking people to vote for data sets they want to be released. In New York, for example, people have requested data on school violence, public restroom locations and bicycle accidents.

Still, asking for the data is often not enough. Software developers in New York have been unsuccessful in getting data feeds of pedestrian and bicyclist injuries and fatalities from the Police Department, said Noel Hidalgo, who is director of technology innovation for the New York State Senate and has been working with developers on building city-data applications. He envisions applications that overlay accident information on city bike maps.

Paul J. Browne, a deputy commissioner of the New York City Police Department, said it releases information about individual accidents to journalists and others who request it, but would not provide software developers with a regularly updated feed. “We provide public information, not data flow for entrepreneurs,” he said.

There have been other scuffles over who has the right to data. Routesy, an iPhone application, uses data from San Francisco’s Municipal Transportation Agency to show train and bus schedules and locate stations on a map. It stopped working for a while because a private contractor working with the agency wanted to charge a licensing fee for the information. The agency now requires its contractor, NextBus, to make the data freely available.

There is evidence that governments’ attitude toward publicizing data is changing. Two years ago, when a Web design and research firm called Stamen Design started a Web site, Crimespotting, that mapped crime data for Oakland, Calif., the city cut off access to the data a week after the site went up.

Bob Glaze, the city’s chief technology officer, said the frequent data requests from the site were disrupting the city’s own crime site. The city eventually changed its mind. And in August, Stamen’s designers unveiled a San Francisco version of Crimespotting with Mayor Newsom at their side.

Some government leaders are making data disclosure an official policy. Mayor Newsom signed an executive order saying city data should be released, and the White House is about to publish a directive expected to give similar instructions to federal agencies.

San Francisco, New York and Washington have all organized contests to encourage software developers to create applications with their data. And the developers are using the data to build businesses. Stamen, for example, uses Crimespotting to show potential clients what it could create for them. Other firms are selling the iPhone apps they have built.

The cities, meanwhile, are to some degree using developers to provide citizens with a service so they do not have to.

“We are increasingly governing in a time when the demand for services exceeds our resources,” said Aneesh Chopra, chief technology officer of the United States. If the contests “spur dozens of innovative applications,” he said, “then we’ve essentially achieved a policy objective at virtually no cost.”

Law.gov is a recent, U.S.-based, law-related open government data project, organized by an open access publisher of government information, called Public.Resource.Org. According to the project’s Website, Law.gov has as its goal to write “a report documenting exactly what it would take to create a distributed registry and repository of all primary legal materials in the United States.”

Law.gov seeks to make U.S. federal, state, and local legal information available free of charge on the Internet, in a manner designed to maximize interoperability and ease of access and to facilitate reuse. The organizers see the U.S. Government’s Data.gov project as a model for Law.gov.

Content for the Law.gov report is to be generated in part through a series of meetings to be held during the first quarter of 2010, and co-organized by faculty or staff at the law schools of Columbia University, Harvard University, Stanford University, the University of California Berkeley, and the University of Colorado; Northwestern University’s Oyez Project; Princeton University’s Center for Information Technology Policy (CITP); Cornell University’s Legal Information Institute; O’Reilly Media; and the Center for American Progress.

A panel about Law.gov (scroll down), and featuring Tom Bruce of Cornell’s Legal Information Institute, John Joergensen of Rutgers University Camden Law Library, Stephen Schultze of Princeton’s CITP, and personnel from Public.Resource.Org, will be held on January 22, 2010, at the workshop entitled Open Government: Defining, Designing, and Sustaining Transparency, at Princeton University’s Center for Information Technology Policy.

Other law-related Open Government Data efforts include the U.S. Government’s Open Government Directive, and the UK Office of Public Sector Information (OPSI)’s Single Legislation Service (SLS) (see details from Dr. John L. Sheridan via Dr. Adam Wyner here and here).

For more information about Law.gov, see posts by Joe Hodnicki and Erika Wayne, and the Law Librarian radio program about the project.

The Supreme Court Open Infrastructure Project, a new legal information dissemination effort, led by scholars including Professor Andrew D. Martin of Washington University and Professor Jerry Goldman of Northwestern University’s Department of Political Science, is now underway.

According to Daniel Martin Katz, the goal of the project is to create a freely available, central online repository of interoperable data about the U.S. Supreme Court, accessible by a wide range of users and easily integrated with other data and applications. Washington University’s Supreme Court Database is to serve as the platform for the repository. The project is related to a National Science Foundation grant devoted to extending the content of the database to include data from 1792 through 1946.

A meeting — attended by personnel from Northwestern University’s Oyez Project, the University of Maryland’s Digital Docket Project, the Computational Legal Studies blog, Cornell’s Legal Information Institute, Stanford Law School, the University of Pennsylvania’s Linguistic Data Consortium, and Washington University’s Center for Empirical Research in the Law (CERL) — to discuss plans for the project took place on 3-4 December 2009, at CERL. Reports on the meeting are available from Daniel Martin Katz and Mark Liberman.

The Supreme Court Open Infrastructure Project appears to offer an interesting example of the application of Open Data principles (see the explanatory article by Miller et al. here, and the related definition of “Open Knowledge” here) being implemented in connection with key legal information, by members of the academic community. This project complements law-related Open Data efforts taking place within government, such as the U.S. Government’s Open Government Directive, and the UK Office of Public Sector Information (OPSI)’s Single Legislation Service (SLS) (see details from Dr. John L. Sheridan via Dr. Adam Wyner here and here).

A selected list of additional law-related data resources (some open and some not) is available here, and a selected list of law-related Linked Data resources is available here.

Absolut Hacker

The Powers That Be swear that they’ll have a mailing list or Facebook Group or something. Any. Day. Now. They could have one now if they’d hire me!

Second Vancouver Open Data Hackathon

When: Wednesday, December 9th, 2009, 5 – 10 pm

Where: City of Vancouver Archives, 1150 Chestnut Street, Vancouver. More: http://vancouver.ca/ctyclerk/archives/about/index.htm

Limited to 30-40 laptops in use at once, and 80 people in the building.

Coffee and light refreshements will be served.

You can RSVP and drop a link to your blog on the wiki. And there’s also a Google Group (thanks to Lisa Mighton for the link)

There’s been a little hiatus here as I have been traveling. I recently learned that Microsoft has launched Codename “Dallas”, a service for purchasing and managing datasets and web services. It seems they are trying to provide consistent APIs to work with different data from the public and private sectors in a clean way. There’s an introduction here.

This type of online data repository seems to be an idea whose time has arrived – I have previously talked about resources like Infochimps, Datamob and Amazon’s Public Data Sets, and there is also theinfo.org, which I seem to have forgotten to mention. A recent commenter on this blog pointed me to the comprehensive knowledge archive network, which is a “registry of open data and content packages”. Then there are the governmental and municipal data repositories, such as data.gov.

Another interesting service, which may have a slightly different focus, is Factual, described by founder Gil Elbaz as a “platform where anyone can share and mash open data“. Factual basically wants to list facts, and puts the emphasis on data accuracy, so you can express opinions on and discuss the validity of any piece of data. Factual also claims to have “deeper data technology” which allows users to explore the data in a more sophisticated way compared to other services like the Amazon Open Data Sets, for instance.

Companies specializing in helping users make sense out of massive data sets are, of course, popping up as well. I have previously written about Good Data, and now the launch of a new seemingly similar company,  Data Applied, has been announced.  Like Good Data, Data Applied offers affordable licenses for cloud-based and social data analysis, with a free trial package (though Good Data’s free version seems to offer more – a 10 MB data warehouse and 1-5 users vs Data Applied’s file size of <100 kb for a single user; someone correct me if I am wrong). The visualization capabilities of Data Applied do seem very nice. It’s still unclear to me how different the offerings of these two companies are but time will tell.

So, the Conservative Party have leaked a leaked copy of the draft Government IT Strategy! I’d been privvy to an early draft through the Local CIO Council and hadn’t really thought anything was worth shouting about. In fact I’m not really sure that another government would do any much different, apart from branding and terminology. Whilst I am a strong believer in, and my dissertation relates to, ”co-production”, I’m not a believer in crowdsourcing per se, it’s a bit like mob rule or, even worse, minority rule or oligarchy, which is apparently the Conservative Party rationale for leakage. I had wondered if it had been a deliberate leak on John Suffolk’s part but I gather from the Cabinet Office that this was not the case, however they do insist it was an early draft and that the feedback will be very useful!

This is the leakage is latest Conservative version of a Conservative ICT non-strategy and not some little way from their earlier rallying cries around “open source”,  “open gov” and “open data”. On the W3C group on e-government, someone recently posted a list of alternative “definitions” for such data and here they are with credit to Winchel “Todd” Vincent III of
<xmlLegal> http://www.xmllegal.org/ This may be developed as part of the groups work but the original is his.

“Unavailable: You simply cannot get the data.  Data is cost prohibitive to publish. There may be security or privacy reasons not to publish.  Or, simply, no one ever thought to publish the data.

Not Translated: Data is available, but exists in a different language than the end user’s language.

Paper: Data is available, but it is only available on paper.

Free: Data is available at no cost and without restrictions.

Fee Based: Data is available, but only for a fee.
— Public: Fee Based: Government provides data for a fee.
— Private: Fee Based: Private company provides data for a fee.

Copyright: Data is available (in some way) but there are copyright restrictions on republication or reuse.

Copyright with License: Data is available (in some way), there is a copyright, but also a license that allows some use (other than all rights reserved).

Public Domain: Data is available (in some way) and is in the public domain, so there are no restrictions on use of the data.

Electronic: Data is available electronically.

Electronic: Web Browser or Paper-Like Electronic Document Format: Data is available but only via a web browser or an electronic document format and not in an easily parsed format (where Images/Graphics, HTML, XHTML, PDF, Word, and Word Perfect do not count as easily parsed formats).

Electronic: Structured Format: Data is available electronically and in a structured format.  A structured format would include delimited text, spreadsheet, XML, and the like.

Electronic: Structured Format: Schema: Data is available electronically and in a structured format.  Additionally, there is a schema available that defines the structured format.

— Government Schema: A government promulgates the schema. The schema may or may not be in the public domain.

— Standards Body Schema: A recognized standards body promulgates the schema.  Schema is licensed under a “copyleft” (perpetual, free, but with restrictions not to modify) or similar license (typical of W3C, OASIS, but not all “recognized” standards bodies).

— Private Schema:  A private company promulgates the schema.  The schema may or may not have licensing restrictions associated with it.

Electronic: Browser/Viewer: Electronic data, whether structured or not, is available only via a web browser or other viewer for viewing.

Electronic: Download: Electronic data, whether structured or not, is available to download.  Here, download means a “manual” download. Some manual user input must be done to download the data (e.g., downloading a spreadsheet or structured text file via an HTTP link or FTP) to the user’s local machine.

Electronic: Web Service: Electronic data, typically structured, is available via a web service (meant in a generic way, not specific to a technology) for machine consumption.  There is some standard, specification, or documented publication rules, such that machines can reliably access the data on an ongoing basis.  The point here is not the format of the data, but the reliability and availability of the connection to the data, so that machines can get to the data feed without human intervention.

Each of these qualities makes the data more or less “open” or “accessible” as a practical matter.  There are  many combinations of these that one could put together.”

If anybody in UK wants to remember the recent history of the National Land and Property Gazetteer (NLPG), they’ll remember the local property data expensively gathered with great efforts spent cleansing it. The authorities who have spent large sums of money are now likely to find this being given away. There is current effort on matching this data with that from the Electoral Register, this is the Coordinated Online Register of Electors (CORE) project. One of the issues around propert data in recent times has been resistance from the Royal Mail which produces the postcode file to allow any fee-free use of the PAF. So local authorities are expected to give away data that has been expensively cleansed in order that private organizations may profit – if that is the Conservative plan – to see it given it away like North Sea Oil, public transport, British Gas etc etc. The comment is that public money paid for it, so the public should have it – but what if they have to pay twice?

A variety of links to interesting and/or helping things on local government and social media, use of data, efficiency  and corporate governance.

Social media tips and how-tos

• A quick start guide to Twitter
  For those looking to begin on Twitter… good stuff from Dave Briggs and Learning Pool
• Editing Wikipedia
  Something about you or your organisation on wikipedia that you don’t like? Dave Briggs highlights some very useful advice on how to go about editing (or not) Wikipedia. Transparency and the truth are your allies on this one.
• 16 social media guidelines used by real companies | Blog | Econsultancy
  Highlights from (and links to) corporate social media policies
• The pieces of the digital engagement puzzle at Helpful Technology
  Steph outlines digital engagement strategies – convening is the key. Brilliant. I love convening (and being convened).
• Presenting our Know and Go series: Bite-sized information to help you start and improve your social knowledge management strategy
  social knowledge management approaches

Efficiency

Linked and open data

Talk About Local » Government data on the ground, making a difference

Free Our Data: the blog » Blog Archive » Gordon Brown announces OS maps to be free online

Democratic structures, engagement  and corporate governance

The EU-Constitution, later rebranded as the Lisbon Treaty did not get a very warm reception. 1. December is Lisbon day, the new Treaty enters into force. And what is the Council of Ministers about to do 30 November? This monday the ministers are about to pass the SWIFT agreement with the United States prior to the parliament getting new oversight competences. An affront against the European Parliament and “good governance” in Europe under the Swedish Council Presidency.

The SWIFT agreement legalises the business espionage of American services on European bank transaction data which is managed by a private Belgium consortial company called S.W.I.F.T. Officially around 3500 files a year are checked for counter-terrorism purposes, results are shared with European security agencies. But the data is extremely poisonous because financial transmission data reveals confidential business information which would seriously undermine the trust in the banking system. In terms of financial markets, transaction data information as such is of monetary value. Unilateral disclosure could distort financial market information allocation. In the summer when it was about to be set as an “A-Point” in the Council I felt it important to sent out a German press release.

Surprisingly the SWIFT debate does not receive much attention in the English news. In Germany it is big news. The Bundesrat, the Chamber of Federal States filed a strong resolution. The Libdems, among them Minister of Justice Leutheusser-Schnarrenberger are very sensitive to the issue. The German banking sector is alerted. In the middle of the month four nations blocked the agreement, among them also France, Finland and Austria. A fierce political battle happens behind the scenes. The current German position is abstention. Ironically Leutheusser-Schnarrenberger gets a lot of critical press now as if she was the driving force, not Chancellor Angela Merkel.

Whatever is decided tomorrow, an adoption would strongly undermine the Lisbon confidence. I fully agree with Elmar Brok (CDU) who called it an affront against citizens of the Union. The Swedish Presidency would be well advised to remove the vote from the agenda for tomorrow, just as a matter of good administration.

Here are the patent applications and grant from the US company Intellectual Ventures.

bulk.resource.org, a service of Public.Resource.Org.

“LOCKSS system has permission to collect, preserve, and
serve this open access Archival Unit.”

This system contains UNSUPPORTED, AS-IS copies of selected
U.S. government archives.

Anyone interested to analyse the zipped documents which contain XML files? A few years ago colleagues from Sweden created the GAUSS patent database, an early attempt to make patent information more accessible.

Commençons par le début : pour faire du journalisme de données, il faut des bases de données. Le premier enjeu du database journalism, avant le modèle économique ou le webdesign, c’est donc l’ouverture des données publiques. Car si les journalistes peuvent collecter eux-mêmes certaines données, ils ont nécessairement besoin, pour les traiter, de bases documentaires existantes.

En effet, la production de sens naît souvent de la confrontation entre deux ou plusieurs bases de données. Le crime mapping, par exemple, met en relation des données géographiques (délimitation des rues et des quartiers) et des données policières (listes de délits) ; de même, le New York Times a mis en parallèle des données sociodémographiques et des données économiques pour produire sa très belle infographie montrant les inégalités face au chômage selon la race, le sexe, l’âge et le niveau de formation.

En France, les principales bases de données géographiques sont détenues par l’IGN (Institut géographique national) et sont accessibles via le site Web de l’établissement public ou via son géoportail. Ce géoportail permet de naviguer en 2D ou en 3D sur des photos aériennes, des cartes et des données géographiques sur l’ensemble du territoire français. Ouvert en 2006, il s’inscrit dans le cadre de la directive européenne INSPIRE (Infrastructure for Spatial Information in the European Community) visant à favoriser la production et l’échange de données géographiques entre Etats membres de l’Union.

L’IGN détient, pour l’ensemble du territoire métropolitain et les départements d’Outre-mer, des bases de données relatives :

- au relief sur la France

- à l’orthophotographie (images aériennes ou satellitales de la surface de la Terre rectifiées géométriquement)

- à l’information cadastrale

- aux adresses

- aux réseaux routiers

- aux limites administratives

- aux toponymes

L’IGN conserve aussi des bases de données géographiques à l’échelle de l’Union européenne

Parmi toutes ces données, certaines sont gratuites, librement exportables, d’autres nécessitent le paiement d’une licence, d’autres encore sont accessibles gratuitement mais sous forme d’échantillon.

Les bases de données librement accessibles en format numérique concernent :

- le référentiel du relief sur la France décrivant la forme du terrain à différentes échelles

- le réseau hydrographique français

- la description de l’ensemble des unités administratives françaises

- le Répertoire Géographique des Communes

- la carte routière 901 IGN sous forme de dalles géoréférencées

- les contours délimitant le monde terrestre et le monde marin

Par ailleurs, l’IGN met à disposition sur son site des fiches techniques de géodésie et de nivellement ainsi que toutes les données issues des différentes stations du Réseau GPS Permanent (RGP).

Les photos aériennes sont également consultables et seront bientôt téléchargeables gratuitement ou à un très bas prix.

Enfin, le géoportail teste une API JavaScript qui permet d’incorporer dans son site Internet des cartographies dynamiques superposant plusieurs couches de données (cartes IGN, photographies aériennes, parcelles cadastrales, routes, bâtiments, limites administratives, réseau hydrographique, réseau ferroviaire…)

Pour le moment, l’API Géoportail est en phase de bêta test, donc téléchargeable gratuitement. A l’avenir, elle nécessitera de souscrire à un contrat spécifique, même si 100 000 pages vues gratuites seront accordées aux sites Internet grand public.

Malgré tout, hors de ces données en accès libre, il faut, pour utiliser les bases de données les plus importantes, acheter une licence coûteuse.

Que conclure à l’issue de cet exposé ?

Les bases de données fournies gratuitement par l’IGN suffisent-elles pour la majorité des cartographies ?

Finalement, qu’offre l’IGN de plus par rapport à ce que Google Map donne déjà ?

N’ayant jamais réalisé de cartographie moi-même, je n’ai pas la réponse à ces questions, mais je suis très curieuse de connaître vos opinions sur la question.

The Google public data functionality isn’t just for the US anymore: Google recently announced that World Bank data is available for remixing. Exciting! Best explained by their blog post on’t.

Di questo servizio abbiamo già parlato ma mi fa piacere riprenderlo attraverso il capitolo 11 di Library Mashups (come sapete, nell’esplorazione dei vari capitoli, non procederemo con ordine , intitolato Mashing Up Open Data with biblios.net Web Sevices e scritto da Joshua Ferraro di LibLime.

All’inizio del 2009 la società che vende servizi per software open source, LibLime, ha rilasciato un servizio di catalogazione gratuito, web based e che poggia su una base di milioni di record bibliografici liberamente adoperabili (openly licensed): biblios.net. In particolare, il capitolo si focalizza sui servizi web (web services) offerti da LibLime insieme al software di catalogazione e alla base dati di record.

I biblios.net Web Services (BWS) poggiano su una conquista fondamentale: gli Open (Bibliographic and not) Data, che a loro volta consentono la libertà e gratuità di intervenire sui record bibliografici, sui metadati prodotti dalle biblioteche e rilasciati/messi a disposizione di tutti attraverso licenze dedicate. Questo dei dati è sempre stato un grosso ostacolo nel mondo bibliotecario – e chi frequenta questo blog da un po’ lo sa bene. Spesso i molti e ricchi e riccamente strutturati metadati che i bibliotecari con grande fatica e sudore di fronte creano quando catalogano i libri, rimangono poi confinati entro ILS (software di automazione e catalogazione) proprietari o comunque chiusi alla possibilità del rilascio libero e del riutilizzo (al netto delle esperienze di catalogazione cooperativa, che però sono un’altra cosa).

Joshua Ferraro, invece, mette subito in rilievo lo slancio che la nascita di licenze come la PDDL (Open Data Commons Public Domain and Dedication Lincese) e di iniziative di apertura dei forzieri dei propri metadati prese da biblioteche come la Library of Congress o la Open Library di Brewster Kahle, ha dato all’impresa di LibLime, di rilasciare nel pubblico dominio così grandi quantità di record bibliografici a disposizione di tutti (di nuovo, se n’era parlato qualche tempo fa). Recod bibliografici che sono stati poi utilizzati dalla stessa LibLime per i suoi servizi e in particolare per la creazione di un livello di accesso e di interrogazione (le famose API, Application Programming Interface) concreto e usabile da tutti gli utenti.

I BWS sono infatti proprio un set di API create per consentire ai programmatori o geek librarian che dir si voglia, di scrivere applicazioni che interagiscano con il database di biblios.net e creare quindi mashup con i dati e i servizi ritornati da questo provider. I web services disponibili sono attualmente:

• Searching for bibliographic and authority records (OpenSearch, SRU/W and Z39.50)
• Retrieving single records (UnAPI)
• Harvesting with OAI-PMH
• Sending data to ‡biblios.net

Verranno invece presto attivati i seguenti:

• Download the ‡biblios.net Dataset (BitTorrent) (coming soon)
• Programming Guide (coming soon)
• API Reference (coming soon)

Uno degli interessanti esempi mostrati nel capitolo, è il mashup creato grazie al SRU target service, che restituisce i dati delle liste di autorità contenute nella base dati di biblios: il catalogatore che stia inserendo nella scheda catalografica di un volume una cosiddetta voce controllata (potrebbe essere quella del nome dell’autore oppure del soggetto/topic con cui si classifica il volume), può attingere on the fly alle voci controllate conservate e messe a disposizione da biblios, attraverso un semplice ed efficace menu ad auto-complete – in questo modo non solo potendo attingere a dati uniformi, controllati e di qualità, ma anche evitando di perdere tempo nell’andare a interrogare separatamente un altro database.

L’altro esempio che Ferraro propone è quello relativo agli strumenti per facilitare la catalogazione cooperativa delle biblioteche, sempre usufruendo dei web services messi a disposizione da biblios. In questo caso si sfrutta la potenza del protocollo OAI-PMH, dei feed RSS e del buon vecchio Z39.50, per costruire un mashup di notifica e aggiornamento delle modifiche avvenute su un record: le biblioteche in una rete cooperativa possono così venire a conoscenza dei cambiamenti intervenuti su una scheda che è stata già acquisita dal catalogo e, se del caso, anche decidere di accogliere quei cambiamenti, sovrapponendo la scheda catalografica modificata con quella del proprio OPAC, in maniera del tutto rapida e automatizzata.

Forse questo capitolo è il più interessante di tutti almeno sotto un profilo: mostra come il catalogo e la catalogazione non solo non sono esclusi, in quanto ambiti di attività tradizionali, dalla creazione di servizi innovativi e mashup, ma possono con la loro ineludibile centralità venire impattati pesantemente dalla creazione di servizi agili, in grado di eliminare le parti più ripetitive e meccaniche di certe attività e aprire la strada alla creatività nell’utilizzo delle informazioni e dei dati contenuti negli OPAC, frutto di anni e anni di lavoro delle migliori menti bibliotecarie…

A bit overdue (I’ve been talking about doing this for a couple of months), but at last there’s now a Ning app for OpenlyLocal local data so anyone who has a UK Ning hyperlocal site (well, anyone in the 90+ councils we’ve opened up data for) can now have information about their council right there in their site.

Like the OpenlyLocal Google gadget (see OpenlyLocal info on your website, Part 1: Google Gadgets), it’s fairly straightforward to use. You need to be the owner of the Ning community to add it, and then it’s automatically available to users as just another tab (like Forums, Videos, Photos, etc). Once you’ve done it you (and the other members of the community) can see the council’s key data, upcoming meetings, members and committees. More features and functionality will be added, but it’s already a very useful addition to any hyperlocal site.

This is what it looks like in action:

So how do you add the OpenlyLocal application to your Ning community (NB Ning Apps need to be added by the network creator). It’s a breeze, and should take no more than a couple of minutes (probably a lot less):

1. Go to the ‘Manage’ Tab on your network:
2. Click on Ning Apps and you’ll be shown the Ning App directory. Quickest thing is just to search for the OpenlyLocal app:
3. Choose the OpenlyLocal application:
4. You’ll then be redirected to the Tab management screeen, where you can change the name of the Tab for the app . By default it’s “Council info :: OpenlyLocal”, but might be better to be just “Our Council” or “Council watch” if space is tight. *Important*: If you do change the name, you must click the Save Tab Settings button, otherwise just click on the link:
   
5. You should now be shown the OpenlyLocal page (if not, just click on the tab), and you should click on the ‘edit settings’ link (in the top RIGHT of the info area, not the ’settings’ link just above it and to the left).
6. Select your local authority, and then “Save Changes”
   
7. The app will then get the data from OpenlyLocal (but some may be hidden – if so so just reload the page).
8. That’s it.

There’s more features coming, but I hope you’ll agree it’s an essential addition to any Ning hyperlocal community. Comments as ever welcome, and the code behind the application will be shortly uploaded to the OpenlyLocal github tools page.

p.s. To remove the app, just go back to the tab management page and click on the ‘x’ beside the tab.

We may be in a recession, but business has been booming in Oxford recently in one sector which is close to my heart … no, not bookshops (though there are still plenty of those) but chocolate and coffee shops. Hotel Chocolat is one Cambridge shop which I’ve been missing ever since moving back to Oxford, so I’m delighted that we now have our own branch on the High Street; similarly with the Fudge Kitchen, now open on Broad Street. The new Missing Bean coffee shop on Turl Street is reputed to be excellent, but it’s been so popular so far that I haven’t yet managed to get a seat there to confirm this for myself; and a new branch of lovely organic Green’s café has just opened on Queen Street — I know that’ll be good because I go to their St Giles branch all the time! (And if, as it seems, Green’s are slowly following long-time Oxford favourite G&D’s around the city, then hopefully we can look forward to an East Oxford branch of Green’s in the future….)

If you too have trouble keeping up, you might be interested to know that you can get an RSS feed of shops newly opened in Oxford from the Oxford OpenGuide wiki. And if you know of a shop that’s opened recently that isn’t on there, then why not add it? You don’t need to write a whole essay for your page to be useful — just fill in the basic facts and someone else (or you!) can always add to it later.

The OpenGuide does include opening hours for shops, but a new Opening Times wiki has also started recently. It makes it very easy to add opening times (I figure that even if nobody else ever looks at the data I add, it’s a useful way for me to keep a note of when the shops I use are open), and also easy to search for things which are open near you — ideal for those moments when you simply have to know where the nearest open coffee shop is!

I’m happy and hopeful at the announcement by the government that “mid-scale” Ordnance Survey data sets will be made freely available from April next year. It means we’ll be able to move a lot more into the “open data” side of Unlock services, offer open postcode and grid reference lookups.

Unlock’s open data gazetteer will become a lot more flexible with detailed footprints for many places rather than just pairs of coordinates. This will make it more worthwhile putting more complex spatial queries into the API (searches for intersections of footprints, searches for buffers around footprints, etc.) as they can be more widely used. One day we will be able to say, in public, “Show me images of all towns within a mile of the banks of the River Tweed”.

Well, the news is circulating but there will now be a 12 week consultation period before concluding exactly what OS data products will become openly available to us and how best the data should be distributed. Ed Parsons offered a ;speculative list of OS data products which he thinks may be opened next year. From our perspective, BoundaryLine, CodePoint and Meridian2 would be a great start, along with the OS Locator mid-range gazetteer which isn’t on Ed’s list.

Several projects having to do with open-access data from governmental and municipal sources have been announced in the past few months. Governments are, of course, already using analytics for a lot of things – this post from A Smarter Planet talks about how the Social Security Administration in the US is “using analytics and predictive modeling to make quicker determinations on disability applications for those in need” and how the US Postal Service is “extracting valuable insights from information on mail delivery to improve on-time delivery performance” (a postal service that improves – now that’s a novelty!), but now (part of) these data will be available to anyone.

In the US, federal data are or will be made available at data.gov. The site seems pretty well-designed, with data sets being queriable online and raw data being downloadable in XML, comma-separated file and other formats. Among the data sets at data.gov, there are some interesting health-related ones like the National Cancer Institute’s Surveillance, Epidemiology and End Results (SEER) database.

The City of San Francisco has its own data clearinghouse, dataSF. Strongly supported by mayor Gavin Newsom, it was launched in August. Given the huge population of programmers in SF, it’s no surprise that mash-up applications using the data started to appear quickly, e.g. “Routesy, which offers directions based on real-time city transport feeds; and EcoFinder, which points you to the nearest recycling site for a given item” (see this Guardian article for more.) New York City has its own NYC Data Mine, which has come under some criticism. It is hosting the BigApps competition, which is meant to “stimulate innovation in the information technology and media industries, and attract and support developer talent to develop web and mobile applications (apps) by using City data.”

The real pioneer in open government data, though, seems to be not the US but Australia. The Australian government recently arranged a “hack day” named GovHack, where they invited programmers to develop mashups and applications around government data during 24h intense hours. Some of the projects that came out of this event were:

The overall winners LobbyClue, by a team comprising members many of whom had never met before the event. LobbyClue is an in-depth visualisation of lobbying groups’ relations to government agencies, including tenders awarded, links between the various agencies, and physical office locations

It’s buggered, mate, In true Australian style, allows you to report buggered toilets, roads, etc, with an easy-to-use graphical interface overlayed on a map. Their idea was to combine this with local government services to fix issues in the community. Built by a team of developers from Lonely Planet.

Know where you live, a stylish presentation of ABS data (along with Flickr Geocoded photos), pulling in relevant information for a particular postcode: rental rates, average income, crime rates, and more. Built by a team of developers who work at News Digital Media.

Pretty cool, isn’t it?

Zoals gepubliceerd op Frankwatching:

Tijdens de Europese ministeriële conferentie over e-overheid op 18 november wordt er door verschillende initiatiefnemers de ‘Open Declaration on European Public Services’ aangeboden. Deze petitie probeert een verbinding te leggen tussen Europese overheidsdienstverlening en een moderne samenleving die op een juiste manier gebruik maakt van internet en webtechnologie. De petitie zal in ontvangst genomen worden namens alle ministers van de lidstaten die zich bezighouden met publieke dienstverlening.

Open overheidsdienstverlening

Wat bedoelt men nou eigenlijk met open overheidsdienstverlening?  Volgens de Declaration zijn 3 kernpunten van belang, te weten transparantie, participatie en zelfredzaamheid. Deze kernpunten zouden moeten leiden tot een overheid die een betere dienstverlening aanbiedt en die nauwer aansluit bij de verschillende behoeften die burgers en organisaties hebben.

Transparantie

Er wordt gesteld dat alle overheidsorganisaties transparant dienen te zijn, tenzij dit expliciet ongewenst is. Dit betekent dat ze informatie rond besluitvorming en functioneren toegankelijk moeten maken. Dit kan bijvoorbeeld stemgedrag zijn, zodat er analyses gemaakt kunnen worden van partijen en politici. Niet alleen in Nederland interessante informatie  voor de journalisten, in Europese landen die worstelen met corruptie een must!

Ook valt te denken aan bijvoorbeeld het aanbieden van de rijksbegroting in XML op Prinsjesdag, zodat journalisten en onderzoekers makkelijk visuele analyses kunnen maken en trends kunnen identificeren.

Participatie

De overheid zou ook de inbreng van de burger makkelijker kunnen maken. De invloed op beleid is nu nog vaak een zaak van hoge heren, lobbyisten, activisten en zeer actieve burgers. Kortom, voor de gemiddelde Nederlander is het niet mogelijk om snel en makkelijk een standpunt of mening naar voren te brengen.

Er zijn ook  hier nog vele andere mogelijkheden te bedenken. Bijvoorbeeld het grootschalig verzamelen van relevante beleidsonderwerpen voor een net aangesteld kabinet.

Zelfredzaamheid

Zelfredzaamheid, de vertaling van ‘empowerment’, van burgers kan vergroot worden. Er is (ongewenste) afhankelijkheid bij burgers van overheidsdiensten, in de vorm van o.a. openingstijden, ingewikkelde formulieren en websites.

Deze onafhankelijkheid hoeft zich niet te beperken tot de relatie tot de overheid. De zelfredzaamheid bij burgers in relatie tot medeburgers, bedrijven en instellingen kan zeker ook vergroot worden. Waarom moet er betaald worden voor info uit het Kadaster, postcodebestanden en KvK-uittreksels?

Waarom kunnen we niet vrij bij informatie die opgebouwd is met onze belastingcenten?

Helaas moet een initiatief als OpenKvK nog de informatie uit de KvK-website peuteren, met allerlei programmeertrucs, en laten ze zien dat het beter kan middels een open API .

Er zijn ook voorbeelden te bedenken zoals portals waar men zowel bankzaken, belastingaangifte en de boekhouding kan doen, maar ook websites die informatie uit het kadaster combineren met KvK -gegevens.

Petitie tekenen?

Als deze ideeën je aanspreken, laat dan snel voor de 19e je stem horen. Want met deze stem open je de overheid van Europa voor iedereen.

Thank you for your letter of 28 October in reply to my email about the postcode database. I am grateful for your speedy response.

Your letter raises a number of points which I feel need to be addressed.

As far as I am aware, this issue is not before courts. Legal action by Royal Mail has forced the developers – ErnestMarples.com – to disable their service.

I am not suggesting placing restrictions upon Royal Mail. The issue here is releasing data which is of public interest. In this case, the developers wish to use the data for solely non-profit purposes with benefits for all citizens.

Any data held by or on behalf of public bodies should be treated as a public, not a commercial, asset. This is the case in the United States and other countries.

In the UK, you may be aware of the Cabinet Office initiative to make public data public. The Digital Engagement blog has further details:
http://blogs.cabinetoffice.gov.uk/digitalengagement/.

A number of local authorities – such as Lichfield DC – are also going down this road.

The government has asked Tim Berners-Lee, inventor of the world wide web, to advise on the open data issue. I am encouraged that the Conservative party has also recognised the need for better digital engagement. In this respect, the appointment of MySociety’s Tom Steinberg as an advisor, though seen as controversial by some, is a positive step.

On the question of the postcode database, I do not see why Royal Mail’s overall profitability should determine treatment of public data, especially where it is being used on a non-profit basis and for public benefit.

You comment on the use of early day motions (EDMs). As a former civil servant who used to brief ministers on replying to EDMs, I am aware of the volume of motions that are tabled before the House of Commons. Whilst some EDMs may be viewed as frivolous, in many cases this is the only way to raise awareness of important local or national issues. What would you propose in place of EDMs?

As to the administration cost, again from my experience, I am sure this pales into insignificance compared with the costs of correspondence between MPs and government departments or answering parliamentary questions.

I hope you can support open access to public data, particularly the postcode database, irrespective of signing an EDM.

I would prefer any replies by email as this saves paper and taxpayers money.

Yours sincerely,

Peter McClymont

The Toronto Open Data Lab took place at Toronto City Hall today as part of the Toronto Innovation Showcase, covering the open data catalogue released by the City (toronto.ca/open doesn’t render well in chrome, so I suggest using another browser).

Inspire Sessions

After some brief introductory remarks by Mark Kuznicki, Peter Corbett spoke briefly about Apps for Democracy, open data, open cities.  He argues that there are only six cities with an open data catalogue, although he believes that will grow significantly in the years to come.  Doing a quick google search implies these cities are amongst them:

• Toronto
• Vancouver
• Washington DC
• San Francisco

There was an interesting discussion during the Q&A about data standardization.  If each open city uses its own unique format for data, that will make application portability difficult.  That said, I’d be happy to see the cities open their data first, and work towards a common, extensible standard as a second milestone.

Next up was Dave Wallace, CIO of the City of Toronto, who talked a little about the process, and also about the data that is be launching today:

• GIS Data
  • Centreline – streets, rivers, rail, trails
  • Address points
  • Business Improvement Areas
  • Parks
  • City Wards and Neighbourhoods
  • Solid Waste Districts
  • Food banks, rent banks
  • Geocoder Web Services
  • Web Map Services
• Apartment Standards
  • By-law infractions against apartment buildings
• Child Care Finder
  • Day Care Centres
  • City and city-subsidized spaces
• Festivals and Events
  • Real-time feed of upcoming events
• TTC Route & Schedules
  • Routes
  • Stops & details
  • Scheduled arrival
  • Times per stop/route

They are looking for community input on the next priorities for future releases.  The Inspire session ended with two ignite presentations by David Suydam (“Testing the untestable”) and Matthew Milan (Open is a Two-Way Street).

Learn

The break-out sessions split the attendees into six groups, and each group spent some time with data experts in the various categories.  We talked briefly about DataTO.org.

The TTC Schedule and Stop Metadata looks like it’ll be text files of tabular data, pretty simple stuff for now.  They’ll list routes, the stops for each route, the scheduled vehicle arrival times for each stop per route.  There are version fields that might imply you can look forward and back over data, or at least use that for comparison from one data pull to another.  This data will is updated fairly irregularly — every six weeks or so.

The apartment standards data is currently only available via a web user interface (Violations Search, Investigation Activity), the data itself may or may not be directly available, there was some confusion on that front.Scraping this data would require a lot of requests, because the data is paginated, and you’d need to make a detail request for each item.

The GIS data is available in centreline data is available in shapefile format.  The address file is available as a pointfile.   The point will be somewhere within the building footprint, rather than, say, a shape or a centroid.  It’s about 510,000 address points, and was born out of 911/emergency data.  It has information about the point, if it’s a school, church.  There’s a shapefile for park data, city wards.  There’s a SOAP web service for accessing the data, intersection searches, address validation, ward searches, “what ward am I in?” “what solid waste district I am in?”  Another foundation data set is the property boundaries from MPAC.  The third way is through a web map service, using GIS mapping application or an OGC-compliant web-map viewer (e.g. Gaia from the Carbon Project).

The festival and event data is available in raw xml file form served out of Domino.  The format is far from ideal, but at least it’s available.

The child care data is pretty limited, but, again, it’s available, which is a start.

Next Steps

It’s a good start.  It’s data that can be used to build applications, and it’s been opened up, and that’s great.  There’s also a central place to find this information now, which is also a good thing.  But there’s still a lot of work to do.

How best do we communicate what new areas of data we’d like to see?  And with existing data sources, how do we ask for more information about the data, addition data in the same subject area, different kinds of feeds/queries/apis, changes to the format of the data, and so forth?  Take a look at the festival and event data by way of example and you’ll find yourself asking a lot of these questions:

• How do I find ‘new’ events?
• Can the format be adjusted to give geocoded location?
• Can the email address format be simplified?  The existing html anchor mailto link seems overly complicated.
• APIs for queries and submitting events would be useful.

So, basically, I’m glad to see this happening, but it’s just the barest scratch on the surface of what’s possible.  Join the conversation on the DataTO group and site.

He was a hardworking farmboy.

A number of cities are embracing the Obama administration’s ‘open up and innovate’ approach to government with initiatives that build on the idea of Apps for Democracy.

I’ve been wondering what opportunities there are for smaller cities and suburban and rural areas to harness this energy. In large urban centers like New York City and DC, there are so many people that building a web-based tool that engages or assists even a small slice of the population can have civic value. These are places where social capital is diluted, but information density is high. What about the opposite?

I was excited to come across a post on the OSTP Blog that describes Manor Labs, a government innovation initiative in Manor, Texas. Manor has 5,800 residents, and it looks like they are going after open and innovative government with the same gusto as the big guns.

Something that caught my eye is the Manor Labs community input metric, which gives the popularity of an idea a stock-like score. The stock on your idea might go up. It might go down. Ostensibly, one might see the stock – calculated in ‘Innobucks’ – change over the course of a campaign to pass a new law. The stocks are tallied together on ‘The Leaderboard.’ At the time of writing, the ‘wealthiest’ individual in Manor is Dustin Haisler, the CIO and Municipal Judge for Manor. The whole thing is a tad complicated, but since the project itself is a pilot, it’s good to throw in some fun metrics, right?

My favorite part of Manor Labs is The Idea Funnel, a visual representation of the suggestions, with their status (Incubation, Validation, Emergence, or Closed). Interestingly, I haven’t seen something like the Idea Funnel in open government contests in larger cities. The funnel shows which ideas have been, for example, endorsed by ‘technical experts’ (I assume these are city servants with area-specific knowledge). It also shows which ideas have been rejected. Openness isn’t just about open data, it’s also about following and contributing to civic processes, and a pipeline like the Idea Funnel gives the public a window into the idea vetting process.

The whole thing is powered by Spigit. Dig it.