Open Data Argument Minus Use

I’m watching BB4’s The Joy of Stats, and it just described on the push for open data in US crime statistics. Today I was planning a conference session about using open content, open data and open access research papers. So the question of “what is open data for?” is at the forefront of my mind and I can’t resist responding to to Tony Hirst's open data skeptic challenge : "What public is intended to be the user of all this public data, and how are they going to use it exactly?"

The question brings to mind this post by Tom Steinberg on Open data how not to cock it up

“The first way we can avoiding cocking up Open Data is to ensure that we always advocate for it in the same way that scientists advocate for Blue Sky science research – we must argue for it as a numbers game – a calculated risk that is worth taking. So, we say loud and clear : we do not expect the majority of government datasets will contain massive wells of untapped value, just as we don't expect that most university research will  yield a new penicillin, or an atom bomb ...”

So maybe much open data is long tail: niche. The chances of it being used might be low. But without it being available there is no chance it gets used.

So what if 99% of open data never gets used. Does this mean it’s pointless to advocate open data?

I wonder if there is an argument for open data that doesn't rely on it being used.

The data is already there somewhere, but it’s locked into proprietary systems, only published as visual images within reports, only exposed for different purposes.Forcing the data out of systems onto spreadsheets and websites at least makes it clearer that the data belongs to the whole council / department / agency, not to the head of finance, or the planning managers etc. Treating data as data has got to be better than treating data as reports.

But what’s most interesting to me is that I think all these opens: open educational resources, open data and open innovation are really about new ways of working. The things you have to do to make your work more open make your work better. Knowing people will see your work makes it better. Inviting other people to see your work as you do it makes the work better. Working openly is just better.

Transparency is ethically better than lack of transparency. Freedom of Information legislation led to publication schedules led to open data. Assuming that government data should be open gives people working inside government a reason to try to make the right things happen. The threat of bad decisions being exposed is a weapon that can be used to argue for better decisions.

Back to Steinberg:

“It may surprise you to hear that my vision of a perfect (but realistic) government, is one that would release nothing, not a jot of data, not a single row or column..... until someone asked for it. When they did ask, my perfect government would then instantly publish that data in a brilliant, helpful format, regularly updated, and running on a lovely webservice that fulfils every data-mashers dreams.”

Comments

[Tony Hirst]

The Tom Steinberg ref is a good one - I'd forgotten that post...

One of the things I've been pondering is the extent to which:
- published data is data that councils have been collecting anyway;
- if so, what have they been collecting it for and how do they use it?
- as far as workflow goes, could the open version of the data be a step in the workflow?

I'm reminded of several occasions years ago when I was advocating for the use of RSS/Atom, and had some success in persuading folk to publish various feeds, of a sort.... But it was quite a common occurrence when I looked at the feeds days or weeks later that they were actually rotten and didn't work properly...

Same with OERs - folk seem to think publishing is the be all and end all... But how about for same data sets/feeds/OERs, the publishers actually demonstrated the data/feed/OER *working with someone else's data/feed/OER, or showing how the data/feed/OER is actually used within the organisation/institution?

If the publishers don't see any utility in the stuff they publish (notwithstanding the aphorism that the best ideas/use cases will be thought up by someone else) why should anyone else?

tony
PS are your feeds working? I subscribe to the Atom feed, but Google Reader doesn't seem to pass the posts through?

[amber thomas]

So here's a view from my friend who has been managing open data projects at a council: http://808kate808.wordpress.com/2010/12/16/killing-open-data-softly/
"reluctance to share stems from early childhood. We’ve all sat there in classes shielding our work so other kids can’t see what we’re doing. Although back then it was because we didn’t want them to see our answers, now it’s because we don’t want people to see our mistakes"