How open should open data be? Transport for NSW at the centre of a data controversy

Some will remember the knots that RailCorp tied itself into in 2009 when attempting to sue three developers for packaging Sydney rail timetables into mobile apps.

How things have changed. Recently the NSW government applauded one of those developers for his mobile app, which has reportedly been downloaded a million times.

However the agency which absorbed RailCorp, Transport for NSW, has now been thrust into the centre of another data controversy, with Fairfax's Ben Grubb reporting a row over how real-time transport data has been released.

The gist of the row is simple. Transport for NSW had worked with PWC to hold the ‘App Hot House’ competition  with a limited number of developers to see what they could do with its real-time data.

The outcome was several good apps, which are now available for the public and have been mentioned (some would say promoted) via various Transport for NSW websites, including 131500.com.au.

However the real-time data used in these apps has, thus far, only been made freely available to the developers who won the App Hot House competition. These developers are now selling their apps via mobile stores, presumably at a profit.

In this situation I can see both sides.

Transport for NSW is conservative, risk-averse and feeling its way in the open data space. The organisation has come a very long way in the last three years and is still addressing the culture change and understanding the impacts and potential risks of providing free data to developers to make apps that people rely on.

By selectively releasing real-time data the organisation can maintain a sense of control and address its accountability requirements while studying how it can best make the data more broadly available.

Meanwhile some in Australia's developer community are frustrated that they didn't get picked as part of the closed group granted access to the data. This group has had no opportunity to innovate on or profit from the information, which a select group of 'insiders' was able to be first to market with their real-time timetable apps.

This could be a permanent commercial disadvantage for the bulk of the developer community. The App Hot House winners have time to build experience working with the data and, as we hear regularly in the corporate and start-up space, first-mover advantage is regularly the difference between success and failure.

So did Transport for NSW do the right thing? Or do developers have a point about the agency restraining trade through selective data release?

In my view there's truth on both sides. Transport for NSW has a legitimate reason to be careful as the custodian of this data - which is both valuable and sensitive to small errors. However developers do have a point that they are missing out - and so might be the public and government (on innovation and competition).

In balance, however, I favour Transport for NSW's perspective. Open data is still very new and an 'undiscovered country' for many government agencies, as well as for the public. 

While it would be fantastic to see the organisation fling open the doors and allow all developers access to real-time data, there are legitimate concerns around data provision and security which make it prudent for Transport for NSW to take a slower and more measured approach to data release.

While app developers may be disadvantaged by late access, the risks for the public if Transport for NSW's systems collapse under the demand for real-time data are much greater. 

Equally, by first working with a small set of developers, Transport for NSW can minimise the risk of events like the NextBus failure in Washington, where the app developer was at fault of their app failing to work correctly, however the Metro system still received an, undeserved, share of the blame.

There were, however, some transparency steps that Transport for NSW could have taken (and I would have recommended if involved) to mitigate the kind of controversy in which they now find themselves.

Firstly the agency could have been extremely public about why it was working with a small number of developers at first and what its longer term plans or hopes were for the data. While some would have still complained, there wouldn't have been a 'data void' to be filled with rumours and speculation. 

The media, and most developers, would have accepted that Transport for NSW, in its custodial role, has a right to pilot the release of real-time data to better understand how to prepare its systems and processes for a broader release in future.

This is an unfortunate and unnecessary controversy. It could have been avoided with some savvy social media and communication planning and turned (as it should rightly be) into a triumph of government organisational culture change and openness. 

Few other government agencies have made the big change that Transport for NSW has made in such a short time.

I hope this huge achievement will not be overlooked and that other agencies don't draw an incorrect conclusion that it is better to bottle up data than to face the media approbation for selective, targeted, pilot open data releases.