I was reading through the comments on my post on Make government data freely available, and found Gav's (RedIguana) comments particularly thought provoking.
...Government also must create national standards for data, for example in terms of controlled vocabularies, etc, to ensure that data can in fact be aggregated nationally. It is no good if each council classifies roads differently, to actually produce a national aggregated dataset of roads with wildly differing definitions.This comment, explored in more depth in Gav's blog, raises an important consideration for government as a whole.
Who in government is responsible for defining consistently usable data standards?
I don't think this is clear at the moment.
I'm not only talking about big picture data standards - such as national inflation, unemployment and population distribution (for which the ABS does an excellent job).
I'm talking about data standards all the way down through state to local levels, such as Gav's example of road hierarchy classification and areas such as geological data.
Currently it seems that every Federal department, every state and even every local council is collecting information on their local environment and areas of responsibility.
However there is no mechanism for combining and making sense of all this data within a single interface.
Looking into the near future I believe that nations that organise their data at all levels to agreed standards, those I call the Data-Haves, will have a significant competitive advantage against those that keep data siloed and on inconsistent baselines, those I call the Data-Have-Nots.
In the Data-Have nations data at all levels can be shared freely online, making it possible to build information maps, cross-reference trends and discover connections and causal events that would otherwise remain concealed.
These informational benefits will allow Data-Have scientists, business people and governments to test and prove new governance, management and scientific theories - leading to discoveries that improve the welfare of their people and their long-term economic success.
Meanwhile the Data-Have-Not nations, who are not sharing data openly, will be rapidly left behind, socially, economically and politically.
This isn't a new scenario - there's a strong case that the Soviet Union, with its rigid and siloed political and economic systems, was unable to survive in the global marketplace because it could not effectively share information between internal groups. It therefore fell further and further behind in an information sharing world.
This leaves me with one question.
How do we help ensure that Australia becomes a Data-Have nation rather than a Data-Have-Not?
Any suggestions?