The economic value of open data to Australia

This morning I attended the breakfast launch of the Open for Business: How Open Data Can Help Achieve the G20 Growth Target report.

The report was written by Nicholas Gruen (former chair of the Gov 2.0 Taskforce) and his team from Lateral Economics, with support from Victoria University and commissioned by the Omidyar Network (the not-for-profit organisation established by eBay's founder).

It makes a compelling economic case for open data, estimating aggregate direct and indirect value for Australia was in excess of $15 billion per year. This was based on estimating the economic value of open data just across the G20's seven priority areas, which I've provided below as a table.

G20 priority area	open data value  per annum to Australia
Anti-corruption	$1.5 billion
Employment	$3.4 billion
Energy	$1.7 billion
Fiscal and Monetary policy	$3.6 billion
Infrastructure	$3.6 billion
Trade	$1.6 billion

Relative progress on open government data areas
Source: http://theodi.github.io/open-data-barometer-viz

The report suggested that Australia was still doing very well in the open data space, ranked 3rd amongst G20 nations (7th or 8th overall globally) - but that there was still much room for improvement and learning from other countries.

During the presentation Martin Tisne from the Omidyar Network said that Mexico and India had demonstrated leadership in opening up education data, while South Africa had taken great steps with open budgetary data - making the point that different nations have excelled in different aspects of openness, but few had demonstrated consistent strength across all aspects of open data.

The report included a great deal of detail on different areas in which governments could achieve economic value through open data - and also highlighted that the cost of realising these benefits could be up to a third of the value received, giving a clear signal of the need for government to invest in this area, not simply allow it to thrive or die on its own with no support.

Both Nicholas Gruen and Tony Shepherd, head of the Commission of Audit, highlighted the need for senior Ministerial leadership, and Gruen noted that no Australian Prime Minister had ever been a passionate supporter of open data, to Australia's detriment where the US and UK had significant political as well as public service leadership for openness.

The presentation also highlighted some of the current pitfalls for entrepreneurs seeking to take advantage of open data while there was no consistent commitment to its release.

Gruen illustrated this point by discussing APSjobs.info, a site created at a past GovHack, that mashes up data on public servant movements from APSJobs.gov.au. He said there was clear added value realised via APSJobs.info, which could be a useful reference tool for recruiters and agencies seeking to identify the best talent.

The report states that:

APSJobs.info's business model was predicated on its development of successful technical methods to 'scrape' the data from pseudo-print PDFs. However frequent changes to the formatting and layout of these files meant continuous re-development of the PDF conversion software to continue to access and add value to the data. The skills required to perform such work (data-mining and text-analysis) are in great demand, and the cost of frequently using such resources exceeded the benefits to Pivotal Analytics.

APSJobs.Info is now defunct - a casualty of government inconsistency.

The Open for Business: How Open Data Can Help Achieve the G20 Growth Target report is highly material in establishing the value of open data to governments and the steps they need to take to realise the economic value that could result from greater release of reusable data.

Hopefully Australian governments will continue to build their commitments to open data and we'll see some of this value filtering back into our economic.

Read full post...

View summary...

Mirror, mirror - can modern societies survive looking into the internet and seeing their own reflections?

One of the most major challenges for governments and societies around the world today is the rapidly declining trust in politicians, institutions and governance systems.

I'm willing to make the claim that politicians today are no more corrupt, self-serving or beholden to special interests than politicians were fifty, a hundred or even a thousand years ago.

Regardless of the political system in place, it takes hard work, compromise, negotiation and a willingness to be pragmatic and flexible in one's values and ethics to achieve high office. Even the cleanest and most ethical politicians have to deal with people with different standards, and must find ways to accommodate diverse views in order to achieve great ends.

I'm also willing to say that it isn't the economic situation. The world has faced huge financial strains in the past, and while governments may rise and fall (as Greece's has done), or constitutions may be redrawn (as in Iceland) the underlying governance systems have rarely changed as a result.

No country changed from a democracy to something else as a result of the GFC, and none changed from 'something else' to a democracy as a result of it either.

Maybe its our institutions - are government agencies, courts, armies and police failing in their jobs?

Well no, to a large degree public services around the world remain highly capable of delivering the services they are required to deliver. Of course there's always room for improvement, and successive governments have changed the configuration and goals of public services - but the core capability remains largely intact, at least to-date.

So what do I believe is causing the rapid decline in trust, potentially the greatest threat facing governments today?

It's the rise of the internet and supporting technologies.

The internet - particularly social media - has become a mirror that society cannot escape.

Every action and decision taken by elected politicians is now almost instantly communicated, critiqued and analysed by thousands or millions of people - many looking for the slightest sign of deviation from a past statement, position or decision.

Websites have sprung up to collate and consider every public political statement - and some of their private ones too. Tools such as Polliwoops ensure that even comments that politicians later delete remain accessible - Google's cache is another way to find deleted media statements made in a politician's early career, and even government-funded platforms such as PANDORA are repositories of deleted Prime Ministerial speeches (removed so as not to 'confuse' voters).

Old-school politicians, who chameleon-like reflect the views of the physical audience in front of them, are potentially finding instant mass communication the greatest challenge. Gone are the days where a politicians could travel from event to event, each time subtly adjusting their message to appeal to the audience at hand.

Now every inconsistency, weasel word, off-the-cuff remark and error of judgement by elected politicians (as well as many unelected ones and public figures) is captured, shared and discussed online.

And that's just the facts - the internet is also full of commentary, predictions, suppositions and lies about political leaders, some benign, some actively trying to understand or help and some trying to bring them down.

The problem isn't that our politicians are flawed, our economies failing and our institutions corrupt - many countries have been there before and survived, even thrived.

The problem is that society is now seeing both truth and fiction in greater quantity and detail than ever before - the mirror of the internet is always on, and no society can choose to look away from it.

It is easier to believe politicians when all we see are the good things they do. It is easier to believe in a system of government when we don't think about the deals done to support it.

However people today now expect some form of purity from their political leaders that has never been achieved in history - they want leaders better than they are, without flaws, with no need to compromise and who appear mystically in a leadership role without years of learning their craft and making mistakes on the way.

How would the political leaders of the past have been effective in today's mirror society?

Would any of them successfully been able to face the mirror without flinching?

Can any aspiring political leaders now make it into parliament without casting a mixed reflection?

Even a political leader who is pure and uncompromised in every way will find that hostile elements - rivals, other parties, lobby groups, disruptive citizens - attempt to distort their reflection into a funhouse caricature that the public reject.

I wonder whether our system of government can survive the relentless focus of this mirror. Whether we'll attract competent politicians, see ongoing mass civil disobedience or simply lose all trust and faith in the people who put themselves forward to be elected and the system they are elected into.

Will we learn to accept that all political systems, their institutions, leaders and decisions, are flawed under sufficient scrutiny. That everyone has something they're not proud of, or can be distorted into inappropriateness, in their past, and accept that our leaders and system are what they are - faults and all?

Will we demand systemic change - that our electoral systems are reformed and the people inhabiting the current system be removed, possibly even tried?

Or will we simply opt-out. Treat politics and our governments as an annoyance that we evade wherever possible and only engage when we have to - leaving us at the mercy of politicians who choose to use their powers for actions not in the interest of the public?

I really don't know which course will be taken in many countries around the world, but I do expect to see many more governments fall over the next twenty years, hollowed out through loss of talent and put into the hands of petty tyrants, or collapsing under their own weight.

However what I hope to see are governments and societies finding ways to truly look at themselves in the mirror. To rationalise that while they can no longer persist with the myth that they are the 'finest of them all' they're actually not that bad looking - despite the wrinkles and scars.

I also hope to see governments recognise that they need to experiment more at the core, not simply around the edges - reverse trends towards political functionaries being the majority of elected members and institute practices which turn parliaments back into the servants of their societies, rather than their masters.

This will take real political courage and will to change.

Ironically political courage may be one thing that increases as the as the mirror's reflections become more and more defined.

Soon anyone seeking to enter politics will need to have courage simply to put themselves forward for election, because if there is a single blemish on their reflection they will be hounded relentlessly.

Standing up to that scrutiny, displeasure, disappointment and abuse in order to make a difference through public office will take enormous public courage.

Ultimately, however, societies will need to find a new accommodations. We will need to accept that there's as many pure politicians as there are unicorns, and when we look into the mirror of the internet the reflection we see isn't solely that of the politicians we elect, it is a reflection of our entire society and the choices we have made to create it.

If we want to feel something other than disappointment or horror when we look at our reflection, our society's reflection, in the internet, then we will have to consciously, personally and collectively, make the decisions that will allow us to gaze on it with pride.

Read full post...

View summary...

Only four weeks to GovHack - register now!

GovHack, the largest open data competition in Australia, is only four weeks away.

With over $70,000 in national prizes, and local prizes in each of the 11 locations around the country, there's not much time left to join the already over 1,000 registered participants to mash-up government data.

Register now at www.govhack.org/register-2014/

Read full post...

View summary...

It's nice to see government agencies share with each other

One of the most frustrating and, I think, silliest things I found when working in Australian government agencies was how almost every department, agency and statutory body developed almost all of its own policies, procedures, software and tools.

There was often 'undercover' sharing - where people in agencies would ask their colleagues in others for copies of their whatever policy, so they could craft one just like it - however there was no central repository where public servants could go and browse standard templated policy documents or access software code developed by other agencies to resolve certain common issues.

At one stage I actively looked at building a research directory for government - either from within an agency, or as a third-party site - where public servants could list the research their agencies had undertaken and the research they needed to access or undertake, so that it could be effectively shared between departments, saving money in re-running research and informing policy decisions.

This attempt didn't get off the ground as senior management strongly felt they had no obligation to share information on the research work conducted with 'THEIR' public dollars with other agencies - even the fact that they'd undertaken it in the first place.

Fortunately most of those senior managers have now retired (literally), and the new crop coming through are realising that, cash and time constrained as they are, that fighting over which agency 'owns' a specific policy, custom software or research that they commissioned gets in the way of productivity, efficiency and cost-effectiveness.

One of the outcomes has been the latest Australian Government whole-of-government website, GovShare.

GovShare had been talked about for nearly five years and has now finally arrived as a central location for agencies to share their work with other agencies to build standardisation and save money across the public service.

As the About page states, GovShare has been designed to support and promote collaboration across the Australian Public Service, and as an online resource it has been provided to APS agencies and their staff to:

• publish, discover and access a broad range of artefacts used within the APS, such as frameworks, guidelines, policies, standards, architectural models, open source software and a host of other ICT and business artefacts; 
• explore APS ICT services and solutions through the Agency Online Services Database and Agency Solutions Database; 
• find skilled people across the APS with expertise in particular fields or products; and 
• contribute to online discussions using the forum.

This type of sharing helps save money and time in government. It allows agencies to collaborate to design the best possible policies, guidelines and software and then customise it if needed for their specific needs.

While the benefits of the site won't necessarily be clearly visible outside of government circles, the efficiencies it will support will help agencies focus their resources on achieving the goals of government rather than on endlessly recreating the policies already in place elsewhere in government.

The site already contains nearly 2,000 'artefacts' for review, reuse and adaptation, and hopefully over the next few years this will swell as more agencies become active contributors as well as takers from the site.

It will also hopefully expand beyond its ICT roots into other business areas - allowing agencies to share standard communication strategy templates, HR policies, procurement guidebooks and financial guidelines - tools and resources that help public servants understand and abide by the rules the government sets in these areas.

Hopefully the site will expand beyond federal government as well, bringing state and local into the fold. These governments often face the same challenges, often with fewer resources, and GovShare could have a large role to play in reducing costs at all levels of government in Australia.

Ultimately it would be wonderful to see 'packages' of policies and guidelines that a newly created agency or statutory body can simply pick up, adapt and use for their operations.

This would be similar to the 'agency on a USB stick' concept that I've been talking about for several years around a set of software platforms and settings that would allow an agency to put in place a solid set of operational systems in a very short time.

However it is, as yet, early days for GovShare. Its success relies on three things, ongoing support from the department hosting it (Finance), active participation by agencies in 'gifting' their work to a common store for other agencies to access and the political will and nous to not kill the program before it bears fruit.

I hope Govshare will succeed, and think the omens are good. Its journey over the next five years will be interesting to watch.

Read full post...

View summary...

For a truly open parliament, look to the UK

The UK Government has just released the alpha version of data.parliament.uk - - a site designed to host open data on the UK Parliament and its proceedings.

(An Alpha for those who don't understand the term is a first release of a software product that is usually tested only by the developers).

The alpha release of www.data.parliament.uk

The site represents a new frontier for parliaments - making information on the discussions and deliberations of politicians far more actively available in ways that it can be reused to inform the public.

The site is sparse at the moment - only a few datasets on parliamentary questions, on how MPs have voted (something we don't publish in Australia despite it being common data available in North America and Europe) and briefing papers.

It will be interesting to see how it grows in form, function and data and whether it becomes a truly useful source for citizens.

Read full post...

View summary...