Suggestions for governments stepping into open data

I've been completing a survey for the Spatial Industries Business Association (SIBA) related to the Queensland Government's open data initiative, where one of the questions asked Can you list or describe any learnings that would be useful in Queensland?

I've provided a number of my thoughts on this topic, having closely observed open data initiatives by government over the last five years, and written periodically on the topic myself, such as:

• Mapping open data site generations
• Is there really an open data El Dorado?
• How should governments educate agencies about open data?
• Infographics: How does Australia compare on government open data released?
• From open data to useful data
• Is it easy for non-programmers to reuse government open data?

To share the thoughts I placed in the survey more broadly - for any value they have for other jurisdictions - I've included them below:

• Data released in unusable formats is less useful - it is important to mandate standards within government to define what is open data and how it should be released and educate broadly within agencies that collect and release data.
• Need to transform end-to-end data process. Often data is unusable due to poor collection or collation methods or due to contractual terms which limit use. To ensure data can be released in an open format, the entire process may require reinvention.
• Open data is a tool, not a solution and is only a starting point. Much data remains difficult to use, even when open, as communities and organisations don't have the skills to extract value from it. There needs to be an ongoing focus on demonstrating and facilitating how value can be derived from data, involving hack events, case studies and the integration of easy-to-use analysis tools into the data store to broaden the user pool and the economic and social value. Some consideration should be given to integrating the use and analysis of open data into school work within curriculum frameworks.
• Data needs to be publicly organised in ways which make sense to its users, rather than to the government agencies releasing it. There is a tendency for governments to organise data like they organise their websites - into a hierarchy that reflects their organisational structures, rather than how users interact with government. Note that the 'behind the scenes' hierarchy can still reflect organisational bias, but the public hierarchy should work for the users over the contributors.
• Provide methods for the community to improve and supplement the open data, not simply request it. There are many ways in which communities can add value to government data, through independent data sets and correcting erroneous information. This needs to be supported in a managed way.
• Integrate local with state based data - aka include council and independent data into the data store, don't keep it state only. There's a lot of value in integrating datasets, however this can be difficult for non-programmers when last datasets are stored in different formats in different systems.
• Mandate data champions in every agency, or via a centre of expertise, who are responsible for educating and supporting agency senior and line management to adapt their end-to-end data processes to favour and support open release.
• Coordinate data efforts across jurisdictions (starting with states and working upwards), using the approach as a way to standardise on methods of data collection, analysis and reporting so that it becomes possible to compare open data apples with apples. Many data sets are far more valuable across jurisdictions and comparisons help both agencies and the public understand which approaches are working better and why - helping improve policy over time.
• Legislate to prevent politicians or agencies withholding or delaying data releases due to fear of embarrassment. It is better to be embarrassed and improve outcomes than for it to come out later that government withheld data to protect itself while harming citizen interests - this does long-term damage to the reputation of governments and politicians.
• Involve industry and the community from the beginning of the open data journey. This involves educating them on open data, what it is and the value it can create, as well as in an ongoing oversight role so they share ownership of the process and are more inclined to actively use data.
• Maintain an active schedule of data release and activities. Open data sites can become graveyards of old data and declining use without constant injections of content to prompt re-engagement. Different data is valuable to different groups, so having a release schedule (publicly published if possible) provides opportunities to re-engage groups as data valuable to them is released.

Read full post...

View summary...

A look into the mind of John Miri

Yesterday I had the opportunity to catch up with John Miri, the former Deputy to the State CTO for Texas, following his presentation at Sitecore's Digital Citizen Engagement event in Canberra.

John is also presenting in Melbourne today, and in Perth next week.

The first thing that struck me about John is how different he is from the stereotype of a government IT professional.

Personable, approachable and possibly the only tea drinker left in the US, John was trained in physics but pursued a career in IT after it was pointed out to him that there were more career opportunities in IT than science.

He came late to government, spending a number of years founding and working in early-stage start-ups before making the leap to public service in 2005, as Director of E-Government and Web Services for the State of Texas, reporting directly to the State CTO.

In that role John was responsible for shepherding the TexasOnline.com program (now texas.gov), implementing 829 new online services, and leading to 83 million citizen financial transactions, with more than $5 billion online revenue.

John is now Editor-In-Chief for the Center for Digital Government and principle of Bluewater Technology Services, a technology consulting company.

John believes that government is at an interesting crossroads - still applying governance principles from the 19th and 20th centuries, while trying to rapidly adapt to the 21st.

He talked to me about the vision that the founders of the US had for their nation, a participatory democracy where citizen involvement in governance didn't end with their vote, where citizens were empowered and supported to contribute to civic life.

John says that with today's technologies it is now possible for societies to realise this kind of vision - to reshape governments to be more participatory without losing the strong institutions and traditions that make democracy possible.

We discussed how government institutions are designed to maintain the status quo, the value of bureaucratic processes in maintaining stable, safe and secure societies, however these strengths can also become weaknesses when politicians and public servants stop asking 'what is the goal of government' and focus on repeating the processes in government - resisting change from within or without.

John asked the question 'what is the role of citizens in delivering government services?' saying that governments need to begin considering citizens as stakeholders and engaging them in the same way agencies engage expert panels, companies and lobby groups.

He also commented on how government's tendency to silo problems and attempt to solve them individually is failing - today's problems are complex and multifaceted, crossing traditional ministerial portfolios and requiring complex and collaborative solutions.

John argued that the current structures in government are poorly suited to solving these problems, and our reliance on subject matter experts - rather than problem solving experts - meant that many problems are being seen through specific lenses and perspectives that made them difficult, if not impossible to solve.

He gave the example of US state road taxes on petrol - designed to cover the cost of maintaining roads. As cars have improved their efficiency, travelling far further - and doing more road damage - on the same amount of petrol, the gap between the funds the tax raise and the maintenance cost has been growing.

John asked a group of road policy experts in government about this issue, and their response was that the solution was simple - raise road taxes. His comment to me was that while the experts may think this was simple to do, it wasn't simple to get tax increases through political processes or sell their value to the public - more participatory processes and more innovative solutions were needed for the long-term.

He said that the increasing size of many of the complex problems that face government today mean that the odds are in the favour of those who advocate for more participative government and Government 2.0.

As traditional approaches to problem solving fail, due to agency silos, expert bias and limited community involvement, governments will be forced to look towards more innovative solutions - involving citizens and reshaping bureaucratic processes.

John also said that digital was an opportunity for governments to do more than simply replicate their business processes online. Rather than mimicing or tweaking paper-based workflows and forms for online use, agencies should use the opportunity to reinvent their business processes.

This involves questioning every assumption - what information is needed, when and how is it needed, how should it be stored, actioned and how should citizens be informed and engaged throughout the entire process.

John says that agencies that simply replicate existing processes online are unlikely to realise the full benefits in cost-savings, accurate completion and citizen satisfaction - an automated mess is still a mess.

He says there are no shortage of example of how technology has transformed business processes and the situation is no different in government. If agencies and politicians can focus on the goals and outcomes they are working towards, rather than bury themselves in repeating the same processes they've used for decades.

John also suggested that a reinvention approach allows room for innovations in how government services are delivered. For example as train timetables become digitalised, why should trains runs at the same time every day?

Would it be possible to adjust train schedules on a flexible basis, managing it like an electricity grid, based on the number of travellers and communicated via electronic messaging boards.

He also asked whether child protection services could be radically reinvented to provide 24/7 access to case workers for children in need. Could a single contact phone number, SMS and email address be used to route case workers to where they are needed most, using GPS and mobile devices to ensure they had the information they needed at all times to maximise their efficiency and protect more children from harm.

In conclusion John was of the view that egovernment, Government 2.0 and the rise of digital citizens who wish greater participation in the democratic process, should not be seen as a threat to traditional democratic institutions - we're not trying to add a third house of parliament.

Instead he said that these movements and emerging technologies should be embraced as a way to realise the original intent and goals of government - to represent, serve and involve citizens. 

Read full post...

View summary...

GitHub launches portal illustrating examples of government-citizen collaboration on open data, open source and open government

In an exciting and useful development, GitHub, the world's best known portal for collaborative software development, has launched a portal illustrating how governments and citizens have worked together to deliver better outcomes.

Now live at government.github.com the portal provides some great examples of GitHub projects that have saved government money and time and delivered better outcomes through citizen participation.

The portal also links to GitHub hosted open civic projects that governments can reuse - at no charge - to enhance what they provide to citizens.

If you've been having trouble explaining to senior management or IT teams how collaborating on software and open data with citizens can deliver better outcomes, then this is a great source to demonstrate how other agencies have reached success.

And, in case you were wondering, policies and laws can be open sourced as well - all of Germany's laws are available through GitHub, ready to be forked, edited and reused by other jurisdictions around the world. Learn more from the OKFN blog. 

Read full post...

View summary...

Has government found its feet in social media?

Earlier today I gave a presentation to the IABC's Canberra chapter on the use of social media within the Australian Government.

The slide deck I used is below, and fairly well carries my point - that government has indeed found its feet in social media, however there's still uneven ground waiting to trip it up if it missteps.

I'm interested as well in whether others agree with my assessment of the 18 Australian Government departments into social media leaders and followers (slide 17).

Has government found its feet in social media? from Craig Thomler

Read full post...

View summary...

The road to public sector IT hell may not be paved with intentions at all

Something that scares me enormously is the house of cards that many (if not most) governments have built with their IT systems.

It can be witnessed every time government agencies get 'MOGed' - Machinery of Government changes where parts of agencies are shifted to other agencies to meet the latest political whim.

In these cases it's not simply a matter of moving tens, hundreds or even thousands of public servants to new offices - in fact in many cases they may not move at all - it is about extracting them from the secure environment, software and network systems of one agency and connecting them (including all their historical records, emails and files) to the network and software of another.

This is a hugely complex and increasingly expensive exercise that can have an enormous productivity and cost hit each time it occurs.

Why is it complex and expensive? Because every agency uses different systems - or different versions of systems - and agencies are now so wedded to these systems after a purchase decision many years earlier that, even though senior bureaucrats recognise the issue, they can not address it without a complete (expensive and time-consuming) overhaul of how government runs its information technology.

Another example is eTax. While I have a great deal of praise for eTax, and it has been very successful by most measures, when the system was originally procured and built it was done in such a way that limited it to the IBM-PC platform. Certainly no-one can blame the ATO for not foreseeing the rise of Apple or the arrival of smartphones and tablets - however the decisions made at the time locked the system into a single platform, which has caused significant pain over the years.

Other examples include the Department of Finance and Deregulation's choice of a document management system as a Web Content Management System for www.australia.gov.au, an entirely appropriate decision at the time based on their well-governed procurement approach, but which led to delays and cost blowouts, constraining the site from what it could have become.

A better known example would be the failure of the Queensland Health payroll system several years ago, where an enquiry is still ongoing. It even has its own website - www.healthpayrollinquiry.qld.gov.au

Indeed, there are hundreds of examples both big and small, where this has occurred - a decision has been taken with the best possible knowledge at the time, or small incremental decisions have been taken over time - all for the right reasons - which have inadvertantly led into blind alleys or very expensive remedial work years later.

And lest you think this is an issue only for the public sector, consider the disaster that was Telstra's bill payment system, the issues our largest banks have had keeping their systems operating, or Virgin's booking system.

With the pace of change accelerating and the increasing limits on public sector employment, the likelihood is that these types of issue will continue to grow and plague IT, becoming even more widespread and expensive.

Agencies could increasingly find themselves trapped into slow and inefficient systems, restricting staff productivity and absorbing more and more of their resources to maintain, with no funds to 'jump tracks' to more future-proofed solutions.

This can even affect the performance of elected governments - who may be forced to change their policies to fit IT limitations. I am already aware of government initiatives that have had to be abandoned (never having seen the light of day) not because they were bad ideas but because the IT constraints in government make them impossible to cost-effectively deliver.

This isn't the fault of public servants or of politicians - seeing that far into the future simply isn't possible anymore. Technology isn't progress linearly and the accelerating rate of change means left-field technologies can appear and radically transform peoples' expectations and strain existing IT systems within a few years (remember the iPhone).

There's many more of these technologies emerging around us. For example 3D printers, capable of printing anything from kitchen utensils to medical devices to firearms, disintermediating physical manufacturers, opening a new front in the ownership of intellectual property and providing access to deadly weapons. There's also unmanned aerial vehicles (UAVs), drones that are capable of live-streaming video, or even carrying weapons, that can be bought online for a few hundred dollars and flown with limited chance of detection by individuals or corporations.

Many others technologies from Google Goggles to driverless cars are in development and could, in increasingly shorter timeframes, radically transform societies.

So when government agencies are still struggling to manage and maintain their legacy green-screen mainframe systems, out-dated (insecure and unsupported) web browsers, where they are locked into increasingly expensive proprietary technologies (due to the cost and resourcing required to migrate - even changing email systems can cost our largest agencies $100 million or more), what are they to do?

There's little time for innovation or for thinking of consequences - the majority of resources in an agency's IT team are committed to maintenance and quick patches on existing solutions.

The likely outcome over time is that we'll start to see more catastrophic IT failures - particularly across the most complex and most essential systems - such as welfare, payroll and grants management.

So how do we fix this? How do we break the cycle before the cycle breaks us?

There's no simply solution, but there's fortunately some trends which work for government agencies facing this challenge - if they're prepared to consider them.

A big area is open source software, which is increasingly being used by agencies in a variety of ways. While open source can run into the same issues as proprietary software, a platform with a large and diverse group of users can combine their IT assets to ensure the system is more useful to agencies and more rapidly updated as the world around it changes.

Another area is cloud-based solutions, which allow a government to more rapidly reconfigure itself to meet the needs of political masters. When software is independent from computer systems and there's a government-wide secure environment which can host software approved for use it can be far faster and cheaper for people moving agencies to retain the files and applications they require.

There's open data - which when made available in machine-readable formats liberates the data from proprietary systems and simplifies how it may be discovered and reused by other agencies (as well as the public).

These trends do not allow governments to replace all their existing systems - however they allow agencies to contain the problem to critical systems, which allowing all other services to be done 'in the cloud'. Imagine, a single email system and intranet across government. A web-based suite of office tools, graphic design tools, finance and HR tools - which can be managed centrally within a government, leaving agency IT teams to focus on the unique systems they can't share.

What does this vision take? Intention, planning and choice.

Governments that fail to proactively and intentionally plan their futures, who simply live on autopilot, will inevitable crash - not today, not tomorrow, maybe not in five years, but eventually - and the damage that their crashes will cause may take decades to recover from.

So for agencies who see themselves as being a continuous entity, with an existence that will exist as long as the state they serve, it is imperative that they plan intentionally, that they engage their Ministers and all their staff in understanding and addressing this issue.

It is not good intentions that will cause agency IT to fail, it is the lack of intention, and that is highly addressable.

CORRECTION: I have been advised by John Sheridan, the Australian Government CTO, there was no cost-overrun on australia.gov.au, it was a fixed price contract.

Read full post...

View summary...