Weird and wonderful uses for open data - visualising 250 million protests and mapping electoral preferences

One of the interesting aspects about open data is how creatively it can be used to generate new insights, identify patterns and make information easier to absorb.

Yesterday I encountered two separate visualisations, designed on opposite sides of the world, which illustrated this creativity in very different ways.

First was the animated visualisation of 250 million protests across the world from 1979 to 2013 (see below).

Based on Global Database of Events, Language, and Tone (GDELT) data, John Beieler, a Penn State doctoral candidate, has created a visual feast that busts myths about the decline in physical protests as people move online and exposes the rising concerns people have around the world.

Imagine further encoding this data by protest topic and displaying trends of popular issues in different countries or states, or looking at the locations of protests in more detail to identify 'hot spots' - in fact John has done part of this work already, as can be read about in his blog (http://johnbeieler.org/)


Second is the splendid Senate preferences map for the 2013 Australian Government election, developed by Peter Neish from Melbourne.

Developed again from public information, this is the first time I have ever seen a map detailing the flow of preferences between political parties, and it illustrates some very interesting patterns.

The image below is of NSW Senate candidates, and thus is the most complex of the states, but shows how this type of information can be visualised in ways never before possible by citizens without the involvement of traditional media or large organisations.

For visualisations of all states and territories, visit Peter's site at http://peterneish.github.io/preferences/

These types of open data visualisation lend themselves to a change in the way the community communicates and offer both an opportunity and a threat to established interests.

Governments and other organisations who grasp the power of data visualisation will be able to cut through much of the chatter and complexity of data to communicate more clearly to the community, whereas agencies and companies who hang back, using complex text and tables, will increasingly find themselves gazumped by those able to present their stories in more visual and understandable forms.

We're beginning to see some government agencies make good use of visualisations and animation, I hope in the near future that more will consider using more than words to convey meaning. 

Read full post...

View summary...

Enter the Queensland Premier's Award for Open Data

The Queensland Government is currently running their first Premier's Award for Open Data, a competition designed to encourage and support the use of data released by the government and raise awareness of how it can add value to the community.

Open data competitions are no longer new for Australia, however it is great to see that Queensland hasn't put limitations on where entrants can come from - unlike previous state open data competitions.

This is a major step forward as it allows the Queensland Government to tap into good ideas from anywhere in the world, rather than limiting them to a geographic area.

The competition only offered $20,000 in prizes spread across four categories ($5,000 in each), howeve received a major boost with support from Microsoft, who is contributing an additional prize, which I value at close to $100,000. This Microsoft Start-Up Accelerator Award will be awarded to the team who develops the most innovative concept with the best start-up potential and will include: · five Nokia Lumia 920s, five Asus VivTab tablets, one four-month course at Founder Institute for the nominated team founder to attend from February 2014 and a maximum of $60,000 worth of Windows Azure for a maximum period of two years.

This size of prize pool is important for open data competitions given the efforts that teams and individuals put into development. I generally also suggest splitting prizes into a range of awards to give more teams the opportunity to win and thereby increasing participation and engagement.

Overall this looks like a good competition and it will be interesting to see the level and range of entries and the winners.

The support offered beyond the competition to entrants will also be interesting to observe, as open data competitions both in Australia and overseas have often suffered from being 'flashs in the pan' - with most apps and services created being abandoned after the end of the competition process.

To find out more or enter the competition, visit data.qld.gov.au/data-event/premiers-awards

Read full post...

View summary...

Is it possible to deliver a government agency's standard IT systems on a single USB?

CSIR Mk 1 with Hollerith
equipment, Sydney 1952
Source: Museum Victoria

The Australian government was one of the earliest adopters of computers and computerisation.

CSIRAC (or CSIR Mk1), the first computer in Australia (and now the oldest surviving first-generation electronic computer), was used by scientists within CSIRO, by the Snowy Mountains Hydro Electric Authority and various university and government departments and agencies between 1949 and 1964 to make sense of 'big data' (for the time) which would have taken years to analyse by hand.

As the fifth stored program computer in the world, CSIRAC programmers could write their programs on punch tapes, check them one step at a time, and store them in the computer to be run again and again.

While computers have gotten a lot smaller, faster and efficient, they still use a similar programming approach to CSIRAC. Programs (software) are loaded into their memory and may then be accessed and run many times.

Of course modern computers use different storage mediums and can store and execute many programs at the same time.

Every government agency has an IT architecture made up of hundreds, if not thousands, of different programs - some run on a mainframe computer, others on desktop computers and still more on servers which allow staff to access the programs remotely from their desktop, laptop or even mobile platforms.

It is a very complex process to manage an agency's IT architecture - some programs may not 'play nice' with others, some may be twenty or more years old and require special hardware and maintenance to keep them operating.

Setting up a new agency can be an even more complex process. Often agencies are 'spawned' from existing departments and 'borrow' much of their IT infrastructure - the software required to run everything from payroll and HR to manage contracts, projects, compliance, Ministerial correspondence and provide the desktop applications required by staff to do their jobs.

Even more complex is the process of combining disparate agencies into a new department. This can require blending two or more sets of software programs into a single solution, with all the data migration and management issues this entails - not to mention addressing security considerations, staff training and avoiding long outages or data loss.

This is where my concept of 'government on USB' comes in.

Why not develop all the core software that a government agency needs to operate as open source shareable software and release it for other government agencies to reuse?

Using this approach it is possible that when a government dictates that a new agency must be formed that the CIO simply pulls out his 'Government Agency USB' and uploads all the required operational software as a complete agency package.

Potentially, via this method, a new agency could have all its core ICT systems in place and operating in days, if not hours.

This approach might seem farfetched, however we're already heading in that direction due to a couple of trends.

Today much of the software an agency needs to run its operations is available through SAAS (Software as a Service) or as cloud-based services - which both basically means that software is stored offsite, maintained by a specialist company and simply accessed and used as needed by an agency - provided they are confident of the security levels.

We're also seeing more and more of the software 'building blocks' of organisations becoming available in open source forms which can be downloaded, adjusted as required by an agency and used, either hosted internally or via a SAAS or cloud provided.

The US has actively been developing and releasing software in open source formats for other governments to use, as has the UK and a few other governments around the world. This offers massive national and international efficiencies for governments who can reuse rather than build or buy software.

The next step is for a government to audit the core systems required to establish a new agency and develop a standard IT Architecture that can be applied for any new agency (with room for specialised modules for unique functions). Then, by selecting from existing open source programs and potentially writing additional services, a government could put together a 'flatpack' IT architecture that any new agency could adopt quickly and easily.

If all the software in this 'flatpack' were open source, it could be easily improved and adjusted over time to meet changing legislative and operational requirements and to integrate ongoing improvements and enhancements.

Then once agencies have adopted this common 'flatpack' of software, it would be significantly easier and cheaper to merge agencies, as they would already be operating in a similar and interchangeable way.

Moving all of government across to this approach would take quite a few years - it's not achievable in a single term - however it would provide ultimately for a 'government on USB'.

This also has implications across the developing world and for newly formed countries, where their government agencies and institutions can suffer from a lack of experience, expertise and money to build the robust IT architecture needed for modern nations.

In the scenario I've described, a new or developing government could simply plug in the 'government on USB' into an agency's systems and establish a sophisticated IT environment to underpin governance in a very short period of time.

Is this simply an unattainable pipedream?

Some may scoff at the notion, however there are many people around the world working on parts of the 'government on USB' model today - albeit many may not be thinking about the bigger picture.

Much of the software required for a government agency is already available in open source form, from HR and financial management systems to desktop applications. It simply hasn't been linked together with a single set-up process.

To explore the concept it would take a government willing to innovate, investing resources and money.

This would be used to model the software requirements of an agency, identify where open source solutions exist (or existing solutions can be modified) and write new open source software where necessary.

Next there would be the need to ensure the solution is secure and to write a single set-up approach that makes it easy for a CIO to roll out the solution quickly.

This may not ultimately be possible or cost-effective, but given the cost of IT architecture changes today when creating, merging or updating agencies, surely it is worth considering.

Read full post...

View summary...

From Gov 2.0 to GovInnovate - expanding the agenda

I'm pleased to note that CEBIT, whose Gov 2.0 Conference has been a great event over the last few years, has recognised the growing innovation agenda in government and broadened this annual conference into GovInnovate.

Now including Gov 2.0, Cyber Security, Service Design and mobile Government (mGov) streams, the GovInnovate conference looks like it will retain a leading position amidst Australian events aimed at government innovators and leaders.

I'll be returning to speak at the conference after an absence of a few years due to other commitments, and I strongly recommend that people involved in government who are interested in the streams above consider whether they can attend.

GovInnovate is being held from 26-28 November in Canberra and more information is available at its website: www.cebit.com.au/govinnovate

Read full post...

View summary...

Social Media specifications guide

One challenge organisations may face with social media is designing their account pages to reflect their common look.

I've seen many organisations place graphics poorly - stretched logos and unintentionally pixelated images - due to not having the specifications to hand when instructing a graphic designer.

Fortunately someone has come up with the below very useful specifications sheet for major social networks.

While this is a 'point in time' resource, as social networks regularly change their designs, it provides a starting point that should help organisations design their account pages to platform constraints.

Read full post...

View summary...