Use of government open data creeping into universities

In September I was honoured to be invited to speak at the first Best Practice in Data Journalism seminar for Australia at Melbourne University, hosted by the Centre for Advancing Journalism.

The event included many of Australia's leading data journalists, together with academics and other representatives working in the data space.

At the event I was glad to see that a number of Masters students in the university were using government open data to create visualisations and derive insights.

Richard Sinnott has now published a public list of some of these projects in the Asia-Pacific data journalism and data visualisation group at LinkedIn, which I've included below.

These have been been developed in ten weeks or less and are works in progress, so consider them appropriately.

Federal government open data mash-up:
http://130.56.249.15/proj/

Visualising Victorian lobbyists, clients, companies, donors: http://115.146.85.10/proj/index.php/lobbyist/graph
(wait for it to load as there is a large complex graph).
Hopefully straightforward to drive/use...

Using Twitter for Melbourne transport congestion:
http://115.146.94.12:8080/web/statistic_date.html
(final scenarios coming soon)

Using Twitter for World Cup 2014:
http://115.146.93.143:8080/TwitterAnalysis/index.html
(can we identify events in games through Twitter sentiment) - pretty much yes as long as enough goals scored!
Also finalising scenarios on are Italians really more emotional than Germans (sentiment +/- on Twitter!)

What politicians are tweeting in Victoria:
http://115.146.84.199:3000/#/politicianList

What politicians are tweeting in Federal government:
http://115.146.84.164/#/politicianList

Don't blame the technology, blame the humans

This week one of the top stories for one of Australia's leading newspapers has been the purported rape of a woman after she met a man using the 'Tinder' matchmaking app.

Rape is a terrible act and unacceptable in every circumstance.

However the newspaper chose to make Tinder the story, not the rape.

Tinder is a mobile application designed to help people meet prospective dates.

It simply alerts users to other Tinder users who fall within a specified age range and gender and are within a certain distance of your location, lettings you know whether you have any mutual friends.

The alerted user then decides whether they like the look of a person from a supplied photo and gives them the option to privately message, and if desired, hook up.

Users must actively choose to turn on Tinder and can reject others without the other user ever knowing.

In other words, Tinder isn't much different to visiting a bar and looking around to see who seems to be your age and your 'type'.

Tinder's popularity stems from its ability to allow users to be discreet when seeking a partner. It gives users control over their dating choices - when they are available, who can be matched to them and whether or not they wish to be approached. In other words it's a little better than hanging around in pubs and nightclubs hoping to meet the perfect partner and avoid undesired come-ons.

In many senses the app is no different to popular online dating websites like HeyCupid and RSVP. it is simply a technological tool adding convenience and control to matchmaking.

Like all other technologies and software Tinder is neither good nor bad, but simply meets a human need.

So when a newspaper uses headline like 'Police warning after Tinder date ends in gang rape' and
'Tinder 'gang-rape victim' withdraws statement' it really gets my back up.

The approach creates an inappropriate connection between a neutral technology and a disgusting human act.

If the lady in question had simply met her alleged attackers in a pub, the newspaper would likely have not reported the story, or reported it very differently - they would not have used a headline 'Police warning after pub date ends in gang rape'.

This type of reporting is part of what holds back the use of digital technologies by government agencies, and it is a damn shame.

When senior managers who don't use social tools only read, hear and see bad news articles which blame or associate specific technologies with human misconduct it can creates an inappropriate association and make digital seems more dangerous to use than it actually is.

I wish I had a dollar for every media story sensationalising the failings of Facebook, Twitter, Google, RSVP, Tinder and other social tools when the real failing is in the human users.

Yes there are risks to all technologies. Cars and other road vehicles kill over 1,000 Australians each year and many other people are hurt when using other technologies, from paper cuts to knife wounds.

However let's try to keep things in proportion. There's few technologies out there specifically designed to harm people and these are usually carefully controlled, like guns and explosives.

The vast majority of technology are neutral, able to be used for good or bad - harming humans through accident or deliberate misuse.

Mitigating these risks is possible. Obsessing over them is unnecessary.

So if you're ever confronted in the workplace by a colleague or manager who quotes a headline like Police warning after Tinder date ends in gang rape' agree with them that the event is tragic and horrible.

But remind them that it was a human act that made it tragic and horrible. It is not necessarily a fault of the technological tool.

IP Australia releases open data including over 100 years of patent, trademark, design and plant breeder rights records

In a great step forward for Australian open data, IP Australia has, for the first time in history, released over 100 years of patent, trademark, design and plant breeder rights records as free publicly accessible and machine-readable data.

Released through data.gov.au (at https://data.gov.au/dataset/intellectual-property-government-open-data), while not a real-time feed, the information contained on Australia's innovation history is staggering.

Note that the main data file is 430Mb as a zip, it can take some time to download and process.

It will be interesting to see how this data might be used over the next year and particularly in future hack competitions, such as GovHack 2015.

How current events play out in search requests - terrorism & related terms in Google trends

While agencies often invest significant money into tools for tracking trends on social media, one of the simplest ways to detect and monitor the rise and fall of key topics and issues online is through Google Trends.

Google Trends tracks the frequency of use of specific search words in Google searches. This represents the majority of online and mobile searches in countries like Australia (93%) and the US (68%)

As a free service, Google Trends has been used over the years to monitor trends in seasonal diseases, such as influenza and dengue fever, to track the relative level of attention paid to politicians, the number of mentions of sports during grand final seasons, and to understand the impact of advertising on product sales.

I used the service back in 2006-2007 to help track a government agency's rebranding program, and have used it subsequently, both with and outside government, to track the level of interest in particular issues and topics.

So today I decided to see what Google Trends can tell us about the level of interest or concern in terrorism, specifically related to ISIS and concerns about muslim extremists.

I chose five main words to track - 'Terrorist', 'ISIS', 'Islam', 'Muslim' and 'Burqa' - which told an interesting story.

Until May 2010 the burqa does not appear to be a particular concern for Australians, with few searches of the term.

However since then it has become more topical, with some interest throughout 2011, then a sudden surge in September 2014 when the 'ban the burqa' movement began to receive significant political support and media coverage.

In contrast, terrorist was a term of interests to Australians in 2004 and particularly in the second half of 2005, with surging interest in July and November of that year. Following this, it settled down into a largely quiescent state, with only a small surge in November 2008 interrupting the mostly flat line.

This changed in August 2014, with a huge rise in searches for the term across Australia resulting in the highest level of searches for the term in the history of Google Trends in September this year.

The same trend can be seen for mentions of ISIS, which were flat until May 2014 and have rapidly escalated since. Early mentions of the term presumably relate to other uses of the term (such as the Egyptian god), with the sudden rise in searches only attributable to the rise of the Islamic State.

Searches for Islam and Muslim have also been rising this year after a long largely flat period. While these terms are the subject of many legitimate searches related to the culture and religion, the recent rise in searches does tend to suggest and correlate with the rise in searches for terror-related terms, indicating that people have linked the terms in some way, at least out of curiosity.

It's possible to compare and contrast these trends with global trends in Google Trends, per the chart below.



This chart provides evidence of growing global interest in terms such as Islam, Muslim and, particularly, ISIS. However it shows little international concern over the burqa or regarding terrorism.

This can be seen in detail when looking at individual countries.

For example while similar trends of increased interest in searching the term ISIS are visible for the US, UK, Canada, Sweden, Japan, Thailand and many others, only a relative few see the burqa as a rising source of concern and many also are not experiencing heightened searches for terms such as islam or muslim.

This may be coincidental, or may reflect political statements and media reports on these topics - a more detailed review of coverage would be needed to confirm direct links.

However given that researchers have found that Google Trends can provide an accurate view of community concerns regarding infectious diseases and product trends, I believe there's sound reasons to suppose a correlation between what leaders say and what people search for.