Use of government open data creeping into universities

In September I was honoured to be invited to speak at the first Best Practice in Data Journalism seminar for Australia at Melbourne University, hosted by the Centre for Advancing Journalism.

The event included many of Australia's leading data journalists, together with academics and other representatives working in the data space.

At the event I was glad to see that a number of Masters students in the university were using government open data to create visualisations and derive insights.

Richard Sinnott has now published a public list of some of these projects in the Asia-Pacific data journalism and data visualisation group at LinkedIn, which I've included below.

These have been been developed in ten weeks or less and are works in progress, so consider them appropriately.

Federal government open data mash-up:
http://130.56.249.15/proj/

Visualising Victorian lobbyists, clients, companies, donors: http://115.146.85.10/proj/index.php/lobbyist/graph
(wait for it to load as there is a large complex graph).
Hopefully straightforward to drive/use...

Using Twitter for Melbourne transport congestion:
http://115.146.94.12:8080/web/statistic_date.html
(final scenarios coming soon)

Using Twitter for World Cup 2014:
http://115.146.93.143:8080/TwitterAnalysis/index.html
(can we identify events in games through Twitter sentiment) - pretty much yes as long as enough goals scored!
Also finalising scenarios on are Italians really more emotional than Germans (sentiment +/- on Twitter!)

What politicians are tweeting in Victoria:
http://115.146.84.199:3000/#/politicianList

What politicians are tweeting in Federal government:
http://115.146.84.164/#/politicianList

Read full post...

View summary...

Don't blame the technology, blame the humans

This week one of the top stories for one of Australia's leading newspapers has been the purported rape of a woman after she met a man using the 'Tinder' matchmaking app.

Rape is a terrible act and unacceptable in every circumstance.

However the newspaper chose to make Tinder the story, not the rape.

Tinder is a mobile application designed to help people meet prospective dates.

It simply alerts users to other Tinder users who fall within a specified age range and gender and are within a certain distance of your location, lettings you know whether you have any mutual friends.

The alerted user then decides whether they like the look of a person from a supplied photo and gives them the option to privately message, and if desired, hook up.

Users must actively choose to turn on Tinder and can reject others without the other user ever knowing.

In other words, Tinder isn't much different to visiting a bar and looking around to see who seems to be your age and your 'type'.

Tinder's popularity stems from its ability to allow users to be discreet when seeking a partner. It gives users control over their dating choices - when they are available, who can be matched to them and whether or not they wish to be approached. In other words it's a little better than hanging around in pubs and nightclubs hoping to meet the perfect partner and avoid undesired come-ons.

In many senses the app is no different to popular online dating websites like HeyCupid and RSVP. it is simply a technological tool adding convenience and control to matchmaking.

Like all other technologies and software Tinder is neither good nor bad, but simply meets a human need.

So when a newspaper uses headline like 'Police warning after Tinder date ends in gang rape' and
'Tinder 'gang-rape victim' withdraws statement' it really gets my back up.

The approach creates an inappropriate connection between a neutral technology and a disgusting human act.

If the lady in question had simply met her alleged attackers in a pub, the newspaper would likely have not reported the story, or reported it very differently - they would not have used a headline 'Police warning after pub date ends in gang rape'.

This type of reporting is part of what holds back the use of digital technologies by government agencies, and it is a damn shame.

When senior managers who don't use social tools only read, hear and see bad news articles which blame or associate specific technologies with human misconduct it can creates an inappropriate association and make digital seems more dangerous to use than it actually is.

I wish I had a dollar for every media story sensationalising the failings of Facebook, Twitter, Google, RSVP, Tinder and other social tools when the real failing is in the human users.

Yes there are risks to all technologies. Cars and other road vehicles kill over 1,000 Australians each year and many other people are hurt when using other technologies, from paper cuts to knife wounds.

However let's try to keep things in proportion. There's few technologies out there specifically designed to harm people and these are usually carefully controlled, like guns and explosives.

The vast majority of technology are neutral, able to be used for good or bad - harming humans through accident or deliberate misuse.

Mitigating these risks is possible. Obsessing over them is unnecessary.

So if you're ever confronted in the workplace by a colleague or manager who quotes a headline like Police warning after Tinder date ends in gang rape' agree with them that the event is tragic and horrible.

But remind them that it was a human act that made it tragic and horrible. It is not necessarily a fault of the technological tool.

Read full post...

View summary...

IP Australia releases open data including over 100 years of patent, trademark, design and plant breeder rights records

In a great step forward for Australian open data, IP Australia has, for the first time in history, released over 100 years of patent, trademark, design and plant breeder rights records as free publicly accessible and machine-readable data.

Released through data.gov.au (at https://data.gov.au/dataset/intellectual-property-government-open-data), while not a real-time feed, the information contained on Australia's innovation history is staggering.

Note that the main data file is 430Mb as a zip, it can take some time to download and process.

It will be interesting to see how this data might be used over the next year and particularly in future hack competitions, such as GovHack 2015.

Read full post...

View summary...

How current events play out in search requests - terrorism & related terms in Google trends

While agencies often invest significant money into tools for tracking trends on social media, one of the simplest ways to detect and monitor the rise and fall of key topics and issues online is through Google Trends.

Google Trends tracks the frequency of use of specific search words in Google searches. This represents the majority of online and mobile searches in countries like Australia (93%) and the US (68%)

As a free service, Google Trends has been used over the years to monitor trends in seasonal diseases, such as influenza and dengue fever, to track the relative level of attention paid to politicians, the number of mentions of sports during grand final seasons, and to understand the impact of advertising on product sales.

I used the service back in 2006-2007 to help track a government agency's rebranding program, and have used it subsequently, both with and outside government, to track the level of interest in particular issues and topics.

So today I decided to see what Google Trends can tell us about the level of interest or concern in terrorism, specifically related to ISIS and concerns about muslim extremists.

I chose five main words to track - 'Terrorist', 'ISIS', 'Islam', 'Muslim' and 'Burqa' - which told an interesting story.

Until May 2010 the burqa does not appear to be a particular concern for Australians, with few searches of the term.

However since then it has become more topical, with some interest throughout 2011, then a sudden surge in September 2014 when the 'ban the burqa' movement began to receive significant political support and media coverage.

In contrast, terrorist was a term of interests to Australians in 2004 and particularly in the second half of 2005, with surging interest in July and November of that year. Following this, it settled down into a largely quiescent state, with only a small surge in November 2008 interrupting the mostly flat line.

This changed in August 2014, with a huge rise in searches for the term across Australia resulting in the highest level of searches for the term in the history of Google Trends in September this year.

The same trend can be seen for mentions of ISIS, which were flat until May 2014 and have rapidly escalated since. Early mentions of the term presumably relate to other uses of the term (such as the Egyptian god), with the sudden rise in searches only attributable to the rise of the Islamic State.

Searches for Islam and Muslim have also been rising this year after a long largely flat period. While these terms are the subject of many legitimate searches related to the culture and religion, the recent rise in searches does tend to suggest and correlate with the rise in searches for terror-related terms, indicating that people have linked the terms in some way, at least out of curiosity.

It's possible to compare and contrast these trends with global trends in Google Trends, per the chart below.



This chart provides evidence of growing global interest in terms such as Islam, Muslim and, particularly, ISIS. However it shows little international concern over the burqa or regarding terrorism.

This can be seen in detail when looking at individual countries.

For example while similar trends of increased interest in searching the term ISIS are visible for the US, UK, Canada, Sweden, Japan, Thailand and many others, only a relative few see the burqa as a rising source of concern and many also are not experiencing heightened searches for terms such as islam or muslim.

This may be coincidental, or may reflect political statements and media reports on these topics - a more detailed review of coverage would be needed to confirm direct links.

However given that researchers have found that Google Trends can provide an accurate view of community concerns regarding infectious diseases and product trends, I believe there's sound reasons to suppose a correlation between what leaders say and what people search for.

Read full post...

View summary...

Is government paying enough attention to privacy in its mobile apps?

Australian internet usage has just reached a tipping point, with more Aussies accessing the internet via their smartphones and tablets than via laptops and desktop computers.

This has been reflected in web usage statistics, with several agencies I talk to reporting that they now receive more of their website traffic from mobile devices than from desktop and laptop computers - particularly when excluding their own staff from the statistics.

There have also now been over 500 mobile apps designed, commissioned or reused by Australian government agencies and councils to deliver information, access services and report issues, including 69 apps from Federal agencies, 80 from Victorian government agencies, 22 from Queensland government agencies and many from local councils around the country.

There's even a few notable games, such as the ABS's Run That Town and Victoria's MetroTrains Dumb Ways to Die.

As a result there's an increasing need for agencies to pay attention to how they design mobile apps to ensure they meet appropriate accessibility and privacy standards.

The latter part of this, privacy, was the subject of a recent study and guide from the Office of the Australian Information Commissioner (OAIC) - Mobile privacy: A better practice guide for mobile app developers.

The guide reported that privacy was a key consideration for citizens, with a 2013 study by the OAIC finding that 62 per cent of Australians opt not to use smartphone apps because of concerns about the way personal information would be used.

The guide also mentioned a similar study in the US by the Pew Research Centre in 2013 that found that 51 per cent of teenage app users had avoided certain apps over privacy concerns, and over a quarter had uninstalled an app because it was collecting personal information they did not wish to share.

Now that's all fine when Australian governments are designing apps properly.

However the OAIC took part in an international 'sweep' on mobile app privacy back in May. As part of this the OAIC examined 53 popular free iOS apps, with a focus on apps produced by or on behalf of Australian businesses AND Australian Government agencies.

The OAIC found that a significant number of these mobile apps did not meet Australian privacy law requirements.

‘Of particular concern was that almost 70% of the apps we looked at failed to provide the user with a privacy policy or terms and conditions that addressed privacy prior to the app being downloaded’, Mr Pilgrim said.

The OAIC also found that almost 25% of the apps examined did not appear to have privacy communications tailored for a small screen.

Only 15% of the Australian-developed apps the OAIC examined provided a clear explanation of how they would collect, use and disclose personal information, with the most ‘privacy friendly’ apps offering brief, easy to understand explanations of what the app would and would not collect and use based on a user granting permission.

I'm sure the OAIC has privately fed back information to agencies on how their apps failed to meet Australian privacy and actions are underway to rectify this.

Other agencies and councils that have developed, are developing or have partnered with commercial mobile apps also need to be aware of the risks they are taking on if they don't adequately meet Australian privacy law.

Under the updated law that came into effect earlier this year, penalties for government agencies and corporations range up to a million dollars - making the omission of a privacy statement or use of user data without clear permission quite an expensive proposition.

Hopefully agencies are aware of the OAIC's report and are ensuring that user privacy is taken into account within their mobile apps.

If not, I hope we see some high profile examples to ensure that other agencies change their behaviour.

Read full post...

View summary...