Showing posts with label analytics. Show all posts
Showing posts with label analytics. Show all posts

Friday, February 27, 2015

It's a good time to rethink the approach to Australia's Census

The first censuses reportedly took place around 5,000 years ago in Egypt, and the approach is mentioned in Exodus, one of the books of the Bible, commissioned by God in order to levy a per capita tax to upkeep the Israelite Tabernacle.

The ancient Greeks, Chinese and Indians all conducted censuses before 1 AD, and censuses were vital for the Romans to levy the taxes that maintained their empire.

More recently, the Domesday book was commissioned by William the Conquerer, also known as William I of England, to properly tax the land he had conquered from the Saxons.

Independently the Inca Empire in South America, who had no written language, conducted censuses in the 15th century using base-ten notation knots in llama hair string.

In modern times over a hundred nations have conducted censuses on a regular or semi-regular basis, with nations like the US and UK conducting them every year and Australia every five.

A census collector from Egypt in 3000BC would probably have found it easy to comprehend the censuses Australia conducted in the 1990s.

While there's more questions today, the method of collecting census information had changed very little until recently.

Up until the introduction of electronic censuses this century, nations sent thousands of census collectors out and distributed millions of census forms for completion.  The collected data was then returned to central points for collation and analysis.

Now the ABS has prepared a paper to government suggesting that they may have better techniques for estimating population statistics rather than conducting an expensive national Census every five years.

This suggestion has caused media controversy, a public backlash and even a campaign to 'save our census'. Australians feel a sense of ownership over the Census, particularly after the fantastic work done by the ABS last census to engage people via social media and interactive tools.

However I think it's a good and appropriate conversation to have right now. We've advanced enormously technologically and scientifically in the last ten years, not to mention the last 5,000.

In fact, if you really think about it, conducting the census in a way that would be familiar to the Romans or William the Conquerer really doesn't make sense for the 21st Century.

With the data collection techniques available now, and the expertise we have in data analytics and prediction, surely we can find more cost-effective and less invasive ways of taking stock of our population than having every household fill in a form on one night.

So while I appreciate the concerns people have about change, and the nostalgia for our census-takers, I applaud the ABS and government for at least considering new approaches to the census.

If we truly want to digitally transform our government and society, we need to challenge practices that have become the norm.

This means not simply updating them (such as making a paper form electronic), but completely rethinking them beginning with our goals, applying the most appropriate approaches to data collection and analysis, and rethinking how we share and use the data for the benefit of society.

This needs to be done with an eye to improving outcomes, avoiding outcomes such as in Canada where it was about money, not quality. It also needs to be done with an effective change management campaign, involving the public in the debate.

Ignoring these two areas risks a messy and difficult process of reconsidering the Census, likely with poor outcomes both for the current government and for Australia.

However a well-thought-out public engagement campaign, combined with clearly superior techniques of collecting and analysing data could be a win for both.


I look forward to the day when children learn about the census in their history books, a system that was great for 5,000 years but was superseded by more effective techniques as humans advanced.

And I look forward to seeing more sacred cows in government and business challenged wisely and effectively with appropriate public engagement.

Read full post...

Wednesday, June 12, 2013

Sentiment analysis: where 'disabled' and 'disability' are often considered negative terms

It's come to my attention that a number of automated sentiment analysis tools include 'disabled' and 'disability' as negative terms.

This means that when calculating whether a particular statement in social media is positive or negative, the use of these words is used by these sentiment analysis tools as an indication that the statement is negative towards the topic of the statement - such as a topic, issue, individual or organisation.

I've checked a number of sentiment dictionaries online and found that both 'disabled' and 'disability' appear frequently as negative terms. However I have not yet been able to confirm whether any sentiment analysis products treat these words in this manner.

This disturbs me, given the efforts of governments and civic organisations in Australia and many other countries to remove negative stigma attached to the word 'disabled', even given its potential application in statements such as 'their system has been disabled'.

It also concerns me that agencies engaging online about disabilities or with disabled people, might accept that the sentiment reported by their social media monitoring tools indicates negativity where in actuality no negativity exists.

I would caution government agencies using automated sentiment analysis tools to get to know they work and check how terms such as 'disability' and 'disabled' are treated in these systems.

I'd welcome comments from makers of sentiment analysis tools to confirm how they treat these words or from agencies using automated sentiment tracking if they've seen these words or others rated negatively or positively in ways which might be misleading and misrepresent the actual sentiment.

Read full post...

Thursday, May 23, 2013

Where are all the public sector web analysts?

It's not unusual for agencies to spend millions of dollars on a new program supporting some group in society and hundreds of thousands of dollars on consultants and research to ensure that the new program is designed to be successful, to track its progress and audit it over time.

It's not unusual for agencies to spend tens of thousands of dollars on an online presence to educate people (such as on a new program) - a website that may provide information, eligibility and application processes as well as a couple of social media channels for engagement.

However it is rare for an agency to spend even thousands of dollars on social media monitoring and website analytics to determine whether the website and online presence for a new program is being effective, where improvements could be made, to track its progress and audit it over time.

I've worked in several government agencies which had some kind of research unit, who spent their time analysing customer and program information to provide insights that help improve policies and service delivery. Unfortunately, even in the five and a half years I spent in the public sector, I saw these units cut in responsibilities, reduced in size, even turned into contractor units simply managing external research consultants.

These units were still new to online when I entered the public service, unsure of how to analyse it or how to weigh the insights they might receive. However when I left, although online had been recognised as an important channel, the capability of research units to integrate it into other analysis had been sadly diminished due to budget cuts.

This trend towards outsourcing or simply disregarding data analysis, at a time when society has more data at its fingertips than ever before, is worrying in government. What trends are going unnoticed? What decisions are being made without consideration for the facts?

However I have a special concern around how government agencies regard web and social media analysis, which in my view are increasingly useful sources of near real-time intelligence and longer-term trend data about how people think and behave.

As I've never been able to afford to have a web analytics expert in one of my teams in government, I've spent a great deal of my own time diving into website and social media stats to make sense of why people visited specific government sites, what they were looking for and where they went when they didn't find what they needed.

I've also used third party tools - from Hitwise to Google Trends - to help identify the information and services people needed from government and to help present this information to agency subject matter experts and content owners to help inform their decisions on what information to provide.

I know that some agencies have begun using social media analysis tools to track what people are saying online about their organisation and programs, often to intervene with facts or customer service where relevant, and this is good and important use of online analytics.

I'm even aware of web and social media analytics being provided back to policy areas to help debunk beliefs, much as I used to give different program areas snapshots of their web analytics to help them understand how effective their content was with the audiences they targeted (when I had time).

With the rise in interest in open data, I guess what I'd like to see in government agencies is more awareness of how useful their own web analytics can be to help them to cost-efficiently understand and meet citizen needs. I would also like to see more commitment of resources to online analytics and analysts within agencies to help their subject matter experts to keep improving how they communicate their program, policy or topic to layman citizens.

It may also be a good time to look into the intersection of open data and online analytics - open analytics perhaps?

I would love to see agencies publishing their web traffic and social media analytics periodically, or  live, allowing government websites to be held accountable in a similar manner to how data on crime statistics helps keep police accountable.

Maybe certain web statistics could even be published as open data feeds, so others might mash-up the traffic across agencies and build a full picture of what the public is seeking from government and where they go to get it. This could even allow a senior Minister, Premier or Prime Minister to have full visibility on the web traffic to an entire state or nation - something that would take months to provide today.

This last suggestion may even overcome the issue agencies have in affording, or for that matter finding, good web analytics people.  Instead external developers could be encouraged to uncover the best and worst government sites based on the data and provide a view of what people really want from government in practice, from how they engage with government online.

Read full post...

Wednesday, April 10, 2013

Web and social media reporting can help Communication get a seat at the decision-makers' table

Yesterday morning I attended the first OPC IT WebEx event for the year, where we heard from three great speakers on intranet development, accessibility and the changing face of the media in Australia.

One particular statement that stuck in my mind was from David Pembroke, CEO of Content Group, who said that it was important for communications people to bring numbers to the table to gain a seat alongside other decision-makers, such as CFOs and CIOs who already have numbers in hand to support their positions.

While most agencies now track the traffic to their website and report raw numbers of followers, comments and mentions on their social channels, I believe there's still a way to go before these numbers are provided in the right way to the right people at the right time to help Communications areas - and particularly Online Communications - have the impact and the influence it deserves.

This has been brought home to me by Slideshare, which recently began sending me reports on the number of views and interactions on the various presentations I've uploaded to the service over the years.

Simply being able to see these basic stats has made me take more notice of the material I'm putting on Slideshare and whether or not it has a wider audience that I should consider when developing my slides.

I'm even considering paying money for an account to get more detailed statistics that will help me finetune material to better match what audiences want.

When working in Government I did place a considerable amount of effort into providing web statistics back to the areas responsible for specific content. I believe this type of reporting is critical to help policy and program areas receive regular and actionable feedback on what they are putting online to help inform their customers, clients, stakeholders and other audiences.

In fact, without web reporting many of these areas only receive ad hoc and irregular feedback on the content they are producing - an annual survey, or some Ministerial Correspondence. This makes it harder for them to understand whether their content is targeted correctly and also means they place much lower emphasis on what they are communicating online - what isn't measured isn't managed or valued.

Now with social media in the picture, web reporting needs to jump to a higher level of competency. While agencies might have made some steps to ensure that various areas of their business are receiving reports on the content they are providing through websites, the new frontier is to provide them with actionable information on what people are saying about their programs and policies across the broader web.

This helps areas within agencies not only assess how people are responding to the information they do provide online, but also gives them some understanding of what questions and issues are being discussed due to the lack of content.

In other words, web reporting helps tell agencies the quality and effectiveness of their own website content. Social media reporting helps tell agencies about the community's content needs beyond existing content.

The benefits to agencies of this social media monitoring are immense, not only can we capture known unknowns, but also the unknown unknowns - intelligence that could shape the entire way a program or campaign is designed and communicated.

It is also very important to differentiate social media monitoring from media monitoring - something that is getting harder to do as media monitoring companies move to bundle social within their media offerings.

Media monitoring tracks what commentators say about an agency and its activities when posturing to a broad audience.

Social media monitoring tracks what your customers and stakeholders are saying about an agency and its activities to each other.

In other words social media monitoring can provides a granular and specific view on what your actual customers think and understand about specific programs and how they interact with them in the real world, while media monitoring only provides a shallow reputational view on what people are saying for an audience - which may simply be an act.

So while there is a clear incentive for Online and Communication teams to roll social media monitoring in as an extension to (traditional) media monitoring, it can be dangerous to consider the intelligence received through both avenues in the same light.

As agencies get better at both web reporting and social media monitoring, and develop standardised ways to communicate actionable insights to the right people, at the right time, we're likely to see more ability for the groups providing these insights to have meaningful influence on agency decisions. This is right and proper - better information leads to better decisions and outcomes.

However it is up to Communication and Online teams and their leadership to recognise how web and social monitoring can advance their ability to positively influence decisions and take the lead on providing insights, otherwise they will find themselves on the margins as more traditional numbers-orientated disciplines take over the responsibility for these activities.

Read full post...

Bookmark and Share