Posts
One of my mantras in professional life is 'you can't manage what you don't measure'.
Therefore it always worries me when I encounter organisations or individuals with a less than firm grasp on how to measure the success or failure of their online properties.
Depending on the type of web property, different metrics are most important for regular tracking and I believe it's the responsibility of top managers to understand the online metrics they use - just as they need to understand business ratios or balance sheets.
After more than twelve years of trial and error, below are the metrics I most and least prefer to use to track different types of online media.
What are the best metrics to use?
Standard websites
Visits
This tracks the total number of visits by users to a website over a period of time (month, week, day). This can include the same unique visitor returning to the site multiple times - which is the same way calls are commonly tracked for call centres.
Visits gives you an overall view of website traffic and, when divided by Unique visitors, provides a measure of 'stickiness' - how often people return to your site.
Note that for an unauthenticated site, a visitor is essentially an IP address, a computer. As multiple people can use a single PC, or a single person can use multiple PCs and it also may track search spiders and other bots, visits doesn't provide a perfect measure of human traffic but it's sufficiently good for trend analysis over time.
In addition, caching by ISPs or organisations can also influence visits - reports based on AOL from a few years ago indicate that visits reports may under report website traffic by as much as 30 percent due to caching - though this is less important today.
In comparison 'readership' is a much looser metric, but is often held in high regard in the print trade.
Unique visitors
Unique visitors tracks the individual IP addresses used to visit a website and as such provides a rough count of the number of actual users of a site, no matter how many times they visit.
This equates to 'reach' for a site - with growth in unique visitors indicating more people are coming to a website.
This is affected by the same IP versus human issue as visits, however is again still far more accurate than 'readership' figures provided by the press or 'viewer' figures provided by TV and radio - which are based on a sample rather than a population (as unique visitors is).
Pageviews
Pageviews are a more specific measure of the views of specific pages within a website, and is most useful for tactical website tracking, allowing the identification of high and low traffic pages and the impact of different navigational or promotional approaches.
Looking at pageviews also provides a psychological view of your audience's top interests - allowing you to quickly prioritise content to be expanded and which can be downplayed.
Pageviews is becoming less important as technology cocktails such as AJAX are more widely used to load part of a page's content automatically or in response to user actions. In these cases a single pageview may not track what the user views in the page.
Authenticated website (transactional services)
Active users
Active users tracks the actual use by authenticated users (real humans) in a time period.
This is the best measure of an authenticated site's success as it tells you how well you've encouraged ongoing use of a website, rather than simply how good a job you've done at getting people to sign up.
Many authenticated sites prefer to talk about Registered users as this is a much larger number, however if a user has registered but never returns, your organisation gains no value from it.
A low ratio of active users to registered users can indicate site problems, and should prompt website managers to ask the question why don't people come back?
Transaction funnels
Transaction funnels track the completion of transactions step-by-step in a service - and isn't necessarily only for authenticated sites.
This provides a website manager with tactical insights into any issues in a transactional process (or workflow), allowing them to diagnose which steps have the greatest abandonment rate and redevelop the process to improve completion.
Generally improving transaction funnels results in more transactions and more active users, which means greater utilisation of the service.
Multimedia (video/audio/flash)
Views
For any type of rich media, the number of views of the media is critical in determining success. However it has to be weighted against the Duration of views to determine if users spent long enough viewing in order to take away the message, or just viewed the first few seconds.
Duration of views
The duration of media views is a more granular measure of the effectiveness of the presentation - tracking whether the media actually communicated its message to users.
Looking at the average duration viewed, compared to the actual duration of the media (where such exists) provides a very strong effectiveness measure.
Shares
One of the keys with the success of media content is how much it is shared with others online - the word of mouth factor. For media with a 'refer to a friend' tool, tracking the use of this will provide a strong indication of how positively users view the material, and therefore how viral it will become. Media that is rarely shared is probably not getting the message across in a memorable way, whereas highly shared material is correspondingly highly memorable - at least for a short time.
Documents (pdf/rtf/docs)
Views
Often 'downloads' is used to track documents. Personally I prefer views as there are some technical issues with tracking downloads of files such as PDFs. In effect the two measures should be identically, but as PDFs, and sometimes other documents, download by segment, they can significantly overreport downloads (which becomes almost as useless as 'hits'), whereas views is a more accurate measure.
There are ways to fix this within reporting systems - which I've largely done in my Agency's system - however this is not possible in all systems.
Social media
Activity by user
Like authenticated sites, the goal of social media is to encourage participation - whether it be forum posts/replies, wiki edits or social network updates/messages.
Each of these represents activity - which may need to be tweaked by the type of social media.
The more activity by users, the more engaged they are with the site and the greater the prospects of longevity.
Views
The other useful measure is views, measuring the passive involvement of users with a social media site. Not all users will actively post, however if they return regularly to view, they are still engaged to some extent with the site.
Commonly the breakdown between active and passive participants is divided as 1/9/90 (Very active/active sometimes/passive observer), however in practice this varies by medium and community.
While that 90 percent doesn't add to the content of the site, they are vital for the other ten percent to participate.
Search
Top searches
Search is also an important area of sites, with the top searches providing another insight into what people want from your site - or what is not easily findable in other navigation.
Tracking this over time provides another perspective on the psychology of your website users. It helps you understand their terminology for navigational purposes and can help prioritise the content you should modify or add to in the site.
Zero results
Any search terms which result in zero results in your site should be looked into as a high priority.
Generally this reflects areas where your website lacks content or uses the wrong context or different language to the audience.
What are the wrong metrics?
Hits
Probably the least useful metric of all time, Hits is still the best known measurement for websites, despite having no practical business uses.
Hits measures the number of files called from a web server, with each separate file accounting for a single 'hit' (on the server).
On the surface this doesn't sound so bad - however webpages consist of multiple files, with the base page, style sheets, graphics and any database calls or text includes each accounting for a separate hit.
A webpage might consist of a single file, or it might consist of 20 or more - meaning that there is no clear relationships between hits and actual page views or user visits to a website.
To increase the number of hits to a website it simply requires the website owner to place more file calls in the page - potentially calling extremely small (1 pixel square) images, therefore hits can be easily manipulated with no effect on the actual number of website users.
So while hits figures are frequently impressive, even for small websites they can easily reach millions each month, they don't provide any useful business information whatsoever.
Monday, September 01, 2008
Use the right online metrics for the job | Tweet |
Sunday, August 31, 2008
Gershon review submitted to federal government | Tweet |
On Friday (29 August) Peter Gershon submitted his review on the status of the $16 billion Australian Government technology budget to the Federal Government for consideration, as reported in a number of places, including this Australian article, Gershon submits govt ICT review.
The Australian article reports that,Sir Peter said his recommendations involve a major program of both administrative reform and cultural change within the Australian Public Service.
I'm eagerly awaiting a look at the recommendations of this review, with my particular interest being in ensuring that the online channel continues to be a lower-cost delivery medium for government information and services, used flexibly to ensure citizens receive the most effective, as well as the most efficient, outcomes.
"With sustained leadership and drive at Ministerial and top official levels, and by providing the enablers of change with the necessary resources, not only in funding terms, but also skills of the right calibre, the Australian Government through implementing these recommendations can drive significant improvements in its use of ICT.
"I am confident that the recommended actions and proposed changes can be successfully implemented over the next two to three years, and will deliver substantial benefits to the Australian Government," Sir Peter said.
As a relative newcomer to the public sector one of my largest challenges has been dealing with how ICT is managed in government in terms of culture, structure and accountability. It is very different to my personal experience working with and managing IT teams in private enterprise.
There are specific laws and standards which are mandatory for government, but do not hold as much force in the private sector - such as around accessibility and security. There are also different drivers when the profit motive is removed.
The largest difference I've noticed has been in the level of application of technology for governance structures that in the private sector are more commonly managed through contracts and professional relationships, making them more adaptable, lower cost and placing business systems under business control.
Judging from the positive outcomes in the UK after Sir Gershon's 2004 Independent Review of Public Sector Efficiency report, I hope that in Australia we move forward with his requirements in order to unlock more of the potential of ICT to be a forward-looking and innovative facilitator in the Australian government's relationships with citizens.
Friday, August 29, 2008
IE8 public beta now available | Tweet |
It's going to be a busy six months for web teams, addressing WCAG 2.0 (the new W3C Accessibility Guidelines) and the release of Internet Explorer 8.
For those who want to stay ahead of the game (like me), the second IE8 beta is now available for public download here.
As I've now switched almost totally to Firefox for home use (due to it's superior Australian English spell checker, ability to resume a session after a PC crash, better support for plug-ins and superior adherence to web standards) and therefore have limited need for IE7 on my laptop, I'm about to give IE8 a try.
It will be very interesting to see how some leading sites (including those I manage) function.
I expect that there will be a rapid take-up for IE8 by home users, up to 35% of the market in the first three months (I'll allude back to this post three months after launch!)
Any organisation who isn't ready for either the IE8 or WCAG 2.0 shift runs the risk of having their website become less accessible to the market.
That's food for thought when there's a legal obligation to meet accessibility guidelines.
Finding egovernment inspiration in Alabama | Tweet |
If you're looking for inspirational egovernment ideas, the CIO 100 Award from CIO magazine are a great place to start. The annual Awards honor organisations creating business value via technology innovation.
For 2008 the State of Alabama was the only state government to win an award, for its egovernment initiatives, involving the delivery of over 130 egovernment services via the Alabama.gov portal.
Some of these services include live online website help, a 'virtual Alabama' powered by GIS technology, the deployment of video conferencing for distance learning, a range of online processes for ordering and renewing licenses, online payment, and many other online services.
While Alabama is the only state to win, a number of US government departments were alsorecognised with awards, including,
- the U.S. Department of Housing & Urban Development
- the California Department of Health Care Services
- the District of Columbia Water and Sewer Authority
- King County, Washington
- the Massachusetts Developmental Disabilities Council, and
- Miami-Dade County Public Schools.
Thursday, August 28, 2008
Australia rated 6th in global egovernment study | Tweet |
Brookings University recently released its report Improving Technology Utilization in Electronic Government around the World, 2008 (link to PDF).
This ranks the government websites of 198 nations, reviewing 6-10 sites in each nation.
Australia ranked 6th, behind South Korea, Taiwan, the US, Singapore and Canada, up from 8th position last year.
As a benchmark this is great - it's better than our Olympic ranking (on substantially less funds per website than we spend on medal winning athletes), and substantially better than our global population ranking of around 50th.
However this study compares governments against other governments, rather than with citizen expectations.
While I do use other governments' initiatives to stimulate my thinking, I'm more interested in what our citizens want.
I also regularly refer to AGIMO's fantastic work on the use of government services online, and the 2006 e-Government Strategy, Responsive Government. There was also the (now superceded) Guide to Minimum Web Site Standards.
However none of these provide a citizen-centric view of what government sites need to provide that can be used to provide numerical ratings for each government site.
I'd love to have such a ranking available as I used to have in the private sector - using Global Reviews - to provide guidance as to what our citizens want, and the relative importance of different functionality. This would greatly assist my team and I'm sure other online groups, to prioritise online developments inline with citizen desires.
Has anyone seen a study in Australia or elsewhere on the community's expectations of how citizens should be able to engage government online?
