Web Analytics Success Measurement For Government Websites

[Cross-posted from the personal blog of Google Analytics Evangelist Avinash Kaushik]

If you know what the desirable outcomes are from your website, it is not hard to measure performance of the website for you and your customers.

Measuring top line success of ecommerce websites is not very complicated, all the sweet revenue based outcomes are there (at the least).

Measuring non-profit websites is a bit complicated, but not really all that hard because we can, with a small amount of love, figure out outcomes to focus on (donations, # of sign-ups for the protest in DC, # of petitions signed, volunteer applications, etc).

Measuring government websites is a bit more complicated, if for no other reason than that it takes a pinch of effort with a dash of imagination to figure out what one is solving for. What are the desirable outcomes one can focus on to measure success?

The above question came to mind from a kind note I got from Ines Jans who is a part of the team that is responsible for www.belgium.be

Ines and team were just starting to think about analytics (because the love their customers!) and asked for some thoughts.

My first question to Ines was, (surprise!):

Q: Tell me a bit more about what your site does, like what are the real goals (or give me some ideas about it) and what challenges you face, what do you expect people to get out of it?

[Best Practice: Always, always, always start any measurement conversation with the above inquiry. The answer will be key to insights, and without it you'll simply be a glorified Reporting Squirrel.]
The answer, which might fit most government websites was:

A: The goal of our site is to be a portal to all the official information there is about Belgium and make information easy to find. Visitors should be able to figure out which Ministry is responsible for what tasks.

[Best Practice: Don't be surprised in your Analysis Ninja quest if you get answers that just start the conversation, rather then give you a prescription for what you need. Squirrels will despair here, but Ninjas will take clues from what they hear and visit the site and come up with a set of important measurable outcomes.]

Based on the answer above and some time spent on the English language site as well as those in other languages (Google Translate!), I came up with the following five questions I could ask data to measure success.

~ Are Visitors able to find the information they are looking for?

~ Are the Visitors satisfied with their experience?

~ What is the most popular content on the site? What area can we prioritize higher than it currently is?

~ How long does it take for someone to find key information they want?

~ Does the right information actually exist on the website? What major things might we be missing on our website?

Let’s take each of these questions one at a time and figure out the best way to answer each using a true Web Analytics 2.0 strategy.

Q1. Are Visitors able to find the information they are looking for?

Given the singular purpose in life of this government website is to be the one stop shop for all the information one could possibly need, it should be pretty obvious that the very first, and magical, thing we would measure is if Visitors to the site are able to find what they might be looking for.

So would you use a web analytics tool?

Here’s the first surprise: No!

There are certainly tertiary ways in which you can answer this question using Omniture’s Site Catalyst or Google Analytics or other wonderful web analytics tools.

But the best way to answer this question?

Ask the Visitors!

Using a simple survey that pop-up on-exit (when Visitors leave the website) you can ask your customers to tell you if they were able to complete their task. No interpretation required.

4Q from iPerceptions, available in 18 languages, is a free on-exit survey you can use. If you don’t want to use a external survey build your own, ask four questions, analyze the data for:

“Were you able to complete the purpose of your visit today?“

The answer to this question becomes the #1 Key Performance Indicator (KPI). You are going to watch it like a hawk, you’ll post it on all your bulletin boards, you’ll set up custom alerts to ensure that your team gets a small electric shock every time this number drops below 65%!

The overall number is good enough, but the data that will be awesomely actionable will be, if you use 4Q: Primary Purpose by Task Completion Rate. . .

You see the second question in 4Q is “Which of the following best describes the primary purpose of your visit?” and a standard report in 4Q will paint the above picture.

Now you not only know if people find what they are looking for, but you also know which tasks are hard to complain.

You need to fix “Complain about the French” : ) because the Visitors are already upset and only 5% are able to complete their task, resulting in them becoming even more mad!

Remember: You don’t need to show the survey to everyone who comes to your site. You can sample just a small percent of your Visitors. You only need 300 responses in a month to get a statistically significant sample of data, and 1,200 if you want to do segmented analysis.

Q2. Are the Visitors satisfied with their experience?
HiPPO’s (the “highest paid person’s opinion”) in the organization, even in the government will love to have a more direct (than task completion rate) answer to the question: Are our Visitors happy with our website?

That’s were it is prudent to measure Customer Satisfaction.

4Q and other surveys of course measure that quite easily: Based on today’s visit, how would you rate your site experience overall?

Measure it. Trend it. Report it. Correlate the trend over time with changes you have made to the site and identify insights (any causal connection between site improvements / campaigns and customer satisfaction?).

An alternative, or additional, way to measure satisfaction is to count and analyze the Contact Us submissions. . . .

Start with the number of submissions. Trend over time.

Drill down into the type of complaints and do atleast rudimentary sentiment analysis (i.e. read & categorize) of actual messages to gauge customer satisfaction.

Remember: When you do surveys you don’t have to torture your Visitors with billions of questions! In researching this post I went to US government sites and I got a ugly 34 question on one single page looooong survey. 34 questions! Most were irrelevant. I would have answered a few, but this showed a fundamental disrespect for your customers. In the end your Visitors are upset and you suffer from a lack of data.

Only ask what you can action.

Q3. What is the most popular content on the site? What area can we prioritize higher than it currently is?

It is not unusual for content sites to produce content. It is even less unusual for them to produce content that they think potential visitors to the site might want.

What is rare is the analysis of what visitors to the site are actually consuming on the site.
Here’s a simple analysis I had learned from Tim Hart (who was with the J. Paul Getty Trust): Measure the distribution of content in each section of your website and the percentage of Visits to each section.

On the y-axis is each of the sections on the belgium.be website. In blue is the amount of content in each section. In red are the percentage of visits where that content was consumed.

Is it not awesome! Insights galore!!

If this were their data, and it is not, it would be pretty obvious that there is huge interest in content about Housing and Economy tiny fraction of the site’s content is about Housing and Economy.

The balance for Family is a lot less lop sided.

While the government might love Justice, Mobility and Health (and boy do they love Environment!), Visitors to the site are a lot less interested in those pieces of content.

Action? You know what people want, how about giving them more of that content?

When it comes time to prioritize the next set of web pages or videos or podcasts, how about giving higher priority to those big red lines?

Sweet right?

You can also do segmented version of this analysis, see what Visitors to English, Dutch, French and German sites prefer. Or within Family what group of content do people like. Etc etc.

Two more ideas to get into your Visitor’s head. . . .

Measure Downloads:

There are a ton of downloads (pdf’s mostly) on the belgium.be website. Forms, applications, useful guides (like how to marry a belgian or how to prepare for your first job) etc.

It is a trivial cost, analytically, to track these downloads using your web analytics tool. Do it. Measure what your Visitors are most interested in.

Yes, yes, yes I see my technical squirrel friends raising their hands and saying you can only track that someone clicked on the download link and not that the download was successful. I know.

For our analysis here just intent is fine.

In fact unless a vast majority of your Visitors are connecting using dial up it is safe to assume the download of small files went through. I know that does not make the squirrels happy. I am sorry. You keep squirreling while we make decisions about how to improve the site.

Outbound Link Tracking:

Another thing you’ll notice about the website (see why it helps to surf a site you are supposed to analyze?) is that there are a ton of links on the site that point to other government websites.
Track ‘em!

Of course the above is not their data :), it’s just an illustration of how absolutely easy it is to track this data.

From the report it is very easy to then figure out what links your Visitors click, which is a great, positive, indicator of the fact that they found what they wanted and also what they were interested in.

Remember: It is not very hard to do any of the above three types of analysis. All you need to get into your customer’s head is move away from “Top Pages Viewed” and “Page Views Per Visitors” and think a bit more creatively.

Q4. How long does it take for someone to find key information they want?

There are some pieces of content that are so darn important that they are heavily linked (say latest news in case of belgium.be) right from the home page, or that you really do want people to find them asap (in the Health section for example the pdf about how to deal with H1N1 virus in belgium).
For these important pieces of content measure Average Time To This Page.

That’s almost three minutes from the time that someone entered the website to the time they found this page (say the one about swine flu).

On average people give a page two and half seconds before they click/leave. Consider how long three minutes is, and how many people might have given up in the process of finding this key information.

Unfortunately not too many tools, Google Analytics and other Paid Solutions included, provide this as a standard metric. I use ClickTracks and that this delightful metric as a standard offering.

I wish others would have it.

Remember: You can use this data to ensure that your best information is found by Visitors to your site quickly. Fix your top / left / right / bottom / whatever navigation you have on the site. Consider creating a prominent “box” on the top right where you “merchandize” these important links. More things like that.

Q5. Does the right information actually exist on the website? What major things might we be missing on our website?

I have consistently advocated my love for internal site search analysis. It is simply da bomb!

Like many other sites belgium.be has a internal site search engine. Typically Visitors who have a harder time with normal navigation (or limited data on a page) will make liberal use of this site search box.

Why not use that data?

Again, this is not their data :).

I recommend looking at the top typed search terms by the Visitor but then also looking at the metric: % Search Exits.

That’s the bounce rate of your search results page. I.E. People come to belgium.be, search for the term hippo and the search results are so bad that 33.33% of the people exit from that page! They don’t even bother to do anything. Just bail. Bounce. Kaput!

Now you know both 1. what information they were looking for, 2. what search results stink and 3. likely because you don’t have the right or enough content about that keyword on your site.

Fix it!

I have one more idea to understand if you are missing information that your Visitors want on your government website.

Use Page Level surveys.

There are free page level surveys available or you can build your own (like the one above from a software vendor’s website).

These can be an excellent way to understand what content is missing from your website. You can of course also use the open text voice of customer (VOC) from surveys like 4Q, look for Visits with Task Completion = No.

Remember: These surveys don’t collect any personally identifiable information (PII) information, and that goes for your web analytics tools as well. Many government sites are extra concerned about privacy, as they should be. Do Please familiarize yourself with the privacy policies of the vendor.

Note: What was not tracked or emphasized. . .

Visits.

Unique Visitors.

Page Views.

Time on Site.

And so many more mundane and perhaps more “famous” web metrics.

I am sure most government or normal websites jump to that first. And why not, they are all staring you in the face when you crack open any analytics tool.

The problem is that these aggregate metrics barely contain any insight. If you focus on them, you’ll be left holding a empty bucket / cry a lot / get fired / not get your government pension / dread meetings with your boss.

 I hope the above ideas inspire you to do more, go beyond the obvious and less than useful.

One last quick example. . .

You can use the same strategy for other sites. Though remember the job the site is trying to do and the desired outcomes will decide which key performance indicators you end up using.

For example for www.recovery.gov in addition to some of the metrics above I would probably also measure Visitor Loyalty and Recency. That’s because the government wants the data provided to be so sticky, and it is updated frequently, that it wants you to come and check it again and again.

In this case perhaps more than downloads I would also measure # of customized graphs created. When I measure content consumed (#3 above) I’ll probably focus on understanding which departments get looked at more on the site (are they the ones most spending money?).

You can also bet I am going to be totally on top of reporting how many complaints we have received on the site for Fraud, Waste & Abuse! Getting a ton of those would be a key performance indicator! : )

Makes sense?

Don’t despair just because you have a government site. Ignore the obvious. Focus on the site’s jobs. Identify key outcomes. Do productive analysis.

Good luck.

Ok now it’s your turn.

Are you responsible for a government website? What are your key performance indicators? What web metrics are important to you? Do you use any of the above strategies? If not, why not? Have you looked at www.belgium.be? What would you have recommend that I did not?

Please share your valuable advice / insights / feedback / critique (on my blog).

Thanks.

PS: Like this post? Perhaps you’ll consider ordering my * new * book: Web Analytics 2.0.

PPS:
Couple other related posts you might find interesting:

• Brand Measurement: Analytics & Metrics for Brand Websites
• Measuring Success of Non Ecommerce Websites
• Measuring Web Analytics Success for Small Business Websites
• Outcomes, baby!! Measure Macro AND Micro Conversions
• Six Recommendations For Measuring Success of Your Blog