Helping Virginians vote

Monday, October 26, 2009 | 5:25 PM

Last year we worked with partners to launch tools that made it easier to find basic voting information like when to register, where to vote, and how to contact your local election office.

We're back at it this year, helping Virginians vote in the upcoming 2009 general election with the Virginia Voting Info Map and gadget, which uses a data feed built by the Virginia State Board of Elections.

If you vote in Virginia, you can enter the home address where you're registered and receive the following information:

  • Your polling place address and directions to get there
  • Candidates on your ballot, along with a link to their websites (when available)
  • Absentee voting information
  • Your local election office address and phone number
Here's a working version for you to try out:



You can easily add the gadget to your site, as the Bob McDonnell campaign has done.

The Voting Information Project, founded by the Pew Center on the States and Google, works with election officials to organize voting data across the United States, giving voters and developers easier access to useful election information. Stay tuned to learn about ways you can help bring the project to your county or state in preparation for the 2010 elections.

Posted by Dan Berlin, Public Sector Engineering Manager

Web Analytics Success Measurement For Government Websites

Tuesday, October 13, 2009 | 10:54 AM

[Cross-posted from the personal blog of Google Analytics Evangelist Avinash Kaushik]

Prickly Problem If you know what the desirable outcomes are from your website, it is not hard to measure performance of the website for you and your customers.

Measuring top line success of ecommerce websites is not very complicated, all the sweet revenue based outcomes are there (at the least).

Measuring non-profit websites is a bit complicated, but not really all that hard because we can, with a small amount of love, figure out outcomes to focus on (donations, # of sign-ups for the protest in DC, # of petitions signed, volunteer applications, etc).

Measuring government websites is a bit more complicated, if for no other reason than that it takes a pinch of effort with a dash of imagination to figure out what one is solving for. What are the desirable outcomes one can focus on to measure success?

The above question came to mind from a kind note I got from Ines Jans who is a part of the team that is responsible for www.belgium.be
belgium.be
Ines and team were just starting to think about analytics (because the love their customers!) and asked for some thoughts.

My first question to Ines was, (surprise!):
Q: Tell me a bit more about what your site does, like what are the real goals (or give me some ideas about it) and what challenges you face, what do you expect people to get out of it?
[Best Practice: Always, always, always start any measurement conversation with the above inquiry. The answer will be key to insights, and without it you'll simply be a glorified Reporting Squirrel.]
The answer, which might fit most government websites was:
A: The goal of our site is to be a portal to all the official information there is about Belgium and make information easy to find. Visitors should be able to figure out which Ministry is responsible for what tasks.
[Best Practice: Don't be surprised in your Analysis Ninja quest if you get answers that just start the conversation, rather then give you a prescription for what you need. Squirrels will despair here, but Ninjas will take clues from what they hear and visit the site and come up with a set of important measurable outcomes.]

Based on the answer above and some time spent on the English language site as well as those in other languages (Google Translate!), I came up with the following five questions I could ask data to measure success.

~ Are Visitors able to find the information they are looking for?

~ Are the Visitors satisfied with their experience?

~ What is the most popular content on the site? What area can we prioritize higher than it currently is?

~ How long does it take for someone to find key information they want?

~ Does the right information actually exist on the website? What major things might we be missing on our website?

Let’s take each of these questions one at a time and figure out the best way to answer each using a true Web Analytics 2.0 strategy.

Q1. Are Visitors able to find the information they are looking for?

Given the singular purpose in life of this government website is to be the one stop shop for all the information one could possibly need, it should be pretty obvious that the very first, and magical, thing we would measure is if Visitors to the site are able to find what they might be looking for.

So would you use a web analytics tool?

Here’s the first surprise: No!

There are certainly tertiary ways in which you can answer this question using Omniture’s Site Catalyst or Google Analytics or other wonderful web analytics tools.
website task completion rate
But the best way to answer this question?

Ask the Visitors!

Using a simple survey that pop-up on-exit (when Visitors leave the website) you can ask your customers to tell you if they were able to complete their task. No interpretation required.

4Q from iPerceptions, available in 18 languages, is a free on-exit survey you can use. If you don’t want to use a external survey build your own, ask four questions, analyze the data for:

Were you able to complete the purpose of your visit today?

The answer to this question becomes the #1 Key Performance Indicator (KPI). You are going to watch it like a hawk, you’ll post it on all your bulletin boards, you’ll set up custom alerts to ensure that your team gets a small electric shock every time this number drops below 65%!

The overall number is good enough, but the data that will be awesomely actionable will be, if you use 4Q: Primary Purpose by Task Completion Rate. . .
primary purpose by task completion rate

You see the second question in 4Q is “Which of the following best describes the primary purpose of your visit?” and a standard report in 4Q will paint the above picture.

Now you not only know if people find what they are looking for, but you also know which tasks are hard to complain.

You need to fix “Complain about the French” : ) because the Visitors are already upset and only 5% are able to complete their task, resulting in them becoming even more mad!

Remember: You don’t need to show the survey to everyone who comes to your site. You can sample just a small percent of your Visitors. You only need 300 responses in a month to get a statistically significant sample of data, and 1,200 if you want to do segmented analysis.

Q2. Are the Visitors satisfied with their experience?
HiPPO’s (the “highest paid person’s opinion”) in the organization, even in the government will love to have a more direct (than task completion rate) answer to the question: Are our Visitors happy with our website?

That’s were it is prudent to measure Customer Satisfaction.

4Q and other surveys of course measure that quite easily: Based on today’s visit, how would you rate your site experience overall?

Measure it. Trend it. Report it. Correlate the trend over time with changes you have made to the site and identify insights (any causal connection between site improvements / campaigns and customer satisfaction?).

An alternative, or additional, way to measure satisfaction is to count and analyze the Contact Us submissions. . . .
belgium.be contact us form
Start with the number of submissions. Trend over time.

Drill down into the type of complaints and do atleast rudimentary sentiment analysis (i.e. read & categorize) of actual messages to gauge customer satisfaction.

Remember: When you do surveys you don’t have to torture your Visitors with billions of questions! In researching this post I went to US government sites and I got a ugly 34 question on one single page looooong survey. 34 questions! Most were irrelevant. I would have answered a few, but this showed a fundamental disrespect for your customers. In the end your Visitors are upset and you suffer from a lack of data.

Only ask what you can action.

Q3. What is the most popular content on the site? What area can we prioritize higher than it currently is?

It is not unusual for content sites to produce content. It is even less unusual for them to produce content that they think potential visitors to the site might want.

What is rare is the analysis of what visitors to the site are actually consuming on the site.
Here’s a simple analysis I had learned from Tim Hart (who was with the J. Paul Getty Trust): Measure the distribution of content in each section of your website and the percentage of Visits to each section.
content vs visit distribution
On the y-axis is each of the sections on the belgium.be website. In blue is the amount of content in each section. In red are the percentage of visits where that content was consumed.

Is it not awesome! Insights galore!!

If this were their data, and it is not, it would be pretty obvious that there is huge interest in content about Housing and Economy tiny fraction of the site’s content is about Housing and Economy.

The balance for Family is a lot less lop sided.

While the government might love Justice, Mobility and Health (and boy do they love Environment!), Visitors to the site are a lot less interested in those pieces of content.

Action? You know what people want, how about giving them more of that content?

When it comes time to prioritize the next set of web pages or videos or podcasts, how about giving higher priority to those big red lines?

Sweet right?

You can also do segmented version of this analysis, see what Visitors to English, Dutch, French and German sites prefer. Or within Family what group of content do people like. Etc etc.

Two more ideas to get into your Visitor’s head. . . .

Measure Downloads:

There are a ton of downloads (pdf’s mostly) on the belgium.be website. Forms, applications, useful guides (like how to marry a belgian or how to prepare for your first job) etc.

It is a trivial cost, analytically, to track these downloads using your web analytics tool. Do it. Measure what your Visitors are most interested in.
tracking downloads

Yes, yes, yes I see my technical squirrel friends raising their hands and saying you can only track that someone clicked on the download link and not that the download was successful. I know.

For our analysis here just intent is fine.

In fact unless a vast majority of your Visitors are connecting using dial up it is safe to assume the download of small files went through. I know that does not make the squirrels happy. I am sorry. You keep squirreling while we make decisions about how to improve the site.

Outbound Link Tracking:

Another thing you’ll notice about the website (see why it helps to surf a site you are supposed to analyze?) is that there are a ton of links on the site that point to other government websites.
Track ‘em!
outbound link tracking
Of course the above is not their data :), it’s just an illustration of how absolutely easy it is to track this data.

From the report it is very easy to then figure out what links your Visitors click, which is a great, positive, indicator of the fact that they found what they wanted and also what they were interested in.

Remember: It is not very hard to do any of the above three types of analysis. All you need to get into your customer’s head is move away from “Top Pages Viewed” and “Page Views Per Visitors” and think a bit more creatively.

Q4. How long does it take for someone to find key information they want?

There are some pieces of content that are so darn important that they are heavily linked (say latest news in case of belgium.be) right from the home page, or that you really do want people to find them asap (in the Health section for example the pdf about how to deal with H1N1 virus in belgium).
For these important pieces of content measure Average Time To This Page.

average time to this page

That’s almost three minutes from the time that someone entered the website to the time they found this page (say the one about swine flu).

On average people give a page two and half seconds before they click/leave. Consider how long three minutes is, and how many people might have given up in the process of finding this key information.

Unfortunately not too many tools, Google Analytics and other Paid Solutions included, provide this as a standard metric. I use ClickTracks and that this delightful metric as a standard offering.

I wish others would have it.

Remember: You can use this data to ensure that your best information is found by Visitors to your site quickly. Fix your top / left / right / bottom / whatever navigation you have on the site. Consider creating a prominent “box” on the top right where you “merchandize” these important links. More things like that.

Q5. Does the right information actually exist on the website? What major things might we be missing on our website?

I have consistently advocated my love for internal site search analysis. It is simply da bomb!

Like many other sites belgium.be has a internal site search engine. Typically Visitors who have a harder time with normal navigation (or limited data on a page) will make liberal use of this site search box.

Why not use that data?
internal site search analysis

Again, this is not their data :).

I recommend looking at the top typed search terms by the Visitor but then also looking at the metric: % Search Exits.

That’s the bounce rate of your search results page. I.E. People come to belgium.be, search for the term hippo and the search results are so bad that 33.33% of the people exit from that page! They don’t even bother to do anything. Just bail. Bounce. Kaput!

Now you know both 1. what information they were looking for, 2. what search results stink and 3. likely because you don’t have the right or enough content about that keyword on your site.

Fix it!

I have one more idea to understand if you are missing information that your Visitors want on your government website.

Use Page Level surveys.

turbotax page level survey-1[1]

There are free page level surveys available or you can build your own (like the one above from a software vendor’s website).

These can be an excellent way to understand what content is missing from your website. You can of course also use the open text voice of customer (VOC) from surveys like 4Q, look for Visits with Task Completion = No.

Remember: These surveys don’t collect any personally identifiable information (PII) information, and that goes for your web analytics tools as well. Many government sites are extra concerned about privacy, as they should be. Do Please familiarize yourself with the privacy policies of the vendor.

Note: What was not tracked or emphasized. . .

Visits.

Unique Visitors.

Page Views.

Time on Site.

And so many more mundane and perhaps more “famous” web metrics.

I am sure most government or normal websites jump to that first. And why not, they are all staring you in the face when you crack open any analytics tool.

The problem is that these aggregate metrics barely contain any insight. If you focus on them, you’ll be left holding a empty bucket / cry a lot / get fired / not get your government pension / dread meetings with your boss.

 I hope the above ideas inspire you to do more, go beyond the obvious and less than useful.

One last quick example. . .

You can use the same strategy for other sites. Though remember the job the site is trying to do and the desired outcomes will decide which key performance indicators you end up using.
recovery.gov

For example for www.recovery.gov in addition to some of the metrics above I would probably also measure Visitor Loyalty and Recency. That’s because the government wants the data provided to be so sticky, and it is updated frequently, that it wants you to come and check it again and again.

In this case perhaps more than downloads I would also measure # of customized graphs created. When I measure content consumed (#3 above) I’ll probably focus on understanding which departments get looked at more on the site (are they the ones most spending money?).

You can also bet I am going to be totally on top of reporting how many complaints we have received on the site for Fraud, Waste & Abuse! Getting a ton of those would be a key performance indicator! : )

Makes sense?

Don’t despair just because you have a government site. Ignore the obvious. Focus on the site’s jobs. Identify key outcomes. Do productive analysis.

Good luck.

Ok now it’s your turn.

Are you responsible for a government website? What are your key performance indicators? What web metrics are important to you? Do you use any of the above strategies? If not, why not? Have you looked at www.belgium.be? What would you have recommend that I did not?

Please share your valuable advice / insights / feedback / critique (on my blog).

Thanks.

PS: Like this post? Perhaps you’ll consider ordering my * new * book: Web Analytics 2.0.

PPS:
Couple other related posts you might find interesting:

Open Government and the Federal Register

Wednesday, October 7, 2009 | 5:18 PM

[Cross-posted from the Google Public Policy Blog]

For more than 70 years, the Federal Register has been the official daily record for federal agency rules and notices, executive orders, and other presidential documents. But at 80,000 pages per year, it lands on your desk with a thud.

That's one reason why we're excited to see that the U.S. government is taking steps to make the Federal Register more open and accessible to citizens. Starting today, the Federal Register will be published in XML format to Data.gov, which will allow third-parties to develop new ways to organize, re-organize, and analyze its contents. Past issues of the Federal Register dating back to 2000 will also be posted online.

We're already seeing cool new tools emerge around this data. This morning Ed Felten of Princeton's Center for Information Technology Policy announced the launch of Fedthread.org, which allows users to annotate and search the Federal Register. The tool also allows users to set up customized RSS feeds for specific search queries, which will allow users to track items and issues over time. Amazingly, the project took only 10 days to create.

The Washington Post describes the Federal Register as the "de facto daily newspaper of the executive branch." Making the Federal Register available on Data.gov is an important step towards making that newspaper more accessible to citizens across the country.

Your world, your base map

| 5:09 PM

[Cross-posted from Google's Content Central Blog]


Today we released new base map data for the U.S. in Google Maps, Google Earth and other Google services. We developed this map by integrating data from many sources, including datasets provided by government agencies and other organizations with an interest in making their mapping data widely available.

One such organization is the USDA Forest Service, which provides comprehensive and detailed data for all of the national forests within the U.S. Using this dataset, we've been able to show the boundaries of U.S. National Forest-managed forests and grasslands in our base map, making these lands more visible in Google Maps. We've also used Forest Service data to improve roads and trails on these public lands.


"An important part of the mission of the USDA Forest Service is informing the public about the forests and grasslands we manage. In order to achieve this, we want information about our lands to be accessible to as large an audience as possible," said David George, Geospatial Service and Technology Center, USDA Forest Service. "We are very pleased that Google has used our geospatial datasets to improve the base map in Google Maps and Google Earth so that everyone can easily discover, navigate and enjoy these public lands."


National parks, forests and other public lands in the Pacific Northwest, based on data from the USDA Forest Service, the U.S. National Park Service and other sources 

We've also created the Base Map Partner Program to enable organizations with mapping data to contribute their data to Google Maps and Google Earth. This program, like those we've introduced for other map content types, offers organizations with mapping data -- and an interest in making it more visible and useful to internet users -- a straightforward process for adding this data to Google's services. We now provide specifications on the types and formats of base map or "vector" data we welcome, including new roads, housing developments, parcel data, points of interest and bike trails. We hope that organizations with mapping data, such as local governments, regional GIS consortia or real estate developers, will take this opportunity to improve and enrich our base map for their community.

If the opportunity is of interest to you, tell us about your organization and your data. If you don't have large datasets to provide, but do have edits to the map to suggest, then try our tool for easily suggesting edits.

We look forward to hearing from you.

Posted by Megan Quinn, Business Product Manager

Google for the Public Sector lands in Oz

Thursday, September 17, 2009 | 6:43 PM

(Cross-posted from the Official Google Australia Blog)

The Australian public sector has a great track record of embracing the web. Australia was the first country to launch a Google election site in 2007, the NSW government map makes public spending more transparent, and the recently announced Government 2.0 taskforce has been charged with accelerating Government onto the web.

Australians spend more time online than on any other media, and more people access government information online than in any other way. The Internet makes government more transparent and accountable and has opened up communication between politicians and their constituents.

In this spirit Senator Stephen Conroy, Minister for Broadband, Communications and the Digital Economy, tonight launched Google for the Public Sector, www.google.com.au/publicsector, a guide to the tools and best practice for the public sector to reach, communicate and engage with their communities.

Many of these products are used by governments around the world to:
To see these tools in action:


Open Source report from Gov 2.0

| 5:41 PM

(Cross-posted from Google's Open Source Blog)

Clearly designed as a conference to start but certainly not finish the conversation, last week's Gov 2.0 Summit assembled an impressive cast of presenters and interviewers. Key White House decision makers, government innovators and industry enthusiasts took the stage and lined the hallways for three days.

Having spent the last five years focusing on helping government adopt Open Source software and its collaboration model, my radar was tuned for explicit mentions / inclusions / endorsements of Open Source software. It appeared that leveraging Open Source software to solve some of the thornier technology problems challenging government (think healthcare and public safety interoperability) had been more implied than expressed in recent months. For the wider community looking for more signs of game change, the event provided plenty of evidence that Open Source is clearly at play.

Paul Rademacher, Google and AllForGood.org, Brian Behlendorf, White House Consultant and Richard Lin, CodeforAmerica.org

The Google-hosted reception Tuesday evening packed the public space at their headquarters on New York Avenue. The event was attended by private industry, publicists and social media converts, non-profit and Open Source community leadership and government attendees and offered a nice opportunity to mix it up after a day of the Gov 2.0 Expo Showcase I sadly missed. Some of the sessions however are video-archived on the web.

Lifting off in a small flurry of debate over the right hash tag for the Gov2.0 Summit, the two day Gov2.0 Summit opened with the and energy and grin of Aneesh Chopra, Federal Chief Technology Officer. Chopra earned a reputation for creative collaboration with industry in his prior role as the Secretary of Technology for the Commonwealth of Virginia and brings the same to the federal scene. Virginia's extensive use of Open Source and open collaboration, as well as that of former D.C. CTO — now Federal Chief Information Officer Vivek Kundra, is well known.

The conference brought attendees through a whirlwind tour of recent innovation in government IT: data transparency projects like Apps for Democracy and resulting mash-ups and visualization as inexpensive and "dirty" Open Source solutions to real problems. Open Source and its exceptional benefits of open standards and interoperability were highlighted in many presentations.

Conference highlights:
  • Beth Noveck provided the most comprehensive picture of what progress had been made by the new administration and its policy road map.
  • Best of Show for Crowd-Rallying: Carl Malamud discussed the need to make judiciary information — data and hearings — truly public in a day where “public” means “on the Internet.” In his speech designed in part for an audience not in the room, his closing comment asserted government operating systems should be Open Sourced brought the crowd to resounding applause.
  • Favorite Projects: Anything visualized — and most frequently enabled by Open Source.
  • Killer App: All things Geo-spatial.
  • Significant Announcement: The General Services Administration (GSA) will begin experimenting with the use of OpenID to manage identity on government web sites.


David Recordon, OpenID Foundation Board of Directors

For the seasoned government attendees, there was in reality not a great deal of new information to be had. That was, in fact, good news; as one government manager shared with me, social media tools like Twitter and GovLoop have made it much easier to stay in touch with what other agencies are up to, plus the 2009 Federal IT Strategy has been broadly distributed and much discussed internally.

The White House will release its new Open Government Directive in a few weeks and will set federal agency wheels in motion. Implementation will be challenging and require the philosophy of change to shift into gear. Industry and government seem to agree that the next non-trivial challenge to technology innovation will be procurement reform.


Guest post by Deb Bryant, Public Sector Communities Manager, Oregon State University Open Source Lab and Producer, Government Open Source Conference

Google Australia: Public sector tips from Down Under

Monday, September 14, 2009 | 10:17 PM

Our Australian team keeps the globe on its toes - they were the first to launch a Google election site. Over the next few days they will be posting best practices from down under - stay tuned!

Posted by Ginny Hunt, Public Sector team