The best way to figure out what works and doesn’t work in your product is to watch a member of your target audience using it. Prior to our recent release we conducted a number of simple focus groups at Judy’s Book where we had people come into the office for individual 30 minute sessions with our site.

One point to note here: don’t ask users what they want. Observe them doing the actions they want to do. This will tell you far more. Also, while individual sessions may seem inefficient, they avoid group think, so you get an honest opinion from each person rather than one collective opinion.

The Process:

The sessions had the following format:

  • Initial impressions with no guidance - we just put the site up on a screen and asked for their reactions.
    • What do you think?
    • Who do you think this site is for?
    • What catches your eye?
    • What’s the first thing you would click on?
  • Targeted Questions
    • How would you find deals in your city?
    • What does that heading mean to you?
    • If you click on that link, where do you think it would take you?
    • What do you think of the store logos on the right of the page?
  • Observing Simple Tasks
    • If you were looking for a digital camera deal, how would you go about doing it?
    • If you wanted to find deals from Amazon, how would you start?
    • If you were looking to see if a store offered free shipping, how would you go about it?

What we learned:

Putting even a handful of users through this exercise is incredibly instructive, and humbling. No matter how well you think your site works, watching someone new try to use it makes you cringe from time to time. When you’re closely involved with something it’s hard to see all its flaws but three 30 minute sessions with new users will bring them all out into sharp relief.

We learned a lot from these sessions but in this post, I’m going to focus on the challenge of displaying online and local deals on the same site. Prior to this release at Judy’s Book, our site showed users online only deals by default and then after clicking on a link labeled “View Local Results” you would be shown the local deals that were relevant. This seemed logical enough until we asked a user to see if there were any local deals she cared about. She didn’t have a clue how to proceed.

Once we showed her, we then asked her to find a deal from Amazon and she started trying to find it in the
Seattle store directory. After seeing a couple of users proceed in this way, it became clear that the distinction between online and local deals was meaningless to users. They just wanted to see the deals relevant to them.

From the user’s perspective, it became clear that they wanted to see the deals they could access in one place. Sitting in Seattle, I can buy from my local Target, but I can also buy from Amazon.com. The distinction between online and local isn’t relevant when I’m browsing deals - it only becomes relevant when I try and act on what I see.

What we did:

This insight led to a fundamental change to the site. Instead of creating silos of online and local content, we blended the two. As a result, users see all deals that are relevant to them, whether online or local. Using filters in the left nav, they can narrow the list if they want.

As a result, on our Seattle Deals page, a user now sees our best local and online deals blended together. This is a much more logical and natural experience. You don’t have to toggle between “Online” and “Local” to find Amazon.com versus your local Target. In hindsight, we probably should have thought of this earlier, but watching a real user wrestle with this issue in person really hammered the point home.

Going Forward:

It’s hard to find the time to invite people in and have them use your site but at the end of the day, it’s a very cost-effective way of improving things. Even though it can be hard at times, and it inevitably leads to changes which can also be problematic, simple usability testing like this is critical. The payoff in terms of feedback, for what is at the end of the day, a very small amount of time and money, is staggering.