I noticed the ad below in Gmail today.
This may seem unremarkable until I tell you that my office is on Eastlake Avenue in Seattle. Wow.
From Google’s perspective, the best part of this is that because the ad in unobtrusive and relevant, it’s actually a positive experience for me.
The best way to figure out what works and doesn’t work in your product is to watch a member of your target audience using it. Prior to our recent release we conducted a number of simple focus groups at Judy’s Book where we had people come into the office for individual 30 minute sessions with our site.
One point to note here: don’t ask users what they want. Observe them doing the actions they want to do. This will tell you far more. Also, while individual sessions may seem inefficient, they avoid group think, so you get an honest opinion from each person rather than one collective opinion.
The Process:
The sessions had the following format:
What we learned:
Putting even a handful of users through this exercise is incredibly instructive, and humbling. No matter how well you think your site works, watching someone new try to use it makes you cringe from time to time. When you’re closely involved with something it’s hard to see all its flaws but three 30 minute sessions with new users will bring them all out into sharp relief.
We learned a lot from these sessions but in this post, I’m going to focus on the challenge of displaying online and local deals on the same site. Prior to this release at Judy’s Book, our site showed users online only deals by default and then after clicking on a link labeled “View Local Results” you would be shown the local deals that were relevant. This seemed logical enough until we asked a user to see if there were any local deals she cared about. She didn’t have a clue how to proceed.
Once we showed her, we then asked her to find a deal from Amazon and she started trying to find it in the
Seattle store directory. After seeing a couple of users proceed in this way, it became clear that the distinction between online and local deals was meaningless to users. They just wanted to see the deals relevant to them.
From the user’s perspective, it became clear that they wanted to see the deals they could access in one place. Sitting in Seattle, I can buy from my local Target, but I can also buy from Amazon.com. The distinction between online and local isn’t relevant when I’m browsing deals - it only becomes relevant when I try and act on what I see.
What we did:
This insight led to a fundamental change to the site. Instead of creating silos of online and local content, we blended the two. As a result, users see all deals that are relevant to them, whether online or local. Using filters in the left nav, they can narrow the list if they want.
As a result, on our Seattle Deals page, a user now sees our best local and online deals blended together. This is a much more logical and natural experience. You don’t have to toggle between “Online” and “Local” to find Amazon.com versus your local Target. In hindsight, we probably should have thought of this earlier, but watching a real user wrestle with this issue in person really hammered the point home.
Going Forward:
It’s hard to find the time to invite people in and have them use your site but at the end of the day, it’s a very cost-effective way of improving things. Even though it can be hard at times, and it inevitably leads to changes which can also be problematic, simple usability testing like this is critical. The payoff in terms of feedback, for what is at the end of the day, a very small amount of time and money, is staggering.
If you’re using the Better Gmail extension in Firefox, you can get a right-click message preview in Firefox.
The goal of enabling users to scan messages without unnecessary clicks is a noble one, but I don’t think it should come at the cost of the expected right button behavior. I think a mouseover preview would have been better.
We’ve been working hard to find creative ways to get hard to find local content into the hands of users - basically, doing work so users don’t have to. As most people in and around the local online sector know, it’s hard to get great local content. I think we’re starting to figure it out - check out our Seattle and Atlanta local deals pages.
The deals rock and they are relevant and unique - this data isn’t available elsewhere on the web. I’m really excited about the quality of the deals on these pages and I’m looking forward to them getting even better.PS: If you want to sign up for a weekly email with the top 10 local deals, just click on “Get a Deal Alert” in the left nav of the pages above.
Seth’s post on follow through and caring about the last inch is a must-read if you haven’t read it already.
Obsessing about the last inch of follow through ensures that the important parts of what you do get just as much (if not more) commitment.
You can’t afford to stop caring about the little things. The equivalent argument when it comes to web sites is: “Well, so few people see that page, we can leave it looking crappy.”
This is incredibly dangerous thinking. The truth is that sometimes you have to cut corners to get something released, but you can’t accept leaving things in a shitty state and you better make sure you come back around and fix it. (By the way, the reverse, obsessing about the little things, is often how great products get built.)
If something isn’t worth doing right, don’t do it, or kill the feature. Otherwise, if it’s up and viewable to your users, then make sure it’s your best work.
Ok, sorry, couldn’t resist the A-Team quote. I just got an email from Casey, one of our developers, who said:
Couponlooker saved me 10% at Godaddy.com <eom>
That’s the kind of email you love to get.
Last week, we rolled out some small tweaks to Couponlooker that should make it even better for finding online coupon codes.
Improving search results is a gradual process and the Tweak-Measure-Iterate cycle is in full effect.
With last night’s release, as Dave mentioned, there were a bunch of changes to Judy’s Book and I’ll be writing more about those next week as we shake out some of the outstanding issues. One feature that I’d like to mention in the near term is Deal Alerts. In the left nav on every page, you’ll see a box that allows you to sign up for a Deal Alert for the page that you’re viewing.
The signup process is streamlined and after you confirm your email address we’ll send you new deals that match the criteria you select. So, for example, you could sign up for “Seattle’s Best Shoe Deals” and receive that regularly via email. In keeping with the notion that you should control the communication you get, you can manage all your alerts from your profile. Membership is not required to sign up but if you do become a member, we bring all your preferences forward.
Like any new feature, perfecting this will take work, but I’m thrilled to have this released. In general, in spite of some issues we need to work out, last night’s release was a big one for us.
When you’re evaluating whether or not your site is meeting its user engagement goals, it’s important to look at the right metrics. What’s right for a social network is not right for a search engine. By the way, if you’re not measuring things like visits, page views/visit, time on site, etc you should start immediately. (We use Google Analytics at Judy’s Book.)
In my notes from the Facebook Seattle Garage, I mentioned that the FB team was talking a lot about page views/user, time on site etc. These metrics make a ton of sense for them. Their product is all about user’s spending most of their online time adding to their data on Facebook, acquiring new friends, downloading new apps, etc.
If you take Couponlooker, these metrics don’t make a ton of sense. Our goal with couponlooker is to have users come to the site, find the coupon code they are looking for in 1-2 pages and then leave. If we see page views/user and time on site start to spike, it could mean that users just love searching for coupon codes, or more likely, it means that there’s something wrong with our search relevance. In Couponlooker’s case, we’re looking for 2-3 page views/user (1-2 pages of search results and a click) and we’re looking for a relatively short time on site. Our goal is to satisfy a user’s need quickly and have them return in the future. Naturally, repeat visits and direct traffic (users who typed in or bookmarked your site) are the life blood of any site.
At the end of the day, you’ve got to measure your performance to see how you’re doing and to improve. Just make sure you measure the right things. Figure out the optimal user behavior and the metrics should be easy to figure out.
I recently signed up for traffic alerts from Traffic.com. Ostensibly interesting and useful, in reality, totally worthless. They don’t put any information in the subject line of the email and as a result I have to click on each one to see it. Given that most days traffic alerts contain no new information (i.e. usual traffic volume), the alerts get annoying very quickly. While this strategy may inflate open rates in the short term, I’ve got to imagine it leads to higher than usual unsubscribe rates.
If Traffic.com put a summary of the message in the subject line with more details in the body, I would have been much more likely to continue with my subscription. This isn’t hard to do, it just requires putting the user first.
