Rahul Pathak : My Personal Blog

Last Friday, we had an important release at Judy’s Book which included several new features. Some visible to consumers, others more in the back end, but all of which should lead to higher quality deals on site.

The primary elements of this release were:

• Deal Approval Queue
• User Posted Local Deals (supported by Local Editors in select geographies)
• Map Improvements
• Fall Promotion Support 

You can see some of the results of this work in Seattle, New York, Atlanta and Chicago.

The entire team came together and cranked to make this happen in a very short period of time and I’m really excited by how things turned out.

For those of you who are interested in the gory details:

Deal Approval Queue

The intent here was to provide a way to support UGC but to enforce quality standards. User contributed deals are of no value if they are spammy, irrelevant, miscategorized etc. The approval queue allows us to solicit content from users but provides a mechanism for us to ensure that only high quality deals make it onto the site. All user posted deals go into the Deal Appproval Queue and remain there until they are explicitly approved. The system also stores the history of approved and rejected deals so we can use it to train spam detection systems.

Content quality is a critical part of the user experience (something Erin & Chris have pounded into me - thanks guys) and you need to be very focused on keeping it high.

User Posted Local Deals

To this point, we had supported adding online deals to the system but we didn’t have a public way for users to contribute local deals. One aspect of the local deal post process that creates complexity is the fact that you have to provide a way for users to add business listings without creating duplicates in your data. In addition, no one wants to type in a complete address, so you have to facilitate search. Oh and by the way, you need to prevent people from spamming your listing database. (Dave has a great post on some of the fun things that happen in local listing spam.)

The Deal Approval Queue was the strategy we came up with to control the flow of data into the system. Because we had a mechanism for screening content, we wanted to make it as easy as possible to participate. This led to us to move from a model where posting was for Judy’s Book members only to a model where all you needed was an email address. If at some point in the future, you sign up for membership, all your deals will be available in your profile.

In addition to supporting user generated content, we’ve been actively recruiting local editors in select geographies to ensure a high quality base of local content. Erin’s written about this on the Judy’s Book blog and I’d touched on the subject earlier in a more generic way in my post on aggregation and UGC.

Map Improvements

The map now displays stores with the most popular local deals in any given category. In addition, you can pan and drag the map and it will continually update in order to provide you with a list of the best local stores that it can.

Fall Promotion Support

We’re currently running a daily giveaway where users can win a $25 gift certificate to a local boutique of their choice and there’s a $1,000 grand prize at the end of the promotion. We needed to put in a significant amount of work to support the marketing efforts around this promotion. The goal was to provide very prominent placement without being intrusive or degrading the user experience for those who weren’t interested in the promotion.

I think the team did a great job coming up with an implementation that works. If you come to the site looking for the promotion, or click on a promotion call out, we use javascript to grey out the screen and present you with an overlay that provides a streamlined way to participate. In addition to signing up with your email address, if you post high quality local deals, you gain additional entries into the giveaway.

The user flow around the promotion is quick and easy when you step through it (as it should be). Features that are easy to use, however, are typically features that required a great deal of thought and effort. This implementation was no exception. In order for something to feel easy for a user, a lot of people put a lot of work in behind the scenes.

Sorry for the recent rash of local deals posts, I’m just excited about the direction we’re going in. I was just checking out the Seattle Deals page today and came across information on a sale at Ian, a clothing boutique in my neighborhood. I had no idea this was taking place. It’s really cool to benefit from the value proposition we’re trying to deliver for our users.

The best way to figure out what works and doesn’t work in your product is to watch a member of your target audience using it. Prior to our recent release we conducted a number of simple focus groups at Judy’s Book where we had people come into the office for individual 30 minute sessions with our site.

One point to note here: don’t ask users what they want. Observe them doing the actions they want to do. This will tell you far more. Also, while individual sessions may seem inefficient, they avoid group think, so you get an honest opinion from each person rather than one collective opinion.

The Process:

The sessions had the following format:

• Initial impressions with no guidance - we just put the site up on a screen and asked for their reactions.
  • What do you think?
  • Who do you think this site is for?
  • What catches your eye?
  • What’s the first thing you would click on?

• Targeted Questions
  • How would you find deals in your city?
  • What does that heading mean to you?
  • If you click on that link, where do you think it would take you?
  • What do you think of the store logos on the right of the page?

• Observing Simple Tasks
  • If you were looking for a digital camera deal, how would you go about doing it?
  • If you wanted to find deals from Amazon, how would you start?
  • If you were looking to see if a store offered free shipping, how would you go about it?

What we learned:

Putting even a handful of users through this exercise is incredibly instructive, and humbling. No matter how well you think your site works, watching someone new try to use it makes you cringe from time to time. When you’re closely involved with something it’s hard to see all its flaws but three 30 minute sessions with new users will bring them all out into sharp relief.

We learned a lot from these sessions but in this post, I’m going to focus on the challenge of displaying online and local deals on the same site. Prior to this release at Judy’s Book, our site showed users online only deals by default and then after clicking on a link labeled “View Local Results” you would be shown the local deals that were relevant. This seemed logical enough until we asked a user to see if there were any local deals she cared about. She didn’t have a clue how to proceed.

Once we showed her, we then asked her to find a deal from Amazon and she started trying to find it in the
Seattle store directory. After seeing a couple of users proceed in this way, it became clear that the distinction between online and local deals was meaningless to users. They just wanted to see the deals relevant to them.

From the user’s perspective, it became clear that they wanted to see the deals they could access in one place. Sitting in Seattle, I can buy from my local Target, but I can also buy from Amazon.com. The distinction between online and local isn’t relevant when I’m browsing deals - it only becomes relevant when I try and act on what I see.

What we did:

This insight led to a fundamental change to the site. Instead of creating silos of online and local content, we blended the two. As a result, users see all deals that are relevant to them, whether online or local. Using filters in the left nav, they can narrow the list if they want.

As a result, on our Seattle Deals page, a user now sees our best local and online deals blended together. This is a much more logical and natural experience. You don’t have to toggle between “Online” and “Local” to find Amazon.com versus your local Target. In hindsight, we probably should have thought of this earlier, but watching a real user wrestle with this issue in person really hammered the point home.

Going Forward:

It’s hard to find the time to invite people in and have them use your site but at the end of the day, it’s a very cost-effective way of improving things. Even though it can be hard at times, and it inevitably leads to changes which can also be problematic, simple usability testing like this is critical. The payoff in terms of feedback, for what is at the end of the day, a very small amount of time and money, is staggering.

We’ve been working hard to find creative ways to get hard to find local content into the hands of users - basically, doing work so users don’t have to. As most people in and around the local online sector know, it’s hard to get great local content. I think we’re starting to figure it out - check out our Seattle and Atlanta local deals pages.

The deals rock and they are relevant and unique - this data isn’t available elsewhere on the web. I’m really excited about the quality of the deals on these pages and I’m looking forward to them getting even better.PS: If you want to sign up for a weekly email with the top 10 local deals, just click on “Get a Deal Alert” in the left nav of the pages above.

With last night’s release, as Dave mentioned, there were a bunch of changes to Judy’s Book and I’ll be writing more about those next week as we shake out some of the outstanding issues. One feature that I’d like to mention in the near term is Deal Alerts. In the left nav on every page, you’ll see a box that allows you to sign up for a Deal Alert for the page that you’re viewing.

The signup process is streamlined and after you confirm your email address we’ll send you new deals that match the criteria you select. So, for example, you could sign up for “Seattle’s Best Shoe Deals” and receive that regularly via email. In keeping with the notion that you should control the communication you get, you can manage all your alerts from your profile. Membership is not required to sign up but if you do become a member, we bring all your preferences forward.

Like any new feature, perfecting this will take work, but I’m thrilled to have this released. In general, in spite of some issues we need to work out, last night’s release was a big one for us.

I was trying to get some information on using a Cingular Blackberry in India and I decided to post a question in LinkedIn’s Q&A section (login required). Within 24 hours, I got some great responses, information about flight training and a strong sense of appreciation for what LinkedIn is doing at the moment. My best answer is posted below.

If EDGE/GPRS, connectivity should not be an issue as long as you have international permissions provisioned on your account. It will definitely be charged at roaming rates (x cost per MB). Local rates may be less expensive (requiring a different sim). However, the BlackBerry only downloads 8-10kb of data intially (usually the text portion of the email). The large attachment resides on the server until you “retrieve” it. So if you stay away from the attachment download, it is a very inexpensive method to get email.

You will also want to check your carrier’s website for international coverage and roaming agreements in the particular area you are traveling to.

LinkedIn’s workflow is pretty good. They enable closing a question, replying to responders and selecting a response as the best response. This allows you to have a dialog and ask follow up questions and also to provide kudos to thoughtful answers. Email notification is used to inform you of the evolution of your Q&A session. They also found a way to monetize these pages with sponsorship banners across the top of the page. Overall, nicely done.

We had a similar feature on Judy’s Book called Expert Shopper that we launched but chose not to invest in. Sticking up a Q&A feature isn’t sufficient - you need to work to ensure that questions get answered quickly. If a user floats for longer than a day without any responses, they lose faith in the feature. On the flip side, if you can guarantee a response, you have an opportunity to create significant positive impact.

Q&A is a powerful addition to a site but it requires an active community or alternatively, a small but dedicated staff who are responsible for answering questions as they arise.

We just released a couple of cool enhancements to Judy’s Book this week. The first is an enhancement to the user tags feature I wrote about last week. Based on deals users save or post, we infer their favorite stores and categories. This provides another lens on what people are interested in. I’m a big believer trying to figure out what users care without having to ask them to tell us what they like. In this case, we gave users tools to post and save deals. Their usage of these tools allows us to surface interesting information. This is a rich area you’ll be seeing more from us on this over the next few months.

Another simple, but cool addition is the JB Newsletter Archive. Here you can browse the current and past Judy’s Book Deals newsletters and check out some great handpicked & often JB-exclusive offers.

Stay tuned on JB - there’s a ton more coming over the next few weeks.

Over the past few weeks, we released and iterated upon a number of features on Judy’s Book aimed at user convenience and making it easier to get and track deals you care about.

Deal Bookmarking and Tagging

This features allows a user to save deals with notes and tags and then review them at their leisure. Saved deals can be marked private and by using tags, users can also generate shopping lists. We roll up people’s aggregate activity to provide a snapshot of the deals/categories and stores that people are saving. This is a small step along the road to providing a personalized lens into our database of deals. As users tag and bookmark deals, we build a list of their favorite stores and categories and we’ll use these in the near future to present more relevant deals to the user.

One of the coolest things for me over the past few weeks was seeing this set of features evolve from a ‘quick and dirty, let’s get it out implementation’ to something much more polished. A great example of this is the User Tags page. The initial version was just a text based list of tags which did the job but wasn’t super exciting:

The current version is a much more interesting, visual way to explore a user’s tags.

Mousing over a thumbnail gives you the deal title, clicking on a tag provides a more detailed view of the deals associated with that tag and the alpha bar and right nav strip allow you to quickly navigate a user’s tags. User favorite stores and categories will be coming very soon. Big round of thanks to everyone that worked on this. It was definitely a collaborative effort and I’m really happy with the end result. Favorite stores and categories (inferred as opposed to reported) will be coming soon.

RSS Feeds Galore

Any page you can generate on our site can be turned into an RSS feed that will update anytime new deals meeting that criteria exist. For example, you can get a feed for Restaurant Deals in Seattle, or for Electronics deals with free shipping and add those to your feed reader. Our goal is for users to be able to obtain deals in any form that’s convenient to them and RSS is an important part of that. Being able to customize the feed to obtain the slice of our data that’s right for you is something I’m really happy that we can deliver on.

Back End Improvements

In addition to user facing features, there have been and will continue to be a number of back end improvements which affect things like expired & spam deal removal and surfacing the best deals. The impact of these changes is less visible in any one go, but add up to a significantly better user experience over time. We’re constantly working on this area of the site and the goal here is to display great deals to the user no matter which page of the site they interact with.

There’s a lot more coming over the next few weeks and while the amount of work ahead of us can feel a little daunting at times, I’m really excited about where we’re headed.

A basic truth of startups is that there are many important tasks that aren’t always fun to do. Another basic truth is that no matter what role you play in the organization, you’ll find yourself doing a lot of these. One great example is partner category mapping.

One of the really fun parts of aggregating local deals is the need to map the category trees of your partners to your own. And when I say fun, it’s about as much fun as getting your wisdom teeth out. It’s especially awesome when working with local data providers because the local classification systems are so granular. For example, there are categories for things like “Junior’s Wallets and Accessories Cases” and “Miscellaneous Sprinkler Systems.” Sometimes you can automate the process of mapping inbound data to your category structures, other times, you have to review a list of thousands of items and make judgement calls.

While this is painful, it’s important to do it correctly to ensure users have a good browse experience. Otherwise, you either end up with data in the wrong place or your ‘uncategorized’ bucket getting incredibly large.

Dave and I were talking about a complex mapping that is on the horizon and that will take a good chunk of time to do correctly. We were exploring automated ways to do it and drew a blank. At that point, I said that I was ‘happy’ to do it by hand. Dave corrected me and replied that while I was probably very willing to do it, he’d be surprised if I was happy about it. He’s right. I don’t think I’d ever choose to do this for fun, but it needs to happen and it will get done. Actually, come to think of it, I’m willing to do it and Dave is happy that I am.

Wait a second. What’s wrong with this picture?

There are two great posts at Probargainhunter about comparison shopping engines which have great information about what users experience at the top sites. Highly recommended for anyone interested in this area of online shopping.

• Tips for smart price comparison shopping
• Price Comparison Engines - Small Fish, Big Fish