Social Design Patterns for Reputation Systems: An Interview with Yahoo’s Bryce Glass (Part I)

Of all the social software built on the web in the last two decades, none are as important yet as little talked about as reputation systems. Reputation systems have driven the entire business at eBay.com, much of the business at Amazon.com, drives activity at Digg.com, powers the moderation system at Slashdot, etc…and yet for all the millions of words written about web design very few of them have been dedicated to this type of software.

That’s why I’m really excited about the recent release of Yahoo’s social design patterns for reputation systems. The following graphic from the pattern library illustrates what Yahoo calls the “competitive spectrum”, which is a way to classify the activity on your site and helps you to choose which reputation pattern might work best for your community.

My first introduction to the patterns happened by accident. I needed a room for this year’s IASummit and I twittered about not having one. I received a tweet back from Bryce Glass, whom I knew of but didn’t know personally. He was gracious enough to let me share a room with him and over the couple of days I was there Bryce told me about some of the fascinating work he’s doing with Yahoo’s various properties.

Bryce Glass is an interaction designer at Yahoo! He currently works in the user experience group supporting the recently-announced Yahoo! Open Strategy. But in the past year, Bryce was the user experience lead for a Reputation Platform that powers the rep systems for many of Yahoo!s properties. In that time, he had opportunity to work with several different community-oriented sites on Yahoo! on improving their reputation systems, increasing user engagement and generally creating friendlier, more-active communities.

In the talk he gave at the summit, Designing your reputation system, Bryce outlined a framework for designing reputation systems that is probably the best starting point for figuring out how to build your own. In the following interview, he starts from ground zero and explains what reputation is, what a design pattern is, and how to start applying them to your own work.

1) So, Bryce, what is your definition of reputation?

I generally use a fairly coarse-grained definition of ‘reputation’ when discussing these patterns or other work we’ve done with reputation at Yahoo! By my simple definition: one’s reputation in a community is both a history of one’s past actions within that community, and a value judgment about the worth of those actions.

Who makes that value judgment? Ideally, the community itself. So contributions that the community values are good, and those it finds objectionable (or simply has no interest in) are bad. And you, as the author or originator of that content, can be judged according to these same values. Quite simply: contribute to the community and—if the community likes it—your reputation rises.

Obviously, it’s a little more nuanced than that, but…

2) So what is a reputation design pattern?

This family of patterns are a small subset of the Yahoo! Design Pattern Library, which has been out ‘in the wild’ for a couple of years now. I’d suspect that many of your readers are at least passingly familiar with that effort (and, if not, my fellow Yahoo Christian Crumlish has a good video update up on our Developer Network.)

These patterns are notable, however, in that they’re the first social design patterns that we’ve included. I know that this is a direction that Christian wants to take the library in, so expect to see more and more of these in coming months. (Identity representation, participation, more reputation, etc.) But… going back to your question… this family of reputation patterns can be thought of a couple of different ways.

First, and foremost, it’s just an attempt to catalog and describe a number of common ways to represent some one’s status within a community. They are patterns that recur across websites, games or other interactive communities and we’ve made some attempt to be as complete as possible in this release. (Tho, for the sake of finally shipping the patterns, we did omit several—I’m hoping to sneak those out over the next several months.) For the designer of a community, these patterns might he helpful in just identifying and naming a reputation feature which they’ve been considering. And providing some very lightweight guidance on where and when to employ the pattern. So you could almost think of these patterns as a ‘Chinese menu’ of options for showing some one’s reputation on your site.

But this was a concern to us as well: through working with Yahoo! property-owners internally (the folks who make decisions about things like our Finance site, or Yahoo! Local, Maps, Answers, etc.) we understand that sometimes a long list of options is actually less helpful than no guidance at all. So we pushed hard to include at least one meta-pattern, The Competitive Spectrum, that attempts to provide guidance about which of the other patterns to employ. It attempts to provide a selection rationale.

In truth, there’s so much more guidance we could have included (the Spectrum is but one of 15 questions that I covered in a presentation on reputation systems earlier this year at the IA Summit.) But this one central question is an important one for anyone attempting to influence the community spirit of any ‘place’ they’re building online: what type of community will this be? A friendly place, where people cooperate? Or a competitive, cut-throat one with winners and losers?

3) Although reputation is sometimes bestowed on an individual by a community, what steps can an individual take to improve their reputation?

Tho’ it may sound… un-profound (is that a word?)… my advice here is simple. To improve your reputation in a community, become valuable to that community! There are a number of simpler ways to state this as well: Don’t be a jerk! Play nice with others! Do unto others as you’d have done unto you. It was all good advice when our mothers gave it to us in our youth, and is still good now.

But it gets tricky. I’d mentioned how reputation is very contextual. So you have to keep in mind the relative values of that specific community. Maybe your idea of ‘being a jerk’ is, in fact, not at all in agreement with the majority opinion of the community. Some communities on the web are built on ‘jerks’ and they’re fine with that reputation. Perhaps they value being a funny jerk, and you just don’t want to be boring in that community.

So I would say… hang out in a community before you start participating and try to learn its values. Then become an active participant, and try to figure out a way to become a valued participant. And, you know what? If it ends up that the communities values simply don’t agree with yours, then maybe that’s a sign you should find a community to frequent that’s better aligned to your values.

4) What are the biggest hurdles in designing for reputation?

I think it’s probably the number and variety of unintended consequences that little design decisions can have further down the line. I’m fond of the article—so I cite it a lot—but Ben Brown, who founded the dating site Consumating, has a great blog-post about the ‘ill-fated points system’ that they used for that site, and the variety of… um… less-than-ideal behaviors that those incentives gave rise to. Early on, Slashdot struggled with many of these same issues, and they’ve re-jiggered their comment karma system several times through the years.

A lot of these learnings are, in some small way, encoded into these patterns that we’ve released, as well as research that we’ve done at Yahoo!, and products that we’ve released, tweaked, tinkered and learned from.

A big hurdle—and if you can solve this, you’re halfway there to having a well-designed and effective reputation system—is appropriately marrying the incentives that you offer your users to the appropriate set of goals that you have for your community. You want to be sure that you’re rewarding folks for behaving like good citizens, and not just rewarding them for no good reason. (Or for vague and misguided reasons like “to keep them engaged” or “so we can have a leaderboard.”)

5) What is the biggest misconception?

My own belief is that community designers today are a little too enamored of the “allure” of these types of systems. They are, indeed, quite powerful patterns. Microsoft, for instance—I think it’s fair to say that they’ve been quite pleasantly surprised with the success of their XBox Live Achievements program: it’s a huge revenue driver (gamers will actually buy or rent games they have little or no interest in strictly to unlock that game’s achievements!) and a fan-pleaser (witness the dozens of community sites dedicated to tips and tricks about unlocking achievements.)

BUT… and this is a big ‘but’… XBox Live is a very specific context. I would place (parts of) that service to the extreme far-end of the Competitive Spectrum. And it should be pointed out that not everyone who plays on XBox Live universally buys into this ‘competitive’ mindset. (Read the comments here for a nice overview of contra-Achievement viewpoints.)

So, it’s a powerful feature and a real showcase example for the pattern that we’ve labeled Collectible Achievements but you should really think twice before employing it on your own community site. Ask yourself: will it inculcate a certain competitive mindset in the community? Do I want that? Who will be motivated by collecting achievements? Will they be too motivated, and act out in anti-social ways to get them? Who will be turned off by them, and leave the community? Am I okay with that?

So a big misconception, currently, I think is “we should be doing this.” We should have an explicit reputation system, with badges, points, voting, thumbing up and down. All the bells and whistles. I hope that sites will soon start to employ a more measured, more intelligent approach to designing these systems.

Read Part II of this interview: Social Design Patterns for Reputation Systems: An Interview with Yahoo’s Bryce Glass (Part II)