Social Design Patterns for Reputation Systems: An Interview with Yahoo’s Bryce Glass (Part II)

In part I of my interview with Yahoo’s Bryce Glass, Bryce explains the basics of Reputation Systems. In the following part II, Bryce digs deeper into the strategic decisions around patterns, like how to determine which pattern to use and who is using reputation patterns best.

6) So let’s say I’m building a site with social networking features. How do I figure out which reputation pattern to use?

My advice would be: start small. Definitely think of which one or maybe two patterns to employ. Consider the spirit and intent of the community that you’re trying to build. What exactly do you hope people will do there? Write reviews? Post videos? Form connections with each other? Consider all of these things, and others, and then try to place your intended community somewhere along that Competitive Spectrum.

And be honest with yourself! I’d posit that the vast majority of consumer-focused social sites are somewhere at or below the ‘Cordial’ level. If you still insist on placing a ‘Top Hunks’ leaderboard on your dating site after all of that? Well… that’s fine, but at least now you’ve made that decision consciously and clearly. (It’s probably still the wrong decision, but…) And, not to re-plug my earlier talk, but more guidance is given there as well on this very question of ‘which pattern?’

7) What is the biggest mistake that designers make when implementing reputation patterns?

I’d say 2 related things: one is employing those more empirical patterns— Points, and Levels, ranked and tracked on Leaderboards— in situations where they’re not appropriate. I feel like I’m belaboring the point, but… if your community values fun, and easy-going interactions with each other and helpfulness? Then don’t destroy that fantastic dynamic by comparing members, one to another. Don’t elevate certain members’ status at the expense of everyone else in the community—’cause resentment, factions and gaming are soon to follow.

And related to this is the mistake of rewarding the wrong types of behavior. Specifically, there’s a tendency to want to reward activity (how many times have I contributed, or how frequently) instead of the quality of those contributions. (Do people like this video? Have they watched it? Responded? Linked to it, or embedded it on their blog? Voted for it, or assigned a rating?) Of course, both are important: you want people who are actively engaged and prolific contributors: but you want those contributions to be quality ones: thoughtfully prepared, formatted along community norms, and above all useful or interesting to the community.

A relevant, and recent, example I could cite is Plurk. Now, I absolutely don’t mean to hate on Plurk. It looks like a fine product (it’s kind of a Twitter-like microblogging platform.) But they’re tracking and displaying some very “official-looking” Karma metrics, and even feature a Leaderboard of Interesting Plurkers. My response to this is two-fold: first is… “why”? What community goals does it further? My guess would be that it’s a desire to promote active, high-use Plurkers to the community, that others might find them and opt to follow them as well.

But the prominent Karma score, and a surface appraisal of how it’s generated, might lead one to believe that Plurk is a competition. And, specifically, a competition won by the amount of stuff you do! (Number of Plurks, number of friends, etc.) Most people can see how badly this could end: if someone really wants to make it onto that leaderboard? They’ll probably try mass-friending and spam-blasts first. (Even if Plurk’s system is smart enough to counter this, the overall effect is still negative.) There is a nod to quality—’Quality Plurking’, however that’s defined—but the emphasis appears to be on Activity. And I’d posit that a karma system for an app like this is somewhat extraneous. It kinda smacks of “wouldn’t it be cool if we…”

I also feel compelled to point out that the particular label they use—’Interesting’—is a loaded one: while very complimentary to those who receive it, it’s can also feel derogatory to those who’re left out. There’s a reason why Flickr has only ever applied the descriptor of interestingness to photos, and not the people that take them—and that reason is that the community folks over there have a wonderful awareness of community spirit, and are sensitive to the effects that labels can have.

8) What is the best example that you know of of a site that implements reputation patterns?

I don’t know if it’s the absolute best, but a site that I’ve praised in the past is Yelp. They’re a review site and they feature a nice variety of reputation indicators. What I like about Yelp is that they seem to have payed close attention to their community, and what motivates people to write reviews, and their reputation system leverages that nicely. It doesn’t work against it.

A really simple example: some of our own research, at Yahoo, indicates that one reason some people may write reviews is just this desire to ‘fill a void’ or provide a review for a product or venue that has none. (I’ve wondered if this isn’t somehow psychologically related to those guys that like to type ‘first!’ into comment fields.) Now of course this isn’t the only thing that motivates someone to write reviews, but it can be a small motivator for some folks.

Yelp must be aware of this tendency, cause they give users a small boon for being the first person to contribute a review for a business. The first review for any establishment will display a ‘first to review’ badge for ever-after. So it’s not a huge thing. They don’t place a lot of importance on it, but there it is: a small and very natural show of appreciation for those users that like to help get the conversation started.

And Yelp does this in a dozen other ways as well. They have specific reputation types that reward funny Yelpers, or helpful ones. They have a special designation (the Yelp Elite — you’ve written about them before, in fact.) So Yelp encourages a wide range of expression from their review-writers: basically, you can be any kind of ‘Yelper’ you want to be, and—as long as the community finds value in your contributions—Yelp has a way of rewarding you. (And Yelp doesn’t ‘rank’ users against each other, or display a leaderboard anywhere on the site.)

And, of course, I’ve already mentioned XBox Live. I think they do a fantastic job. I believe that they employ just about every pattern from the set that we’ve published, and probably a couple others besides. And all for great effect, for a very specific purpose. BUT… they’re a fairly competitive context, so I think they get a lot of leeway to do things that a lot of social community sites should probably not be doing.

9) What is Yahoo’s strategy in getting these out to the community? Are you simply being altruistic? Wouldn’t these help your competitors?

There are a couple of dimensions to my answer here. First, I’d say that the Pattern Library, in general (which has been open since February of 2006, btw) is a good fit for Yahoo!s stated goals of openness and transparency.

Secondly, there’s nothing especially proprietary about the information or opinions embedded in the patterns. Christian, who I’ve mentioned, actually vets all of our public patterns with our Legal team, so if there actually were some sooper-sekrit game-changing reputation business logic in there…? Well, that probably wouldn’t make it outside the firewall. 😉

But, also, these patterns are in large part drawn from examples and experiences of competitors, as well as products that we’ve shipped at Yahoo! So in a way, it’s not so much ‘getting them out’ to the community as giving them back to the community. There is some work involved in these patterns (compiling, researching, refining and writing them out) but the benefits for us are innumerable: the ability to positively influence the community, be seen as thought-leaders in social software. Heck, just taking part in a smarter dialog about the place of reputation systems… it’s all good.

Thanks for the interview, Bryce!

Resources:

• Yahoo’s Social Design Patterns for Reputation
• Bryce’s Designing your Reputation System slide deck
• Bryce’s personal blog