June 19th, 2008

5 ways to improve reputation systems

by Joshua Porter  |   8 Comments

As more and more companies “go social” (as the NYTimes just did), we’ll see a growing need for well-designed reputation systems. Reputation systems can be defined as systems that help people judge the reputation of others in order to make better decisions about what to buy, who to listen to, or generally what to do.

A new article on Boxes and Arrows, On A Scale of 1 to 5 is a nice introduction to the topic and a good read for folks building reputation systems. The authors provide 5 design takeaways:

1. List the behaviors you want to encourage and those that you want to discourage
2. Be transparent
3. Keep your reputation system flexible
4. Avoid negative reputations
5. Reflect reality

The authors also provide a helpful framework to keep in mind while designing, paying close attention to three questions:

1. Who is doing the rating?
2. What, exactly, is being rated?
3. If people are being rated, what behaviors are we trying to encourage or discourage?

One thing that my research has shown is that people often make up their mind about the trustworthiness of others based on how they write…they get a sense about who to trust from the prose that people use…even if it’s merely a product review. While a Netflix-style “people like me” feature can be valuable…they aren’t trusted outright…and it doesn’t replace what people will glean from their own inspection of the actions of others.

One thing I didn’t like about the article was how they explained reputation systems as helping people manage risk. While this is fine for economics class…this is an incredibly distancing term for design. When I’m shopping on Amazon I’m not managing risk…I’m trying to decide! I’m making a decision, and I’m uncertain about what to do. Using the term “Risk management” confers zero empathy…but when we design to help others make important decisions in their life we’re much more empathic…well I’m quibbling. That small quibble aside…it’s a good article.

I also like how the authors pay homage to the heavyweights in this arena. While reputation and rating systems have been around for many years, Amazon and eBay are only recently getting the credit they deserve. These are massive, scalable, and generally effective designs that have pushed the entire state of web design forward. While we can look at these two sites and recognize what’s happening, we inevitably take for granted the tens of thousands of hours of research and design that led to them. They are the canaries in the coal mine of reputation system design.

Grab the RSS Feed