September 12th, 2006
Digg’s Design Dilemma
by Joshua Porter | 
55 Comments | shortlink: http://bokardo.com/p/460
This past week’s Digg controversy is one in a growing number of incidents that suggest that a small group of users are having an undue influence on the promotion of stories. In response, Digg is changing the way that it handles votes by adding complexity to its ranking algorithm. I think that’s the wrong approach, so here’s another idea: change the actual design of the site…that’s the real problem.
The most recent controversy happened on September 5th, when someone named jesusphreak posted Digg the Rigged?, an in-depth article exposing some of the curious details of recently-popular stories on digg. Many of the stories, jp pointed out, were dugg by members of the Digg Top 30, or the 30 most popular digg members (popular being measured by number of stories submitted that were promoted to the frontpage). The Top 30 includes Digg founder Kevin Rose.
This was not the first time that someone has pointed out this phenomenon. On April 18 of this year Macgyver at ForeverGeek posted Digg Army, which included screenshots of who dugg two recent articles on the site. Each article had the exact same 16 people digging it in the exact same order. Of the first 19, 18 were the same. Included in that list of people was, again, Kevin Rose. ( for an in-depth history see Tony Hung’s excellent: A Brief History of the Digg Controversy)
These incidents, taken together, are more than coincidence. They strongly suggest that Digg is being gamed by a small number of users, artificially inflating the value of stories that might not deserve such attention. This flies in the face of the democratic ideal of the site. And so far, nobody has claimed that the two articles I mentioned are false: Digg exposes most of the voting activity for all to see. A small group of users is definitely having a large effect on popular stories.
But before we get outraged at the corruption of it all, we should give everyone involved the benefit of the doubt and consider how this might have happened without evil influences.
Don’t blame the users
The users of Digg aren’t to blame. They’re simply playing by the rules as outlined by what they can and cannot do on the site. They’ve figured out how to play in the environment they find themselves in, and that’s OK. Jason Calacanis, creator of digg-clone Netscape.com, said in his post One User, One Vote: “The top users earned their spot and they should be reward for their contributions–not penalized.”. I agree with that…it’s not like there were any rules to follow.
Blame the design
Instead of blaming the users, blame the design of the site. From the ranking system, to the friends feature, to the display of content, to the ease with which users vote, the design of Digg.com conspires to make it haven for gaming. Not only is the pile-on digging activity possible on the site, it is actually enhanced and made easier by the very design of it!
Here are the features in question:
- Rankings list
If you want people to compete, rank them. This is a big part of the reason why there is gaming on Digg. Getting a higher ranking becomes an incentive to game because if you do then others will notice and you’ll get recognition. (that’s important to people, even in social software) In addition, with the recent offer by afformentioned Calacanis to pay people for this type of work, high rankings may also be a source of income. - Friends feature
The Digg friends feature is the means by which the top users promote stories so quickly and with such synchronicity. In particular, the friend’s history page shows the stories that friends have dugg in reverse-chronological order, so that the newest diggs are at the top of the page. By refreshing this page often, top Digg users (who are all friends in the system) can stay up-to-the-moment with each other. During waking hours, a quick 30 diggs will draw further attention to any story, making a pile-on more likely. - Exposing who diggs what
At the bottom of each dugg entry is a list of people who have dugg it, and serves as the evidence that the two articles above used to expose the issue going on at Digg. This is a perfect example of what in psychology is called “social proof”. Social proof is something that is “proved by society”. In other words, the mere fact that others are doing makes it seem like it is what should be done. We learn that way, by mimicking the actions of others. So, when we see someone else digging something, we would be much more likely to digg it ourselves. Or, to put it another way, we let others influence our decisions and help make them for us. - Stories at a distance
It is very possible to interact on Digg, digging stories and burying others, without actually reading a story. That’s because Digg only shows summaries of posts. If you want to read a post, you actually have to click on them and go to the external site to do so. Many people will make this extra effort. But many people won’t.In It’s all a Farce Anyway Tara Hunt recounts an interesting (and scary) conversation with people who game digg. They submit stories and ask their friends to digg them. After a post reaches a certain digg count they say: “people just hit digg if they are remotely interested in the topic.”. This, again, is the effect of social proof, exacerbated because the stories are at a distance and it is extra effort to read them.
- Ease of voting
While it takes extra effort to read posts, it takes almost no effort to digg them. This might be backward…digg is essentially making it possible to vote without knowing what you’re voting on. Although the digg feature is amazing, an excellent example of technology that makes our lives easier, it is also in danger of trivializing them.This reminds me of a story by Derek Powazek in his book Design for Community, where he makes the point that the harder it is for someone to comment on something, the better the comments are. In other words, people who jump through hoops (or pay attention long enough) to comment are the ones who really care about the subject matter, they’re invested in the story and see value in taking the time to respond.
Other factors
The Digg community is protective. Stories that are about digg get a lot of positive attention there. If you want to get noticed, for example, simply write a post entitled “10 Ways to Get Dugg”. That will get them interested. However, it has also been noted that many Digg community members react strongly to anti-Digg content, often burying it when it reaches the front page of the site, effectively censoring it. This has the unintended effect of making it seem like Digg the service is censoring all non-Digg content (which isn’t necessarily false, either).
Also, people use Digg in many different contexts. I’ve dugg stories myself that I just want to read later…stories that I don’t have time to read right now but that seem valuable to me and I want to be able to find them later. It could be that others do this activity as well, causing votes where none should happen. When you give people tools, they don’t always use them as prescribed.
The result: no independence in voting
The result of all these factors is that Digg breaks the cardinal rule of voting: independence. As outlined in James Surowiecki’s book The Wisdom of Crowds, independence arises when a person makes a decision (votes, diggs) without the direct influence of others, on their own, by making up their own mind. Of course, there will always be influences on that decision…what others have said, where their political party is leaning, their current situation, but in the end they need to have the privacy of their vote. On Digg, no votes are private, and when you make them you can’t help but notice the way others are voting.
If we compare this to how people vote in Presidential elections, we see how different it is. In those, we anonymously vote. The anonymity of the vote is key…once we start exposing who voted on what we’re gaming the system because we are inevitably influenced by others votes. And the ranking of voters just solidifies this…imagine if we could see how others voted over time in Presidential elections…
Digg vs. Del.icio.us
The voting on Digg is in contrast to a site like Del.icio.us, where voting (saving a bookmark) is done more independently, often without having any idea whether or not someone else even viewed it, let alone voted on it. Del.icio.us isn’t immune to gaming, however, as there is a popular list, and it’s very easy to simply copy those bookmarks into your own, driving up the numbers just like on Digg.
So far, however, Del.icio.us seems to be more immune to the outcries of gaming. This may result from a smaller user population, as it is nowhere near the size of Digg. But I think it has more to do with the nature of the tool. On Del.icio.us, the main value is personal, as people use it to store bookmarks that are valuable to them. On Digg, the bookmarking utility is secondary to the voting, in both the interface and the wording used on the site.
Digg’s response
Later this past week, Digg responded to the controversy by changing its algorithm:
“This algorithm update will look at the unique digging diversity of the individuals digging the story. Users that follow a gaming pattern will have less promotion weight. This doesn’t mean that the story won’t be promoted, it just means that a more diverse pool of individuals will be need to deem the story homepage-worthy.”
I think this is the wrong approach. By keeping the above features the same…Digg is asking for gaming. As gaming occurs, they’ll have to change their promotion algorithm. Then more gaming will occur to override the new algorithm, which they’ll then have to change. In any social system gaming will occur, but I think Digg’s problems are much more basic: their features simply beg to be gamed. Better to focus on the independence of voting, not on the algorithm. By making much of the ranking and voting hidden, the diversity of the site would increase. It’s exposing information that leads to sameness.
What would change mean?
Even if Digg were to change, however, to alter some of the features above to make voting more independent, we still couldn’t be sure that they would work. People test the boundaries of all social tools, finding every which way to bend them to do something useful. Sometimes it’s fine, sometimes it really does hurt the quality of the site.
Digg couldn’t just say “let’s move the digg voting widget somewhere else” and be done with it. That would introduce a new set of problems, based on the new context. However, they did add a new feature lately whereby the Digg widget shows up right on the posts themselves. That could potentially solve a lot of these problems, getting the voting mechanism much closer to the content people should be reading before voting on. Though it isn’t clear whether or not this is part of the solution, it seems like a step in the right direction. (I’m trying it out below – you may not see it if you’re in an aggregator that strips scripts)
Links to this Post
Comments
1. Tony 9:44am, Tue 12th, 2006
That has got to be one of the best written articles on how a system-failure — not a personnel failure — creating one of the biggest blowouts on Digg.
The point that you make about how it takes almost no time to “digg” an article (well, you just click it) as opposed to actually _reading_ it is dead on … its like the grease which enables every feature whereby someone could manipulate the system.
In other words weighted voting + easy-to-check-what-friends-are-digging + easy-to-digg == circle-digg controversy.
If anyone is interested in Digg as Game Theory (and how it simply cannot be the wisdom of crowds) there are a few good references) I have posted quite a few already. Hope no one minds:
http://www.deepjiveinterests.com/2006/09/08/digg-as-democratization-of-news-a-fallacy-from-the-start
Cheers
tony.
2. Calvin Yu 10:01am, Tue 12th, 2006
Good points, but by making diggs private you would remove any public oversight to possible gaming in the system. Would Digg have openly acknowledge that a problem existed if no one was ever going to be able to prove it was true or not?
3. Gene 10:05am, Tue 12th, 2006
Josh, this is an excellent post. The challenge with Digg is that it’s not just a social news site–the community features are as much a part of its character as the voting. If you take away the community features in the interests of making the voting independent, you strip away much of what makes Digg Digg.
When Kevin & co (don’t know them, hear they’re nice guys) focus on changing the algorithm, they’re in effect saying “all users are potential gamers.” As we saw this week, that alienates people. Another option is to be more transparent about the algorithm with certain users, building a “circle of trust” with some of the top diggers.
(By the way, some similar ideas are here: http://www.shmula.com/197/digg-as-a-game)
4. Emrys.Roberts 10:35am, Tue 12th, 2006
I am actually a fairly new user to digg, and I have to say that I really love the site. After I read your article, I do see how digg could very easily be abused. I in no way presume to think I know what game theory is and what exactly it is all about, but I do think I am getting the concept. That is unfortunate that it can happen, and I do wish people wouldn’t do that. I do realize however, that there is only so much safeguarding we do against that. People will always hack the system and try to find the loop holes. It’s only human nature. That’s why I do agree with the latest update digg is making to the site. It’s unfortunate that it will affect the top users of the site, but I don’t see it as punishing them. By making it harder all around to make the front page, i believe it creates a more realistic setting. It’s not easy to make front page news, even when it’s the major news companies deciding what the news is. This way we’ll be the ones deciding again and not the top users of the site. I am glad at how much the contribute to the site, but I don’t want it to be easier for them to make the front page then anyone else. Thats not the point. You submit a story, people decide if it’s digg worthy and that’s it. I don’t digg because I want to get popular on the site. I digg because I love to see if anyone else cares about the stories I submit, and vis versa. I do love the site and I am going to stick with them. I takes a lot of trial and error to create something successful. Be patient. With everyone’s help, the site will become what most people want it to be. : )
5. Michal Migurski 11:28am, Tue 12th, 2006
I’m not so sure that a system which accretes attention to a small number of stories is necessarily a slap in the face of a democracy – there are plenty of legitimately, democratically-elected statesmen who don’t deserve much attention, either. =) Switching to a blind-voter system like Reddit could help alleviate some of the pressure, but it may also kill the swarming pile-on aspect of Digg that makes it a lot of fun. The trick is to make it easier for new players to join in the action and increase the long-term diversity of the output, rather than making the site uniformly fair and dull for all.Del.icio.us is a bookmarking tool, Digg is something else entirely.
6. William 1:10am, Wed 13th, 2006
What about simply hiding the digg count, comments and voting history of an article from a user until that user has voted?
In that way, you prevent some of the abuse and you don’t allow a user to participate in the discusson (comments) of the article until, hopefully, they have read it, but, at the same time, once they have voted, the history of the article and its diggs is laid bare.
7. Muhammad Saleem 5:37am, Wed 13th, 2006
A very systematic look at how the system is failing at Digg. Thanks.
8. Rudi Cilibrasi 6:10am, Wed 13th, 2006
Technical note: It’s mathematically impossible to design a collaborative ranking (voting) system that prevents strategic voter-collusion (to do takeovers) and provides for the necessary features that you would need for any system to be called good. This theorem is sometimes known as Arrow’s Voting Paradox or Arrow’s Impossibility Theorem. It’s a limitation on the possible sorts of logical structures we may build, ever. Here is the essential message to me: Get used to it. Strategic collusion is unavoidable so it is best to adapt or try to mitigate the harm but realize it’s technically impossible to stamp out collusion and simultaneously preserve those other features that we associate with group choice making protocols. Such a system cannot logically exist.
9. Tom 6:31am, Wed 13th, 2006
Good analysis. Thanks for the article.
10. Dave 8:08am, Wed 13th, 2006
The funny thing I’ve noticed is that everyone is complaining about what is essentially democracy. The only difference between typical democracy and digg’s form of it is that digg allows you to see who votes.
The thing is, if people want to game digg for the sake of influencing the news that users read (essentially using digg as a form of advertising) then your suggestions about anonymous voting won’t work. The top diggers are, no doubt, not all motivated by fame or notoreity or a sense of helping out the digg community. Some of them will be posting what they want the users to hear/read. Essentially using digg as a soapbox or billboard.
Sure Digg is making it easy to game, but it would be just as easy to game if you didn’t know who was voting or if you had to “read” the article before voting. “Why?” you may ask. Well, if I submit a story I can just send an email out to a mailing list I’ve created for my friends or send an instant message to them all. And they can do the same. However, if voting is made anonymous users will be unable to detect that it’s the same people digging the stories.
Users on digg have been accused of being hired by companies like Microsoft and Apple to digg up pro-MS/Apple stories and digg down stories that are pro-MS/Apple opponents. And this seems to me just like big businesses funding the politicians that will act favourbaly toward them once they get into power. The masses are usually ignorant of what they are voting on or who they are voting for. It doesn’t seem to be any different with digg.
11. linportal 8:26am, Wed 13th, 2006
Very good article! Thank you.
Also, I caught myself thinking about the same thing you mention, how users digg on the articles they’ve not even read. I did it more than few times, so you’re right.
They probably should do something about that, if at all possible.
12. Bronwen 8:47am, Wed 13th, 2006
Well-considered information. I thought Digg was gaming with web content. I didn’t think of it as an information source, it presents as a digital sport. I like what you have provided here. The conections, logic, and references you’ve made are serious insights into social computing. My time was well-spent reading your work. Thanks
13. Jayel Aheram 9:04am, Wed 13th, 2006
Most people do not read the articles…? I do not understand that concept at all. Is this a fact?
14. unclewiggly 9:15am, Wed 13th, 2006
That was a very well thought out analysis. I agree with William on making the voting results hidden until one votes. The only piece I see missing that no one has mentioned so far is removing BuryIt. I don’t mind so much that friends collaborate to get stories to the front page faster. What I object to is that others have the power to effectively cancel my vote. There is no analog to that in a Democracy. Remove Bury It and more varied stories will appear on the site, thus attracting or keeping more varied users. Remove Bury It and the effects of collaborative voting become minimized. Its childish anyway. Keep it and the stories that do get to the front page will continue to be those that have been censored by those that take the trouble to do so. This results in a site that revolves around the interests of that small group.
15. aggies11 11:20am, Wed 13th, 2006
Digg is a “Social” bookmarking site. Voting (election) is not a social process. Part of the allure of digg, is the Social aspect.
“These can’t just be a coincidence”, unfortunately the vast majority of people’s misunderstanding of statistics and probability, means that above statement is not very usefull. Look at the whole (In a room full of 30people, what are the chances that two of them share the same birthday? >%90. When you tell most people that, they don’t believe it). Statistics are counter-intuitive.
Look at how many digg users there are. How many diggs happen each day. What is the chance of some stories having groups of 30 people all digg together in rapid sucession? Pretty damn high.
You do make a point, that because of the social nature of digg, with friends, people will vote in groups. Not to “game” the system, but because people share interests, and have friends lists. This means the chances of groups digging together goes up. This isn’t bad though.
The top 30 diggers are an even poorer example. They are the most active diggers and submitters. They digg *alot*. They are gonna show up everywhere, and often together (because they are all there “first”, being the most active diggers).
This article, while written in a reasonable and in telligent tone (which is a good thing), is a result of misunderstanding (while common, and understandable) of statistics/probability and a misunderstanding of the nature of digg.
If enough people find an article interesting and digg it, then it’s simply that, a story people find interesting.
If people on digg, in general, do not find stories against digg interesting and bury them, it’s simply that, the people of digg choosing what they want to see on digg. Which is the whole point.
Part of digg’s appeal is that it’s “FUN”. Take the above away, and thats what it stops being. Which defeats the purpose.
Aggies
16. murtlest 1:07pm, Wed 13th, 2006
This site tried it out but it seems like those suggestions just takes the fun out of digg.
17. Joe 1:08pm, Wed 13th, 2006
Nicely summarized. I really think that Digg should consider allowing RANKED (top ranked) users to have a bit more features to allow them to edit stories and decide if it is really true. This is already done at one of digg like site at TechTagg.com. Seems like a way to clean up garbage.
18. Nikunj Mehta 2:39pm, Wed 13th, 2006
Value of information
Novel and true information has more value than stale and misleading information. Big media editing process is inherently biased and produces a small community of information evaluators. On the other hand democratic, i.e., social editing often creates the least common denominator of information, not the same as novel and true information.
Approximation to optimal editing
The academic editorial process works quite effectively by producing blind votes, subjective review, and reviewer selection. The process usually advances the community and its agenda.
It is not quite optimal because it is slow, biased towards the community goals, requires a qualification for entry, and can be rigged in favor of certain groups.
Conclusion
Democracy unfortunately does not produce good information. Neither does capitalism. You ought to have a meritocracy to distribute valuable information and foster excellence. Would be great if someone could address some of these drawbacks of academic the editorial process and introduce it to mass communication.
19. Adam Darowski 1:20pm, Thu 14th, 2006
Awesome, awesome post. And I’m not even a digg user. I’m someone working on creating his first community site and this information is invaluable.
20. Irene Brawn 11:47am, Fri 22nd, 2006
Its a very popular topic on the Internet currently. The main thing is that everybody knows about Digg, everybody visits Digg and votes, and everybody beleives in the “clear†democratization of news so that the main goal will be reached.
21. Ira Roth 3:06am, Tue 13th, 2007
I am usually one of those people who don’t like big articles but this one I read from end to end. I particularly liked William’s idea – “What about simply hiding the digg count, comments and voting history of an article from a user until that user has voted?” which I have found to work notably well in voting polls when you have to vote and only after that you can see the votes. I’ve even see polls where you can chose “I don’t want to vote”, then you can see the results and you can’t vote anymore because chosing “I don’t want to vote” is counted as a vote. I believe that strategies like this will really help for the fair-minded voting.
Well, I really don’t think that hiding comments is any solution because sometimes great discussions arise because of user interaction via comments. But I do think that hiding votes and digg count will help if not solve the problem with equity.
22. Web Design Wales 5:21pm, Wed 14th, 2007
It’s hardly surprising that this is going on – with spammers and ruthless marketers trying desperately to push their latest “Top 10 Something or other” article, it seemed inevitable that Diggs could be ‘bought’, or at least that the process could be corrupted. It’s a shame
23. Pozycjonowanie 4:37pm, Thu 5th, 2007
That was a very well thought out analysis. I agree with William on making the voting results hidden until one votes. The only piece I see missing that no one has mentioned so far is removing BuryIt. I don’t mind so much that friends collaborate to get stories to the front page faster. What I object to is that others have the power to effectively cancel my vote. There is no analog to that in a Democracy. Remove Bury It and more varied stories will appear on the site, thus attracting or keeping more varied users. Remove Bury It and the effects of collaborative voting become minimized.
24. One night in paris 5:35am, Fri 6th, 2007
I caught myself thinking about the same thing you mention, how users digg on the articles they’ve not even read. I did it more than few times, so you’re right.
They probably should do something about that, if at all possible.
25. HOODIA Safety 6:32am, Thu 12th, 2007
Very nice site, helpful articles – good job, thanks
26. flickr news blog 4:28am, Thu 19th, 2007
Amazing tutorial! Really apreciate that – thanks!
27. Lukas 10:12pm, Wed 25th, 2007
Amazing job thank you so much
28. Matthias 10:18pm, Wed 25th, 2007
Your work is very much appreciated thanks
29. Gardening 12:12am, Sat 28th, 2007
Several fresh articles about gardens and gardening
30. akademi türkiye 9:24am, Thu 3rd, 2008
thanks
31. iyinet webmaster forumu 2008 seo yarışması 2:01pm, Mon 4th, 2008
thanks
32. kız oyunları 2:03pm, Mon 4th, 2008
Thank you
33. rüya tabirleri 12:27pm, Wed 6th, 2008
Thank you.
34. iyinet webmaster forumu 2008 seo yarışması 12:48pm, Wed 6th, 2008
thanks very good
35. Dijital FotoÄŸraf Makinesi 12:38pm, Mon 7th, 2008
It is not quite optimal because it is slow, biased towards the community goals, requires a qualification for entry, and can be rigged in favor of certain groups.
36. interaction design 8:12am, Sun 18th, 2008
Hi Joshua,
I think the response of Digg to change its algorithms is a typical engineering response. They immediately think the flaw is in their coding or algorithm because that of course hold the key to all problems (sarcasm mode off again). I agree with you that the problem lies a lot deeper then only the algorithm.
Thanks for the article.
37. iç çamaşır 4:53pm, Tue 30th, 2008
t is not quite optimal because it is slow, biased towards the community goals, requires a qualification for entry, and can be rigged in favor of certain groups.
38. payday advances 2:30am, Thu 19th, 2009
Digg is good social web. May be one of the best because of good idea of makers. I post a lot of articles there.