Tech.Memeorandum’s Filtering Illustrates Web 2.0’s Most Important Skill

On your first glance at the tech.memeorandum home page, you won’t see anything all that special. You’ll see some links to blog items down the left-hand side labeled “Top Items”, and some smaller links to the right labeled “New Item Finder”. It looks like a hundred other blog aggregators being released nowadays…no big deal, right?

So why does Robert Scoble call it “THE PLACE to come to every day” and Richard MacManus call it the “brilliant new blog news service”?

Because it is more special than it appears. Blog.memeorandum is a blog aggregator that accurately shows what topics are coursing through the blog veins at any given moment, with the latest, most popular topics at the top. For example, the new blog search engine by Google was one of the hottest topics in the past week, garnering a spot high in the list and a large headline as a result. The bigger the topic, the bigger the headline.

What makes it special is that it is much more accurate than other aggregators. Hundreds of sites aggregate blog feeds, but few are able to glean what’s most relevant. Many are able to find the most popular topics of the moment, but most can’t get rid of the unpopular topics. The ability to show only what’s new and being talked about while hiding what’s not is the advantage that tech.memeorandum has. Its a single page of highly relevant information.

It’s a challenge being taken up by many developers. Technorati has its popular page. Icerocket has “Hot Topics” listed across the top of its results pages. Bloogz has it’s ranking page. Bloglines has it most popular links page. All of these services are trying to do the same thing: show what’s going on right now.

So why are developers focusing so much on aggregating? Why are they building search engines for blogs? Well, I think it’s because we’re living in an attention economy, and so optimizing our systems to provide people with the latest interesting news becomes an obvious way to get attention. Getting people’s attention is easiest when we can already see what many are paying attention to. If the high profile bloggers are paying attention to something, then everyone else probably will, too.

The Google Approach

It’s the Google approach. Make a service (probably a free one) that everyone wants to use, and you can then create a plethora of paid-for services around it that feed off its attention. It’s the old get ’em in the door tactic. It’s what great salespeople have done since the dawn of commerce.

And in the commerce of Web 2.0, getting someone in the door is the major driver of success. Because the competition of everything on the Web is so fierce, and everyone wants our attention in every way, we’ll be much more likely to try a service or product from someone we know rather than someone we’ve never heard of before. That’s why Google is so powerful right now. Their Search is so great that we’re more than willing to give their other services a try. Sure, a competitor might have a better blog search tool, but now that Google has one I’m pretty confident that they’ll make that one the best, too. I have no switching costs this way, and that makes my life easier.

The Real Challenge: Filtering

The real challenge, and thus a crucial skill of Web 2.0, is filtering out noise. Finding just the right algorithm to parse through hundreds or thousands of feeds and decide what topics are really worth paying attention to is not easy. It’s like being in a stadium of screaming people and trying to figure out what conversations are the most important.

Tech.memeorandum takes a slightly different approach to filtering than sites like Technorati or Bloglines. Instead of trying to index anything and everything like those two do, tech.memeorandum starts with a relatively small set of bloggers and then branches out to other blogs only if they are linked to. This is a successful attempt, I think, at keeping down the noise. If you stick with popular bloggers, then you’ll catch most of what is important. Indeed, they often define what is important!

Just like the assault of copycats on Google’s Pagerank algorithm, there will be attempts at creating systems similar to tech.memeorandum. This is a matter of course, and a main theme running through Web 2.0. When someone comes up with a good filter, others quickly figure out how to emulate it.

Looked at in a slightly different way, Tech.memeorandum is a news recommendation system. It recommends what news to read. It is very similar to the stalwarts in the old media. The most important services there were the newspapers and the nightly news, because they recommended to us what we pay attention to (note that I’m using the past tense here). It’s hard to tell if tech.memeorandum will become a stalwart in the new media, but it sure has a good filtering algorithm at the moment. It makes great recommendations of what to read.

Recommendation Systems and Web 2.0

The other day I wrote about movie recommendation systems and didn’t really tie it in to Web 2.0 like I wanted to. So let me try and sum up here:

Recommendation systems are the end goal of Web 2.0. They are how Web 2.0 will change the daily lives of “normal” people. It’s fun and exciting to talk about RSS and REST and semantic markup, but what we’re really after isn’t technology, it’s utility. What we’re really after is being able to see the greatest movies of all time, listen to the best music out there, and hear the most important news without having to wade through all the junk to get to it. It’s the getting rid of stuff that makes recommendation systems valuable.

Of course, the Web as Platform doesn’t filter by itself. Simply having a bunch of content from which to draw doesn’t do that much for us other than provide an exciting opportunity. With no effort in filtering we’re left with simple aggregation blogs that copy everything, word-for-word, the wheat and the chaff. With more effort in filtering we have valuable filters like tech.memeorandum that can pinpoint the important content and hide the rest. That’s why filtering is way up high in the skillset of Web 2.0.