Which Movie to Watch? An Overview of Recommendation Systems

During lunch at work one day this week we were talking about movies, one of our favorite topics. Both Jared and Christine suggested watching the new Val Kilmer movie: Kiss Kiss, Bang Bang. They said it was quirky, funny, clever, and just a great story. They highly recommended it.

But I got to thinking. Why is their recommendation powerful? Is it because they are two people whose opinion I trust, or because nothing else was recommended to me? What if, for example, someone else whose opinion I value recommends a different movie as highly as Kiss Kiss, Bang Bang? What should I do then?

MovieLens, a Movie Recommendation Tool

So I turned to my new movie recommendation engine called MovieLens, created by some fine folks at University of Minnesota. They are members of GroupLens Research, and have been working on recommendation systems for over a decade. To make its predictions, MovieLens uses a technique called “collaborative filtering”, which takes actions by both me and others to produce a set of recommendations for movies I should see. They explain more about the service on their about MovieLens page.

Unfortunately, Kiss Kiss, Bang Bang was not in their database yet, as it hasn’t even been released. (Jared and Christine belong to some sort of movie club that often gets to view movies ahead of release).

But it did have recommendations for me. Since I started using it, I’ve entered ratings for over 100 movies in MovieLens, and it has enough information to recommend hundreds of other titles. By combining my ratings of movies with ratings from thousands of other people who have seen the movies I haven’t, the number one recommended movie for me at the present moment is: Dear Frankie, a drama.

Now I have a choice: Kiss Kiss, Bang Bang or Dear Frankie?

There are many movie recommendation systems like MovieLens. People adore the recommendations of the Netflix web site. Blockbuster has one. Walmart suggests movies to watch. Who needs Ebert when we’ve got systems like these?

Recommendation Systems Everywhere

Recommendation systems are a growing trend, perhaps you’ve seen some of these:

• iTunes “Top Songs”
• Amazon “people who bought this also bought…”
• Bloglines “similar blogs”
• Del.icio.us “most popular” bookmarks
• NYTimes “most emailed articles”

Just to name a few. As Movielens demonstrates, recommendation systems are systems that recommend content for us by looking at certain factors including what other people are doing as well as what we are doing. The basic algorithm of collaborative filtering is to compare the trends in how I rate movies to those of other people. If other people rate movies similar to me, then they become part of my “neighborhood” of like-minded users. The movies that get recommended to me will be the ones that my neighborhood rates highly that I haven’t seen yet.

Recommendation Systems are User-Focused

Up until recently, recommendations from friends have been the most helpful way for us to find out about new things. In the future, recommendations will come from computer systems like Movielens in addition to friends. They will further infiltrate our daily lives, and I think we’ll be happy about it.

There is a lot of room for improvement in recommendation systems. Right now most of our systems prioritize content based solely on time: newest at the top, oldest at bottom. We see ads for new movies in a much higher proportion than we do old movies, even though old movies might be just as good or as important to us as the new ones.

The prioritization, of course, isn’t ours. It’s the prioritization of movie studios or distributors or whomever is trying to make money on the release of the movie. Since the prioritization isn’t ours, the relevance is not, either. This is why Netflix has grown so quickly, because it is using collaborative filtering to prioritize users over the movies.

The following list is a few ways how recommendation systems could prioritize content more helpfully. Some of these are similar to Movielens, while others are based more on other ways in which we prioritize information in our daily lives.

Ways to Prioritize

• Newness: This is how most systems prioritize things, by how new they are. New information has a higher potential of being great than old information, and therefore it is highly interesting.
• Time-sensitivity: Prioritized by whether something needs attention now vs. later. The flight status of your plane trip tomorrow would be very important. A month away…probably not.
• Popularity: Prioritized by how many other people are paying attention to it. So, anything that is getting attention now will be quickly routed your way. Anything that doesn’t won’t be seen until you get to it.
• Personal Relevance: Prioritized based on some criteria that you deem relevant. This could be your line of work, your interests, your hobbies, your family members.
• Social Network Relevance: Prioritized based on whether or not it is recommended by someone in your personal network. If someone you know (or someone close to someone you know) recommends something, that gets priority over anybody who you don’t know.
• Authority-based: Prioritized based on some metric of authority, which would take into consideration many other user’s actions. This would have to be based on some metric over time: those outlets that are most visited or referred to are given the most authority. Roger Ebert, for example, is not just the most popular movie reviewer. He’s the most authoritative movie recommender.
• Collaborative: Prioritized based on your own actions compared with others. Whereas prioritization by personal relevance would be based on some criteria you choose, collaborative-based prioritization would be based on some algorithm outside of your direct manipulation. This is what Movielens does.

By combining these various prioritization schemes, recommendation systems could make intelligent decisions about which bits of content to show us now and which we should pay attention to that we already aren’t.

We will soon see recommendation systems everywhere, and start desiring them in parts of our lives that don’t have them yet. For example, I would love to have a great recommendation system for blogs. I’ve used Bloglines, which is decent, but its not my primary blog reader and I find that most of the recommendations are for similar blogs, not just blogs of similar quality. I want to read blogs about all sorts of topics, not just about web design. Way back in November I emailed the folks building Rojo and suggested that their recommendations feature has the potential to be the #1 feature of their software. I think this is still true.

Always showing the latest things can be valuable for people. Avid moviegoers want to see what’s latest, because they’ve seen most of the old movies. On the whole, however, we don’t judge quality by equating it to newness. We judge it on other criteria, be it authority, our social network, or personal interest. Old things are sometimes even better. In fact, many people sadly say that the Golden Age of film has already passed.

The Golden Age of Recommendation Systems, however, has just begun.