This is closely linked to another concept, the representativeness heuristic. Certainly, a 15-year-old who is interning at Morgan Stanley is not representative of 15-year-olds, let alone “teenagers” at large.

This is why Chris is wrong on this one. True, anecdotes are a kind of qualitative “data” — but only insofar as they provide information about the person telling it.

As a designer, whenever I hear an anecdote about use, I immediately think of alternate scenarios and consider who might represent them. Ideally, then you go out looking for those people and hear their stories.

This is a basic tenet of psychological research: intervening variables are legion. So while we as members of the Western world prefer to reduce our data into bite sized chunks, let’s keep in mind that our tests might have returned different results if it hadn’t been raining that day, or if our subjects hadn’t had a bit of a headache, or if we weren’t worried about our child being sick, or if the tester hadn’t just gotten that good bit of news. Psychological studies show that moods affect performance, and as a race, we’re a pretty moody bunch. So while we’re busy ticking off boxes about what was easy or hard, keep in mind that there’s a whole other level that is usually not accounted for. And that level is qualitative variability.

Your last point regarding focusing on an exception rather than a rule reflects a recognized psychological phenomenon. People do tend to rely on anecdote in their everyday lives and they also tend to find the anecdote that suits their personal preference. Dan Ariely covers this quite nicely in his book, Predictably Irrational, Chapter 9, “The Effect of Expectations: Why the Mind Gets What It Expects.” I suggest that this is why Apple doesn’t do focus group testing. (But actually, focus group testing is problematic for a variety of reasons, which I’m happy to discuss in greater detail elsewhere.) Testing tends to reduce information and design to a low common denominator; this is not the locale from which dramatic and influential change occurs. This is the most difficult aspect of testing. In an old notion but a true one: humans are resistant to change, even when it’s for the better (covered in Ariely’s book mentioned above; Drew Westen’s book, The Political Brain; and George Lakoff’s book, The Political Mind).

So ultimately, while an anecdote does not constitute a complete data set, it is vital from a human perspective, primarily because we respond and remember things from a narrative standpoint and not a data-driven one. What all of this means is that quantitative data is almost useless without interpretation (performed by moody humans) along with some explanatory qualitative data that serves to frame it.

So we are left with anecdotes, opinions, experiences, conversations… Is that so bad?

It’s no different than writing a book and letting the marketing/media outlets determine how it will be perceived.

Putting “visibility” over “viability” it seems.

If something does not get read does that mean it’s not a useful work? Maybe. But “tidying” up your titles just to beef up your SEO so you’ll get read, or get comments isn’t the way either.

When we reduce everything to a crisp little sound bite like @john’s example above not only is the soul of the experience stripped away, but a sentiment, story, comment may go “viral” (such as “Teenagers don’t use twitter”), thus destroying any chance of capturing any viable data.

One of my favorite movies, Six Degrees of Separation, has a great scene toward the end where Ouisa Kittredge (Stockard Channing’s character) says of their recent experience with the class-jumping Will Smith: “I will not turn him into an anecdote, it was an experience. How do we hold onto the experience?”

Indeed – an anecdote is typically used to illustrate an already held conviction, and simplify or abstract away from the raw data which might be more difficult to understand – and more human.

I think the reason these studies get so much press is just a combination of laziness by blog authors and a desire to write catchier, more provocative headlines.

“Teenagers don’t use twitter” will get more reads in your RSS feed than “Many suggest that adoption of twitter by those under 25 significantly lags their adoption of other social computing technology.”

Just doesn’t have the same ring, does it?

In other words ONE anecdote is only ONE data point.

In your example, Alex is only slightly less guilty than Gigi of using anecdotal data to form a conclusion. Her knowledge of the one-call rule can only be anedotal.

In many ways, Alex is actually worse because she has used her idiosyncratic anecdotes to form a hard-and-fast rule, whereas Gigi is keeping an open mind because she knows that exceptions are possible. Gigi might even have other anecdotal data to work with (i.e., she felt sparks between her and Conor that would transcend any rule).

“The plural of anecdote is not data” is itself a kind of folksy anecdote, one not supported by the mountains of amazing qualitative research whose entire foundation is essentially nothing but piles of anecdotes.

I’m not saying you aren’t just cautioning us to be more analytical and rational (which is not a bad thing) but this piece taken alone does seem to be suggesting that the story is less important than the data or experience behind its creation.

Being an anthropologist by study/background, it is the experiencing not the data collection that makes observational research so powerful and important for decision making.