Algorithms and Community: Voice Wins

I've meant to write this post for sometime now, and thanks to Looksmart, I finally can. The company has asked me to join a small group of writers in thinking out loud about the implications of search and media, and nothing is as clarifying as a deadline to get my thoughts in a row.

Back in early August I penned a post titled "Why Google Needs to Buy Wikipedia." Clearly, Wikipedia, as a non profit, isn't for sale, but I was trying to make a point about the intersection of algorithm, on the one hand, and community, on the other. It was the start of an idea, but not a conclusion.

In that post I wrote: "Google Maps is not very good. At the real world. And while Google is trying to fix that by allowing map editing, I don't sense Google will be very good at fostering and nurturing the kind of communities that will allow Maps to self correct....Put another way, because Google is not very good at communities that self-correct into reasonable quality, and if it's going to realize the vision it might (of turning the entire world into, well usable data) it's going to have to get a lot better, a lot faster."

There was a seed of an idea in there, finally it's time to give it a bit of water and sunlight.

First, let me pull back. I believe that in the media world, there are several buckets of what one might call "value creation." There is clearly value in traditional approaches to content creation - editors, producers, and writers corralled into media-making factories like the New York Times or NBC (I call this kind of media "Packaged Goods Media"). There's value in a different kind of media, media created by ongoing conversations between communities of media consumers on blogs, social networks, and sites like Digg (I call that kind of media "Conversational Media.")

Finally, there's value in the aggregation and curation of media, whether it's packaged goods or conversational. Curation (or put more traditionally, editing and filtering) is increasingly valuable in the information economy - there are simply too many potential streams of information for anyone to grok, and we need trusted sources to curate it all for us. Google News is a good example of that curation, as are Digg, NewsVine, Wikipedia and any number of other sites - every vertical seems to have similar service - from music (http://www.thesixtyone.com/) to women's interest (http://www.kirtsy.com/).

But here's the rub: There's a critical difference between curation based on algorithm (Google News) and curation based on human insight (Digg or Wikipedia) - and that difference can be summed up in one word: Voice. In short, sites that allow people to be part of the curation process have voice, and sites that are driven by algorithm, don't.

No matter how hard we try, we can't come up with an algorithm that creates a truly human voice. Sure, we can mimic it, but until we solve the Turing Test, the only computer that can create a human voice is, well, a human. And when you put lots of humans together, and give them all a chance to express their voices, you get community-driven media.

Now, how does this all relate to Google Maps and Wikipedia?

In my earlier post, I said "Google Maps isn't very good." That was kind of a cheap shot, because in fact the application is great - if what you need is a Map. But the promise of Google Maps goes well beyond looking at a map - currently you can get driving and walking directions, find businesses nearby, calculate traffic delays, and the like. But it's the promise of what might be layered on top of that where things get really interesting.

If you're like me, you've counted on Google Maps (or a product like it) to get you somewhere, only to find that a road no longer exists, is one way the wrong way, or has a house built in the middle of it. Even more maddening are the routes that seem determined to waste significant amounts of your time (and gasoline) because they are ignorant of local knowledge about better routes (all of these things happened to me when I used Google Maps extensively this summer in Massachusetts).

What might make this algorithmic application better? Why, getting the community involved, of course (often called "crowdsourcing"). When you let the community have a voice in media, that media can get a lot better. Imagine if I could have input all my learnings about better routes while driving around Massachusetts? But crowdsourcing without leadership can digress to chaos. It takes extraordinarily talented community leaders to set, exemplify, and adapt a community's mores toward a common good. Without these leaders, a community will almost always fail. At Wikipedia, a small circle of devoted editors lead the way. At Digg, the same is true. In short, great community media needs great leaders, humans who understand how to conduct the particular chorus that is a site's collective Voice.

My conclusion is simply this: I believe it's impossible to write an algorithm to lead a community. Only people - and uniquely talented people at that - can lead a community. Hence, if Google wants Maps to get better, it has to have leaders who solicit, curate, direct, and cultivate a community. So far, that's not been the business Google - or any other search company - has been in. As search and media evolve, however, I'm certain that must change.