The Human in the loop

I’ve worked with big systems for most of my career (first with server systems, and now with cloud services). Scale and performance have always been a concern, and I’m impressed with systems that work at very large scale. Search systems like Google and Bing are particularly impressive in their scale.

And yet, I’m always delighted with systems that are able to effectively use the human in the loop.

Using the human in the loop is not a new idea. When problems in computer vision failed to get tractable, research began looking at interactive computer vision – where you use the human in the loop. Amazon’s Mechanical Turk (what Jeff  Bezos calls Artificial Artificial Intelligence) is a straightforward example of such interactive pattern recognition.

But how do you engage the human? Yes, there is “wisdom in the crowd”, but how do you get the crowd to participate in solving the problem you’re interested in solving? Giving money is one solution – which is what Amazon’s Mechanical Turk does. But that strategy has backfired in some cases. There used to be an early review site called which actually paid top reviewers a portion of the profit that their reviews generated. It didn’t work out very well. And yet, which pays no money out to its reviewers, continues to grow at enormous rates (80% growth last year). Before Facebook harnessed their user-base to translate their website into Spanish, Turkish and a host of other languages, I would have never thought that would have worked.

And the most impressive example for me way back when was – a “semantic search engine” that harnessed its community of users to answer questions. And yes, there have always been bulletin boards and mailing lists, but I think Yahoo was the first website to do it in a broad way.

I think the first trick is to create a sense of community. Creating a warm, inviting website where users want to hang out. Yahoo used to do that very well. Yelp also does that. Facebook, of course, has a very engaged community. The second is to set incentives just right. Yelp in particular, highlights top reviewers. It rewards top reviewers non-monetarily at Yelp events and such.

It requires right-brain creativity to do both – create a “social” website where users want to hang out, and then nudge them appropriately to engage – which is why I think it is rarer and more delightful to watch when it happens.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s