Weighing in on the ‘Wisdom of Crowds’

Psychology and market research have a long history of seeking to understand the self – my motivations, my behaviour, my hopes, my dreams, what I would buy or not buy. Where Freud led, Gallup followed, and most research is based on the assumption that we are experts in our own behaviour.

When it comes to methodological rigor, opinion polling sets the gold standard in market research, and when polling, researchers ask you to predict how you are going to vote in the election. The same is true of NPD research, asking how likely people are to buy a product is believed to be the best way to predict how the product will perform in the marketplace. ‘Me research’, we call it.

Ten years ago, a book intrigued us enough to create a different way of doing things. In his seminal 2004 book, The Wisdom of Crowds, James Surowiecki tells the story of how British Victorian scientist Francis Galton attended a 1906 Plymouth farmers’ fair which featured a contest to guess the weight of an ox. Seeing this as an opportunity to experiment , he calculated the mean of 787 contestants’ guesses (representing the collective wisdom of the crowd), and discovered that the crowd’s judgment was essentially perfect. They guessed that the ox would weigh 1,197 pounds; in fact, it weighed 1,198 pounds.

What Galton stumbled upon that day was the simple but powerful truth at the heart of The Wisdom of Crowds: under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them.

Of course, like many great discoveries, Galton had initiated his study with an entirely different hypothesis and expected outcome. An unashamed anti-democracy elitist, he planned to make a point about the ‘average voter’ – that they were capable of little, with judgment that could not be trusted. In his mind, the average guess of the group would be way off. For an accurate answer, you would need to ask an expert.  Sound familiar?

The lesson Galton learned that day – that a ‘crowd’ can be trusted to give an accurate prediction – had revolutionary implications for research. Perhaps people’s expert opinion on their own behavior wasn’t worth all that much. Perhaps the crowd could get there better. We called this, not ‘me’ research but ‘we’ research – playing to people’s innate social sense to understand and predict others’ behavior. Human beings are unreliable witnesses to their own behavior, so better to ask people how they think other people will behave, than to ask them to predict what they themselves will do.

In 2005, BrainJuicer turned this insight into action. We launched our revolutionary and award-winning idea screening solution, Predictive Markets, where the crowd picks winners by buying and selling shares in ideas. If the name sounds familiar, it’s because it was inspired by findings from the Iowa Electronic Markets (IEM), founded in 1988 to predict the outcome of elections. Across more than 700 elections, the IEM proved to be more accurate than the most accurate poll 75% of the time – dethroning polling as the gold standard to outcome prediction.

Ten years and 35,000 ideas tested later, Predictive Markets has, too, become a familiar, accepted and established research approach. From forecasting the future of fuel to backing Bollywood blockbusters, from predicting the coolest summer drinks to the hottest German singers, from Xmas toys to X-rated toys – and how to name, claim, and find fame – we’ve learned a lot this past decade.

As we celebrate the 10th Anniversary of Predictive Markets, let’s count down our 10 key takeaways about concept writing and screening:

No. 10. Less is more
Aim for 25 words in a description, not the traditional 80+. Length makes people linger with uncertainty rather than reassuring them.

No. 9. Show, don’t tell
Include intuitive visuals (even if you don’t have a pack shot) rather than explaining in detail. You won’t have the luxury of the latter in market.

No. 8. Insight, not insight statements
Your concepts should be insightful, but insight statements at the start of concepts don’t help. Begin with them if it helps you, but remove them before testing.

No. 7. Brand can help or hinder
Testing unbranded concepts isn’t an automatic disadvantage. On average, they score virtually the same as branded ones. Get the brand right, and it can propel your brand to 5-Star success, but an inappropriate brand can lose you points.

No. 6. The crowd is truly wise
Predictive Markets isn’t just fun, fast and easy; it’s accurate. Our most recent validation (on new UK beer and cider launches) saw another strong correlation with market shares: +0.82.

No. 5. You can do more with a crowd
Even in a specialist healthcare category, the Predictive Markets crowd predicted the same winners as standard approaches with physicians – at just 10% of the cost!

No. 4. If you’re in a hurry, don’t worry
It’s possible to tap into the ‘Wisdom of Crowds’ overnight with our CrowdSort version of the tool.

No. 3. Projection aids stability
Using the share dealing metaphor, we see much lower variability in scores between countries and categories than with traditional concept screening.

No. 2. Projection boosts discrimination
The projection sharing metaphor also boosts discrimination. Within projects, we see a much greater spread of results than traditional screening, pulling apart the ‘wheat from the chaff’ for easier, quicker decision-making. It can also spot polarisers, which can be the sign of a breakthrough idea.

No. 1. You can test just about anything!
While purchase intention only makes sense for things you could imagine buying yourself, the share dealing metaphor can be applied to individual variables through to holistic, less tangible concepts that aren’t for sale. Not just NPD ideas, but packs, names, brands, claims, celebrities, singers, politicians, movies… and intimate products, which people are less likely to admit buying!

Want to learn more? Register/request the recording for BrainJuicer’s March 12 webinar: The 10 Things We’ve Learned About Concept Testing.

By Orlando Wood (MD – BrainJuicer Labs), Jim Rimmer (Global Research Director) and Deanna Cullen (Marketing Director).

Read more by Brainjuicer in our Clubhouse.