The 4 conditions required to build a wise crowd.
There is a time every year that I see more of my best friends than I do at any other time of the year. What have they been up to? How’s their work going? Have they got any holidays planned? I couldn’t tell you any of that. We’ve been watching rugby, and for 7 thrilling weeks in February and March each year it’s the only thing we talk about.
The Six Nations Championship for me is a sporting contest unmatched by any other on earth. It’s exhilarating, frantic, surprising and captivating, often all at the same time. The championship focuses on six European rugby nations as they battle for rugby supremacy in a rousing contest that sees each country pitch 15 of their sons against 15 from each other nation, with a total 15 fixtures taking place over 5 intoxicating match days.
For proud Scotland fans like me, it is typically a bruising experience. Scotland won the final Five Nations tournament in 1999, before I was old enough to care, and before Italy joined in 2000 to make it a six-country festival of rugby. Since then the competition hasn’t been terribly kind to either nation, with Italy finishing last on 13 occasions and Scotland taking home the ‘wooden spoon’ four times.
It is perhaps for that reason that in the last few years that I have looked for more ways to bring excitement and intrigue to the championship. If Scotland is going to lose, how can I still win? My answer was a little bit of innocent and recreational gambling among friends (although I still lost…).
For the last couple of seasons, I have been running a Six Nations predictions league for my friends, and it is the predictions league that has inspired this blog. The rules are fairly simple – predict the scores for each block of three matches every weekend throughout the championship to win points. If you correctly predict the result (ie. the match winner) and the margin of victory (8 points or more, or 7 points or less) you get 3 points. 2 points are awarded if you get the match winner right but not the margin.
This year, inspired by James Surowiecki’s book, The Wisdom of Crowds, I was curious to see how my friends would perform as a group, not just as individuals. In the book, Surowiecki argues that the aggregation of information in groups can be better or more accurate than if made by any single member. So would our collective wisdom outperform us as individuals?
Including me, there was appropriately 6 of us that entered the league. We still all submitted our own predictions, and to get a crowd prediction I simply took an average of the scores for each match.
With six matches played, the wisdom of the crowd topped the ladder. On matchday 2, the crowd had had a perfect weekend, correctly predicting the winners and margins of each fixture. This would prove to be the only time the crowd or any individual achieved this throughout the tournament.
The crowd would lose it’s lead after the eighth fixture however and never again regain top spot. It would eventually finish in second place, two points behind the leader. Second place was a great result for the crowd, but I was disappointed that it didn’t finish on top (and not just to give something with which to wipe the smile off of my friend Neil’s smug face). So what do I take from this?
One of Surowiecki’s lessons is that diversity in a crowd is important. The fact that the six of us were all male, white, Scottish, under 34 but over 30 is not a good start. A crowd cannot be wise if everyone picks the same answer. Collectively we learnt this the hard way, all believing the pre-tournament hype that this could be “Scotland’s year”, each of us predicting that Scotland would win a tight match against Wales in Cardiff in the opening fixture on the opening day of the championship. Wales won that match at a canter, sending the Scotland team and its fans (including me) home “tae think again”. They say it’s the hope that kills you…
Surowiecki also claims that for a crowd to show wisdom independence of thought is crucial. That is there should be no opportunity for a crowd member to influence another’s thinking. In my less than bulletproof experiment, this would seem to be backed up in the results. After six matches, around the time that the crowd lost its lead, I found myself starting to take notice of my friends’ predictions on the shared spreadsheet used to collect them before I recorded my own. It stands to reason that my own predictions were therefore influenced by those that went before me. In the last two rounds of fixtures I even deliberately made opposite predictions to the league leader even if I agreed with them, knowing that to put myself in a winning position I had to win points he didn’t. I’d be astonished if others weren’t doing the same thing, and for each one of us that did, the crowd’s chances of winning were diminished.
“The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes.” James Surowiecki
Surowiecki also argues that a further critical condition for a crowd to prove itself wise is decentralization. Crowd members need to be able to draw on local knowledge and use different approaches. It is highly likely however that the members of the prediction league were all reading the same rugby articles and blogs and watching the same and listening to the same pundits.
Too much communication can also make a group, on the whole, less intelligent. It is perhaps telling that the winner of the league was the one of us who don’t live in London and therefore didn’t have the same opportunity to hang out with the rest of us and talk rugby.
The mechanism I used to turn the private predictions into a collective decision was not without its flaws. My crowd was also on the small side which meant the aggregated data had little variety.
At 100%Open we have been running online crowds for our clients for several years, connecting them with global networks of consumers and entrepreneurs to provide better, faster and cheaper insights, ideas and innovations that they would ever manage on their own. We believe in the wisdom of the crowd because we experience it every day. Maybe now is the time for you to embrace it too.
This year Scotland didn’t win and I didn’t win, but next year will be Scotland’s year. Surely. The wisdom of the crowd would in all likelihood paint a very different picture.