Smarter together: learn the building blocks of collective intelligence

Transcription

In this lesson, you will learn about one type of collective intelligence that emerges when many people attempt to guess something. In a nutshell, the idea is that the consolidated guess of many, the crowd’s guess, is often much more accurate than individual guesses. This phenomenon is known as “the wisdom of crowds”. Crowd forecasting simply leverages this stunning ability of crowds to make reliable collective guesses about the future. We’ll go into forecasting in the next lessons, but first, let’s learn about the wisdom of crowds.

The phenomenon of the wisdom of crowds was discovered in extremis in 1906 by the British scientist Francis Galton, one of the foremost statisticians of all time. I say “in extremis”, because Galton achieved the ripe old age of 84 years old! – and died just a few years after publishing this last discovery.

Galton was visiting a country fair when he came across a crowd of several hundred people whose attention was focused on an ox. They were participating in a contest to try to guess the weight of the animal, or to be precise, to guess the weight of the meat that would come from butchering it. Each participant paid a small entry fee and wrote their best estimate on a ticket. The most accurate of these estimates would then win the contest, along with its meaty prize.

Why would Galton find this interesting? Well, he realized that he could use these estimates to find out whether a large number of non-specialists could make a smart collective judgment. At the time, the United Kingdom was debating the idea of a more democratic form of government, making the wisdom of crowds a heated topic. As a member of the kingdom’s elite, Galton was initially skeptical of the ability of laypeople to make accurate judgments.

To investigate the matter, Galton arranged to retrieve all the tickets after the contest. He arranged the 787 estimates in ascending order from the lowest, at around 400 kg, to the highest, at around 700 kg. He then identified the middle-most estimate, called the median. The median is very special because it is the only estimate that isn’t considered as too low or too high by a majority of the participants, meaning there are exactly the same number of estimates above as there are below the median. This brought Galton to declare the median to be a measure of the vox populi, the people’s voice. In other words, the choice of the crowd as a whole.

This median estimate, the crowd’s estimate, was 548 kg, just 5kg away from the actual ox’s weight of 543 kg. Galton later computed the average of the estimates – another way to consolidate the crowd’s guess – and it turned out to be even more accurate, a mere half a kg away from the actual weight.

The 3 principles of crowd wisdom

The wisdom of crowds obeys 3 laws, which Galton’s example illustrates perfectly. The first is that collective judgment is generally smarter than most individual judgments. We can test this claim by comparing the average error of an individual within the crowd, to the error of the full crowd average. In this particular case, the average individual error – that is, how much error you’d expect picking someone at random – was about 4.5%, while the error of the crowd as a whole was essentially 0%. So this checks out nicely, leading us to the first law of crowd wisdom :

1. “the error of the average is smaller than the average error.“

The second law is that crowd size matters: 2. Larger crowds are smarter. Look at how the error of the crowd diminishes as we include more and more individual estimates in the average.

The third law is visible in the shape of the crowd-wisdom curve. 3. It exhibits diminishing returns. As you increase the size of the crowd, the error falls sharply at first, and then more and more slowly. A group of 10 has an error that is 3 times smaller than the average individual’s. But it takes 100 people to divide the error by 3 again. And then a crowd of 450… The implication is that most of the collective-intelligence benefit is obtained at the start: even small crowds are a lot smarter than individuals, and you don’t need a huge crowd to be a lot wiser.

The math behind crowd wisdom

Ok, so far we have described the wisdom of crowds phenomenon, but we haven’t truly explained how it works. By what magic can a smart collective judgment emerge from the combination of many flawed individual judgments? Of course, it’s not magic… it’s mathematics. The central idea is that everyone’s judgment contains both information and error. In many cases, every judgment is flawed in a different way. So when they are combined, information adds up, while errors cancel each other out.

For example, in the case of Galton’s ox, someone may have underestimated its weight by 50 kg, while someone else could have overestimated it by 47 kg. When the two erroneous judgments are combined through averaging, the result is just 1.5 kg below the real weight.  An almost perfectly accurate judgment.

Scott Page’s diversity theorem

The mathematical formula that powers the wisdom of crowds is called “the Diversity Theorem”. It was discovered by the American sociologist Scott Page. It looks complicated, but what it says is very simple, yet profound: collective error equals average individual error, minus the diversity of the estimates. In other words, the size of a groups’ mistake is reduced when individual errors are smaller – that’s obvious – but it is also reduced when different people make different mistakes. That’s the diversity component. Because of the minus sign here, the more people disagree, the smarter the group becomes. That’s why a diversity of opinions, which implies a diversity of errors that can cancel each other out, is so important to collective intelligence.

So you can read this formula as simply that a group’s collective intelligence depends as much on individual expertise as on diverse opinions. Expertise and diversity are complementary and interchangeable. If most people aren’t experts, you can compensate by seeking a higher diversity of opinions. Or if everyone thinks alike, then they’ better be very knowledgeable.

Expert vs Amateurs: predicting jury awards

Here’s an example from an experiment that was run back in 2011 by a group of American and Israeli researchers. It looked at people’s ability to predict the results of trials in the United States, specifically the size of jury awards. Some participants were experienced attorneys with 20 years of professional experience, while others were students at Stanford University Law School, very bright no doubt, but with no particular expertise. As you can see, the average mistake of a student was twice as large as the average mistake of an attorney. Expertise makes a huge difference! Yet, when the estimates of several students are combined, the diversity of opinions in the group makes up for what it lacks in individual expertise. The students’ collective error diminishes so that a group of 14 is just as smart as an expert. And a group of 15 students is slightly smarter still…

But experts can also disagree, and they often do, so their opinions can also be combined for more accurate judgments. Combining just 2 expert opinions increases accuracy by 25%. And now it takes 42 students to match the performance of two attorneys. Sadly, experienced attorneys are bad at taking advice : they give less weight to their peer’s estimates and therefore fail to reap the benefits of the crowd’s wisdom.

James Surowiecki’s collective intelligence recipe

All this talk about wise crowds is all well and good, but haven’t we all encountered deeply stupid groups? Has each and every one of us not personally witnessed that even smart people – perhaps even ourselves – can become stupid when they become part of a crowd? Of course, collective stupidity also exists. There are smart crowds and stupid crowds, and most crowds are indeed naturally stupid. To make a crowd smart, you need to organize it properly. The recipe for collective intelligence, adapted from the famous book by American journalist James Surowiecki, tells us how:

First of all, everyone must be incentivized to think, and to do so independently. Independence allows for a diversity of opinions to flourish, by allowing and encouraging individuals to express their unique perspectives. The enemy of intelligence is conformity, not the crowd itself. A smart group should seek to surface all diverging opinions instead of seeking consensus. Then, a final step ensures the group can make decisions despite the disagreements: objective aggregation.

The most relevant aggregation mechanism depends on what a given crowd is trying to do. If it is tasked with choosing a leader, a majority vote is a good solution. If it is asked to guess the weight of an ox, a mathematical average of all estimates will do fine. If it is trying to predict the future, letting people place bets on outcomes is a powerful way to consolidate divergent forecasts. In essence, this is what crowd forecasting is about: using crowd wisdom to predict the future.