Learn the 4 habits of the very best forecasters

Full transcript:

So now that you know what forecasting is, how do you become good at it? On second thought, is there even such a thing as a good forecaster, or is the future so unpredictable that even the smartest forecaster might as well just flip a coin?

Fortunately, we have an answer to this question! Forecasting is a matter of skill, not just luck, and this skill can be learned.

How do we know this? Between 2011 and 2015, the US Intelligence Community – the spy agencies – ran a forecasting contest in which thousands of amateur forecasters, like you, competed to see who could best predict the outcomes of important world events.

Forecasters competed to answer questions such as:

Will the Kurdish Regional Government hold a referendum on independence this year?

Will India or Brazil become a permanent member of the UN Security Council in the next two years?

This contest was part of a larger research project aimed at improving the forecasting capabilities of US intelligence analysts.

The project succeeded beyond expectations. According to some reports, amateur forecasters in the contest even outperformed professional intelligence analysts with access to classified information, by 30%!

But forecasting performance also varied widely across individuals, so the most valuable outcome of this research project is how it sheds light on what makes a forecaster better than another. The good news is that it isn’t so much about what you already know, how expert you are in a domain, but much more about how you think!

But first let’s talk about what isn’t required to be a great forecaster. Having a high IQ helps, but it’s really not the best predictor of who becomes a great forecaster. And while the best forecasters tend to be good with numbers, they seldom use equations when making their forecasts. Nor is it critical to be an expert. In fact, research shows that experts are often among the worst forecasters! Expertise is generally more useful to explain what has already happened, than to forecast what will happen.

So if it’s not I.Q., math or expertise, what does make a good forecaster? The secrets of the best forecasters are condensed into three principles we call the “ABCs of great forecasting”.

A is for “Attitude”: seek out opposing points of view, enjoy uncertainty

The best forecasters are humble. They’re comfortable with uncertainty and don’t take it personally when their forecasts don’t pan out.

They are actively open-minded. That means they seek out clashing points of view. They like to debate with themselves, learn from others who disagree with them, and use language like “on the one hand…, on the other hand…”

The best forcasters are also growth-oriented. They have what psychologists call a “growth mindset.” That is, they believe they can improve with practice (and the research shows they can!). They consider their failures as part of a learning curve, and they don’t give up.

B is for “Break it Down”: divide complex problems into manageable parts

It’s important to break any question you forecast down into relevant parts – that is, identify the important factors that are involved so that you know what you need to research.

For example, if you are trying to make a forecast about the chances that France will win the next football world cup, you might need to break the problem down into at least the following components:

Does France currently feature top playing talent?

Does the team have a good coach?

When is the last time that France won a world cup?

B is also for “Balance”: see problems both in general and specific ways

A great forecaster also has to balance how they look at a question by taking a microscopic view, called the “inside” view, and a macroscopic view, also called the “outside” view.

The “inside view” refers to the unique attributes of a question. These are key factors that are relevant only in the specific context of the current prediction you are trying to make.

In contrast, the “outside view” considers only what has happened in similar situations in the past. To take the outside view, you need to identify similar situations and compute base rates. As described earlier, base rates are the average occurrence of a particular outcome, that is, how often on average does the event play out a certain way in similar situations.

A great forecaster will start with taking the outside view on the situation, and then take a closer look with the inside view.

Consider our example about forecasting the chances that France will win the next world cup. Do you have any idea? It’s ok if you don’t, but if you do, take a few seconds to write your estimate on a piece of paper… before we work on it together.

Ok, let’s see if we can come up with a reasoned estimate. Taking the outside view, we ask: How often does France tend to win the world cup on average? If we look at the most recent 10 world cups, we see that France won twice. That gives us a base rate of 20% chances of winning. At the same time, we also see that in recent WorldCup history no team has ever won twice in a row. So that’s a second base rate of 0%! If we combine the two base rates by, for example, averaging them, that gives us a 10% chance that France will win.

The inside view will then help us refine that estimate by looking at the specifics of the current situation. We observe that France currently has some top talent, with Kylian Mbappé among the top 10 best payers in the last FIFA ranking. Furthermore, France’s coach Didier Deschamps, himself a world-cup champion, is the same who has coached the team to victory the last time around. These observations taken together might drive us to revise our base-rate estimate upwards by 5%.

By combining the outside and inside views, we may forecast that France has a 15% chance of winning the next world cup.

C is for “Commitment”: make small adjustments to your prediction

It also takes commitment to be a great forecaster: making a forecast isn’t just a one-time job! The key to succeeding is to update your predictions regularly. Look at how these three forecasters are updating their probability forecasts over several months. As events change, so should your predictions. Maybe a key actor will be forced to resign – Kylian Mbappé could have an injury – or an unexpected event could change the terms of the competition – for example, if a favorite team like Argentina loses early in the tournament. These changes matter, and the best forecasters make sure to update their predictions as the situation on the ground changes.

But be careful not to overreact to new developments, like Forecaster A, in red, does in this example. The natural tendency is to consider that whatever happened most recently is also the most important, but that is often not the case. Research shows that the best forecasters don’t overreact by making a few big changes to their forecasts, but instead prefer to continuously make many small adjustments over time. In this example, Forecaster C, in grey, is likely the most reliable forecaster.

Congratulations! You are now ready to take on any prediction challenge that comes your way. Remember that the key to success is to practice the ABCs of great forecasting:

• Having the right attitude: a mix of humility, active open-mindedness and a growth midset;
• Breaking the problems down into manageable parts;
• Balancing the outside and inside views by considering the base rates and the specfics;
• And committing to update your predictions regularly and thoughtfully.

If you can master the ABCs, you will own the future.