Ask What Is Missing to Avoid Survivorship Bias

It's often easy to focus only on what is in front of you, on the things that survived. However, it is important to ask what is missing, as that often provides important insight. That which passes the selection process may have special properties that allowed for survival, or it may be a coincidence and simply correlational.

For example:

When deciding where to put more armour on fighter planes, don't focus on where the bullet holes are on planes that come back. Focus on where the bullet holes are not, which are presumably on the planes that don't come back. More armour should be placed where the engines are, as planes that are hit there do not come back.

Judging a portfolio based on the current mutual funds in the portfolio does not tell the whole story. The current funds were the ones that survived, but the ones that lost money did not survive and got removed from the portfolio. Similarly, mutual funds typically go through an incubation period, where the low performers get removed. However, the mutual funds that remained are not necessary special. They just survived, giving the appearance that there was something special about the fund.

In the parable of the Baltimore stockbroker, he sends his predictions to thousands of people. The ones that responded got the correct predictions. He does the same to the survivors and repeats this until only a few are left. To these survivors, it appears that the Baltimore stockbroker has predictive powers, but in reality there were many others that got the wrong predictions and did not survive.

References

How Not to Be Wrong - The Power of Mathematical Thinking by Jordan Ellenberg