Why investment managers' performance tends to be disappointing after they are hired

Institutional investors often have the discouraging experience of hiring an investment manager who, despite good historical performance, has disappointing performance after being hired. One reason for this is the difficulty most institutional investors have assessing the a priori probability that an investment process is effective. Another is an excessive reliance on good past performance.

Some sophisticated institutional investors compute the probability of observing a performance history as good as the manager’s or better, assuming that the manager’s investment process is not effective. In statistical parlance, this is a test of the null hypothesis that the manager’s true performance is zero (i.e., due to luck).

If this probability is substantial, then the observed performance is consistent with the hypothesis that it is due to luck. If this probability is small, then the assumption that the observed performance is due to luck implies a great coincidence and it may be more reasonable to assume that it was due to an effective investment process.

Typically, the historical performance is attributed to an effective investment process if the computed probability of achieving the observed performance or better from luck is below 5% or thereabouts. What most investors who use this approach do not realize is that this choice is arbitrary and virtually assures disappointing future performance.

Suppose the probability of achieving the observed performance or better from luck is 1%. Is that sufficiently small to attribute the observed performance to an effective investment process? Doesn’t the answer depend on what is the investment process? Suppose the performance was achieved by choosing stocks at random? Or, to put it more accurately, suppose the performance was achieved through a process that one cannot imagine should work? Or, to put it even more accurately, suppose the investment process has an a priori probability of being effective of 0.000001%?

On the other hand, suppose the probability of achieving the observed performance or better from luck is 25%. Is that sufficiently large to attribute the observed performance to luck? Doesn’t the answer depend on what is the investment process? Suppose one cannot imagine that the investment process would not work? Or, to put it more accurately, suppose the investment process has an a priori probability of being effective of 99%?

What the typical statistical testing leaves out is the a priori probability that an investment process is effective.

What investors ought to estimate is the probability that the investment process is effective, given the observed performance. This depends on both the a priori probability that the investment process is effective and the probability of achieving the observed performance or better from luck (i.e., given that the investment process is not effective). Investors almost never compute, numerically or subjectively, the probability that the investment process is effective, given the observed performance. The reason is that they are usually unable to assess an investment process from an a priori perspective.

To see why the typical approach of computing the probability of the observed performance or better due to luck fails, consider that there are thousands of managers and only those with good past performance show up at the institutional investor’s door.

Here is a dramatic illustration of the kind of thing that goes on. Assume there are 1000 managers and that none of their investment processes are effective. Approximately fifty of them should have past performance that is good enough, due to good luck, so that the computed probability of observing performance as good or better is 5% or less. Naturally, these fifty show up at the institutional investor’s door and the results of his statistical tests are a foregone conclusion. The expected future performance of the fifty managers is zero. The investor’s statistical tests are virtually worthless.

The institutional investor’s only protection is to have a very good a priori reason for thinking that an investment process should work.