Skill, Luck, and the Problem of Telling Them Apart

Every bull market manufactures geniuses.

Some of them are. Most of them were standing in the right place when the wind blew. The uncomfortable part is that from the inside, and from the outside, the two look identical for a surprisingly long time.

Here's how to tell them apart, including when the person you're evaluating is you.

One year proves almost nothing

A single year of returns, even an excellent one, contains far more noise than anyone wants to admit while they're the one enjoying the result.

Short samples are too short to separate a repeatable process from a favorable roll of the dice, no matter how good the story sounds while it's happening. This isn't a technicality. It's the central fact, and almost every mistake in this area comes from ignoring it.

Factor exposure wearing a skill costume

A manager who beat the market because the portfolio carried meaningful small-cap, value, momentum, or sector concentration wasn't necessarily skilled. They may simply have been exposed to something that happened to be rewarded during that stretch.

Run proper attribution and control for those exposures, and apparent skill often shrinks considerably. Sometimes it disappears almost entirely. The return was real. The explanation was wrong.

This applies to individual investors at least as much as to professionals. If your good year came from being overweight one sector that happened to lead, you learned less about your process than it felt like you did.

The thousand coin flippers

Here's a way to feel the scale of the problem without any statistics.

Imagine a thousand managers, each flipping a coin every year to decide whether they beat the market. Pure chance makes it overwhelmingly likely that some will look brilliant for three, four, even five years running. Nothing behind it but a favorable sequence.

The industry produces a great many track records every year. Some will look exceptional by chance alone, and there is no way to tell which ones from the track record alone.

Read that last clause again, because it's the part people resist. Not "it's hard to tell." You cannot tell, from the record by itself. You need something else.

A reported number is a range, not a point

Any estimate of a manager's true skill, or your own, should be thought of as a range rather than a precise figure.

A reported four percent above a benchmark might represent a genuinely strong repeatable edge. It might represent something close to zero that got lucky this particular year. Both are consistent with the same reported number, and treating a point estimate as if it carries no uncertainty around it is among the more common ways investors fool themselves.

You don't need advanced statistics to hold the intuition: the shorter the record and the smaller the edge, the less confidently you should treat it as proof of anything.

Persistence helps, with a caveat

A better question than "did this beat the market" is "does this outperformance persist across different periods and different market environments."

The caveat matters though. A lack of persistence doesn't automatically prove a lack of skill. Markets change, mandates change, and a manager's asset base can grow large enough to blunt a strategy that worked well at smaller scale. Persistence is meaningful evidence. It isn't the whole verdict.

The five questions that actually work

Since the track record alone can't settle it, ask about the process instead. After any outcome, win or lose:

Did the thesis play out the way you expected it to?

Was the position sized appropriately given the conviction and the risk involved?

Did the gain, if there was one, come from the mechanism your thesis predicted, or from something else that happened to push the price the same direction?

Did risk stay within the plan you set out in advance?

And the sharpest one: would you make the same decision again today, knowing only what you knew at the time, regardless of how it actually turned out?

That last question is the whole discipline compressed into a sentence. A good decision that lost money is still a good decision. A bad decision that made money is still a bad decision, and it's considerably more dangerous, because it teaches you the wrong lesson and you'll repeat it with more size.

Attribute decisions, not just returns

Most investors who do any self-assessment stop at the portfolio's overall return. Go further and attribute the individual decisions that produced it: the buy, the sizing, the sell, any macro tilt layered on top, any hedge applied.

This is more demanding than almost anyone actually does, professional or otherwise. It's also the habit that separates an investor who is genuinely improving a process from one who is accumulating a track record and hoping it holds.

What to do with this

Take your best trade of the past year. Not your biggest, your best, the one you feel smartest about.

Now answer the third question honestly: did it work for the reason you thought it would?

If the stock went up because of something your thesis never mentioned, you got paid for being lucky. That's fine, keep the money. But don't file it as evidence about your process, because it isn't, and the file is where the damage happens.