Survivorship bias: why trader leaderboards mislead

Every copy-trading platform has a leaderboard: the top traders of the last 90 days, ranked by return. It feels like research. It is mostly a list of survivors, and survivors look better than the average trader for a reason that has nothing to do with skill.

The coin-flip test

Start with 1,000 traders who have no edge at all. Each month, each one has a 50% chance of finishing green, purely by chance. After 12 months, about 7% of them (roughly 70 traders) will have 9 or more green months. That is simple binomial arithmetic: 299 of the 4,096 equally likely month-by-month outcomes contain nine or more green months.

Those 70 traders will sit near the top of any leaderboard. They will have a clean-looking record and a story about why it worked. None of them had an edge. The other 930 are not on the list, so you never see them.

Three ways the list flatters itself

1. Dead accounts disappear. A trader who blew up or quit simply drops off the ranking. The average you see is the average of whoever is left.

2. The time window is chosen for you. A 90-day ranking rewards whoever was positioned for the last 90 days of market direction. In a rising market, leveraged longs fill the top ten. The same names often vanish when the regime changes.

3. Return is ranked, risk is not. A leaderboard sorted by ROI puts the biggest risk-takers first. The ones whose bets landed look brilliant, and the ones whose bets did not are gone (see point 1).

Three questions to ask about any top trader

• How long is the record, and across more than one market regime? Under three months is a streak, not a record. Check how the trader behaved in a falling or sideways market too.
• What did the worst stretch look like? Maximum drawdown and the longest losing run say more than the headline return. See our note on reading a drawdown chart.
• Is the sample big enough? A few dozen trades cannot separate skill from luck. Our note on the first 50 live trades covers what small samples can and cannot show.

How we apply this to ourselves

We have the same problem in reverse. A strategy book that only displayed its best strategies would be a leaderboard of survivors. So the lab page lists every strategy in the book, including the ones below a profit factor of 1.0 and the ones we have killed, and our kill criteria are set before a strategy goes live. Even then, a short live record is still a short live record.

The honest summary

A ranking tells you who is ahead right now. It does not tell you whether they are ahead because of skill, and it hides everyone who dropped out along the way. Treat a leaderboard as a place to start asking questions, never as the answer.