Every free parameter in a backtest is a question the data has to answer. A trading strategy answers those questions with closed trades, not with bars of price history. This note sets out the rule of thumb KreamEdge applies to the in-sample window: at least 30 closed trades per effective parameter, and never fewer than 400 trades in total. It then covers how we count a parameter, the case where the rule rejected one of our own strategies, and the overfitting it does not catch.
This is research methodology. The figures below come from historical backtests on KreamEdge’s internal research bench, not from a live track record, and nothing here is a trading signal. We would not trade any strategy described here as-is: it is research only. Past performance is not indicative of future results. Not financial advice. See our community page for the full disclaimer.
Key points
- KreamEdge requires at least max(400, 30 × effective parameters) closed trades in the in-sample window before a backtest result counts as evidence.
- The number 30 is a practitioner heuristic, not a theorem. It is three times the 10-events-per-variable rule from regression studies, because trade returns are fat-tailed and carry less information per observation.
- A parameter is any choice that restricts which trades happen. That includes rule conditions and fixed filters as well as the numbers the optimizer varies.
- Position sizing cannot fix a low trade count. Halving position size moved one KreamEdge trend sleeve from 114 to 121 trades, because the selection rule decides how often a strategy trades.
- The rule limits how many parameters one fit can support. The number of configurations tried needs a separate budget, and the out-of-sample window needs its own floor of 80 trades.
What is a good rule of thumb for trades per parameter?
A workable rule of thumb is at least 30 closed trades in the in-sample window for every effective parameter, with an absolute floor of 400 trades whatever the parameter count. KreamEdge applies it as max(400, 30 × effective parameters). No statistics paper fixes the number 30 for trading. It is a practitioner heuristic, set deliberately stricter than the regression rule it borrows from.
The closest academic anchor comes from medical statistics. Peduzzi and colleagues (Journal of Clinical Epidemiology, 1996) simulated logistic regressions and found estimates stable at 10 or more events per predictor variable. Below that, coefficients were biased and confidence intervals unreliable. Later work, such as Vittinghoff and McCulloch (2007), argued that 10 can sometimes be relaxed.
Backtests are a harsher setting than clinical trials. Trade returns are fat-tailed, they cluster by market regime, and a trend strategy’s result often rests on a small minority of large winners. Each closed trade therefore carries less information than one independent clinical event. Tripling the ratio to 30 is KreamEdge’s margin for that difference.
Systematic traders agree on the shape of the rule more than on its number. On the Better System Trader podcast, Dave Bergstrom said he will not allow strategies below “a certain trade to parameter ratio” (episode 79), and Bert Mouler said to keep the trade-to-parameter ratio high, counted on out-of-sample trades (episode 64). Neither gave a figure. Michael Bryant named the unit that matters:
You judge the amount of data you need based on number of trades, not necessarily number of bars of price data.
Michael Bryant, Better System Trader episode 36
The guests who did give numbers gave absolute trade counts, not ratios:
| Source | Number stated | What it applies to |
|---|---|---|
| John Ehlers, BST episode 152 | More than 20 trades; about 40 in practice | Each in-sample window of an intraday walk-forward |
| Jeff Swanson, BST episode 196 | Over 100 trades | What remains after filters cut the baseline |
| Stefan Friedrichowski, BST episode 178 | 3 degrees of freedom | A mean-reversion strategy, counted explicitly |
| KreamEdge | max(400, 30 × effective parameters) | The full in-sample window of one fitted strategy |
Why count trades instead of bars?
A strategy learns from closed trades, not from bars. Ten years of daily data is about 2,500 bars. A selective strategy that trades 15 times a year has produced only about 150 outcomes over the same decade. The optimizer fits its parameters to those 150 outcomes, so the trade count, not the bar count, decides how many parameters the data can support.
The same logic applies to any component that switches the strategy on and off. Adrian Reid described a regime filter that flips four times in a backtest holding 500 trades. The filter’s own sample is the four switches, not the 500 trades:
You’ve optimized something on basically four data points, four on-offs, which is a real big mistake.
Adrian Reid, Better System Trader episode 206
Adding history helps less than removing parameters. Rob Carver pointed out in episode 180 that the precision of a parameter estimate grows with the square root of the sample length. Doubling the history from five years to ten improves that precision by a factor of about 1.4, not 2. Removing one parameter, by contrast, cuts 30 trades from the floor straight away.
What counts as a parameter? Restrictions, not optimizer settings
A parameter is every choice that restricts which trades happen, not only the numbers the optimizer varies. KreamEdge’s effective count adds three things: the free parameters, the conditions inside the chosen entry and exit rules, and fixed filters such as “close above the 200-day average”. A fixed filter does not become free because nobody tuned it in today’s run.
Practitioners count the same way. Michael Bryant (episode 36) counts a moving-average crossover as two restrictions, one per lookback, plus one more if the price field is also a choice. Andrea Unger (episode 16) said he never counts his degrees of freedom, but that a generic 200-day moving-average filter is “still a constraint”.
Carver’s broader point is that manual tinkering counts too. Swapping a simple moving average for an exponential one after looking at the backtest is a fit, even though no parameter changed:
You think, oh, my soul is clean, I haven’t changed any parameters, I’ve done no fitting. You have… you’ve completely changed the model.
Rob Carver, Better System Trader episode 180
Not every restriction costs a full parameter. KreamEdge weights each kind of parameter on a table that is fixed before any run, never adjusted per candidate after seeing results:
| Kind of parameter | Weight in the count | Reason |
|---|---|---|
| Continuous value fitted in this search | 1.0 | A full degree of freedom |
| Integer with 8 or fewer plausible levels | 0.5 | Fewer distinct values to fit |
| Binary on/off gate | 0.25 | One choice between two states |
| Frozen value, never tuned on this data | 0 | The data never chose it |
| Frozen value, tuned earlier on overlapping data | Half its original weight | Freezing removes it from today’s search, not from yesterday’s |
Every condition also divides the sample, which is how parameter counts and trade counts interact in practice. Scott Andrews described splitting gap trades into more size categories, which cut the average category to about 125 trades, and then adding a 200-day filter, which halved it again:
So as you start adding in just a few simple ideas, you end up with too few trades to really trust the data.
Scott Andrews, Better System Trader episode 18
How the trade floor scales with the parameter count
Under max(400, 30 × parameters), the flat 400-trade floor binds for any strategy with up to 13 effective parameters. Above that, each extra parameter costs 30 more in-sample trades. A 15-parameter search needs 450 trades and a 20-parameter search needs 600. KreamEdge therefore caps every optimizer run at 15 free parameters and aims for 10 or fewer.
| Effective parameters | 30 × parameters | Required in-sample trades | Trades per year over a 7-year window |
|---|---|---|---|
| 5 | 150 | 400 | about 57 |
| 10 | 300 | 400 | about 57 |
| 15 | 450 | 450 | about 64 |
| 20 | 600 | 600 | about 86 |
| 25 | 750 | 750 | about 107 |
The flat 400 comes from KreamEdge’s own bench rather than from theory. When bench strategies were grouped by trade count, the share with a positive forward CAGR was 36% below 200 trades and 85% between 400 and 700 trades. That is an observational pattern on already-filtered candidates, not proof that more trades make a strategy better. It does show that results resting on fewer than 200 trades told us very little.
The rule also caught KreamEdge’s own model. When the scoring model carried 11 fitted weights, the floor was 330 trades. Across 29 quality-filtered strategies on the bench, the mean was 374 trades and the median 283, so the median strategy sat about 14% below its own floor. That measurement pushed the research toward smaller per-family parameter sets, now fewer than 10 values each.
When the rule says no: 114 trades in 7.4 years
The rule exists to refuse strategies, and it refused one of KreamEdge’s. Our incumbent trend sleeve produced 114 trades over the full 7.4 years of history available. An honest count of 8 to 10 effective parameters puts its floor at 240 to 300 trades, even under a relaxed 100-trade minimum. The sleeve failed by construction, not by bad luck.
The 8 to 10 count came from the weighting table, not from the optimizer’s settings. It covers five fitted score coefficients, a trailing stop, a holding-duration limit, and two gate thresholds at half weight because they were tuned earlier on overlapping data. The conditions inside the entry rule account for the upper end of the range.
Two probes tested whether the trade count could be engineered upward. Raising the cap on entries per bar reproduced exactly 114 trades, so the cap was never binding. Halving position size to 20 slots gave 121 trades at 9% exposure. Sizing cannot create fills: the selection rule decides how often a strategy trades, so a trade count this low is structural.
The floor itself had already been relaxed once, from 400 to a minimum of 100 trades, because 7.4 years of history could not support the original. That change was a dated amendment applied to every candidate, and it came with mandatory luck controls in exchange. Even the relaxed floor rejected the sleeve.
That left two honest options. The first was a labeled, dated exception, tracked on paper only, with the stronger luck controls attached. The second was to pick a different, denser strategy designed to clear the floor and run it through the full validation stack from scratch. What the process excludes is a floor changed for one candidate after seeing its result.
Trades per parameter is not enough: count the trials too
The trades-per-parameter rule limits how many parameters one fit can support, but it says nothing about how many fits were run. Bailey, Borwein, López de Prado and Zhu (Notices of the AMS, 2014) show that with five years of daily data, trying more than about 45 independent configurations makes an in-sample Sharpe ratio of 1 likely by chance alone, with an expected out-of-sample Sharpe of zero.
Their Minimum Backtest Length is a second budget, set in years of data per configuration tried. A strategy can clear 30 trades per parameter and still be the lucky winner of 10,000 trials. Dave Walton made the same split in episode 51: the number of rules is one driver of data-mining bias, and the number of variants searched is another.
KreamEdge prices the selection itself. The whole shortlist is re-run on price series whose day-to-day order has been shuffled, which keeps each series’ return distribution but destroys any real pattern. The best real configuration must beat the 95th percentile of the best shuffled configurations: at least 20 shuffles for research, at least 100 before promotion. The test only counts when it re-runs the whole selection process, not just the finalists at their already-fitted values.
Out-of-sample windows need their own trade floor
A holdout window needs enough trades to be measurable, and that floor is separate from the in-sample rule. KreamEdge marks a holdout with fewer than 80 trades as “insufficient data” rather than as a failure. Below that count the holdout result is noise, and grading it would confuse “failed the check” with “could not be checked”.
The 80 comes from grouping 61 bench strategies by holdout trade count. The share with a positive holdout CAGR was 36% below 20 trades, 35% between 20 and 40, and 94% between 80 and 120. Only two strategies fell between 40 and 80. Before the change, the floor was 5 trades, and holdouts with as few as 9 trades were being graded as failures.
Each trade floor in the KreamEdge process has one purpose, and they are never mixed:
| Floor | Value | Question it answers |
|---|---|---|
| In-sample window | max(400, 30 × effective parameters) | Can this data support this many parameters? |
| Holdout window | 80 trades | Is the out-of-sample result measurable at all? |
| Walk-forward fold used to calibrate trade cones | 30 closed trades per fold | Can this fold anchor an expected-range estimate? |
| Forward-test window | 120 bars or 30 trades | Is the window long enough to report? |
How to apply the rule to your own backtest
Apply the rule before you look at performance, not after. Write down how each kind of parameter is counted, count the effective parameters, compute the floor, and only then read the backtest. A floor chosen after seeing the result is a fitted parameter in its own right. The steps below are the order KreamEdge follows.
- Fix the weighting table before the first run: what counts as a full parameter, a half, a quarter or zero.
- Count effective parameters: free parameters, plus rule conditions, plus fixed filters, plus values frozen after tuning on the same data at half weight.
- Compute the floor as max(400, 30 × effective parameters) and count closed trades in the in-sample window only.
- If the strategy falls short, remove parameters or widen the universe with the same parameters applied to every instrument. Do not resize positions to manufacture trades.
- Log every configuration you try, including manual edits, and budget the trials against the years of data.
- Set the holdout floor in advance, and report a short holdout as insufficient, not as failed.
- Compare variants at matched exposure and roughly matched trade count. A gate that removes most trades can raise the Sharpe ratio without improving the signal.
Trades per parameter FAQ
How many trades per parameter does a backtest need?
KreamEdge requires at least 30 closed trades in the in-sample window per effective parameter, with an absolute minimum of 400 trades. The 30 is a practitioner heuristic, three times the 10-events-per-variable rule from regression studies, because trade returns are fat-tailed. No published trading study fixes an exact ratio, and practitioners on the Better System Trader podcast who use one did not give a number.
Does a fixed filter like a 200-day moving average count as a parameter?
Yes. A fixed filter restricts which trades happen, so it reduces the sample the remaining parameters are fitted on. KreamEdge counts it at full weight if it was tuned on overlapping data, at half weight if it was frozen after an earlier fit on that data, and at zero only if the data never chose it. Trading author Andrea Unger called such a filter “still a constraint”.
Is 30 trades enough to validate a trading strategy?
No. In the KreamEdge rule, 30 is the number of trades per effective parameter, not the total. A whole strategy needs at least 400 in-sample trades, and its holdout needs at least 80 to be measurable. Thirty closed trades is the minimum KreamEdge uses for a single walk-forward fold that calibrates an expected-range estimate, which is a much narrower job.
Can adding more symbols raise the trade count?
Yes, if the same parameters apply to every symbol, pooling across instruments adds trades without adding parameters. The caveat is independence. Trades on correlated instruments overlap in time and in risk; trader Adam Grimes noted on Better System Trader episode 188 that 20 stock positions behave more like one risk than 20. Count pooled trades, but do not treat them as fully independent.
What is the difference between trades per parameter and minimum backtest length?
Trades per parameter limits how many parameters one fit can support. Minimum Backtest Length, from Bailey, Borwein, López de Prado and Zhu (2014), limits how many configurations can be tried on a given number of years of data. With five years of daily data, the paper puts that limit at about 45 independent configurations. A sound backtest needs to respect both budgets.
Sources
- Better System Trader podcast, hosted by Andrew Swanscott: episodes 16 (Andrea Unger), 18 (Scott Andrews), 36 (Michael Bryant), 51 (Dave Walton), 64 (Bert Mouler), 79 (Dave Bergstrom), 152 (John Ehlers, July 2018), 178 (Stefan Friedrichowski), 180 (Rob Carver, February 2021), 188 (Adam Grimes), 196 (Jeff Swanson) and 206 (Adrian Reid, November 2021). Episode archive.
- David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance”, Notices of the American Mathematical Society, 2014. SSRN abstract.
- Peter Peduzzi, John Concato, Elizabeth Kemper, Theodore R. Holford and Alvan R. Feinstein, “A simulation study of the number of events per variable in logistic regression analysis”, Journal of Clinical Epidemiology 49(12), 1996. PubMed record.
- Eric Vittinghoff and Charles E. McCulloch, “Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression”, American Journal of Epidemiology, 2007.
On the sourcing: material in block quotes and quotation marks is transcribed from the episode transcripts, cleaned for punctuation and filler words but never for meaning, with an ellipsis marking any omission. Everything else attributed to a guest is a paraphrase of their argument in KreamEdge’s words. Guests are named as the episode titles name them. The KreamEdge figures come from the internal research bench and journal; they describe historical backtests on a research setup, and no figure here is audited.
KreamEdge has no affiliation with Better System Trader, any guest or any author cited above, and no citation is an endorsement.
Closing
Thirty trades per parameter is a blunt rule, and that is the point. It forces the count to happen before the equity curve is visible, it makes fixed filters and manual edits pay their way, and it refuses strategies that look good on too few outcomes, including our own. It does not replace a budget on trials or a measurable holdout. It is the first gate, not the only one.
If you want to discuss parameter budgets and backtest methodology with other builders, join the free channels on the community page.
We would not trade any strategy described here as-is: it is research only. The figures come from historical backtests, past performance is not indicative of future results, and nothing in this note is financial advice or a recommendation to buy or sell any instrument.
0 Comments