Before you trust your system.
Try to break it.
Independent stress-testing for racing systems, models and methodologies.
You have the system and the results. We test whether they survive scrutiny: against the market, against new data, against variance, and against the assumptions the backtest depends on.
Confidential analysisNDA availableIndependent assessment
A profitable backtest is the beginning of the investigation. Not the end.
A profitable historical result can be created by a small sample, selection bias, overfitting, market movement, changing conditions or a handful of exceptional results. Any of them can make a system look stronger than it is.
We do not begin by trying to prove your system works. We begin by trying to find out why it might not.
Ten questions a system has to answer.
ROI and return structure
Is the return broad and repeatable, or dependent on a small number of exceptional results?
Market benchmarking
Did the system find information the market missed, or simply identify horses the market had already priced correctly?
Out-of-sample testing
Does the rule still work on races it was never built on?
Overfitting and rule fragility
Do small changes to thresholds destroy the result?
Segment analysis
Where does it hold, where does it weaken, and does it depend on narrow conditions?
Drawdown and variance
How deep and how long are the losing runs, and would the return have survived them?
Time stability
Does the result hold across seasons and years, or rest on a period that has already passed?
Sensitivity testing
Which assumptions can the result not do without?
Data quality
Is anything in the data known only after the race, missing, duplicated or mistimed?
Implementation reality
Was the price the backtest assumed a price anyone could have taken?
Scope depends on the system and the data supplied. Not every audit includes every test.
Bring us a system you believe works. Our job is not to agree with you. Our job is to interrogate the evidence.
Proof of process. Not a claim of edge.
An independent written assessment.
Every engagement is scoped individually. An audit covers six areas:
- Performance reconstruction
- ROI, strike rate, return distribution, losing sequences and drawdown.
- Market benchmarking
- Performance against Betfair Starting Price and the relevant prices actually available.
- Validation
- Out-of-sample testing, time stability and sensitivity.
- Segment analysis
- Where the system works, weakens or depends on narrow conditions.
- Data and implementation
- Leakage, missing data, duplicates, timestamp quality, liquidity and realistic price availability.
- Written findings
- What holds, what does not, what is unresolved and what should be tested next.
Four kinds of statement, kept apart
- Observation
- What the data shows.
- Evidence
- What survives a test that could have gone the other way.
- Hypothesis
- What may explain the result but is not yet tested.
- Recommendation
- What we would do next.
We keep these categories separate so a hunch is never mistaken for a result.
Sometimes the most valuable conclusion is: do not add another rule.
Independent means independent.
No promised conclusionsNo manufactured improvementsNo claim of an edge without evidenceNo historical performance presented as proof of future returns
Five stages. One question throughout.
- TalkA confidential initial conversation.
- ScopeAgree what is being tested and what data exists.
- ProtectAn NDA and secure data transfer where required.
- Stress testIndependent reconstruction and analysis.
- FindingsWritten conclusions, weaknesses and the next investigations.
An optional follow-up consultation is available once you have read the findings.
Your system remains your system.
- NDA
- Available
- Data transfer
- Secure route, agreed with you
- Publication
- Never without your permission
- Your material
- Used only for the agreed analysis
Proprietary rules, code and datasets are not sent with an enquiry. A secure transfer route is agreed after the initial conversation.
Three ways to engage.
Terms are agreed before any work begins.
- Initial consultationA private conversation about how the system was built, what data exists and what you want independently tested. It decides whether a full audit would be useful, with no obligation to proceed.Scoped and quoted individually.
- System auditIndependent reconstruction and stress-testing of one system against the supplied data, with written findings and the next investigations.Scoped and quoted individually.
- Advanced and ongoing researchLarge datasets, multi-rule models, bespoke reconstruction and continuing forward testing as a system is developed and run.Scoped and quoted individually.
Research and analytical consultancy: Elite Pass provides research and analytical consultancy. Historical results do not guarantee future performance. Nothing in an audit is financial advice, a betting recommendation or a guarantee of profitability, and decisions made using the analysis remain the client’s own.
People with a system, and a question about it.
- Independent racing-system developers
- Quantitative racing researchers
- Professional and semi-professional bettors
- Racing-data businesses and model developers
- Syndicates and owners of proprietary systems
If you have a system and a genuine question about whether its edge is real, this service is for you.
Think your system has an edge?
Let’s find out what happens when somebody tries to break it.
All enquiries are confidential.
Received. Your enquiry is treated as confidential and we will reply by email.
We could not accept that enquiry. Please check the name and email address and try again, or write to the address in the site footer.
Submit a confidential enquiry
A short description is all we need to scope a conversation. Do not send proprietary rules, code or datasets here. A secure transfer route is agreed after the initial conversation.
Read next
- Do horse racing systems actually work? What happens to a racing system when it is tested against the price that was actually available.
- How should a horse racing system actually be tested? Twelve steps for testing a racing idea properly, and why the answer is allowed to be no.
- Evidence and methodology How every number on this site is defined, sourced and checked.