Iterating Without Overfitting

Why greyhound models are so easy to overfit

Your model should summarise what you believe matters in a greyhound race. Once you start changing it to fit what already happened, it stops summarising belief and starts memorising history. Greyhound racing is noisy: dogs are bumped, miss the break, fall or get crowded at the first bend. Those random events only cancel out over hundreds of races if your sample is honest.

The danger is that there are a thousand knobs to turn: track, grade, weight range, trap draw, recent form over two runs or four, days since the last race, sectional times. Each one offers a small chance to push a backtest figure upwards, and enough nudges will make almost any historical dataset tell the story you want. Overfitting lets the data write the story; underfitting ignores it. The skill lies in the uncomfortable middle.

Out-of-sample testing with a holdout period

The most useful idea in the chapter is simple: never test a model on the data you used to design it. Split your history into three slices. The training set is where you design features, choose weights and form the model. The holdout set is a block of races straight after it, used to test the model once only. The live set is the races you bet on with real money after the model is frozen.

The holdout is sacred. If you peek, tweak and peek again, you have used it up and the test becomes a tidy piece of fiction. A common pattern is to hold back the most recent six or twelve months, build on everything earlier, then roll the window forward as the live record grows. The exact window matters less than always keeping a gap of untouched data between deciding and judging.

Too many variables and hidden data leaks

Each variable is a small bet that it will keep working tomorrow, and early on many are guesses dressed up as features. A useful check is races per variable in your training set. With around fifty races for every coefficient you estimate, you are probably fine. With eight per coefficient, you are mostly fitting noise. A model with fewer variables you understand deeply usually beats one full of variables you half-understand.

Watch for leaks too: any feature using information you would not have had when placing the bet. Finishing positions, post-race sectional times and weights from the result are obvious examples. Subtler ones include a dog's end-of-season grade used for earlier races, or a race dated after the one being predicted. Describe each variable in a plain sentence; if you need the word after, it is probably leaking. Robust features also survive removal tests.

Treating every model change as a written experiment

The chapter asks you to become your own sceptical colleague. Before adding, removing or reweighting anything, write the change in one sentence including the data field it uses, state what you expect to improve and roughly by how much, pick a minimum holdout sample, and decide in advance what counts as success. Finally, agree not to rerun the test if it fails. Without these answers you are fiddling, not experimenting.

Keeping only the changes that work and quietly reverting the rest is a search, not an experiment, and it behaves like a slot machine in practice. The worked example tests days since a dog's last race on a holdout of one thousand races, with a rule set beforehand about the improvement needed. Run it, look once, decide, move on. Record every change in a simple model log with date, reason, sample size and result.

Knowing when to leave a greyhound model alone

The hardest lesson is that some weeks, and some months, the right answer is to change nothing. Poor reasons to touch a working system include a quiet week, a forum thread about someone's favourite factor, boredom or simply wanting a feeling of progress. None of them justify touching a model that is working.

Good reasons look different: a persistent, implausible gap between holdout and live results, a real documented change in the racing such as a new track surface, grading policy or calendar shift, or a specific testable hypothesis with a sample size in mind. Setting a minimum interval between changes, such as reviewing every two or four weeks, makes a model harder to overfit and lets each change breathe, since a feature that disappoints in week three may just be unlucky.

A simple iteration cycle for steady improvement

The chapter offers a six-step cycle. Freeze the current model for the next betting period. Observe live results without changing anything, noting surprises. Hypothesise by writing down at most one or two possible changes and why. Test each on the holdout with the pre-agreed sample size and rule. Decide to adopt, reject or park the change based on the result. Record the decision, reason and sample size in your model log.

Big changes are almost always a mistake because they swap a system you understand for one you do not, often at exactly the moment overfitting confidence is highest. Small changes, tested honestly and accumulated over months, are how most quantitative models improve. The aim is not a perfect model but a process that improves a good one slowly without fooling yourself.