Currently working on the following across several seas:
Step 1: Define the counterfactual properly. The “unmodeled guess” would be what policymakers would assume is absent in the model, and would be some combination of:
- Persistence: effort and catches stay roughly where they were
- Naïve closure arithmetic: a closure “protects” everything inside its boundary and costs the fleet proportionally to its footprint
- Proportional redistribution: displaced effort spreads uniformly over the remaining grounds
- Expert judgment: what experiential knowledge would say without simulation
The “naive” baseline assumes fishers don’t strategically adapt. That is DISPLACE’s whole point: fleet adaptation most likely occurs in response to area-based management by displacing effort and continuing to fish elsewhere. The comparison shows whether it is real.
Step 2: Use natural experiments where the truth is observable. Real closures happened after the model existed (Baltic cases, Kattegat, Mediterranean fishery-restricted areas). For those, we would retrieve before/after VMS tracks, logbooks, landings, and fuel data. We would then feed the model only data up to the year before the closure, let it predict the post-closure period, and score it. The unmodeled baseline would make the same prediction for the same period.
Step 3: Score on metrics where behaviour matters most.
- Spatial allocation: what fraction of post-closure effort landed in the cells the model assigned it to, versus where persistence/proportional baselines put it? (use some metrics for that…)
- Displacement distance and direction: did effort shift to the actual remaining grounds, at roughly the modelled magnitude?
- Economics: revenue and fuel use per fishing fleet segment: compare mean absolute error for each predictor.
- Probabilistic calibration: run stochastic simulation to capture uncertainties and compare with frequency-type statements… did the modelled frequencies match realised frequencies?
DISPLACE doesn’t need to be right; it needs to be less wrong than the guess. If a retrospective showed, say, that the model beat persistence by 30% on spatial allocation while the naive baseline missed displaced hotspots entirely, that would be a good case to demonstrate that taxpayer money is not misused.