I don't know what your strategies and workflow look like but "tinkering" here sounds a lot like overfitting, doesn't it? If you're adjusting parameters or adding new mechanisms that increase the AR in the backtest, you can introduce all sorts of biases into it even with OOS testing.



You have to be confident that your strategy wins this black swan event not by chance though. I've also played around with the MATIC coin some time and the backtests were often defined by that burst of volatility in may. But if there are 2-3 trades happening that capture these insane price jumps, I just don't know how I can know if that is random luck or if the script can actually be on the right side of these reliably. And since I'm pessimistic, I rather exclude those times from the backtest personally. Though I also don't include any mechanism in the scripts so far that specifically react to these moments.