Yep, I'm with Grant and don't really understand what your approach is here.

Are you looking for feedback by posting here, or are you just using the forum as a place to dump your notes? I think most of the rest of us just use a document on our local machines for some of the observations you are making.

But each to their own. To clarify - are you live with any of these strategies or do you have a method you are following that is taking you closer to going live with these? It sometimes seems like you are generating metrics just for the sake of generating them, however big numbers are meaningless (at least to most readers here I would suspect) if they are not robust.

Have a read of this (if you haven't done so already): https://financial-hacker.com/build-better-strategies-part-3-the-development-process/ and see if this can help focus the mind.

I welcome feedback, but mostly second

I don't generate them. those are actual backtest results. all of them

meaningless?

monthly income of 34196129$ doesn't spark your interest?

live since October

define robustness!

mine is simple - it's robust if you became filthy rich.
after the fact only

have read all of that blog

sure looks bad. but it's useful

I'm sure JCL or one of the many other experts here could better define robustness, but I suppose I generally look to see how similar my backtest performance was when compared to forward testing or live trading. This will show how sensitive the indicative results during the backtest are to parameter settings / period of test / market conditions / asset etc...

Are you seeing annualised returns of 1000%+ during live trading in line with some of the recent results you are posting here? Does your win rate / drawdown etc. align with your expectations from the development process?

and what do you think I do?



are you from the past?

it's the worst crash since March 2020



either way equity curve goes - high linearity is the least likeable attribute of it

yes, they do align. but only because my expectations are sane

I will answer that by explaining what I did (without implying to be mr know it all!).

I've picked February & March 2020 as my in-sample period (1M data). Why? Because February had a relative low volatility, but March was sky high. So this short period contained at lot of valuable information. Then I ran a ton of backtests from February 2020 till May 2021, just to see how my models (I use ML, hence 'models') behave in- and out-of-sample. By doing so, you see exactly how easy it is to over-fit a model.

In- and out-of-sample


Out-of-sample

@Grant

I'm missing your point

it feels like you are stating obvious

---------

also - there is a bit of difference when working with machine learning

the idea being - when you over fit ML model,
you can't go back and cherry pick out the good parts out of trained model cause it's gibberish

at the very least - not as easy

---------

about mr know it all

well... excuse my tone - I deliberately try to be dense, direct and open

I think that's just the fastest way to knowledge

you have made me think about things that would otherwise fly by unnoticed, so thanks

I appreciate that 'tone' sir, so no problem! I find the different approaches on this forum and financial-hacker refreshing to read.

Yes, there's nothing magical about my approach in general, but why would you not out-of-sample test your strategy first on recent historical data, and once it behaves stable, test it on a demo account? By skipping the first part, the uncertainty is even higher and so will be your failure rate, so you're wasting more time in your development process.

As for ML, I'm well aware of the bias-variance tradeoff. I apply techniques like cross-validation and regularisation to control this issue as much as possible, but granted, it's tough, esp with financial market data.

most problematic period of time

conflicting rules identified

curve looks reasonably nice if we throw them out