6 registered members (rki, Martin_HH, AndrewAMD, squik, Ayumi, alibaba),
1,150
guests, and 5
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
Re: Is it possible to have "too much history" for backtesting?
[Re: swingtraderkk]
#429532
09/13/13 18:05
09/13/13 18:05
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
Well I suppose "reasonable period" is subjective, unfortunately. I hate that myself, because I don't like introducing subjectivity. However, my theory is that for COMPARISON purposes, you must ensure consistency.
For example, I could look at a Performance Report and see that a bot is "good" or maybe "excellent". Those would be subjective evaluations. However, my theory is that I could not look at two bots and know -- in an objective way -- if one was better than the other... at least without having knowledge that they were developed with the same amount of history. One way I think you could answer that question would be knowing the relationship of the BarPeriod with the total amount of data.
Not knowing that relationship means you would also affect the statistical accuracy of the logic, I believe (note: I confess I am NOT a statistics or math expert!). The reason why is because of this example: 240minute barperiod strategy across 10 years data is not comparable to 15minute barperiod strategy across 10 years
There would be 16x more opportunities for trades in the 15minute strategy across that dataset. Therefore, it is not a fair comparison with the 240minute strategy across the SAME dataset.
Now don't get me wrong... I'm not advocating necessarily that it's bad to get rid or avoid testing on max history available. I'm only suggesting that perhaps it is not best for COMPARISON purposes.
If you believe that max history is the best philosophy, then one way you could accomplish that is like this: 1) determine the History ratio of your longest BarPeriod strategy across the max history available 2) Adjust testing history on all other BarPeriods to match
This provides a more similar way to COMPARE different bots without introducing the unfair bias of additional trade opportunity history. You could also "maybe" argue that more history is less relevant and therefore potentially "bad".
Separately, in my design process, I'm comparing first with the limited/equalized dataset but also testing with max history. The max history test is just for my information, but may not be appropriate for comparison purposes.
---- Btw, to answer your question -- I think I mean "reasonable period for optimization purposes" ... because in the stage where I would be limiting history, it is mostly for the purpose of finding the best trading logic.
|
|
|
Re: Is it possible to have "too much history" for backtesting?
[Re: swingtraderkk]
#429744
09/17/13 19:54
09/17/13 19:54
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
First, I should preface that "I don't know" if this is a good idea. I'm just throwing it out there, and testing it myself. It might be a terrible idea, or maybe it has some value? I'm not sure yet.
You make a good argument about the overall market's (permanent) changes. I agree with that, that bots should be robust enough to hopefully go with that flow.
My intent was to find some sort of common ground to compare strategies of different barperiods. On this level, I'm not looking at them from a timeframe perspective as much as just barperiods.
It seems like a fractal problem. Each new bar presented is another opportunity to test the trading logic. On some level, it doesn't matter how many ticks it took to build that bar.
I agree that judging a strategy's worth should not be done necessarily by restricting possible testing history. But for comparing strategy A with strategy B, of different barperiod logics, perhaps there is some value to equalizing them on some level. But it seems like you'd have to keep it within bounds... for example, reduce data but ensure enough trades in both strategies.
Again, I'm not sure if it's a good idea. For example, I just jumped from testing a 287-barperiod strategy to now a 15-barperiod strategy. It seemed pretty much useless to equalize the history ratio in that example.
|
|
|
|