dusktrader,

I'm still confused as to what the objective of the "reasonable period" is?

reasonable period to backtest for positive expectancy, or reasonable period for optimisation purposes?

Well I suppose "reasonable period" is subjective, unfortunately. I hate that myself, because I don't like introducing subjectivity. However, my theory is that for COMPARISON purposes, you must ensure consistency.

For example,
I could look at a Performance Report and see that a bot is "good" or maybe "excellent". Those would be subjective evaluations. However, my theory is that I could not look at two bots and know -- in an objective way -- if one was better than the other... at least without having knowledge that they were developed with the same amount of history. One way I think you could answer that question would be knowing the relationship of the BarPeriod with the total amount of data.

Not knowing that relationship means you would also affect the statistical accuracy of the logic, I believe (note: I confess I am NOT a statistics or math expert!). The reason why is because of this example:
240minute barperiod strategy across 10 years data
is not comparable to 15minute barperiod strategy across 10 years

There would be 16x more opportunities for trades in the 15minute strategy across that dataset. Therefore, it is not a fair comparison with the 240minute strategy across the SAME dataset.

Now don't get me wrong... I'm not advocating necessarily that it's bad to get rid or avoid testing on max history available. I'm only suggesting that perhaps it is not best for COMPARISON purposes.

If you believe that max history is the best philosophy, then one way you could accomplish that is like this:
1) determine the History ratio of your longest BarPeriod strategy across the max history available
2) Adjust testing history on all other BarPeriods to match

This provides a more similar way to COMPARE different bots without introducing the unfair bias of additional trade opportunity history. You could also "maybe" argue that more history is less relevant and therefore potentially "bad".

Separately, in my design process, I'm comparing first with the limited/equalized dataset but also testing with max history. The max history test is just for my information, but may not be appropriate for comparison purposes.

----
Btw, to answer your question -- I think I mean "reasonable period for optimization purposes" ... because in the stage where I would be limiting history, it is mostly for the purpose of finding the best trading logic.

Seems that Daniel decided to contribute to this discussion with the latest post on his blog

http://mechanicalforex.com/2013/09/long-...fitability.html

Edit:

And here's one of the older articles that dealt with the topic:

http://mechanicalforex.com/2010/09/why-are-five-years-statistically.html

Unfortunately, it's missing important graphs.

dusktrader,

I can agree with almost everything you are saying - and I have my own issues re ranking bots. Where you lose me is when you abstract the bars from the actual time period you are looking at.

1000 240 min bars = 16000 15 mins bars = the exact same history = The exact same time period you wish to have a return in so you can live = the exact same time period that the market changes in.

The types of market changes that make strategies/algos less profitable occur over the exact same periods of years whether you measure them in 240 min bars or 15 min bars.

Implicit in your logic is an assumption that the permanent and cyclical changes you wish to include (or avoid) in your backtesting happen 16 times faster on a 15 min chart compared to a 240 min chart. This does not make sense to me.

I can understand your arguments for less history in general, (I'm not convinced though ;-)), but if you decide that x years is the period you will test for your 15 mins algo, then I think the only valid comparison for the 240 min algo is the same x years.

First, I should preface that "I don't know" if this is a good idea. I'm just throwing it out there, and testing it myself. It might be a terrible idea, or maybe it has some value? I'm not sure yet.

You make a good argument about the overall market's (permanent) changes. I agree with that, that bots should be robust enough to hopefully go with that flow.

My intent was to find some sort of common ground to compare strategies of different barperiods. On this level, I'm not looking at them from a timeframe perspective as much as just barperiods.

It seems like a fractal problem. Each new bar presented is another opportunity to test the trading logic. On some level, it doesn't matter how many ticks it took to build that bar.

I agree that judging a strategy's worth should not be done necessarily by restricting possible testing history. But for comparing strategy A with strategy B, of different barperiod logics, perhaps there is some value to equalizing them on some level. But it seems like you'd have to keep it within bounds... for example, reduce data but ensure enough trades in both strategies.

Again, I'm not sure if it's a good idea. For example, I just jumped from testing a 287-barperiod strategy to now a 15-barperiod strategy. It seemed pretty much useless to equalize the history ratio in that example.