Hi, I'm just struggling how to properly test my strategies i play with:)
At the moment i have a question about the capital allocation factors.

I understand that if the FACTORS flag is set the OptimalF factors are generated from the whole training+test period - not only from the training period.

a) is this done dynamically step by step while the test period is running on the test period? Or is the whole training+test period used to allocate optimalF factors BEFORE running the actual WalkForward Test?

b) is there a FLAG or method i could use to only train the OptimalF Factors from the training period only?

Thanks:)

It is done by running a complete walk-forward test and then calculating the factors from this test.

Taking factors only from a certain period must be done by script. For this, first generate the parameters, then set only the FACTORS flag, and exclude trades from outside the desired period with an if(..) condition, like this:

if(year() < 2011 or month() < 6) enterLong();

or

if(year() >= 2011 and month() >= 6) quit();

for only calculating factors until June 2011. Setting StartDate and EndDate won't work here because it would also affect the WFO cycles.

Ok thanks! I will use that. Your help is very appreciated!

Wouldn't it be more realistic to have the Factors be created "on the run" as if you were retraining every week in live trading for example? Instead of getting them "from the future":)

I can't think of a practical way of implementing that in a script though

OptimalF factors need a relatively large trade history. So they should not be calculated on the run for the past weeks - those factors could be way off and misleading.

Ok sorry to be nagging, but i didn't mean to calculate them from a few weeks only, but from the beginning of the training period right into the test period to the point where zorro "walks forward" at the moment. Simply not taking into account the rest of the test period that lies "in the future" from the single trade perspective.

Could that be implemented somehow? I get huge discrepancies in backtesting with vs without factors.
I am just testing your tip from above though.. sounds pretty reasonable.

I just don't feel good backtesting if you take into account information that would practically lie in the future:)

Please correct me if I'm wrong here in my understanding... but FACTORS are mostly used for trade weighting purposes... ie, to know "how much" of a component your strategy should buy or sell.

Since I believe this is the case, what I normally do is work with flat-lots only (ie, minimum Lots=1) to develop the strategy first. You want to make sure the trading logic is correct/profitable before applying any OptimalF "enhancements"

Once the trading logic is in place, then you can use OptimalF to weight your trade basket accordingly, so that the most profitable trades historically get precisely the optimal allocation factor of your funds.

To me it is a mistake and very misleading to allow Lots > 1 while developing the strategy.

Again, please correct my line of thinking if this is not correct. I'm still a n00b too. But I personally feel this philosophy has served me well in my discretionary trading.

This is absolutely correct. Factors and Margin should not be used for developing, only for the final trading. If you have a losing portfolio system, but convert it to winning with OptimalF factors, the profits come likely not from the system but from selection bias.

Aside from that, you can generate factors at runtime in a retraining process. But unlike parameters, factors normally need not be updated often. It's normally sufficient to do that once a year.

OK, my strategy was developed like that - without the Factors flag, exactly for the reasons you wrote... but now everything gets more complicated...
I first added a few Algos, then a lot of additional assets. Somewhere on the way i noticed the strategy wasn't profitable anymore if i excluded the FActors flag. But with it, it is hugely profitable.
So im actually now trying to discern what is reasonable and what isn't. Perhaps i should go a few steps back.

I'm still playing around with different methods of Factor allocation for only a limited timeperiod. I'll report back when im done.

My opinion is that you should not try to define "reasonable" or put it in a box. Because I'd say 100% of your definition of that comes from what the media reports, and that is definitely wrong (or at least limiting).

Instead, keep an open mind, a clean slate. My personal intention is to just see what shakes out and learn along the way. Perhaps you will find that your definition of "reasonable" is some combination of high profit factor and low drawdown. You'll probably get a sense for it over time.

One use of OptimalF that I am exploring, that I think should be "ok", is for the purpose ONLY of knowing which pairs/algos/directions to trade. In other words, let your backtest run all variations of pairs/algos/directions and then based on OptimalF, trade only those proven profitable... but do not apply the OptF factor to the trade weight.

In other words... test every combination, but only keep the ones that show any profitability at all. Still test only with minimum flat lots.

Depending on your strategy, my guess is that some assets will show "excellent" performance while others will seem only "mediocre". However, OptimalF does know best... so once you find the cream-of-crop, you can then apply OptimalF to maximize your bang-for-buck.

Im back from holidays:p
Ok, i rechecked all my Algos(I'm having fun with them), and what i did is this:

I tested them without FACTORS
I tested them with fixed FACTORS(eg. everything on 0.1 - no idea if that helps, but i managed to test Z12 like that)
I tested them with FACTORS which where only generated in the Training period.
What i learned is you should really proceed step by step - if a script isn't profitable without FACTORS, it probably will never be - thus i ditched a few:(

Also i thought a lot about SelectionBias. I can't write a script without it - i mean you change a thing and hit Test- if it doesn't work out, you change something until you get a straight equity curve. implementing the PARAMETERS is great help for that! I first didn't grasp it - but i think a script is less biased WITH PARAMETERS than without.

Last thing to overcome SelectionBias: Im running my backtests on a MacBookPro with WinOnX. I can't download any PriceData there(but btw its much faster than on Windows8 on the same MBP), so my SelectionBias only runs until early 2013. Thus i can see if it still performs after that later.