Can you please confirm if the figures calculated to produce the Profit factor or OptimalF are affected by the spread?

I'm thinking this is not the case.

In several cases I've seen a strategy that performs well, at least according to the PF and OptF. But after Testing, I cannot seem to keep it profitable. Of course setting Spread = 0 does make it profitable.

Thanks

Maybe another way I could ask that question is:

Is spread taken into account during Training? (at the time that PF and OptF are generated)

If not, is there a way to force the (real) spread during Training? I know I can hardcode the spread with spread=X but I want it to use the real spread from AssetsFix.dta for each asset.

The reason why is that I want to be able to prospect in the universe of assets to see which other assets could be profitable, after WFO, using the exact parameters of a formerly-optimized asset. For example, if EURUSD works well, I want to know if the identical settings can also work with USDCAD... but I need to take into account the current spread for each asset. I want to base success on either the OptF or PF readings, which are produced during the WFO cycle.

THANKS

I did some further testing and was able to answer this for myself... the answer is that YES, both OptF and ProF are affected by the spread during Training.

So then, the real question I'm digging at is:

What types of factors would cause an asset to "look great" on a .fac report (ie, great Profit factor, great OptimalF rating)... yet seem to fail miserably during the Test.

There must be some other factor I'm missing that could cause this situation.

OptimalF and lot volume is always 1 in training. So the training results will normally be different than the test results. But a bad result in the test means normally also a bad result in training. Maybe there are other differences between test and training in your script, aside from the trade volume? Do you get the same number of trades?

dusktrader,

A couple of other places to look:

1) overfitting to past data - some of the great work you did on using binary logic to optimise days / times etc. could lead to overfitting. Do you still have the divergence on simpler minimally optimized algos.
Relatedly - there recently was a thread on degrees of freedom which describes this phenomenon better. However, beware elimination of degrees of freedom by testing and rejecting many optimizable parameters for a single value - that process introduces all new data mining biases.


2) how long are you testing vs how long did the test period run?
Not sure what an algo would have to be trained and tested on to be fit for the last week and a half of market action! Come to think of it Dec wasn't great either :-)

I'm very proud I restricted my discretionary trading drawdown% to double digits only this past week ;-)

Thanks guys. I'm still working at this. I'm about ready to toss GBPCHF in the trash, and assume it might just be the pair. I have no problem doing that, it's just that I'm having a hard time reconciling why the WFO Test (1 Lot still) does not look better, given these WFO cycle results:

cycle 1) GBPCHF: 53.0 29 52.0 5.00=> 4.482
cycle 2) GBPCHF: 53.0 29.0 53.0 6.99=> 3.560
cycle 3) GBPCHF: 58.0 29.0 48.0 5.01=> 1.766
cycle 4) GBPCHF: 60.0 29 49.0 6.00=> 2.878
cycle 5) GBPCHF: 60.0 29.0 46.0 8.00=> 2.404
cycle 6) GBPCHF: 61.1 35.0 46.0 2.98=> 1.301
cycle 7) GBPCHF: 44.0 29.0 47.0 6.00=> 1.237
cycle 8) GBPCHF: 54.0 37 51.0 3.99=> 1.279
cycle 9) GBPCHF: 61.1 37.0 50.0 8.00=> 3.337
cycle 10) GBPCHF: 49.0 29.0 53.0 7.00=> 1.649
last cycle) GBPCHF: 50.0 34.0 49.0 8.00=> 1.107

Every cycle returns a nice PR from what I can tell, yet the Test itself shows a loss (and the equity chart looks bad as well).

I know the strategy has some merit, because the results on EURUSD look great (both Train and Test). I carried it to GBPCHF after some prospecting procedures that showed good WFO Train results and then tweaked the optimizations for GBPCHF (I did also test the raw EURUSD params, which were not as promising looking).

Regarding overfitting: yes I know its possible. I hope I haven't introduced too much, it is constantly on my mind. But even in the case of overfitting, I would think a good Train result should also have a (decent) Test result. Usually when I see a bad Train result, then I may not even run the Test because I know it too will be bad.

To answer your second question... the history period I've chosen more philosophically than scientifically, and I'm not sure if it's a good choice. I kept it starting at 2008 only because I felt that history prior to that was less relevant (ie different market personality) for a shorter-timeframe logic. So far I have not seen a reason to think this starting point is wrong.

@jcl I'll have to go back and check the # of trades between Train/Test... I haven't done that yet.

If you mean the number after the "=>", that's not the result used for the .fac generation. It's only the objective result from the parameter generation run.

This number is indeed normally far better than the WFO result, which is out of sample and resembles real trading.

I think I figured something important out here; a clue. After studying the .fac report some more, I realized that the reason I had chosen GBPCHF was because of it's excellent OptF component rating on GBPCHF:S (short).

In looking more closely at this report (which I've sorted descending based on best OptF to worst)... here are the other GBPCHF OptF results:
GBPCHF:S 0.808 (short)
GBPCHF 0.141 (combined long and short)
GBPCHF:L 0 (long)

It showed up on my radar as one of the top 7 assets while prospecting, which was primarily due to the short component (only). Typically when I see a good component, I assume it means both shorts/longs will also be good. But apparently not in this case.

If I simply disable long trades on this logic altogether, I get the reasonable result I'm expecting. It never occurred to me that I might want to do this LOL. In looking at the chart of GBPCHF, it makes sense to me because long term (since 2008) it has been in a steady downtrend. If I was trading discretionary, I might only take shorts on this pair.

It's easy enough to add a switch for assets to restrict them to only-long or only-short. I think I can even do it automatically, since I take an OptF measurement every step of the process (out of laziness to turn it off). I will give that a go and see if I can make more progress on this pair.

Hi Dusktrader!
If you disable long trades due to this fact, you are doing some sort of data snooping, because in real trading you would not know about this downtrend. (When does it start? When does it end? It could have ended right after switching on the EA)
Greetings roy

Maybe phantom trade both directions, and only enable real trades for a direction when its phantom equity is on the increase?