1 registered members (flink),
767
guests, and 5
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
Are all backtest results potentially flawed?
#463193
11/21/16 12:22
11/21/16 12:22
|
Joined: May 2016
Posts: 180 Prague
pcz
OP
Member
|
OP
Member
Joined: May 2016
Posts: 180
Prague
|
I haven't received an answer for reported problems with slippage in another thread yet so I'll try to reformulate the issue and also provide a code to reproduce the whole thing. It seems that when you set slippage greater than zero, the results might be optimistically biased. The effects can be quite profound even with values around 5, which is the default in new Zorro. It can effectively ruin your account. You can try for yourself with this simple mean reversion strategy:
//Asset settings in AssetFix.csv:
// EUR/USD,1.12000,0.00005,-0.418,0.0560,0.0001,10.00,2236,0,100000,0,
function run()
{
Asset = "EUR/USD";
StartDate = 20140101;
EndDate = 20141231;
if(IBS() < 0.2 && NumOpenLong == 0)
enterLong();
if(IBS() > 0.8 && NumOpenShort == 0)
enterShort();
}
In Zorro 1.51 it gives me AR 50% and SR 1.15 - not that bad, right? Now lets add line Slippage = 0; With zero slippage AR is 0% and SR 0.01. Isn't that weird? Lets try larger value, for example one minute (i.e. 60). Now the AR is 1293% and SR is 11.12. The number of trades and entry / exit times are the same as with slippage set to 0, yet the results are very different. This is for example the first trade for both extreme cases (i.e. Slippage = 0 and Slippage = 60 in the second line):
Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3617,1.3628,108.00,0.00,Reverse
Osc_spread,Long,EUR/USD,8201,1,2014-01-07 19:00,2014-01-08 02:00,1.3613,1.3630,163.99,0.00,Reverse
Exported price data for the asset (around open / exit time):
07/01/14 19:00, 1.36175, 1.36184, 1.36163, 1.36163
07/01/14 19:01, 1.36163, 1.36163, 1.36154, 1.36157
...
08/01/14 02:00, 1.36283, 1.36290, 1.36283, 1.36289
08/01/14 02:01, 1.36289, 1.36290, 1.36274, 1.36275
As you can see with slippage the entry and exit prices are way out of line. Sphin noted that in the new beta version there should be a fix for some slippage issues but I tried it and the results are the same so I assume it was a different problem than this one I described. Also - I asked for this months ago: it would be really useful to increase the precision of price representation in log files. With only four decimal places it's really hard to debug the fine details.
|
|
|
Re: Are all backtest results potentially flawed?
[Re: Finstratech]
#463203
11/21/16 15:07
11/21/16 15:07
|
Joined: May 2016
Posts: 180 Prague
pcz
OP
Member
|
OP
Member
Joined: May 2016
Posts: 180
Prague
|
I noticed that some systems behave very differently with and without the TICKS flag. Try setting it and the EXTRADATA flag. I haven't played with the slippage yet though. Will try. TICKS + EXTRADATA (especially the second flag) change the results somewhat but there's still a huge difference even between 0 and 5 sec slippage backtest. You can try it for yourself with the code provided above.
|
|
|
Re: Are all backtest results potentially flawed?
[Re: Sphin]
#463206
11/21/16 16:18
11/21/16 16:18
|
Joined: May 2016
Posts: 180 Prague
pcz
OP
Member
|
OP
Member
Joined: May 2016
Posts: 180
Prague
|
I noted that What's New page promotes a slippage bug to be solved in 1.52.0 but this version seems to be not available yet because the actual beta on the download page is 1.51.9. Did you already get 1.52.0 there? Sorry, I misunderstood your last post. I thought that it's fixed in 1.51.9. So we'll see if it is fixed in the next version. Nevertheless it's worth noting that all previous backtests should be done again as Slippage > 0 has been default.
Last edited by pcz; 11/21/16 16:21.
|
|
|
Re: Are all backtest results potentially flawed?
[Re: pcz]
#463208
11/21/16 19:03
11/21/16 19:03
|
Joined: Jun 2016
Posts: 9 Germany, BW
MattY
Newbie
|
Newbie
Joined: Jun 2016
Posts: 9
Germany, BW
|
Hey, test with this, so you are always on the pessimistic and more real side: Setting Slippage to a negative amount simulates asymmetric slippage that is always in adverse direction of the trade. Asymmetric slippage is illegal, but some trading platforms allow the broker to automatically apply asymmetric slippage for reducing the trader's profit. I trade with FXCM and the slippage seems to be much greater on the negative side (hope they don't do this on purpose....).
|
|
|
Re: Are all backtest results potentially flawed?
[Re: DavidC911]
#463216
11/22/16 08:40
11/22/16 08:40
|
Joined: May 2016
Posts: 180 Prague
pcz
OP
Member
|
OP
Member
Joined: May 2016
Posts: 180
Prague
|
Does it mean that all Z system's results are rather optimistic?
If so, how can we deal with that issue since we can't modify the Slippage or Fill parameter of the Z system?(or can we?) Not necessarily. I've tried the same for my other strategy and greater values of slippage had actually decreased its performance. But I have no clue which strategies might be affected.
|
|
|
Re: Are all backtest results potentially flawed?
[Re: pcz]
#463218
11/22/16 13:01
11/22/16 13:01
|
Joined: Nov 2016
Posts: 4 Taipei
DavidC911
Guest
|
Guest
Joined: Nov 2016
Posts: 4
Taipei
|
Not necessarily. I've tried the same for my other strategy and greater values of slippage had actually decreased its performance. But I have no clue which strategies might be affected. Thanks for your info, pcz. I'll keep this potential flaw in mind.
|
|
|
|