2 registered members (AndrewAMD, 7th_zorro),
1,285
guests, and 4
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
Bug? Backtest performance getting worse with smaller timeframes
#465643
05/07/17 18:50
05/07/17 18:50
|
Joined: Dec 2014
Posts: 206 Germany
Smon
OP
Member
|
OP
Member
Joined: Dec 2014
Posts: 206
Germany
|
This strategy uses only simple pending orders and shows a whopping 360% annual profit in H4, but an annual loss in M1. This is how I thought the backtest works with pending orders: bar crosses limit order -> simulated trade opened (add spread and slippage) bar hits stop -> book a loss bar hits TP -> book a profit bar hits both TP and SL within the same candle -> book a loss. Using a smaller Timeframe, some trades that the backtester booked as a loss before, because it wasn't sure if TP or SL was hit first should turn into winners and performance should get even better.
/*
Mean Reversion Strategy
ATR Band - looks similar to BollingerBands, without widening of the bands during spikes in Volatility
Entry: Pending order at the perimeter of the ATR Band
Exit: TP at the middle of the ATR Band
*/
function run()
{
//settings
set(PLOTNOW);
StartDate = 20100101;
EndDate = 20161231;
LookBack = 500;
BarPeriod = 240; //set this to 60, 15 or even 1 to see what I mean
NumWFOCycles = 8;
//PriceSeries
vars Price = series(price());
//Optimization Parameters
var ATR_multi = 1; //optimize(2,1,4,0.2); //optimization disabled
//static parameters
TimeFrame = 240 / BarPeriod; //keep Timeframe for the ATR Bands at H4 at all times
//ATR Bands
vars middle = series(LowPass(Price,10));
vars upperATR = series(LowPass(Price,10)+ATR_multi*ATR(100));
vars lowerATR = series(LowPass(Price,10)-ATR_multi*ATR(100));
//Trendfilter
// vars Trend = series(LowPass(Price,200)); //Trendfilter disabled
//TradeManagement
Stop = 2*ATR(25); //optimize(2, 1, 6, 1)*ATR(25);
TakeProfit = middle[0];
EntryTime = 1;
//TradeLogic
if (true) //(rising(Trend)) Trendfilter disabled
{
Entry = -lowerATR[0];
reverseLong(1);
}
if (true) //(falling(Trend))
{
Entry = -upperATR[0];
reverseShort(1);
}
PlotBars=300;
plot("upperATR",upperATR,MAIN,GREY);
plot("lowerATR",lowerATR,MAIN,GREY);
}
|
|
|
Re: Bug? Backtest performance getting worse with smaller timeframes
[Re: Smon]
#465644
05/07/17 19:16
05/07/17 19:16
|
Joined: Feb 2017
Posts: 369
Dalla
Senior Member
|
Senior Member
Joined: Feb 2017
Posts: 369
|
You cannot change BarPeriod and expect you system to behave the same way. I think what you are looking for is TICKS mode. Check here http://zorro-project.com/manual/en/mode.htmBasically just use set(TICKS); to enable
|
|
|
Re: Bug? Backtest performance getting worse with smaller timeframes
[Re: jcl]
#465691
05/09/17 20:32
05/09/17 20:32
|
Joined: Dec 2014
Posts: 206 Germany
Smon
OP
Member
|
OP
Member
Joined: Dec 2014
Posts: 206
Germany
|
Sorry, I don't get it.
I cite Kris from Robotwealth (I'm a member there): "If you look closely at the two 4-hour plots, you can see a number of trades that lasted for a single bar in the first plot, but which are absent altogether from the last one. Most of these seem to be winning trades, so their absence probably accounts for the difference. I have no explanation for why they are missing, but it does appear to be due to the granularity of the simulation. Definitely a question for the Zorro developers – this is pretty important!"
I looked into the log files and found some interesting differences. I set
StartDate = 20170101; EndDate = 20170404;
The first strange difference is the date of the first respective trade, which Kris already mentioned:
H4: 2017.01.16 08:00 H1: 2017.01.13 14:00 M1: 2017.01.17 05:36
Here is a trade that was taken in H4 and H1, but not in M1:
H4: 18.01.2017 12:00 exit at the same bar H1: 18.01.2017 13:00 exit at 15:00 M1: no trade
The trade looks legit on the chart, even if it's scary accurate. I dind't calculate my ATR Bands at this time yet, but I'm willing to do so if I have to to get this riddle solved.
The first trade that comes from the same signal is this short trade and fortunately, it's an interesting one with significant differences. It was very close to SL in between. I pencilled down the stops from the STEPWISE function, as I don't know how to let these write into the logfile yet:
H4: Entry 2017.01.17 04:00 @ 1.06499 Exit 2017.01.18 04:00 @ 1.06978 STOP 1.0730 H1: Entry 2017.01.17 05:00 @ 1.06489 Exit 2017.01.18 02:00 @ 1.06901 STOP 1.0729 M1: Entry 2017.01.17 05:36 @ 1.06466 Exit 2017.01.17 11:29 @ 1.07061 STOP 1.0724
The M1 trade was slightly different Entry with STEPWISE: Entry at 6:25 @ 1.0655
The highest HIGH between Entry and Exit of the trade was the 11:00 (UTC) BAR with 1.07196
M1 got stopped out even if it missed the SL with 4,4 PIPS (maybe due to different price data? I got the HIGH from tradingview.com) H1 missed the STOP by 5,0 PIPS and didn't get stopped out (due to the 0,6 PIP difference or due to the Timeframe???)
However, I re-read the manual and found the section about BarPeriod and TimeFrame pretty easy to understand and straightfoward. If I set BarPeriod to H1 (60) and Timeframe to 240 / BarPeriod = 4, then my ATR should be calculated just as if I set BarPeriod = 240, as 4*60 = 240. Wrong???
I understand that the backtester will produce very different results with market orders, as it doesn't know how the signal and price will change during an H4 bar. But with pending orders, Entry, SL and TP should have the exact same values from my understanding, because these values only change at 00:00, 04:00, 08:00 and so on... And even if the backtester can't tell the entry time down to the minute in H4 (of course not), it shouldn't have a hard time to come up with the exact same prices for entries and exits in such a liquid market (EURUSD).
I searched for the email of the helpdesk, but I couldn't find it.
|
|
|
Re: Bug? Backtest performance getting worse with smaller timeframes
[Re: jcl]
#465720
05/10/17 15:14
05/10/17 15:14
|
Joined: Dec 2014
Posts: 206 Germany
Smon
OP
Member
|
OP
Member
Joined: Dec 2014
Posts: 206
Germany
|
@pcz, thanks! I wouldn't have noticed the huge slippage. @jcl, yes, as far as I understand my own code and as I intended this system, the price is 1 ATR(100) in H4 away from both TP and SL when the trade gets stopped in. "Thus many bars will trigger Entry and TakeProfit at the same time." Of course, I'm all with you but again, as I learned from the robotwealth course, Zorro makes sure to not give overly optimistic results. It does so by counting a loss, if SL and TP are both hit within the same candle. Isn't that correct? If it's not, it should be implemented. And if SL isn't hit, well - why doesn't the backtester just calculates the Pips between the former pending order and the TP? "You can not test such a system in low resolution." I would aggree if we weren't talking about pending limit orders with static SL and TP. I think the backtester isn't well designed if it can be confused with such simple trades. The only thing that probably needs less ifs and elses to simulate are Binary Options. "From what I understand, the code is also strongly dependent on BarPeriod since the price series is derived from it." I don't understand the difference between Pending Limit orders calculated once per H4 bar or every 240 bars on M1....? From how TimeFrame is explained in the manual I conclude, that TimeFrame acts like I had set a different BarPeriod for all subsequent functions. For how the script is working it shouldn't make a difference if I set BarPeriod = 1 and Timeframe = 240 or BarPeriod = 240 and TimeFrame = 1 (or comment out TimeFrame alltogether). I'm still learning. This script was just an exercise, so TimeFrame was used for practicing reasons and I used these ATR Bands because I'm using them in my manual trading. I fixed them to H4 in my MT4, to always have the same price levels.
|
|
|
|