Backtest accuracy

Posted By: tomna1993

Backtest accuracy - 05/07/21 17:29

Hi all,

I'm trying to backtest one signal just to get an information about accuracy of backtesting. For that I use 1 year of MNQ data, the bars in dataset have 1 second bar period (the dataset is in t6 format).

I ran two tests. In the first one I entered the trades in the run function, in the second I used tick function to enter the trades. As I understand from manual, when I have 1sec data and I use 1 minute bar period there will be 60 ticks, so Zorro will step into tick function 60 times during the completion of the run function. (I hope I didn't misunderstood that, please correct me if yes eek)

The test is about place a trade with 10000USD, set up the Stop to -1R and the TP to +1R and check in the log how much of the trades are wrong. Wrong trades means they are out from the +9900 -> +10100 or -9900 -> -10100 zones.

1.4% of the trades are wrong In tests where I enter the trades in run function.
In case of using the tick function I get 13% wrong trades.

Am I missing something? I think this should be otherwise, when I use just the 1 minute bars (when enter trades in run function) I should get worst accuracy then with the tick function.

Does someone have statistics about the accuracy, and can give me a hint how can I reach the highest accuracy with tick mode?
Thank you!

Below you can see my code:

Code
#include <profile.c>

vars Open, High, Low, Close, BarHeight; 

var SignalEntry, SignalStop, SignalTakeprofit, Contract;

bool signal;

static var Time = 0;

function tick()
{
	if(signal && !(TradeIsOpen))
	{
		enterLong((int)Contract,SignalEntry,SignalStop,SignalTakeprofit);
		signal = false;
	}
}

function run() 
{
	set(PLOTNOW); 
		
	if(is(INITRUN))
	{
		set(LOGFILE);
		set(TICKS); // comment out when enter trade in run function
		Time = timer();
		
		StartDate = 20200101;
		EndDate = 20201231;
		
		StartMarket = 1330;
		EndMarket = 2000;
		setf(BarMode, BR_LEISURE);
		
		signal = false;
		
		PlotScale = 8;
		
		LookBack = 100;
		
		ColorUp = 0x00BC0A;
		ColorDn = 0xDE0505;
		
		Slippage = 0;
		Spread = 0;
		PIP = 25./1;
		PIPCost = 50./1;
		
		BarPeriod = 1; //use 1min bars to test, get around 7000 signals in a year
		
		asset("MNQ_2020_1sec");
	}
	
	Open = series(priceOpen());
	High = series(priceHigh());
	Low = series(priceLow());
	Close = series(priceClose());
	
	if (LongSignal)
	{	
		plot("Bull k", (Low[0] - ATR10[0]*0.5), DOT|MAIN, BLUE);
		SignalEntry = High[0]; // enter the trade on level of signal High
		SignalStop = Low[0]; // stop on Low of the signal bar, ST is -1R
		SignalTakeprofit = High[0] + BarHeight[0]; // add the signal bar height to the signal bar, TP is +1R
		Contract = 10000/((SignalEntry - SignalStop)*2); // calculate enough contracts to get the best accuracy 
		signal = true; // used when backtest with tick function
		//enterLong((int)Contract,SignalEntry,SignalStop,SignalTakeprofit); // comment out when enter trade in tick function
	}

	if (is(EXITRUN)) 
	{
		printf("\nTime needed: >>> %.3f sec <<<",(timer()-Time)/1000); // print time needed to complete the test
	}
}

Posted By: tomna1993

Re: Backtest accuracy - 05/07/21 22:14

I tried to test the accuracy with tick data. No luck yet. I think a mode flag or some parameters should be wrong. I don't understand why 10% of the trades are wrong even with tick data. There are 1 second bars with 200-300 price change and I run the backtest on 15sec bar period.
Posted By: jcl

Re: Backtest accuracy - 05/10/21 10:04

If you mean with "wrong" that your trade is not filled at your desired stop price: Welcome to the world of trading. Read on https://manual.zorro-project.com/stop.htm how stop exits are filled in backtests.
Posted By: tomna1993

Re: Backtest accuracy - 05/11/21 22:02

It's clear for me why I had "wrong" trades. It's because of the data. When there is no price on my entry level, then the trade should open on the next tick, of course this will make the results a little bit inaccurate. This is completely clear for me. But what I still don't understand is how can be a test with OHLC prices (use just the run function) more accurate than a test with 15 ticks (with tick function). So when I use just the run function and the trigger bar OHLC prices are far away from the entry level, why Zorro opens the trade on the entry level and not on the close of the bar? (see example)


Attached picture example.png
Attached picture example_tickFunction.png
Posted By: AndrewAMD

Re: Backtest accuracy - 05/12/21 00:26

Try setting Fill to different values and see which is most appropriate:
https://zorro-project.com/manual/en/fill.htm

When in doubt, consider a conservative assumption.
Posted By: jcl

Re: Backtest accuracy - 05/12/21 11:46

To answer the question: your tick version is a lot more accurate than the bar version. If you don't use ticks, Zorro makes assumptions of the curve inside the bar, and that's why you got different entries.
Posted By: tomna1993

Re: Backtest accuracy - 05/12/21 13:29

I'll try to test with tick data, it will be hard because even the generation of t1 file is hard and Zorro can't backtest one year of MNQ tick data because it's so big. But I will try to separate the one year data to smaller pieces and hopefully it will work.
Thanks to both of you for the informations!
© 2024 lite-C Forums