Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
1 registered members (AndrewAMD), 945 guests, and 8 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Mixed frequency history (M1+D) #477126
05/20/19 09:53
05/20/19 09:53
Joined: Apr 2019
Posts: 20
K
kankan Offline OP
Newbie
kankan  Offline OP
Newbie
K

Joined: Apr 2019
Posts: 20
I was wondering whether even in theory is possible to generally have daily history data but only for some of the days have M1 history (and use it).

The motivation is that I trade many assets, but very very infrequently and only intraday for 1 day. So instead of downloading full M1 history for all the assets which would take a month from my broker, I wanted to download just the days I needed for the backtesting. Since those are earnings, they are well defined.

I am a little bit confused if this is possible at all and how I would approach this.
Sounds like AssetFrame and TimeFrame could be useful for skipping periods with no data until the date that has M1 history, then running at 1-minute intervals (or maybe running at 1-minute intervals the whole time and skipping zero-intraday data?)

Also, does the data have to be stored as M1 with many empty periods, or can it be mixed?

Thank you for any pointers.

Re: Mixed frequency history (M1+D) [Re: kankan] #477127
05/20/19 10:21
05/20/19 10:21
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
It must be stored as M1 with many gaps. If it's a portfolio, let the first asset be the one with the least gaps, because it determines the bars. If the gaps don't overlap, use a dummy asset as the first.

Re: Mixed frequency history (M1+D) [Re: jcl] #477207
05/31/19 13:01
05/31/19 13:01
Joined: Apr 2019
Posts: 20
K
kankan Offline OP
Newbie
kankan  Offline OP
Newbie
K

Joined: Apr 2019
Posts: 20
So I assembled the history in UTC, single file per asset, that has a mix of daily and 1-minute bars, with the timestamp at the end of the bar as written in the manual.
I've been reading about it and I am still very confused on how to set up the script to make use of this data. With the below setup, I get "SMA Lookback period exceeded by 286360 bars" error. I am guessing it's expecting 200*1440 minute bars before the StartDate. Do I need to make my own bar() function to make use of this data properly?


The data is stored like this. I don't need empty bars where there is no data, correct?

2018.07.31 13:33 190.02 190.31 189.95 190.21 0 834 // MINUTE
2018.07.31 13:32 190.31 190.34 189.91 190 0 1233 // MINUTE
2018.07.31 13:31 190.34 190.5 190 190.3 0 4165 //MINUTE
2018.07.30 20:00 191.98 192.2 189.07 189.91 0 1.59208E+07 //DAILY
2018.07.27 20:00 194.93 195.19 190.21 190.98 0 1.41448E+07 //DAILY
2018.07.26 20:00 194.61 195.96 193.61 194.21 0 1.43066E+07 //DAILY


The assets file first has a DUMMY asset, then another entry for AAPL


The very basic script where I am opening a long trade on a specific date that has daily resolution data:
Code:
function run() {
	BarPeriod = 1;
	StartDate = 20130301;
	LookBack = 200*1440;  // enough for the 200day SMA
	LifeTime = 448;  // automatically close trades at the end of the day
	assetList("History\AssetsEarnings.csv");
	History = "m.t6";
	asset("");  // not sure if I need the DUMMY here...
	asset("AAPL");
	
	TimeFrame = 1440;  // daily SMA
	vars dailyPrices = series(priceClose());
	vars smaDaily = series(SMA(dailyPrices, 200));
	 
	// buy after open on 2018-07-31... the day that has minute resolution data
	if(year(0)==2018 && month(0)==7 && day(0)==31) { 
		TimeFrame = 1;  // run on minute intervals during trading
		if(ltod(ET, 0)==931 && NumOpenLong == 0) {
			enterLong();
		}
	}
	else {
		TimeFrame = 1440;
	}


Re: Mixed frequency history (M1+D) [Re: kankan] #477209
05/31/19 14:15
05/31/19 14:15
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
A 1440 timeframe when you have mostly daily bars is many years long and your lookback period is 1000 years. You must calculate the sma by script with only 1 bar per day, and use a normal lookback period of 200 or so.

Re: Mixed frequency history (M1+D) [Re: Petra] #477221
06/01/19 12:42
06/01/19 12:42
Joined: Apr 2019
Posts: 20
K
kankan Offline OP
Newbie
kankan  Offline OP
Newbie
K

Joined: Apr 2019
Posts: 20
Thank you Petra. To play with it, I removed the SMA calculation and then things run as expected.

However, I am still confused with what a Bar really refers to (and therefore BarPeriod). I thought it was a time interval, say 1-minute, and regardless of the gaps in the history I have, it will be a fixed number of them between StartDate and EndDate.

But, it seems that it refers to the actual historical bars, more like candles actually. Is this correct? So in my case, a daily candle (when there are no minute candles), is treated as 1 bar, much the same as an intraday 1-minute candle. Therefore, when I set Lookback to 200, that can refer to a varying amount of time actually, depending on the available history before the StartDate?

Am I understanding this correctly?

Also, I am unsure what you mean by "calculate sma by script". Do you mean I manually have to build a series of just daily prices?

Re: Mixed frequency history (M1+D) [Re: kankan] #477223
06/02/19 04:48
06/02/19 04:48
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
Yes the bar length depends on historical bars. You really need daily prices for the sma, not 1440 bars.

Re: Mixed frequency history (M1+D) [Re: Petra] #477230
06/03/19 16:49
06/03/19 16:49
Joined: Apr 2019
Posts: 20
K
kankan Offline OP
Newbie
kankan  Offline OP
Newbie
K

Joined: Apr 2019
Posts: 20
This is how I managed to build a daily SMA, however this seems extremely fragile to me (for backtesting). For example, if there were no trades at 16h ET, there would be no bar and we wouldn't produce a daily close. There must be a more robust way of doing this, no? Is there some way to track an internal clock during backtesting?
Code:
// capture NYSE market close and the Friday close that falls on Sunday 23h UTC
static int numSkippedBars = 0;
if(lhour(ET, 0)==16 || hour(0)==23) {
	TimeFrame = min(-1, -numSkippedBars);
	numSkippedBars = 0;
}
else {
	TimeFrame = 0;
	numSkippedBars++;
}
vars dailyPrices = series(priceClose(0));
vars sma = series(SMA(dailyPrices, 200));




I also tried this, but it doesn't work correctly and I'm not sure why. It correctly gets the daily close when there are intraday bars and on the first day of the week only. Everything else gets skipped (TimeFrame=0). Why?
Code:
FrameOffset = 21*60-dst(ET, 0)*60;
TimeFrame = frameSync(1440);

vars dailyPrices = series(priceClose(0));
vars sma = series(SMA(dailyPrices, 200));




What would be a proper way to build a daily close series?
Thank you.


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1