Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
3 registered members (AndrewAMD, TipmyPip, Edgar_Herrera), 804 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Page 1 of 2 1 2
Multiple timeframes: prevent loading unnecessary data #485098
01/22/22 14:47
01/22/22 14:47
Joined: Jan 2022
Posts: 57
N
NorbertSz Offline OP
Junior Member
NorbertSz  Offline OP
Junior Member
N

Joined: Jan 2022
Posts: 57
Hello!

I want to use my script for analyzing multiple timeframes, 5 minutes to daily, all with 200 bars LookBack.
As I understand that the normal way to do that is something like this:

Code
BarPeriod = 5;
LookBack = 12*24*200;             // = 5 minutes * 12 * 24    (1 day)    *   200

TimeFrame = 1;
//...analyze the last 200 bars of M5...
TimeFrame = 3;
//...analyze the last 200 bars of M15...
TimeFrame = 12;
//...analyze the last 200 bars of H1...
TimeFrame = 12*4;
//..analyze the last 200 bars of H4...
TimeFrame = 12*24;
//..analyze the last 200 bars of D1...


My problem with this method is the unneccessarly huge amount of data loaded to RAM. Because with live trading (MT4 brigde) the script loads 12*24*200 bars of 5 minutes data per Asset to be able to calculate the Daily bars. But I just only need the last 200 bars for all of the TimeFrames, so basically 12*23*200 bars are useless at 5 minutes.

For larges timeframes, it could be much better and faster loading directly the larger bars, not calculate it by the smallers. Therefore I don't need to load 12*23*200 bars per asset, only 5*200.

Is there a way to do that?

Thank you!

Last edited by NorbertSz; 01/22/22 15:02.
Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485099
01/22/22 17:12
01/22/22 17:12
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
You can set LookBack =200, if you indeed need only 200 data points of each timeframe.

If it varies - try restricting length of series as needed : vars cls1H = series (priceC(), 200);



Last edited by Zheka; 01/22/22 17:27.
Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485100
01/22/22 21:09
01/22/22 21:09
Joined: Jan 2022
Posts: 57
N
NorbertSz Offline OP
Junior Member
NorbertSz  Offline OP
Junior Member
N

Joined: Jan 2022
Posts: 57
When I do this in run():

Code
LookBack = 200;
StartDate = Now;
asset("EURUSD");
BarPeriod = 1;
TimeFrame = 1;
vars Price = series(priceOpen());


I see this in Zorro terminal:

Code
...
Load EURUSD prices.. 600 min, gap 1378 min
Trade: testBarperiod EURUSD 2022-01-22
Lookback 200 bars, 2022-01-21..2022-01-21.
...


When I do this:
Code
LookBack = 200;
StartDate = Now;
asset("EURUSD");
BarPeriod = 1;
TimeFrame = 1000;   //!!!!
vars Price = series(priceOpen());


I see exactly the same in Zorro terminal:

Code
...
Load EURUSD prices.. 600 min, gap 1378 min
Trade: testBarperiod EURUSD 2022-01-22
Lookback 200 bars, 2022-01-21..2022-01-21.
...


It's impossible that 600 mins are enough since these are 1000 minutes bars, and I need 200 of them for LookBack. So LookBack is calculated only by BarPeriod, regardless of TimeFrame.

That's why I should do the LookBack in this case for proper operation:

Code
BarPeriod = 1;
LookBack = 1*1000*200 // It's just 200 when I switch to TimeFrame=1000;


...what I want to avoid because of startspeed and memory savings.
According to the code above, for my calulcations I only need:

- 200 bars of 1-minute
- 200 bars of 1000-minute

but this way I get 200,000 pieces of 1-minute, and calculate dynamically the 200 bars of 1000-minutes.

Last edited by NorbertSz; 01/22/22 21:16.
Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485101
01/22/22 22:06
01/22/22 22:06
Joined: Jul 2017
Posts: 783
Z
Zheka Offline
User
Zheka  Offline
User
Z

Joined: Jul 2017
Posts: 783
1) to create 200 of 1000min candles you need/strongly advised to set(PRELOAD) such history from a *.t6 file.
2) then , try my second suggestion (and read the manual on series() )

Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485104
01/23/22 08:41
01/23/22 08:41
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
Limit the series to 200, because that saves memory. But you will always need 12*24*200 lookback bars for that strategy. You cannot change the bar period during the lookback. Only the time frame.

Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485105
01/23/22 10:19
01/23/22 10:19
Joined: Jan 2022
Posts: 57
N
NorbertSz Offline OP
Junior Member
NorbertSz  Offline OP
Junior Member
N

Joined: Jan 2022
Posts: 57
Okay, I see, thank you for the answers!
But there is two problems with this method.

As testing I choosed M1 to H4 timeframes, and calculations for 400 bars:
Code
BarPeriod = 1;
LookBack = 1 * 60 * 4 * 400;

Problem 01

I attached a plotted result. As you can see there are a lot of useless data here, because my broker do not even store this amount of M1 candles. But I need them for calculating the old H4 candles. These are easily avaliable of couse if I go by BarPeriod = 60*4, TimeFrame = 1. But the above way I can't get the old H4 bars, because these are calculated by Zorro with the M1 candles - wich are not exists.

(I also want to avoid that I mix the previously downloaded historical M1 data and my broker's data.)

Problem 02

First loading of months of M1 data is incredibly slow and unneccessary if I calculating with 60 assets.

Possible solution

A solution can be that I run multiple Zorro instances - each one with different BarPeriod, and not changing TimeFrame at all. These scrips could be fast and simple, but for this I need to make a messaging bridge between them - for example some TXT files or interprocess communication.

Have you got a better idea?
Maybe I miss something? Maybe there is a way to download H4 candle data even if my BarPeriod is 1?

Attached Files m1.png
Last edited by NorbertSz; 01/23/22 10:57.
Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485106
01/23/22 11:59
01/23/22 11:59
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
Use M1 data and PRELOAD. As Zheka said.

https://manual.zorro-project.com/mode.htm


Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #485118
01/24/22 15:52
01/24/22 15:52
Joined: Jan 2022
Posts: 57
N
NorbertSz Offline OP
Junior Member
NorbertSz  Offline OP
Junior Member
N

Joined: Jan 2022
Posts: 57
Thank you!

Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #486323
07/29/22 18:18
07/29/22 18:18
Joined: Jan 2022
Posts: 57
N
NorbertSz Offline OP
Junior Member
NorbertSz  Offline OP
Junior Member
N

Joined: Jan 2022
Posts: 57
I'm stuck with my strategy, because the solutions mentioned above are not meeting the demands.

My problem is still that I need exactly X bars of H4 data and exactly X bars of M1 data. Unfortunately I can't PRELOAD the whole amount of these data from the computer, because:

  • Huge amount of RAM, unnecessarily, because Zorro's way is loading X*60*4 M1 to generate the needed X H4 bars. So it means (X*60*4 - X) amount of unnecessary M1 data: I only need X of that.
  • I want flexible code to run on multiple assets, anytime, just by pressing the Trade button. I don't want to start hunting for historical .t6 data for every time when I start trading, with 60+ assets I trade, especially for exotic ones. And theoretically it's not a problem, because my broker gives me the bars I need for my strategy. But I just don't know how can Zorro handle in that way, so that's why it's a problem.
  • Plus, maybe the historical prices that I can get in .t6 to PRELOAD from an independent source has maybe not the same prices that my broker shows me in the trading session after the LookBack. So I need the data exactly from my broker, not from outer sources.


So if I would make a single-timeframe strategy, it woud be very-very easy to get these X bars of datas: my broker just serves it by BarPeriod = 1, or BarPeriod = 60*4. The problem comes in when I want to use them together, in a multi-timeframe strategy, because Zorro forces to load X*60*4 M1 data to generate the X amount of H4 data - but this amount of M1 data is not avaliable from my broker - and also the RAM problem as mentioned above.

I only want to load X amount of M1 candles and X amount of H4 candles, AND for higher bars: not calculating, but loading from the broker.
For example it would be perfect if Zorro would load the datas 2 times at start: X bars for M1, and X bars for H4.
Is it possible somehow? Do you know an other solution?

In live trading I could run two instances of Zorro, one is in BarPeriod = 1, and the other is in BarPeriod = 60*4, then make some communications between them. But of course I want to backtest the whole strategy, and I would like to not change the code for testing.

I appreciate any ideas about reaching these goals.

Thank you!

Last edited by NorbertSz; 07/29/22 18:19.
Re: Multiple timeframes: prevent loading unnecessary data [Re: NorbertSz] #486324
07/30/22 15:09
07/30/22 15:09
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
If you want X bars M1 data and X bars H4 data, but want to avoid generating the H4 from M1, read them from two different datasets. One for H4 and one for M1. Or get H4 from a pre-loaded dataset and load the M1 data as lookback period from the broker.

I think using 2 Zorros for this is guns on sparrows.




Page 1 of 2 1 2

Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1