Gamestudio Links
Zorro Links
Newest Posts
How to: plot a frequency spectrum in bars?
by Sempervirens. 05/12/21 02:45
Backtest accuracy
by AndrewAMD. 05/12/21 00:26
Test one asset with more bar period in one run
by AndrewAMD. 05/12/21 00:22
System with more take profit
by AndrewAMD. 05/12/21 00:20
EXANTE broker
by madpower2000. 05/11/21 17:02
No Trades when Backtesting Period is longer
by TimmiHendrix. 05/11/21 15:47
Last Candle for Testing and Train?
by jcl. 05/11/21 12:14
DataSplit for Advise
by TimmiHendrix. 05/11/21 10:40
AUM Magazine
Latest Screens
Hecknex World
Scheherazade's Journey
Paradox Vector
The Space Between
Who's Online Now
3 registered members (AndrewAMD, Sempervirens, jyd), 470 guests, and 3 spiders.
Key: Admin, Global Mod, Mod
Newest Members
ShadowAx, TimmiHendrix, contactmmfx, msupl7, Robert_
18644 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Tick data and high-precision backtesting development #481785
10/30/20 19:05
10/30/20 19:05
Joined: Oct 2020
Posts: 7
A
AVL Offline OP
Newbie
AVL  Offline OP
Newbie
A

Joined: Oct 2020
Posts: 7
Hello.

I've just started to use Zorro S and unfortunately met some difficulties with tick data.

I'm a little disappointed that such a professional product missing so essential things for high-precision backtesting like Volume (Ask/Bid Size) in .t1 format. Yes, users could use .t6 for this matter, but it's illogical and not sound at all. I have hundreds of gigabytes of tick data in .csv, which I was going to transfer to .t1 (or .t6, unfortunately), and after the first export attempt, I got an error that the memory limit is exceeded (2605 MB). Well, not good at all. Considering that some .csv files are 10GB+ in size. Then I started to look for explanations in the manual and found that Zorro can't even backtest more than 2 years using TICKS parameter. Well, this is finally very bad, all competitors can handle this.

I'm writing this not to critique but to point on the significant drawback. I like a lot what you have developed, but you should improve this vital part of the software. Otherwise, it can't be called high-precision backtesting.

Attached Files Seriously.png
Re: Tick data and high-precision backtesting development [Re: AVL] #481788
10/31/20 14:53
10/31/20 14:53
Joined: Feb 2017
Posts: 1,254
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,254
Chicago
The memory limit is because Zorro is 32-bit. oP Group is working on adding a 64-bit version, as noted here:
Quote
Features planned for the next releases:
Overcoming the 32-bit limit for training and backtests (Zorro S; in development).
https://zorro-project.com/manual/en/new.htm

Re: Tick data and high-precision backtesting development [Re: AVL] #481789
10/31/20 15:00
10/31/20 15:00
Joined: Oct 2020
Posts: 7
A
AVL Offline OP
Newbie
AVL  Offline OP
Newbie
A

Joined: Oct 2020
Posts: 7
Originally Posted by AndrewAMD
The memory limit is because Zorro is 32-bit. oP Group is working on adding a 64-bit version, as noted here:
Quote
Features planned for the next releases:
Overcoming the 32-bit limit for training and backtests (Zorro S; in development).
https://zorro-project.com/manual/en/new.htm


This is great news. Still, would be great if developers enhance possibilities of .t1 by adding additional volume fields and making an option for "order fill check" during the backtest.

Re: Tick data and high-precision backtesting development [Re: AVL] #483039
04/26/21 11:31
04/26/21 11:31
Joined: Apr 2021
Posts: 14
Kolarovo
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 14
Kolarovo
Hello,

Does Zorro S able to handle tick data normally? I wanted to translate my tick data from .txt (+10Gb file size) to t1 but unfortunately the free Zorro version can't handle the data. It handled one month of data, converted it to t1 which I was able to check in History editor, but it's nonsense for me to export data monthly and convert it. I read about performance problems too with tick data backtests. At this point I'm not sure I want to put more time in Zorro or even buy Zorro S.

Can someone give information about how the process goes with the 64bit version and how it will handle accurate tick-by-tick backtests?
Thank you!

Re: Tick data and high-precision backtesting development [Re: tomna1993] #483040
04/26/21 12:39
04/26/21 12:39
Joined: Feb 2017
Posts: 1,254
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,254
Chicago
Originally Posted by tomna1993
Hello,

Does Zorro S able to handle tick data normally? I wanted to translate my tick data from .txt (+10Gb file size) to t1 but unfortunately the free Zorro version can't handle the data. It handled one month of data, converted it to t1 which I was able to check in History editor, but it's nonsense for me to export data monthly and convert it. I read about performance problems too with tick data backtests. At this point I'm not sure I want to put more time in Zorro or even buy Zorro S.

Can someone give information about how the process goes with the 64bit version and how it will handle accurate tick-by-tick backtests?
Thank you!
There is no 64-bit version. If there was, it would work by consuming more memory - this would be the only difference from the 32-bit version. You can read how tick-based backtesting works in the manual.

For now, you can split up your huge 10GB file into smaller CSV files, convert each to t1 format, and backtest with one t1 file at a time.

Re: Tick data and high-precision backtesting development [Re: AndrewAMD] #483043
04/26/21 18:10
04/26/21 18:10
Joined: Apr 2021
Posts: 14
Kolarovo
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 14
Kolarovo
Okay, thanks for info!

Re: Tick data and high-precision backtesting development [Re: AVL] #483079
05/01/21 10:02
05/01/21 10:02
Joined: Apr 2008
Posts: 348
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 348
Austria
For ask bid volumes, use only .t2. Not .t6 or .t1, they are for prices only.

Convert your data to .t1 and .t2, and use .t1 for the price history and .t2 for evaluating order flow or getting the volumes of ask bid quotes. Look here: https://manual.zorro-project.com/ordercvd.htm

Re: Tick data and high-precision backtesting development [Re: AVL] #483142
05/05/21 19:27
05/05/21 19:27
Joined: Apr 2021
Posts: 14
Kolarovo
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 14
Kolarovo
Thank you Petra! I'll try this.


Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1