Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
5 registered members (Nymphodora, AndrewAMD, TipmyPip, Quad, Imhotep), 847 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Tick data and high-precision backtesting development #481785
10/30/20 19:05
10/30/20 19:05
Joined: Oct 2020
Posts: 7
A
AVL Offline OP
Newbie
AVL  Offline OP
Newbie
A

Joined: Oct 2020
Posts: 7
Hello.

I've just started to use Zorro S and unfortunately met some difficulties with tick data.

I'm a little disappointed that such a professional product missing so essential things for high-precision backtesting like Volume (Ask/Bid Size) in .t1 format. Yes, users could use .t6 for this matter, but it's illogical and not sound at all. I have hundreds of gigabytes of tick data in .csv, which I was going to transfer to .t1 (or .t6, unfortunately), and after the first export attempt, I got an error that the memory limit is exceeded (2605 MB). Well, not good at all. Considering that some .csv files are 10GB+ in size. Then I started to look for explanations in the manual and found that Zorro can't even backtest more than 2 years using TICKS parameter. Well, this is finally very bad, all competitors can handle this.

I'm writing this not to critique but to point on the significant drawback. I like a lot what you have developed, but you should improve this vital part of the software. Otherwise, it can't be called high-precision backtesting.

Attached Files Seriously.png
Re: Tick data and high-precision backtesting development [Re: AVL] #481788
10/31/20 14:53
10/31/20 14:53
Joined: Feb 2017
Posts: 1,718
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,718
Chicago
The memory limit is because Zorro is 32-bit. oP Group is working on adding a 64-bit version, as noted here:
Quote
Features planned for the next releases:
Overcoming the 32-bit limit for training and backtests (Zorro S; in development).
https://zorro-project.com/manual/en/new.htm

Re: Tick data and high-precision backtesting development [Re: AVL] #481789
10/31/20 15:00
10/31/20 15:00
Joined: Oct 2020
Posts: 7
A
AVL Offline OP
Newbie
AVL  Offline OP
Newbie
A

Joined: Oct 2020
Posts: 7
Originally Posted by AndrewAMD
The memory limit is because Zorro is 32-bit. oP Group is working on adding a 64-bit version, as noted here:
Quote
Features planned for the next releases:
Overcoming the 32-bit limit for training and backtests (Zorro S; in development).
https://zorro-project.com/manual/en/new.htm


This is great news. Still, would be great if developers enhance possibilities of .t1 by adding additional volume fields and making an option for "order fill check" during the backtest.

Re: Tick data and high-precision backtesting development [Re: AVL] #483039
04/26/21 11:31
04/26/21 11:31
Joined: Apr 2021
Posts: 41
Slovakia
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 41
Slovakia
Hello,

Does Zorro S able to handle tick data normally? I wanted to translate my tick data from .txt (+10Gb file size) to t1 but unfortunately the free Zorro version can't handle the data. It handled one month of data, converted it to t1 which I was able to check in History editor, but it's nonsense for me to export data monthly and convert it. I read about performance problems too with tick data backtests. At this point I'm not sure I want to put more time in Zorro or even buy Zorro S.

Can someone give information about how the process goes with the 64bit version and how it will handle accurate tick-by-tick backtests?
Thank you!

Re: Tick data and high-precision backtesting development [Re: tomna1993] #483040
04/26/21 12:39
04/26/21 12:39
Joined: Feb 2017
Posts: 1,718
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,718
Chicago
Originally Posted by tomna1993
Hello,

Does Zorro S able to handle tick data normally? I wanted to translate my tick data from .txt (+10Gb file size) to t1 but unfortunately the free Zorro version can't handle the data. It handled one month of data, converted it to t1 which I was able to check in History editor, but it's nonsense for me to export data monthly and convert it. I read about performance problems too with tick data backtests. At this point I'm not sure I want to put more time in Zorro or even buy Zorro S.

Can someone give information about how the process goes with the 64bit version and how it will handle accurate tick-by-tick backtests?
Thank you!
There is no 64-bit version. If there was, it would work by consuming more memory - this would be the only difference from the 32-bit version. You can read how tick-based backtesting works in the manual.

For now, you can split up your huge 10GB file into smaller CSV files, convert each to t1 format, and backtest with one t1 file at a time.

Re: Tick data and high-precision backtesting development [Re: AndrewAMD] #483043
04/26/21 18:10
04/26/21 18:10
Joined: Apr 2021
Posts: 41
Slovakia
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 41
Slovakia
Okay, thanks for info!

Re: Tick data and high-precision backtesting development [Re: AVL] #483079
05/01/21 10:02
05/01/21 10:02
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
For ask bid volumes, use only .t2. Not .t6 or .t1, they are for prices only.

Convert your data to .t1 and .t2, and use .t1 for the price history and .t2 for evaluating order flow or getting the volumes of ask bid quotes. Look here: https://manual.zorro-project.com/ordercvd.htm

Re: Tick data and high-precision backtesting development [Re: AVL] #483142
05/05/21 19:27
05/05/21 19:27
Joined: Apr 2021
Posts: 41
Slovakia
T
tomna1993 Offline
Newbie
tomna1993  Offline
Newbie
T

Joined: Apr 2021
Posts: 41
Slovakia
Thank you Petra! I'll try this.


Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1