Gamestudio Links
Zorro Links
Newest Posts
Oanda instruments
by Morris. 06/05/20 10:24
Indicator coding
by Grat. 06/05/20 09:29
MT4/5 - wrong M1 time
by Grat. 06/05/20 08:34
fce for close signal
by Grat. 06/05/20 06:09
TMF - Chandelier
by danatrader. 06/05/20 05:03
The Black Book
by danatrader. 06/05/20 04:29
AUM Magazine
Latest Screens
The Space Between
Pogostuck: Rage With Your Friends
Worst Case Z
AckCon'18 - Lotter vs the World 2 - Preview Release
Who's Online Now
10 registered members (Mio, 3run, kvm, onoff, Grat, alibaba, AndrewAMD, FLD, jenGs, 1 invisible), 482 guests, and 6 spiders.
Key: Admin, Global Mod, Mod
Newest Members
Frank_Shieh, PBSeven, Hardi01, Giuseppe, AdamWu
18461 Registered Users
Previous Thread
Next Thread
Print Thread
Rating: 5
Page 1 of 3 1 2 3
Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! #473310
06/27/18 19:39
06/27/18 19:39
Joined: Dec 2014
Posts: 171
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 171
Germany
I created T1 and T6 files with Tick Data from Dukascopy as described here: http://statsmage.com/zorro-on-steroids-ii-data-conversion/ http://statsmage.com/zorro-on-steroids-ii-data-conversion/

The whole process took two weeks on my system and I had to upgrade my hard drive to store these CSV files. So you will basically save two weeks, maybe a hard drive and if you help to keep this torrent alive for everyone -> win-win situation...!


If you want it, fire up your BitTorrent client (qTorrent is a decent one without adware) and get it. You should be selective about what you download. Not many will want to backtest with a variable spread on T1 data and hack Zorro to do this, so maybe deselect this file: T1_Forex_History_2006-2017_Bid_hack.7z

All data adds up to around 50 GB (compressed by 70%!)


Grab the torrents here: https://www.dropbox.com/sh/rbgocbu2cwx2ur3/AADGQm38GHjckW_y5CRsfYOfa?dl=0

If you find any mistakes in the dataset, please post your findings here!

Readme:

All credits to statsmage.com!


Do yourself a favor and enable NTFS compression for your Zorro History folder! It will save you 40% disk space!

This data comes in different packages:

Dukascopy started recording Tick Data between January 2003 and the December 2005, depending on the currency pair. The datatypes are M1, Tick Data Ask (Zorro default) and Tick Data Bid. The later can empower you to backtest on Tick Data with variable Spread. Details on statsmage.com or in the attached PDF "Zorro on Steroids III - Backtesting with Variable Spread.pdf"

The different torrent packages are:

-Tick Data Ask and Bid from January 2003 to December 2005 (some currencies records start in 2003 and some in 2005
-Tick Data Ask from January 2006 to December 2017
-Tick Data Bid from January 2006 to December 2017
-Tick Data Ask and Bid from January 2018 to June 2018 (will be replaced in January 2019 by the dataset of the complete year)
-M1 Data from January 2003 to December 2017
-M1 Data from January 2018 to June 2018 (will be replaced in January 2019 by the dataset of the complete year)

Included are all Major Forex Pairs and all Crosses (all possible combinations of AUD, CAD, CHF, EUR, GBP, JPY, NZD, USD)


Read the attached PDF file "Zorro on Steroids II - Data Conversion"
This thread might be interesting too: http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=463658

If you can afford the disk space and bandwidth, please don't delete the torrent and help to upload. I'm not willing to lever a potential several hundred gigabytes of upload from my private internet connection. -> Don't be a Leecher - become a Seeder and be awesome! ;-)

If you want to create these files, install python 2 from within the Cygwin setup. It didn't work for me when I installed it directly!

The original script published on http://statsmage.com/ works only with Zorro S. I edited the original script a little so you can use Zorro free in a two - steps process (or rather 3 steps if you count Tickdownloader in)

Set the time zone to (UTC) Coordinated Universal Time. Tickdownloader will export two identical CSV files (for example EURUSD_tick.csv and EURUSD_tick_UTC+0_00.csv). If you forgot this - the first file will contain the UTC time without daylight savings. You can easily delete the second file to save space during the process. Check "Automatic export to CSV" and only export tick data. The conversion to M1 will be done by the Zorro script.


Call the python script by opening the Cygwin Terminal. Enter "cd /cygdrive/d/zorro/strategy/dataconversion" if the python script is in D:ZorroStrategyDataconversion (without the quotes!) and press Enter.

Enter "python convert_all_assets.py /cygdrive/e/tickdata/" if your CSV files are in E:tickdata

Don't press any keys while the Cygwin terminal is active and a script is running. It might break the whole process!

If you want to optimally use your computer's resources, distribute the CSV files on different folders (as many as you have physical CPU Cores) and if possible on different physical hard drives as well, as the reversion step is limited by your hard drive's speed. An SSD really rocks here.

Edit the the ConvertData_all_assets.c script according to your needs (first couple of lines)

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: Smon] #473315
06/28/18 11:54
06/28/18 11:54
Joined: Feb 2015
Posts: 605
Milano, Italy
M
MatPed Offline
User
MatPed  Offline
User
M

Joined: Feb 2015
Posts: 605
Milano, Italy
I do not use tick data, but great job and great sharing!

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: MatPed] #473320
06/28/18 17:51
06/28/18 17:51
Joined: Dec 2014
Posts: 171
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 171
Germany
You can select only T6 Data (M1) as well! It's an extra archive within the torrents.

Last edited by sdh309795gaas; 06/28/18 18:21.
Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: Smon] #473425
07/04/18 14:33
07/04/18 14:33
Joined: Dec 2013
Posts: 14
O
oligodar Offline
Newbie
oligodar  Offline
Newbie
O

Joined: Dec 2013
Posts: 14
Thank you !!!!!!!

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: oligodar] #473427
07/04/18 14:50
07/04/18 14:50
Joined: Dec 2013
Posts: 14
O
oligodar Offline
Newbie
oligodar  Offline
Newbie
O

Joined: Dec 2013
Posts: 14
I am gonna seed it al least till the end of year 2018. If you create new torrent with 2018 year I am gonna seed it as well.

Thank youu!!!!

Last edited by oligodar; 07/04/18 14:54.
Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: oligodar] #476308
02/16/19 17:02
02/16/19 17:02
Joined: Dec 2014
Posts: 171
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 171
Germany

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: Smon] #476309
02/16/19 17:13
02/16/19 17:13
Joined: Aug 2017
Posts: 102
Spain
B
Brax Offline
Member
Brax  Offline
Member
B

Joined: Aug 2017
Posts: 102
Spain
Awesome!

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: Brax] #476314
02/17/19 15:38
02/17/19 15:38
Joined: Aug 2018
Posts: 87
O
OptimusPrime Offline
Junior Member
OptimusPrime  Offline
Junior Member
O

Joined: Aug 2018
Posts: 87
Thank you so much.. sdh309795gaas

I can't load the Torrent with UTorrent. It gives an error message about an invalid "Bencoding"

Last edited by OptimusPrime; 02/17/19 16:05.

Thanks so much,

OptimusPrime

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: OptimusPrime] #476321
02/17/19 19:18
02/17/19 19:18
Joined: Dec 2014
Posts: 171
Germany
Smon Offline OP
Member
Smon  Offline OP
Member

Joined: Dec 2014
Posts: 171
Germany
Try qBittorrent. The torrent seems okay. I have already uploaded the package 1,76 times.

Re: Dukascopy T1&T6 files for all Forex Pairs and Crosses - get it! [Re: Smon] #476428
02/24/19 22:26
02/24/19 22:26
Joined: Feb 2019
Posts: 14
Other
J
j45p41 Offline
Newbie
j45p41  Offline
Newbie
J

Joined: Feb 2019
Posts: 14
Other
Hi,

first of all - great effort!

I am interesed in the M1 data for forex crosses. Maybe I have missed something obvious (sorry if I have) but how to you update the assets.fix file with correct spread, price, etc?

Thanks!

Page 1 of 3 1 2 3

Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1