I created T1 and T6 files with Tick Data from Dukascopy as described here:
http://statsmage.com/zorro-on-steroids-ii-data-conversion/ http://statsmage.com/zorro-on-steroids-ii-data-conversion/The whole process took two weeks on my system and I had to upgrade my hard drive to store these CSV files. So you will basically save two weeks, maybe a hard drive and if you help to keep this torrent alive for everyone -> win-win situation...!
If you want it, fire up your BitTorrent client (qTorrent is a decent one without adware) and get it. You should be selective about what you download. Not many will want to backtest with a variable spread on T1 data and hack Zorro to do this, so maybe deselect this file: T1_Forex_History_2006-2017_Bid_hack.7z
All data adds up to around 50 GB (compressed by 70%!)
Grab the torrents here:
https://www.dropbox.com/sh/rbgocbu2cwx2ur3/AADGQm38GHjckW_y5CRsfYOfa?dl=0 If you find any mistakes in the dataset, please post your findings here!
Readme:
All credits to statsmage.com!
Do yourself a favor and enable NTFS compression for your Zorro History folder! It will save you 40% disk space!
This data comes in different packages:
Dukascopy started recording Tick Data between January 2003 and the December 2005, depending on the currency pair. The datatypes are M1, Tick Data Ask (Zorro default) and Tick Data Bid. The later can empower you to backtest on Tick Data with variable Spread. Details on statsmage.com or in the attached PDF "Zorro on Steroids III - Backtesting with Variable Spread.pdf"
The different torrent packages are:
-Tick Data Ask and Bid from January 2003 to December 2005 (some currencies records start in 2003 and some in 2005
-Tick Data Ask from January 2006 to December 2017
-Tick Data Bid from January 2006 to December 2017
-Tick Data Ask and Bid from January 2018 to June 2018 (will be replaced in January 2019 by the dataset of the complete year)
-M1 Data from January 2003 to December 2017
-M1 Data from January 2018 to June 2018 (will be replaced in January 2019 by the dataset of the complete year)
Included are all Major Forex Pairs and all Crosses (all possible combinations of AUD, CAD, CHF, EUR, GBP, JPY, NZD, USD)
Read the attached PDF file "Zorro on Steroids II - Data Conversion"
This thread might be interesting too:
http://www.opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=463658If you can afford the disk space and bandwidth, please don't delete the torrent and help to upload. I'm not willing to lever a potential several hundred gigabytes of upload from my private internet connection. -> Don't be a Leecher - become a Seeder and be awesome! ;-)
If you want to create these files, install python 2 from within the Cygwin setup. It didn't work for me when I installed it directly!
The original script published on
http://statsmage.com/ works only with Zorro S. I edited the original script a little so you can use Zorro free in a two - steps process (or rather 3 steps if you count Tickdownloader in)
Set the time zone to (UTC) Coordinated Universal Time. Tickdownloader will export two identical CSV files (for example EURUSD_tick.csv and EURUSD_tick_UTC+0_00.csv). If you forgot this - the first file will contain the UTC time without daylight savings. You can easily delete the second file to save space during the process. Check "Automatic export to CSV" and only export tick data. The conversion to M1 will be done by the Zorro script.
Call the python script by opening the Cygwin Terminal. Enter "cd /cygdrive/d/zorro/strategy/dataconversion" if the python script is in D:ZorroStrategyDataconversion (without the quotes!) and press Enter.
Enter "python convert_all_assets.py /cygdrive/e/tickdata/" if your CSV files are in E:tickdata
Don't press any keys while the Cygwin terminal is active and a script is running. It might break the whole process!
If you want to optimally use your computer's resources, distribute the CSV files on different folders (as many as you have physical CPU Cores) and if possible on different physical hard drives as well, as the reversion step is limited by your hard drive's speed. An SSD really rocks here.
Edit the the ConvertData_all_assets.c script according to your needs (first couple of lines)
