M1 tick data format

Posted By: jbhunter

M1 tick data format - 03/21/20 04:38

I am wanting to experiment with some 1 minute tick data (M1). To start with I can get SPY for ~$15 of about 20 years in csv format to mess around with. I am curious now the path to get csv to zorro format files. For formatting I am good with c# and the ZHistoryEditor seems to be dotNet, I was thinking of leaning on that to figure out how to parse and output the zorro format. Does anyone know if ZHistoryEditor is open source? Am running the wrong path from a price and/or effort point of view? Is there an easier path? Any suggestions welcomed.

Thanks
Posted By: ugofx

Re: M1 tick data format - 03/21/20 08:48

Look at CSVtoHistory.c
In case of problems search with that name in the forum.
Posted By: jbhunter

Re: M1 tick data format - 03/24/20 05:13

This script is very handy. Hopefully I can use this for my data. If anything the comments show the format there is code to decipher output for the zorro history files.

Thanks!
Posted By: jbhunter

Re: M1 tick data format - 03/26/20 20:21

I have some data now. Some of the dates were YYYY-mm-dd and some were YYYY.mm.dd. I changed them all to YYYYmmdd. The data looks like below.

Date,Time,Open,High,Low,Close
20200102,093000,3244.67,3247.4,3244.67,3247.22
20200102,093100,3247.19,3247.22,3245.01,3245.22
20200102,093200,3245.07,3245.54,3244.16,3244.66
20200102,093300,3244.89,3247.69,3244.89,3247.61
20200102,093400,3247.38,3248.08,3246.92,3246.92
20200102,093500,3246.89,3249.15,3246.89,3249.09
20200102,093600,3249.02,3249.45,3248.2,3249.45
20200102,093700,3249.38,3249.63,3248.49,3248.54

I am using a format in CSVtoHistory.c like below.
string Format = "%Y%m%d,%H%M%S,f3,f1,f2,f4"

I am getting .t6 output files. However when trying to use in backtest I get "No asset data for SPX".

When load the .t6 files in ZHistoryEditor am getting "2000.12.31 00:00" for all of the dates. It is picking up the open/high/low/close data. And the spread and vol are showing zeros.

Any thoughts on what is happening here? I notice in the help files there is mention of using Verbose=7 to debug. I tried setting this in the zorro.ini and in the .c script but I am not seeing any log/debug output. Any help appreciated. Thanks!
Posted By: jcl

Re: M1 tick data format - 03/27/20 07:02

"No asset data" means the SPX data is missing in your asset list. The format string is missing a "+". And no debug output means that your attempts to set Verbose have not yet been successful.
Posted By: jbhunter

Re: M1 tick data format - 03/27/20 12:20

jcl, thanks for the help.

I added the + for decending order.

I added an asset list, now I am getting Error 55. No valid ticks, No bars generated.

I added set(LOGFILE) to CSVtoHistory and am now seeing the debug info. It appears to be grabbing the date in OA format. I am not sure what the 0.39722222 and 0.39791667 refer to.

DTA: 34 MB allocated
20040906,093200,1114.75,1114.75,1114.65,1114.65

38236.00000000 0.39722222 1114.75000 1114.75000 1114.65000 1114.65000
20040906,093300,1114.65,1114.65,1114.65,1114.65

38236.00000000 0.39791667 1114.65000 1114.65000 1114.65000 1114.65000 ....... 1528310 records
1528310 lines read
Posted By: jbhunter

Re: M1 tick data format - 03/27/20 12:47

I finally figured this one out. I changed my format of the date in my date to 9:32:00, and my format spec to

string Format = "+%Y%m%d,%H:%M:%S,f3,f1,f2,f4,f5,f6";

I also had to pad my date with ,0,0 at the end to zero out the spread and vol data. This appears to be working. Is it possible to parse data without the spread and volume data?

Thanks!
© 2024 lite-C Forums