Using Option Chains at 1 Minute Intervals

Zorro does support option strategies, and can read historical option data from sources such as IVolatility.

It seems that for options Zorro may select the option data file based on date only, assuming that the option file contains EOD option data for that day only.

So if you are working on an intraday option strategy, and option data from IVolatility, which contains the options chains at each minute of the day, how do you select the correct option change at the time of the current bar ? It's not entirely clear how "Asset" function can select the correct data for a given bar that is being processed.

If anyone has done this before, any guidance would be appreciated.

The tutorial has an options strategy where you can see how it works. But for using 1 minute options data you must obviously set the bar period also to 1 minute.

There are some subtle differences when you use 1 minute data, for instance you must then update the contracts differently in backtest and in live trading.

In backtest the options chain has also the prices, so load it at any bar. But live it loads no prices but can take up to 10 minutes, so better load it live only once per day.

Is there a descripttion in the manual for these subtle differences? I did not find these.
At the moment I am struggeling at importing IVolatility one minute data. I am using the sample file they are providing, but the CSV to Options script seems to have a problem with the import format:

// 2019-10-29 09:30:00,9327,SPX,2019-10-30,1800,C,E,SPXW 191030C01800000,1234.1,1237.3,2019-10-29 09:11:36,2019-10-29 09:11:36,7,7,W,W,0,0.3538,3034.4788,0.99993,0,0,0,0.0632,-1,0,2019-10-29 09:27:37,2019-10-29 09:30:00
string Format = "%Y-%m-&d %H:%M:%S,,,i,f,s,s,,f,f,,,i,i,,,i,,f"; // from the sample above

For backtesting you need not care about subtle differences. But when live trading you should know how long it takes to download the contract chain from your broker. If longer than one bar period, you can obviously not load it at any bar and must load it less frequently. Preferably once per day and outside market hours. If you have problems, we have a service for programming your options strategy.

The ivolatility option formats are very different and change all the time, so you must normally adapt the code to the format of your data. We also have a service for that.

I managed to adjust the import script but due to the size I run into the next problem.
Is there a way to store the .t8 data in e.g. daily files?

To put options data for a whole year is way to much data for the 32bit memory limit. Could history files named other than ASSET_YEAR.t8 ?
Unfortunately, I did not find anything in the manual.

Can multiple .t8 file be concatinated?

Yes. A t8 file consists of nothing more than an array of CONTRACT structs, so you can assemble that as you please into an ASSET.t8 file. The dataset functions, file functions, and time functions included with Zorro make this easy.

Yes, but the problem is, that the options data of one year exceeds the memory limit.

So my question is, can I use history files that are organized like this:

SPX_2019-01-01.t8
SPX_2019-01-02.t8
.
.


When I try it, Zorro tells me it does not find SPX.t8. At the moment I am trying to play with the "History" Variable and set it to the appropriate file for every day but this did not succeed yet:

History = strf("*_%s.t8", strdate("%Y-%m-%d",ldate(ET,0)));

My filenames are:
SPX_2019-08-01.t8
SPX_2019-08-02.t8
.
.


Any other thoughts? If this is not possible, does it mean you can only backtest option strategies when the data of one year does not exceed around 3 GB?

Again, yes.

Load one day file at a time into data set A. Process it into daily data. Append it to data set B. Clear data set A and repeat for the next day, and so on, until data set B has all of the days.

This can all be done within a 32-bit footprint.

ok, so if I read each day separately (A) and append it to (B) by using "dataAppend", I will have a .t8 file on disk which is lets say 100 GB for one year.
When I use this file for backtesting, than the memory limitation does not come into play?

No, there will be processing in between. Dataset A gets processed into A_proc. Then you append A_proc to B. Obviously, A_proc will be much smaller than A.

In processing it, your objective is to summarize what happened to each contract during the day.

Rereading your question, it looks like you are asking about daily history files being the final file, not "daily contract data" (i.e. one data point per day).

Sure, why not. But you'll need to run one backtest per t8 history file, so you'll need to set up batching.

By the way, this is illegal because History needs to point to a permanent and unchanging string, not a temporary string.

basically what I am trying to do is to backtest an SPX option strategy on 1 min intervalls.
But I realized I cannot import the data as the dataset becomes too large...
And apparently zorro only allows for yearly history files.

Hmm, meanwhile I tried to load the contract data manually:
dataLoad(1, strf("SPX_%s.t8", strdate("%Y-%m-%d",ldate(ET,0))),9);

but ldate is in this case the actual date, not the historical date. I guess Zorro loads the "oldest" data it finds for the asset and runs from there. Trying to set StartDate did not help.

Actually, you're on the right track! You can dynamically load contract data as the test progresses.

I don't understand what problem you're having with dataload, though.

I have found the problem. I only had the SPX .t8 files in History folder. But I need SPX .t6 files in the folder as well....

If you’re manually loading t8 files, you can locate them in any folder that you please.

The Option chain is loaded into memory now but contractFind seems to have a problem. I can export the contract chain with contractPrint() and it looks ok.
Does contractFind pick the right contract depending on the timestamp of the chain?

Really having difficulties with intraday option data :-(

By default, it uses the current run() timestamp. Or you can override it by setting the Now variable to a particular time where you want to load contracts.
https://zorro-project.com/manual/en/contract.htm
https://zorro-project.com/manual/en/date.htm (Now)