Easiest way to get 1m History?

I like Zorro's backtesting speed and parameter optimization. Would like to utilize that.

I would like to feed it whole MATICUSDT price history.

How do I do that? It fails to download it from Binance via Download script (did log in successfully).

From the help file - found a script example that can pull ohlc`s from coinapi and wrote:

```
function run()
{
set(LOGFILE);
BarPeriod = 1;
asset("MATICUSDT");
vars Prices = series(price());


// Download Bitcoin OHLC prices from Coin.io
int Month, Year = 2021, MinutesPerMonth = 31*1440;
for(Month = 1; Month <= 12; Month++) {
string URL = strf("https://rest.coinapi.io/v1/quotes/BINANCE_SPOT_MATIC_USDT/history?period_id=1MIN&time_start=%04d-%02d-01&limit=%d&apikey=123-API-KEY",
Year,Month,MinutesPerMonth);

string Name = strf("MATICUSDT_%04d.t6",Year);
assetSource(Name,URL,
"X-CoinAPI-Key: 123-API-KEY",0,
"[,time_period_end,%Y-%m-%dT%H:%M:%SZ,price_high,price_low,price_open,price_close,,volume_traded");
assetHistory(0,FROM_SOURCE);
}

}
```

It does show it in browser but for some reason - Zorro fails to pick it up. Maybe I already exhausted request quota - not sure.

Why it's such a hassle?

The Download script is easiest.

You say downloading via Binance did not work. What went wrong? What is your asset list configuration? What error message did you receive? What did you do to correct an error on your end based on what the error messages told you?

Share your logfile.

https://zorro-project.com/manual/en/account.htm (Asset List)
https://manual.zorro-project.com/binance.htm (Binance plugin)
https://zorro-trader.com/manual/en/trouble.htm (troubleshooting)
https://zorro-trader.com/manual/en/errors.htm (error codes)

Glad to see an answer. Means forum is alive.

Got this line in AssetsFix.txt file:

MATIC/USDT,0.730530,0.000150000,0.1665,-0.4580,0.00010000,0.10000,7.305,0,1000,0.0,MATICUSDT

As I understand - the really important bit is the last item that identifies symbol.

If I change it to anything else like this:
MATIC/USDT,0.730530,0.000150000,0.1665,-0.4580,0.00010000,0.10000,7.305,0,1000,0.0,MATICUSDT_INVALID

And try to download M1, Zorro says:
!BrokerHistory2: Symbol MATICUSDT_INVALID not found

If I change it back and try to download 2020-2021, whole thing runs smoothly but the both t6 files are only 47KB long.
They basically are empty.

If I try to download Ticks, it says:
Error 063: Tick history not available

--------------------

Tried to do whole thing manually.
Downloaded bunch of csv`s through binance api.
Did some cleanup so it looks like this:

Date,Open,High,Low,Close,Volume,QuoteVolume
26/04/2019 18:00:00,0.00263000,0.01052000,0.00263000,0.00639000,157258876.10000000,1092100.13082700

Modified CSVtoHistory script format to:
string Format = "%d/%m/%Y %H:%M:%S,f3,f1,f2,f4,f6";

And ran import. Seemingly successfully.

Ran CSVfromHistory to see what has made in and it looks decent:
2021-01-01T00:00:00,0.01742,0.01742,0.01742,0.01742,0.00000,5312.29980
2021-01-01T00:01:00,0.01741,0.01741,0.01740,0.01740,0.00000,79141.60156
2021-01-01T00:02:00,0.01739,0.01739,0.01735,0.01735,0.00000,157942.59375
2021-01-01T00:03:00,0.01736,0.01736,0.01735,0.01735,0.00000,46928.89844

But when I try to run Ehler's AMFM script or Workshop4 or Workshop5, it just says:
Error 047: No bars generated

Sometimes:
Error 042: no MATICUSDT parameters at walk 0
^ tried to change MATIC/USDT in assets list to MATICUSDT for the name too

And Zorro Chart Viewer says:
Unspecified error

Tried setting various StartDate and EndDate. Nothing changed.
Tried setting BarPeriod to 1, 5, 60. Nothing changed.

Why does it dislike that imported data?

Also - failed to find the log files. Tried to set LOGFILE flag but that produced nothing.

It sounds like Binance does not store very much historical data. Audit the t6 files you downloaded for some hints.

Next, there are many helpful suggestions on how to handle each particular error code. Start there:
https://zorro-trader.com/manual/en/errors.htm (error codes)

Next, did you audit your t6 files? Use the History script (or the Z History Editor from the download section). Is the t6 data sorted from newest to oldest (and not the other way around)? Is the file naming convention identical to what is described in the manual?You probably put it in the wrong spot. Post your script.

> It sounds like Binance does not store very much historical data.

Probably untrue. At least for my case. Got data since its inception 29-4-2019.

Did visit manual errors section. Didn't find it helpful.

> Next, did you audit your t6 files?

Not yet.

> Is the t6 data sorted from newest to oldest (and not the other way around)?

Hoping that this is the culprit. Will test that now. <------------------

> Is the file naming convention identical to what is described in the manual?

Think there's no issue with this.

> You probably put it in the wrong spot. Post your script.

```
function run()
{
set(LOGFILE);
```


-----------------------------------------

I guess it's the sorting issue. Will try to reverse it and import again.




-----------------------------------------

Noticed there's a Sort function on the history viewer.

Sorted, saved.

-----------------------------------------

IT'S ALIVE!!!!!!!!!!!!!!!!



I'm so excited.... Such a playground.

Thank you, Andrew! Much appreciated.

You can also use dataSort on the dataset prior to saving to t6 file.
https://www.zorro-project.com/manual/en/data.htm

Well... I need to thoroughly RTFM. At least now it kind of seems worth it.

One more quick question - going live would still require manual data feeding or Zorro would somehow figure it out and pick up price data via Websocket?

-----------------------------

For the sake of learning, squashed together this script:

```
#include <profile.c>

function run()
{
BarPeriod = 1;
StartDate = 20210729;
EndDate = 20210809;
Verbose = 2;
LookBack = 300;
asset("MATICUSDT");
set(LOGFILE|PLOTNOW);
set(PARAMETERS);

vars Prices = series(priceClose());

int SrsiPeriod = optimize(5, 5, 60, 1, 2);

vars SRSI = series(StochRSI(Prices, SrsiPeriod, 3, 3, MAType_KAMA));

int AtrMultiplier = optimize(0.1, 0.1, 4, 0.1, 1);
int AtrPeriod = optimize(3, 3, 100, 1, 1);

Stop = AtrMultiplier*ATR(AtrPeriod);

if(crossOver(SRSI,30))
enterLong();
else if(crossUnder(SRSI,70))
enterShort();
}
```

Little SRSI cross strat that runs on KAMA just because I could feed it in.

First question would be - how come there's only 1 SRSI param for period?
So far I've seen that SRSI runs on 2 periods: 1 for RSI, 1 for stoch.

Second question would be about ridiculous results:

```
Read stoch_rsi.par
Test: stoch_rsi MATICUSDT 2021
Warning 035: MATICUSDT outlier at #7021 2021-08-01 1.0057->1.0877
Warning 035: MATICUSDT outlier at #14040 2021-08-08 1.0779->1.1966
Warning 035: MATICUSDT outlier at #14041 2021-08-08 1.1966->1.0872
Monte Carlo Analysis... Median AR 16657%
Win 1985$ MI 5492$ DD 225$ Capital 733$
Trades 1308 Win 49.5% Avg +15.2p Bars 10
AR 8989% PF 2.22 SR 0.00 UI 0% R2 1.00

Chart...
Copied to Clipboard!
```

What's an outlier? Imported price data is still invalid?

What's up with that annual return?
I guess it doesn't take fees into account but still - number looks stupid.

How would you go about debunking that?

Nevermind. Figured out what's happening. All in all - Zorro looks decent.
Hopefully live trading doesn't flop.

Time to learn.

Hi,

Binance has data since inception and you can retrieve them all with Download.c script or with assetHistory function.

Problem with Download.c script is that you will need to click Download each 1500 ticks. Thats almost 1 time per day as a day has 1440 ticks.

I have a simple script that download the full year and I am happy to share it here. Please note that I am not a coder, and the script hangs at the end... but the data is downloaded.

Also note this report from yesterday: https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=484037#Post484037 Using this method also insert 1 minute gap between each 1500 ticks.



That code works well with Binance Futures plugin. I did not test with the other Binance plugin.

About your Outlier warning. I use Outlier = 0; (as in the script above) to deactivate the Outlier detection. With crypto is also important to have this line in your run() function because trading is 24/7 non stop:



Finally, be very careful with Asset conf in the .csv. It's critical to have the correct values for PIP, PIPCost, and LotAmount. I have this line for MATICUSDT (Binance Futures):



You can find the base values at: https://www.binance.com/en/futures/trading-rules but you will need to calculate some of them:

LotAmount = "Min. Trade Amount". It's the second column: 1 for MATIC
PIP = "Min. Price Movement". It's the second part of the third column (Price in USDT). 0.0001 USDT for MATIC
PIPCost: Just multiply LotAmount with PIP: 1*0.0001 = 0.0001
MarginCost: It's de initial margin rate (calculated from leverage) and you can find it here: https://www.binance.com/en/support/faq/360033162192. If you want to work with 50x leverage the initial margin rate is 2%, so MarginCost = -2 in the config file.

Hope this helps.

@CpOo works well. afaik the new update fixes gap issue too

Quick question guys also new to this, but the free data here on the Zorro site is that 1minute SPY options data?:

Most prob not since it's only 150 MB in size. AFAIK, you need to pay for intraday options data at cboe.com ot other sources.

Do you know how to produce synthetic 1 min options data from the 1 min SPY data, and output to the History folder?

I don't think it's a smart idea. A small difference between real & theoretic option prices wouldn't be a problem for EOD data, but it could have a big impact on the quality of your strategy in a high frequency scenario. I don't work with these data, so I could be wrong on this.

Anyway, you could create these prices in a spreadsheet, using a Black & Scholes or Cox-Ross-Rubinstein model (more precise), save it as an CSV file and convert it by using the dataParse function (see https://manual.zorro-project.com/history.htm)