Yahoo has changed their protocol
Posted By: jcl
Yahoo has changed their protocol - 04/18/17 14:17
We got several complaints today that downloading data from Yahoo does not work anymore. The reason is that Yahoo switched their servers to a new HTTPS protocol last weekend.
We've just uploaded a new beta version, 1.56.4, which downloads Yahoo data with the new protocol. You can get it from
http://opserver.de/down/ZorrroBeta.zip. Alternatively, you can still use your old version, but download Yahoo data from Quandl, f.i. assetHistory("YAHOO/AAPL",FROM_QUANDL);
You must register at Quandl for getting a Quandl key. Also, Quandl does support Stocks, but not all ETFs in their Yahoo database.
Update: For downloading all the Z8 assets with the beta version, you can use a small script like this:
void main()
{
assetList("AssetsZ8");
while(asset(loop(Assets)))
assetHistory(Asset,FROM_YAHOO);
}
Posted By: jmb
Re: Yahoo has changed their protocol - 05/08/17 21:40
It seems that some other things have changed at IB.
The Zorro manual states (or it did last time I looked) that you need to be subscribed to "US Value Bundle PLUS" data subscription for Z8.
To make this work, I found I needed these:
- AMEX (Network B/CTA)
- NASDAQ (Network C/UTP)
I cancelled my entry for "US Value Bundle PLUS".
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/17/17 01:02
Still have some problems downloading from Yahoo.
I tested the Heatmap script built-in Zorro. I tried both V1.56 and V1.58, both of them carshed like below:
Heatmap compiling..............
Bad date format - can't parse Historyhistory.csv
0 AEIS bars read
0 AMAT bars read
More assets..
0 AMD bars read
Time 00:01:06
Error 111: Crash in script
Time 00:01:06
What did I do wrong?
By the way, does the data from Yahoo have volume information? What's the right function to get the volume info.
Thank you
Jeff
Posted By: jcl
Re: Yahoo has changed their protocol - 05/17/17 08:15
Yes, Yahoo data has volume info. Or had, at least - look into the "history.csv" with a HTML browser and you'll get the answer for the new download problems.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/17/17 09:07
IE and Chrome can't open it.
Which is the function to get volume info? Like I use priceClose(0) for the close price.
Thank you
Jeff
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/17/17 09:47
"history.csv" comes with Zorro, I didn't modify it, It should have been working.
Jeff
Posted By: jcl
Re: Yahoo has changed their protocol - 05/17/17 11:25
Had you managed to open it with your browser, you had seen this message:
Yahoo
Will be right back...
Thank you for your patience.
Our engineers are working quickly to resolve the issue.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/18/17 01:05
No, I tried both IE and Chrome, didn't work out. I asked one of my friend tried, not either. Which browser do you use?
Jeff
Posted By: Jens
Re: Yahoo has changed their protocol - 05/18/17 06:47
Jeff, you have to rename the "history.csv" to "history.html" in order to open it in Chrome or IE.
Posted By: jcl
Re: Yahoo has changed their protocol - 05/18/17 08:06
I have been informed meanwhile that it's unlikely that the "Yahoo engineers" will fix it in the next time.
We'll have to look for an alternative to Yahoo for ETF prices via API. Unfortunately, all I found so far is not free.
Posted By: Jens
Re: Yahoo has changed their protocol - 05/18/17 08:15
Posted By: jcl
Re: Yahoo has changed their protocol - 05/18/17 08:40
That's their browser button. It works only in a browser and cannot be used for automated download.
Posted By: jcl
Re: Yahoo has changed their protocol - 05/18/17 11:06
Ok, we found that you can download the same ETF data from Google. But I don't know how long, their API is also depreciated. For the moment, it works. So when you want to test or run Z8, run this script before:
void main()
{
string Code;
int N = assetList("AssetsZ8");
while(--N >= 0)
{
string URL = strf("https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",Assets[N]);
string Content = http_transfer(URL,0);
file_write("History\\history.csv",Content,0);
free(Content);
dataNew(1,0,7);
dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","History\\history.csv");
printf(" %s",Assets[N]);
dataSave(1,strf("%s.t6",Assets[N]));
}
}
Posted By: Sphin
Re: Yahoo has changed their protocol - 05/18/17 17:51
it's unlikely that the "Yahoo engineers" will fix it in the next time.
Very unlikely:
https://forums.yahoo.net/t5/Yahoo-Finance-help/Is-Yahoo-Finance-API-broken/td-p/250503/page/3:
"This feature was discontinued by the Finance team and they will not be reintroducing that functionality."
This has a massive impact, imagine only how many R analysis' work with Yahoo API ...
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/19/17 04:04
I have account and data subscription with IB, and I run your scripts Heatmap and MVO, can I simple change "assetHistory(Names[N],FROM_YAHOO)" to "assetHistory(Names[N],FROM_IB)"? Would it work?
Is your code a seperate script I should run before I run other scripts? If so, should I run it every time before I run others? or I should put it somewhere in your Heatmap and MVO script?
Can I get stocks data from Google?
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/19/17 04:28
I tried, and found this
google compiling...........
Parse Historyhistory.csv... UCTT
Parse Historyhistory.csv... UVE
Parse Historyhistory.csv... BSX
Parse Historyhistory.csv... LOW. FRME
Parse Historyhistory.csv... EQIX
Parse Historyhistory.csv... MTN
Parse Historyhistory.csv... UGI
Parse Historyhistory.csv... TREX
Error 111: Crash in script: main()
My assetList comprises all stocks.
Posted By: hatten
Re: Yahoo has changed their protocol - 05/19/17 05:40
This script for google is unfortunately not working. It yields the following:
googleDownload compiling......
Parse Historyhistroy.csv... XRT
Parse Historyhistory.csv... XLY
Bad - can't parse Historyhistory.csv XLV
Parse Historyhistory.csv... XLU XLP XLK XLI XLF
Parse Historyhistory.csv... XBI
Parse Historyhistory.csv... UPRO
Error 111: Crash in script: main()
Interesting enough, a second run does not yield the same
log:
googleDownload compiling...........
Parse Historyhistory.csv... XRT
Error 111: Crash in script: main()
Without the download functionality, the usefullness of zorro is very limited if any. What options do we have?
Posted By: jcl
Re: Yahoo has changed their protocol - 05/19/17 06:38
Dowloading from Google works for me with the default Z8 assets, but maybe not with old Zorro versions or when you have added other assets. If Google does not support the asset, the parse function returns 0 and the script crashes. Printing an error message instead is left as an exercise to the reader.
Google is no final solution since they will also discontinue the API some day. I'm afraid the final solution will possibly not be free.
-The Google download can be run before any script with Yahoo download. The script will detect that the data is already up to date. Google has also stocks, but major stocks are no problem anyway since you get them free from Quandl. The problem are ETFs.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/19/17 07:27
Thank you for your always prompt reply. Like I said, I have IB data subscription, how can I use it?
Thanks
Posted By: jcl
Re: Yahoo has changed their protocol - 05/19/17 07:36
Not directly. IB delivers only 1 year history.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/19/17 07:55
So the Quandl is the only possible way to do this by subscribing it?
Posted By: jcl
Re: Yahoo has changed their protocol - 05/19/17 08:26
Not for stocks, but for ETFs. But we'll try to find an alternative solution.
Here's a modified Google download script that does not crash anymore when an asset is not found:
void main()
{
int N = assetList("AssetsZ8");
while(--N >= 0)
{
string URL = strf("https://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",Assets[N]);
string Content = http_transfer(URL,0);
if(!Content) continue;
file_write("History\\history.csv",Content,0);
dataNew(1,0,7);
if(!dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","History\\history.csv"))
continue;
printf(" %s",Assets[N]);
dataSave(1,strf("History\\%s.t6",Assets[N]));
}
}
Posted By: hatten
Re: Yahoo has changed their protocol - 05/19/17 10:50
While the google download script now indeed no longer causes a crash, there is a large discrepancy in the test results of the Z8 system with the google data. The drawdown is more substantial (The previous test was made with the 1.58.0 beta version, whose result I fortunately still had in the log directory), AR was about 18% with leverage 2 and now it is down to 13%. Is googles data adjusted for dividends, splits etc.?
Posted By: jcl
Re: Yahoo has changed their protocol - 05/19/17 12:30
Only for splits. AFAIK dividends are not included. This affects the result substantially. Also the Google data has a lot more outliers and gaps than the Yahoo data. So this is only a temporary solution.
We will implement a final solution in the next Zorro update.
Posted By: hatten
Re: Yahoo has changed their protocol - 05/19/17 19:39
If I read their webpage correctly eoddata delivers historical end-of-the-day data for a very affordable one time payment.
Moreover it says:
"If you purchase access to 5 years historical end of day quote data then you can download all 5 years and in 3 years time there will be 8 years for you to download. Once you have purchased access, you can download your historical end of day data as many times as you like. If your PC ever crashes and you loose your data simply download it again."
So as time progress you get also access to new data.
Posted By: boatman
Re: Yahoo has changed their protocol - 05/20/17 00:58
For anyone using the quantmod library in R, Joshua Ulrich (the package's author) released a workaround that you can get by updating quantmod to the development version. In R, do either:
remotes::install_github("joshuaulrich/quantmod", ref="157_yahoo_502")
or
devtools::install_github("joshuaulrich/quantmod", ref="157_yahoo_502")
More info on the GitHub page:
https://github.com/joshuaulrich/quantmod/issues/157This works for me. I was able to successfully use quantmod::getSymbols() to query the Yahoo API.
Posted By: Sphin
Re: Yahoo has changed their protocol - 05/20/17 19:52
I search for a Yahoo API substitute also for another reason than Zorro and I stumbled upon a site, I don't know really what to think of it.
alphavantage.co offers free (you can apply a free API key) intraday, daily, weekly and monthly prices for US stock, ETFs etc. Their history e.g. of TLT goes back to 2002, of SPY back to 2001. The output format is JSON but this is to handle I guess. I did not verify the quality of their prices so far.
They declare their aim as "democratizing access to institution-grade financial analytics" but they have no address or company name on their site and even the domain name alphavantage.co is registered by an agent it seems. The IP address 216.239.36.21 of their server belongs to a Google net block. As said, I don't know how reputable they are.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/23/17 03:29
Still no solution?
Posted By: Sphin
Re: Yahoo has changed their protocol - 05/24/17 18:13
Also an elegant
solution by Ed0906. Following are code snippets of his approach. It works for now, the question is how long ...
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 05/25/17 08:18
Thank you for the info, seems there are some solutions out there. Unfortunately I am not a programmer, hope Zorro will catch up.
Jeff
Posted By: Jens
Re: Yahoo has changed their protocol - 05/25/17 15:19
This is a really nice solution and working. I tested the C# class.
Posted By: Sphin
Re: Yahoo has changed their protocol - 05/25/17 18:49
I use the php solution in my website and it's also working for now. But no one knows if and when Y will tighten its security's features. The cookie's validity of 1 year is e.g. very convenient ...
Posted By: Jens
Re: Yahoo has changed their protocol - 05/26/17 06:40
Another way is to use the .NET webbrowser control in order to load the whole website from yahoo, boerse-frankfurt etc. Then we get a HTML document object and can walk through the tables and it's rows and cells to get the quotes. This is working for me, too.
Posted By: Jens
Re: Yahoo has changed their protocol - 05/27/17 17:08
For many ETFs I get historical prices only for 2 years via the described Yahoo interface. For the same ETFs I get 5 years or more at boerse-frankfurt.de.
Posted By: vinsom
Re: Yahoo has changed their protocol - 05/27/17 23:39
I made this .Net exe from previous links to C# code
source code attached.
Put the exe in zorro folder.
Then call it with this in Zorro
var dataFromYahoo(string Code, int mode)
{
string Format;
if (mode == FROM_YAHOO | UNADJUSTED)
Format = "%Y-%m-%d,f3,f1,f2,,f4,f6";//UnadjAdj Close
else if (mode == FROM_YAHOO)
Format = "%Y-%m-%d,f3,f1,f2,f4,,f6";//Adj Close
exec("YahooFinanceAPI.exe", Code, 1);
//exec("YahooFinanceAPI.exe", "AAPL 1",1);
dataNew(1, 0, 7);
int numRecord = dataParse(1, Format, "History\history.csv");
printf("n%s %d records read from YAHOO", Code, numRecord);
dataSave(1, strf("%s.t6", Code));
return numRecord;
}
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/01/17 09:09
I made this .Net exe from previous links to C# code
source code attached.
Put the exe in zorro folder.
Then call it with this in Zorro
var dataFromYahoo(string Code, int mode)
{
string Format;
if (mode == FROM_YAHOO | UNADJUSTED)
Format = "%Y-%m-%d,f3,f1,f2,,f4,f6";//UnadjAdj Close
else if (mode == FROM_YAHOO)
Format = "%Y-%m-%d,f3,f1,f2,f4,,f6";//Adj Close
exec("YahooFinanceAPI.exe", Code, 1);
//exec("YahooFinanceAPI.exe", "AAPL 1",1);
dataNew(1, 0, 7);
int numRecord = dataParse(1, Format, "Historyhistory.csv");
printf("n%s %d records read from YAHOO", Code, numRecord);
dataSave(1, strf("%s.t6", Code));
return numRecord;
}
Thank you but I don't quite understand this.
Where is the exe code?
What's the purpose of the code here? Should I put it into the script hwere I use FROM_YAHOO?
What's the purpose of the attachment?
I am new here but I really need this working.
Jeff
Posted By: vinsom
Re: Yahoo has changed their protocol - 06/01/17 21:06
You need to compile the attached code with MS Visual studio (can't attach exe).
This will generate the exe YahooFinanceAPI.exe.
Put this in the Zorro folder.
Then get the history with the zorro code snippet above, that will call the exe
generated, with exec("YahooFinanceAPI.exe", Code, 1);.
Cheers
Posted By: konorti
Re: Yahoo has changed their protocol - 06/02/17 08:05
Sorry I dont have MS Visual Studio. Can you please upload the exe to wetransfer.com or similar site? Thanks!
Update: i tried to build with msbuild but didnt compile because of error.
Posted By: vinsom
Re: Yahoo has changed their protocol - 06/02/17 23:54
Here it is.
https://we.tl/Jfs1kuwNRJneed the .Net framework to work.
Download it from microsoft.
Regards
Posted By: konorti
Re: Yahoo has changed their protocol - 06/03/17 06:05
Thanks! I also modified the "Download" strategy to use your exe.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/07/17 06:08
Hi all,
Can I use the google script provided by jcl to download Canadian stocks history from Yahoo, I tried but got this
google compiling..............
Bad - can't parse Historyhistory.csv
Time 00:01:14
Last time I used an assetList with all US stocks, it worked.
I copied the link "https://www.google.com/finance/historical?q=AC.TO&startdate=01-Jan-2000" into the browser, actually it worked and I got the data up to date.
What's wrong here?
Thank you
Jeff
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/07/17 08:37
Thanks! I also modified the "Download" strategy to use your exe.
Hi,
I am using MVO script and an assetlist with all Canadian stocks. I tried your method to download data from YAHOO. I put the .exe to the Zorro fold and your "var dataFromYahoo(string Code, int mode)" snippet as a function into the MVO script ahead of run function, changed orignal "assetHistory(Names[N],FROM_YAHOO);" to "dataFromYahoo(Names[N],FROM_YAHOO);". When I run it, for each of the assets, it seemed that a blank window pooped up waiting for some input, but I could not type in anything. When I crossed it out, Zorro seemed to download data. But at the end, the data is all old, not updated!
What's wrong with my script?
Thank you
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/08/17 08:41
Hi vinsom,
Are you Dennis Lee on Github? Is this .exe for Yahoo's cookie and token? I have tried but did not work out. I use win10 with .net built-in.
What went wrong?
Thanks
Posted By: konorti
Re: Yahoo has changed their protocol - 06/08/17 12:14
I think pop-up empty window is fine. This works the same as original Download script that you have to write asset name and press "Load Yahoo Data" then. Try the the code snipet from Download_Y script as I modified a little. Naming of the CSV file didnt work for me in the original script, so I changed it.
Zorro should print something like this:
Download_Y compiling...........
FXCM.dll => FXCM plugin mounted
Login FXCM.. at UTC 06-08 12:10
Downloading TLT..
Parse Historyhistory.csv..
2017-06-07,125.239998,125.519997,124.790001,124.919998,124.919998,11951200
42893.00000000 125.2400 125.5200 124.7900 124.9200 11951200.0000 .
2017-06-06,125.690002,125.870003,125.400002,125.519997,125.519997,12841500
42892.00000000 125.6900 125.8700 125.4000 125.5200 12841500.0000 nTLT 3741 records read from YAHOO
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/09/17 02:25
I think pop-up empty window is fine. This works the same as original Download script that you have to write asset name and press "Load Yahoo Data" then. Try the the code snipet from Download_Y script as I modified a little. Naming of the CSV file didnt work for me in the original script, so I changed it.
Zorro should print something like this:
Download_Y compiling...........
FXCM.dll => FXCM plugin mounted
Login FXCM.. at UTC 06-08 12:10
Downloading TLT..
Parse Historyhistory.csv..
2017-06-07,125.239998,125.519997,124.790001,124.919998,124.919998,11951200
42893.00000000 125.2400 125.5200 124.7900 124.9200 11951200.0000 .
2017-06-06,125.690002,125.870003,125.400002,125.519997,125.519997,12841500
42892.00000000 125.6900 125.8700 125.4000 125.5200 12841500.0000 nTLT 3741 records read from YAHOO
Thank you very much for your reply. I tried your method in my MVO script with a Canadian stocks, I got this,
MVO compiling...........
Parse Historyhistory.csv..
8-Jun-17,155.25,155.54,154.40,154.99,21250798
39447.00000000 155.2500 155.5400 154.4000 21250798.0000 .
7-Jun-17,155.02,155.98,154.48,155.37,21069647
39082.00000000 155.0200 155.9800 154.4800 21069647.0000
AC.TO 4001 records read from YAHOO
284 Bars 2016-03-28..2017-05-11 generated
Test: MVO AC.TO 2016..2017
Assets CATEST666.csv
In my script, I set the StartDate=20160601, no EndDate, but the bars generated here is from 20160328 to 20170511. Also, the price of AC.TO is bit above 17$ but here it's 155.
Any thoughts?
Best
Jeff
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/13/17 12:29
Finally a solution is working, thank you team Zorro
Posted By: firecrest
Re: Yahoo has changed their protocol - 06/13/17 13:12
Hi Jeff1228,
Would you be able to share the solution? I am trading Z8. Thanks.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 06/14/17 02:28
Zorro releases a new version 1.59, then you use Alphavantage, then you go!
Posted By: konorti
Re: Yahoo has changed their protocol - 06/20/17 08:30
I downloaded beta version 1.59.4 but I cant download history with it.
It freezes in the message:
Downloading AGG..
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/20/17 12:35
Hi,
Can Zorro team improve a script for downloading data from
https://stooq.com/it canbe useful instead using Yahoo or else ?
Posted By: firecrest
Re: Yahoo has changed their protocol - 06/20/17 12:53
I downloaded beta version 1.59.4 but I cant download history with it.
It freezes in the message:
Downloading AGG..
Maybe you need to log onto IB before downloading.
Posted By: konorti
Re: Yahoo has changed their protocol - 06/21/17 07:30
So far we didnt need to have IB account to download from yahoo. I didnt change my fxcm login sttings and was able to download from yahoo.Did this change? Can somebody confirm? Can somebody confirm that beta version was also fixed not just normal version (i cant install normal version as my company laptop thinks it is a virus...)
Posted By: GreenBoat
Re: Yahoo has changed their protocol - 06/22/17 08:28
Yahoo download doesn't work anymore, that's what this thread is all about.
Posted By: jcl
Re: Yahoo has changed their protocol - 06/22/17 10:24
Beta 1.59.4 had a bug with AV download. Use 1.59.5.
Posted By: konorti
Re: Yahoo has changed their protocol - 06/22/17 13:09
Thanks!
Yesterday I also wrote a code which easily downloads fromm Google, only Assets names should be enterred into the code. Enough to "test" the code. For me 1 minute was enough to download 80 assets' history:
int download (string ass)
{
string Code = ass;
string URL = strf("http://www.google.com/finance/historical?q=%s&startdate=01-Jan-2000&output=csv",Code);
string Content = http_transfer(URL,0);
file_write("Historyhistory.csv",Content,0);
dataNew(1,0,7);
if(dataParse(1,"%d-%b-%y,f3,f1,f2,f4,f6","Historyhistory.csv"))
dataSave(1,strf("History%s.t6",Code));
}
void main()
{
while(loop( // Here you can list all assets
"AGG",
"ALFA",
"AMZA",
// ....
))
download(Loop1);
}
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/22/17 20:16
I have a question : how to download data from alpha vantage
I've tried this code but it says : {
"Error Message": "the parameter apikey is invalid or missing. Please claim your free API key on (
https://www.alphavantage.co/support/#api-key).
function run()
{
NumYears=5;
assetHistory("MSFT",FROM_AV);
}
THANKS
Posted By: firecrest
Re: Yahoo has changed their protocol - 06/23/17 03:48
I have a question : how to download data from alpha vantage
I've tried this code but it says : {
"Error Message": "the parameter apikey is invalid or missing. Please claim your free API key on (
https://www.alphavantage.co/support/#api-key).
function run()
{
NumYears=5;
assetHistory("MSFT",FROM_AV);
}
THANKS
I think you need to apply for the key from AV and set it up in the .ini in Zorro folder.
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/23/17 10:15
THANKS for your help, it's ok for AV
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/23/17 12:13
I have an another question, for example, I've tried to use the SP500 data from Yahoo to trade on SPX500 in FXCM. It's worked in backtest but when I try to use the script in live : it says :
no asset data for ^GSPC
the script is :
function run()
{
StartDate = 20100101;
EndDate = 20171231;
BarPeriod = 60*24; // 2 hour
LookBack = 600;
set(PRELOAD|LOGFILE|PLOTNOW);
assetHistory("^GSPC",FROM_AV);
asset("^GSPC");
PlotHeight1 = 800; // height of the chart with results
PlotWidth = 2000;
/////////////////////////////////
vars Price = series(priceClose());
vars high=series(priceHigh());
vars low=series(priceLow());
vars zerolag=series(KAMA(Price,10));
vars fastema = series(EMA(Price,50));
vars slowema = series(EMA(Price,80));
asset("SPX500");
if(fastema[0]>slowema[0])
{
enterLong();
}
if(fastema[0]<slowema[0])
{
exitLong();
}
}
there is a solution to solve the problem, to use R but this solution takes more time, more memory so it's no efficient solution.
May be you have a suggestion.
Thanks
Posted By: konorti
Re: Yahoo has changed their protocol - 06/24/17 19:59
Sorry maybe this is a dumb question. Why not download history from FXCM?
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/26/17 17:15
iF I use Yahoo (SP500 DAILY DATA) in my backtest, I obtain these résultats :
strategy compiling..........
Test: strategy ^GSPC 2008..2018
Monte Carlo Analysis... Median AR 9%
Profit 4651$ MI 67$ DD 672$ Capital 841$
Trades 746 Win 76.8% Avg +757.5p Bars 2
AR 9% PF 2.58 SR 1.07 UI 3% R2 0.98
Chart... ok
Copied to Clipboard!
If I use fxcm data (sp500 daily data ) , I obtain these results
strategy compiling...........
Test: strategy SPX500 2008..2018
Monte Carlo Analysis... Median AR 42%
Profit 1850$ MI 25$ DD 694$ Capital 592$
Trades 620 Win 70.8% Avg +34.3p Bars 3
AR 50% PF 1.51 SR 0.51 UI 10% R2 0.87
It's clear that there a difference. In fact, I use open, high low and close data from Yahoo to do my backtest. These data are different from fxcm. That's why, I want use daily data for live trading in daily mode
Posted By: konorti
Re: Yahoo has changed their protocol - 06/29/17 20:38
Did you see any clear difference in trades in file: Logtesttrades.csv?
I would test it also visually during only last months (to see trades on bars) what is difference between them.
And maybe checking the history with file ZHistoryEditor.exe and trades from log file...
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/29/17 20:51
I haven't check the history but i think that the difference of performance is possibly caused by the nature of data, specially the high , low and close, and orders are not executed at the same moment,
Posted By: konorti
Re: Yahoo has changed their protocol - 06/29/17 21:13
Did you enter into AssetsFix.csv the line:
^GSPC,50,0.1,0,0,0.01,0.01,0,2,1,0.02,
Posted By: stephane97490
Re: Yahoo has changed their protocol - 06/30/17 15:56
I've changed like your recommandation : it returns
Error 053: ^GSPC unavailable at this time
Load NAS100 prices.. 5100 h0.00 0.00 0.00
Trade: model2 2017-06-30
Lookback period (145 bars)
Error 030: asset ^GSPC missing in INITRUN
Error 011: Function KAMA called with invalid parameters (no price)
Error 011: Function EMA called with invalid parameters (no price)
the script is :
function run()
{
NumYears=15;
StartDate = 20100101;
EndDate = 20171231;
BarPeriod = 60*24;
LookBack = 145;
set(PRELOAD);
//asset("^GSPC");
//if(is(INITRUN))
assetHistory("^GSPC",FROM_AV);
asset("^GSPC");
vars Price = series(priceClose());
vars high=series(priceHigh());
vars low=series(priceLow());
vars zerolag=series(KAMA(Price,10));
vars fastema = series(EMA(Price,50));
vars slowema = series(EMA(Price,80));
/////////////////////////////////
vars slowema = series(EMA(Price,80));
asset("NAS100");
if(fastema[0]>slowema[0])
{
enterLong();
}
if(fastema[0]<slowema[0])
{
exitLong();
}
printf("%.2f ",zerolag[0]);
printf("%.2f ",fastema[0]);
printf("%.2f ",slowema[0]);
}
Posted By: konorti
Re: Yahoo has changed their protocol - 06/30/17 18:05
Next week i will try to replicate your code to google history as i dont have zorro paid version to access AV history to see what is the problem.
Posted By: konorti
Re: Yahoo has changed their protocol - 07/04/17 07:16
Sorry I tried, it is the same problem for me. There is a way I think to import csv as data, and use as data and not as price history. I havent done that so I cant help.
Posted By: Jeff1228
Re: Yahoo has changed their protocol - 07/04/17 08:36
I have been using AV for a while, I found the latest data (EOD) I could download from AV has been the data 2 trading days ago, not like other sites just a few hours after the trading hour is over I can get the latest EOD data. Has anyne found the same? What's wrong here?
Thank you
Posted By: stephane97490
Re: Yahoo has changed their protocol - 07/06/17 09:16
thanks
Posted By: stephane97490
Re: Yahoo has changed their protocol - 07/06/17 09:17
I've noticed that AV depends on yahoo data, so when yahoo isn't working, AV doesn't deliver or updates their data
Posted By: hatten
Re: Yahoo has changed their protocol - 08/17/17 20:59
It has been mentioned in the past that data downloading through zorro from AV is slow because of the JSON format. According to their update website however the API now also allows to download csv files instead of JSON.
Posted By: mhdus
Re: Yahoo has changed their protocol - 11/13/17 16:22
Barchart has launched an alternative:
https://www.barchart.com/solutions/press...nance-api-users@jcl, could you consider this for integrating into the Zorro data source options?
Posted By: Hredot
Re: Yahoo has changed their protocol - 11/13/17 16:31
@mhdus
I might be wrong, but isn't 2 years of daily history a ridiculously small time frame which makes this service essentially useless?
Posted By: mhdus
Re: Yahoo has changed their protocol - 11/13/17 16:43
Ok, I overlooked this...
Then maybe not really an alternative but rather an interesting additonal source as it has intraday data and futures data.
Posted By: pascalx
Re: Yahoo has changed their protocol - 11/13/17 18:26
"2 years of daily history"
"3 months of n-minute history"
Does not look very useful indeed.
Posted By: LorraineJones
Re: Yahoo has changed their protocol - 03/14/21 21:39
I've been using
this one for the last few months and you can get the last 10 years of history.