Hi,

I have developed a plugin to use IG with Zorro, and today I'm releasing the initial version. You can find instructions on how to install here:
https://github.com/dan-lind/igzplugin

Please let me now of any issues you encounter, problems with the documentation, or other questions that you might have. I'll answer as fast as I can :-)

As pointed out, this is the first release. Don't expect it to be free of bugs just yet :-)

Regards,
Daniel

Thank you Dalla, nice job!

Hello,
I'm tryng to install the plugin.
I have followed the istruction
https://github.com/dan-lind/igzplugin

but at the and of the procedure the plugin does not compare in zorro's menu list "Account"

For sure I have done some mistakes

I have a doubt about point 2 for the second part :
"2.) Add *${yourJREinstallPath}jrebin* and ${yourJREinstallPath}jrebincom.iggroup.api.client to the front of your Path environment variable(here is a howto)."

I suppose that I should have this file
"com.iggroup.api.client"
in the directory "${yourJREinstallPath}jrebin" but I haven't this file ....
Maybe the problem is this?

another question:
insise the zip file "igplugin.zip" there is the file"igplugin-0.1.jar" I must to launch before zorro? I tried but I failed


PS
as JRE I have
C:Program FilesJavajre1.8.0_131bin

I hope someone can help me

Thanks

Hi

Thanks for the feedback. There was a typo in step 2 of the instructions, I have updated them now. Please check again. Also please download the zip-file again, I'm not sure I had the correct compiler options the first time around, but I have verified that it's correct this time around.

Also, you shouldn't start the jar-file yourself, Zorro will do that by itself.

/Daniel

Hi Daniel
thanks a lot!!
you are very kind .
I hope in the future that I will be able to return the favor....

With the new zip file and the new PATH , now its Ok!! :-)
The plugin is visible in zorro's menu Account

Now all the market are closed so I can't test anymore...
I will try tommorrow (if some market will be open)

I have another doubt about how I have to change the assetFix

For example for EUR/USD I supposed that I have to set the two variables like this

Name= EUR/USD
Symbol= CS.D.EURUSD.CFD.IP

or as an alternative
Name= CS.D.EURUSD.CFD.IP
Symbol= EUR/USD

(of course there are the other columns in the AssetsFix.csv)

Great to hear!

The first alternative is correct. Name is the name that will be displayed in zorros asset drop down, so that can be whatever you want it to be, as long as you set symbol to be the value expected by the broker.

GoodMorning Daniel
I m tring now and it's OK
I used the zorro's script "trade test" and I have bought and the I have sold one lot
In the same time I was checking in the IG's web site (my private area) and the position and the order are the same that I have commanded with zorro :-)

Next step will be launch a z script :-)

in the istruction
https://github.com/dan-lind/igzplugin
you write:
"Start Zorro and check if the Account drop-down-box shows IG as an available broker. Enter your login details, pick a script of your choice and press Trade."

maybe it's more clear if you specify that the "Login details" is not the first IG' account but that created for the companion API
praticaly the same that peoplet have to use

https://labs.ig.com/sample-apps/api-companion/index.html


post scriptum
Sorry for my English, I suppose that there are many errors but I hope that The concept is understandable (also with the help of google translate :-) )

One Info for all
I have a sever in EC2 Amazon with "windows server 2012"
To configure the Environment Variables "Path" I follow the istruction

https://lsdbtech.wordpress.com/2015/05/28/windows-server-2012-r2-change-environment-variables/

Great to know!

However I'm not sure what you mean when you say "that the "Login details" is not the first IG' account but that created for the companion API praticaly the same that peoplet have to use "

What to you mean with "first IG account" ? It almost sounds like you created a separate account just to create the api key?

Hello
I mean : "first IG account" is the first account created when I opened the account with IG
I use it here
https://www.ig.com/it/login
to enter in the private area of the web site

Then I created API Key and (both for real and demo account)

Later
Following the istruction here "http://labs.ig.com/gettingstarted"

Getting started
To begin using our API all you need to do is:
Create an account
Generate your API key
Start coding

I create a new account (I mean , ID and passowrd) to use
in this link
http://labs.ig.com/sample-apps/api-companion/index.html
Buy this second account has only this functionality to check API request like to find the IG'S symbol of EUR/USD

Daniel
are you using zorro with z script?

I m using z7 it 's work but I think it isn't ok to 100%
indeed today Zorro have stil in on the market with a currency, Instead in the ig platforms I was FLTAT

I think there are strange Behaviors and some messages appear
"!Severe error occured, check igzplugin.log logfile!"

I am carrying on to try zorro's z7 (in demo of course) :-)

Hi

You dont need to create a second account if you dont want to, you can create an api key on your original account.

What does the igplugin.log say? Check in the ig folder under the plugin directory of zorro

I don't trade any z systems myself at this point.

Hi Daniel

Today the zorro's Behavior seems to be correct
But there are still the message error in zorro'log

I have attached the log in this message

In the files log I have replaced my username with "XXXXX" and my key with "YYYYYY"

in the zorro's file log "Z7demo.log" the last message that I founded:

"[Wed 03.05.17 17:03] 30175 +106 +22 /// 1.0922/1.09241.0915/1.0916
!Severe error occured, check igzplugin.log logfile!
[Wed 03.05.17 18:03] 30171 +106 +19 /// 1.0916/1.09281.0904/1.0914"

and in file "igzplugin.log" I founded:


2017-05-03 17:29:22.124 [ThreadPoolTaskScheduler4] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when refreshing session token
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://demo-api.ig.com/gateway/deal/session/refresh-token": demo-api.ig.com:443 failed to respond; nested exception is org.apache.http.NoHttpResponseException: demo-api.ig.com:443 failed to respond
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:666)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
at com.danlind.igz.ig.api.client.RestAPI.refreshSessionV1(RestAPI.java:758)
at com.danlind.igz.adapter.RestApiAdapter.refreshSessionV1(RestApiAdapter.java:282)
at com.danlind.igz.brokerapi.BrokerLogin.refreshAccessToken(BrokerLogin.java:82)
at com.danlind.igz.brokerapi.BrokerLogin.lambda$startRefreshAccessTokenScheduler$12(BrokerLogin.java:88)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.http.NoHttpResponseException: demo-api.ig.com:443 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:89)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:652)
... 14 common frames omitted
2017-05-03 17:29:22.128 [ThreadPoolTaskScheduler4] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.RuntimeException: null
at com.danlind.igz.adapter.RestApiAdapter.refreshSessionV1(RestApiAdapter.java:290)
at com.danlind.igz.brokerapi.BrokerLogin.refreshAccessToken(BrokerLogin.java:82)
at com.danlind.igz.brokerapi.BrokerLogin.lambda$startRefreshAccessTokenScheduler$12(BrokerLogin.java:88)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

That's weird. Seems to be happening almost exactly every two hours, at least of you look at the last three occurrences in your log. But it doesn't seem to affect trading as a long position was opened after one of the failures? Please let me know if this is not the case.

I'll try to look more into this issue later this week.

Hi Daniel
I confitm that it doesn't seem to affect trading as a long position was opened after one of the failure

If could help you I saw that you can the report in real time here
http://54.93.127.181/z7.htm

Good Night

Thanks for confirming.

I have verified the issue locally as well now.
I think I may have found a fix as well, but I want to run some more tests locally before preparing a new release.

/Daniel

Quick update: I think I have fixed the issue which caused the errors you are seeing. However I also found another potential issue that I fixed. Since the markets are closed I can't verify the fix right now, so I'll try to confirm and hopefully release an update on Monday.

Hi Daniel
Great Job! :-)
When you will released an update me too I will try the news release

Thanks

Have a good sunday

Hi,

A new version has been released that should hopefully make connections more stable.
Download it here: https://github.com/dan-lind/igzplugin/releases/tag/v0.12

/Daniel

Hi Daniel
I Have installed the new versions
In the next days I will report the outcome of the test

Thanks

Hi Daniel
there are still some errors
Maybe it seems there are less errors

The log is attached

Hi

These errors suggests that there were problems connecting to the demo API, i.e. the server is not responding at all or not responding in a time. I saw that too when running my own tests. It's not that worrying as I think the demo server is also used for development purposes for IG, that is they will sometimes restart it to push new releases etc to the demo environment.

If you plan to run additional tests, could you please delete your log-file before starting Zorro? It makes it easier for me to determine exactly what is relevant and not in the log.

Thanks for your help :-)

@kmerlo, wanted to check if you ran any more tests? Any new issues?

Hi Dalla
I still using the api, but sorry in this day I have some other commitments. I hope to be more precise in the next days :-)

Menatime I have detected some minor strange Behaviour, maybe tree....

For the first now I don't remember exactely (I ' sorry), but should be that in zorro the position was flat instead in IG the position still open...

the second : in zorro
System State: 30414 +35 +0
instead in IG the account is 30427
I have stoped Zorro and then I have restarted it, but the money still different

third:
some message errors during the week end, so I changed in file the Z.INI the variable Weekend from "2" to "3"
I will check the next week end if the modify is usefull:-)

I understand, just curious to hear any results :-)

The most obvious issue is that there are still a lot of instances where IGs servers does not seem to be responding at all. I'll see if I can confirm if there are some times where at least the demo server is expected to be down.

There are also some issues related to exceeding the api key limits, which is a bit more of an issue. How many assets is this system using?

Hi daniel,
I am testing only z7, so only 3 assets

Hmm, that´s strange. Did you at any time restart the strategy?
3 assets shouldn't be enough to hit the limit unless you restart many times over. Or if there is a bug of course

hi
I think to have restar the strategy 4 o 5 time.

today there ara this error message


2017-05-16 15:36:59.613 [ThreadPoolTaskScheduler1] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when refreshing session token
org.springframework.web.client.HttpServerErrorException: 503 Service Unavailable
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
at com.danlind.igz.ig.api.client.RestAPI.refreshSessionV1(RestAPI.java:758)
at com.danlind.igz.adapter.RestApiAdapter.refreshSessionV1(RestApiAdapter.java:275)
at com.danlind.igz.brokerapi.BrokerLogin.refreshAccessToken(BrokerLogin.java:82)
at com.danlind.igz.brokerapi.BrokerLogin.lambda$startRefreshAccessTokenScheduler$13(BrokerLogin.java:92)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2017-05-16 15:36:59.632 [ThreadPoolTaskScheduler1] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task.
java.lang.RuntimeException: 503 Service Unavailable
at com.danlind.igz.adapter.RestApiAdapter.refreshSessionV1(RestApiAdapter.java:287)
at com.danlind.igz.brokerapi.BrokerLogin.refreshAccessToken(BrokerLogin.java:82)
at com.danlind.igz.brokerapi.BrokerLogin.lambda$startRefreshAccessTokenScheduler$13(BrokerLogin.java:92)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Same as before then.
The part "503 Service Unavailable" means that the server is not responding.

I Daniel
today I tried z12

http://54.93.127.181/z12.htm

when I have launched the script I have received this message

"
Error 030: asset USD/JPY missing in INITRUN
Error 011: Function ATR called with invalid parameters (no price)
Error 011: Function BBANDS called with invalid parameters (no price).
"

plus other messages

But do you use the API and all it's fine for you o also you have some strange error messages?


PS
later the script has open some position so, it was working but I can't know if se the first message cause that some trade will done badly... :-(

Hi

I will check the details later, but it looks like you haven't set up the symbol name for USD/JPY in the AssetsFix file. I'm doing testing also and have fixed a couple of issues. Hoping to push a new release this week.

You have a character in your AssetsFix for USD/JPY has an extra character at the end that you need to remove. So change

CS.D.USDJPY.MINI.IP"
to
CS.D.USDJPY.MINI.IP

I done a stupid error

I found an other similar error
IX.D.NASDAQ.IFM.IP"

I correct the two errors

But now I have an other stange error
Error 053: CS.D.GBPUSD.IP unavailable at this time

But with the same "AssetFix.csv" when I test the z7 script I haven't this error... :-(

and I have only "z.ini" I don't make the fiel z7.ini or z12.ini

Thanks

Roberto

In the file log "igzplugin.log" I found for other symbol this messagge
.....error was {"errorCode":"error.public-api.exceeded-api-key-allowance"}....

Unfortunately the default API limits are quite low for IG, and the error message is telling you that you have exceeded your limit.

Try to raise this setting in application.properties (same directory as the igzplugin.log)

Change
plugin.refreshMarketDataInterval=30000
to a higher value, like 45000, 60000, or even 70000.

This will reduce the number of requests per minute.

You could probably extend the quotas, but you would have to contact IG support for that, and I'm not sure it would be free anymore.

I have release v0.13 of the plugin. Some of the errors related to not finding order confirmations should be gone now.

I Daniel
in your site "https://github.com/dan-lind/igzplugin/releases" I find only the source code file
The last time you made the executable file too
Are you planning to put the executable file also this time ?
If not, now how I can to compile the source code?

Thanks

Bye

R

There was a problem with the uoload apparently. The zip-file is there now.

I suspect though that you might still have issues with Z12?
Let me know how it goes.

One thing to try if you still get {"errorCode":"error.public-api.exceeded-api-key-allowance"} when starting Z12 (or other systems for that matter).

Try setting
Preload = 1 for the Z-system.
Explained how here under "Setup & Trading":
http://www.zorro-trader.com/manual/en/zsystems.htm

Hi Daniel
I still have problems
Here the log for z7

NB
I am using zorro 1.58

Hi,

Are you running multiple strategies in parallell with Z7?
There are two errors here:
error.public-api.exceeded-account-historical-data-allowance
and
error.public-api.exceeded-api-key-allowance

The first is saying that you are trying to download more ticks than you are allowed each week (default 10000 per week).

The second is same as before, too many API calls during 1 minute.
Since this shows up directly when you start this strategy, it must mean you have something else running that is already consuming your API quota.

Another release has been published, small fix for timing issues when confirming orders.

@kmerlo, still issues running Z systems?

Hi Daniel
no I had some issues with my VPN server after to be installed zorro 1.58
I could not fix it because this week I'am in holiday and here I haven't wire line internet.
In the next days I will try the new release

PS
It is very strange that last time I had consumed my API quota because for one week I had not use the IGplugin but only the MT4plugin... :-(

Hi. I've trying to trade Sprint markets. Previously I've been doign this via the MT4 bridge by making requests to the IG API through there. Problem is, I need to trade using bars of 15s and 30s, and can't do this via the MT4 bridge. So I was thinking of grabbing the bars via another bridge, say FXCM, and then just makign the http requests from within Zorro. But I've had no luck with getting POST to work to login to IG yet.

Any ideas on how to accomplish this? (I doubt the data allowances from the IG API are sufficient).

Hi

I'll help if I can of course, however I'd like to keep this thread as a discussion for the plugin only. Please PM me or start a new thread, and I'll try to answer there.

Regards
Daniel

Hi MaskofZorro,

You can download up to 10 years of free data using FXCM's Basic Historical Data Downloader https://www.fxcmapps.com/apps/basic-historical-data-downloader/ . Hope it helps with what you're looking for.

Jason

Hi Daniel

unlucky I still have issues running Z systems
I use zorro 1.58 and the new igplugin

I tested z7
Zorro stalled (goes in freeze) :-(

It looks to me like you are not running the latest plugin, can you please double check and also make sure you restarted Zorro after updating the plugin?
These errors
404 Not Found
happened frequently in an earlier version of the plugin.

You mentioned Zorro stalled, did this happen during the weekend?
This has happened to me too.
Try setting Weekend = 7 in the Z.ini file

yeh during the week end
Ok I will retry with : Weekend = 7 in the Z.ini

I saw also that there seems to be issues with some of the orders, most likely related to a rounding error. Will try to look into that during the week.

Hi Daniel
Zorro this morning was blocked
I closed it now
I will use MT4 Bridge for understanding if the problem is IG's Plugin o some configuration in my VPN server...

I'm currently running Z7 on an EC2 instance, will check if this happens to me as well.

A new version of the IG plugin has been release.
Most importantly this release fixes an issue where Zorro would sometimes block/freeze after not getting any return values from the plugin

https://github.com/dan-lind/igzplugin/releases/tag/v0.2

Thanks for your efforts in creating this plugin Daniel! I have not tested it deeply yet and also just on a demo account, but the basic functions from the TradeTest script work nicely (almost, see below*).
It's quite unfortunate that IG limits the non-trading requests to 30 per minute/account. With this limit I can only update 8 assets (e.g. with the Download script, saving all assets to AssetsNew.csv) before hitting the limit. I've asked for an increased limit, let's see if they comply with my request.

* When opening a position on a currency pair with Hedge=4, Zorro freezes. Other symbols work fine, see the logs below:

EUR/USD Mini, Hedge=4, Buy Long from TradeTest.c --> freeze


Germany 30 Cash 1-EUR, Hedge=4, Buy Long from TradeTest.c --> works

I assume you downloaded the latest version? :-)

It should not freeze of course, but your issue is that the API is complaining about your stop loss. Unfortunately the documentation of the API is not very helpful here, but your stop loss looks very big (0.5 will translate to 5000 pips). Does it work if you make it smaller?

Actually, I can't reproduce this with the latest version. Make sure you have the latest release, and restart Zorro after installing it.

Hello Dalla,
thanks for your work
I have a doubt, when there is a new release of the plugin I have to replace all file or only ig.dll ?

I've decided to change the way you test the plugin, not using z script but with my script or the script tutorial's

I will keep you updated when I will have some results

Have a a good time

Most important is to replace the ig.dll and the igplugin-x.x.jar .
And to restart Zorro to reload the new plugin.
Sometimes there can also be new entries in the application.properties, but I'll make sure to mention that specifically in the release notes going forward.

My bad, I checked for a new version yesterday in the morning and didn't read or see your post (had the forum tab opened since morning). It works fine now, no more freeze and the position is opened

Good to know, this release was pretty much supposed to fix the exact issue you described. You got me worried there for a second ;-)

Hi,
I followed the instructions and could successfully connect via the IG bridge. I did also successfully open and close a trade using the Zorro TradeTest script. I'm using the latest Zorro Beta 1.59.91 and the version of the IG plugin included there.
I'm just wondering why I'm constantly getting the message "!Multiple accounts are not yet supported!" in the Zorro message window? And my real account equity is not shown, it just shows 0. Maybe a related issue?
Assuming the error relates to the fact that I do have 3 live accounts with IG: the default CFD Trading account, a CFD MetaTrader account (so far used for trading with Zorro and the MT bridge) and a stocks trading account. I had to set up the MT account manually, the other 2 were automatically set up after registering with IG (in Germany). According to the logs the plugin is connecting to the intended account (the default CFD Trading account which is funded).
Can I just ignore the error message? Any chance to have the account equity shown correctly? I could not find anything related in the log, not even at debug level...

Hi,

The equity not showing is definitely related to the error message you are seeing. However I have no idea what causes it... I also have 3 live accounts with IG, Two for CFD (SEK & EUR) and an MT4 account. I have never seen this error message. It has to be some Zorro configuration I think. I'll try to get an answer from Zorro support and get back with an answer.

/Daniel

Hi,

I've released a new version which should fix this issue.
https://github.com/dan-lind/igzplugin/releases/tag/v0.15

Can you please download and see if this helps?
Instructions on how to install can be found here:
https://github.com/dan-lind/igzplugin

Hi Daniel,
my apologies, I had no time to check back here until now - many thanks for your fast response and the fix! I will test it tomorrow during the day and let you know...
Thanks again,
Markus

Hi Daniel,
just installed and tested: both problems fixed, no more error messages and the account equity is also shown correctly, great!
Many thanks
Markus

Great to hear!

Daniel

I tried to run a Z system (tried Z12 and Z7) via the IG Plugin but quickly ran into the quota exceeded for historical data problem ... I guess it happened because I forgot to set Preload = 1 in Z.ini :-(
IG were so kind to reset the quota once but now I'd like to be 100% sure not to run into the same issue again.
So has anyone managed to successfully run one or more of the Z strategies via the IG Plugin and could provide some hints how to make sure there is no issue with the IG API quotas?
(By the way, enhancement idea for the IG plugin: add a configurable threshold for historical data requests per strategy launch...)
Many thanks in advance.

Additional comment: it seems as if one can set up several API keys per IG account (one per "application") and the quota may be applied per key. I had tried again with a fresh key and Preload = 1 in Z.ini but still ran into the problem (not sure if that just happened because the quota was already exceeded by the first key, though).

Hi

This is unfortunately the biggest issue with the IG API.
Like mentioned in the installation instructions, the best way to avoid these kind of issues is to keep your historical data up to date, and use the PRELOAD flag.

I currently tend to download historical data from FXCM/Oanda before starting strategies. I know this is not optimal, but doing so will avoid the problems you are describing.
Doing this allowes me to start both Z12 and Z7.

Adding a threshold for data download would not really help; Zorro expects to get the data it is asking for, and if the plugin doesn't provide, it won't start the strategy.

Using multiple API keys might help for number of requests, but as far as I understand it will not increase the 10000 data point limit for historical data.

A threshold would still help as it is much better to see a strategy not starting (and a message such as "warning: attempting to load too much historical data, check your local data and configuration") than just exceeding the quota limits and having to wait for one week before trying again... Ok, maybe not the highest-priority enhancement but not useless either. In fact the API should be doing that.
Anyway, I will try updating my local data from other sources and then give it another shot.

I agree with this suggestion
Even myself once for error I finished the ig's quote..

PS for DALLA
in the next days I'll try the new release and I'll post the feedback
Thanks for all!

I updated all my history files via FXCM and tried starting Z7 and Z12 using the IG plugin. Deliberately on Sunday to be sure I am not missing any latest price data. I did also check the _2017.t6 files for completeness using ZHistoryEditor. As expected, data is included up to last Friday night and no gaps except the week-ends.

Z7 started without errors with the following Zorro messages:

Z7 .x ..
ig.dll => IG plugin mounted
Z7.55: PH PR M 100 H 5 W 2 V 2 .
Login IG.. at UTC 07-16 13:16.
Load EUR/USD prices.. 680+33 h, gap 41 h
Load USD/JPY prices.. 680+33 h, gap 41 h
Load GBP/USD prices.. 680+33 h, gap 41 h

Trade: Z7 2017-07-16
Lookback period (480 bars) .
Read Z7.c

I understand "gap 41h" is due to the weekend gap, so no issue. However, I don't understand where the +33h is coming from? 3 x 33h x 60 min/h = 5940 historical bars consumed ... :-(

Hoping IG will reset the quota at the beginning of the calendar week I tried starting Z12, too:

Z12
Login IG... at UTC 07-16 13:17
Z12.54: PH PR M 100 H 5 W 2 V 2
Load EUR/USD prices.. 2800+211 h, gap 41 h
Load USD/CHF prices.. 2800+211 h, gap 41 h
!Severe error occured, check igzplugin.log logfile!
!Severe error occured, check igzplugin.log logfile!
Error 053: CX.D.GBPUSD.CASH.IP unavailable at this time
!Severe error occured, check igzplugin.log logfile!

[...]

!Severe error occured, check igzplugin.log logfile!
Error 053: CX.D.XAGUSD.CASH.IP unavailable at this time
V 1.600 on Sun 17-07-16 15:17:06 (Zorro S subscription)

Trade: Z12 2017-07-16
Read Z12.fac
Read Z12.par
Lookback period (500 bars)
[184: Thu 17-03-16 16:09] 1.0722/1.07441.0713/1.0736
Error 033: GBP/USD missing in INITRUN
Error 011: Function ATR called with invalid parameters (no price)
Error 011: Function BBANDS called with invalid parameters (no price).
[185: Thu 17-03-16 20:09] 1.0736/1.07711.0708/1.0768

Of course, the igzplugin log shows:
ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when getting historic prices for epic CX.D.GBPUSD.CASH.IP, error was {"errorCode":"error.public-api.exceeded-api-key-allowance"

Again, I wonder where the +211h is coming from? The data files were up-to-date ...

Obviously PRELOAD was set and recognized. So what else was wrong?
Any help much appreciated. Many thanks in advance!

From the manual

"Loading AUD/USD prices... 910+111 h

The PRELOAD flag was set and historic price data was available. Zorro read 910 hours price data from its history files and downloaded additional 111 hours price history from the broker's server. For portfolio strategies the number of downloaded hours can differ from asset to asset."

These messages however are generated by Zorro. You say that your files are up to date, but it looks like zorro disagrees. Also, are you running with BarPeriod 1? If not, then the historical bars consumed is probably not as bad as you think.

I had seen this in the manual. And I understand your conclusion "zorro disagrees". However, it does not fit my previous checks of the data. Eventually the handling of (no) data during the week-end is not as I would expect (see the messages in the log below regarding EUR/USD data for this week-end).

Thanks for the hint regarding BarPeriod - indeed there should be much less bars consumed (hourly bars for Z7, 4h bars for Z12). Which even more confuses my why the quota is exceeded so quickly... Is it possible to debug what (amount of) data is requested from the API by the IG plugin?

What I realized later, after my previous post: the additional hours of data loaded are different, depending on which broker/plugin is used. Here are 2 logs from Z7 start (using Verbose = 3), first FXCM then IG immediately after:

Z7
Login FXCM.. at UTC 07-16 20:33
Z7.55: PH PR M 100 H 5 W 3 V 3
Load AssetsFix
!EUR/USD 07-14 21:13:00 to 07-16 20:33:42 failed
Preload EUR/USD 40800 min, gap 48 h
663 Bars 2017-06-06..2017-07-14 generated
EUR/USD: 0..49565, 2017-06-06..1899-12-30
Preload USD/JPY 40800 min, gap 48 h
USD/JPY: 0..49565, 2017-06-06..1899-12-30
Preload GBP/USD 40800 min, gap 48 h
GBP/USD: 0..49565, 2017-06-06..1899-12-30
V 1.600 on Sun 17-07-16 22:33:49 (Zorro S subscription)

Trade: Z7 2017-07-16
Lookback period (480 bars)
[182: Fri 17-06-16 08:03] 1.1155/1.11691.1152/1.1169

[...]

Session end at 20:33
Logout.. ok



Z7
Login IG... at UTC 07-16 20:34
Z7.55: PH PR M 100 H 5 W 3 V 3
Load AssetsIG
!CX.D.EURUSD.CASH.IP 07-14 21:13:00 to 07-16 20:34:34
!=> 07-13 18:45:00 to 20:30:00, 108 ticks
Load EUR/USD prices.. 680+27 h
663 Bars 2017-06-06..2017-07-14 generated
EUR/USD: 0..49673, 2017-06-06..1899-12-30
Load USD/JPY prices.. 680+26 h
USD/JPY: 0..49673, 2017-06-06..1899-12-30
Load GBP/USD prices.. 680+26 h
GBP/USD: 0..49673, 2017-06-06..1899-12-30
V 1.600 on Sun 17-07-16 22:34:36 (Zorro S subscription)

Trade: Z7 2017-07-16
Lookback period (480 bars)
[182: Fri 17-06-16 08:02] 1.1156/1.11681.1152/1.1168

[...]

Session end at 20:34
Logout.. ok


Based on the above message "07-13 18:45:00 to 20:30:00, 108 ticks" from the IG plugin I checked my local EURUSD_2017.t6 file once again: the data during this timeframe is included! So I wonder what the plugin was doing about these 108 ticks and why?

I just did the same test with Z12 now after 11PM (week-end is over) and have the same effect: with FXCM Zorro reports 2800 +1h for all prices, with IG it reports 2800 +205h for the first 5 price series, then the quota exceeded issue kicks in again:

2017-07-16 23:06:03.601 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.EURUSD.CASH.IP
2017-07-16 23:06:03.604 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.EURUSD.CASH.IP
2017-07-16 23:06:04.419 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDCHF.CASH.IP
2017-07-16 23:06:04.435 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDCHF.CASH.IP
2017-07-16 23:06:05.346 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.GBPUSD.CASH.IP
2017-07-16 23:06:05.347 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.GBPUSD.CASH.IP
2017-07-16 23:06:06.139 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDCAD.CASH.IP
2017-07-16 23:06:06.140 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDCAD.CASH.IP
2017-07-16 23:06:06.697 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDJPY.CASH.IP
2017-07-16 23:06:06.701 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDJPY.CASH.IP
2017-07-16 23:06:07.658 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.AUDUSD.CASH.IP
2017-07-16 23:06:07.659 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.AUDUSD.CASH.IP
2017-07-16 23:06:07.810 [main] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when getting broker account info: {"errorCode":"error.public-api.exceeded-api-key-allowance"}

Please let me know if there are any additional debugging options I should apply. The plugin could be really great but given the data limitations by IG debugging and eventually throttling data consumption appears to be a crucial prerequisite.

First thing first, just to make it clear.
The error you are seeing
errorCode":"error.public-api.exceeded-api-key-allowance"
Means that there has been too many calls in a too short time period. So your historical quota has not been consumed, when that happens the message is slightly different.

If you open the logback.xml (same directory as plugin log file) and change log level from info to debug for only for this line
<root level="info">

You should get some more info on exactly what data zorro is asking for.

Have you also made sure that you are indeed using the same assets for the IG plugin and FXCM? Could you please post your assets file for both?
The +1 vs +205 has nothing to do with the plugin as far as I know.

Asset files are quite the same, except the Symbol names, of course:

AssetsFXCM.csv:
Name,Price,Spread,RollLong,RollShort,PIP,PIPCost,MarginCost,Leverage,LotAmount,Commission,Symbol
AUD/USD,0.77311,0.00005,0.2400,-0.5100,0.0001,0.087100,9.0000,74.820,1000.0,0.600,AUD/USD
EUR/CHF,1.07962,0.00004,0.0800,-0.2600,0.0001,0.092630,75.0000,13.334,1000.0,0.800,EUR/CHF
EUR/JPY,123.3590,0.00600,-0.3000,0.0900,0.0100,0.081066,10.0000,100.002,1000.0,0.800,EUR/JPY
EUR/USD,1.13795,0.00005,-0.0200,0.0100,0.0001,0.087100,10.0000,99.115,1000.0,0.600,EUR/USD
GBP/USD,1.51364,0.00003,0.0500,-0.1200,0.0001,0.087100,14.0000,94.170,1000.0,0.600,GBP/USD
GER30,10884.00,1.00000,-0.1000,0.0000,1.0000,0.100000,6.0000,181.400,0.1,0.000,GER30
NAS100,4185.00,2.0000,-0.1000,0.0000,1.0000,0.087100,2.0000,182.257,0.1,0.000,NAS100
NZD/USD,0.67752,0.00007,0.6000,-1.3200,0.0001,0.088713,7.0000,85.864,1000.0,0.800,NZD/USD
SPX500,2032.40,0.50000,-0.5000,0.0000,0.1000,0.087100,8.0000,221.278,1.0,0.000,SPX500
UK100,6869.00,2.0000,-0.1000,0.0000,1.0000,0.131830,6.0000,150.923,0.1,0.000,UK100
US30,17520.00,2.0000,-0.5000,0.0000,1.0000,0.087100,6.0000,254.332,0.1,0.000,US30
USD/CAD,1.24466,0.00010,-0.2100,0.0900,0.0001,0.069980,10.0000,87.101,1000.0,0.800,USD/CAD
USD/CHF,0.92203,0.00022,0.1500,-0.5800,0.0001,0.094460,50.0000,17.419,1000.0,0.600,USD/CHF
USD/JPY,117.4890,0.00400,0.0200,-0.0400,0.0100,0.074130,10.0000,87.095,1000.0,0.600,USD/JPY
XAG/USD,17.2350,0.04600,-0.0009,0.0000,0.0100,0.435470,5.0000,150.107,50.0,0.000,XAG/USD
XAU/USD,1260.82,0.47000,-0.0124,0.0000,0.0100,0.008710,11.0000,99.834,1.0,0.000,XAU/USD

AssetsIG.csv:
Name,Price,Spread,RollLong,RollShort,PIP,PIPCost,MarginCost,Leverage,LotAmount,Commission,Symbol
AUD/USD,0.77311,0.00005,0.24,-0.51,0.0001,0.0871,9,0,1000,0.6,CX.D.AUDUSD.CASH.IP
EUR/CHF,1.07962,0.00004,0.08,-0.26,0.0001,0.09263,75,0,1000,0.8,CX.D.EURCHF.CASH.IP
EUR/JPY,123.359,0.006,-0.3,0.09,0.01,0.081066,10,0,1000,0.8,CX.D.EURJPY.CASH.IP
EUR/USD,1.13795,0.00005,-0.02,0.01,0.0001,0.0871,10,0,1000,0.6,CX.D.EURUSD.CASH.IP
GBP/USD,1.51364,0.00003,0.05,-0.12,0.0001,0.0871,14,0,1000,0.6,CX.D.GBPUSD.CASH.IP
GER30,10884,1,-0.1,0,1,0.1,6,0,0.1,0,IX.D.DAX.IFD.IP
NAS100,4185,2,-0.1,0,1,0.0871,2,0,0.1,0,IX.D.NASDAQ.IFD.IP
NZD/USD,0.67752,0.00007,0.6,-1.32,0.0001,0.088713,7,0,1000,0.8,CX.D.NZDUSD.CASH.IP
SPX500,2032.4,0.5,-0.5,0,0.1,0.0871,8,0,1,0,IX.D.SPTRD.IFD.IP
UK100,6869,2,-0.1,0,1,0.13183,6,0,0.1,0,IX.D.FTSE.CFD.IP
US30,17520,2,-0.5,0,1,0.0871,6,0,0.1,0,IX.D.DOW.IFD.IP
USD/CAD,1.24466,0.0001,-0.21,0.09,0.0001,0.06998,10,0,1000,0.8,CX.D.USDCAD.CASH.IP
USD/CHF,0.92203,0.00022,0.15,-0.58,0.0001,0.09446,50,0,1000,0.6,CX.D.USDCHF.CASH.IP
USD/JPY,117.489,0.004,0.02,-0.04,0.01,0.07413,10,0,1000,0.6,CX.D.USDJPY.CASH.IP
XAG/USD,17.235,0.046,-0.0009,0,0.01,0.43547,5,0,50,0,CX.D.XAGUSD.CASH.IP
XAU/USD,1260.82,0.47,-0.0124,0,0.01,0.00871,11,0,1,0,CX.D.XAUUSD.CASH.IP


Thanks for the hint on "error.public-api.exceeded-api-key-allowance" vs. "error.public-api.exceeded-account-historical-data-allowance" (the latter I finally got just now after some more testing, unfortunately. So no more testing for one week). At least this confirms that the historical data consumption is not the main issue and the slight differences in the messages reported by Zorro about historical data were just distracting...

So what about this API limit (https://labs.ig.com/faq) then:
Per-account non-trading requests per minute: 30

Here is what I got while starting Z12 without any previous testing for (a lot) more than one minute:

2017-07-17 13:56:32.965 [main] INFO c.danlind.igz.brokerapi.BrokerLogin - Connecting to IG Real-account as xxxxxx
2017-07-17 13:56:34.630 [main] INFO com.danlind.igz.ZorroBridge - Login successful
2017-07-17 13:56:34.630 [main] INFO com.danlind.igz.brokerapi.BrokerTime - Subscribing to Lightstreamer heartbeat
2017-07-17 13:56:34.633 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer account updates for account: XXXXX
2017-07-17 13:56:34.778 [main] INFO com.danlind.igz.brokerapi.BrokerTime - Getting initial time from restAPI
2017-07-17 13:56:34.850 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.EURUSD.CASH.IP
2017-07-17 13:56:34.852 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.EURUSD.CASH.IP
2017-07-17 13:56:35.571 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDCHF.CASH.IP
2017-07-17 13:56:35.572 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDCHF.CASH.IP
2017-07-17 13:56:37.001 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.GBPUSD.CASH.IP
2017-07-17 13:56:37.002 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.GBPUSD.CASH.IP
2017-07-17 13:56:37.641 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDCAD.CASH.IP
2017-07-17 13:56:37.707 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDCAD.CASH.IP
2017-07-17 13:56:38.415 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.USDJPY.CASH.IP
2017-07-17 13:56:38.419 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.USDJPY.CASH.IP
2017-07-17 13:56:38.943 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: CX.D.AUDUSD.CASH.IP
2017-07-17 13:56:38.944 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: CX.D.AUDUSD.CASH.IP
2017-07-17 13:56:39.668 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: IX.D.NASDAQ.IFD.IP
2017-07-17 13:56:39.670 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: IX.D.NASDAQ.IFD.IP
2017-07-17 13:56:40.657 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: IX.D.SPTRD.IFD.IP
2017-07-17 13:56:40.658 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: IX.D.SPTRD.IFD.IP
2017-07-17 13:56:41.183 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer tick updates for market: IX.D.DOW.IFD.IP
2017-07-17 13:56:41.184 [main] INFO com.danlind.igz.handler.AssetHandler - Subscribing to Lightstreamer volume updates for market: IX.D.DOW.IFD.IP
2017-07-17 13:56:41.314 [main] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when getting historic prices for epic IX.D.DOW.IFD.IP, error was {"errorCode":"error.public-api.exceeded-api-key-allowance"}
org.springframework.web.client.HttpClientErrorException: 403 Forbidden

(I switched back from DEBUG to INFO for the above log as debug is extremely verbose)

I wonder if each of these subscription requests (I'm counting 19 or 20 of these before the error) is just one restAPI request or if there are a few more behind the scenes? If so, the 30 per minute limit could be hit. I guess this could only be avoided by active request pacing by the plugin.

I realize that I had a discussion with IG support some time ago when I had a similar issue. Basically their logs were saying that I was hitting the application limit of 60 requests per second even though I was nowhere near that rate.

I suggest you contact IG support just to see if you are running into that same issue. Provide them with the api key you are using, date, time and time zone when you hit the limit.
If it helps, the guy who helped was named Matt Bailey

BTW, if you want to count the exact number of requests being made, you should be able to do that with the DEBUG setting. There are a couple of other besides the ones that your are seeing (starting account subscription, getting account time zone, the actual login etc), but since you are not close to 60 this should not be the issue.

In my conversation with IG I actually mentioned the Per-account non-trading requests per minute: 30, but Matt was positive that this was not the limit I was hitting according to the logs.

Ok, then this may well explain the issue. Even based on the INFO messages I'm counting 28 requests (2x13 for the instruments +2 for account and heartbeat). So if there are a few more the 30 per minute is reached. I don't think they put this in the FAQ and don't enforce it. Maybe they relaxed the limit for you individually as you are developing a plugin that has good potential to boost their client base?
I am in contact with IG support, let's see what they respond...

That sounds familiar also. I was close to 30, but not really at 30 when counting the requests.

Basically support said that my application API limit was also set to 30 rather than 60 for some reason. So they increased it to 60, and after that I've not had any problems.

To be honest I find the FAQ a bit confusing on the limits, specifically
Per-app non-trading requests per minute: 60
Per-account non-trading requests per minute: 30

To me it seems like the per-account limit of 30 would always be hit before the per-app limit, since all app keys belongs to an account.
I've sent a mail to support asking them to clarify this as well, I'll post back here when I get an answer.

No feedback from the API experts at IG yet, despite several reminders to the helpdesk front line ...

That's weird. Vacation times? I've never had to wait more than a couple of days for answer from support, and that was true before I made this plugin as well

For me, updating the Assets file (Download script -> "Save Assets to") triggers the limit at my IG account (the current limit is "Per-account non-trading requests per minute: 30"). The limit is triggered after the 12th asset in the list (it used to be after the 8th asset one month ago with previous versions of Zorro and the IG plugin). When I asked the Dev support for a limit raise, their response was:


Since you're already using the Lightstreamer lib, is there a way to prevent reaching the limit, e.g. by queuing the requests or using this trick?

I'm getting quick answers to each request but only from first line support: "I will follow up with the developers again to get a response from the as quickly as possible."...

Anyway, since these limits appear to be a showstopper for the plugin so far, could you consider dhrv's suggestion?

Hi,

Using the download script, if you really need to download many assets, you should download one at a time rather than all at once. As mentioned in the installation notes, using IG to download lots of historical data is not a good idea.

I've also asked support about the limits for the rest api. All requests that require an api key (which is all requests made by the plugin) counts towards the application limit of 60. So rather than asking them to just raise your limit, ask why you are getting rate limited after 30 requests rather than 60.

I'm not sure rate limiting the api calls would even work, as I suspect Zorro would time out if the api call takes to long. I will check this with zorro support team. Anyways, I'm away on vacation for the next 5 weeks, so I won't prioritize this specific suggestion during that time.

Thank you very much. I think the Zorro developers should be able to introduce a parameter to adjust for such delays ... enjoy your vacation!

This was purely about updating the Asset list variables (spread etc.), not about downloading historic prices (.t6 files), but yes, I always update the Asset list successfully in small batches Still I wanted to report this since you guys were discussing something similar. It really all comes down to IG's API limit. I imagine it's hard for the Zorro Devs and for you to implement a queuing or connection throttle while retaining the guarantee to be able to modify orders/close trades at all times. So don't worry about this too much and enjoy your holidays!

As long as the throttling would only affect the INITRUN of a strategy, it should be easy to implement into the Zorro core, at least. I'll continue bugging IG for an increase of the 30 requests limit as this doesn't make sense to me either.

Finally IG have increased my limits: "account non trading request per minute to 60 and account trading requests per minute to 100" - an I could successfully start Z12, Z3, and Z7. Of course I have been waiting 2-3 mins between starting each of the strategies.
Ok, one issue fixed, now let's see how it looks while trading ...

Zorro provides two functions for adjusting delays: SET_DELAY and SET_WAIT. But they must be realized in the plugin. Most broker plugins support them, but I don't know if they are implemented in the IG plugin.

Hello,
I am pretty new to this coding, I am trying to connect a demo account to Zorro using your plugin, but I do not succeed. The message is get from Zorro is:
Login IG..
!java.lang.NoClassDefFoundError!com/danlind/igz/ZorroBridge
BrokerLogin:No Data!

I do not really know what to do as I think I followed all the steps from the readme file. Any hint on what to do?
Thank you

If you are using the latest zorro release there might be an issue which I discovered only yesterday. Will try to post an update today.

Can you try with this new release?

https://github.com/dan-lind/igzplugin/releases/tag/v0.22

Also make sure you've followed these instructions
https://github.com/dan-lind/igzplugin

Hello I am at work currently so I will only try at noon. I indeed use the latest version of Zorro. I will keep you posted. Thank you

Hi,
Now I have an error message. Link to the log file is here: https://pastebin.com/ygvn5yzG

It looks like you didn't read the Readme after all? &#128539;
The error message is pretty obvious

Exception when creating session with api key <your demo key here>, error was {"errorCode":"error.security.api-key-invalid"}

I did, my bad didn't really know where to look in the log file. Now I have a different error. I updated the symbols name as requested (I took the CSV displayed previous posts ago), but I am not sure if the error is due to this. Is it ok if I created a AssetsIG.csv file with the following:

Name,Price,Spread,RollLong,RollShort,PIP,PIPCost,MarginCost,Leverage,LotAmount,Commission,Symbol
AUD/USD,0.77311,0.00005,0.24,-0.51,0.0001,0.0871,9,0,1000,0.6,CX.D.AUDUSD.CASH.IP
EUR/CHF,1.07962,0.00004,0.08,-0.26,0.0001,0.09263,75,0,1000,0.8,CX.D.EURCHF.CASH.IP
EUR/JPY,123.359,0.006,-0.3,0.09,0.01,0.081066,10,0,1000,0.8,CX.D.EURJPY.CASH.IP
EUR/USD,1.13795,0.00005,-0.02,0.01,0.0001,0.0871,10,0,1000,0.6,CX.D.EURUSD.CASH.IP
GBP/USD,1.51364,0.00003,0.05,-0.12,0.0001,0.0871,14,0,1000,0.6,CX.D.GBPUSD.CASH.IP
GER30,10884,1,-0.1,0,1,0.1,6,0,0.1,0,IX.D.DAX.IFD.IP
NAS100,4185,2,-0.1,0,1,0.0871,2,0,0.1,0,IX.D.NASDAQ.IFD.IP
NZD/USD,0.67752,0.00007,0.6,-1.32,0.0001,0.088713,7,0,1000,0.8,CX.D.NZDUSD.CASH.IP
SPX500,2032.4,0.5,-0.5,0,0.1,0.0871,8,0,1,0,IX.D.SPTRD.IFD.IP
UK100,6869,2,-0.1,0,1,0.13183,6,0,0.1,0,IX.D.FTSE.CFD.IP
US30,17520,2,-0.5,0,1,0.0871,6,0,0.1,0,IX.D.DOW.IFD.IP
USD/CAD,1.24466,0.0001,-0.21,0.09,0.0001,0.06998,10,0,1000,0.8,CX.D.USDCAD.CASH.IP
USD/CHF,0.92203,0.00022,0.15,-0.58,0.0001,0.09446,50,0,1000,0.6,CX.D.USDCHF.CASH.IP
USD/JPY,117.489,0.004,0.02,-0.04,0.01,0.07413,10,0,1000,0.6,CX.D.USDJPY.CASH.IP
XAG/USD,17.235,0.046,-0.0009,0,0.01,0.43547,5,0,50,0,CX.D.XAGUSD.CASH.IP
XAU/USD,1260.82,0.47,-0.0124,0,0.01,0.00871,11,0,1,0,CX.D.XAUUSD.CASH.IP


Error messages:

2017-09-18 20:05:04.852 [main] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when getting historic prices for epic EUR/USD, error was <html><head><title>Page Not Found</title><h3>Error 404</h3><h3>Sorry, the requested page is not available.</h3></body></html>
org.springframework.web.client.HttpClientErrorException: 404 Not Found

2017-09-18 20:05:04.853 [main] ERROR c.danlind.igz.brokerapi.BrokerAsset - Error when subscribing to EUR/USD
org.springframework.web.client.HttpClientErrorException: 404 Not Found

2017-09-18 20:05:04.975 [RxCachedThreadScheduler-6] ERROR c.danlind.igz.brokerapi.BrokerAsset - Error subscribing to volume observable for EUR/USD
com.lightstreamer.ls_client.PushUserException: Incorrect instrument setup

I really appreciate the time & effort you put into this plugin!

If you want to use another asset list than asset fix, load it with assetList in your strategy like this

assetList("AssetsIG");

http://zorro-project.com/manual/en/asset.htm

Editting my previous post, the symbols were not ok.
Now I think I have the correct one, but I do get this:

2017-09-18 21:13:32.126 [main] ERROR c.danlind.igz.adapter.RestApiAdapter - Exception when getting historic prices for epic EUR/USD, error was <html><head><title>Page Not Found</title><h3>Error 404</h3><h3>Sorry, the requested page is not available.</h3></body></html>
org.springframework.web.client.HttpClientErrorException: 404 Not Found

2017-09-18 21:13:32.127 [main] ERROR c.danlind.igz.brokerapi.BrokerAsset - Error when subscribing to EUR/USD
org.springframework.web.client.HttpClientErrorException: 404 Not Found

and if I try the GER30, I get this:

2017-09-18 21:14:24.574 [RxCachedThreadScheduler-3] ERROR c.danlind.igz.brokerapi.BrokerAsset - Error subscribing to tick observable for IX.D.DAX.IFD.IP
com.lightstreamer.ls_client.PushUserException: Insufficient permissions

2017-09-18 21:14:24.710 [RxCachedThreadScheduler-5] ERROR c.danlind.igz.brokerapi.BrokerAsset - Error subscribing to volume observable for IX.D.DAX.IFD.IP
com.lightstreamer.ls_client.PushUserException: Insufficient permissions

I am a bit lost, sorry for this... I do not understand if my settings are wrong or if I cannot access the value due to any other limitation

EUR/USD is not a correct symbol name, it should be more in line with the other examples you have given. Not sure about Insufficient permission, you would need to ask IG support about that.

I will get to IG and keep you posted. Regarding EUR/USD, I do not understand one point: my line in the Assets.csv file is the following:
EUR/USD,1.13795,0.00005,-0.0200,0.0100,0.0001,0.087100,10.0000,99.115,1000.0,0.600,CS.D.EURUSD.CFD.IP

It is pressty close to the example I saw before. The symbol name I put is not EUR/USD but CS.D.EURUSD.CFD.IP which is what I get from the API companion. Then why Zorro is telling me about EUR/USD?

Can you post your strategy and your current AssetsIG file?

Here is the AssetsIG.csv file saved in the History folder, and here is the strategy: https://pastebin.com/6BFHZVJA. Nothing fancy, just the test trade strategy with the assetList("AssetsIG"); line added
Log file here: https://pastebin.com/F1vLBtuS

Can't see the asset file? My suspicion would be that something is wrong with that, because apparently it's trying to subscribe to EUR/USD rather than the long weird epic name used by IG.

Here are the log file, the assetIG file and the asset file. Strategy is the same as previous. I get the code is trying to subscribe to EUR/USD but I don't get why it would use the epic name for DAX and not for EUR/USD

Not sure why you have both an Assets and an AssetsIG? Anyway, I cannot see anything obvisouly wrong with the AssetsIG file. I would try a much simpler strategy first without all PanelGet etc, just to see if you can make it work. I'm pretty confident that the issue you are experiencing is not connected to the plugin.

Ok I think I figured it out, and your plugin is working fine. Many thanks for the support.

Hello,
Sorry to disturb again, I am trying to download data using the download script. I entered the correst symbol for EUR/USD, and I get this message:

2017-09-24 17:59:01.357 [main] WARN c.d.igz.brokerapi.BrokerHistory - Zero ticks returned for requested date range 2017-09-24T12:59:01 - 2017-09-24T17:59:01

If I change the years (instead of 2017-2017 I input 2016-2017) I also get the same error message plus another one:

2017-09-24 18:08:23.391 [main] WARN c.d.igz.brokerapi.BrokerHistory - Zero ticks returned for requested date range 2016-12-31T20:48:59 - 2017-01-01T01:48:59

Is the download script working with the plugin or should I write my own script? What is the reason of such an error?

Sorry to bother you again, I am still learning!

I would not recommend using the plugin to download historical data, other than to backfill the lookback period when starting a script. IGs limits for historical data is VERY low, which means you will easily exhaust your weekly limit. If you do, you will not even be able to start a script if even a single minute of data is missing.

That said, it should work with the download script, IF you stay within the time range given in the FAQ: https://labs.ig.com/faq.

My guess for the error message: You have tried to download data from today = Sunday. IG is not open until late in the evening on Sundays, so there are no ticks to be found within the range stated in the error message.

The second error is probably because you are trying to fetch data too far back in time, again refer to the FAQ.

Hello Dalla & community,

I attempted to connect to IG using the IG plugin. I read through this thread but was not able to resolve this on my own leaving me with a bit of a bruised ego.

These are the steps I performed:
- Installed Zorro V1.60
- Installed IG plugin as per instructions in the manual and on GitHub
- Generated API keys for both real and demo IG accounts
- Updated API keys in application properties.file of IG plugin
- Updated File AssetsFix.csv with IG symbols extracted from IG API Companion

I failed when attempting to run any of the scripts, for example 'TradeTest'and 'Download'scripts receiving the error message: Severe error occured, check igzplugin.log logfile! Can't connect!.

The result is the same after multiple attempts over several days. I also tried using (1) username + password and (2) username + API key to connect to the server, again with the same result.

Any advice on how to tackle this would be greatly appreciated.

Best regards,
Kris

Hi

You said that you installed Zorro 1.60. The newest version is 1.66. Try with that first.
To be honest I don't really know what the problem is, can't remember seeing error 400. I would expect 403 if something was wrong with the login credentials.

Hi Dalla,

Thanks for the quick response, I will try with V1.66. In the meantime I found out that IG has a login restriction for Singapore where I am located:

**************************************************************************
Login Restrictions
The /session API is subject to the following country-specific restrictions:
Singapore - all login passwords must first be encrypted using the relevant IG SDK function below.

To encrypt a password:
call /session/encryptionKey which gives a key and timestamp
create a RSA token using the key.
encrypt password + "|" + timestamp
See sample applications for further details.

**************************************************************************

Source: https://labs.ig.com/loginrestrictions

Could this be a problem for the plugin?

Thanks,
Kris

The password is not encrypted at this point, although that might be a good thing to implement in a future version. Unfortunately I don't have that much time at the moment, so I wouldn't expect this to change soon. I'd be happy to take a pull request if your up for implementing it yourself.

Hi Dalla,

I am not exactly a skilled coder but I might give it a shot...