Sierra Chart Plugin

Posted By: AndrewAMD

Sierra Chart Plugin - 09/20/18 00:42

I am proud to announce that the Sierra Chart Plugin is now in BETA, and so I am in need of some testers.

Documentation and source code is here:
https://github.com/AndrewAMD/SierraChartZorroPlugin

The latest DLL can be found here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/tree/master/releases

I am here to answer any questions and fix any bugs you may have found. Please let me know.

Thanks,
Andrew
Posted By: Zheka

Re: Sierra Chart Plugin - 09/20/18 10:24

Thank you very much, Andrew, for your work!
I will test it, but only in a month or so...

Meanwhile:
- does the plug-in support true limit orders?
- how does one retrieve historical data for a futures/options contract?
how/why would one use GET_DTCSECURITYDEF BrokerCommand?

Thank you!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/20/18 12:15

1) Yes, not not via SET_LIMIT but directly in the enterLong/Short calls

2) Call the explicit asset symbol directly. It is the only way.

3) I will put together a demo for the last one.
Posted By: Zheka

Re: Sierra Chart Plugin - 09/20/18 14:20

1) You mean, by specifying a TakeProfit var in the enterLong, the plug-in will send a real limit order to the book, right?

2) What "symbology" should be followed? Sierra's?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/20/18 14:57

1) The plugin is NFA mode (orders) only, so this depends on how Zorro handles pool trades.

2) Sierra's. If you can pull up a chart for that explicit symbol, it should be available. This is why I needed to put in the workaround (more on that later).
Posted By: Zheka

Re: Sierra Chart Plugin - 09/29/18 18:47

Hi,Andrew,

re getting historical prices: as I understand, BrokerHistory2 fills in a t6 struct.
So, one has then to manually "repackage" them to t8, correct?
Does SC plug-in fill in the fVal field in a t6 struct for Spread?

It would be great to get the plug-in to recognize that the instrument is a "contract" and switch to filling in a t8 struct.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/29/18 19:28

Originally Posted By: Zheka
re getting historical prices: as I understand, BrokerHistory2 fills in a t6 struct.
Yes.

Originally Posted By: Zheka
So, one has then to manually "repackage" them to t8, correct?
I suppose filling t8 can be one way to do it. There may be other ways. Anyways, the plugin cannot do this for now, so it must be manual.

Originally Posted By: Zheka
Does SC plug-in fill in the fVal field in a t6 struct for Spread?
For market, yes. Historical data does not have spreads, so that is set to zero.

Originally Posted By: Zheka
It would be great to get the plug-in to recognize that the instrument is a "contract" and switch to filling in a t8 struct.
Sierra Chart is not eager to support options much further than market and trading. Last time I checked, the symbol definitions did not clarify whether they were puts or calls, not even the strike price. (That is, the fields are available in the security definition response messages, but they were left blank.) There might have been a multiplier, not sure. Maybe they added some features ever since?

Depending on what's available, I might be able to put in another custom BrokerCommand to fill a CONTRACT struct for a given explicit SC symbol.

I will look into this.

EDIT: Strike price is included.
Posted By: Zheka

Re: Sierra Chart Plugin - 09/29/18 21:34

Quote:
For market, yes. Historical data does not have spreads, so that is set to zero.
What do you mean by "do not have spreads"? Normally both bid and ask data ARE available in SC..

Quote:
Last time I checked, the symbol definitions did not clarify whether they were puts or calls, not even the strike price. (That is, the fields are available in the security definition response messages, but they were left blank.) There might have been a multiplier, not sure
I read your messages on the DTC forum. Do I understand correctly that this is a problem for forming a contract list, but for a directly specified symbol - it is not?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/30/18 01:42

For no historical spreads, I meant for M1 (T6) historical bars. Here is the DTC message:
https://dtcprotocol.org/index.php?page=d...RECORD_RESPONSE

However, I believe tick history is able to give spread, since the historical data indicates ask and bid price.

As for the second question:
https://dtcprotocol.org/SupportBoard.php?ThreadID=33413

Oh, it looks like they do have strike price.

But as they said: "The security definition does contain the strike price. The other fields are not set."

This is for every individual options security.

So according to this thread, I will not be able to get options by underlying soon, nor can I determine whether an options security is a put or a call.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 10/01/18 15:02

Originally Posted By: Zheka
So, one has then to manually "repackage" them to t8, correct?
Random note, the plugin is able to buy and sell options symbols directly. So you technically don't even need to touch t8 structs.

If you explicitly need t8's to come from the plugin, I might have a workaround idea but it most likely involves manual labor from the strategy writer.
Posted By: Zheka

Re: Sierra Chart Plugin - 10/01/18 19:47

Quote:
the plugin is able to buy and sell options symbols directly
I understand that. The question is about simplifying delivery of historical data.

If SC only has 1-min "trade" data - that would be of almost no use (for backtesting): trades do not happen often in a particular strike, and so the last trade tick might be too old to be meaningful.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 10/02/18 12:41

Originally Posted By: Zheka
The question is about simplifying delivery of historical data.
Did you have a question?

Originally Posted By: Zheka
If SC only has 1-min "trade" data - that would be of almost no use (for backtesting): trades do not happen often in a particular strike, and so the last trade tick might be too old to be meaningful.
The type of historical data available (ticks and/or M1 for any security type) depends on:
1) If applicable, your Sierra Chart Service Package type
2) If applicable, your Sierra Chart Data or Historical Data service
3) If applicable, your broker that Sierra Chart is connected to, and the type of historical service it provides.

As you can see, there are many confounding variables here. smile

In any case, the plugin does both ticks and M1 for any security type, if available on DTC.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 10/17/18 15:25

Bump. Beta testers wanted. Sierra Chart does have a free trial, by the way.

I believe I will be implementing GET_BOOK soon, as Sierra Chart has very good support for market depth.
Posted By: AndrewAMD

BETA: Sierra Chart Plugin - 10/27/18 15:21

NEW BUILD (0.9.1.1), still in beta (testers wanted).
Download DLL here

Documentation is here:
https://github.com/AndrewAMD/SierraChartZorroPlugin

New feature: Added brokercommand GET_BOOK



Attached picture mdepth.PNG
Posted By: AndrewAMD

BETA 0.9.2.0: Sierra Chart Plugin - 11/03/18 15:44

NEW BUILD (0.9.2.0). Beta testers wanted.
Download DLL here

Documentation is here:
https://github.com/AndrewAMD/SierraChartZorroPlugin

Bug Fix: Problem receiving T1 tick data when server decides to send bar messages instead of tick messages.
Posted By: Trading4pips

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 15:48

Hi Andrew,

I'm answering your call for beta testers for the Sierra Chart plugin.

I'm currently using SC 1841 and Service package 5.

I've copied the SC Zorro Plugin v 0.9.2.0 to the Zorro plugin folder and created a history file to include the current futures SC symbols: ESZ18 and NQZ18.

When I run a strategy an ES_2018.t6 file is created in the History folder, some data appear to be placed in the file and then I receive a couple of errors. Any guidance you can provide would be greatly appreciated.

Here is a snip-it of the messages I'm receiving:
...
Subscribing to Symbol ESZ18...
Market depth subscribed.
Data is being downloaded from a remote source. Download will start when this is done.
Load ES 2018.. 85678 min - ES history unavailable!
Error 056: Can't download ES 2017 history
Error 046: TRND EMA LookBack exceeded by 37 bars (128)
Logout.. ok
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 16:20

Can you post your script?

Also, what broker/service are you using with Sierra Chart?
Posted By: Trading4pips

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 18:17

I'm using the SC Sim1 account to test the plugin. I'm not subscribed to SC's real-time data, I use Interactive Brokers for RT data. For this test, I was just trying to pull historical data and run the script.

Also, in the user field, I tried leaving it blank (to which I receive an ES history unavailable message). So I run the script with only zlib in the user field and it appears to pull ES data from 2018-04-02 (based on the Warning message: Warning 047: ES first bar at 2018-04-02).

Based on the above warning, I thought a first step would be to use the plugin to successfully pull historical data. So I created the following Asset file: Assets-SC-ES.csv (spacing appears messed up below, but it lines up properly in Excel):

Code:
Name	Price	Spread	RollLong	RollShort	PIP	PIPCost	MarginCost	Leverage	LotAmount	Commission	Symbol
ES	2700	0.25	0	0	0.25	12.5	6000	0	1	5	ESZ18



I then took the Gap study and modified it slightly:

Code:
// Display gaps in historical data.
// Red lines = gap length in days. 

#define ASSET_LIST "History\Assets-SC-ES.csv"

function run()
{
	StartDate = 2017;
	//StartDate = 2018;
	LookBack = 0;
	set(PLOTNOW);
	assetList(ASSET_LIST);
	//asset("EUR/USD");
	var m = minutesAgo(1);
	plot("Gap",m/(60*24),NEW,RED);
}



Findings:
1. With a StartDate = 2018 the script runs without any errors, but the chart starts on April 2, 2018 rather than January.

2. With a StartDate = 2017 I receive the following message:
GapFinder_2018 compliling..........
Error 055: ES History missing (2017 8192 HistoryES.t6)
Test: GapFinder_2018 ES 2017..2018
Asset HistoryAssets-SC-ES.csv
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 18:22

If you know your historical data begins in 2018, why are you setting StartDate at 2017 and expecting it to work?
Posted By: Trading4pips

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 19:47

The historical data from SC goes back years. So I thought the plugin would connect to SC and download the required data. The data for the ES was downloaded by the plugin when I first ran it, but it only downloaded it from April 2, 2018.

My assumption was that when I tell it to start in 2018 it would download data for all of 2018, when I change the start date to 2017 it would download missing data starting January 1, 2017.

Am I wrong on this assumption?
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 19:50

You set ES to symbol ESZ18, so you are limited to ESZ18.
Posted By: Trading4pips

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 19:59

Ok, then that's not what I really want, I would like to pull the data for the continuous ES contract.

In SC I use ESZ18 as the symbol, but under the Advanced Settings I have Continuous Contract set to: Continuous Futures Contract - Date Rule Rollover.
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 20:16

Alas, Sierra Chart's DTC server does not support the export of Continuous Contracts, or any indicator for that matter.

I believe you do have two options:

1) Using Zorro, download all of the individual contracts' historical data and construct a continuous contract indicator from that data.

2) Maybe you can export Continuous bars directly from Sierra Chart to text/csv, and then convert it to t6 using a zorro script. I have not tried this yet.
Posted By: Trading4pips

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/15/18 23:39

It is really unfortunate and surprising that their server does not support continuous contracts since SC uses them.

I appreciate your suggestions, but I think option 1 is not realistic due to the amount of work that would be involved in constructing a continuous contract.

When I get some time, I'll try to research option 2 to see how one might convert the downloaded data to t6 files. If you are taking feature requests, this would be a great option to add to the plugin laugh
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 11/16/18 00:11

Take a peek at the CSVtoHistory script, it should point you in the right direction.

Given the DTC implementation, it is not realistic to try to output continuous contracts from the plugin. The SC staff told me finding futures symbols by underlying is on their to-do list, but they never got around to it.
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/08/19 18:23

Oh wow, Sierra Chart finally came through with the underlying search for futures!

https://dtcprotocol.org/SupportBoard.php?ThreadID=33413

It looks like I can pull this out of beta in the next month! laugh
Posted By: firecrest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/11/19 10:17

Thank you so much for the Sierra Chart plugin. Sierra Chart is link to IB. I am thinking of what is the advantage of using Sierra Chart plugin instead of just directly using IB plugin.
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/11/19 13:56

If you need lower latency and every last tick, use the Sierra Chart plugin.

If you want access to Sierra Chart's historical data and/or live streaming data service, use the Sierra Chart plugin.

If you need options, use the IB plugin.

If you need all of the above at the same time, use both plugins simultaneously.

Here's why:
* The engineers at Sierra Chart have gone so far as to reverse engineer the IB interface so that they are working directly with IB's proprietary protocol rather than a library. They prefer to work directly with sockets.
* Likewise, my plugin also works directly with sockets (using Boost Asio as a frontend for the socket API).
* Finally, the Sierra Chart plugin supports SET_HWND while the IB plugin currently does not. This means you can get every last tick while running your strategy live.
* But Sierra Chart is not eager to support listing options symbols by underlying using their integrated DTC protocol. (Their protocol supports it, but that's irrelevant if the server does not.)
* Anyways, the IB plugin works well with options.
Posted By: tradingest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 13:21

Hi guys,

but is more fast to use LMAX account with Sierra plugin or with MT4 plugin?
We always need to have the platforms opened, MT4 or Sierra

thanks
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 13:44

Originally Posted By: tradingest
Hi guys,

but is more fast to use LMAX account with Sierra plugin or with MT4 plugin?
We always need to have the platforms opened, MT4 or Sierra

thanks
Until the MT4 plugin supports GET_HWND, the Sierra Chart plugin will be faster.

One other notable difference is that you must use the Sierra Chart plugin in NFA mode, since the DTC protocol only processes orders and not "trades".
Posted By: tradingest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 13:56

Is it possible to have a more information about GET_HWND?
What does means?

thanks AndrewAMD
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 14:07

Originally Posted By: tradingest
Is it possible to have a more information about GET_HWND?
What does means?

thanks AndrewAMD
It means Zorro will be notified when a new tick has arrived.
Posted By: tradingest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 14:28

Originally Posted By: AndrewAMD
Originally Posted By: tradingest
Is it possible to have a more information about GET_HWND?
What does means?

thanks AndrewAMD
It means Zorro will be notified when a new tick has arrived.


but for this we can use function tick() within Zorro.
I don't understand what is the difference...
Using function tick() the script is performed for each tick.
For me it is very important to understand well the diffrence and why Sierra plugin is more fast respect to MT4.
Also with MT4 plugin is it possible to obtain the notification for each tick

Is it possible to have a more details about it to use the plugin in the best way?
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 15:06

Originally Posted By: tradingest
Using function tick() the script is performed for each tick.
Not necessarily.

There is a chain...

BROKER <--> PLUGIN <--> ZORRO <--> tick()

Without SET_WHND:

* Broker says there is a tick.
* Plugin retains tick info or does nothing.
* (pause indefinitely)
* Broker says there is a tick.
* Plugin retains tick info or does nothing.
* (pause indefinitely)
* Zorro's timer says that it is time to check for a new tick on all Assets.
* Zorro gets the tick.
* Zorro invokes tick().

With SET_WHND:

* Broker says there is a tick.
* Plugin tells Zorro there is a tick.
* Zorro gets the tick.
* Zorro invokes tick().
Posted By: tradingest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/16/19 15:18

Originally Posted By: AndrewAMD
Originally Posted By: tradingest
Using function tick() the script is performed for each tick.
Not necessarily.

There is a chain...

BROKER <--> PLUGIN <--> ZORRO <--> tick()

Without SET_WHND:

* Broker says there is a tick.
* Plugin retains tick info or does nothing.
* (pause indefinitely)
* Broker says there is a tick.
* Plugin retains tick info or does nothing.
* (pause indefinitely)
* Zorro's timer says that it is time to check for a new tick on all Assets.
* Zorro gets the tick.
* Zorro invokes tick().

With SET_WHND:

* Broker says there is a tick.
* Plugin tells Zorro there is a tick.
* Zorro gets the tick.
* Zorro invokes tick().


very good...wonderful...now is more clear...thanks so much

less steps means more speed
Posted By: tradingest

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/27/19 10:45

AndrewAMD can you help me to join LMAX account with SierraChart?

I follow these steps but doesn't work
https://www.sierrachart.com/index.php?page=doc/LMAX.php
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 01/27/19 13:10

Originally Posted By: tradingest
AndrewAMD can you help me to join LMAX account with SierraChart?

I follow these steps but doesn't work
https://www.sierrachart.com/index.php?page=doc/LMAX.php
I don't use LMAX, so I cannot help you.

But you can try the Sierra Chart support boards. Try searching for your errors/problems first.

https://www.sierrachart.com/SupportBoard.php?ForumID=0
Posted By: tomaslolo

Re: BETA 0.9.2.0: Sierra Chart Plugin - 02/06/19 10:32

Hello, I want to try this Sierra plugin. Thank you very much for your work! I follow these instructions:

1.- "To install the plugin, simply place the SierraChart.dll file in the Plugin folder where Zorro is installed."

2.- "Zorro automatically scans that folder at startup, and lists all broker DLLs in the [Broker / Account] scrollbox "

But when I open Zorro S 1.96 version does not shows Sierra Chart in "[Broker / Account] scrollbox "

Am I missing anything?

Thank you
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 02/06/19 12:36

You need to download the DLL for now.

Get it here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/tree/master/releases
Posted By: tomaslolo

Re: BETA 0.9.2.0: Sierra Chart Plugin - 02/06/19 12:50

Yes, SierraChart.dlll is already placed in Zorro Plugin folder.

But does not appear in "Account/broker" scrollbox.



Attached picture SierraCHART_not_showing.jpg
Posted By: AndrewAMD

Re: BETA 0.9.2.0: Sierra Chart Plugin - 02/06/19 13:56

I'm not having this problem, using the latest plugin release.

1) Make sure you're using the latest plugin release.
2) Close Zorro and open it again to refresh the broker list.



Attached picture 001.PNG
Posted By: tomaslolo

Re: BETA 0.9.2.0: Sierra Chart Plugin - 02/06/19 14:33

Solved. I was not downloading the .dll the right way.

Sorry for my mistake and thank you for your effort!

Best Regards
Posted By: nandv

Re: Sierra Chart Plugin - 03/17/19 13:31

I did a simple assetHistory test and got this error, please help!

code:


error:




I tried to search zorro manual and can't find any references to NumberOfDays or StartDateTime

This is on Zorro 2.04 and SC 1882, with the latest SC plugin from your github.

Thanks!

Posted By: danatrader

Re: Sierra Chart Plugin - 03/17/19 13:45

Hi Andrew,

can you give a little start support?

Connecton to DTC works.
Account Sim1 detected.

So from my uderstanding next step is download price data?
So it tells me "!1: NumberOfDays or StartDateTime fields must be non-zero.", although I have 2019-2019, maybe it's just information.

It runs forever, when I cancel I receive the information below.

Download...
Login 0 SierraChart..
!1: Connected to SC DTC Protocol server. Service=scdataallservices|SymbolSettings=scdataallservices
!1: WARNING: Trading not supported
!1: TradeAccounts detected: Sim1
!1: Using: Sim1
!1: Balance synchronized..
!1: NumberOfDays or StartDateTime fields must be non-zero.
!1: Failed to confirm historical orders
!1: Sync hist orders failure.
Can't connect!
Done


I just see the latest post, ame here, all newest... only sim account to receive data.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/17/19 15:22

Okay, we have beta testers!

Quote:
NumberOfDays or StartDateTime fields must be non-zero.

I believe those are DTC message fields. Let me look into this.
Posted By: danatrader

Re: Sierra Chart Plugin - 03/17/19 16:16

Maybe I am wrong, but trade log on sierra chart also then gives basically the same infomation as

!1: Failed to confirm historical orders
!1: Sync hist orders failure.

I guess maybe the Problem could be that SC Sim Account does't have any orders, so they can't be retrieved.

Might be different if SC Sim Account is for example a Rythmic Papertading account (real demo account).

I also tried the earlier version online, same messages.

If you need moore log, etc, I may proviide it.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/17/19 16:33

I believe SC does not supply the history of your orders when using Sim1.

I might be able to suppress these error messages and replace it with a less confusing warning.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/19/19 18:08

Solved: Sierra Chart changed the behavior of their DTC server. It now requires the NumberOfDays field to be filled, even though the documentation says it is not required.

https://www.dtcprotocol.org/index.php?pa...R_FILLS_REQUEST

I will update the DLL this evening.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/20/19 00:22

Just released version 0.9.2.1.

DLL can be found here. Thank you!
Posted By: danatrader

Re: Sierra Chart Plugin - 03/20/19 19:58

Great, looks different now!

Thank you so much.
Posted By: 7th_zorro

Re: Sierra Chart Plugin - 03/25/19 02:47

Using sierra chart, can I display zorro backtest result data?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/25/19 12:01

No.
Posted By: 7th_zorro

Re: Sierra Chart Plugin - 03/30/19 06:59

I am looking forward to update about support continuous futures contract.
Thank you for nice plugin.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/30/19 11:51

As stated before, Sierra Chart’s DTC server does not support continuous contracts. For the plugin to output continuous contracts, I would have to code the reconstructions myself, which I do not have the time for right now.

Currently, you have two options:

1) The user can manually reconstruct the continuous contract as a Zorro script.


2) The user can export CC CSV directly from Sierra Chart, and import it as T6 for Zorro. I talk about this on my blog:

https://www.vitaltrades.com/2019/03/17/continuous-futures-data-in-zorro-1-data-acquisition/
https://www.vitaltrades.com/2019/03/24/continuous-futures-data-in-zorro-2-parsing-csv-to-t6/
Posted By: 7th_zorro

Re: Sierra Chart Plugin - 04/04/19 08:25

Does sierra chart bitmex connection use rest or websocket?
I heard that bitmex have order rejection problem.
Do you have any experience about that?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/04/19 11:11

Originally Posted By: 7th_zorro
Does sierra chart bitmex connection use rest or websocket?
Neither. Bitmex set up a DTC service just for Sierra Chart users. It comes complete with live streaming of market data and market depth, historical intraday, live trading, simulated trading, etc.

Originally Posted By: 7th_zorro
I heard that bitmex have order rejection problem.
Do you have any experience about that?
I don't use Bitmex, but Sierra Chart has declared the Bitmex interface to be stable. You can read about setting up the Bitmex/SC connection here.
Posted By: vinsom

Re: Sierra Chart Plugin - 04/30/19 12:55

Hi Andrew,
Do you know if with this plugin is possible to short crypto futures on Bitmex ?

Thanks
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/30/19 12:59

If your broker allows it, then the plugin allows it.

For now, you have to manually enter futures symbols in your asset list, as GET_FUTURES is not functional.
Posted By: vinsom

Re: Sierra Chart Plugin - 04/30/19 21:11

Sorry andrew another question, how would you set the leverage with this plugin ?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/30/19 21:41

Originally Posted By: vinsom
Sorry andrew another question, how would you set the leverage with this plugin ?
Your only option is explained under "Setting BitMEX Leverage":
https://www.sierrachart.com/index.php?page=doc/BitMEX_Trading.php#SettingBitmexLeverage
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/03/19 13:40

Hi all,

As of Zorro 2.10, it looks like Virtual Hedging will only be a Zorro S feature from that point.

For this plugin in particular, you need NFA flag enabled, and the best Hedge setting is Hedge = 5. This will require Zorro S.

If you are using the free Zorro, you can only use NFA flag enabled and Hedge = 0. This can especially be an issue for portfolio systems using my plugin - in which case, consider Zorro S.
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 09:34

I've tried very hard to connect SierraChart to Zorro.



Login 0 SierraChart..
!1: Connected to SC DTC Protocol server. Service=interactive_brokers.trading|SymbolSettings=interac
!1: TradeAccounts detected: DUXXXXXX,Sim1
!1: Using: DUXXXXXX
!1: Balance synchronized.
!1: TradeAccount is not set
!1: Failed to confirm historical orders
!1: Sync hist orders failure.
Can't connect!

I've tried both with Sierrachart logged into IB, or with the Sierrachart data service.

I tried various ports, TLS on off, zlib, real/demo accounts etc

I am using the latest Sierrachart 1915 and plugin versions as well as Zorro Beta 2.11, also tried non-beta 2.03.

I am able to connect to DTC from python and download data.

login string (tried a variety of alternatives here, including empty string which also works):
ta:DUXXXXXX h:127.0.0.1 p1:11099 p2:11097

script run is:
void main()
{
// Lookback = 0;
StartDate = 20190201;
EndDate = 20190315;
GapDays = 3; // detect gaps
Verbose = 2;
assetList("AssetsFix");
asset("EUR/USD");
assetHistory(0,1);
}

Please help!
Thanks a lot.


On Sierrachart the error is:

DTC client #27. 127.0.0.1 | Received logon request. | 2019-05-16 05:30:04
DTC client #27. 127.0.0.1 | Sent successful Logon response message to client. Username: _. Result text: Connected to SC DTC Protocol server. Service=interactive_brokers.trading|SymbolSettings=interactive_brokers.trading | 2019-05-16 05:30:04
DTCServer (1) | Socket gracefully closed by remote side. | 2019-05-16 05:30:04
DTCServer (1) | Close event error. Windows error code 10053: An established connection was aborted by the software in your host machine. | 2019-05-16 05:30:04 *
DTCServer (1) | Graceful close received or receive error. Shutdown complete. Closing socket now. | 2019-05-16 05:30:04





Attached picture sierrasettings.png
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 13:16

I also tried an older version, 1.96, same issue
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 13:33

I'm on a demo account, would that be the issue?

I don't have problems accessing dtc data through python
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/16/19 13:41

This might be an IB specific issue, but I cannot confirm this.

I tweaked the DLL based on your error log. Can you try it out?

See attachment.

Attached File
scplugin.zip  (100 downloads)
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 13:51

I tried it without IB, same issue, trying your plugin now.
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 13:56

works!

so the issue was that there were no historical orders?!

Thanks so much!

I wonder though what was different for me compared to other who haven't had this issue?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/16/19 14:08

Same issue on other brokers? This means Sierra Chart changed the DTC server again.

I tweaked it again, but this time the historical order requests syncs up the particular trade account. (Apparently they decided they require this field now.)

See attached.

Attached File
scplugin.zip  (107 downloads)
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 14:15

I was having this issue regardless of whether I was connected to IB or just using SC data service.

The first plugin you provided above fixed that.

I have an issue now, the plugin starts downloading all history for that instrument (3 gb for USDNOK) in sierrachart, while being stuck on "Data is being downloaded for a remote source. Download will start when this is done"

What's a good way of restricting 1) how much data SC will download off the web, and 2) more importantly, how much data gets transferred to Zorro, as this seems to be rather slow.

Thanks again!
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 14:26

The second plugin you provided recreated the problem!

Back to failed to confirm historical orders and Sync hist orders failure. Can't connect.


The first plugin you posted worked, and changed sync hist orders to warning only.

TradeAccount not set occurs on all versions of the plugin so far.


Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/16/19 14:37

Try the download script.

OK, well I might have time to debug the latter issue this weekend.
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/16/19 14:47

The second plugin is not able to connect, neither with the download script, nor with the script I posted.

The first plugin seems works, when I change to "trades" and remove the slash from the name, i.e. EURUSD instead of EUR/USD. At least SierraChart starts downloading data.
Posted By: h4h4rd

Re: Sierra Chart Plugin - 05/24/19 06:36

Hey is there any news regarding this? Again, the first plugin you sent works for me right now, the second one doesn't. For downloading that is, I haven't tried live trading. I can't imagine I'm the only one with this issue. Or do you believe there is something wrong with my setup?

Thanks,

h4h4rd
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/24/19 10:54

Sorry, I did not have time to look into it yet. I will provide an update when I get the time.
Posted By: bondo1

Re: Sierra Chart Plugin - 06/26/19 09:22

Hey Andrew, just wanted to say thanks for taking the time to write this dll, very much appreciated!

Secondly, I'm having the same problem as h4h4rd, but I'm trying to download historical prices from Bitmex - similar/same error messages showing up on Zorro (using Zorro S) and on Sierra. Was there any update to this?

Thanks again for taking the time to do all this!

EDIT: So I tried the dll you posted here; https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=477098#Post477098 ; and I can connect/retrieve data, gonna compare it to the data drawn as if I was just using Sierra. Also, I'll check the other
dll that h4h4rd said reproduced the issue.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 06/26/19 12:44

I do have a bugfix update, haven't gotten around to uploading it until now.

Attached is version 0.9.2.4.

Attached File
SierraChart_0_9_2_4.zip  (105 downloads)
Posted By: bondo1

Re: Sierra Chart Plugin - 06/26/19 23:43

Hey Andrew,

Thanks for the update! I just tried it out and no issues on my end with historical data now.

For anyone wondering, I'm using a demo of Sierra as provided by Sierra, but I had the Login fields set to Real when I downloaded price history, gonna try out what happens if I set it to Demo later tonight and re-download the data.
I personally use CQG and IQFeed, so I'll run it using both data sets to see if I get any issues and report back here.

Also, can't thank Andrew enough, thanks for keeping up with this man, big ups!

Posted By: bondo1

Re: Sierra Chart Plugin - 06/28/19 00:11

Just wanted to ask,

I think I read earlier in this thread that to download data, we should be using the symbols listed by Sierra. Using the new plugin, I just tried it with a couple of equity symbols and I'm getting no foreseeable problems. Gonna try it with some futures and cryptocurrency exchanges in a bit as well.

Couple of things;

If I wanted to test using a symbol from another data feed that isn't Sierra, say IQFeed, how would I/would I be able to do that with the plugin?
(E.g.

File > Data/Trade Settings > Select IQFeed
//
File > Find Symbol > [Input/Type out IQFeed Symbol]
//
Run Zorro download.c script > [Type IQFeed Symbol, not Sierra's Symbol] into the download widget]? )

I tried the above steps with LYFT and I was able to download the asset prices and display a price graph doing the above steps. I attached a picture as well.

My questions is, how do I know that I pulled from IQFeed and not Sierra's data feed? I'm assuming the message log from Sierra would let me know that (?) Here are some select messages that might help with the clarification;
-----------------------

// After selecting IQFeed and inputting LYFT as the symbol, I tried to chart the price in Sierra's software. In the message log I received notification that the data was from IQFeed

IQFeed | Starting real-time market data updates for: LYFT. ID: 3 | 2019-06-27 16:51:41.485
IQFeed | Requesting security definition data for: LYFT. ID: 3 | 2019-06-27 16:51:41.485
IQFeed | Using primary service for historical data for LYFT | 2019-06-27 16:51:41.485
Intraday data recording state for symbol LYFT is set to download 'Pending'. | 2019-06-27 16:51:41.485
Delaying start of download for LYFT | 2019-06-27 16:51:41.486
HD Request # 7 | Downloading Intraday chart data for LYFT to the file LYFT.scid. Service: dtn | 2019-06-27 16:51:46.214

// When I tried to download/chart the data using the plugin, I received this notification (I'm assuming this means that the data is being downloaded from IQFeed > since dtn = IQFeed, but just wanted to make sure : D)

Sent successful Logon response message to client. Username: _. Result text: Connected to SC DTC Protocol server. Service=dtn|SymbolSettings=dtn | 2019-06-27 16:52:52.195

-----------------------


And lastly, I think I read in the Zorro manual that imported data converts into UTC (correct me if I'm wrong), so the timestamps we see on the time scale are in UTC as well, even if I set my Sierra settings to my local time?


Attached picture Test1.png
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 06/28/19 02:24

Sierra Chart connects to one data service at a time.

You receive your time stamps in UTC, both via DTC and via all plugins. This is good because it is impossible to get wrong.
Posted By: bondo1

Re: Sierra Chart Plugin - 06/28/19 03:22

Hey Andrew,

Thanks for the clarifications!

As I was playing around with loading different asset data, I noticed that for Bitmex, it's skipping regular time periods, specifically the same weekend day every single week [See picture below].
I highlighted the areas of interest.

I'm using the download widget that runs from Zorro's included download.c script and after checking that the asset was stored in the Zorro History folder, I plot the price to check the data according to this script;

//Run to check price curve data

function run()
{
//Defining a 1 minute period
BarPeriod = 1;

//Defining the two assets used in spread calculation
asset("nameofasset");
var assetprice = priceClose() ;

//Plots
plot("assetprice",assetprice,NEW,BLUE);
}

Not sure if this is due to the plugin, or Sierra's side? Or do I need to do something to the download.c script from Zorro?

There's definitely weekend data, I loaded historical charts off Sierra using the same symbol and was able to display the entire history (plus, almost all cryptocurrency exchanges are 24/7 barring
downtimes.

**** Also, I checked BTC data off of Poloniex and received a similar result (regularly occurring, missing data that seems to be a weekend day), see second attached picture. I checked my Sierra settings and it's set to load weekend data.



Attached picture IncompleteLoad3.png
Attached picture IncompleteLoad4.png
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 06/28/19 11:53

But you’re plotting it with a Zorro script. By default, Zorro ignores weekends.

Download the Z History Editor from the Download section of the site and inspect the t6 files yourself. Then tell me what you see.
Posted By: bondo1

Re: Sierra Chart Plugin - 06/29/19 01:55

Hey Andrew,

I loaded the data up in Z History Editor and was able to see continuous data (including the weekends), I checked this on ES futures as well as some cryptocurrencies. Thanks again for the assist [I read up on the weekend bit after you posted about it, and for anyone who didn't know, I think the Zorro Manual > Time and Calendar Functions > BarMode should be helpful]. I'm gonna play with the code and try to get Zorro to chart all data (including weekends).

For anyone who's curious, I've been loading 1 minute data (so t6), but I'll mess around with downloading/checking tick data and report how that goes.

Also, gonna code a couple simple strategies/phantom trade them to see how the plug-in works for trading execution and will also post results - Thanks again to Andrew, the update is going swimmingly!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/18/19 14:03

Just a quick update - I have plans to spin off the sockets backend into its own DLL library with a C API, since it depends on Boost. Maybe even make the library directly accessible from Zorro.

This way, the plugin itself would not have a direct dependency on boost. This will make code management much easier.

Also, I dislike my current multi-threaded design - there's no need for it. I can call windows handles using sockets, so networking can still be asynchronous while still single-threaded.

I can afford to be a perfectionist since there's no deadline, and this is not a funded project. But at the end of the day, I'm trying to make this repository as easy to maintain as possible.
Posted By: bondo1

Re: Sierra Chart Plugin - 09/28/19 23:56

Hey man, just wanted to say thanks for continually updating us on this, your work is very much appreciated! I have yet to deploy live capital using your plugin (been working through a bunch of coding examples), but I'm excited to try it out
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 09/29/19 13:49

If it works well for demo/paper trading, then it should work well for live trading. Either way, please let me know if you encounter any issues.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/17/19 16:16

I'm thinking about giving the Sierra Chart a try for historical data. I'm happy with my own broker plugin for live trading and live pricing feed, but not quite happy with my current historical EOD and M1 data feed setup. Can I use the Sierra Chart plugin for historical data ONLY -- basically what assetHistory() would provides. Thanks.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/17/19 17:56

Yes.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/17/19 19:00

Hi Andrew. How would I do that?

I got SC working with a simple script and it's downloading D1 and M1 historical data perfectly. I had to point the account to SierraChart.dll in Accounts.csv, but then I cannot specify my own plugin for my live trading.

Ideally, I would like my use of SC to be something like assetHistory(FROM_SC,xxx), while all trading and real-time data in Live mode goes through my broker plugin.

Thanks again for your work on the Sierra Chart plugin!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/17/19 19:27

It's not set up to do that in the way you describe.

Currently, you can use the broker arbitrage method, which should meet your needs:
https://zorro-project.com/manual/en/brokerarb.htm
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/18/19 23:25

I work with a lot of assets, so that method probably wouldn't work.

How about if I sponsor op Group to create a version of assetHistory() for Sierra Chart by wrapping your DLL.

Any objections?


Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/19/19 00:07

It’s in beta, and I planned on rewriting the backend in the near future.

In other words, don't do it.

For your own purposes, you can instead dynamically load the DLL and invoke BrokerOpen, BrokerLogin, and BrokerHistory2, all entirely from a Zorro script.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/19/19 01:01

Great! Look forward to it. Any idea on ETA?

My need isn't urgent. My multiple data provider approach via Zorro script, while not elegant, is working so far. Sierra Chart has all the data I'm looking for a very reasonable price, an would be an elegant solution.

Thanks again for your hard work on this!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/19/19 01:47

No ETA, just delays. wink
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/20/19 20:46

Got some time today, so I thought I'd give it a try. It crashed in BrokerLogin. I did use my real username/login.

I've never loaded a DLL function before in. Any guidance would be appreciated

Code

#include <stdio.h>
#include <default.c>
#include <contract.c>

int __cdecl BrokerOpen (char*, FARPROC, FARPROC); 
API(BrokerOpen,Plugin\\SierraChart)

int __cdecl BrokerLogin (char*, char*, char*, char* );
API(BrokerLogin,Plugin\\SierraChart)

int __cdecl BrokerHistory2 (char* , DATE , DATE , int , int , T6* );
API(BrokerHistory2,Plugin\\SierraChart)


void run()
{

   StartDate = 20190101;
   EndDate = 20191031;
   Verbose = 7|DIAG;

   String temp = "AAPL";
   History = "_SC.t6";

   BokerOpen("SierraChart",0, 0);
   _POS("11111");

   BrokerLogin ("USERXXX", "PWDXXXX", "Demo", "0000");
   _POS("22222");

   T6* myT6;
   int items = BrokerHistory2 (temp, dmy(20190101), dmy(20191031), 1440, -1, myT6);
   FILE *fp;
   string fname = strf("History\\%s%s",temp, History);
   fp = fopen(fname, "wb");
   fwrite(myT6 , items , sizeof(myT6) , fp );

   asset(temp);    

   var Price = priceClose();
   printf("price = %.4f\n",Price);

}


Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/21/19 00:40

Why did you hand over null pointers to fpError and fpProgress? I bet my plugin crashes on calling BrokerError and BrokerProgress. (Obviously, there's no reason to check if the pointers are valid tongue )

Also, char* Accounts is an output, so why are you supplying a string literal?

Mind you, my plugin was designed to be used by Zorro. laugh
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/21/19 01:29

Since I am not Zorro, I don't know how to pass the correct arguments :-)

So, what should I put in as arguments for pointers to fpErrpr and fpProgress?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/21/19 03:35

Make up your own function and pass a function pointer. You know what to do, right? Just do what the manual says the functions do.

Also, your BrokerHistory2 call is wrong. You don't even know how much memory you need - you need to call GET_MAXTICKS, and then call malloc.

~~~~ IN GENERAL~~~~
Any time that the plugin outputs memory (such as T6 ticks or strings), the caller is responsible for supplying the memory. You have supplied memory zero times. Fix this.

If you're not sure whether or not it's an output, read the manual.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/21/19 16:52

Hi Andrew, my programming skills are not close to your level by any measure. This DLL loading thing is completely new to me.

I've read the manual and continued to hack at it and still doesn't work. I even copied SierraChart.dll to SierraChart2.dll, put in the root Zorro directory, and tried a different way of loading the DLL.

I'm getting a compile error about wrong type CONV:POINTER::DOUBLE at the BrokerOpen line. I don't think I'm passing the function pointers in correctly. I tried my best with my rusty skills. Here's my stripped down code. I'll worry about BrokerHistory once I get pass BrokerOpen :-)

Any thoughts?

Thanks.

Code
#include <stdio.h>
#include <default.c>
#include <contract.c>

int __cdecl BrokerOpen (char*, FARPROC, FARPROC); 
BrokerOpen = DefineApi("SierraChart2:BrokerOpen");

int __cdecl BrokerLogin (char*, char*, char*, char*);
BrokerLogin = DefineApi("SierraChart2:BrokerLogin");

int __cdecl BrokerHistory2 (char* , DATE , DATE , int , int , T6* );
BrokerLogin = DefineApi("SierraChart2:BrokerHistory2");

int BrokerError(char* message) {
	printf(message);
}

int BrokerProgress(DWORD progress) {
	printf("stuff");
}

void run()
{

   StartDate = 20190101;
   EndDate = 20191031;
   Verbose = 7|DIAG;

	string temp = "AAPL";
	History = "_SC.t6";

	string temp1[100];
	BrokerOpen(temp1, &BrokerError, &BrokerProgress);

	char retAcct[1024];
	BrokerLogin ("XXXXX", "XXXXXX", "Demo", retAcct);

}


Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/21/19 17:30

Try casting your pointers to void* or FARPROC
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/21/19 18:02

No dice. same syntax error.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/21/19 18:16

Well, maybe I can take a peek at it this weekend evening.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/21/19 19:18

Many thanks!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/22/19 02:52

This should get you started. When the connection fails, it crashes. I don't know why, and I don't care. tongue Bon appetit.
Code
#include <default.c>

int __cdecl BrokerOpen (char* Name, void* fpError, void* fpProgress); 
API(BrokerOpen,SierraChart)

int __cdecl BrokerLogin (char* User, char* Pwd, char* Type, char* Accounts);
API(BrokerLogin,SierraChart)

// int __cdecl BrokerHistory2 (char* , DATE , DATE , int , int , T6* );
// API(BrokerHistory2,SierraChart)

int fError(char* message) {
	printf("\n%s",message); 
	return 0;
}

int fProgress(DWORD prog) {
	return wait(0);
}

void run()
{
	if(!Test){quit("#Only [Test] mode supported.");return;}

	char outName[64],outAccount[1024];
	memset(outName,0,64);
	memset(outAccount,0,1024);
	
	printf("\ncheckpoint a");
	int o = BrokerOpen(outName,(void*)fError,(void*)fProgress);
	printf("\ncheckpoint b, outName: %s, o: %d",outName,o);
	BrokerLogin ("","","Demo",outAccount);
	printf("\ncheckpoint c");
	
	quit("#Done.");
}
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/22/19 21:00

You da man!

Looks like it connected!

Let me see if I can figure the rest out :-)
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/25/19 05:02

Andrew, looks like it's working pretty well in TEST or TRADE mode with multi-asset strategies too. I will post the script here when I am comfortable there are no bugs.

However, I noticed that the volume data looks suspicious. It ramps up, zero, ramps up, zero. See the attached chart for AAPL.

Do you see this in your charts as well?

Attached picture AAPL-chart.PNG
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/25/19 12:59

It’s by design. Now you can calculate volume per bar by calculating the difference between them. Plus it matches the volume you get in [Trade] mode.

Did you know that if you have M1 data, and you use a larger BarPeriod than 1, then Zorro discards all of the volume data in between each bar? For example, if you set BarPeriod to 5, Zorro discards four fifths of your volume data.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/25/19 16:42

I see, makes sense now.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/25/19 20:47

Hi Andrew, your timestamps appear to be off.

See the attached image comparing the EOD SPX data from Sierra Charts vs. Yahoo.

I think you mean to timestamp the bar just before midnight, but because it's stamped as midnight, the bar data is actually assigned to the next day. I noticed this on 1-minute bars as well yesterday, but the problem at 1-minute resolution was annoying but not a big enough deal. But on daily bars, that's a pretty big deal.

Your thoughts?

Attached picture SPX-diff.PNG
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/25/19 21:21

That's a bug. I'll have a fix later this week.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/25/19 21:42

Great! Thanks!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/26/19 00:12

Here it is. Can you give it a spin?

Attached File
SierraChart_0_9_3_0.zip  (96 downloads)
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/26/19 01:21

The EOD bars appear to be correct, the M1 bars do not. You SC data is 1 minute ahead.
It'd be nice if you fix that too so the bar charts look the same as conventional trading platforms.

BTW, since you are putting UTC timestamps in the T6 files, your DLL should also adjust for daylight savings time. Or is the sierra chart data already in UTC time? As you know the UTC offset is different when in DST.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/26/19 01:55

DTC is already UTC, so there is no adjustment.

Zorro’s T6 specification is that intraday historical time stamps should align with the end of the period. Whereas for daily, the stamp should be at midnight UTC at the very beginning of the day. I am operating in accordance with this specification.
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/26/19 14:01

Good to know it's already in UTC time.

I read the manual on timestamps again and it's a bit confusing. So I asked support this question and they gave this answer:

Question:
When creating an M1 t6 file, what is the timestamp of an M1 bar that covers the period from 11:16:00 - 11:16:59. Is it 11:16:00 or 11:16:59 or 11:17:00 ?

Answer:
it should be 11:16:59.

Unless History.c is displaying it wrong, it looks like your timestamp for this example is 11:17:00.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 11/26/19 14:27

You mean to tell me that when you download historical data from the official oP Group plugins that they actually have the seconds offset by one before the end of the period?
Posted By: SBGuy

Re: Sierra Chart Plugin - 11/26/19 14:32

No. I'm using my own data from an ActiveTick source.

It's probably because they are time-stamping theirs at the beginning of the M1 bar.

You're probably right about my question being asked incorrectly.
Posted By: wzschultz

Re: Sierra Chart Plugin - 12/05/19 01:31

This is a problem that I encountered early on with the way SierraChart's stores its bars.

For backtesting, one typically wants the [A,B) interval or (A,B] interval covered by the bar to be designated by time B.
That way it is less likely that you will peek into the future during your backtest. At the time B, you are allowed
to observe bars labelled with B or before.

However, because SierraChart is repeatedly re-drawing the bar on the chart as it it is forming, they want to publish the bar
to the internal ACSIL DLL code with a more constant time stamp. At least this is how I rationalize the observation
that they use the "A" timestamp (beginning of bar) everywhere instead of the "B" end-of-bar timestamp.
Posted By: SBGuy

Re: Sierra Chart Plugin - 12/20/19 19:04

Thanks to Andrew's help, I'm pretty happy with my version assetHistorySC() that gets historical data from Sierra Charts like the standard assetHistory() function in Zorro. It works in Test, Live PRELOAD. It initially loads at least 6 years of EOD data, and as much M1 data as you have in your Sierra Charts setting. Then it adds new data to the T6 file as needed (usually in Live mode) on subsequent runs.

The only problem is Zorro crashes when you close the Zorro instance. I think it has to do with dangling DTC sockets to the Sierra Charts program when the Zorro instance is closed. Aside from having to click and clear the crash message, I haven't seen any problems.

I like using SC because it's a single source for high quality historical EOD and M1 data, with dividend adjusted prices, for all kinds of symbols starting at $26/month. Before SC, I had to piecemeal my data sources depending on what my strategy requires.

Here is the code. Use and modify at your own risk :-)

Code

int __cdecl BrokerOpen (char* Name, void* fpError, void* fpProgress); 
API(BrokerOpen,Plugin\\SierraChart)
int __cdecl BrokerLogin (char* User, char* Pwd, char* Type, char* Accounts);
API(BrokerLogin,Plugin\\SierraChart)
int __cdecl BrokerHistory2 (char* theasset, DATE tStart, DATE tEnd, int nTickMinutes, int nTicks, T6* ticks);
API(BrokerHistory2,Plugin\\SierraChart)

int fError(char* message) {
	//printf("\n%s",message); 	// uncomment to display brokerError messages from SC
	return 0;
}
int fProgress(DWORD prog) {
	return wait(0);
}

function connectToSierraChart()
{
	//-----------------------------------
	//  	connect at INITRUN 
	//-----------------------------------
	if (!Init)	return 0;

	char outName[64],outAccount[1024]; memset(outName,0,64); memset(outAccount,0,1024);

	int o = BrokerOpen(outName,(void*)fError,(void*)fProgress);
	if (o!=2)	{
		printf("%s BrokerOpen FAILED!\n", outName);
		quit("#quit...");
	}
	o=BrokerLogin ("","","Demo",outAccount);	// don't need login credentials here for localhost SC
	if (!o)	{
		printf("%s BrokerLogin FAILED!\n", outName);
		quit("#quit...");
	}

}


int assetHistorySC(string theAsset)
{
	if (!Init)	return 0;

	int barperiod = BarPeriod;
	//----------------------------------
	//		setup start and end dates
	//----------------------------------
	int M1back = (LookBack*BarPeriod)/(7*60); 	// days of lback
	int DailyLBack = (LookBack+2*UnstablePeriod);
	int start = min(ymd(wdate(NOW)-6*365), ymd(dmy(StartDate)-2*ifelse(barperiod<1440,M1back, DailyLBack)));	//6 years or more
	int end;
	

	string t6Out[100];
	if (barperiod<1440) {
		sprintf(t6Out,strf("History\\%s%s",theAsset,History));
		barperiod = 1;
		end = ymd(wdate(NOW)+1);	// +1 to get most current M1 bar
	}
	else	{
		sprintf(t6Out,"History\\%s.t6",theAsset);
		end = ymd(wdate(NOW)-1);	 // -1 to get up to yesterday
	}
	
	//-------------------------------------
	// check to see if exiting t6 is in range	
	//-------------------------------------

	if (file_length(t6Out))	{
		int records = dataLoad(33,t6Out,7);	
		int fileStart = ymd(dataVar(33,-1,0));	
		int fileEnd = ymd(dataVar(33,0,0));
		
		//printf("fileStart=%d  fileEnd=%d\n",fileStart, fileEnd);
			
		// if Test or Train, don't download if data in range
		if (	(!Live && EndDate<=fileEnd && barperiod==1440)
			|| (Live && end<=fileEnd && barperiod==1440)
			)	
		{
			printf("\n[%s] historical data up to date... no download\n");
			dataNew(33,0,0);
			return 0;	// range already inside
		}	
		// if not enough data, its probably a newly listed stock/asset with limited history
		if (start<fileStart)		
			printf("\nWARNING:  historical data for [%s] may not be early enough\n",theAsset);

		// set start to last date in file, to upate to current date
		start = fileEnd;		

	}

	printf("Downloading data [%s]... Start=%d   End=%d\n", theAsset, start, end);

	//----------------------------------
	//			index symbol conversions
	//----------------------------------
	if (theAsset == "SPX") theAsset = "$SPX";
	if (theAsset == "DJI") theAsset = "$DOWI";	
	if (theAsset == "INX") theAsset = "$INX";
	
	//----------------------------------
	//				get data		
	//----------------------------------

	if(once(Init)) connectToSierraChart();

	int reqbars = (int) (dmy(end) - dmy(start))*ifelse(barperiod==1440,1,1440);	// D1 or M1 data only 
	//printf("SC: items allocated = %d\n",reqbars+10);

	T6* myT6=malloc((reqbars+10)*sizeof(T6));
	int items = BrokerHistory2 (theAsset, dmy(start), dmy(end), barperiod, -1, myT6);	
	printf("SC: items received = %d\n",items);
	if (!items)	{
		printf("ERROR:  No historical data returned from SierraChart... maybe out of data cache range.\n");
		free(myT6);
		quit("#quit");
	}

	FILE *fp;
	string tempT6 = strf("History\\temp-%s.t6",theAsset);
	fp = fopen(tempT6, "wb");
	fwrite(myT6 , sizeof(T6), items, fp );
	fclose(fp);
	free(myT6);
	
	dataLoad(33,tempT6,7);	// load it into 33
	dataSort(33);				// make sure it's sorted 
	
	//---------------------------------
	//	pre-pend to existing	t6
	//---------------------------------
	if (file_length(t6Out)>0)	{
		dataLoad(22,t6Out,7);
		dataMerge(22,33);		
		dataSave(22,t6Out);
		dataNew(22,0,0);	
	}
	else	{
		dataSave(33,t6Out);
		dataNew(33,0,0);	
	}

	file_delete(tempT6);
	return 0;
}


Posted By: JamesHH

Re: Sierra Chart Plugin - 02/07/20 04:06

Originally Posted by AndrewAMD
I am proud to announce that the Sierra Chart Plugin is now in BETA, and so I am in need of some testers.

Documentation and source code is here:
https://github.com/AndrewAMD/SierraChartZorroPlugin

The latest DLL can be found here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/tree/master/releases

I am here to answer any questions and fix any bugs you may have found. Please let me know.

Thanks,
Andrew


Hello Andrew,

I am very interested in using your plugin!

However, I want to make sure it will be suitable for what I am trying to do before spending time on it.

I want to use it for market making on the BitMEX exchange. So it would be placing passive limit orders using POST_ONLY orders (I believe the POST_ONLY flag can be set in the SC plugin).

This also needs to be low-latency. Can Zorro react with your plugin (place orders upon receiving new market data), say within 1ms?

Support at SC told me that their DTC server is suitable for low-latency trading.

Thanks
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 02/07/20 15:12

My setup is low-latency, since it implements SET_HWND for immediate reactions to incoming data. However, I have not done any latency tests, so I cannot comment on whether or not it is sub-ms.

(Note: This project will remain in beta until I rewrite the backend.)
Posted By: JamesHH

Re: Sierra Chart Plugin - 02/07/20 17:59

Originally Posted by AndrewAMD
My setup is low-latency, since it implements SET_HWND for immediate reactions to incoming data. However, I have not done any latency tests, so I cannot comment on whether or not it is sub-ms.

(Note: This project will remain in beta until I rewrite the backend.)


That still sounds very good. Do you know if Zorro runs at that low latency in general (aside from your plugin)?

Also, there is no problem with sending limit orders to the BitMEX exchange? I read something about limit orders earlier in this thread but I could not understand the discussion.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 02/07/20 19:41

Originally Posted by JamesHH
Also, there is no problem with sending limit orders to the BitMEX exchange? I read something about limit orders earlier in this thread but I could not understand the discussion.
Not sure what you mean. Limit orders are supported.

If BitMEX has problems, that has nothing to do with my plugin. Remember, this plugin works with Sierra Chart.

When in doubt, test on a paper account and in Sierra Chart's simulation mode.
Posted By: JamesHH

Re: Sierra Chart Plugin - 02/08/20 06:55

Originally Posted by AndrewAMD
Originally Posted by JamesHH
Also, there is no problem with sending limit orders to the BitMEX exchange? I read something about limit orders earlier in this thread but I could not understand the discussion.
Not sure what you mean. Limit orders are supported.


At the start of the thread, Zheka asked "- does the plug-in support true limit orders?" and then: "1) You mean, by specifying a TakeProfit var in the enterLong, the plug-in will send a real limit order to the book, right?" and you replied "1) The plugin is NFA mode (orders) only, so this depends on how Zorro handles pool trades." I don't understand what NFA mode does, but as long as I can send limit orders all is good.

Quote

When in doubt, test on a paper account and in Sierra Chart's simulation mode.


Yes, I plan to test it. BitMEX also has a testnet that SC can connect to, where you can paper trade with the BitMEX API.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 02/11/20 14:39

In Zorro, you place "trades". But "trades" are a high-level interface for order placement. When a plugin is "NFA mode", it means the broker only understands orders.

This does not change the fact that limit orders are supported. Just verify that it behaves as you expect.
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/13/20 00:23

I attempting to use the plugin in Trade mode to get live prices, but after Zorro connects to the server I am receiving a flood of unknown message type errors:

!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
!1: Received unknown msg type: 140
[Stop] at 00:16
Logout.. ok


This is using the following simple script:

void run() {
BarPeriod = 1;
LookBack = 10;

assetAdd("XBTUSD-BMEX");
asset("XBTUSD-BMEX");

vars Prices = series(price(0));

print(TO_WINDOW, "\nPrice: %f", Prices[0]);
}


Am I doing something wrong?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/13/20 00:49

It's probably because Sierra Chart changed their DTC protocol. Let me look into this. (Maybe by Sunday? My schedule is bad.)
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/13/20 02:41

Originally Posted by AndrewAMD
It's probably because Sierra Chart changed their DTC protocol. Let me look into this. (Maybe by Sunday? My schedule is bad.)



Great. Thank you.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/13/20 11:05

It looks like they added some new market depth messages. I'll need to add some more message handling.

If you don't need market depth, you can probably ignore it until I have a fix.
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/15/20 01:40

Originally Posted by AndrewAMD
It looks like they added some new market depth messages. I'll need to add some more message handling.

If you don't need market depth, you can probably ignore it until I have a fix.


OK. I think it works.

This is not specific to the plugin, but I have never used Zorro in Trade mode before. It looks like I will only get price updates at the end of each 1 minute bar, with BarPeriod = 1? So I need Zorro-S to get a stream of tick data?

Also, about market depth, eventually I would need it. Are the bid/ask prices real? I mean that Spread is always 0.5, so does that mean it is really 0.5 on the exchange?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/15/20 02:44

The spread you get in Trade mode is real. Whereas the spread you set in your asset list is strictly used for Test and Train mode to emulate a spread.

Quote
It looks like I will only get price updates at the end of each 1 minute bar, with BarPeriod = 1? So I need Zorro-S to get a stream of tick data?
I believe tick() works for both Zorro and Zorro S. But you can only use BarPeriod 1 or greater in free Zorro. So if you're only using run() and BarPeriod is 1, then you only get price updates once per run(), which is once per minute.
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/17/20 05:57

Originally Posted by AndrewAMD
I believe tick() works for both Zorro and Zorro S. But you can only use BarPeriod 1 or greater in free Zorro. So if you're only using run() and BarPeriod is 1, then you only get price updates once per run(), which is once per minute.


I didn't know about tick(). It does seem to be working with free Zorro.

My understanding of the manual is that in tick(), priceClose() gives the ask quote. So how does one get the trade price when a trade occurs?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/17/20 12:45

This is the Sierra Chart Plugin thread. Please stay on topic.

These sorts of questions belong on "Starting with Zorro":
https://opserver.de/ubb7/ubbthreads.php?ubb=postlist&Board=49&page=1
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/18/20 01:20

Apologies, I have reposted my question there.

Btw, I assume obtaining market depth with be specific to the Sierra Chart Plugin (since I didn't see any documentation on this in the manual), but I guess you will explain this once the plugin adds that functionality.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/18/20 11:20

Originally Posted by JamesHH
I assume obtaining market depth with be specific to the Sierra Chart Plugin (since I didn't see any documentation on this in the manual), but I guess you will explain this once the plugin adds that functionality.
Not true, it is standard.
https://manual.zorro-project.com/ordercvd.htm
https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=474654#Post474654
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/18/20 16:47

Oh, I misunderstood. I thought you hadn't implemented market depth yet, but I assume you are saying that recent changes in Sierra Chart broke it.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/18/20 16:50

Correct. My plugin "talks" to the sever. They changed the "language". laugh
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/21/20 23:28

Sorry, but so far I am not finding this plugin to be usable. Here are some issues:

1) After starting Zorro in Trade mode, the status remains red for about 2:45 minutes before the script starts doing anything. This makes it very time consuming to debug scripts.

2) The flood of megabytes of error messages makes it that much more difficult to try and see what is going on.

3) At one point I was receiving quote updates in the tick() function. Now there are no updates showing in the log (even though I know the quotes have updated on SC). Also when I try to stop my script zorro keeps running after I select Yes to stop running the script.

I made some changes to my script and then tried to revert them to what seemed to be working before. Is there some error in my script that could be causing the problems?


vars bid, ask;

void run() {
resf(BarMode, BR_WEEKEND);

BarPeriod = 1;
TickTime = 1;
LookBack = 2;

//bid = series(0, -LookBack);
//ask = series(0, -LookBack);

assetAdd("XBTUSD-BMEX");
asset("XBTUSD-BMEX");
}

void tick() {

//var new_ask = priceClose();
//var new_bid = new_ask - Spread;

/* shift(ask, new_ask, LookBack);
shift(bid, new_bid, LookBack);

if (bid[0] != bid[1]) print(TO_WINDOW, "\nBid changed");
if (ask[0] != ask[1]) print(TO_WINDOW, "\nAsk changed");
*/
print(TO_WINDOW,
"\n%2i:%2i:%2.3f => Bid: %f, Ask: %f", hour(), minute(), second(),
//new_bid, new_ask);
priceClose() - Spread, priceClose());
//print(TO_WINDOW, "\nPrice: %f", Prices[0]);
}
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/21/20 23:49

I'll look into this tomorrow.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/23/20 01:00

There appears to be an error/failure under unknown circumstances. I will not be fixing this because 1) Sierra Chart changed the protocol on me, and 2) I planned on overhauling the plugin in the first place. The overhaul is low on my priority list because I'm working on a much bigger project right now.
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/23/20 02:44

Originally Posted by AndrewAMD
There appears to be an error/failure under unknown circumstances. I will not be fixing this because 1) Sierra Chart changed the protocol on me, and 2) I planned on overhauling the plugin in the first place. The overhaul is low on my priority list because I'm working on a much bigger project right now.



Fine, but I guess that means Sierra Cart is unusable with Zorro in the meantime.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/23/20 13:35

I do not know of any other plugin implementations of Sierra Chart for Zorro. If you know C++ and sockets, you can always implement your own.

The other issue is that my debugger said that the crash was internally in Zorro.exe (not in the scope of my code), so that makes it a wee bit harder to debug in the first place. Even if the crash is elsewhere, my plugin can still be the culprit. And then if Zorro is the culprit, I might need to file a convoluted bug report. This kind of stuff is very time-consuming.
Posted By: JamesHH

Re: Sierra Chart Plugin - 03/23/20 19:23

Originally Posted by AndrewAMD
I do not know of any other plugin implementations of Sierra Chart for Zorro. If you know C++ and sockets, you can always implement your own.

The other issue is that my debugger said that the crash was internally in Zorro.exe (not in the scope of my code), so that makes it a wee bit harder to debug in the first place. Even if the crash is elsewhere, my plugin can still be the culprit. And then if Zorro is the culprit, I might need to file a convoluted bug report. This kind of stuff is very time-consuming.



I see what you mean by time consuming. I'm not sure if you have access to the Zorro source code, but they should give it to you if you don't.

My plan for now is to write a basic bot that connects directly to the Sierra Chart DTC server. I have done socket programming in C++, but I will not attempt to write a Zorro plugin, at least for now.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 03/23/20 19:35

I do not have access to the source.
Posted By: JesseL

Re: Sierra Chart Plugin - 04/24/20 10:01

Hi Andrew,

I am new to both Zorro and your plugin, and just started trying the plugin for simulation trading. My Sierra subscribes to their exchange data and order routing service. Two questions here.

1. Can I use Sierra's special orders with Zorro in trading mode, like Limit Chase or OCO orders?

2. In Zorro's trading mode, it looks Zorro subscribes to the tick data from Sierra, and builds the 1M bar by itself. Because of a little bit latency, the bar's OHLC prices are not exactly aligned with Sierra's historical data on the chart. I compared them by setting PRICETYPE to 2. It's more significant in night session. Is it possible to directly subscribe to Sierra's bar data in trading mode? I understand assetHistory is only for initialization.

Thank you!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/24/20 11:50

Quote
1. Can I use Sierra's special orders with Zorro in trading mode, like Limit Chase or OCO orders?
Although these order interfaces are available via the DTC server, they do not jive very well with Zorro's high level trading interface, so it does not make much sense for me to program this into the plugin.

Quote
2. In Zorro's trading mode, it looks Zorro subscribes to the tick data from Sierra, and builds the 1M bar by itself. Because of a little bit latency, the bar's OHLC prices are not exactly aligned with Sierra's historical data on the chart. I compared them by setting PRICETYPE to 2. It's more significant in night session.
It's not latency. The historical timestamps have different meanings in Zorro than in Sierra Chart. Zorro defines a bar by end-of-period, whereas Sierra Chart supplies the timestamp of the very first tick received per bar. The plugin does the translation for you.

Quote
Is it possible to directly subscribe to Sierra's bar data in trading mode? I understand assetHistory is only for initialization.
What Sierra Chart supplies via the DTC server is what you get.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 05/27/20 01:40

I fixed a struct alignment issue. New version of DLL can be found here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/raw/master/releases/0.9.3.2/SierraChart.dll
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/18/20 01:20

Hi Andrew,

I've been enjoying using SierraChart for all my historical data needs, but I don't use it for trading.

I would like to follow up this post regarding the way you handle historical volume.
https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=478669#Post478669

I started to using daily volume in my strategy and your continuous summing of the bar volume is causing a bit of a challenge. I'm wondering if I can convince you to not do that for the follow reasons:

1. It is not consistent with the way Zorro create historical .t6 files from other sources such as YAHOO etc. EOD .t6 files have only daily volume in the volume field.

2. The raw data in the SierraChart Data folder contains only daily volume for EOD data, not continuously summed up.

Your thoughts?

Thanks.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 07/18/20 14:42

I might have time to change the volume behavior in a week or two.
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/18/20 14:47

Many thanks!
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 07/18/20 18:24

Version 0.9.3.3 for download here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/raw/master/releases/0.9.3.3/SierraChart.dll

Now the default behavior is to do standard per-bar volume, no session calculations.

I didn't test it due to lack of time.
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/18/20 21:00

Awesome! Thanks Andrew for the quick turnaround. EOD data looks good and works well.
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/22/20 15:23

Thanks again Andrew. By the way, there's just one minor annoyance with using SC as a historical data source only.

Related post: https://opserver.de/ubb7/ubbthreads.php?ubb=showflat&Number=478796#Post478796

Whenever I exit out of Zorro, windows gives a Zorro crash modal message. I think you mentioned this was expected. Is this something that the Zorro developers can easily fix, or is this something in your DLL?

It think it may have something to do with the dangling network connection to the SC application?

Is there a function to gracefully disconnect from SC or unload the DLL after I've received the historical data? I only use SC in the beginning of a trade/test session.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 07/22/20 15:34

This is because the plugin was designed to operate as a Broker Plugin and not for users to dynamically reload DLL's for direct API access, so this creates side effects.

I have no plans on fixing this, because my work will go out the window once I revamp the plugin.
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/22/20 15:47

I see. So it is something that only you can fix inside the DLL and not something that Zorro developers can do to "clean up" some dangling this or that?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 07/22/20 16:11

Maybe it's because my plugin is multi-threaded? Anyways, I have plans to gut out the multithreading, because it's now 100% unnecessary.
Posted By: SBGuy

Re: Sierra Chart Plugin - 07/22/20 16:23

Ah yes, that's probably it.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 08/11/20 22:41

Version 0.9.3.4. Bug fix: Zorro thread is not blocked correctly when orders are placed.
https://github.com/AndrewAMD/SierraChartZorroPlugin/raw/master/releases/0.9.3.4/SierraChart.dll
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/07/21 03:42

I am proud to announce that the Sierra Chart plugin has been completely overhauled and has been released!

Basically, the plugin no longer depends on Boost. The plugin is both single-threaded and asynchronous (SET_HWND is supported). GET_FUTURES is now supported for certain Sierra Chart services. All dependencies have been included in the Github repo.

Version 1.0.0.1 can be downloaded here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/releases/tag/v1.0.0.1

The documentation is completely different. Read up over here:
https://github.com/AndrewAMD/SierraChartZorroPlugin
Posted By: SBGuy

Re: Sierra Chart Plugin - 04/07/21 13:18

Thanks for the overhaul! I look forward to testing it.

As you may recall, I use it strictly for historical data download/preload. How have you change things with regards to using your DLL as a replacement assetHistory() function while trading and real-time data is through another broker plugin?
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/07/21 13:37

Originally Posted by SBGuy
I look forward to testing it.
Please do.
Originally Posted by SBGuy
How have you change things with regards to using your DLL as a replacement assetHistory() function while trading and real-time data is through another broker plugin?
Status has changed from "it works" to "it works (and probably faster too)".

I might otherwise add a standalone brokercommand so that you don't have to manually load the DLL.
Posted By: SBGuy

Re: Sierra Chart Plugin - 04/07/21 15:24

I don't think brokerCommand will help me because I get real time data and trade via a different broker plugin, selected from the Zorro dropdown list.

Just ran the new DLL and it seems to work fine.

Except that I had to change the default port sequence to: 1099, 11097, and 11098

The old sequence was 1099, 11098, and 11097 before for Listening Port, Historical Data Port, and the TLS Historical Data Port.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/07/21 15:42

Good. Let me know if there are issues.
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/10/21 19:46

Bugfix update, version 1.0.0.2 can be downloaded here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/releases/tag/v1.0.0.2
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/16/21 13:21

Bugfix update, version 1.0.0.3 can be downloaded here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/releases/tag/v1.0.0.3
Posted By: jbhunter

Re: Sierra Chart Plugin - 04/21/21 12:39

I am attempting to get some historical data from SierraCharts using the plugin. I have been successful for daily bars just using the zorro assetHistory(). However I am attempting to use SBGuy's assetHistorySC function posted earlier to get intraday historical data for some equities. I have set BarPeriod=1. Function below and some example output. The data bars received is is only going back about 6 months or so. It always seems to get around 50k items from Sierra. I have set Global Settings...Day/Trade Service Settings...Common Settings...Maximum Historical Intraday Days to Download to 1825 (5 years). I have noticed that I actually see data in SierraCharts go back much further, so I believe the data is there in the scid file from Sierra, but I cannot get it all to zorro.

Any thoughts? I believe the right parameters are called in the BrokerHistory2. Could this be another SierraCharts setting required? Any help would be appreciated.

A line from the Sierra log
Code
HD Server | HD Request #1 | Socket: 2 | Symbol: BRX | ServiceCode:  | RecordInterval: 60 | MaximumDaysRequested: 0 | Start date-time: 1999-12-31  00:00:00 | ClientReqID: 1 | Username: _. | Thread:7384 | 2021-04-21  08:17:44.869



Zorro Output
Code
 StartDate=0 EndDate=2021  NumYears=3  BarPeriod=1
Cmd=C:\Users\User\Zorro\Zorro.exe -h -stay -run Sierra_GetAsset_BC.c
 asset=BRX
Downloading data [BRX]... Start=19991231   End=20210422

#1: Launching thread...
#1: Trying endpoint 127.0.0.1:11099 (TLS 1.2)
1: Connected to SC DTC Protocol server. Service=scdataallservices|SymbolSettings=scdataallservices
1: WARNING: Trading not supported
1: TradeAccounts detected: Sim1
1: Using: Sim1
1: Warning: no balance confirmed for this account!
1: 0 historical orders detected.
1: 0 open orders detected.
1: 0 open positions detected.SC: items allocated = 11207530
BrokerHistory2 (BRX, 19991231, 20210422, 1, -1, myT6)

#2: Launching thread...
#2: Trying endpoint 127.0.0.1:11097 (TLS 1.2)
#2: Logon successful.
2: Data is being downloaded from a remote source. Download will start when this is done.
#2: Joining thread...SC: items received = 50821
t6 = History\temp-BRX.t6
LOADED TEMP
Saving to, C:\Users\User\Zorro\History\Sierra\BRX.t6




Code
int assetHistorySC(string theAsset)
{
	if (!Init)	return 0;

	int barperiod = BarPeriod;
	//----------------------------------
	//		setup start and end dates
	//----------------------------------
	int M1back = (LookBack*BarPeriod)/(7*60); 	// days of lback
	int DailyLBack = (LookBack+2*UnstablePeriod);
	int start = min(ymd(wdate(NOW)-6*365), ymd(dmy(StartDate)-2*ifelse(barperiod<1440,M1back, DailyLBack)));	//6 years or more
	int end;
	

	string t6Out[100];
	if (barperiod<1440) {
		sprintf(t6Out,strf("%s%s.t6",History,theAsset));
		barperiod = 1;
		end = ymd(wdate(NOW)+1);	// +1 to get most current M1 bar
	}
	else	{
		sprintf(t6Out,"History\\%s.t6",theAsset);
		end = ymd(wdate(NOW)-1);	 // -1 to get up to yesterday
	}
	
	//-------------------------------------
	// check to see if exiting t6 is in range	
	//-------------------------------------

	if (file_length(t6Out))	{
		int records = dataLoad(33,t6Out,7);	
		int fileStart = ymd(dataVar(33,-1,0));	
		int fileEnd = ymd(dataVar(33,0,0));
		
		//printf("fileStart=%d  fileEnd=%d\n",fileStart, fileEnd);
			
		// if Test or Train, don't download if data in range
		if (	(!Live && EndDate<=fileEnd && barperiod==1440)
			|| (Live && end<=fileEnd && barperiod==1440)
			)	
		{
			printf("\n[%s] historical data up to date... no download\n");
			dataNew(33,0,0);
			return 0;	// range already inside
		}	
		// if not enough data, its probably a newly listed stock/asset with limited history
		if (start<fileStart)		
			printf("\nWARNING:  historical data for [%s] may not be early enough\n",theAsset);

		// set start to last date in file, to upate to current date
		start = fileEnd;		

	}

	printf("Downloading data [%s]... Start=%d   End=%d\n", theAsset, start, end);

	//----------------------------------
	//			index symbol conversions
	//----------------------------------
	if (theAsset == "SPX") theAsset = "$SPX";
	if (theAsset == "DJI") theAsset = "$DOWI";	
	if (theAsset == "INX") theAsset = "$INX";
	
	//----------------------------------
	//				get data		
	//----------------------------------

	if(once(Init)) connectToSierraChart();

	int reqbars = (int) (dmy(end) - dmy(start))*ifelse(barperiod==1440,1,1440);	// D1 or M1 data only 
	printf("SC: items allocated = %d\n",reqbars+10);

	T6* myT6=malloc((reqbars+10)*sizeof(T6));
	printf("BrokerHistory2 (%s, %d, %d, %i, -1, myT6)\n", theAsset, start, end, barperiod);
	int items = BrokerHistory2 (theAsset, dmy(start), dmy(end), barperiod, -1, myT6);	
	printf("SC: items received = %d\n",items);
	if (!items)	{
		printf("ERROR:  No historical data returned from SierraChart... maybe out of data cache range.\n");
		free(myT6);
		quit("#quit");
	}

	FILE *fp;
	string tempT6 = strf("History\\temp-%s.t6",theAsset);
	printf("t6 = %s\n", tempT6);
	fp = fopen(tempT6, "wb");
	fwrite(myT6 , sizeof(T6), items, fp );
	fclose(fp);
	free(myT6);
	
	dataLoad(33,tempT6,7);	// load it into 33
	dataSort(33);				// make sure it's sorted 
	printf("LOADED TEMP\n");
	//---------------------------------
	//	pre-pend to existing	t6
	//---------------------------------
	if (file_length(t6Out)>0)	{
		printf("Appending to, %s\n", t6Out);
		dataLoad(22,t6Out,7);
		dataMerge(22,33);		
		dataSave(22,t6Out);
		dataNew(22,0,0);	
	}
	else	{
		printf("Saving to, %s\n", t6Out);
		dataSave(33,t6Out);
		dataNew(33,0,0);	
	}

	file_delete(tempT6);
	return 0;
}
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/21/21 12:53

No idea, but you're not using the latest plugin. Download it here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/releases/tag/v1.0.0.3

The documentation has completely changed, too. Read:
https://github.com/AndrewAMD/SierraChartZorroPlugin

Also, I have no plans on supporting that method of access. Use AssetHistory:
https://manual.zorro-project.com/loadhistory.htm

I might eventually put in a separate brokerCommand if I have time to spare.
(Or I can expedite it with a sponsorship.)
Posted By: SBGuy

Re: Sierra Chart Plugin - 04/21/21 15:08

Hi Andrew,

I had this same 180 day problem back in 2019 and you lifted the limit. Here's what you said. Perhaps forgot to include the IGNORE_MAX_DAYS thing in the new plugin? Would appreciate you putting it back in. It apparently didn't cause any problems since 2019.

=========================================================================
I hacked off max_days with a macro IGNORE_MAX_DAYS.

Can you see if it fixes your issue?

You can download it here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/raw/master/releases/0.9.3.1/SierraChart.dll
Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/21/21 15:54

It is impossible to have that problem with the latest version of the plugin, since MaxDaysToReturn is never set.

I don't know why you are linking to the old version, the new one is much better!
Originally Posted by AndrewAMD
No idea, but you're not using the latest plugin. Download it here:
https://github.com/AndrewAMD/SierraChartZorroPlugin/releases/tag/v1.0.0.3

The documentation has completely changed, too. Read:
https://github.com/AndrewAMD/SierraChartZorroPlugin

Also, I have no plans on supporting that method of access. Use AssetHistory:
https://manual.zorro-project.com/loadhistory.htm

I might eventually put in a separate brokerCommand if I have time to spare.
(Or I can expedite it with a sponsorship.)
Posted By: jbhunter

Re: Sierra Chart Plugin - 04/21/21 16:22

Thanks for the quick responses. When I try to use assetHistory(ticker,1) I get results like this.


Code
  StartDate=0 EndDate=20210421  NumYears=3  BarPeriod=1.000000
Cmd="C:\Users\User\Zorro\Zorro.exe" Sierra_GetAsset_test -stay
 ass=BRX
Load AssetsSierra
Login 0 SierraChart..
!1: Connected to SC DTC Protocol server. Service=scdataallservices|SymbolSettings=scdataallservices
!1: WARNING: Trading not supported
!1: TradeAccounts detected: Sim1
!1: Using: Sim1
!1: Warning: no balance confirmed for this account!
!1: 0 historical orders detected.
!1: 0 open orders detected.
!1: 0 open positions detected. at UTC 04-21 16:17
(null): 6 MB allocated
!2: short read
!2: Connect failure.
!BRX 01-01 00:00:00 to 04-21 16:17:57 failed
!BRX 01-01 00:00:00 to 04-21 16:16:57 failed
!BRX 01-01 00:00:00 to 04-21 16:15:57 failed
!BRX 01-01 00:00:00 to 04-21 16:14:57 failed
!BRX 01-01 00:00:00 to 04-21 16:13:57 failed
!BRX 01-01 00:00:00 to 04-21 16:12:57 failed
!BRX 01-01 00:00:00 to 04-21 16:11:57 failed
!BRX 01-01 00:00:00 to 04-21 16:10:57 failed
!BRX 01-01 00:00:00 to 04-21 16:09:57 failed
!BRX 01-01 00:00:00 to 04-21 16:08:57 failed
!BRX 01-01 00:00:00 to 04-21 16:07:57 failed
!BRX 01-01 00:00:00 to 04-21 16:06:57 failed - BRX history unavailable!
Error 056: Can't download BRX 2021 history
(null): 6 MB allocated
!BRX 01-01 00:00:00 to 12-31 23:59:59 failed
!BRX 01-01 00:00:00 to 12-31 23:58:59 failed
!BRX 01-01 00:00:00 to 12-31 23:57:59 failed
!BRX 01-01 00:00:00 to 12-31 23:56:59 failed
!BRX 01-01 00:00:00 to 12-31 23:55:59 failed
!BRX 01-01 00:00:00 to 12-31 23:54:59 failed
!BRX 01-01 00:00:00 to 12-31 23:53:59 failed
!BRX 01-01 00:00:00 to 12-31 23:52:59 failed
!BRX 01-01 00:00:00 to 12-31 23:51:59 failed
!BRX 01-01 00:00:00 to 12-31 23:50:59 failed
!BRX 01-01 00:00:00 to 12-31 23:49:59 failed
!BRX 01-01 00:00:00 to 12-31 23:48:59 failed - BRX history unavailable!
Error 056: Can't download BRX 2020 history
(null): 6 MB allocated
!BRX 01-01 00:00:00 to 12-31 23:59:59 failed
!BRX 01-01 00:00:00 to 12-31 23:58:59 failed
!BRX 01-01 00:00:00 to 12-31 23:57:59 failed
!BRX 01-01 00:00:00 to 12-31 23:56:59 failed
!BRX 01-01 00:00:00 to 12-31 23:55:59 failed
!BRX 01-01 00:00:00 to 12-31 23:54:59 failed
!BRX 01-01 00:00:00 to 12-31 23:53:59 failed
!BRX 01-01 00:00:00 to 12-31 23:52:59 failed
!BRX 01-01 00:00:00 to 12-31 23:51:59 failed
!BRX 01-01 00:00:00 to 12-31 23:50:59 failed
!BRX 01-01 00:00:00 to 12-31 23:49:59 failed
!BRX 01-01 00:00:00 to 12-31 23:48:59 failed - BRX history unavailable!
Error 056: Can't download BRX 2019 history
Logout.. ok


When I update to the latest plugin version I get this


Code
 StartDate=0 EndDate=20210421  NumYears=3  BarPeriod=1.000000
Cmd="C:\Users\User\Zorro\Zorro.exe" Sierra_GetAsset_test -stay
 ass=BRX
Load AssetsSierra
Login 0 SierraChart..
!M: Connecting to 127.0.0.1:11099...
!M: Requesting binary encoding...
!M: readloop_handler1: End of file
Can't connect!
Posted By: SBGuy

Re: Sierra Chart Plugin - 04/21/21 16:23

Ah, very good Andrew. Yeah, the old link was just for your reference.

Sorry, jbhunter, don't know what's wrong with your setup. I have 1,000 days max on SC setting and was able to get 1,000 days of M1 data for your BRX example into a .t6 file. I am using the version 1.0.xx of the plugin

My code may have changed since I posted it. For what it's worth, here's the latest code. It's not the best code, but it works for me. Just put BarPeriod as the second argument.

You'll have to debug your problem further on your own.

Code
int assetHistorySC(string theAsset, int theBP)
{
	if (!Init)	return 0;

	int barperiod = theBP;
	//----------------------------------
	//		setup start and end dates
	//----------------------------------
	int M1back = (LookBack*BarPeriod)/(7*60); 	// days of lback
	int DailyLBack = (LookBack+2*UnstablePeriod);
	int start = min(ymd(wdate(NOW)-6*365), ymd(dmy(StartDate)-2*ifelse(barperiod<1440,M1back, DailyLBack)));	//6 years or more
	int end;
	
	
	if (!EndDate) 
		if (hour(NOW)>=21) EndDate = ymd(wdate(NOW));
		else EndDate = ymd(wdate(NOW)-1);

	string t6Out[100];
	if (barperiod<1440) {
		sprintf(t6Out,strf("History\\%s%s",theAsset,History));
		barperiod = 1;
		end = ymd(wdate(NOW)+1);	// +1 to get most current M1 bar
	}
	else	{
		sprintf(t6Out,"History\\%s.t6",theAsset);
		if (hour(NOW)>=21) end = ymd(wdate(NOW));
		else end = ymd(wdate(NOW)-1);	 // -1 to get up to yesterday		
	}
	
	//-------------------------------------
	// check to see if exiting t6 is in range	
	//-------------------------------------

	if (file_length(t6Out))	{
		int records = dataLoad(33,t6Out,7);	
		int fileStart = ymd(dataVar(33,-1,0));	
		int fileEnd = ymd(dataVar(33,0,0));
		
		//printf("fileStart=%d  fileEnd=%d\n",fileStart, fileEnd);
			
		// if Test or Train, don't download if data in range
		if (	(!Live && EndDate<=fileEnd && barperiod==1440)
			|| (Live && end<=fileEnd && barperiod==1440)
			)	
		{
			if (Live || DEBUG) printf("\n[%s] historical data up to date... no download\n");
			dataNew(33,0,0);
			return 0;	// range already inside
		}	
		// if not enough data, its probably a newly listed stock/asset with limited history
		if (start<fileStart)		
			if (Live || DEBUG) printf("\nWARNING:  historical data for [%s] may not be early enough\n",theAsset);

		// set start to last date in file, to upate to current date
		start = fileEnd;		

	}

	if (start==end) return 0;
	
	if (Live || DEBUG) printf("Downloading data [%s]... Start=%d   End=%d\n", theAsset, start, end);

	//----------------------------------
	//			index symbol conversions
	//----------------------------------
	if (theAsset == "SPX") theAsset = "$SPX";
	if (theAsset == "DJI") theAsset = "$DOWI";	
	if (theAsset == "INX") theAsset = "$INX";
	
	//----------------------------------
	//				get data		
	//----------------------------------

	if(once(Init)) connectToSierraChart();

	int reqbars = (int) (dmy(end) - dmy(start))*ifelse(barperiod==1440,1,1440);	// D1 or M1 data only 
	//printf("SC: items allocated = %d\n",reqbars+10);

	T6* myT6=malloc((reqbars+10)*sizeof(T6));
	int items = 0;
	int i;
	for (i=0;i<=2;i++)
	{
		items = BrokerHistory2 (theAsset, dmy(start), dmy(end), barperiod, -1, myT6);	
		if (Live || DEBUG) printf("SC: items received = %d\n",items);

		if (!items)	{
			if (Live || DEBUG) printf("ERROR:  No historical data returned from SierraChart... maybe out of data cache range.\n");
			connectToSierraChart();
			wait(1000);
		}
		else	{
			break;
		}
	}
		
	FILE *fp;
	string tempT6 = strf("History\\temp-%s.t6",theAsset);
	fp = fopen(tempT6, "wb");
	fwrite(myT6 , sizeof(T6), items, fp );
	fclose(fp);
	free(myT6);
	
	dataLoad(33,tempT6,7);	// load it into 33
	dataSort(33);				// make sure it's sorted 
	
	//---------------------------------
	//	pre-pend to existing	t6
	//---------------------------------
	if (file_length(t6Out)>0)	{
		dataLoad(22,t6Out,7);
		dataMerge(22,33);		
		dataSave(22,t6Out);
		dataNew(22,0,0);	
	}
	else	{
		dataSave(33,t6Out);
		dataNew(33,0,0);	
	}

	file_delete(tempT6);
	return 0;
}


Posted By: AndrewAMD

Re: Sierra Chart Plugin - 04/21/21 16:32

Originally Posted by jbhunter
When I update to the latest plugin version I get this


Code
 StartDate=0 EndDate=20210421  NumYears=3  BarPeriod=1.000000
Cmd="C:\Users\User\Zorro\Zorro.exe" Sierra_GetAsset_test -stay
 ass=BRX
Load AssetsSierra
Login 0 SierraChart..
!M: Connecting to 127.0.0.1:11099...
!M: Requesting binary encoding...
!M: readloop_handler1: End of file
Can't connect!
Read the documentation on Github. The port numbers changed:
https://github.com/AndrewAMD/SierraChartZorroPlugin
Posted By: jbhunter

Re: Sierra Chart Plugin - 04/21/21 17:44

Thanks SBGuy. I might try giving that a shot.

When trying the latest plugin version 1.0.0.3, I had to change the ports to get it to login. After fixing this I am getting these errors shown on the Sierra side...

HD Server | Thread:7228 (152) | Encryption stream error encountered when processing received data. Failed to read incoming decrypted data from the incoming binary I/O stream. error:1408F10B:SSL routines:ssl3_get_record:wrong version number | 2021-04-21 13:41:15.703
Posted By: jbhunter

Re: Sierra Chart Plugin - 04/21/21 18:00

Oh geez, I had the "Historical Data Port" and "TLS Historical Data Ports" swapped. I thought I had tried that already. It is working. I am also now getting multiple years with assetHistory(ticker, 1)

Anyways thanks for the plugin and support!
© 2024 lite-C Forums