About marketVol() function

Posted By: pascalx

About marketVol() function - 11/02/17 17:49

Hello.

I was wondering how the marketVol function is supposed to be used. When invoked, it returns unexpectedly small values only. When looking at daily asset data from stooq, market volume numbers are very large, going into hundreds of millions, whereas Zorro returns values in the ones and tens. Upon investigation, it looks like the application returns only the first minute volume value, and not the sum of all values within a bar. Consider this script:

Code:
#include <default.c>
#include <contract.c>

void main()
{
	BarPeriod = 1440;
}

void run()
{
	DATE date = wdate(0);
	long yyyymmdd = ymd(date);
	double vol = marketVol(0);
	printf("n%d volume: %f", yyyymmdd, vol);
}



And the attached image.

I am using the data from the Zorro website. But even if marketVol would return the sum, the number would still be relatively small. Like lets say 5 (approx. volume per minute) * 1440 (minutes) = 7200 (volume per day). This is far from the expected volume of hundereds of millions. In what unit are the Zorro data volume numbers?

The Manual says "trade volume per minute":
http://zorro-project.com/manual/en/price.htm
But if this is really only ever returning minute volume, how can the user access the daily/hourly/etc volume, when the given function argument takes a bar (as per BarPeriod) only?

I don't understand this. Any help appreciated. Thanks.

Attached picture marketVol.png
Posted By: AndrewAMD

Re: About marketVol() function - 11/02/17 19:12

That's because daily data (from Stooq etc) is stored differently than M1 data.

From an earlier post by jcl :

Originally Posted By: jcl
No, in D1 history there is no difference of start and end, since it has no hours. The date of the open is identical to the date of the close. This is considered by Zorro. Otherwise the calculation would be based on a wrong time, since a date literally begins at 00:00 and shifted to UTC you would get a date boundary between open and close.

But you would think Zorro would add up those bars anyways, right?
Posted By: pascalx

Re: About marketVol() function - 11/02/17 21:05

Originally Posted By: AndrewAMD
That's because daily data (from Stooq etc) is stored differently than M1 data.

If the volume of M1 data is different to the volume of D1 data, how does someone convert the M1 volume to D1 colume when using BarPeriod = 1440 and M1 tick data?
Posted By: AndrewAMD

Re: About marketVol() function - 11/02/17 23:19

Whoops, I forgot to address the elephant in the room:

Decentralized instruments will always have different tick volumes and/or volumes between exchanges. The SPX500 CFD is one of those instruments.

Next, I was unclear:
Quote:
If the volume of M1 data is different to the volume of D1 data, how does someone convert the M1 volume to D1 colume when using BarPeriod = 1440 and M1 tick data?

Assuming data is coming from the same data vendor (and using the same metric, not confusing tick volume with actual volume), 1440 M1 data points should always add up to the equivalent D1 bar. A D1 has a DATE float value rounded down to the nearest integer to indicate which day.

Could this be a Zorro D1 volume retrieval bug?
Posted By: jcl

Re: About marketVol() function - 11/06/17 09:03

No, Zorro does nothing with volume. So what you do with the value returned by marketVol is completely up to the broker plugin and to your script. The "volume per minute" is a suggestion what _could_ be returned.
Posted By: pascalx

Re: About marketVol() function - 11/06/17 12:41

Originally Posted By: jcl
No, Zorro does nothing with volume. So what you do with the value returned by marketVol is completely up to the broker plugin and to your script. The "volume per minute" is a suggestion what _could_ be returned.


Damn. This is troubling. I see MT4/5 does not even support volume data at all:
Quote:
marketVal: Not supported.
marketVol: Not supported in historical data, tick frequency in live data.

http://zorro-project.com/manual/en/mt4plugin.htm

Why is that? MT4 does know about volume.
Wouldn't it be natural to pass that in marketVol instead of 'tick frequency', whatever that is?
https://docs.mql4.com/predefined/volume

So you have marketVal and marketVol as additional, optional values. Any particular reason why 2?
Are 2 the maximum of optional data values someone ever needs?

This idea of a function called marketVol(int) (assuming "Vol" stands for "Volume") could return something other than volume looks like a design flaw. If the broker sends some other data for it, then ideally the user should not require to add checks in his code prior calling marketVol to make sure it is indeed a volume value. marketVol should just return 0, indicating that it has no known volume data from that current data source.

To accommodate arbitrary data values you would expect a marketVal(int index, int offset) function, where the given index selects the optional data values at their natural position. I would even consider to add a new data structure such as:

Code:
typedef struct T4X
{
	DATE time;	
	float fHigh, fLow;
	float fOpen, fClose;
	float* fVal;
} T4X;



Where the pointer of fVal would point to an additional series of arrays. This would allow for true arbitrary optional data values.

I would suggest to mark marketVol(int) and marketVal(int) as deprecated and add new functions marketSpread(int), marketVolume(int) and marketVal(int index, int offset).
marketSpread(int) and marketVolume(int) should return spread and volume if the system knows that the values are indeed what they promise, for example by the broker plugin, or t6 data files, otherwise 0.

Now to the next problem: How to correctly collect the optional asset data for a given bar. The current design gives the user no possibility to collect correctly. So marketVol(int) and marketVal(int) are useless in many cases. The proposed marketSpread(int) and marketVolume(int) functions could automatically solve this problem, because since they know what the value is, they can internally collect the value correctly. For example while priceHigh(int offset) collects the highest value within the bar, marketVolume(int) would collect the sum of all values within the bar. Sounds fair?

Now, what about marketVal(int index, int offset)? It probably needs a third parameter taking an optional function pointer to tell how data should be collected. So marketVal(int index, int offset, void* collectFunction), where the collect function could look something like

Code:
var collect(var collectedValue, var nextValue)
{
	return collectedValue + nextValue;
}

var collect(var collectedValue, var nextValue)
{
	return max(collectedValue, nextValue);
}



The collect function gets called by zorro for each data point within the given bar. This would enable the user to specify how zorro should collect the optional asset data. void* collectFunction should be a proper function pointer if Lite-C supports it.

Anyone agrees with this?
Posted By: jcl

Re: About marketVol() function - 11/06/17 14:33

That's unfortunately not our's to decide. Only prices are delivered by all brokers, any other data is up to the broker. If they deliver tick frequency instead of volume, marketVol returns the tick frequency. If they deliver spread, marketVal returns the spread. If they deliver nothing, the functions return 0. Different function names or bigger structs won't help.

For this reason, it makes no sense to guarantee a certain volume format. The only requirement is that marketVol returns at least some proxy of volume. Since almost all volume based indicators only use volume changes and not absolute volume, they will work. How useful such indicators are with a MT4 connection where volume is normally the tick frequency, is a different question.
Posted By: AndrewAMD

Re: About marketVol() function - 11/06/17 16:22

jcl,

Can we add a new brokerCommand that allows the plugin to specify the output format of *pVolume? This way, Zorro would have no trouble determining how to handle the volume.

* New brokercommand: GET_BROKERASSET_VOLUME_TYPE - It will return the below values:
* 0: Not specified. Volume must be manually handled. (What Zorro does now.)
* -1: Absolute-point-of-reference number of ticks.
* -2: Number of ticks since last BrokerAsset call for this asset.
* -3: Frequency of ticks in ticks/minute.
* 1: Absolute-point-of-reference number of lots traded.
* 2: Number of lots traded since last BrokerAsset call for this asset.
* 3: Frequency of lot trades in lots/minute.

And then finally, on the user side, let the Zorro user specify how to interpret the historical volume with a similar type of flag.
Posted By: jcl

Re: About marketVol() function - 11/06/17 16:50

Yes, that's certainly possible. I'll put it on our list.
Posted By: pascalx

Re: About marketVol() function - 11/06/17 18:33

Originally Posted By: jcl
That's unfortunately not our's to decide. Only prices are delivered by all brokers, any other data is up to the broker. If they deliver tick frequency instead of volume, marketVol returns the tick frequency. If they deliver spread, marketVal returns the spread. If they deliver nothing, the functions return 0. Different function names or bigger structs won't help.

For this reason, it makes no sense to guarantee a certain volume format. The only requirement is that marketVol returns at least some proxy of volume. Since almost all volume based indicators only use volume changes and not absolute volume, they will work. How useful such indicators are with a MT4 connection where volume is normally the tick frequency, is a different question.


I don't see how this blocks a proper implementation of spread/volume/arbitrary asset data. In the case of MT4/MT5 the documentation clearly states that we have access to a volume value. We also have access to spread. Yet Zorro only delivers a tick frequency. Why?
https://docs.mql4.com/predefined/volume
https://docs.mql4.com/marketinformation/marketinfo

If the broker decides to send a garbage or no value for volume, and MT4/MT5 hands that garbage/no value to Zorro, why is that a problem Zorro would need to tackle, when the problem is on Broker/MT4 side? You also would not tackle the event if the broker would send garbage for open/high/low/close right? You just take the value as is in the remote platform and hand it over.

Furthermore I don't understand how the current implementation can be any satisfactory, nor do I understand how Andrews proposal fixes it all.

Summary of issues:
- Cannot access/use optional asset data values with BarPeriod > 1 (e.g. when backtesting .t6 data)
- Cannot retrieve volume and spread with MT4/MT5
- Limit of 2 optional asset data values. Cannot use 3 or more
- marketVol/marketVal might not be what they pretend to be / user has to find out / adapt code to different use cases

Originally Posted By: jcl
Different function names or bigger structs won't help.


I did not simply propose different function names or bigger structs. Was my previous text really that non-descriptive?
Posted By: jcl

Re: About marketVol() function - 11/07/17 09:24

MT4 delivers no volume, and cannot deliver it since it's mainly a Forex and CFD trading program. Forex and CFD have no volume. You confuse that with the tick frequency.

You can see that it's really the tick frequency when you display marketVol() on EUR/USD live with an MT4 connection, and at the same time observe the ticks in the MT4 terminal. That's why the manual mentions "tick frequency" for MT4 marketVol, not "Volume". And no, there's no historical spread by MT4 either, at least as to my knowledge.
Posted By: pascalx

Re: About marketVol() function - 11/07/17 12:24

Originally Posted By: jcl
MT4 delivers no volume, and cannot deliver it since it's mainly a Forex and CFD trading program. Forex and CFD have no volume. You confuse that with the tick frequency.

You can see that it's really the tick frequency when you display marketVol() on EUR/USD live with an MT4 connection, and at the same time observe the ticks in the MT4 terminal. That's why the manual mentions "tick frequency" for MT4 marketVol, not "Volume". And no, there's no historical spread by MT4 either, at least as to my knowledge.

Hi. Yes I might be mistaken on this matter.

A quick google search indeed revealed that the functionality labeled as "volume" in MT4 returns a tick frequency.

Still there appears to be spread information according to the MT4 manual, at least for live feeds. Why is that not bridged?

I will ask a buddy who has great experience with MT4 and forward his assessment on this matter - if he has any.

Do you have opinions about the other 3 issues I mentioned?
Posted By: jcl

Re: About marketVol() function - 11/07/17 12:37

I do not really understand the spread issue. All brokers deliver live spread and it's available in the "Spread" variable. marketVal is for an arbitrary additional parameter, not necessarily for spread.

As to retrieving more than 2 additional parameters from the broker - can you give me an example of which other parameters from which broker you would need?
Posted By: pascalx

Re: About marketVol() function - 11/09/17 10:08

Originally Posted By: jcl
I do not really understand the spread issue. All brokers deliver live spread and it's available in the "Spread" variable. marketVal is for an arbitrary additional parameter, not necessarily for spread.

As to retrieving more than 2 additional parameters from the broker - can you give me an example of which other parameters from which broker you would need?

Ok the spread makes sense. I don't have an example for more than 2 additional parameters. It might be no practical issue, and just a theoretic one, considering we are asking that question and are not sure of the answer. Ideally we would not have to ask that question.

Ok down to 2 issues/questions. Any opinions on points 1 and 4 ?

1. Cannot access/use optional asset data values with BarPeriod > 1 (e.g. when backtesting .t6 data)
2. Cannot retrieve volume and spread with MT4/MT5 (no issue)
3. Limit of 2 optional asset data values. Cannot use 3 or more (no practical issue)
4. marketVol/marketVal might not be what they pretend to be / user has to find out / adapt code to different use cases
Posted By: jcl

Re: About marketVol() function - 11/09/17 16:23

To 1., I'm not sure that I understand the problem - can you give an example?

To 4., you can find the returned data by any specific API on the API page in the manual.
Posted By: pascalx

Re: About marketVol() function - 11/09/17 16:56

Originally Posted By: jcl
To 1., I'm not sure that I understand the problem - can you give an example?

In the opening post of this topic I added a sample script that shows the problem. When looking into the actual t6 data (with ZHistory tool) then it becomes obvious that the given values are unexpected.

Originally Posted By: jcl
To 4., you can find the returned data by any specific API on the API page in the manual.

Yes. Unfortunately not all pages show this information.

The following do:
http://zorro-trader.com/manual/en/mt4plugin.htm
http://zorro-trader.com/manual/en/oanda.htm
http://zorro-trader.com/manual/en/fxcm.htm
http://zorro-trader.com/manual/en/ib.htm

And these do not:
http://zorro-trader.com/manual/en/ig.htm
http://zorro-trader.com/manual/en/ally.htm
http://zorro-trader.com/manual/en/dukascopy.htm
http://zorro-trader.com/manual/en/nxcore.htm

But as I said earlier this could easily be improved by providing functions that really return what they claim to return.

For example instead of a marketVol(int) function you could have a marketTickFrequency(int) and marketVolume() function that internally are aware if they return what they promise.

For example

Code:
void marketVolume(int offset)
{
	if (plugin == ig || assetData == t6)
	{
		return volume[offset];
	}
	return 0;
}



Otherwise the user needs to implement and maintain such checks himself (if he needs them).
Posted By: AndrewAMD

Re: About marketVol() function - 12/06/17 13:18

Originally Posted By: AndrewAMD
* -1: Absolute-point-of-reference number of ticks.
* 1: Absolute-point-of-reference number of lots traded.

jcl,

I should offer some clarification on how I expect this to work.

Zorro should anticipate that the absolute point of reference volume should occasionally reset. For example, the DTC protocol defines volume as "volume for this session". If, in this case, the connection is lost and must restart, volume will probably return to zero.

Pseudo-code:
if (AbsVolume[0] < AbsVolume[1]) VolumeSinceLastBrokerAssetCall = 0;
Posted By: jcl

Re: About marketVol() function - 12/06/17 13:30

Yes. The point of reference can be anything. It is normally of no importance, IB has not even documented it.

If the broker returns accumulated volume, you can convert it to volume per minute in this way:

VolumePerMinute = (marketVol(0)-marketVol(1))/BarPeriod;
Posted By: AndrewAMD

Re: About marketVol() function - 12/06/17 14:23

jcl,

I saw this in the beta trading.h headers:
Code:
#define GET_PRICETYPE	150 // type of prices returned by the API
#define SET_PRICETYPE	151 
#define GET_VOLTYPE	152 // type of volume returned by the API
#define SET_VOLTYPE	153

As of this writing these are not documented - I look forward to it.

I assume that if I supply absolute or relative volume from the plugin, Zorro will already know how to translate this into the appropriate volume.

For example, if a user's historical volume is volume at that bar, and the plugin provides volume since the last brokerasset call, I expect Zorro to accumulate volume for every given bar. Are my expectations correct?
Posted By: jcl

Re: About marketVol() function - 12/06/17 16:10

No, the marketVol function just returns the volume. It does not modify it in any way.
Posted By: AndrewAMD

Re: About marketVol() function - 12/06/17 16:30

It sounds like if a user wanted to replicate a volume-per-bar (such as for a money flow index indicator), then this is what he will need to do:

* Set future brokercommand SET_VOLTYPE to absolute volume (either ticks or real)
* Therefore, future brokercommand GET_VOLTYPE will acknowledge this setting.
* User will then detect changes in absolute volume and use that as real volume or tick volume for a given bar.

Is this correct?

If so, I would definitely use the absolute ticks setting for any forex plugin and an absolute volume setting for stocks, options, and futures.

Question: Were you planning on adding these brokercommands to the IB, FXCM, and Oanda plug-ins?

I also understand that, depending on how the plugin collects history data, whether from newest to oldest or oldest to newest, that it might require absolute volume to be accumulated in reverse order. That is...
* Newest volume is zero
* Previous tick is higher
* The tick before that is even higher.

So you can leave that to the end user to straighten out. (Or I might publish a handy helper function or two in the headers.)
Posted By: jcl

Re: About marketVol() function - 12/07/17 16:03

I know no API where you can set the volume type. So SET_VOLTYPE is at the moment only theoretical, a placeholder for a possible future feature that will probably be implemented in Zorro, not in the API plugin. Normally, trade volume is relative to a starting point in live trading, and absolute in historical data. Tick volume is always absolute. History data is always stored from newest to oldest.
Posted By: AndrewAMD

Re: About marketVol() function - 12/22/17 14:28

jcl,

Is there a way that I can alter historical data in-place? Suppose I want to edit only the marketVal values - all of them. Will I need to edit the historical files explicitly, or is there a high-level Zorro approach available?

My plan is to have M1 historical-only volume in marketVol, and then I can calculate absolute volume in marketVal. Then my plugins will put live absolute volume in marketVal.

This way, I can compare historical fluctuations in absolute volume with real fluctations in absolute volume.

_____________________________
UPDATE:

Well, I did figure out how to read the T6 files, it's literally just a binary array of T6 structs, with the newest entries in the front. I just need to overwrite that file.

_____________________________
UPDATE #2:

On the plugin side, for live data, I see a way to explicitly set an output for MarketVol (double *pVolume) but not for MarketVal, so it appears that my approach outlined above will not work.
Posted By: roshanbaba

Re: About marketVol() function - 02/26/18 08:15

can somebody code the money flow index or the aonbalance volume for zorro.
© 2024 lite-C Forums