MT4/5 - wrong M1 time

Posted By: Grat

MT4/5 - wrong M1 time - 05/30/20 06:34

Hi,

Zorro FX data is in UTC
Quote
All downloaded prices are usually aligned to UTC time...


but in the script "Zorro.mq4" - command CMD_HISTORY return time candle with this code:

Code
int time = (int)iTime(Asset,timeframe,shift);
arr[4] = time + 60*timeframe; // assume tick start time


but i missing shift from servertime to UTC time.
For this I thinking data download from MT4 is wrong. Many server is +2GMT but I see also +1GMT nad +3GMT.

Is a true?

If yes, I know, how to make correct in the Zorro.mq4.

Milan
Posted By: Grat

Re: MT4/5 - wrong M1 time - 05/30/20 09:23

I had compare data with come with Zorro and data downloaded from MT4

at first - data which is downloaded from MT4 - TimeZone is OK


[Linked Image]

but if use data from Zorro - have a different Timezene

[Linked Image]


Attached picture zdate1.jpg
Attached picture zdate2.jpg
Posted By: Grat

Re: MT4/5 - wrong M1 time - 05/30/20 09:58

how to repare:

in the file zorro.mq4

row (353-354):

int start = (int)arr[1]+(timeframe*timeZone);
int end = (int)arr[2]+(timeframe*timeZone);

and row 77 ( not important )
extern int timeZone = -2; // if server is +2GMT

is only temporary - becouse must be add the DST

MT4 servers are GMT +2 or GMT +3 depending on New York DST (Daylight Savings Time)

( not tested !! )
Posted By: Grat

Re: MT4/5 - wrong M1 time - 05/30/20 19:14

maybe this is a better:

BrokerZone = UTC+2;
Posted By: Petra

Re: MT4/5 - wrong M1 time - 05/31/20 09:39

As far as I know the MT4 bridge corrects the time so that it is UTC when it is received from Zorro.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 05/31/20 10:13

I testeted - look picture above.
First have data dowloaded from MT4
SEC - use data comming with Zorro.

Is a different.
Posted By: Petra

Re: MT4/5 - wrong M1 time - 05/31/20 10:31

Hm, Zorro entered a trade at a bar of 16:15 and in your screenshot it looks as if the same bar is in the MT4 chart at 18:15. Is 16:15 UTC not the same as 18:15 local time? Or do I not understand something?
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/01/20 08:47

with year 2019 ( zorro data ) -> zorro open trade in 16:15 - on the MT4 is real 18:15 - this is a correct. MT4 have a 2 hour shift from UTC.

But with data downloaded from MT4 - zorro stored with date comming from MT4, not the shift to UTC. In this case Zorro open trade on the same time like see in the MT4 chart.
This is correct, becouse Zorro don't know how many is shifted MT4 server. Now is standard +2/3 depend the DTS. But any obscure brokers have different timezone.

In the MT4 can be calculace this shitmend but only if is market time. During weekend send the wrong.

In the MT4I have:

Code
        extern int nAddH=2;  
        int nDow=TimeDayOfWeek(TimeLocal());
        if(nDow==0 || nDow==6){
            nGMT=-60*60*nAddH;
            Print("Week: ",nGMT);
        }else{
            nGMT=(int)(TimeGMTOffset()-TimeCurrent()+TimeLocal());
            Print("WORK: ",nGMT," DW: ",nDow);
        }
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/01/20 12:34

Your screenshots above all look ok to me, so I can't see the problem. Can you show a screenshot with no time shift?
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/01/20 17:05

If see on first jpg - Zorro open in 11:00 the same like in MT4 - 11:00 ... this data is download from MT4 - data 100% not in UTC but in MT4 server zone
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 06:46


I make test (EUR/USD) with this code:
Code
	function run()
	{
		NumCores = -1;
		StartDate 	= 20200401;
		BarPeriod = 15;	// 15 min
		LookBack = 1;
		NumYears = 1;
		MaxLong = MaxShort = 15;
		//AssetMarket = 2;
		//BrokerZone = UTC+2;
		set(STEPWISE);
		assetList("Assets4");
		if (is(INITRUN)){
			assetHistory(Asset,1);
		}
		ticket();	
			printf("\nOpen: %.5f High: %.5f Low: %.5f",priceOpen(),priceHigh(),priceLow());
	}



1. data comming with ZORRO ( UTC ) 2020.04.06 07:00
Quote


[761: Wed 20-04-01 00:15] (1.10247)
Error 011: MACD EUR/USD invalid parameters (failed) at bar 761
Open: 1.10268 High: 1.10321 Low: 1.10230
[762: Wed 20-04-01 00:30] (1.10323)
Open: 1.10247 High: 1.10340 Low: 1.10243
[763: Wed 20-04-01 00:45] (1.10301)
Open: 1.10323 High: 1.10340 Low: 1.10257
[764: Wed 20-04-01 01:00] (1.10324)
Open: 1.10301 High: 1.10370 Low: 1.10288
[765: Wed 20-04-01 01:15] (1.10273)
Open: 1.10324 High: 1.10340 Low: 1.10257




2. now - delete file EURUSD_2020.t6 and run this code again:

Quote


[762: Wed 20-04-01 00:30] (1.10314)
Open: 1.10239 High: 1.10331 Low: 1.10236
[763: Wed 20-04-01 00:45] (1.10291)
Open: 1.10314 High: 1.10331 Low: 1.10249
[764: Wed 20-04-01 01:00] (1.10315)
Open: 1.10291 High: 1.10362 Low: 1.10283
[765: Wed 20-04-01 01:15] (1.10262)
Open: 1.10315 High: 1.10329 Low: 1.10252



the same datetime, but different price.

other date:

Data from ZORRO ( UTC)
[55: Tue 20-02-11 14:00] (1.09041)
Open: 1.08998 High: 1.09046 Low: 1.08955
[56: Tue 20-02-11 14:15] (1.09096)
Open: 1.09041 High: 1.09124 Low: 1.09022
[57: Tue 20-02-11 14:30] (1.09142)
Open: 1.09096 High: 1.09179 Low: 1.09095
[58: Tue 20-02-11 14:45] (1.09174)
Open: 1.09142 High: 1.09188 Low: 1.09128
[59: Tue 20-02-11 15:00] (1.09208)
Open: 1.09174 High: 1.09235 Low: 1.09158
[60: Tue 20-02-11 15:15] (1.09224)
Open: 1.09208 High: 1.09254 Low: 1.09192

Data from MT4 +2UTC
[50: Tue 20-02-11 12:45] (1.08992)
Open: 1.08978 High: 1.09039 Low: 1.08912
[51: Tue 20-02-11 13:00] (1.09035)
Open: 1.08992 High: 1.09041 Low: 1.08950
[52: Tue 20-02-11 13:15] (1.09089)
Open: 1.09037 High: 1.09118 Low: 1.09016
[53: Tue 20-02-11 13:30] (1.09136)
Open: 1.09090 High: 1.09174 Low: 1.09090
[54: Tue 20-02-11 13:45] (1.09169)
Open: 1.09135 High: 1.09181 Low: 1.09124
[55: Tue 20-02-11 14:00] (1.09202)
Open: 1.09168 High: 1.09229 Low: 1.09152



OLD DATA
ZORRO/UTC. 0:00 x x x 1:00 x x x 2:00 x x x 3:00
MT4+3 22:00x x x23:00x x x 0:00 x x x 1:00

NEW DATA ( from MT4)
ZORRO/UTC. 0:00 x x x 1:00 x x x 2:00 x x x 3:00
MT4+3 0:00 x x x 1:00 x x x 2:00 x x x 3:00


setting:
BrokerZone = UTC+2; is OK

but if need
BrokerZone = UTC-2; - not work, still is only UTC




Attached picture Snímek obrazovky 2020-06-02 v 13.09.58.png
Attached picture Snímek obrazovky 2020-06-02 v 13.11.37.png
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/02/20 09:55

Check the log. You should see a message "MT4/5 time zone shift xxx hours". Of course only when you don't set BrokerZone or edit the Zorro EA. Do you get that message?
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 10:22

<b>before delete data :</b>
I trying -
BrokerZone = UTC+2;
BrokerZone = UTC-2;
//BrokerZone = UTC+2;

no change

testData compiling...............
Login 0 MT4/5..
!ZorroMT4.dll V4.23.
!Gleneagle Securities Pty Limited trading as Global Prime FX connected at UTC 06-02 10:14
Open: 0.00000 High: 0.00000 Low: 0.00000
Logout.. ok
Test: testData EUR/USD 2020
Assets Assets4
Open: 1.09399 High: 1.09411 Low: 1.09354

[277: Sun 20-04-12 00:00] (1.09290)
Open: 1.09367 High: 1.09402 Low: 1.09290

<b>first log after delete data:</b>
Open: 1.09399 High: 1.09411 Low: 1.09354

[277: Sun 20-04-12 00:00] 1.09367/1.09402\1.09290/1.09290 -0.00006
Open: 1.09367 High: 1.09402 Low: 1.09290

testData compiling...............
Login 0 MT4/5..
!ZorroMT4.dll V4.23.
!Gleneagle Securities Pty Limited trading as Global Prime FX connected at UTC 06-02 10:20
!EUR/USD 01-01 02:00:00 to 06-02 12:20:02
!=> 06-02 03:28:00 to 08:27:00, 300 ticks
Load EUR/USD 2020..
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 10:23

data download from MT4 with - BrokerZone = UTC+2;

Load EUR/USD 2020..
Copied to Clipboard! 155762 min
Open: 0.00000 High: 0.00000 Low: 0.00000
EUR/USD 155762 ticks read
3648 Bars 2020-04-07..2020-06-02 generated
Logout.. ok
Test: testData EUR/USD 2020
Assets Assets4
Open: 1.09354 High: 1.09355 Low: 1.09290

[274: Sun 20-04-12 23:15] 1.09446/1.09476\1.09276/1.09367 -0.00006
Open: 1.09446 High: 1.09476 Low: 1.09276
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 10:25

data downloaded from MT4 without

!=> 06-02 05:25:00 to 10:24:00, 300 ticks
Load EUR/USD 2020.. 155879 min
Open: 0.00000 High: 0.00000 Low: 0.00000
EUR/USD 155879 ticks read
3696 Bars 2020-04-07..2020-06-02 generated
Logout.. ok
Test: testData EUR/USD 2020
Assets Assets4
Open: 1.09349 High: 1.09389 Low: 1.09341

[277: Sun 20-04-12 00:00] 1.09345/1.09402\1.09290/1.09290 -0.00006
Open: 1.09345 High: 1.09402 Low: 1.09290
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 10:37

Verbose = 7+DIAG;

Open: 1.11277 High: 1.11286 Low: 1.11272

[196: Wed 20-01-15 00:15] 1.11282/1.11303\1.11277/1.11303 -0.00006
Open: 1.11282 High: 1.11303 Low: 1.11277

testData compiling...............
Login 0 MT4/5..
!ZorroMT4.dll V4.23.
!Gleneagle Securities Pty Limited trading as Global Prime FX connected at UTC 06-02 10:35
!EUR/USD 01-01 00:00:00 to 06-02 10:35:59
!=> 06-02 05:37:00 to 10:36:00, 300 ticks
Load EUR/USD 2020..

Load EUR/USD 2020..
Copied to Clipboard! 155891 min
Open: 0.00000 High: 0.00000 Low: 0.00000
EUR/USD 155891 ticks read
388 Bars 2020-01-12..2020-01-17 generated
Logout.. ok
Test: testData EUR/USD 2020
Assets Assets4
Open: 1.11298 High: 1.11307 Low: 1.11290

[196: Wed 20-01-15 00:15] 1.11296/1.11315\1.11291/1.11310 -0.00006
Open: 1.11296 High: 1.11315 Low: 1.11291
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 10:43

No shift time. This is a normal, because Zorro don't know MT4 Timezone. During the market time is easy get the shift. But in the SA/SU not work. And in Zorro.mq4 is this code:

int time = (int)iTime(Asset,timeframe,shift);
arr[4] = time + 60*timeframe; // assume tick start time

In this case must be say in Zorro for each brokerPlugin correct timezone and during store in .t6 transform into UTC timezone. I don't have the source this plugin.

I make my workaround - change M1 candle timezone to UTC. But must be change every DST 2/3
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/02/20 10:46

Thank you. I still cannot see the reason of the problem from here, but please contact Support. They will sort that out. And do not set BrokerZone or modify the EA. Otherwise you'll likely get wrong timestamps.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/02/20 12:10

For the other users,

[Linked Image]

Attached picture realtime.jpg
Posted By: danatrader

Re: MT4/5 - wrong M1 time - 06/02/20 17:42

I had weird issues, not too long ago, just asked, my broker told me GMT +3.

I may have issues caused by that?
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/03/20 06:53

Normally not. The screenshot looks ok, so I am not sure about the problem. But if there are really wrong timestamps with Grat's broker for some reason, the support will find out.

It happens that a MT4 server has temporarily wrong data. I have not yet heard of wrong timestamps, but theoretically it's possible.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/03/20 07:04

I work on the detail report, where is problem, but in short:

a) in the real time.... zorro and MT4 is correct timezone
b) if zorro downloaded historic data from MT4 - don't correct shift to UTC -> this is a basic problem. I'm a 99% sure about this.
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/03/20 07:12

Historical data is time zone adjusted with the same offset as live data. Wrong timestamps are possible when the live and historical data come from different time zones. But I have not yet heard of such a case.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/03/20 08:06

How to find this offset? In the DLL?
Posted By: jcl

Re: MT4/5 - wrong M1 time - 06/03/20 08:11

Yes, it's an internal variable in the DLL. It cannot be accessed externally at the moment, but we can change this dependent on what the support finds out.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/03/20 08:20

In this case, check, how find this offset during SA/SO - mayby this is a wrong.
Now I busy with other project, but evening I return back and prepare detail survey. I read both data ( MT4 .hst and ZORRO .t6) with python and compare.
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/04/20 11:56

I promise python compare:

how to read in python hst file from MT4:
Code
import numpy as np
import pandas as pd

def read_hst(filepath):
    with open(filepath, 'rb') as f:
        ver = np.frombuffer(f.read(148)[:4], 'i4')
        dtype = [('time', 'u8'), ('open', 'f8'), ('high', 'f8'), ('low', 'f8'),
                     ('close', 'f8'), ('volume', 'i8'), ('s', 'i4'), ('r', 'i8')]
        df = pd.DataFrame(np.frombuffer(f.read(), dtype=dtype).astype(dtype))
        df['time'] = pd.to_datetime(df['time'], unit='s')
    return df

dMT4=read_hst("EURUSD15.hst")
dMT4.head()


output is:
Code
time	open	high	low	close	volume	s	r
0	2019-06-27 16:00:00	1.13665	1.13678	1.13618	1.13666	1489	0	0
1	2019-06-27 16:15:00	1.13665	1.13672	1.13638	1.13644	989	        0      0
2	2019-06-27 16:30:00	1.13640	1.13671	1.13608	1.13627	1557	0	0
3	2019-06-27 16:45:00	1.13626	1.13628	1.13564	1.13585	1274	0	0
4	2019-06-27 17:00:00	1.13587	1.13707	1.13574	1.13699	1592	0	0


how to read in python history file from ZORRO:

Code
def read_t6(filepath):
    with open(filepath, 'rb') as f:
        dtype = [('time', 'f8'), ('high', 'f4'), ('low', 'f4'), ('open', 'f4'),
                     ('close', 'f4'), ('val', 'f4'), ('vol', 'f4')]
        df = pd.DataFrame(np.frombuffer(f.read(), dtype=dtype).astype(dtype))
        df['time'] = pd.to_datetime(df['time'],unit='D', origin=pd.Timestamp('1899-12-30'))
        df=df.set_index("time")
        ohlc_dict = {'open':'first', 'high':'max', 'low':'min', 'close': 'last'}
        
    return df.resample('15Min').agg(ohlc_dict)
dT6=read_t6("EURUSD_2020.t6")
dT6.head()


Now I work on the compare this dataset - datetime

come later....
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/05/20 07:56

compare datetime

[Linked Image]

Attached picture Snímek obrazovky 2020-06-05 v 12.39.25.png
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/05/20 08:28

Data, which Zorro get from MT4 is in different TZ

[Linked Image]

Attached picture Snímek obrazovky 2020-06-05 v 13.22.16.png
Posted By: Grat

Re: MT4/5 - wrong M1 time - 06/05/20 08:34

I thinking, is a problem with DST. Because now is server +3 hour's, but in the 2020/01/01 to change DST only +2hour's. For system used short period ( Timeframe < 240 ) don't depend. But for system working with H4 and D1 must have a different results.

that's all folk...

P.S: if anybody want, I send jupyter script for playing on sand laugh
© 2024 lite-C Forums