Gamestudio Links
Zorro Links
Newest Posts
loading historical data 1st time
by AndrewAMD. 04/14/23 12:54
Trade at bar open
by juanex. 04/13/23 19:43
Bug in Highpass2 filter
by rki. 04/13/23 09:54
Adding Limit Orders For IB
by scatters. 04/11/23 16:16
FisherN
by rki. 04/11/23 08:38
AUM Magazine
Latest Screens
SHADOW (2014)
DEAD TASTE
Tactics of World War I
Hecknex World
Who's Online Now
0 registered members (), 1,012 guests, and 8 spiders.
Key: Admin, Global Mod, Mod
Newest Members
rki, FranzIII, indonesiae, The_Judge, storrealba
18919 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Handling spotty data #426569
07/25/13 10:03
07/25/13 10:03
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
How can I know if a downloaded set of data "could" be used, or if it would be better to just discard it?

How does Zorro behave while testing, if it comes across a large gap?

I downloaded EUR/DKK and some of the file sizes look abnormal:
EURDKK_2013.bar --> 1627 KB
EURDKK_2012.bar --> 2910 KB
EURDKK_2011.bar --> 3214 KB
EURDKK_2010.bar --> 2194 KB
EURDKK_2009.bar --> 553 KB
EURDKK_2008.bar --> 798 KB
(presumably there is nothing earlier, as I had requested back to 2002)

So it seems the years 2008 and 2009 are definitely spotty. Should I leave them as is? Delete them? or just set dummy files with size 0 KB?

Would there be any benefit to (removing files and) re-trying the download?

In my strategy, I want it to use a basket of currencies going back to 2002 for testing. If a pair does not have data available, I was hoping Zorro would just deal it and then eventually jump in with that pair once it sees that data is available. (ie, give a warning but keep trying across the test)

Re: Handling spotty data [Re: dusktrader] #426601
07/25/13 15:40
07/25/13 15:40
Joined: Jul 2000
Posts: 27,935
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,935
Frankfurt
Set up dummy files with size 0 for 2008 and 2009.

For checking the spottyness of a price file, you can just print the price chart and look for gaps. Alternatively you could write a script that checks the prices for missing data or analyzes them in many other ways.

Re: Handling spotty data [Re: jcl] #426614
07/25/13 21:12
07/25/13 21:12
Joined: Jun 2013
Posts: 1,609
D
DdlV Offline
Serious User
DdlV  Offline
Serious User
D

Joined: Jun 2013
Posts: 1,609
Pursuant to my question in the other thread, should 2008 & 2009 be "0 dummied" for all assets? (Assuming we're talking FXCM data...)

Re: Handling spotty data [Re: DdlV] #426625
07/26/13 09:55
07/26/13 09:55
Joined: Jul 2013
Posts: 522
D
dusktrader Offline OP
User
dusktrader  Offline OP
User
D

Joined: Jul 2013
Posts: 522
That's a great idea to plot the price LOL... I never thought of doing that!

DdlV, I am referring to FXCM data (currently downloading for several pairs back to 2002).

My assumption/plan is that I will create a 0 KB placeholder file for every pair and every year back to 2002... that is either missing or appears spotty.

Re: Handling spotty data [Re: dusktrader] #426632
07/26/13 10:40
07/26/13 10:40
Joined: Jul 2000
Posts: 27,935
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,935
Frankfurt
DldV: No, not for all assets, only for exotic pairs and CFDs. The data from the major pairs is fine in 2008 and 2009.


Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1