Gamestudio Links
Zorro Links
Newest Posts
Data from CSV not parsed correctly
by EternallyCurious. 04/18/24 10:45
StartWeek not working as it should
by Zheka. 04/18/24 10:11
folder management functions
by VoroneTZ. 04/17/24 06:52
lookback setting performance issue
by 7th_zorro. 04/16/24 03:08
zorro 64bit command line support
by 7th_zorro. 04/15/24 09:36
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:48
Zorro FIX plugin - Experimental
by flink. 04/14/24 07:46
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (flink, AndrewAMD), 656 guests, and 1 spider.
Key: Admin, Global Mod, Mod
Newest Members
EternallyCurious, howardR, 11honza11, ccorrea, sakolin
19047 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Out of Sample #480293
05/30/20 12:05
05/30/20 12:05
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
What is more significant?

Leave some Data out of training and run test against previous unseen data, or run test 1000x against DETREND = SHUFFLE?;

Re: Out of Sample [Re: danatrader] #480295
05/30/20 13:06
05/30/20 13:06
Joined: Feb 2017
Posts: 1,725
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,725
Chicago
Why not both?

Re: Out of Sample [Re: danatrader] #480300
05/30/20 20:02
05/30/20 20:02
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
Time.
How decide on contraindication?

Re: Out of Sample [Re: danatrader] #480302
05/30/20 21:12
05/30/20 21:12
Joined: Feb 2017
Posts: 1,725
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,725
Chicago
Reject when either says it’s bad. grin

Re: Out of Sample [Re: danatrader] #480304
05/30/20 22:05
05/30/20 22:05
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
Probably, better safe than sorry.

Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

Last edited by danatrader; 05/31/20 09:03.
Re: Out of Sample [Re: danatrader] #480324
06/01/20 01:30
06/01/20 01:30
Joined: Feb 2017
Posts: 1,725
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,725
Chicago
Originally Posted by danatrader
Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

You should think about what you're testing, and why.

If you optimize to dataset A, and you shuffle dataset A 1000 times, your optimized unshuffled configuration will almost necessarily reign king. This is not useful at all.

Whereas if you optimize to dataset A, then check it against an OOS dataset B, then shuffle dataset B 1000 times, the results are actually useful. (You want the first OOS test to outperform random data with statistical significance.)


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1