Gamestudio Links
Zorro Links
Newest Posts
Trailing on Ticks
by danatrader. 07/13/20 06:21
Misreading Monitor Resolution
by Dooley. 07/11/20 23:42
Help with Martingale Script
by Qw3rty. 07/11/20 12:17
Help with code
by Qw3rty. 07/11/20 12:03
Tick Volume
by danatrader. 07/11/20 06:43
FAC Train mode
by danatrader. 07/10/20 18:13
A9
by 3run. 07/09/20 19:13
AUM Magazine
Latest Screens
The Space Between
Pogostuck: Rage With Your Friends
Worst Case Z
AckCon'18 - Lotter vs the World 2 - Preview Release
Who's Online Now
4 registered members (danatrader, AdamWu, alibaba, MatPed), 389 guests, and 7 spiders.
Key: Admin, Global Mod, Mod
Newest Members
dasanc, DavidFdez, TraderKevin, deus777, carolthesecond
18476 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Out of Sample #480293
05/30/20 12:05
05/30/20 12:05
Joined: Mar 2019
Posts: 189
D
danatrader Online OP
Member
danatrader  Online OP
Member
D

Joined: Mar 2019
Posts: 189
What is more significant?

Leave some Data out of training and run test against previous unseen data, or run test 1000x against DETREND = SHUFFLE?;

Re: Out of Sample [Re: danatrader] #480295
05/30/20 13:06
05/30/20 13:06
Joined: Feb 2017
Posts: 1,024
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,024
Chicago
Why not both?

Re: Out of Sample [Re: danatrader] #480300
05/30/20 20:02
05/30/20 20:02
Joined: Mar 2019
Posts: 189
D
danatrader Online OP
Member
danatrader  Online OP
Member
D

Joined: Mar 2019
Posts: 189
Time.
How decide on contraindication?

Re: Out of Sample [Re: danatrader] #480302
05/30/20 21:12
05/30/20 21:12
Joined: Feb 2017
Posts: 1,024
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,024
Chicago
Reject when either says it’s bad. grin

Re: Out of Sample [Re: danatrader] #480304
05/30/20 22:05
05/30/20 22:05
Joined: Mar 2019
Posts: 189
D
danatrader Online OP
Member
danatrader  Online OP
Member
D

Joined: Mar 2019
Posts: 189
Probably, better safe than sorry.

Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

Last edited by danatrader; 05/31/20 09:03.
Re: Out of Sample [Re: danatrader] #480324
06/01/20 01:30
06/01/20 01:30
Joined: Feb 2017
Posts: 1,024
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,024
Chicago
Originally Posted by danatrader
Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

You should think about what you're testing, and why.

If you optimize to dataset A, and you shuffle dataset A 1000 times, your optimized unshuffled configuration will almost necessarily reign king. This is not useful at all.

Whereas if you optimize to dataset A, then check it against an OOS dataset B, then shuffle dataset B 1000 times, the results are actually useful. (You want the first OOS test to outperform random data with statistical significance.)


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1