Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
2 registered members (Quad, aliswee), 835 guests, and 5 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Out of Sample #480293
05/30/20 12:05
05/30/20 12:05
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
What is more significant?

Leave some Data out of training and run test against previous unseen data, or run test 1000x against DETREND = SHUFFLE?;

Re: Out of Sample [Re: danatrader] #480295
05/30/20 13:06
05/30/20 13:06
Joined: Feb 2017
Posts: 1,718
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,718
Chicago
Why not both?

Re: Out of Sample [Re: danatrader] #480300
05/30/20 20:02
05/30/20 20:02
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
Time.
How decide on contraindication?

Re: Out of Sample [Re: danatrader] #480302
05/30/20 21:12
05/30/20 21:12
Joined: Feb 2017
Posts: 1,718
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,718
Chicago
Reject when either says it’s bad. grin

Re: Out of Sample [Re: danatrader] #480304
05/30/20 22:05
05/30/20 22:05
Joined: Mar 2019
Posts: 357
D
danatrader Offline OP
Senior Member
danatrader  Offline OP
Senior Member
D

Joined: Mar 2019
Posts: 357
Probably, better safe than sorry.

Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

Last edited by danatrader; 05/31/20 09:03.
Re: Out of Sample [Re: danatrader] #480324
06/01/20 01:30
06/01/20 01:30
Joined: Feb 2017
Posts: 1,718
Chicago
AndrewAMD Online
Serious User
AndrewAMD  Online
Serious User

Joined: Feb 2017
Posts: 1,718
Chicago
Originally Posted by danatrader
Run 1000 times against the artifical price curve of the previous unseen data, or against the artificial curve of the previously used train data.

Or again both?

You should think about what you're testing, and why.

If you optimize to dataset A, and you shuffle dataset A 1000 times, your optimized unshuffled configuration will almost necessarily reign king. This is not useful at all.

Whereas if you optimize to dataset A, then check it against an OOS dataset B, then shuffle dataset B 1000 times, the results are actually useful. (You want the first OOS test to outperform random data with statistical significance.)


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1