Ahead bias when retrain

Posted By: JRA

Ahead bias when retrain - 05/22/18 11:10

I have a question: If the average trade duration is 5 bars (example) and Zorro retrain on every bar, can the WFO be "overlapped"? Is there possible the "ahead bias"?

For example:

NumWFOCycles = 5 * DataSplit/(100-DataSplit); // And the trade runs 6 days.

Thanks.
Posted By: jcl

Re: Ahead bias when retrain - 05/22/18 12:18

Yes, it is possible. You must set DataHorizon when WFO cycles can overlap.
Posted By: JRA

Re: Ahead bias when retrain - 05/22/18 18:28

Sorry but I can not find a practical example of DataHorizon.
When the DataHorizon is set correctly for me (Train sample bars long) I donīt get trades.
Can you writte a little example please?

Thank you very much.

Posted By: JRA

Re: Ahead bias when retrain - 05/23/18 08:32

All the examples on Zorro ML examples than I found (Financial Hacker or Robot Wealth) uses:

WFOPeriod = 252*24; // 1 year

But without set the asset before.
I run two "train and test" on the same algo, first setting the asset on the slider and the second on the script, the results are very differents


This explains why when set the DataHorizon correctly I get no trades.
Are the "succesfull" examples peeking on the future?


Attached picture L_X_ES_30.png
Attached picture L_X_ES_30(Ticker).png
Posted By: jcl

Re: Ahead bias when retrain - 05/23/18 14:21

If your profit goes down when you set DataHorizon, then you have indeed been peeking into the future. But if your script does not trade at all, then debug it and find the reason, for instance a wrong trade logic or a mistake on cycle setup.
Posted By: JRA

Re: Ahead bias when retrain - 05/23/18 16:28

Originally Posted By: jcl
If your profit goes down when you set DataHorizon, then you have indeed been peeking into the future. But if your script does not trade at all, then debug it and find the reason, for instance a wrong trade logic or a mistake on cycle setup.


StartDate = 20000101;
BarPeriod = 1440; // 1 day
LookBack = 100;

WFOPeriod = 252; // 1 year
DataSplit = 90;
DataHorizon = 25; // the 10% of WFO, also the test period

Itīs correct? The rest of the script donīt matters, mine or yours.
Posted By: jcl

Re: Ahead bias when retrain - 05/24/18 10:27

No, it is not correct. Look here: http://manual.zorro-project.com/dataslope.htm

I do not know your data horizon, but it is certainly not 25.
Posted By: JRA

Re: Ahead bias when retrain - 05/25/18 08:05

Sorry JCL but here are a very important question without answer:

When everyone uses the "WFOPeriod" without setting the asset before the script, is peeking ahead.
This happens with all scripts I can found, not only mines.
This question is explained in the manual "need to set the asset before", but it opens the door to: are the OOS test a full Out of sample?

I think is not possible to train all the data sample without know the test periods from start to end, almost without timestamps to know if a test period has been crossed the "trained zone".

My script donīt use future bars explicitly, dontīt have the PEEK flag, the only future leak comes from the train.

Thanks.


PD: If the WFO is correct Iīm rich before Christmas.



Posted By: Spirit

Re: Ahead bias when retrain - 05/25/18 08:46

Peeking ahead means you use future information for trading. Then your result is worthless no matter how you set the asset before the script! laugh So, hope of getting rich may be wrong...
Posted By: JRA

Re: Ahead bias when retrain - 05/25/18 08:54

Originally Posted By: Spirit
Peeking ahead means you use future information for trading. You cannot fix it by setting the asset before the script! laugh


Yes, you can, because when Zorro knows the asset can split correctly the cycles.
You can see the images attached up, the only change is the asset set.

And more: if you train the algo without setting the asset on the script , and set it before test, you donīt get trades, because test period OOS do not exists.

From the manual:

// calculate NumWFOCycles from the test period in days
// DataSplit and LookBack must be set before
function setWFO2(int daysTest)
{
asset(Asset); // requires NumBars, so asset must be set
int barsTest = daysTest * 1440/BarPeriod;
Posted By: jcl

Re: Ahead bias when retrain - 05/25/18 12:03

Calling asset() is needed to select the asset. It has absolutely nothing to do with peeking bias.

I do not really understand your problem with the WFO setup. Anyway, when you post your code and explain in more detail what you want to achieve, people can most likely help you. If you don't want to reveal your code here, you can buy a support ticket and let Support fix your setup.
Posted By: JRA

Re: Ahead bias when retrain - 05/25/18 14:53

This is the code. I get different results when the asset is selected on the scrollbox or settled in the script.




Code:
// XGBoost on S&P500

#include <r.h>
#include <profile.c>
#include <contract.c>

var neural ........................
{	
...................................	
...................................	
}	
	
function run()
{	
	NumCores = 8;
set(RULES|PLOTNOW|OPENEND);
	StartDate = 20020101;
//	EndDate = 20180515;
	BarPeriod = 1440;
	LookBack = 400;
	asset("ES");  // Fantastic results when comment and select it from scrollbox
   DataSplit = 90;
   WFOPeriod = 25 * DataSplit/(100-DataSplit); /* One month test and one year train, but better profits when lower test period, this made the suspicions */
	

	vars Close = series(priceClose());
	var desv = 2;
	var volumeDown = marketVol();
	var volumeUp = marketVal();
	
/* And the signals and transformations.............
.......................................
........................................ */

	if(Train) 
	{
		 
		Hedge = 2;
		NumSampleCycles = 4;
		LifeTime = 1;
		if(adviseLong(NEURAL+BALANCED, 0, 
			Sig1,Sig2, Sig3, Sig4,Sig5,Sig6,Sig11,Sig16,Sig17,Sig18,Sig20) > 0.5)
		{
			enterLong(1);
		}
		if(adviseShort() > 0.5)
		{
			enterShort(1);
		}
	}
	
	if(!Train)
	{
		var LongPrediction = adviseLong(NEURAL+BALANCED,0, Sig1,Sig2, Sig3,Sig4,Sig5,Sig6,Sig7,Sig11,Sig16,Sig17,Sig18,Sig20);
			
		var ShortPrediction = adviseShort();

		if(LongPrediction > 0.5 && ShortPrediction < 0.5)
			{	Stop = 4*ATR(20);
		      TakeProfit = 6*ATR(20);
				reverseLong(1);
			}

		
		if(ShortPrediction > 0.5 && LongPrediction < 0.5)
			{  Stop = 4*ATR(20);
		      TakeProfit = 4*ATR(20);
				reverseShort(1);
			}
		 PlotWidth = 800;
	    PlotHeight1 = 600;
	}
 
}

Posted By: Spirit

Re: Ahead bias when retrain - 05/25/18 18:38

you got the asset call right this time but the datahorizon is still not set. so you have still the peeking bias.
Posted By: JRA

Re: Ahead bias when retrain - 05/25/18 18:47

I have never seen the "DataHorizon" in a script, included in any Zorro version or in the web.
And "DataHorizon" does not exist for some versions ŋ?,I canīt compile it.

And yes, itīs exactly like the Deeplearn.c. Is this one peeking bias too? shocked
Posted By: Spirit

Re: Ahead bias when retrain - 05/25/18 19:07

you need datahorizon when you have peeking bias. the deeplearn.c and the other examples have no peeking bias, but in your script with only 25 bars test period you have a large overlap and large peeking bias.

if you have an old zorro with no data horizon, no problem, then you must just simulate the data horizon in your script by checking the overlap and not trading during that time. That was described in the old manual. hope this helps!
Posted By: JRA

Re: Ahead bias when retrain - 05/25/18 21:52

I have the newest versions, and DataHorizon it’s compiling again. I have goblins on my pc or the compiler has remote control. grin



© 2024 lite-C Forums