Linear Regression in objective function

Posted By: Brax

Linear Regression in objective function - 11/16/17 16:33

Hi,

I am currently doing some research on optimization criteria and iīve got stuck trying to apply LinearRegSlope*R2 to optimize parameters.

This code gives me an error regarding i need more lookback period:

Code:
var objective()
{
    return LinearRegSlope(ResultsDaily,Bar-StartBar)*R2;   	
}


Iīve also tried with Day instead of Bar-StarBar, and even putting directly a number lower than lookback for the period gets a crush.

ŋHow could i achieve this?

The alternative is using (WinTotal-LossTotal)*R2, but this is not what i really want.
Posted By: jcl

Re: Linear Regression in objective function - 11/17/17 09:14

Day is correct for getting the regression slope of the equity curve, but be aware: ResultsDaily is in chronological order, not reversed like a series. So use the negative slope. Lookback must be set to the number of days in the simulation, or above.
Posted By: Brax

Re: Linear Regression in objective function - 11/20/17 13:13

Ok, I understand.

The problem with increasing lookback is i run out of enough bars... So i canīt properly test this criteria.

I thought i wouldnīt need lookback for this, as in other platforms is not necessary, just take the whole equity and go.
Posted By: jcl

Re: Linear Regression in objective function - 11/20/17 13:54

Then set lookback to 0 when you don't need it. If Lookback is not 0, Zorro assumes that you need it for something.
Posted By: Brax

Re: Linear Regression in objective function - 11/20/17 20:26

I see your point, but at least in this case if i set lookback to 0, then no optimization is performed (everything returns 0) and the system is useless.

The only solution as it seems to be is just having at least double historic data than simulation period, if not, i canīt use this function.

So, i am afraid iīll forget about this unless someone has an alternative solution.

Thanks.
Posted By: jcl

Re: Linear Regression in objective function - 11/21/17 07:45

Why so complicated? If you need lookback in the test, but not in the objective function, then set lookback in the run function to the value you need, and to 0 in the objective function.

The main purpose of variables like lookback is preventing errors. You normally adapt them to your script, not your script to the variables.
Posted By: Brax

Re: Linear Regression in objective function - 11/21/17 12:25

Sorry jcl but it still doesnīt work...

Iīve done as you state, and with other criteria goes fine but when i apply LinearRegSlope whether with Day, Bar-StarBar or a fixed number it crushes.

Quite strange indeed...

Thanks.
Posted By: jcl

Re: Linear Regression in objective function - 11/21/17 13:13

Ok. I cannot tell why "it crushes" because I have not yet used an indicator on something else than a data series. But here is how to look into such a problem.

First, check if ResultsDaily is really nonzero in the objective function. I believe it is, but otherwise the problem is clear.

Next, call LinearRegSlope with something simple, like an array { 1, 2, 3 } of length 3, and check if it still "crushes". If so, then LinearRegSlope only works with a real data series for some reason, and you need the more common polyFit function. When you're at this point, post again and I'll help.
Posted By: Brax

Re: Linear Regression in objective function - 11/21/17 19:32

Hi jcl.

ResultsDaily is nonzero and itīs reversed as you stated.

LinearRegSlope(ResultsDaily,Day) with LookBack = 0 doesn't seem to work properly. It always returns 1, maybe there is an issue with the reversed order of the series.

polyfit(0,ResultsDaily,Day,1,1) letting LookBack with its original value produces different results in training, but in test mode the system results aren't profitable when other criteria do.

I think i need to reverse the series or apply the slope over the differencies, need some help.

Thanks.


Posted By: jcl

Re: Linear Regression in objective function - 11/22/17 14:40

I've checked now, but all the indicators definitely work with LookBack=0. Lookback=0 just switches off the error message.

So your problem has a different reason that is unrelated to LookBack. Without seeing your code this is of course hard to tell. It's not reversing the series - this just changes the sign of the slope.

Some lines of code for testing LinearRegSlope wit no LookBack - maybe it helps:

Code:
void run()
{
	LookBack = 40;
	vars Prices = series(price(),40);
	if(Bar > 40) {
		LookBack = 0;
		printf("nSlope = %.6f",LinearRegSlope(Prices,40));
		LookBack = 40;
	}
}
Posted By: Brax

Re: Linear Regression in objective function - 12/04/17 20:34

Hi.

Sorry for not writing anything last days, iīve been kind of busy lately...

Coming back to the point discussed here, i must say i havenīt been able to make LinearRegSlope work properly in objective function. Something strange happen with this function as far as iīve tested.

I am using polyfit instead, but i think something is not working or maybe i am not getting the results i was expecting.

Letīs take this piece of code:

Code:
static int criteria = 6;
static int minTrade = 0;

var objective()
{
    if(NumWinTotal+NumLossTotal <= minTrade) {     
        return 0;
    }
    else { 
        switch(criteria){
            case 0: //PRR
                var wFactor = 1./sqrt(1.+NumWinTotal); 
                var lFactor = 1./sqrt(1.+NumLossTotal);
                var win = WinTotal, loss = LossTotal;
               
                if(NumWinTotal > 2) win -= (NumWinTotal-2)*WinMaxTotal/NumWinTotal;
                if(NumLossTotal > 2) loss -= (NumLossTotal-2)*LossMaxTotal/NumLossTotal;
                return (1.-wFactor)/(1.+lFactor)*(1.+win)/(1.+loss);
            case 1: //PF
                return WinTotal/max(1.,LossTotal);
            case 2: //WinRate
                return (var) NumWinTotal/max(1.,NumWinTotal+NumLossTotal);
            case 3: //CAR
                return WinTotal-LossTotal;
            case 4: //Calmar
                return (WinTotal-LossTotal)/max(1.,DrawDownMax);
            case 5: //Sharpe
                return ReturnMean/ReturnStdDev;
            case 6: //K-Ratio
                var coeff[2];
                LookBack = Bar-StartBar;
                polyfit(coeff,ResultsDaily,Bar-StartBar,1,1);
                return coeff[1]*R2;
                // LookBack = 0;
                // return LinearRegSlope(ResultsDaily, Day)*R2;
        }
    }
}

function run()
{
	set(LOGFILE+PARAMETERS);
	BarPeriod = 4*60;
	LookBack = 500;
	StartDate = 2005;
	EndDate = 2015;
	
    asset("EUR/USD");

	vars Price = series(price());
	vars Filtered = series(BandPass(Price,optimize(30,20,40),0.5));
	vars Signal = series(FisherN(Filtered,500));
	var Threshold = optimize(1,0.5,1.5,0.1);

	Stop = optimize(4,2,10) * ATR(100);
	Trail = 4*ATR(100);

	if(crossUnder(Signal,-Threshold))
		enterLong(); 
	else if(crossOver(Signal,Threshold))
		enterShort();
    
	plot("Filtered",Filtered,NEW,BLUE);
	plot("Signal",Signal,NEW,RED);
	plot("Threshold1",1,0,BLACK);
	plot("Threshold2",-1,0,BLACK);
	PlotWidth = 1024;
	PlotHeight1 = 400;
}


Optimizing this system with almost any criteria produces an annual return ranging from 80% to 100%. However with K-Ratio it barely achieves an 60%.

Not sure if K-Ratio isnīt as good as i thought or if iīve implemented it wrongly (most probably). The manual says something about polyfit having a cap of 1000 in TimePeriod parameter, so maybe here is the issue. I am also setting lookback to the whole number of bars in the simulation as this seems to work better.

This is everything iīve figured out. Any suggestion or advice will be welcomed.
© 2024 lite-C Forums