John Ehlers' Autocorrelation Periodogram

Posted By: boatman

John Ehlers' Autocorrelation Periodogram - 11/19/14 10:14

Hello jcl and others

I was wondering if you could help me translate John Ehlers' Autocorrelation Periodogram into Lite-C. The indicator is found in the book 'Cycle Analytics for Traders'. The indicator uses an autocorrleation process to calculate the dominant period, and I believe will do so with less lag than Zorro's DominantPeriod function.

I will firstly describe the indicator's calculation, then I will post the TradeStation code that appears in the book, and finally I will post my attempt and describe where I am stuck.

The indicator is calculated by firstly pre-filtering the data using the roofing filter. I chose low and high cutoff periods of 10 and 50. Then, we correlate the output of the filter with the output delayed by each of a series of lag values, using Person correlation. Ehlers autocorrelates using 10 to 48 bars of lag, and I will try to do the same. Each value of lag is assigned a correlation coefficient between -1 (perfect anticorellation) and 1 (perfect correlation).

After calculating the autocorrelation for each value of lag, Ehlers extracts the cyclic information using a discrete Fourier transform of the autocorrelation results. This is accomplished by correlating the autocorrelation at each value of lag with the cosine and sine of each period of interest. The sum of the squares of each of these values represents relative power at each period from trigonometry. An EMA is used to smooth the power measurement at each period.

Next, the automatic gain control function is used to normalize the spectral components. Finally, the dominant cycle is extracted using a centre of gravity algorithm.

Ehlers uses a heat map display to show the strength of each period at each bar. I don't think this is possible in Zorro, but that shouldn't stop us from extracting the dominant cycle using this method.

I've attached the pages from 'Cycle Analytics for Traders' that describe the indicator, including the TradeStation code. The TradeStation code is also below:

Code:
{
Autocorrelation Periodogram
© 2013	John F. Ehlers
}

Vars:
AvgLength(3), M(0),
N(0),	 
X(0),
Y(0),
alpha1(0), HP(0),
a1(0),
b1(0),
c1(0),
c2(0),
c3(0),
Filt(0),
Lag(0),
count(0),
Sx(0),
Sy(0),
Sxx(0),
Syy(0),
Sxy(0),
Period(0), Sp(0),
Spx(0),
MaxPwr(0), DominantCycle(0), Color1(0), Color2(0), Color3(0);

Arrays:
Corr[48](0),
CosinePart[48](0), SinePart[48](0),
SqSum[48](0),
R[48, 2](0),
Pwr[48](0);

//Highpass filter cyclic components whose periods are shorter than 48 bars
alpha1 =  (Cosine(.707*360 / 48) +  Sine (.707*360 / 48) - 1) / Cosine(.707*360 / 48);
HP =  (1 - alpha1 / 2)*(1 - alpha1 / 2)*(Close - 2*Close[1] +
Close[2]) +  2*(1 - alpha1)*HP[1] - (1 - alpha1)* (1 - alpha1)*HP[2];
//Smooth with a Super Smoother Filter from equation 3-3
 

a1 =  expvalue(-1.414*3.14159 / 10); b1 =  2*a1*Cosine(1.414*180 / 10); c2 =  b1;
c3 =  -a1*a1;
c1 =  1 - c2 - c3;
Filt =  c1*(HP +  HP[1]) / 2 +  c2*Filt[1] +  c3*Filt[2];
 

//Pearson correlation for each value of lag For Lag =  0 to 48 Begin
//Set the averaging length as M M =  AvgLength;
If AvgLength =  0 Then M =  Lag;
Sx =  0;
Sy =  0;
Sxx =  0;
Syy =  0;
Sxy =  0;
For count =  0 to M - 1 Begin X =  Filt[count];
Y =  Filt[Lag +  count]; Sx =  Sx +  X;
Sy =  Sy +  Y;
Sxx = Sxx + X*X; Sxy = Sxy + X*Y; Syy =  Syy +  Y*Y;
End;
If (M*Sxx - Sx*Sx)*(M*Syy - Sy*Sy) > 0 Then Corr[Lag] =
(M*Sxy - Sx*Sy)/SquareRoot((M*Sxx - Sx*Sx)*(M*Syy - Sy*Sy)); End;
For Period =  10 to 48 Begin CosinePart[Period] =  0;
SinePart[Period] =  0; For N =  3 to 48 Begin
CosinePart[Period] =   CosinePart[Period] +
Corr[N]*Cosine(370*N / Period);
SinePart[Period] = SinePart[Period] + Corr[N]*Sine(370*N / Period);
End;
SqSum[Period] =    CosinePart[Period]*CosinePart[Period] +
SinePart[Period]*SinePart[Period];
End;	(Continued )
 
For Period =  10 to 48 Begin R[Period, 2] =  R[Period, 1];
R[Period, 1] =  .2*SqSum[Period]*SqSum[Period] +
.8*R[Period, 2]; End;

//Find Maximum Power Level for Normalization MaxPwr =  .995*MaxPwr;
For Period =  10 to 48 Begin
If R[Period, 1] > MaxPwr Then MaxPwr =  R[Period, 1]; End;
For Period =  3 to 48 Begin
Pwr[Period] =  R[Period, 1] / MaxPwr; End;
 
//Compute the dominant cycle using the CG of the spectrum Spx =  0;
Sp =  0;
For Period =  10 to 48 Begin
If Pwr[Period] >=  .5 Then Begin Spx =  Spx +  Period*Pwr[Period]; Sp =  Sp +  Pwr[Period];
End; End;
If Sp <> 0 Then DominantCycle =  Spx / Sp;
Plot2(DominantCycle, “DC”, RGB(0, 0, 255), 0, 2);

{
//Increase Display Resolution by raising the NormPwr to a higher mathematical power (optional)
For Period =  10 to 48 Begin Pwr[Period] =  Power(Pwr[Period], 2);
End;
}



And here is my so far incomplete attempt:
Code:
function run()
{
//=====John Ehlers' Autocorrelation Periodogram=====

//Roofing Filter used on prices
vars RoofPrice = series(Roof(Price, 10, 40)); //standard roofing filter
vars RoofAGC = series(AGC(RoofPrice, 1)); //output normalized using AGC algorithm

//Ehlers Autocorrelation Periodogram

var corr[48]; //an array that will store the correlation coefficient for each value of lag from 1 to 48
for (i = 1; i < 49; i++)
	corr[i] = Correlation(RoofAGC, RoofAGC+i, i); //assign a correlation coefficent to each value of lag. 

//discrete Fourier transform of autcorrelation results
}



As you can see, I'm stuck on how to do the discrete Fourier transform. I just don't understand how to translate the TradeStation code for this part into Lite-C. I'm also unsure of how to best handle the arrays. Any advice or tips would be really appreciated, and hopefully enable me to translate the remaining TradeStation code.

Thanks in advance!

Attached File
Posted By: boatman

Re: John Ehlers' Autocorrelation Periodogram - 11/20/14 02:19

Hello again

I've taken my attempt at Ehlers' autocorrelation periodogram further, but am still stuck. When I run the script below, I get a crash. Any suggestions? I've tried to convert this from the TradeStation code, but I must admit that I am a little lost.

Also, I found that you can't declare an array and set the size using a previously declared variable. eg var my_array[LowCutoff]; results in a syntax error even when LowCutoff is declared. Why is this?

Obviously I'm still a learner when it comes to programming, so any advice is greatly appreciated.

Code:
//===== Ehlers Autocorrelation Periodogram =====

function run()
{
vars Price = series(price());

// Roofing filter applied to price series
var LowCutoff = 10;
var HighCutoff = 48;
vars RoofPrice = series(Roof(Price, LowCutoff, HighCutoff));
vars RoofPriceAGC = series(AGC(RoofPrice, 1)); //normalize output with automatic gain control

// Pearson correlation for each value of lag
var corr[48]; //declare an array to store the correlation coefficients for each value of lag up to the HighCutoff
int lag;
int AvgLength = 0; //Averaging length for the correlation algorithm. Ehlers recommends a value of 3.
int M; //used to set AvgLength to the lag value if AvgLength is not set
for (lag = 0; lag <= HighCutoff; lag++)
	{
		if (AvgLength == 0)
			M = lag;
		else M = AvgLength;
	
		corr[lag] = Correlation(RoofPriceAGC, RoofPriceAGC+lag, M); //assign a correlation coefficient for each value of lag to the array corr[]
	}
	
//Discrete Fourier transform. Correlate autocorrelation values with the cosine and sine of each period of interest. The sum of the squares of each value represents relative power at each period
var cosinePart[38]; //array for storing the cosine values
var sinePart[38]; //array for storing the sine values
var sqSum[38]; //array for storing the squared and summed values of the sine and cosine parts
int period;
int N;
for (period = LowCutoff; period <= HighCutoff; period++)
	{
		for (N = 3; N <= HighCutoff; N++)
		{
			cosinePart[period] = cosinePart[period] + corr[N]*cos((370*N*PI/180)/period);
			sinePart[period] = sinePart[period] + corr[N]*sin((370*N*PI/180)/period);
		}
		sqSum[period] = cosinePart[period]*cosinePart[period] + sinePart[period]*sinePart[period];
	}
	//Smooth power measurements using an EMA
var R[2][35];
for (period = LowCutoff; period <= HighCutoff; period++)
	{
		R[1][period] = R[0][period];
		R[0][period] = 0.2*sqSum[period]*sqSum[period] + 0.8*R[1][period];
	}
//Apply automatic gain control with decay factor 0.995
vars maxPwr = series(0);
maxPwr[0] = 0.995*maxPwr[1];
for (period = LowCutoff; period <= HighCutoff; period++)
	{
		if (R[0][period] > maxPwr[0])
			maxPwr[0] = (R[0][period]);
	}
	
var pwr[45]; 
for (period = 3; period < HighCutoff; period++)
	{
		pwr[period] = R[0][period] / maxPwr[0];
	}	

//Compute dominant cycle using centre of gravity algorithm
var Spx = 0;
var Sp = 0;
var DomPer; //Dominant cycle period
for (period = LowCutoff; period <= HighCutoff; period++)
	{
		if (pwr[period] >= 0.5)
			{
				Spx = Spx + period*pwr[period];
				Sp = Sp + pwr[period];
			}
	}
if (Sp != 0)
	DomPer = Spx/Sp;	
	
plot("DominantCyclePeriod", DomPer, NEW, BLUE);
	
	
}

Posted By: byakuren81

Re: John Ehlers' Autocorrelation Periodogram - 11/20/14 15:59

hello, the original code contains a serie of mistake :
- its 360 and not 370 so you should have :
corr[N]*cos(2*N*PI/period);
corr[N]*sin(2*N*PI/period);
-for (period = 3; period < HighCutoff; period++)
{
pwr[period] = R[0][period] / maxPwr[0];
}
here R[0][period] has been calculated only from period = LowCutOff and not for the values between 3 and LowCutOff-1
-also you are calculating on 39 terms (48-10+1) in many of your loops whereas your arrays can contain only 38 values
-I can see other index problems in your arrays exceeding their initial dimensions
Posted By: byakuren81

Re: John Ehlers' Autocorrelation Periodogram - 11/20/14 16:11

could look more like that :

//===== Ehlers Autocorrelation Periodogram =====

function run()
{
vars Price = series(price());

var LowCutoff = 10;
var HighCutoff = 48;
vars RoofPrice = series(Roof(Price, LowCutoff, HighCutoff));
vars RoofPriceAGC = series(AGC(RoofPrice, 1));

var corr[48];
int lag;
int AvgLength = 0;
int M;
for (lag = 0; lag <= HighCutoff; lag++)
{
if (AvgLength == 0)
M = lag;
else M = AvgLength;

corr[lag] = Correlation(RoofPriceAGC, RoofPriceAGC+lag, M);
}
var cosinePart[HighCutoff+1];
var sinePart[HighCutoff+1];
var sqSum[HighCutoff+1];
int period;
int N;
for (period = LowCutoff; period <= HighCutoff; period++)
{
for (N = 3; N <= HighCutoff; N++)
{
cosinePart[period] = cosinePart[period] + corr[N]*cos(2*N*PI/period);
sinePart[period] = sinePart[period] + corr[N]*sin(2*N*PI/period);
}
sqSum[period] = cosinePart[period]*cosinePart[period] + sinePart[period]*sinePart[period];
}
var R[2][HighCutoff+1];
for (period = LowCutoff; period <= HighCutoff; period++)
{
R[1][period] = R[0][period];
R[0][period] = 0.2*sqSum[period]*sqSum[period] + 0.8*R[1][period];
}
vars maxPwr = series(0);
maxPwr[0] = 0.995*maxPwr[1];
for (period = LowCutoff; period <= HighCutoff; period++)
{
if (R[0][period] > maxPwr[0])
maxPwr[0] = (R[0][period]);
}
var pwr[HighCutoff+1];
for (period = LowCutoff; period <= HighCutoff; period++)
{
pwr[period] = R[0][period] / maxPwr[0];
}
var Spx = 0;
var Sp = 0;
var DomPer; //Dominant cycle period
for (period = LowCutoff; period <= HighCutoff; period++)
{
if (pwr[period] >= 0.5)
{
Spx = Spx + period*pwr[period];
Sp = Sp + pwr[period];
}
}
if (Sp != 0)
DomPer = Spx/Sp;

plot("DominantCyclePeriod", DomPer, NEW, BLUE);


}
Posted By: byakuren81

Re: John Ehlers' Autocorrelation Periodogram - 11/20/14 16:21

I programmed in different languages all Ehlers tools in the past and was in contact with him as they are many mistakes in his last book, I encourage you to check the code in the following chapters of this book and compare as he will use it again and sometimes with the mistakes corrected or with new ones...Ehlers did an amazing work and brought many new tools for traders however to my mind the one you have to focus on are the super smoother and other decyclers and oscillators as the band pass filter and improvements brought to classical tools as RSI. Moreover computing the dominant cycle is relevant only in the case of stocks as currencies are more trending than mean reverting, even if Ehlers showed in the previous books some example on forex strategies his work has been more concentrated on stocks. Finally this code will not all the time give back a value because of the final test on the power which has to be bigger than 0.5, you can see the same on his graph, sometimes the dominant cycle remains constant equal to the last value where power was bigger than 0.5
If you are a beginner in programming I encourage you to start by simple things and espacially things containing no mistake otherwise you will be quickly fed up !
good luck to you
Posted By: boatman

Re: John Ehlers' Autocorrelation Periodogram - 11/21/14 01:38

Hi byakuren81

Thank you so much for you help and feedback. I learned a great deal from studying the code you provided, and will now try to implement it.

I will do as you advise and focus on the super smoother, decyclers and band-pass filter, as well as the improvements to the RSI.

I was also very interested to read your comments on the applicability of the dominant cycle to mean reverting markets such as the stock market. I recently read Keith Fitschen's book 'Building Reliable Trading Systems' and he talks about this extensively for stocks, commodities and forex. He writes that his research indicates that currencies paired with the US dollar are generally trend-following instruments, while the rest are generally counter-trend instruments, at least on daily bars. On intra-day bars, he believes there is a greater bias towards trend-following. I plan to look into this further, and would be more than happy to share my results if you're interested.

Thanks again for taking the time to reply, it really was a great help.

Cheers
Posted By: simplex

Re: John Ehlers' Autocorrelation Periodogram - 02/21/15 18:01

Originally Posted By: byakuren81
I encourage you to check the code in the following chapters of this book and compare as he will use it again and sometimes with the mistakes corrected or with new ones.


Hi boatman & byakuren81,

I'm a complete newbie when it comes to lite-C, but an experienced MQL coder looking for alternative platforms.

I'm also working on this Ehlers' stuff in MQ4 at the moment, which should result in similar code structures.

Actually I interpret your code that the upper and lower cutoff periods are user's inputs. Correct? If so, you should change the power decay factor 0.995 (or 0.991 according to page 103 of Ehlers' book) accordingly. That hardcoded value is only valid for the standard set of parameters (10,48). I developed the general formula for this. If you should be interested, and would share it.

And you seem to utilize a Zorro standard function for the Roofing Filter part of your algorithm. Is this based on a 1-pole or a 2-pole high-pass? Should be 2-pole in this case to work correctly.

My initial results regarding the Dominant Cycle output were not really promising on FX data. Much too busy, even on Daily prices. I'm not sure whether this is caused by the nature of the FX data itself or by a bug that I introduced. Will carry on ...

Cheers, simplex
Posted By: boatman

Re: John Ehlers' Autocorrelation Periodogram - 02/22/15 11:00

Hi Simplex

I'm really interested in your general formula for power decay factor. Would you mind sharing it?

I can confirm that the Roofing Filter implementation in Zorro is a 2-pole highpass filter in series with Ehlers' Super Smoother.

I too have not had a great deal of success with this version of the Dominant Cycle in forex and have recently been diverting my focus elsewhere.
Posted By: simplex

Automatic Gain Control - 02/22/15 13:13

Hi boatman,

Here is how I coded it in MQ4 (should be more or less the same in lite-C):

/* -----------------------------------------------------------
agcDecayFactor
calculate decay factor for automatic gain control (AGC)
Concept see John Ehlers: Cycle Analytics For Traders, 54f
----------------------------------------------------------- */
double agcDecayFactor(const int _lowerCutoff, const int _higherCutoff, const double _acceptableSlope = 1.5) {
double hc = (double)_higherCutoff / 2.0; // calculate half the cutoff periods
double lc = (double)_lowerCutoff / 2.0;
double delta = MathAbs(hc - lc); // delta >= 0, so division in expo is always safe
double accSlope = -MathAbs(_acceptableSlope); // make sure gain slope in decibels is negative
double expo = accSlope / (delta * (delta + 1.0)); // calculate exponent to solve logarithm (base: 10)
double out = MathPow(10.0, expo);
return(out);
}


I'll also attach a scan showing how I derived my formula from Ehlers' text on pp. 54-55. Hope it's clear enough. If not, don't hesitate to ask.

Are you aware of the fact that Ehlers sometimes uses the decay factor K directly (0.991), sometimes its sqare root (0.995)? In the autocorrelation chapter, he uses 0.995 in his code listing (8-3), but mentions 0.991 in the text on page 103. Since there is another mistake in the code (370 vs. 360 degrees), I would think that 0.991 is correct. What's your opinion?

Cheers, simplex




Description: AGC: derivation of decay factor formula
Attached picture AGC_formula_simplex.jpg
Posted By: boatman

Re: Automatic Gain Control - 02/23/15 08:22

Hi Simplex

Thanks for sharing your derivation of general formula for the AGC decay factor. Very useful!

Those errors you pointed out had escaped my attention previously. I can't see any reason to use the square root of the decay factor in code listing 8.3, so unless I am missing something, I have to agree with you that 0.991 is correct.
Posted By: cifr

Re: Automatic Gain Control - 10/29/16 03:20

Hi simplex,

Appreciate your formula, however I'd like to say this is not the general equation, because delta only equals to 19 in this specific case, and the 20 is not derived from delta + 1, its origin is from the same book page 13, that 20 is a constant in this equation, if the delta changes your formula could be wrong.

the correct equation should be K = 10^(-3/(20*(longP-shortP))

no offence, just want to help laugh
Posted By: johnnyp

Re: Automatic Gain Control - 10/11/17 20:14

I know this topic is old, but it seems worth mentionning...

The zorro examples in this thread all run the prices through a roofing filter AND THEN through an automatic gain control (AGC) before calculating the correlation coefficients.

Ehlers code on the other hand, doesn't use an AGC at that point.

Besides wouldn't the AGC mess up some of the correlations in the data?
© 2024 lite-C Forums