Detrend=SHUFFLE?

Hello,

I am educating myself how to use Zorro in the Robotwealth course and recently stumbled over a neat illustration of the Detrend = SUFFLE parameter.

In this course the authors conclude:
"If the StartDate and EndDate of your simulation don’t use all available price data, then the shuffled price curves will have different statistical properties to the data used in your backtest because they start at a different price level, but all finish at the same level."

They then go on to illustrate that you only get the same start and endpoint of your shuffled price curve, if you include all of the available data.

However, I have been unable to reproduce this behavior myself, no matter how much data I use there are alsways curves that start and/or end differently. So my question is whether the SHUFFLE functionality has been altered in later versions and if it is possible to shuffle the data while maintaining its statistical properties, like mean, sd, trend etc.?

No, shuffled curves should still start and end all at the same points. You can check that by printing the prices at the start and at the end of the historical data.

Ok then can you please explain why I get price curves with offset start/end values when I run a script like this over my entire data?

function run()
{
set(PLOTNOW);
StartDate = 20150101;
EndDate = 20161231;
BarPeriod = 1440;

NumTotalCycles = 50;
Detrend = SHUFFLE;

PlotWidth = 600;
PlotHeight1 = 300;
ColorUp = ColorDn = 0;
plot(strf("#Cycle_%d", TotalCycle), priceClose(), MAIN, color(100*TotalCycle/NumTotalCycles, BLUE, RED, GREEN));
}

see also the attached output. The values are relatively close, but it was my understanding that they should start/end at exactly the same value.

Your chart does not match start and end date and does not show the lookback period.

ups sorry, I copied the code in and changed the start and end date afterwards, my bad.
of course I included the entire historical data set I have in my history folder, by setting startdate to 20020101 and no enddate.

Edit: Thanks for the hint with the Lookback, though. Setting it to 0 makes the starting point equal, but the enddates still diverge slightly.

See script and output below:

function run()
{
set(PLOTNOW);
StartDate = 20020101;
BarPeriod = 1440;
LookBack = 0;

NumTotalCycles = 5;
Detrend = SHUFFLE;

PlotWidth = 1200;
PlotHeight1 = 600;
ColorUp = ColorDn = 0;
plot(strf("#Cycle_%d", TotalCycle), priceClose(), MAIN, color(100*TotalCycle/NumTotalCycles, BLUE, RED, GREEN));
}

You're right. I see in my notes that the LEAN flag must be set for Detrend=SHUFFLE. That info didn't make it into the manual. The reason is probably that without LEAN, open and close prices are stored in the ticks and affect the shuffling.

This will be either added to the manual or worked around so that the LEAN flag is not needed anymore.

Ah yes, now it works like it should, thanks a lot jcl!