Different Result depending on Order of adviseLong / adviceShort

Posted By: GPEngine

Different Result depending on Order of adviseLong / adviceShort - 01/11/14 21:22

I have read http://zorro-trader.com/manual/en/advisor.htm and http://zorro-trader.com/manual/en/tutorial_pre.htm (Workshop7). I have been working under the assumption that the models built by adviseLong and adviseShort were independent. I tested this assumption and found it is not the case. I get very different results depending on whether I call adviseLong followed by adviseShort, or the other way around.

Code:

#define LONG_FIRST

function run() {
  if (is(INITRUN)) {
    BarPeriod = 80;
    BarOffset = 40;
    LookBack = 245;
    NumWFOCycles = 6;
    StartDate = 20130601;
    EndDate = 20131230;
    set(NFA + TICKS + RULES + TESTNOW + PLOTNOW + LOGFILE);
    TradesPerBar = 2;
    Weekend = 1;
    Hedge = 4;
  }

  vars ol = series((priceOpen(0) - priceLow(0)) / PIP);

  var v00 = APO(ol, 4, 7, MAType_WMA);
  var v01 = APO(ol, 148, 245, MAType_TRIMA);
  polyfit(0, ol, 24, 2, 1);
  var v02 = polynom(0, -1);

  TakeProfit = 3 * ATR(100);
  Stop = 3 * ATR(100);

  int al;
  int as;
#ifdef LONG_FIRST
  al = adviseLong(PERCEPTRON, 0, v00, v01, v02);
  if (al > 0) {
    enterLong();
  }
  as = adviseShort();
  if (as > 0) {
    enterShort();
  }
#else
  as = adviseShort(PERCEPTRON, 0, v00, v01, v02);
  if (as > 0) {
    enterShort();
  }
  al = adviseLong();
  if (al > 0) {
    enterLong();
  }
#endif

  plot("al", al, LINE + NEW , BLUE);
  plot("as", as, LINE, RED);
}

[Train] with #define LONG_FIRST
==> Annual loss -28325p
[Train] without #define LONG_FIRST
==> Annual +42% +9366p

I attached the Performance Chart as long_then_short.png and short_then_long.png, respectively.

Additional observations.
Whichever model is first will have little variation and will always have the values -100 or 100. Whichever model is second will have reasonable variation and the values vary depending on cycle. I observed +/- 48, 55, and 53. See red/blue chart in the attached images.

I expected it to not matter which one is called first, since the outcome of only the very next enterLong/enterShort forms the prediction with which to train the model.

Quote:

Prediction Value to be trained for prediction by the current Signal combination. Either a positive value for suggesting a trade, or a negative value for advising against a trade, or 0 for using the result of the next trade for prediction. This parameter is only used in the training run and has no meaning in test or trade mode.

Can you please shed some light on this?

Attached picture long_then_short.png

Posted By: GPEngine

Re: Different Result depending on Order of adviseLong / adviceShort - 01/12/14 00:34

I seem to get more stable results when I do

Code:

if (Train) {
  Hedge = 2;
} else {
  Hedge = 4;
  set(NFA);
}

Posted By: GPEngine

Re: Different Result depending on Order of adviseLong / adviceShort - 04/26/14 15:39

Reviving this thread. This is still a huge problem for me.

These two scripts give different results. Can anyone explain why? The only difference is the order that adviseLong+enterLong and adviseShort+enterShort appear in the code.

adviseLong first --> Annual +9% +555p
adviseShort first --> Annual +37% +985p

Code:

string selected_asset;

//#define SHORTFIRST

var EntryL;
var EntryS;
var TrailL;
var TrailS;

function run() {
  if (is(INITRUN)) {
    BarPeriod = 127;
    LookBack = 501;
    StartDate = 20130601;
    EndDate = 20131230;
    set(NFA + TICKS + MUTE);    
    set(RULES);
    set(FACTORS);
    set(TESTNOW + PLOTNOW);
    
    if (Train) {
      Hedge = 2;
      Detrend = 1;
    } else {
      Hedge = 4;
      Detrend = 0;
    }
    TradesPerBar = 2;
    Weekend = 1;
    EntryTime = 1;
    ExitTime = 24 * 60 / BarPeriod;
    NumWFOCycles = 3;
  }

  while(selected_asset = loop("AUD/CHF", "CAD/JPY", "EUR/AUD")) {
    asset(selected_asset);

    var atrx = ATR(329);

    EntryL = 1.6;
    EntryS = 1.6;
    TrailL = 1.2;
    TrailS = 1.2;

    var v00 = WillR(102);
    var v01 = CCI(192);
    vars Price = series(price());
    var v02 = HighPass(Price, DominantPeriod(Price, 7));
    
    plot("v00", v00, NEW+LINE, BLUE);
    plot("v01", v01, NEW+LINE, BLUE);
    plot("v02", v02, NEW+LINE, BLUE);

#ifdef SHORTFIRST
    if (adviseShort(PERCEPTRON, 0, v00, v01, v02) > 0) {
      Entry = EntryS * atrx;
      Trail = TrailS * atrx;
      enterShort();
    }
#endif
    if (adviseLong(PERCEPTRON, 0, v00, v01, v02) > 0) {
      Entry = EntryL * atrx;
      Trail = TrailL * atrx;
      enterLong();
    }
#ifndef SHORTFIRST
    if (adviseShort(PERCEPTRON, 0, v00, v01, v02) > 0) {
      Entry = EntryS * atrx;
      Trail = TrailS * atrx;
      enterShort();
    }
#endif
  }
}

Posted By: dusktrader

Re: Different Result depending on Order of adviseLong / adviceShort - 04/26/14 18:31

Hi GPE,
this might sound like a really stupid question but -- I can't figure out why you think the order of trades _wouldn't_ affect the outcome?

Generally speaking, why do you think the ordering of longs or shorts in any strategy should necessarily produce equal (or similar) results?

Sorry if I'm over simplifying; I might be missing something important you're trying to point out.

Posted By: GPEngine

Re: Different Result depending on Order of adviseLong / adviceShort - 04/27/14 15:46

During training, adviseLong and adviseShort return True unconditionally. So, Zorro makes a trade (2 trades in this case) unconditionally at every single bar. This is how Zorro knows what target to train against. These trades all need to be independent with no interference. The account state, other open trades, etc., should not play a role.

Posted By: GPEngine

Re: Different Result depending on Order of adviseLong / adviceShort - 04/27/14 15:54

I see that the order long/short might play a small role in the Test phase, when perhaps both orders compete for entry at the same bar. But I think this would be rare -- it would have to mean that both adviseLong and adviseShort return True in Test mode at a certain bar.

But, anyway, that's not what happening. I am getting different generated code in Data/ . Meaning Zorro's Train output is depending on the order of adviseLong/adviseShort. That shouldn't happen.

Posted By: jcl

Re: Different Result depending on Order of adviseLong / adviceShort - 04/27/14 17:02

Guessing and assuming is no good when you develop a system - you need to definitely _know_ what your script is doing. For this you have the log. Just compare the logs of different script versions and you can see immediateley how and why your script changes affect the single trades.

As you see, the order of long/short can heavily affect performance unless your trades are really independent. But this is not the case here due to your hedge setting. Orders can not "compete", but spread losses by netting always depend on which order comes first.

Posted By: GPEngine

Re: Different Result depending on Order of adviseLong / adviceShort - 04/27/14 19:15

That is good advice.
In my own testing I have concluded that I have once again been burned by the fact that set(NFA) is not compatible with Train and causes training targets to be corrupt due to interference with each other that would not happen during Test/Trade mode.

Do you agree that this is possible?

Posted By: jcl

Re: Different Result depending on Order of adviseLong / adviceShort - 04/28/14 14:39

NFA and Virtual Hedging are trade execution modes, so they have no effect on training. They can not cause training targets to be corrupt.