0 registered members (),
1,397
guests, and 7
spiders. |
Key:
Admin,
Global Mod,
Mod
|
|
|
Re: DT design process: step 1: identify the edge
[Re: Sundance]
#432220
11/02/13 16:04
11/02/13 16:04
|
Joined: Jun 2013
Posts: 1,609
DdlV
Serious User
|
Serious User
Joined: Jun 2013
Posts: 1,609
|
Wow! Thanks dusktrader for taking the time to share in such detail what's obviously the result of much time and trial & error. And here I am still stuck "on the edge"... Like others, I will absorb as time permits; and hopefully the time will come when I can return you something equally useful... Thanks again.
|
|
|
Re: DT design process: step 1: identify the edge
[Re: Sundance]
#432222
11/02/13 16:09
11/02/13 16:09
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
Thanks very much for your feedback guys! I've been working on polishing the logic a bit further and I've been able to extract a few more routines into helper functions (that keeps the run() function cleaner and easier to follow). I'll post the code below. Next I'm going to work on dynamic switching of parameter ranges depending on the asset. My thinking is that if I incorporate a reasonable parameter range that is also personalized for each asset... that it could have a major impact.
function fridayClose(int fridayclose)
{
//allows Friday trading up until NYSE 3pm; close trades and don't allow after this
if(fridayclose && dow() == FRIDAY && lhour(ET) >= 15)
{
exitLong("*");
exitShort("*");
return 1; //condition met; indicate no further trades
}
return 0; //condition not met; safe to take new trades
}
function hourOpen(int hourblockstart, int hourblockend)
{
//blocks new open trades between selected hours
//uses NYSE time, including DST
if ( (lhour(ET) >= hourblockstart) && (lhour(ET) < hourblockend) )
return 0; //between blocked hours, do not allow trade opens
else
return 1; //no conditions met, allow trades by default
}
function todayOpenCombo(var dayopencombo)
{
//allows optimizer to specify the best combo of days for opens
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will return whether
//current dow() is in the combination
int dayopencombobits = dayopencombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if (dayopencombobits & (1 << today)) return 1; //current dow() is in the combo
else return 0; //current dow() not in combo, do not allow trade opens
}
function todayCloseCombo(var dayclosecombo)
{
//allows optimizer to specify the best combo of days to close by NYSE 4pm
//bit position 0 = Monday
//bit position 1 = Tuesday
//bit position 2 = Wednesday
//bit position 3 = Thursday
//bit position 4 = Friday
//bit position 5 = Sunday
//given a combination #, the function will determine if we are beyond
//a combo close time, close all trades if necessary, and return 1
//if no further trades allowed today
int dayclosecombobits = dayclosecombo+.5; //truncate to rounded int
int today = dow() - 1; //Mon is 0
if (today == 6) today = 5; //bump Sun to 5 (no Sat, keep binary range 0-63)
if ((dayclosecombobits & (1 << today)) && lhour(ET) >= 16)
{
exitLong("*");
exitShort("*");
return 1; //current dow() is in the combo; indicate no further trades
}
else return 0; //current dow() not in combo, safe to take new trades
}
function marketOpenCombo(var marketopencombo)
{
//allows optimizer to specify best markets to initiate trades
//bit position 0 = New York 8am-5pm Eastern
//bit position 1 = Sydney 5pm-2am Eastern
//bit position 2 = Tokyo 7pm-4am Eastern
//bit position 3 = London 3am-12pm Eastern
//given a combination #, the function will determine if current time is within
//a market part of the combination (returns 1 to allow trading if true)
int marketcombobits = marketopencombo+.5; //truncate to rounded int
if ( (lhour(ET) >=8) && (lhour(ET) <17) && (marketcombobits & (1 << 0)) ) return 1; //inside New York
if ( (lhour(ET) >=17) || (lhour(ET) <2) && (marketcombobits & (1 << 1)) ) return 1; //inside Sydney
if ( (lhour(ET) >=19) || (lhour(ET) <4) && (marketcombobits & (1 << 2)) ) return 1; //inside Tokyo
if ( (lhour(ET) >=3) && (lhour(ET) <12) && (marketcombobits & (1 << 3)) ) return 1; //inside London
return 0; //default - current market not in combination, don't allow trade opens
}
function checkEquity(var emode)
{
//emode 1 = standard: sets phantom/normal mode only (via Lots)
//emode 2 = switch hitter: always in market (Lots=1), fades direction (via dir)
//emode 3 = reward success with weighting: increase trades based on degree of improvement
//emode 4 = mean reversion: trade when equity curve falls (Lots=1), sit out when it rises (Lots=-1)
vars EquityCurve = series(EquityLong+EquityShort); //includes all phantom equity
var dir; //indicates normal trade direction (dir=1) or reverse (dir=-1)
//narrower curves
//var slow = 50;
//var fast = 10;
//wider curves
//var slow = 100;
//var fast = 10;
//mega-wide curves
var slow = 200;
var fast = 10;
//uber-wide curves
//var slow = 300;
//var fast = 10;
//optimized curves
//var slow = optimize(50,50,300,12);
//var fast = 10;
vars EquityLP = series(LowPass(EquityCurve,fast));
var EquityLPfalling = LowPass(EquityLP,slow);
var EquityLPrisingBigger = LowPass(EquityLP,slow*3.2);
var EquityLPrisingBig = LowPass(EquityLP,slow*1.5);
//plot("EquityLPslow",LowPass(EquityLP,slow),1,BLUE);
//plot("EquityLPfast",LowPass(EquityLP,fast),0,GREEN);
if(EquityLP[0] < EquityLPfalling && falling(EquityLP)) { //drawdown
if (emode==1) Lots = -1; //set phantom trade mode
if (emode==2) return 1; //fade: take signals when losing
if (emode==3) { //reward success with weighting
Lots = -1; //set phantom trade mode
return 1; //allow max 1 phantom trade in drawdown
}
if (emode==4) Lots = 1; //mean-reversion: start trading when equity curve falls
}
else { //positive equity curve
if (emode==1) Lots = 1; //set normal trade mode
if (emode==2) return -1; //fade: take reverse signals when winning
if (emode==3) { //reward success with weighting
Lots = 1; //set normal trade mode
if (EquityLP[0] > EquityLPrisingBigger && rising(EquityLP)) return 3; //very big rising
else if (EquityLP[0] > EquityLPrisingBig && rising(EquityLP)) return 2; //big rising
else return 1; //rising but not yet significantly
}
if (emode==4) Lots = -1; //mean-reversion: stop trading when equity curve rises
}
}
function checkTradesPerCycle()
{
//require minimum 30 trades per WFO cycle or stop training
static int LastWFOCycle = 0, LastNumTrades = 0;
if(Train && (WFOCycle != LastWFOCycle) )
{
if(LastNumTrades > 0 and LastNumTrades < 30)
{
char tradecount[100];
sprintf(tradecount,"Not enough trades per cycle: %d",LastNumTrades);
quit(tradecount);
}
LastWFOCycle = WFOCycle;
}
LastNumTrades = NumWinTotal+NumLossTotal;
}
function calculateMargin(int direction)
{
//calculate risk Margin based on OptimalF and trade direction
Capital = 1000; //simulated account balance
var riskCapital = 600; //basis to trade with
if (direction && OptimalF>.001) //long trade, historically profitable
return OptimalFLong * riskCapital;
else if (!direction && OptimalF>.001) //short trade, historically profitable
return OptimalFShort * riskCapital;
else if (is(TRADEMODE)) //non-historically profitable = phantom live trades only
Lots = -1;
return 0; //no Margin allocated for non-historically profitable
}
function checkModifiers()
{
int reversedir = 0; //default normal trade direction (0) unless specified otherwise
int fridayclose = 0; //enforce auto-close and no trades after NYSE 3pm Friday
int hourblockstart = 0; //block trade opens beginning at NY hour
int hourblockend = 0; //block trade opens ending at NY hour
int dayopencombo = 63; //optimize(54,1,63,1); //combo of days to open; 63=every day
int dayclosecombo = 0; //optimize(33,1,63,1); //combo of days to close after NYSE 4pm; 0=none; 63=every day
int marketopencombo = optimize(12,1,15,1); //combo of markets to allow trade opens; 15=every market
if ( (!fridayClose(fridayclose) //close NYSE 3pm on Friday
|| !todayCloseCombo(dayclosecombo) ) //close NYSE 4pm on selected days
&& todayOpenCombo(dayopencombo) //open on selected days only
&& marketOpenCombo(marketopencombo) //open during selected markets only
&& hourOpen(hourblockstart,hourblockend) ) //open during selected hours only
return 1; //ok to place new trades
else
return 0; //no trade, restricted by a modifier
}
function run()
{
set(PARAMETERS+FACTORS);
StartDate = 20080101;
EndDate = 20130531;
BarPeriod = 15;
LookBack = 600;
DataSplit = 70; //70% training, 30% OOS test
NumWFOCycles = 5;
if(is(TESTMODE)) NumSampleCycles = 15; //oversampling on Test only, not Train
if (Train) { RollLong = 0; RollShort = 0; } //help prevent asymmetry in parameters & profit factors
checkTradesPerCycle(); //stop Train early if not enough trades
int maxtrades = 1;
int reinvestprofits = 1; //invoke margin setting during trade logic
//equity-curve trading
checkEquity(1); //emode 1: normal/phantom trading
//reversedir = checkEquity(2); //emode 2: switch hitter
//maxtrades = checkEquity(3); //emode 3: reward success
//checkEquity(4); //emode 4: mean-reversion mode
while(asset(loop("NZDJPY","EURUSD","AUDUSD","AUDCHF","AUDJPY","USDCHF","USDJPY")))
{
//edge trading logic
var TimeCycle = optimize(64,55,75,1,0);
var TimeFactor = optimize(2.6,0.2,5,0.2,0);
//Stop = BarPeriod*PIP; //simple stop level
Stop = ATR(200) * optimize(1.99,1,15,0.5,-3); // allow 3% tolerance for preferring low stop distances
Trail = ATR(200) * optimize(8.5,3,13,0.5);
vars Price = series(price(0));
vars MA1 = series(SMA(Price,TimeCycle));
vars MA2 = series(SMA(Price,TimeCycle*TimeFactor));
if (checkModifiers())
{
//OK to trade, let's evaluate signals then
if (crossOver(MA1,MA2) && rising(MA1))
{
if (reinvestprofits) Margin = calculateMargin(1); //long
//enterLong(); //standard entry
reverseLong(maxtrades);
}
else if(crossUnder(MA1,MA2) && falling(MA2))
{
if (reinvestprofits) Margin = calculateMargin(0); //short
//enterShort(); //standard entry
reverseShort(maxtrades);
}
}
}
PlotWidth = 1100;
PlotHeight1 = 800;
}
|
|
|
Re: DT design process: step 1: identify the edge
[Re: Geek]
#434609
12/22/13 12:21
12/22/13 12:21
|
liftoff
Unregistered
|
liftoff
Unregistered
|
Totally blown away by your work dusktrader ... a real contribution to this community. I just spent the last 3 hours digesting your post. I have to admit I have learnt so much and my whole conception of the design process has been greatly improved. This will definitely be the template on which I build my own design process. Thank you again.
|
|
|
Re: DT design process: step 1: identify the edge
[Re: ibra]
#438498
03/16/14 00:00
03/16/14 00:00
|
Joined: Jul 2013
Posts: 522
dusktrader
OP
User
|
OP
User
Joined: Jul 2013
Posts: 522
|
Well I suppose its all a matter of perspective. There are some combinations that barely produce the minimum required trades during WFO cycles. But that's not the reason I use oversampling. My feeling is that oversampling in general is a way to improve test results by reducing the effect of random entries with your logic (ie, success due to luck only).
My understanding is that oversampling will never harm the accuracy of your simulation result. On the contrary, it adds more realism to the simulation, in my opinion.
In another thread somewhere on this forum, jcl confirmed exactly how oversampling works, but I will try to explain it again here from memory:
As an example, if your strategy looks for entries on BarPeriod 15, that means during simulation, it would only look at one price sample every 15 minutes. That should theoretically be fine, as long as you have "enough" trades overall to be statistically significant when looking at the Performance Report analysis.
However, by using the NumSampleCycles oversampling feature, you have the opportunity to introduce more simulated trades "while maintaining the trend, periodic behavior, and other characteristics", per the manual.
In laymens terms, the simulation looks at the "what if" scenario of where exactly the 15 minute bar begins. If the strategy logic is truly robust, then it shouldn't really matter too much if the 15-minute bar begins at 9:15, 9:16 or 9:17. That is what oversampling does -- it shifts where that demarcation point is, at least for simulation purposes.
So my understanding is that it runs and re-runs the strategy assuming the demarcation point of the bar is at 9:15, 9:16, 9:17, etc... and then takes the average result of those tests.
Technically speaking, I believe it works like this: NumSampleCycles=5
cycle 5 --> 9:19 bar boundary cycle 4 --> 9:18 bar boundary cycle 3 --> 9:17 bar boundary cycle 2 --> 9:16 bar boundary cycle 1 --> 9:15 bar boundary
If the strategy logic is robust enough, it shouldn't matter too much where the bar boundary is. So oversampling helps validate robust logic.
|
|
|
Re: DT design process: step 1: identify the edge
[Re: dusktrader]
#438522
03/16/14 13:47
03/16/14 13:47
|
Joined: Jan 2013
Posts: 68
ibra
Junior Member
|
Junior Member
Joined: Jan 2013
Posts: 68
|
|
|
|
|