Major future features

Zorro's basic functionality is now finished, aside from some small issues such as Win95/XP compatibility of the image viewer, or support for US accounts on the Z1/Z2 systems. We will not do much with Zorro but work on Gamestudio in the next time, but we're already thinking about what major Zorro features could be implemented next year. Here are some possibilities. We can not implement all at once, but one after the other, so please let us know what you would prefer to come first.

- Debugger: Single step through scripts line by line while observing the variables.

- HTTP module: Download life data from commercial websites, f.i. the VIX or other indices, and use it for trade signals.

- Monte Carlo simulation: Gives a more realistic, less random value for the drawdown and its derived parameters such as annual return.

- Genetic optimizer: Finds profit peaks in the parameter space.

If you have other ideas for major features, or have a preference within of the listed features above, please let us know.

There are 2 things that I think would be interesting:
1. that other data sources could be used, either by using a script to import from .csv files or by connecting directly to another broker

2. when optimizing, it would be interesting to have a range of profitable parameters returned rather than only one. This would help avoid getting an optimized result that is the only profitable set of parameters for that strategy. For example, imagine a moving averages cross system which has a very profitable result when slow ma period is 22 and fast ma period is 13 and any other set of parameters is either producing a loss or a very small profit. Then that system wouldn't be robust compared to a system that would show to be profitable for say slow ma period between 12 and 20 and fast ma period between 25 and 36.

I hope point 2. makes some sense...

Thanks
Guiom

Being able to select a range of dates for a test period would be useful.

I second both of Guiom's points as what I would consider the most useful and a monte carlo simulator would be excellent. Thanks for your continuous hard work JCL it's appreciated.

Thanks for the suggestions. CSV data can already be imported, using the file and string functions. How the code would look in detail of course depends on the imported data.

From a Programmers Point of View: Trading is extremely risky, therefore bugs in software like zorro can have dramatic impact (probably you do not take any responsibly for tradining losses due to software bugs - so probably no impact for you, but at least for you customers)-

Due to this reason it would make sense to let Zorro connect to an internal conitec Server, which checks if there are currently any high risk bugs in the running version. If there are any, Zorro permits the user to start trading or informs him (if he's currently trading).

Of course it would be better to avoid such errors, but as you've already proven you are not able to prevent them always.. Such a functionality could at least decrease the impact of trading bugs slightly.

The risk usually comes not so much from bugs but more from bad strategies. Anyway I imagine that traders don't want their trading controlled by our server. Having it controlled by the broker server is bad enough .

I would like to see a "visual mode" similar to the one on the mt4 strategy tester where you could watch the trades being taken one by one to check that the strategy you wrote was being implemented properly. This can also give you ideas as you watch the charts unfold.

Also I would still like the option of starting with a notional balance of a certain amount and derive a % gain or loss from that level by taking trades of a set percentage of your balance. Maybe this could be an option? Please?

I think both is already possible. For watching trades one by one, just let the script print a message window with all trade parameters that you want when a trade is opened or closed. You can use the "broker" function for that. For adding a balance at start, just do

if(is(INITRUN)) Balance = 10000;

I love it when you say "just" do this or that :-)

Ok, I'll have a try.

Would it be possible to import MT4 strategies and use them with Zorro?

Theoretically yes. The language is the same, you just have to write a C header that translates the variables and functions. This is however a diligence challenge and no real fun.

Hi


I have a request about something that i'm sure many MT4/5 coders are probably struggling with (at least I am...) :

"TA" and "price" functions are not multi-timeframe.

I explain:

If i test/run my script in Daily charts, I set BarPeriod=1440.

However, all TA functions such as ATR(), EMA(), LowPass(), etc only have a "int TimePeriod" argument, but no equivalent of the MT4 "timeframe" argument.

Therefore, when i call



this will return the ATR(Daily,60) value if i'm running my script with BarPeriod=1440 and TimeFrame=1.

If i switch to another BarPeriod, say 5,
then ATR(60) will return the equivalent of ATR(M5,60).

There is no way to get the ATR(Daily, 60) whatever BarPeriod/TimeFrame i would be using for my test.

What makes things even more confusing is that the "TimePeriod" argument of TA functions not only depend on BarPeriod, but ALSO on TimeFrame...

How do we (simply!) solve this ?

I would be really happy to have a separate "int timeframe" argument that clearly specifies the timeframe on which the "timeperiod" argument applies, just like MT4 has.

These 2 arguments together would then make the TA function calls completely independent on the BarPeriod and TimeFrame global variables that otherwise affect the trade functions

My "daily ATR" request woudl then always look identical like this (for instance):



regardless of the BarPeriod and TimeFrame arguments that are set to run the script.

Note that this should also likely apply to the price functions (priceOpen(), etc).

The current way it is implemented is very confusing, and makes it very difficult to code scripts that can be run on any BarPeriod values.

Thanks.

Sq

In lite-C, TimeFrame is a simple variable, not an extra argument to all functions. For a daily ATR(60) just write:

ATR(60 * 1440/Barperiod)

or

TimeFrame = 1440/Barperiod;
ATR(60);

No extra functions required.

I would like to see this too.
this is surely why my backtest in Zorro is different. And i know now why, after reading the TimeFrame-function:



i think it would be better, if the run-function will be called every Tick.


i think what helps here is always test in M1 and if we like to have values from other TF's we have to use the TimeFrame-function like:



but i don't know if this works and if it works tick-wise instead Open-Bar-wise

I remember that an every-tick run function was implemented in an early Zorro version - but I think this feature is no longer supported, as apparently no one found any use for it. But if you can give me an example for what purpose an every-tick run function could be used in strategy testing, we can certainly re-implement it.

The shortest bar periods I've seen in serious strategies were M5, but on those short time frames the prices are mostly noise and systematic trading is difficult. Most profitable strategies use H1, H4, or D1.

If I understand everything correctly, multitimeframe-trading forward or backward should be made from bottom-up.
How will zorro handle this:
I run a backtest in the 1 minute timeframe and fetch the ATR-value of the 60 minute timeframe, will I get the value from the actual 1 hour candle or from the past 1 hour candle?

If yes, then function run in tick makes really no sense

You can get both:

ATR(n * 60) // ATR from the last n*60 minutes up to the current minute

or

TimeFrame = 60;
ATR(n); // ATR from the previous n 1-hour bars

ok, when playing with BarPeriod and TimeFrame yesterday, i came up with something similar to your second solution.
Although it is not ideal as you expose it, because it means you are messing up with TimeFrame, and if you want to use other TimeFrame-sensitive functions (TA or price functions), then you have to make sure you set TimeFrame every time.
Or restore it to its previous value once you've used it...
Which is the solution i came up with yesterday.

Here is actually how i "solved" this :
- i created 2 "general-purpose" functions to set and restore TimeFrame;
- for EACH TA function( ATR(),EMA(), etc), i created a function made of an extra "layer" on top of the Zorro function, which saves TimeFrame, calls the Zorro TA function, restores TimeFrame, and returns the result...

Here is an example with ATR():




It is heavy, but it works fine, and avoids mistakes with TimeFrame.
And it also gives a clear "atr(timeframe,length)" API.

No messing around with TimeFrame in the "active code", therefore no risk of miscoding it.

Just for THAT single reason : NO risk of hard-to-find errors in the code, i think this approach is much more robust.
You should really consider providing this kind of improvement in the APIs, this will make coders life easier, and less questions/answers in the forum in the future as well;
And believe me, i know exactly how it feels to be bombarded all day long with the same questions again and again because people don't know how to (correctly) use your APIs...


Back to my script example with ATR above :
i coded a simple run() function to compare the 2 methods :
- the first one you provided : ATR(60 * 1440/Barperiod)
- the second one, coded with my "enhanced" atr() function, which is equivalent to your second suggestion;

The script is very simple, it will plot both results on the same sub-window, just to make sure they return the same value;
Which it does!

It also plots a second subwindow showing the values of BarPeriod and TimeFrame values as they were set at the beginning of the script;

Now the catch is this :
although both atr methods give the same result (that was expected), if you change the BarPeriod value in the script, then the ATR value will give you a totally different range of values, although the curves have the same shape...

Try with BarPeriod=60, then BarPeriod=5, then 1440, you'll see what i mean.
They should all give the same value ranges and the same curve shapes.
The only one that is correct is when BarPeriod=1440 (120-160p)
In M1/EURUSD, it gives you a DAILY range of ... 2.5-4pips


There's a bug somewhere.
Either in the way i am using ATR(), although both suggested solutions give the exact same (wrong) results,
or in the ATR() function itself!


Another somewhat confusing element is LookBack.
It must be set to a large enough value to accomodate the "longest" "timeperiod" parameter used in any of the TA or price functions.
When playing with daily timeframes, for instance ATR(60 days), if you want to run the script on M1 candles, you will need LookBack > 100*1440 (apparently, any lower value gives an error).
If we set BarPeriod to 1440, then only LookBack can go down to 100 only.

The minimum value also seems to be affected by the value of TimeFrame.

So i came up with this work-around :

LookBack = 100*1440/BarPeriod/TimeFrame+1;

which will accomodate up until ATR(60 days) at least.

But what if i want to calculate ATR(150 days) ?
Then i have to go back and change LookBack again.

This is quite important for backtesting because changing LookBack will greatly affect how far in the past we can backtest.
No way I would set it once for all to 100*1440;
We couldn't run any test with BarPeriod=1440...



Sq

Thanks for the bug report! I can confirm the problem - ATR indeed returns a wrong value when TimeFrame is > 1. It's a bug. I'll forward this to the developers. This will be fixed.

- Your method to emulate the MT4 atr is correct, but a little heavy. You need only 2 lines for replacing all time frames with function arguments:



- Your method to set the Lookback period does not look right. Maybe there's some misconception. LookBack is the number of bars before the test period. For 150 days, it's 150*1440/BarPeriod.

FYI, the recently uploaded version 1.03 beta contains a new ATR function for different timeframes and different price series.

Great, jcl!
I'll check this out as soon as i download the 1.03.

Thanks.
Sq

Will you add a function like MT4 iCustom()?

And I have been looking for a robust testing software for Binary Option.
Are you involved in?

iCustom is a specific function for MT4; what do you want to do with it in Zorro?

If the binary option is based upon an existing asset, you can write and test a trade strategy just as for normal assets. You can theoretically even trade it automatically when you control your bookie's software or website with Zorro's "keys" function. We do not recommend trading binary options, though.

I just want to convert some custom indicators in MT4 to Zorro code and use it by iCustom.
When I make a hybrid robot which contains many custom indicators, Is it useful if there is that function?
Or no need to do it, Just copy and paste Zorro's version of the code into the zorro?

Just write the indicator and either copy it directly into your script, or include it with the #include statement.

I just did test for weekly(43200) chart.
but it did not work and even there are only bars until 1440 in the period slider.

Could I test for more than 1440 bars?

Sure, AFAIK BarPeriod has no limit. If a bar period of one week makes any sense is another question.

Please, can you ask such questions in the Starting with Zorro forum? This thread is for a) future and b) major features only.

Im wondering whether it might be possible to add some more JPY pairs like the following:



These pairs are pretty popular and usually get decent spreads that make strategies still viable.

Could you please build in Monte Carlo Simulation?
I think it is one of the important processes for trading system design.

You're right. Implementing a Monte Carlo simulation for the performance report is on our schedule with rather high priority.

I saw that you are planning a genetic optimizer. Could it be possible to make this accessible for the user from the API? A simple genetic algorithm is not suitable for financial time series as they are noisy and strategies usually exhibit a multimodal solution space. CMAES for example works well for optimizing strategies although there are others that are simpler. CMAES is open source and could be implemented as an example and then the user could write a DLL for the optimizer API and implement their own optimizer. You probably have to add some new functions to the Lite-C language though, for example Optimizer(CMAES). Could this be implemented?

Yes. It makes sense to use a DLL plugin for the optimizer. Users can then modify the optimizer or add their own optimizing algorithms.

Great! I haven't tested all the features yet, but I have been considering writing my own trading app but Zorro is very similar to what I would have done myself.

I think http://www.myfxbook.com/ can be used by using API.
If I can use it in Zorro, It is very useful.

Hi jcl,

i'am new to Zorro and just got Z1 running and now trying to create/convert my EA to Zorro. What i really miss is the possibility to draw object within a chart. For example when i use a function to identify a 'cup' formation i would like to draw some lines and after testing to confirm that the algorithm has detected the 'cup' formation the right way.

This would be a great help also in debugging scripts. What do you think?

Can you tell me more about how drawing lines in a chart can help developing strategies? Do you mean just horizontal lines? Or do you want to draw an artificial price curve for testing the cup detector?

Drawing lines on a chart won't help in developing a strategy but it would help in confirming that it works as expected. As for the example of the 'cup' i would draw lines that will show me the cup. so i can see if i also would identifiy this as a valid cup or not. So i can adjust the given paramter to the advise function.

Or more simpler: draw some lines above and below a consolidation zone

Ok. So you'd need a function like "line(bar1,price1,bar2,price2,color)" - would that do?

Yepp. That would do the job. Line, Symbol, Rectangle something like this. Just as you would do it in MQL code.

Ok, that goes on our to do list.

Count me interested for the functionality, too. It's not so useful when trading, but having basic graphics primitives available while developing new strategies is very useful.

A question.
When i look at the manual 'Workshop 7: Machine Learning. Price Action Trading' the first example. Before the call of the enterLong() function i would draw a line like line(2, priceLow(2), 0, priceLow(0),white) to mark the found price pattern. So when testing i can look at the chart an directly see the patterns.

This is what i think of. Is it possible :-)


UPS i didn't saw the last two posts. Fine i'am happy to see that functionality coming

Marking a found pattern is already possible. You can draw a line with the plot function, like this:

static bool DoLine = false;
if(patternstart) DoLine = true;
if(patternend) DoLine = false;
if(DoLine) plot("Pattern",priceLow()-30*PIP,0,RED);

This draws a red line under all found patterns at 30 pips distance from the price.

Ah. I read the manual about the plot function but had no clue how to draw a line like i wished. This is a workaround that at first glance looks okay. So the line consists of many dots. Do you still pin the line function on your to do list?


PS: ...and funny again you used the 'its already possible' sentence

Yes, the line function will be implemented in a future version, as my suggestion above is just a workaround for the moment.

A minor feature but one that I would find very handy is an "always on top" for the zorro window makes it easier to find the window when working on scripts. Or a dock to the edge of the script window.

Hi jcl,

i have one more question reagarding the plot function. I would like to draw/mark some bars in the past. When i'am at bar 0 i know now that the last x bars form a special formation so i would like to mark them. How can i accomplish this?

thx in advance

At the moment only the current bar can be marked, not past bars. So you'd have to set the PEEK flag and check the bar formation in the future for marking the current bar. The upcoming line function will also allow to mark past bars.

Okay. then i will wait for the line function. Possible time frame? :-)

Will you add a harmonic trading function?

Can you show me an example of a harmonic trading indicator?

You can download here.
http://www.fx-mt4.info/view-review/zup-v93-indicator-harmonic-price-pattern-recognition

AFAIS, this indicator just paints a quad or butterfly in a chart. What do you want to do with it in Zorro?

An automated harmonic trading.
You can find a more info about it.
http://www.fxgroundworks.com/

That looks like a website that sells scam products. I find also no info on that site how their filters are supposed to work - for implementing an indicator, I need to know what output signal this indicator should generate.

Do you think a harmonic works?
You said in the manual that fibonacci numbers don't work as it is linked to them.

I don't know if it works. But a price curve has normally an autocorrelation range of less than 4 or 5 bars. So there is no reason to assume that a butterfly or some other figure can give insight in the further price movement. And I have not heard that anyone has found so far any evidence that such methods work.

It will be useful if the Zorro chart shows times of the each bars.

Why Zorro says errors when I try to save a chart image?

For saving a chart image, just click on Image / Save As Picture.

The chart shows the time, but in a relatively low resolution. I'll check if the chart x axis can get a finer resolution.

Would be great to have a debugger implemented in Zorro in the near future as well as the Monte Carlo draw down algorithm.

Also if the user had a choice to customize the Zorro Color background and text etc as you can in SED.

Is there a lite-c library to send messages over a socket, like for example with websockets/socket.io or similar?

A debugger is planned. Modifying the background and text color would require some structural changes, as it's then not a standard object anymore, so this might get a low priority.

For socket sending, there are no native lite-C functions, but you can use the Windows socket functions.

Zorro has only a few color code. If you can add many colors it would be good when testing colorful indicators.

Other instruments? Such as equities, ETFs, non-CFD indexes, etc. Maybe someday options?

Implies associated data structure changes (i.e., adding Volume).

And appropriate broker(s) so it'd work.

Do you plan to publish a roadmap?

Thanks.

You can use any instrument, as long as a broker supports it and offers an API or MT4 for access. Volume, interest etc. are just a data curve. They can be added the same way as an instrument and accessed with the asset() function. The prodedure is described under "Data Import".

We can not publish roadmaps because most new features are paid by customers. So the release date of a feature does not depend on us. But you can see under "What's new" which features are planned for the future.

Thanks jcl. Yes, I had read that info in the Data Import. But I'm also mindful of the level of capabilities & effort involved. I was coming from the PoV of a better OOBE - immediate ease of use as well as for those whose alternatives are limited by NFA/SEC...

Thanks JCL for the new possibility of drawing lines!!!

Is it possible to add a plot function for viewing the monthly net profit of the strategy over the backtest period and also in the performance report or by a separate log file?

You can do that already. Use the plotBar function - you can find examples of doing those statistics in the profile.c script library.

Thanks! Will check this out.

Could you add "The Kinetics algorithm"?

http://www.forexeasystems.com/forex-expert-advisor-forex-kinetics/user-guide#algorithm

This is a commercial algorithm. It is not published, so I can't add it. Although the algorithm idea might make some sense, it's unknown if it's profitable at all. The equity curves on that website are fake - you can see from the MT4 menu that they are obviously taken in-sample.

Any progress on the Monte Carlo analysis algorithm?

Not really, as it was not begun yet. But you can vote here for it if you want it more urgently than the other listed features.

One vote for Monte Carlo .

I don't think I officially voted for Monte Carlo feature either. I did learn a bit about it though since reading on this forum. Please add my vote too. I'd rather have this feature more than some other things I may have asked for.

Is there a way for the forum to hold a voting poll? You could put a bullet list of requested features there and sometimes can allow multi-votes too.

yap, a simple poll with each user given 10 vote points and a max of 3 pts for each feature would be great.

one vote for IB API connection in the next release ... much obliged

+1 for Even faster test/training by using multiple CPU cores (for Zorro S)

Feature Requests:
1: Dukascopy JForex API Plugin
http://www.dukascopy.com/swiss/english/forex/api/jforex_api/
or Dukascopy FIX API Plugin
http://www.dukascopy.com/swiss/english/forex/api/fix_api/
2: test/training by using CUDA or OpenCL (GPU) cores.

thanks,

I vote for Dukascopy JForex API Plugin as well.

Thank you.

Hi jcl. There have been requests for API implementations for Dukascopy, IB, and Oanda (and I see that IB and Oanda are on the What's New roadmap). I'm not qualified at this point to vote for one over the other especially as much of the reasons I'd use to vote are not directly observable (hence above they're listed alphabetically ). But for planning purposes I would definitely like to know your decision as to which and when.

Thanks.

Please keep in mind that some brokers have minimum account sizes for using API on their accounts. As far as I know they are:
Dukascopy: $100k
IB: $10k
Oanda: none

Hi guys, the thing that would help me the most is to see the manual updated and expanded.

It would be nice to have some specifics in there around how the multisession plugin works with MT4. I.E. do you drop a Zorro expert adviser on a different chart for each strategy, or just one chart?

Also just a bit more depth on some topics...I feel like sometimes I have to read multiple sections to understand one thing fully.

Last but not least, it can be hard to find things in the manual, as there is no way to search it(correct me if I'm wrong), as you drill down on each subject. It would make a huge difference for me if there was a way to ctrl-f for all the text in the manual, and quickly find what I am looking for.

I believe these couple of things would greatly enhance the zorro experience for new users.

Thanks.

Thirstywolf

You can use google to limit a search to a website. For example..,

site:http://www.zorro-trader.com/manual/ "margin"

Click "Help" for opening the manual and click "Search" for searching in it. There is also an alphabetical index. Reading a manual without knowing how to search for things is certainly a cumbersome task.

I see. So if I open the manual directly from the Zorro Control Panel there is search functionality. I have always just opened it through the Zorro website. Thank you very much for this handy information. On the plus side, I know the manual pretty well now.

I see that it has been requested before, but also I would like to have the ability to connect Zorro with Interactive Brokers. However, IB has a requirement that the minimum amount is 10kUSD when opening an account, which is above the 7kUSD limit that Zorro (free edition) is using.

Besides that would I have two more wishes but I guess that these are out of the scope of Zorro. One is to be able to run Zorro on a Unix (Ubuntu) computer. My understanding is that Zorro is developed for Windows machines. I have not yet tried to use Wine on a Ubuntu machine to see whether it is then able to run Zorro.
The other one is to be able to run Java software, developed as API for IB's TWS on Zorro without having to translate it to another programming language (with the risk of making translation errors).

My own report. Zorro on wine works for Train and Test, meaning the behavior is the same. but it cannot connect to FXCM. I think there is a problem with the networking dll (I have no idea). So you need a windows machine to download History or Trade. I have also had trouble with SED.exe in wine. It prints a blank undismissable dialog when it wants to say that an open file has been modified. So, I don't use the supplied text editor. Other than that, we kinda got lucky on this point.

Interactive brokers connection for Zorro S please. could use a real broker, FXCM is meh so so broker. Interactive Brokers support for zorro S please.

KK