PnL calculation

Hello,

i am just trying out Zorro and writing a few simple scripts to get familiar with it. I wrote a script that trades only on a single day, that goes long at 12:00PM and exits the trade at 16:00. I have tested this with EUR/USD data from the Zorro-website.



Now Zorro produces a performance report, a trade log and a CSV-file with the trades. The log says i entered at 1.2978 and exited at 1.2992 for a PnL of $1 (+14 pips).

The CSV says i entered at 1.29781 and exited at 1.29896 for a PnL of $0.84 (+11.5 pips).

The performance report says i made +10.4 pips or $0.84.

Why are there such differences, and how do you calculate the $0.84 ? A gain of 10 pips (the minimum of the three) should be $1 straight.


I also did the same on USD/JPY. Again the exit-prices from the csv and the log are differing by ~3 pips. And the PnL calculation is off: according to the csv-file i lost 3.1 pips or $0.32. This cannot be correct with usd/jpy exchange-rate at 78.xx.


Maybe i have to adjust some flags or something?

One last question: can Zorro correctly calculate the PnL in USD of, for example, a EUR/CHF trade?

Regards,
Purri

The trade entry, exit, and profit values are recorded in the CSV file.

The log does not record trades. It just records the events during the simulation. It lists the asset price at the moment of entry and exit. The price at which you really bought and sold the asset is recorded in the CSV, and it's normally different due to Spread, Slippage, and Rollover. Look here for details: http://manual.zorro-trader.com/spread.htm.

The EUR 0.84 in your above example is the profit of that trade. Where values are rounded, 84 cents are displayed as $1 and 10.4 pips are displayed as 10p.

The "$" sign in the message window has nothing to do with US-Dollars, it just means money. What currency it is depends on the simulated broker account. The default account is a micro lot EUR account.

Thanks, i see. So the $0.84 from the CSV file are in this case actually Euros. But shouldn't it then be 0.88 ? ( $1.15/1.29896 = 0.884 ) The $0.84 would imply an eur/usd exchange-rate of around 1.37.

I understand its very difficult to calculate correct fx pnl during backtests, especially if you trade a portfolio of many different pairs, some of which might not even include the account base currency. Even more so if you want to account for historical roll rates. Im not entirely convinced static exchange and roll rates are 'good enough'. Some pairs are +-50% compared to 10 years ago, or even 5.

No, the $0.84 are correct, but your objection is justified. PnL calculation is not really difficult, but there are many factors involved.

For spread, rollover, and conversion from price differences to profit, the current broker's parameters are used, not the historic rates. The reason behind that is that you want to use a developed system today, at current rates, and not in the past. So the simulation does not really simulate history, it simulates the present, but with a history price curve. For the conversion from asset price differences to account money, the broker publishes a value named "PipCost" or similar on his website. This value is used for the conversion. It changes from time to time when the counter currency and the account currency are different.

You can find details about the account simulation in the manual under "Data import/export". There are about 10 parameters per asset that determine how PnL is calculated. You can set up different parameters for emulating different accounts, also in US-$, so you then won't have the conversion issue.

jcl: how are the 0.84 correct? +11.5 pips on a microlot would be $1.15 in profit which converts to €0.88 with the then prevailing exchange-rate.

A trade profit has not much to do with the prevailing exchange rate. Brokers use their published pip costs for converting price differences to profits. They do not use the dollar exchange rate for obvious reasons. The exchange rate is different any day and traders would not appreciate when it affected all their trade results.

The profit of your trade is about 10.44 pips. The difference to 11.5 is the slippage, which is not rounded because it's a statistical value. When you multiply 10.44 pips with 0.0800 EUR pip cost, you'll get 0.8352 EUR. That's your 84 cents - the exact profit of your trade when you had entered it today with a real FXCM EUR account. I think that's the result you're normally interested in when testing a system.

Entry and exit prices in the CSV contain the spread, but they do not contain the other trade costs - rollover, commission, and simulated slippage. The costs affect the profit, but not the entry and exit price, although for slippage this could maybe make sense. But slippage is simulated separately at the moment.

I hope this helps understanding trade profit. For your experiments you can set Slippage to 0 - then you should really get 11.5 pips profit.

jcl: actually they do use the exchange rates for pnl - im sure traders would appreciate it even less if FXCM came up 4% short in their account like in this example. I certainly wouldnt.

I did never trade with fxcm, but Oanda, IB and lmax do it like that (and everyone else i suppose). What happens behind the scenes is that you pnl will stay in USD for 2 days and then settles in your account currency. You can google "tomorrow-next" for settlement conventions in fx.

If you do a 10 year backtest, the exchange-rate in eur/usd varies by 60%, its anywhere between 1 and 1.60. You cannot just assume a static eur to usd conversion factor, the pnl of the simulation will invariably be off significantly.

I have tried quite a few of the more well knowen affordable backtesting platforms (ninja, neoticker, multicharts and so on) and none can really do multi-currency backtests. Let alone account for rolls, so one could simulate carry-trade strategies. It would be nice if Zorro could somehow come up with a wholistic solution to this.

Hmm, the Fx brokers that I know all calculate profits in the same way as FXCM.

For EUR/USD, the current Pip-EUR conversion rate is 0.0800. You can normally see the conversion rate when you start your broker's platform, f.i. the FXCM trading station, and look into the currency detail view. The rate is named like "PipCost" or "PipValue" dependent on the broker.

The rate does indeed change, but infrequently. It does not follow the daily dollar price. The current value, 0.0800, was the same in 2012. So your trade had earned exactly the same 84 cents profit when you had executed it back then on a real EUR account. In 2011 the rate was lower, like 0.078.

A change of the rate does however not affect the PnL in the simulation. The simulation only uses the current pip cost, not the USD exchange rate and also not the broker's historic pip costs. Otherwise the equity curve would not reflect the performance of your strategy, but also the USD price or the broker's conversion rates.

Anyway: if you want to convert the profits of your trade to another currency at the real exchange rate, you can do that in the script. You can set PipCost to a different value, so you can use the price of an asset, f.i. EUR/USD, for calculating an artificial PipCost that changes daily.

jcl: i can only recommend to think this through one more time.. its just not how it works.

And obviously PnL of a simulation should reflect what the PnL would have been at that time - not what it would be with a future exchange-rate..

Ok. At least I can say that I did my best in this thread, although with no final success .

Maybe we can still archive success. Here is an experiment:

Make a trade with 100K units of EUR/USD on a fxcm euro demo account. Lets suppose you make +15 pips on this. Your PnL will be $150. These $150 will then be automatically converted to Euro with the actual excahnge rate at 1.2960 (current rate) - so PnL will show up in your account report with $150 / 1.2960 = €115.7

This isnt rocket science and can easily be verified with a few clicks.


Here is another way to put it: Say you buy 100K units of eur/usd at 1.2500. That means you have now a credit of 100K EUR and a debit of 125K USD. Now you are in luck and the eur/usd exchange-rate rises to 1.2550 and then you sell 100K units of eur/usd to go flat which essentially means you exchange your 100K Euro credit into USD with a rate of 1.2550 to repay your 125k USD debit. For your 100K Euro you now get 125500 USD - so $500 are left over, which is your proft. Now these $500 will be converted to Euro at a rate of 1.2550 (~€398).

You could ovserve the actual currency balances with (for example) an oanda demo account, they do list the exposures in their platform. But no matter if you express this in floating pip values or any other representation, this is how it works, and has always worked in fx.

I'm a little pessimistic about the success, as my lengthy explanations so far seem not to have been arrived. So, my final attempt with a small calculation and no much words:

Your profit = 15 pips
The pipcost = 0.0800 EUR
What ends up in your account = 15*0.08*100 micro lots = 120.00 EUR.

You can now complain to FXCM that your profit ought to be EUR 115.7 due to the exchange rate, but they will insist to pay you EUR 120. I can really not help it . But that's what you get and what the program simulates.

No they wont. I can post a screenshot with an actual trade on fxcm demo and its pnl on monday. Would that help?

http://i.imgur.com/lFWRlGg.jpg

As you can see, there is a trade with 1.7 pips loss on a minilot for a pnl of €-1.31

$1.7 / 1.29966 = €1.31

According to you, it should be €1.36

No, it's 1.309. You can follow my example above for the calculation, only pipcost is 0.77 EUR for the account in your screenshot because it's a minilot account. Microlot pipcost are the mentioned 0.08 EUR. You can see the pipcost in your asset detail view.

1.7 * 0.77 = 1.309 EUR. Rounded up it's the 1.31 that you can see in your screenshot.

On the broker website you'll usually find all parameters explained for calculating the PnL of your trade. For FXCM, it's here: http://support.fxcm.com/fxts/user-guide/rates/

- You might wonder why the pipcost is 0.77 on a minilot account and 0.08 on a microlot account. These values are also really used in their trading API. FXCM gave us the explanation that this has technical reasons due to the internal parameter representation by their server software. But this is a detail specific to FXCM and not relevant for general trading.

In their trading API, the broker specifies how the PnL is calculated. We have no choice here and can not invent our own PnL calculation method. Otherwise Zorro would record different trade profits than FXCM in real trading. The platform receives the current pipcost value through the API, and that value is then stored in the account parameters and also used for the simulation.

But your 0.77 is the same thing: 1 / 1.29966 = 0,7694

The pip-cost will fluctuate according to the exchange-rate, internally probably with 5 digit precision, rounded off to 2 digits is what fxcm displays in the platform as pip-cost. Your 0.08 is probably 0.07694 rounded to 2 digits. Its not a value that is arbitrarily set by the broker.

Your link(that describes the behavior of a USD account) actually says that it is fluctuating too. In an USD account xxx/USD pairs have a fixed pip-cost of $1 per pip and minilot. But for example xxx/JPY pairs will hae a fluctuating pip-cost. Consequently in an EUR account, xxx/EUR pairs (that do not exist) would have a fixes pip cost, other paris don't.

No, they have no internal 5 digit precision - or in fact they have, but the remaining digits are normally zero. The values internally used by the broker API are really the same as displayed in the FXCM platform, with no hidden digits.

The values do indeed fluctuate. FXCM changes them from time to time and adapts them to the exchange rate. When you look into the file "AssetsMini.dat" in your History folder, which is for simulating a mini lot account, you can see that the value was 0.78 in 2012. So in this way you're right that the simulation does not reflect the current situation. But you can update that file anytime to your current account parameters with the procedure described under "Data Import".

How about the follwing solution to conclude this discussion: You use a script function that modulates your profits with the exchange rate.

I can not recommend this, for several reasons. For instance it adds a level of overfitting to the simulation and reduces the quality of trained parameters. So your strategy will probably become less profitable. But if you want to do that nevertheless, I can post a short function here that replaces the broker's pip cost with an exchange rate that you can select. Would that work for you?

jcl: that sounds like it would help. My initial idea would have been to calculate the pip-values for each pair and each day (daily is sufficient for this) and store it in a csv-file or similar. I suppose its possible to write a Zorro script to update said files. The pip-values could then be adjusted by looking it up everytime the strategy detects a new date.

With eur/usd and an EUR-based account its easy, because you only need the eur/usd rate for calculation. But with i.e. usd/chf its more complicated, you need the eur/chf rate to calculate the pip-values. Thats the reason i would have suggested pre-generated files, but maybe you have a better idea?

There is a more wholistic solution that is used by other backtesting software, where each asset is set up with information in which currency it is quoted. For example, if you have a portfolio of stocks of Citi, BP and BMW, you have postions as C/USD, BP/GBP and BMW/EUR, the pnl is calculated in the quote currency, and in a second step converted to the account currency according to fx rates. Likewise with USD/CHF it is represented as a position of asset USD quoted in currency CHF (= the usd/chf rate). The pnl in CHF is then converted to the account currency of eur with the eur/chf rate. But i suppose you need a full blowen symbol manager and a major redesign for this and im not sure if this is compatible with the light-weight spirit of zorro.


Not to belabour the point from the previous post, but the screenshot i posted is a microlot demoaccount, and pip-cost is displayed as 0.08 in the software - while clearly in the report 0.077 is used. That seems like they are infact rounding the 0.077 (and possibly even more digits) to 2 digits for display purposes. And i would be surprised if this value wouldnt be adapted on at least a daily basis, and the day-to-day changes are behind the first 2 digits, so they are not displayed, while still used for pnl calculation. But i can ask a fxcm rep next week for details.

http://i.imgur.com/ew3Cp4R.jpg

I dont understand your objection to this. If i want to simulate a trade in for example the year 2005, i want to use the pip-value from 2005 for the pnl-calculation. Using the the pip-value from 2013 would make no sense - i want to know what the result would have been - not the result with the pip-value of 2013. And it would be forward-peeking, the pip-value of 2013 is not knowen in 2005. Its not fitting anything, it just simulates what would have happened in 2005.

Ok, here's the function:



In your run function, call

convertProfit("EUR/USD");

when your account currency is EUR and the counter currency is USD.

- No, forward peeking is something quite different. What you're doing here is training the strategy to trade more often in the months with better exchange rate. But there's most likely no correlation between your trade signals and the exchange rate, so you'll get distorted parameters and less profit in real trading.

At least that's what I suspect. I don't think that many people simulate a strategy this way, so there are no experiences how this really affects the profit, and my objections are purely theoretical. Just try it!

You can easily check the result of your method with walk forward analysis f.i. with the strategies from workshop 5, 6, or 7, or with your own strategies. Let me know how it worked.

Thanks for the function. I did plot pipcost on a eur/usd chart and it works.

But i dont yet understand how often i need to call this. Do i have to call this function on each run() call? Or does Zorro remember it throughout the rest of the simulation if i set it once?


I dont know, to me this just makes sense. If i make a trade today, my pnl will be with current rates. When i look back in 5 years time, this fact will not have changed, regardless of what the exchange rates will be in 5 years. And since a simulation should be as close to reality as possible, i would then, if i had to do a simulation of this trade, of course simulate the rates from today, because that is just what happened. If i would use rates from 2018 to convert pnl, the outcome would be different to what actually did take place in the past, thus lessening the quality of the simulation. But lets just disagree..

You have to call it when you want the rate to adapt. Normally on each run(), but I think once per day would do also. The function is fast though, so it won't matter much for simulation speed how often you call it.

The simulation must only be close to reality where the trade signals are involved. In all other regards, implementing 'realism' does no good when it affects your strategy parameters. System developers normally try to remove external factors, not add them. I would certainly not include factors such as the account currency. But that's just my advice - Zorro is a tool and you can use it in any way you want.

Theoretical discussions are futile anyway. You can easily test your ideas with the workshop strategies or your own strategies, and then see if they work or not.

Hi, one more question: i entered a limit order for a specific price:

enterShort(1, -1.2930);

But the fill comes back with 1.29274 according to the CSV. I assume the spread of 2.6 pips has been deducted from the limit price - although a limit entry should be no worse than the limit price, in contrast to stop or market orders. Is it possible to change this behaviour?

Also i dont understand the slippage setting, which is set to 10 seconds by default. The docs are a bit ambigous about how this works exactly. I understand the minimum barperiod in Zorro is 1 minute?

All prices in stops, limits, targets, indicators etc. are generally Ask prices - no matter if it's for a long or short order. If you want to use the Bid price instead, you must subtract resp. add the spread.

Slippage determines a range on the price curve that contains the likely price at which the order is filled or closed. So with 1-minute bars and 10 seconds slippage the order is filled from within 1/6 of the estimated price curve inside the bar.

Still, in live trading a limit-order will fill no worse than the limit-price (if it is filled at all).

It does not. What you entered was the Ask Price. Short orders fill at the Bid price.