Corrupted Z12.trd?

Hi. Windows (Server 2016) wanted to do Updates, so today (Sunday) I clicked Stop, No to Close Trades, Yes to Stop Trading, exited Zorro, and did the Updates, including the reboot after.

Now Z12 will not Trade. It logs in to the broker, loads prices, prints "Trade: Z12 <date>" and then up pops the "Zorro Automaton has stopped working" window. I have also tried with "- diag", but no additional information is recorded.

The same VPS also runs Z7 (from a different directory) and it has restarted fine.

So, it seems the Z12.trd file has somehow been corrupted? How can I verify this and repair the trd file? Is there a trd utility somewhere?

Thanks.

Maybe a virus?

If it were the .trd you would see an error message, what you had looked to me more like a broken zorro.exe or z12.x. Better reinstall it.

Thanks Petra. Reinstall done. Unfortunately same crash.

FYI: Reinstall above was done into existing directory. Have also now done fresh install into new directory and copied the trd file into it - same crash.

I have also renamed the trd file and Traded Z12. A new trd file is created and Trade starts fine. The problem seems clearly to be the trd file has been corrupted. Is there any utility to examine and/or repair trd files? Or a manual way to examine/fix it?

Its binary, so you cannot look into it easily, but you could load it with loadStatus and different SaveMode flags to check which parts of it are readable.

Thanks Petra. I will see if that gives any clues. Do the SaveMode modes cover the entire contents of a trd file?

Thanks again, Petra. loadStatus crashes trying to load the AlgoVars. Sliders and Trades seem OK.

Now, how can I fix the AlgoVars?!

You could reset the SV_ALGOVARS flag. They will then not be loaded. But that would not work for Z12 where you have no code access. Delete the broken .trd file and close the trades manually.

Thanks @jcl.

Would the code below work? It creates a MUCH smaller trd...

@jcl: Would appreciate your thoughts before I try using the post-surgery trd. 116k went to 12k. Is this expected? 90% of a trd is the algovars?

Thanks.

I estimate for algovars a size of about 1 kb multiplied with the numbers of algos and assets.

Thanks @jcl. That puts the size reduction is in the same ballpark, so I'll give the tiny trd a try.

FYI, I save a copy of the trd file most every weekend. I tested starting Zorro (in a test environment, not live) with these backups, and had to go back a number of weeks to find an un-corrupted trd. Zorro ran continuously over that period, and the weekly trd files have different sizes. Since Zorro was never stopped and restarted, the trd file was apparently never read - only written to. Unfortunately, especially given the passage of time, I have no hope of correlating the corruption to anything else. But from these circumstances a reasonable suspicion could be a rare pointer/buffer problem in Zorro?

HTH.

@jcl: The saga continues. The tiny trd allowed Z12 to start. But:

a) Intermingled with the "resumed" messages were a number of "closed" messages. The "closed" messages were all for real trades, and all exactly 1 lot. Is this expected?

b) After the "resumed" messages came a slew of Trails, Stops, and Longs. I assume this is normal as Z12 adjusts to the new market conditions it finds? (Although without the benefit of the missing AlgoVars...)

c) Lastly, the net result of the above seems to be that the account balance being reported by Zorro is accurate, the number of open trades matches the number of open positions in the broker's web interface, but Zorro's view of the value of these open positions, including which are winning vs. losing, is incorrect. Is anything to be done?

FYI, the trd file has grown from 12k to 57k.

Thanks.

Closed means that the trade was not open anymore. You probably closed it?

Open value includes swaps and commissions, so it can differ.

Thanks @Petra.

- No, I didn't close anything. Wanted to keep everything unchanged on the hope the trd could be fixed. Also very suspicious that in every case exactly 1 lot was closed - no trades of 1 lot were ever opened.

- Open value is ~1/2 what the broker shows, even though balance is the same. Swap & commission could account for that?

There can be 1000 reasons of differences, hard to tell with so vague information. Compare the different trade in the platform trade list with the same trade on the trade status page. There you can see all parameters of the trade and calculate the open value yourself.

Thanks @Petra. Yes, I am aware of that process, and a full investigation would also include the trades done when Zorro "reconciled" to the new world it found after being disconnected for many days...

Before embarking on that, I was hoping @jcl or someone who knows the internals of Zorro and trd processing could add insight into why _exactly_ 1 lot closes during resume, and why the open value might have gone off (while balance is correct).

Hmm, how am I supposed to know your trade sizes and open values? If you want, you can get a support ticket and send them the log, trd file, and trade status page. They will look through all and explain the open value and why 1 lot was closed. I don't think Zorro internals are needed.

Thanks @jcl. Clearly you are not supposed to know what you can't. The question was explicitly if what you DO know (Zorro internals) could shed some light as to why on restart with AlgoVars removed:

- Zorro would at startup time of trades resume enter only multiple close orders across all Assets of exactly 1 lot - is 1 lot some sort of default or fall-back?

- Zorro would "lose its mind" and not know the correct values of open trades, while reporting correct account balance.

Thanks.

No, 1 lot is not a fallback. It's really the stored trade size.

The open value is simply the difference of current price and open price multiplied with the number of contracts.

Thanks @jcl. Well, I now know what happened. I made a mistake (no surprise there ), Zorro's message of closing 1 lot is in error and misdirected the investigation, and I would appreciate your help in how best to proceed.

Mistake: After determining I could successfully create a trd with the AlgoVars removed, in my exuberance I didn't double-check close enough and converted an old trd. Consequently, when Z12 finally was able to start, it was presented with an old trade list.

Message: Z12 startup resumed all the virtual trades and checked the real trades against the broker. Real trades found were resumed. Real trades not found were correctly listed as closed - but rather than printing the lots of those Real trades, Zorro printed "1", which I believe is a bug.

Help: Zorro is now running with a list of virtual trades that no longer match reality, since it's lost the intelligence of what happened to those trades in the interim. What to do? I suspect the response will be to close all trades in the broker interface and let Z12 start over; but are there any other choices to consider?

Thanks.

Hmmm... Or is Open Value not used in determining exit and I should just let the positions ride and all will become right over time?

Or are the Stops based on the (incorrect) opening prices and something needs to be done?

When it's really messed up, I would close all open trades and restart. A wrong .trd file can have all sorts of wrong stops or other parameters.

Normally .trd files should not go corrupted. The only scenario I can imagine is a hard server crash just at the moment when the .trd file is written.

Thanks @jcl. Please clarify - I see 2 scenarios:

1) In the Broker platform, close all trades. Zorro retains its virtual trades, and will re-enter real NET trades to match.

2) Click Stop, Close all Trades, and Stop Trading. In the Broker platform, verify all trades are gone - if any remain, close them. Delete the trd. When Zorro restarts, it will create a new, empty trd and restart completely fresh with no trades - virtual or real - resumed.

Which scenario did you mean?

2.

I have been trading live on Interactive Brokers with Zorro.

Unfortunately, my laptop crashed and burned - nearly to ashes...

Obviously, the open lives trades stated on the IB platform - is it possible somehow to re-reconnect those trades with Zorro again so that Zorro continues to manage them?

I see in the documentation a function

brokerTrades (int Filter): int,
Call the GET_TRADES command. Cancel all currently open trades, and replace them with the open positions on the broker account (if any).
https://manual.zorro-project.com/brokercommand.htm

Could this function be helpful in re-creating the .trd file?

Maybe there is an example script somewhere for this task?

brokerTrades will not recreate the original trades from the .trd, but create new ones. Theoretically a .trd can be constructed from that, but that's a forensic task and probably not worth the effort. Close the open positions and start over.

JCL - thank you!