Asset History Questions

Some of these issues of this big topic have been hinted at elsewhere, but I can't recall seeing explicit recommendations...

a) Should separate History be maintained for each Broker used? (Obviously at the expense of disk, potentially lots of disk)
b) Does this matter less for EOD History? I.e., can any Broker's History be used for EOD?
c) Does it matter more and more the finer the Trading time period is?
d) If I read the manual right, some of the Z Systems are Tested using a simulated Oanda account but with FXCM data? When is this mix-and-match OK? At what BarPeriod or other measure does it become critical to use the target Broker's History?
e) If one has multiple History files for an Asset, possibly with partially overlapping data, can those safely be combined? Is there a helper script to do so?

And so on... As I said, a big topic...

You need different histories when a) the prices are broker dependent and b) the differences matter for your strategy.

Thanks @jcl.

Actually, for that portion, I was asking more metrics and/or "rule of thumb" questions... Let me try a different way:

a) For long-term periods between trades - f.i. Z8, Z9, strategies that don't trade intra-day, etc. - EOD data from anywhere is good enough? Whatever differences there might be won't matter?

b) For shorter periods, the example mentioned above was data from one Broker while simulating trading at a different Broker. Is there a metric or "rule of thumb" to tell when likely "the differences matter" without the time, trouble, disk space, etc. of actually downloading a whole 'nother set of data to test?

c) In cases where we've ended up with multiple History files for an asset that we wish to combine, is there a helper script to accomplish this?

Thanks.

Z8 and Z9 trade stocks and ETFs. They are traded on exchanges. Their prices are not made up by the broker.

As a rule of thumb, when you trade Forex, CFDs, Cryptos, or any other asset where any broker has different prices, and your system runs on ticks or short bars less than 10 minutes, then you should compare backtests with prices from different sources and find out if the results are affected.

Thanks @jcl. That's much clearer.

Z8/9 were meant as examples of long time-frame trading more than price source. A better example would have been Z10 where Crypto prices are Broker-dependent - since it uses daily bars Broker intra-day price differences don't matter, correct?

Thanks.

Yes, for daily prices the broker differences are negligible.

@jcl,

Thanks!

For combining/merging History files, is there a helper script I've missed?

Thanks.

@jcl,

I assume, then, there is no existing helper script for this.

If I undertake to do this, are there particular dangers or other issues one should beware of in combining History files?

Thanks.

The t6 files are nothing more than dataSets with six fields. Use the file and dataset functions.

The simplest way is merging files with the copy command. Bill Gates helps: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/copy