Hourly data from IB

Posted By: MegaTanker

Hourly data from IB - 01/10/22 13:41

assetHistory has a mode for hourly data, but the manual only lists Bittrex and CryptoCompare for this mode. I tried using it with IB and while it looks like it kind of works, it produces a kind of corrupt history that looks like the image below. I'm assuming this means that this feature does not properly support IB? It would be a nice option considering minute data takes a day to download for a single asset due to their API limits.

[Linked Image]
Posted By: AndrewAMD

Re: Hourly data from IB - 01/10/22 13:49

I think it's more correct to say that IB has bad historical data.
Posted By: MegaTanker

Re: Hourly data from IB - 01/10/22 14:33

Maybe, but is it THAT bad? Even downloading something liquid like QQQ, these artifacts still exist (though they're more rare). And when viewing the graph in the TWS, the data does not have these errors.
Posted By: AndrewAMD

Re: Hourly data from IB - 01/10/22 14:56

You assume that the data displayed in TWS is the same data output by the Gateway API.
Posted By: Grant

Re: Hourly data from IB - 01/10/22 15:27

Andrew, would this mean that TWS (probably) has an internal data correction feature?

It's a shame that this reputable broker, which was once a front-runner in trading technology, hasn't fixed these basic issues.
Posted By: Petra

Re: Hourly data from IB - 01/10/22 17:28

Neither bad data nor TWS correction, but IB hourly data is not probably not implemented in assetHistory. Load M1 and then convert it to H1, or post a feature request for IB H1 with assethistory.
Posted By: Zheka

Re: Hourly data from IB - 01/11/22 16:43

If you are at least slightly familiar with Python you can use ib_insync and its ready-to-use notebooks for basic tasks: https://nbviewer.org/github/erdewit/ib_insync/blob/master/notebooks/bar_data.ipynb
Posted By: MegaTanker

Re: Hourly data from IB - 01/11/22 20:07

While we're on the topic the IB api... Does anyone know exactly what their BID_ASK mode returns for the Open and Close? The doc states "Time average bid" for Open and "Time average ask" for Close but I'm not sure what that means and I don't find it explained anywhere in the doc. The bars it returns look strange. See below comparison of TRADES mode vs BID_ASK mode

[Linked Image]
[Linked Image]
Posted By: 7th_zorro

Re: Hourly data from IB - 01/12/22 00:33

Try this.

Outlier = 0;
Posted By: Zheka

Re: Hourly data from IB - 01/12/22 01:23

'Time average bid' is the average of all bids during the bar; same for asks. Similar to what Zorro delivers with price().

The bid-ask spread is negligible relative to a 1-hr high-low range - so you see these marks as almost one on the chart.
Posted By: MegaTanker

Re: Hourly data from IB - 01/12/22 11:00

That makes sense and explains the small size of the candles. But probably not the best choice to use for backtests with longer timeframes. The average ask can be very different from the actual closing ask of the candle which is when you would actually buy the asset.
Posted By: Zheka

Re: Hourly data from IB - 01/12/22 11:11

Yes, i guess the use of it is for benchmarking execution algos.

Have a look at ib-insync.
Posted By: MegaTanker

Re: Hourly data from IB - 01/12/22 19:37

I'm familiar with that package, I used it for the barplot in the previous post.

Unfortunately IB is returning the same nonsense for this asset using python as it does in Zorro. TRADES mode seems to work, but BID_ASK or BID or ASK returns tons of bogus outliers.
© 2024 lite-C Forums