Issues tracker

Some information on system priced leg trades is available here: https://www.ice.com/publicdocs/System_Priced_Legs_Enhancement.pdf In practice, those can be very far off the best bid and best ask. These trades should be counted only for the volume, but not for the price.

During winter months, when Eastern Time is UTC-4, the last hour of Nasdaq extended trading session (7 to 8pm ET) extends into the following UTC day. This is assigned D-2127 internally.

Currently our historical data discards Saturday test data on CME, however our live data gateways expose the test data because it’s useful for Databento’s own internal monitoring to ensure that our live gateways are running properly. We plan to discard this test data in live as well to ensure point-in-time behavior. We’ll likely have to hide test data from customers. This is assigned D-2036 internally and will likely only be addressed in Q3.

Currently, the InstrumentDefMsg provided by our definition schema does not contain the underlying leg information. This is assigned internal ticket D-2032.

We currently leave out MDOrderPriority from our normalized MBO messages, due to 2 reasons: This behavior is CME-specific. Most other venues adopt ITCH-based behavior, where ordering is based on timestamp instead. If we supported MDOrderPriority, it would bloat the data for all of the non-CME venues. ts_event serves the same purpose as MDOrderPriority for all instruments except for interest rate options and instruments where there's a LMM. We confirmed this with CME GCC and also from practical experience of comparing simulation against live order matching. We actually recommend most users to infer MDOrderPriority from ts_event since it makes their order book implementation more reusable for other venues. Two proposed solutions were considered: Exposing raw message payloads and PCAPs. 'Supplementing' our normalized MBO data with venue-specific fields. We consider approach 2 to be inferior: If we tried incorporating other fields, we'd be replicating the venue's original structure in DBN with unnecessary indirection and copies and losing the benefit of normalization — so we might as well expose the raw message payload. For many venues, the raw data has nested message tree structures, and if we replicated it, either in Databento Binary Encoding or an unstructured encoding like JSON, we'd lose a lot of the benefits of our zero-copy binary encoding. The only limitation of approach 1 is that it's bandwidth-intensive, obviously as most venues require you to receive their raw multicast data on their extranet over a physical cross-connect. At this time, we reject approach 2 and are working towards approach 1. This is tracked on our roadmap here. This is tracked internally as D-2130.

A continuous contract with a statistics-based roll rule such as GC.n.0 can be incorrectly ranked if the exchange is late to publish statistics.

Our DBEQ.BASIC data has some records with instrument_ids that do not map to a symbol (or an empty symbol field). These all correspond to test symbols ATEST, CTEST, and PTEST, as well as subcategories of these such as PTEST W and ATEST C. Only test symbols are affected. Internal tracking number: D-2527

As CME sends the data, instruments with fractional pricing may have a display_factor that ends in e.g. 12.0 instead of 12.5. This caveat is noted on their page: EXCEPTION: For products that tick in modified fourths (30-Day Fed Funds options and Rough Rice options), the decimal '.5' is denoted in the price display. The Display Factor will be sent as '01' but should be treated as '00'. For example, an actual price of '12.5' would be displayed as '12' even though the display factor value '01' normally dictates a display of '12.5'. Therefore, client systems must include the .5 in the fractional conversion to decimal calculation for these products. Databento should apply this rule to the display_factor field for customer's convenience. Internal tag: D-2693

For some CME options symbols, the strike_price provided in the definition schema is not taking into account the correct display factor of the underlying future. We are still investigating the full extent of the affected products and how to work around this in the meantime. The following options parent symbols are affected at this point: 0SR,1YH,1YM,1YZ,2YH,2YM,2YZ,A4D,A6I,A6J,AE7,CZL,CZM,ESR,LN1,LN2,LN3,LN4,LNE,MC3,MC4,MC5,MC6,NC3,NC4,NC5,OC6,OLD,OMD,OZL,OZM,OZR,S0,S03,S04,S2,S23,S24,S3,S33,S34,S4,,S5,,SR1,SR3,ZL1,ZL2,ZL4,ZM1,ZM2,ZM4,

For NBBO quotes (MBP records with publisher_id set to 30, OPRA.PILLAR.OPRA) the publishers with the NBBO are indicated in the bid_ct_00 / ask_ct_00 fields. Prior to 2023-08-29 these fields are always set to 0.

This is typically available latest 11:00 UTC+0 the next day. It seems the dataset has not been appended to since the 29/04 session. Reproduce: curl -G 'https://hist.databento.com/v0/metadata.get_dataset_range' -u <API KEY>: -d dataset=GLBX.MDP3; {"start_date":"2017-05-21","end_date":"2024-04-29"} For an individual instrument: curl -X POST 'https://hist.databento.com/v0/timeseries.get_range' -u <API KEY>: -d dataset=GLBX.MDP3 -d symbols=ESM4 -d schema=ohlcv-1d -d start='2024-04-29' -d end='2024-05-01' -d encoding=json -d pretty_px=true -d pretty_ts=true -d map_symbols=true -d limit=1 | jq { "detail": { "case": "data_end_after_available_end", "message": "The dataset GLBX.MDP3 has data available up to '2024-04-30 00:00:00+00:00'. The end in the query ('2024-05-01 00:00:00+00:00') is after the available range. Try requesting with an earlier end.", "status_code": 422, "docs": "https://databento.com/docs/api-reference-historical/basics/datasets", "payload": { "dataset": "GLBX.MDP3", "start": "2024-04-29T00:00:00.000000000Z", "end": "2024-05-01T00:00:00.000000000Z", "available_start": "2017-05-21T00:00:00.000000000Z", "available_end": "2024-04-30T00:00:00.000000000Z" } } }

I am seeing across all files in my TBBO batch, across a range of futures markets, an average of about 62 entries where the above condition applies... here is an example of a single row. ts_event 2024-02-18 23:00:00+00:00 rtype 1 publisher_id 1 instrument_id 17077 action T side N depth 0 price 5016.5 size 44 flags 0 ts_in_delta 18276 sequence 3879 bid_px_00 5021.5 ask_px_00 5008.0 bid_sz_00 3 ask_sz_00 20 bid_ct_00 2 ask_ct_00 1 symbol ESH4 Name: 2024-02-18 23:00:00.030977523+00:00, dtype: object Is this expected behavior?

Trying to retrieve the following data set : params = dict( symbols='SR3.c.0', dataset='GLBX.MDP3', schema='ohlcv-1d', stype_in='continuous', start= '2017-06-01', end='2024-02-29', ) receiving the following error: databento.common.error.BentoError: Error streaming response: Response ended prematurely

This is related to: https://roadmap.databento.com/b/n0o5prm6/feature-ideas/support-more-secure-ftp This is assigned D-2126 internally and we plan to addressing this in Q2 2024.

Our MBP-1/MBP-10 schemas currently only provide side information on trades (to indicate the aggressing side).