Issues tracker

Submit issue and bug reports here. For feature requests and enhancements, use our Roadmap instead. For other issues, or if you prefer to reach us privately, contact support.

Trending
  1. Extend Nasdaq coverage to include last hour of extended trading, when it crosses into next UTC day

    During winter months, when Eastern Time is UTC-4, the last hour of Nasdaq extended trading session (7 to 8pm ET) extends into the following UTC day. This is assigned D-2127 internally.

    Jack C

    0

  2. DBEQ.BASIC historical data contains "merged" bars

    For DBEQ.BASIC, on our ohlcv-1s/m/h/d schemas, on each interval we should publish a message for each venue where a trade happened. Since 2024-07-01, historical data for DBEQ.BASIC has been published with "merged" OHLCV data, hence the trade volume is not for the specific publisher_id, but across all the venues which comprise DBEQ.BASIC.

    Renan Gemignani
    #bug#data-quality

    2

  3. Saturday test data shows up on CME live data

    Currently our historical data discards Saturday test data on CME, however our live data gateways expose the test data because itโ€™s useful for Databentoโ€™s own internal monitoring to ensure that our live gateways are running properly. We plan to discard this test data in live as well to ensure point-in-time behavior. Weโ€™ll likely have to hide test data from customers. This is assigned D-2036 internally and will likely only be addressed in Q3.

    Tessa Hollinger
    #bug#data-quality

    0

  4. Leg information is not provided for composite instruments in CME and ICE

    Currently, the InstrumentDefMsg provided by our definition schema does not contain the underlying leg information. This is assigned internal ticket D-2032.

    Renan Gemignani
    #symbology

    0

  5. MDOrderPriority is not provided for CME Globex MDP 3.0

    We currently leave out MDOrderPriority from our normalized MBO messages, due to 2 reasons: This behavior is CME-specific. Most other venues adopt ITCH-based behavior, where ordering is based on timestamp instead. If we supported MDOrderPriority, it would bloat the data for all of the non-CME venues. ts_event serves the same purpose as MDOrderPriority for all instruments except for interest rate options and instruments where there's a LMM. We confirmed this with CME GCC and also from practical experience of comparing simulation against live order matching. We actually recommend most users to infer MDOrderPriority from ts_event since it makes their order book implementation more reusable for other venues. Two proposed solutions were considered: Exposing raw message payloads and PCAPs. 'Supplementing' our normalized MBO data with venue-specific fields. We consider approach 2 to be inferior: If we tried incorporating other fields, we'd be replicating the venue's original structure in DBN with unnecessary indirection and copies and losing the benefit of normalization โ€” so we might as well expose the raw message payload. For many venues, the raw data has nested message tree structures, and if we replicated it, either in Databento Binary Encoding or an unstructured encoding like JSON, we'd lose a lot of the benefits of our zero-copy binary encoding. The only limitation of approach 1 is that it's bandwidth-intensive, obviously as most venues require you to receive their raw multicast data on their extranet over a physical cross-connect. At this time, we reject approach 2 and are working towards approach 1. This is tracked on our roadmap here. This is tracked internally as D-2130.

    Tessa Hollinger
    #http-api ๐Ÿ”—#raw-api ๐Ÿ”—

    1

  6. Orders outside likely price bands on CME

    e.g. 198,000.00 order here: 2024-09-15T12:00:05.054156821Z 2024-09-15T12:00:04.707396583Z 160 1 4358 A A 21643.000000000 1 8 6855758107682 40 0 2751 2024-09-15T12:00:05.054156821Z 2024-09-15T12:00:04.707396583Z 160 1 4358 A A 22000.000000000 3 8 6853082252068 40 0 2745 2024-09-15T12:00:05.054156821Z 2024-09-15T12:00:04.707396583Z 160 1 4358 A A 23500.000000000 1 8 6855315723493 40 0 2748 2024-09-15T12:00:05.054156821Z 2024-09-15T12:00:04.707396583Z 160 1 4358 A A 198000.000000000 1 8 6855095244654 168 0 2747

    Tessa Hollinger
    #bug#data-quality

    0

  7. Latency tails and performance issues

    There are a few latency hotspots that we're working on. In general, the median latency is acceptable but we'd like to confine the tails. BBO-1x and OHLCV-1x There have been reports of latency spikes around our subsampled schemas, especially BBO-1x which would be used for a more latency-sensitive use case than OHLCV-1x and cannot be replicated from trades, sometimes stretching to a few seconds. These may be due to the memory bandwidth burst when publishing 600k to 1.2M instruments on the dot at the same time. Specific times of day Shared by one of our users, it appears that ts_out - ts_recv spikes to 1-3 seconds on certain days, e.g. attached screenshot for Sep 18, during Fed rate cut announcement. Mass cancels After stripping out a live gateway, we find that we're keeping to 5.1~6.5 us median, but 300-800 us 99.9th percentile. This appears to be due to latency spikes around mass cancels. dataset=GlbxMdp3 quantiles=[(0.0, 1565), (0.1, 2787), (0.25, 3597), (0.5, 5739), (0.9, 87679), (0.95, 214527), (0.99, 428031), (0.995..., (0.999, 658943)] dataset=GlbxMdp3 quantiles=[(0.0, 1615), (0.1, 2447), (0.25, 3187), (0.5, 5143), (0.9, 30079), (0.95, 68479), (0.99, 173567), (0.995, ..., (0.999, 312831)]

    Tessa Hollinger
    #performance

    0

  8. Live latency estimates inaccurate due to new subscription behavior

    For example, running the example in https://databento.com/docs/examples/basics-live/live-latency/ will result in large numbers in the tens of seconds: 00p 32008.355ms 50p 32042.988ms 90p 32079.569ms 99p 32079.666ms Each latency sample appears to be monotonically increasing by a small amount i.e. If you increase the number of samples in above script, the latency numbers will appear even larger, suggesting that the client starts to fall behind. The problem is not noticeable if the user were instead to modify the script to only measure latency for 1 instrument ID or symbol, since the symbol mappings are negligible in that case. This appears related to changes made to the subscription or symbol resolution behavior recently and how long it takes to receive all symbol mappings.

    Tessa Hollinger
    #bug#raw-api ๐Ÿ”—

    1

  9. Parts of site do not reflect zero usage fees for users on flat-rate subscriptions

    Since our site was originally designed with usage-based billing in mind only, there are parts of it like the Data usage page, pricing, and pricing calculator pages that might not correctly show $0 consumption fees once you're on a flat-rate plan like our Enterprise (soon renamed Unlimited) and upcoming Standard plans. If you discover an issue around this or have any confusion with pricing once you're on an flat-rate plan, do contact support for assistance. We're in the middle of auditing our site and making updates to improve UX for flat-rate subscription users. Most recently, the Billing page has been overhauled for this.

    Tessa Hollinger
    #bug#portal ๐Ÿ–ฅ

    0

  10. PTP issues in Aurora I (DC3)

    Due to our switches flip-flopping between an incorrect grandmaster, we occasionally see PTP issues at DC3. This is usually biased negative by at least -37 seconds because this leads to the TAI/UTC offset being momentarily before flipping over. In particular: Root cause - Switch loses sync with parent likely due to Cumulus/performance issues.Switch loses sync with its parent and enters holdover mode. In holdover mode, switch sets the PTP_UTC_REASONABLE flag to false.PTP NIC stops using the provided UTC offset of 37 and instead uses the default of 0.Outbound NIC by default respects the UTC offset valid flag. We've filed the issue with the switch manufacturer, but it's unlikely to be fixedโ€”instead we'll be switching out to a different switch model in DC3 within September 2024. This issue should not affect our infrastructure in NY4 which has already been moved to the new switches. Based on our recent logs, we expect to see this at least twice per month. 08/06/24 02:17:05.035055 | 29780 | INFO | OSS | 19-8ff4c89e | Adapter 0 timestamp clock got Out-of-Sync with reference, TimeSkew: -55499999970 ns 08/21/24 07:31:54.314801 | 29780 | INFO | OSS | 19-8ff4c89e | Adapter 0 timestamp clock got Out-of-Sync with reference, TimeSkew: -55500001097 ns

    Tessa Hollinger

    0

  11. Create a test issue

    This is a test issue

    Enda

    0

  12. IFEU/NDEX - System priced leg trades are being counted towards OHLCV bars

    Some information on system priced leg trades is available here: https://www.ice.com/publicdocs/System_Priced_Legs_Enhancement.pdf In practice, those can be very far off the best bid and best ask. These trades should be counted only for the volume, but not for the price.

    Renan Gemignani

    0

  13. Continuous contracts with statistics-based roll rules are not ranked correctly if statistics data is published late

    A continuous contract with a statistics-based roll rule such as GC.n.0 can be incorrectly ranked if the exchange is late to publish statistics.

    Nicholas James Macholl
    #bug#data-quality#symbology

    0

  14. Partially missing test symbol symbology in DBEQ.BASIC

    Our DBEQ.BASIC data has some records with instrument_ids that do not map to a symbol (or an empty symbol field). These all correspond to test symbols ATEST, CTEST, and PTEST, as well as subcategories of these such as PTEST W and ATEST C. Only test symbols are affected. Internal tracking number: D-2527

    Zach Banks
    #symbology

    1

  15. Imprecise `display_factor` for CME Globex instruments with fractional pricing

    As CME sends the data, instruments with fractional pricing may have a display_factor that ends in e.g. 12.0 instead of 12.5. This caveat is noted on their page: EXCEPTION: For products that tick in modified fourths (30-Day Fed Funds options and Rough Rice options), the decimal '.5' is denoted in the price display. The Display Factor will be sent as '01' but should be treated as '00'. For example, an actual price of '12.5' would be displayed as '12' even though the display factor value '01' normally dictates a display of '12.5'. Therefore, client systems must include the .5 in the fractional conversion to decimal calculation for these products. Databento should apply this rule to the display_factor field for customer's convenience. Internal tag: D-2693

    Zach Banks

    0