What is Pendulum Flow Orderbook Archive?

The archive is organized into hourly Parquet files, letting you choose a bounded interval and read selected columns or row groups with DuckDB or another Parquet engine. Rebuilding depth needs both full book snapshots and subsequent price_change rows; trade prices alone are insufficient.

Capture eras must remain visible in any derived dataset. V3 includes receive-time, witness and sequence information, while earlier collectors differ in schema and fidelity. An hour awaiting publication is different from a missing recording; audit gaps and hour boundaries rather than assuming continuous coverage.

Who it is for

Researchers rebuilding historical order books.
Engineers building local Polymarket datasets.
Analysts requiring inspectable provenance and gaps.

Dataset scope and fields

Hourly Parquet
Read a chosen hour without downloading everything.
Snapshots and deltas
Combine book snapshots and price_change rows for resting quantities.
Trades and lifecycle
Access trade prices, new markets, resolutions and tick-size changes where recorded.
V3 capture provenance
Inspect receive timestamps, witness fields and sequence data.
Checksums and manifests
Verify file hashes and review event counts before processing.
Coverage audit
Review absent hours and era-specific collection limits.
Remote columnar reads
Use DuckDB or another reader to select columns and row groups.

Data preparation and analysis

Reconstruct one book

Choose an audited V3 interval, load a snapshot and apply token-level changes in receive order, handling the hour boundary.

Study size sensitivity

Rebuild both sides, walk price levels for a chosen size and compare with the same-time midpoint.

Keep an inspectable extract

Store source paths, checksums, capture era and filters with the derived dataset.

Pricing and total cost

Public downloads are free without an account or API key. Donations support capture, storage and delivery.

Dataset access and delivery

Select a time range in the coverage index and read that era’s format.
Inspect the hourly manifest and checksum, then query one market and a few columns.
Sort by the era’s time/sequence semantics and record missing intervals.

Read the V3 format

Useful links

What to check

Capture versions have different schemas and fidelity.
V3 receive timestamps use microseconds; older mirrored data commonly uses milliseconds.
Collector sequences are not global exchange ordering across machines.
Publication delay differs from a missing recorded hour.
Hourly files can exceed 1 GB; use a columnar engine rather than a spreadsheet.

A closer look

Ratings & reviews

Editorial rating

4.0/ 5

0 user reviews

PredictionToolkit editorial

Our take on Pendulum Flow Orderbook Archive

Public files, checksums and explicit audits make the archive inspectable. Combining capture eras still requires careful treatment of schema and timestamp semantics.

Assessment criteria: Task usefulness, documented scope, setup clarity, and access and cost transparency.
Assessment date: 2026-10-01
Documentation reviewed: 2026-10-01

Documentation-based assessment. How we rate