Benchmarks

TPC-H at SF=1 / SF=10 / SF=100 / SF=1000 — AWS cloud runs (single node vs DuckDB/Polars; 4-node mesh vs Trino/PySpark; 1 TB on one box) plus the strict single-machine protocol on an Apple M4 Max. Median ms, same hardware and same Parquet files.

TPC-H, all 22 queries, SF=1 / SF=10 / SF=100 / SF=1000, measured two ways:

• AWS cloud runs — commodity c7i.4xlarge boxes: single node vs DuckDB / Polars, and the 4-node peer mesh vs Trino / PySpark on the identical cluster — plus the 1 TB run on a memory-class box. This is the “what you get on rented hardware” view.
• Single-machine strict protocol — Apple M4 Max, per-query process isolation, thermal gating, 2σ verdicts, five engines. This is the tightest-controlled per-query view.

Same Parquet files per track, shipped pip install ematix-flow defaults, no per-query tuning anywhere.

AWS cloud runs (v0.13.0, July 2026)

All AWS numbers: c7i.4xlarge (16 vCPU / 32 GB, us-east-2), TPC-H Parquet loaded from S3, all 22 queries value-validated, 5 trials × 2 warmups, sum of per-query medians. ematix-flow runs the shipped pip install ematix-flow defaults — no flags, no tuning. Raw per-query JSONs with full provenance (instance, AZ, git SHA, env) are stored with every run.

Single node — vs DuckDB 1.5.4 / Polars 1.42.1

Polars (standard build, per-query process isolation) is shown with its completion count: its totals cover only the queries it finished, so they understate its true totals — the comparison is conservative in its favor. What it can’t finish on this box is honest physics for an in-memory engine: five SF=100 queries (and two at SF=10) exceed 32 GB, and one query has no Polars SQL variant.

The SF=100 total includes the July 2026 join-side correction (default-on, no flags): Q09 — formerly a memory-cliff outlier that could swing 16–75 s on the 32 GB box — was root-caused to hash joins built on the wrong (150M/80M-row) side, whose ~12 GB peak evicted the page-cached working set. The planner now measures string-filter selectivity by sampling the actual file at plan time and builds on the small side; Q09 runs a flat 5.8 s (DuckDB: 6.2 s steady on the same box). Per query, ematix-flow is faster on 13 / 22 with 3 ties; every loss is under 0.7 s. The full diagnosis — cold-IO parity, page-cache eviction traces, before/after trials — ships with the raw run JSONs.

ClickHouse — both of its modes (SF=100, July 2026)

ClickHouse got its own head-to-head because it’s the engine most often cited against us on large data. Same c7i.4xlarge, same Parquet files, ClickHouse’s own published TPC-H queries and table DDL (vendored verbatim from their benchmark kit), newest stable server at the time (26.6.1.1193). Two modes, because ClickHouse is really two stories:

The headline: even after granting ClickHouse a 24-minute ingest and 41 GiB of duplicated storage, its partial 19-query suite is still slower than ematix-flow’s complete 22-query suite running straight off the shared files. And the two ClickHouse modes fail different queries — on this 32 GB box there is no ClickHouse configuration that completes all 22.

Per query (ClickHouse, both modes; DNF = did not finish):

Honesty notes, all reproducible from the raw run JSONs:

1. Q12 on parquet is a ClickHouse engine bug, not physics: Not found column materialize(l_commitdate) in block from its parquet reader on a column-to-column predicate. The same query runs fine on native tables (1.42 s).
2. Q02/Q05/Q08 on MergeTree each exceeded a 600 s-per-execution bound with a pathological low-parallelism join phase (~2 of 16 cores busy after all input rows were read; EXPLAIN shows an ordinary hash-join chain). Reproduced on both the embedded engine and the standalone server; the same queries complete on parquet views of the same data. Loading into the native format made these queries worse, not better.
3. Q09 on parquet needed 21.5 GiB against the 18.5 GiB cap that keeps the box alive (two earlier runs at higher caps took the whole machine down via untracked page-cache pressure). DuckDB and ematix-flow complete Q09 on the same box.

Context for ClickHouse’s own published numbers: their “TPC-H SF100 in 19.8 s” runs on a 59-core / 236 GiB ClickHouse Cloud node, and their join-improvement benchmarks use 32 vCPU / 128 GiB — several times this box, always on pre-ingested MergeTree. Nothing on this page compares against those machines; every number here shares one box and one set of files.

Distributed — 4-node clusters, vs Trino 482 / PySpark 4.1.2

Same cluster shape for every engine within each row (coordinator + 3 workers), same S3 data. SF=1 and SF=10 rows ran on 4 × c7i.2xlarge; SF=100 on 4 × c7i.4xlarge — every engine in a row shares its hardware. ematix-flow’s mesh is auto-detected per query (EMAT_MESH unset = AUTO): small queries stay single-node, large scans fan out over Arrow Flight.

Two honesty notes. PySpark’s SF=100 originally DNF’d 18/22 — that was our rig’s fault, not Spark’s: Amazon Linux mounts /tmp as a 16 GB RAM-backed tmpfs and Spark’s default shuffle dir sits there, so the shuffle-heavy queries ran out of “disk” no matter how large the EBS volume was. With shuffle scratch on EBS it completes all 22. The ematix totals were refreshed Jul 16 on the current build: AUTO now measures instead of predicting — each join query probes its execution modes untimed (native single-node, mesh, mesh with broadcast joins) and runs the trials on the fastest, with every probe row-verified against the other modes (that verification caught — and we then fixed — a distributed-planner shape that silently dropped rows, before any of it shipped in a release). Result: AUTO 46.0 s, vs forced-mesh 57.2 s and forced-single 57.1 s on the same fleet. The Jul 15 run (twin routing only) read 55.3 s; Jul 11 read forced-mesh 58.8 s / AUTO 59.1 s; the Jul 7 run predated the join-side correction at 61.9 s.

Two readings worth calling out:

• One node vs their cluster: single-node ematix-flow at SF=10 (4.76 s) outruns the 4-node Trino cluster (56.4 s) by ~12× and the 4-node PySpark cluster (65.0 s) by ~14× — on a quarter of the hardware.
• The cluster now beats one box at SF=100: 4-node AUTO (46.0 s) outruns single-node (51.8 s) — the first configuration where adding nodes lowers the TPC-H total instead of just adding capacity. AUTO gets there by refusing to guess: per query it runs whichever mode its own row-verified probes measured fastest, so it inherits the best of the native engine (Q05/Q12/Q15/Q21 stay single-node) and the mesh (broadcast-join fan-out on the scan-heavy joins) at once.

SF=1000 — one terabyte (July 2026)

The scale where “just use a single-node engine” is supposed to stop working, and where the cluster engines are supposed to earn their keep. Bigger memory-class boxes than the rest of this page: single-node engines ran on one r7i.8xlarge (32 vCPU / 256 GB); the cluster engines got 4× r7i.4xlarge — twice the aggregate cores and RAM of the one box. Same TPC-H Parquet dataset (~373 GB on disk), all 22 queries, 5 trials × 2 warmups, sum of per-query medians, value-validated.

ematix-flow at SF=1000 is the same story as SF=1: shipped defaults, no flags, complete suite. The single-node leg runs all 22 in 417.0 s on one box (v0.14.2, NO_DISTRIBUTE=1). Turn the mesh on — still one box, now with a loopback peer — and AUTO runs it in 384.2 s, ahead of the single-node leg on the same machine (faster on 16 of 22 queries). Per query the mode memo probes twin / mesh / mesh+broadcast untimed and runs the fastest, so AUTO inherits the native engine on the join-light queries and the broadcast-join fan-out on the scan-heavy ones (Q10, Q17, Q21) at once — the same “cluster beats one box” crossover the SF=100 row shows, here on a single machine. DuckDB completes all 22 on the same box class at 467.9 s. The cluster engines, on twice the hardware: Trino finishes all 22 in 4121.3 s — 10.7× slower than ematix AUTO — and PySpark never gets past Q04 (the Spark master repeatedly removed the application on the shuffle-heavy joins; the raw log ships with the run stamp). Raw per-query JSONs with full provenance: 20260717T164020Z (ematix AUTO), 20260716T193957Z (ematix single), 20260712T175438Z (DuckDB), 20260712T222000Z (Trino / PySpark).

Single-machine strict protocol (Apple M4 Max)

The tightest-controlled per-query view: five engines, all 22 queries at three scales — SF=1 (~1 GB, fits in cache), SF=10 (~10 GB, production shape), SF=100 (~100 GB, out-of-core) — one M4 Max, the same Parquet files, per-query process isolation, thermal gating, 2σ verdicts.

How to read the table below: each row is one TPC-H query; each column is an engine’s median time in milliseconds (lower is better). The fastest engine per row is highlighted in teal, and ematix-flow’s whole column is tinted so you can scan down it. ”—” means that engine couldn’t run the query at that scale — see caveats. Switch scale factor with the tabs above the table.

What changes across scale

Fast across the whole sweep — no soft spots (v0.12.0, 2σ verdicts). At SF=1 the working set is L3-resident and per-query constant cost dominates — ematix-flow is fastest on all 22 / 22 (2.51× geomean). At SF=10 the workload turns memory-bound: fastest on 17, with 5 queries a statistical tie (Q01 / Q03 / Q05 / Q08 / Q09, ematix nominally ahead in 3) and none slower — 1.47× geomean. At SF=100 — out-of-core, each query in its own process — fastest on 20, 2 ties (Q01, Q16), none slower — 1.37× geomean, +35% on the sum of medians. A handful of queries that were marginal in earlier releases (Q10 / Q18 at SF=100, Q05 at both scales) moved into the win column through engine work in 0.10–0.12: runtime-bloom build-side swaps, narrow-key decode, and the shuffle-free cluster-key aggregation (RANGE.AGG Stage 2) — each landed behind a strict interleaved A/B with a 2σ bar.

Concurrent-stream throughput

New in v0.12.0: the same strict harness measures N ∈ 100 simultaneous query streams per engine (seeded query permutations, solo engines, first batch discarded). ematix-flow’s concurrency-aware scheduling — a cross-process registry sizes partitions, the reader’s decode fan-out, and the rayon pool from one per-process core share — holds its throughput lead at every stream count (DuckDB shown as the columnar reference):

Caveats

• Polars can’t run several canonical TPC-H shapes; we feed it semantically-identical hand-translated variants. Q05 still overflows Polars’s default 32-bit row index (the bigidx build would fix it) and shows ”—” at SF=10 / SF=100; a few more SF=100 queries (Q08, Q10, Q19, Q21) likewise exceed it.
• Postgres is a row-store OLTP engine, included as a familiar single-node reference — not a columnar-analytics peer. At SF=100 it ran with a 90 s per-query cap; 16 / 22 heavy queries timed out (shown ”—”), so its SF=100 geomean covers only the 6 that finished.
• DuckDB runs at defaults (in-memory read_parquet). ematix-flow runs the production preset — target_partitions = cores, plus the fused-aggregate, dict-group-count, push-LeftSemi, transitive-semi, runtime-bloom, scale-relative-broadcast, and cluster-key single-phase aggregation rules. That’s the same config you get from pip install ematix-flow; no per-query tuning, one fresh context per query.
• Thermal note (SF=10 / SF=100): back-to-back runs on an M4 Max drift ~5–20 % as the package heats; the isolated passes run under caffeinate -i with warmups + 20-trial medians to bound it.

Reproducing

# ematix-flow + DuckDB — strict protocol (solo passes, per-query process
# isolation, thermal gating, env.json provenance, 2σ verdict diff)
cargo build --release --features triangulation --example tpch_triangulation_bench
scripts/bench/strict_22q.sh --sf 1 --engine ematix --isolate --out /tmp/em-sf1
scripts/bench/strict_22q.sh --sf 1 --engine duckdb --isolate --out /tmp/dk-sf1
python3 scripts/bench/strict_diff.py --a /tmp/em-sf1/strict-22q-summary.md \
  --b /tmp/dk-sf1/strict-22q-summary.md --out /tmp/verdict-sf1.md \
  --label-a ematix --label-b duckdb
#   → repeat with --sf 10 / --sf 100

# Concurrent-stream throughput (QPH, memory-guarded)
scripts/bench/strict_throughput.sh --sf 10 --streams "1,10,100" \
  --partitions auto --max-inflight 10 --min-free-gb 6 --out /tmp/tput-sf10

# Polars — the in-process triangulation harness
TPCH_DATA_DIR=examples/tpch/data/sf10 TPCH_SKIP_DUCKDB=1 TPCH_SKIP_EMATIX=1 \
  TPCH_OUT=/tmp/polars.md cargo run --release -p ematix-flow-core \
  --example tpch_triangulation_bench --features triangulation

# PySpark — needs JDK 17+ (brew install openjdk)
JAVA_HOME=$(/usr/libexec/java_home) \
  SPARK_DRIVER_MEM=16g SPARK_SHUFFLE_PARTS=64 \
  python scripts/bench-tpch-pyspark.py \
  --data-dir examples/tpch/data/sf100 --trials 5 --warmups 1

# Postgres — load the Parquet via ematix-flow's own ADBC ingest,
# then run each query under EXPLAIN ANALYZE.

The 2026-07-03 tables are the v0.12.0 release benchmark: ematix-flow and DuckDB re-measured at all three scales in one session with the exact tagged code under the strict protocol; full raw evidence (per-invocation runs, verdict diffs, env.json) ships in the repo at bench-results/release-v0.12.0/ and bench-results/sched-arc-2026-07-03/ (throughput). Polars / PySpark / Postgres columns are carried baselines from earlier same-machine runs (raw logs in bench-results/refresh-2026-05-30/). Release-over-release history lives in the repo CHANGELOG.