Skip to content
ParticleDB
Release candidateParticleDB 1.1.19-rc.17 is not yet GA or ready for production data.Release status →

ClickBench methodology

ClickBench measures an analytical workload over one flat table. It does not establish transaction correctness, general application throughput or performance on every data model.

This page publishes methodology, not a leaderboard placement. Internal candidate measurements are not accepted upstream results. A ranking needs the dated comparison snapshot, hardware and filters; it can change as other entries change.

A submission must identify an immutable public binary URL and SHA-256, server version and edition, client version, upstream revision, operating system, hardware and storage. Publish the entry scripts, schema, all query definitions, raw results and failures. Installation on a fresh machine must not depend on a private source checkout, an existing binary or an unpublished SDK.

A result from one binary is not evidence for a different installer download. If the build or its optimization profile changes, repeat the measurements before replacing the reported artifact.

Declare the dataset, row count, load command and persisted on-disk size. Document storage-engine and collation settings. For a segment-table entry, state its supported operations explicitly; a read-only analytical result does not establish UPDATE/DELETE support or mixed transactional performance.

Follow the ClickBench rules at the submitted upstream revision:

  • Run all 43 queries. Preserve the first cold run and the two following hot runs; do not add internal-suite warmups.
  • A true cold run requires a server restart and OS page-cache clearing before each query’s first run. Otherwise disclose the limitation using upstream’s required tag.
  • Disable query-result caching. Document other caching and configuration choices.
  • Measure sending, executing and receiving results. Do not replace result delivery with a null-output mode.
  • Keep failed or unavailable measurements visible as nulls; do not omit them to improve an aggregate.

Use the upstream aggregation rules. A normalized timing ratio above one means slower than its reference, not a speedup. Link the exact comparison view before quoting a score or placement.

Have another developer run the public instructions on a fresh matching machine. Preserve correctness checks, binary hashes, full logs and deviations. This is an independent reproduction, not upstream acceptance or a guarantee that every repetition falls within a fixed percentage.

Shared methodology · Release availability · OLAP comparisons