Benchmarks and methodology
Performance depends on the workload and configuration. A useful result includes enough information to reproduce both its behavior and its timing.
This documentation revision does not carry forward comparative headlines without a matching, reproducible result bundle. Application examples are correctness checks, not performance claims.
Workloads
Section titled “Workloads”| Workload | What it measures | Methodology |
|---|---|---|
| Analytical queries | Load, cold and warm query behavior | ClickBench |
| Operational transactions | A defined transaction mix under concurrency | Transaction workloads |
| Application analytics | Queries over representative operational data | OLAP comparisons |
| Key-value | Durable SET, GET, INCR, list and hash throughput against Redis | Key-value: ParticleDB and Redis |
Required result bundle
Section titled “Required result bundle”Every published run must identify:
- Server and client versions and the source revision where available.
- Hardware, operating system, CPU count, memory and storage.
- Dataset source, scale and query or transaction definitions.
- Indexes, transaction settings and durability settings.
- Warmups, timed repetitions and the reported statistic.
- Per-query or per-operation results, raw logs and failures.
- Competitor versions and matching settings for a comparison.
Run discipline
Section titled “Run discipline”Run one benchmark at a time on an otherwise idle machine. Follow the named suite’s protocol: ClickBench uses its upstream cold/hot sequence without additional warmups; internal suites with two warmup iterations must retain them. Repeat internal comparisons and name the reported statistic. For before/after comparisons, run both versions on the same machine in the same session.
The 10-million-row cases generally give a more stable comparison than small samples. Larger cases can amplify variation from machine load.
Compare like with like
Section titled “Compare like with like”Do not compare relaxed persistence with durable commits, cached data with a cold run, different query semantics, or failed work with successful work. Report the gaps as well as the favorable results.
Current validation
Section titled “Current validation”Documentation and result-format checks do not establish a performance ranking. Measurements must be repeated in an idle environment before a new number is published.
This site uses Google Analytics cookies to count visits. Privacy · License