KalDB vs OpenSearch ClickBench Benchmarks
We are publishing ClickBench benchmark results for KalDB against OpenSearch using the KalDB vs OpenSearch comparison harness and its cold/hot protocol. The run covers all 43 ClickBench query slots, includes two repetitions, completes without failures, and shows KalDB faster on a majority of the weighted query scores.
The current version of KalDB is pinned to OpenSearch 2.11.1, so the raw numbers in this post use OpenSearch 2.11.1 as the comparison target. We have also run the harness against the latest OpenSearch version and seen similar numbers, but the updates for that latest version have not landed in the main branch yet. Once the latest OpenSearch version update lands, we will publish official benchmarks from that configuration.
The headline result: KalDB’s total weighted score was 25.9% lower than OpenSearch 2.11.1 across the full ClickBench query set.
Benchmark Data
The raw per-query data below includes cold latency, hot latency, weighted score, and KalDB/OpenSearch 2.11.1 score ratio for every ClickBench query slot.
Lower is better for cold, hot, and score. Hot is the faster of the two hot
runs for each repetition, averaged across repetitions. The ratio is
KalDB score / OpenSearch 2.11.1 score, so values below 1.0 mean KalDB was
faster.
| Query | KalDB | OpenSearch 2.11.1 | Score ratio | ||||
|---|---|---|---|---|---|---|---|
| Cold (ms) | Hot (ms) | Score (ms) | Cold (ms) | Hot (ms) | Score (ms) | ||
| Q0 | 160.5 | 78.0 | 98.7 | 120.0 | 60.2 | 75.1 | 1.313 |
| Q1 | 279.5 | 116.2 | 157.0 | 168.5 | 75.5 | 98.8 | 1.590 |
| Q2 | 710.5 | 519.9 | 567.6 | 825.8 | 614.2 | 667.1 | 0.851 |
| Q3 | 555.9 | 163.7 | 261.8 | 380.5 | 230.7 | 268.2 | 0.976 |
| Q4 | 541.6 | 173.3 | 265.4 | 387.4 | 204.5 | 250.2 | 1.061 |
| Q5 | 10002.8 | 9571.3 | 9679.2 | 12157.0 | 12361.7 | 12310.5 | 0.786 |
| Q6 | 306.9 | 232.2 | 250.9 | 46.1 | 7.5 | 17.1 | 14.658 |
| Q7 | 295.0 | 109.2 | 155.6 | 279.7 | 74.4 | 125.7 | 1.238 |
| Q8 | 1492.7 | 943.3 | 1080.6 | 1399.7 | 773.7 | 930.2 | 1.162 |
| Q9 | 1529.0 | 976.7 | 1114.8 | 1477.7 | 976.8 | 1102.0 | 1.012 |
| Q10 | 843.5 | 199.4 | 360.4 | 573.9 | 104.4 | 221.7 | 1.625 |
| Q11 | 1174.6 | 304.3 | 521.8 | 779.0 | 180.6 | 330.2 | 1.581 |
| Q12 | 458.0 | 248.0 | 300.5 | 955.0 | 175.6 | 370.4 | 0.811 |
| Q13 | 5218.7 | 3171.8 | 3683.5 | 4064.3 | 3110.4 | 3348.9 | 1.100 |
| Q14 | 8592.4 | 7638.6 | 7877.1 | 12743.6 | 11815.6 | 12047.6 | 0.654 |
| Q15 | 814.1 | 497.1 | 576.3 | 900.9 | 514.5 | 611.1 | 0.943 |
| Q16 | 13369.0 | 11626.4 | 12062.1 | 20493.8 | 19650.1 | 19861.0 | 0.607 |
| Q17 | 13314.8 | 11596.4 | 12026.0 | 21368.5 | 20261.1 | 20538.0 | 0.586 |
| Q18 | 17378.3 | 15164.0 | 15717.6 | 25740.1 | 24203.6 | 24587.8 | 0.639 |
| Q19 | 236.0 | 45.0 | 92.7 | 59.3 | 8.7 | 21.4 | 4.339 |
| Q20 | 2127.1 | 20.3 | 547.0 | 1459.9 | 137.6 | 468.2 | 1.168 |
| Q21 | 2244.9 | 35.5 | 587.8 | 1980.5 | 161.5 | 616.3 | 0.954 |
| Q22 | 4442.7 | 37.9 | 1139.1 | 4528.6 | 533.6 | 1532.4 | 0.743 |
| Q23 | 2298.4 | 43.4 | 607.1 | 1616.9 | 142.4 | 511.0 | 1.188 |
| Q24 | 478.9 | 231.4 | 293.3 | 266.5 | 101.9 | 143.0 | 2.050 |
| Q25 | 483.3 | 226.8 | 290.9 | 401.7 | 112.2 | 184.5 | 1.577 |
| Q26 | 538.5 | 239.7 | 314.4 | 385.8 | 98.5 | 170.3 | 1.846 |
| Q27 | 25769.5 | 22727.8 | 23488.2 | 29737.9 | 28731.9 | 28983.4 | 0.810 |
| Q28 | 62616.5 | 58641.7 | 59635.4 | 85866.2 | 84561.4 | 84887.6 | 0.703 |
| Q29 | 1863.2 | 1495.2 | 1587.2 | 2065.0 | 1594.2 | 1711.9 | 0.927 |
| Q30 | 2204.1 | 1324.5 | 1544.4 | 2038.4 | 1190.8 | 1402.7 | 1.101 |
| Q31 | 2618.3 | 1511.2 | 1788.0 | 2382.1 | 1488.6 | 1712.0 | 1.044 |
| Q32 | 11029.8 | 9746.9 | 10067.6 | 11526.1 | 10364.7 | 10655.0 | 0.945 |
| Q33 | 462.7 | 265.4 | 314.7 | 1746.7 | 493.6 | 806.9 | 0.390 |
| Q34 | 27110.8 | 24698.2 | 25301.3 | 30815.2 | 30023.3 | 30221.2 | 0.837 |
| Q35 | 7622.3 | 7030.3 | 7178.3 | 10279.8 | 9537.2 | 9722.9 | 0.738 |
| Q36 | 333.8 | 72.4 | 137.8 | 1386.4 | 83.4 | 409.1 | 0.337 |
| Q37 | 315.4 | 54.2 | 119.5 | 1825.9 | 53.6 | 496.7 | 0.241 |
| Q38 | 643.5 | 80.4 | 221.1 | 1401.6 | 66.0 | 399.9 | 0.553 |
| Q39 | 8108.9 | 6092.5 | 6596.6 | 9631.8 | 8712.6 | 8942.4 | 0.738 |
| Q40 | 446.6 | 96.6 | 184.1 | 414.4 | 137.8 | 206.9 | 0.890 |
| Q41 | 516.1 | 114.2 | 214.7 | 489.5 | 158.7 | 241.4 | 0.889 |
| Q42 | 327.1 | 66.8 | 131.8 | 306.3 | 71.3 | 130.1 | 1.014 |
Total: KalDB 209.1 s, OpenSearch 2.11.1 282.3 s, KalDB/OpenSearch 2.11.1 score ratio 0.741, overall KalDB score improvement 25.9%.
Score formula: 0.25 * cold + 0.75 * min(hot1, hot2); totals sum weighted scores across all 43 query slots.
Conclusion
KalDB’s total weighted score was 25.9% lower than OpenSearch 2.11.1, with a KalDB/OpenSearch 2.11.1 score ratio of 0.741. KalDB was faster by weighted score on 24 of 43 query slots, while OpenSearch 2.11.1 was faster on 19.
The biggest gains come from the larger optimized queries, which drive the overall score. These results are preliminary. In the coming days, we will publish final results, the reproducible benchmark harness, and the steps needed to reproduce the run. We will also keep optimizing min/max-style queries, simple aggregations, grouped distinct counts, and selective lookup paths.