KalDB replaces brittle managed OpenSearch setups with a low-cost, high-throughput log analytics platform built for S3. Real engineers report memory leaks, crashes, slow queries, and data loss.
Real issues reported by engineering teams running managed OpenSearch at scale
Nodes die, indexing stops, clusters need frequent restarts. Memory leak behavior persists across versions.
Expensive queries bring clusters to their knees. High-cardinality aggregations cause cascading failures.
Connection blips and unhealthy node states, especially on small clusters or poorly tuned deployments.
CVEs, misconfigured permissions, difficult-to-audit role-based access. Security adds operational burden.
Unable to recover data after outages. Backups and recovery are painful and often incomplete.
Teams spend weeks tuning shards, heap, and ingestion. Capacity planning is a full-time job.
Confusing error messages, lack of actionable diagnostics. Average GitHub issue resolution time: 79+ days.
From OpenSearch forums and community discussions
"After ~19,400 indexed documents, the process gets killed because it uses too much memory... increasing heap doesn't help."
"90% memory usage, 80% CPU spikes — our cluster was on the brink of collapse during peak hours."
"OpenSearch restarted and we lost our data. We had no way to recover weeks of logs."
"We spend more time managing OpenSearch than actually using the data. It's become a full-time job."
Why OpenSearch struggles with modern log workloads
JVM pointer compression causes practical heap limits. GC sensitivity causes catastrophic failures beyond 32GB.
Aggregations and heavy queries blow up memory/CPU without careful index design.
Shards, replication, disk growth, and upgrades add combinatorial complexity.
UI changes and minor version bumps risk behavior changes. Upgrades are risky.
Original architects have departed. Institutional knowledge is declining in the community.
Average GitHub issue resolution time: 79.6 days. Critical bugs linger for months.
How KalDB addresses each OpenSearch limitation
| OpenSearch Problem | KalDB Solution |
|---|---|
| Memory leaks & GC crashes | S3-backed storage; stateless compute nodes with no heap management |
| CPU spikes during aggregations | On-demand indexing; query compute scales independently |
| Cluster instability | Stateless architecture; no cluster state to synchronize |
| Security vulnerabilities | Simplified attack surface; data stays in your S3 bucket |
| Data loss & recovery issues | S3 provides 99.999999999% durability; automatic recovery |
| Operational complexity | No shards, no heap tuning, no capacity planning |
| Troubleshooting gaps | Simple architecture with clear failure modes |
Not a bolt-on. S3 is the foundation, not an afterthought. True cloud-native architecture.
Migrate in hours, not months. Your existing tools and dashboards just work.
Battle-tested at scale. Handling petabytes of logs for one of the world's largest collaboration platforms.
No license games, no usage restrictions. Fork it, modify it, deploy it anywhere.
Try KalDB open source today or talk to us about production deployment