Replacing OpenSearch on Edge Devices

Our Context & Requirements

We use OpenSearch deployed on an embedded edge device. Its primary use cases are:

1. Log ingestion and reporting: Edge devices ship logs to OpenSearch locally, which then off-boards to the cloud back-office. This is the main workload.
2. Remote debugging: Engineers remotely access the edge device and query logs through OpenSearch's UI and API to diagnose issues.
3. Edge analytics (minor): Light aggregation and analytics on log data — not a heavy analytics workload, just ad-hoc queries.

The critical constraint: we are on an embedded device with limited CPU and memory. Every MB of RAM and percent of CPU matters. OpenSearch — being Java-based — is a heavy lift in this context.

Non-Negotiables

• Low memory footprint — ideally under 100MB total (JVM can't do this)
• Low CPU overhead — idle should be near-zero
• Full-text log search — text matching, filtering, keyword search
• API access — remote dashboards and tools query through an API
• Simple local hosting — single binary or lightweight standalone service, no cluster dependency
• Retention management — ability to set TTL on logs

Nice-to-Haves

• Simple UI for remote debugging sessions
• Light aggregation/aggregation capabilities
• REST API compatible with OpenSearch/Elasticsearch query DSL (migration ease)
• Ability to ship logs to cloud back-office from this engine

Why OpenSearch is a Poor Fit

OpenSearch is the open-source fork of Elasticsearch. It's a powerful, distributed, JVM-based search and analytics platform. For our edge device context, it has several fundamental mismatches:

• JVM overhead: OpenSearch runs on the JVM, which requires 500MB–2GB of initial heap allocation. This is not configurable away — it's how the JVM works, and you want this headroom for garbage collection.
• Distributed architecture DNA: OpenSearch is built for clusters with multiple shards, replicas, and coordinating nodes. Our use case is a single node on a single device — but OpenSearch doesn't know that and provisions accordingly.
• Memory-hungry for basic use: Even a minimal OpenSearch installation with a single node consumes 1–3GB RAM under light load. Edge devices rarely have that much available.
• Slow startup: JVM cold starts take 30–90 seconds — unacceptable if any edge device reboot or health check requires a quick service availability.
• Operational complexity: JVM tuning, heap sizing, GC tuning, and JVM monitoring add operational overhead that doesn't make sense for a single-node edge deployment.

Replacement Candidates

Here are the leading alternatives, evaluated for our specific edge device context:

1. ZincSearch

Go <100MB RAM Elasticsearch API compatible

ZincSearch is a lightweight alternative to Elasticsearch written in Go. It was built specifically to address the JVM overhead problem that Elasticsearch/OpenSearch carry.

Strengths

• Single binary Go application — no JVM, no dependencies, fast startup (under 3 seconds)
• Runs on less than 100 MB of RAM — 20–30x lighter than OpenSearch, which is transformative for edge devices
• Elasticsearch/OpenSearch API-compatible — supports the same query DSL, making migration straightforward. Existing query code may work with zero changes.
• Embedded mode available — can run as a Go library embedded directly in your application, removing even the network service overhead
• Built-in web UI — simple search UI for remote debugging sessions
• Built-in authentication — security built in, not bolted on
• Full-featured FTS — facets, ranked search, boolean operators, stemming, synonyms, phrase search
• OpenObserve (by the same team) — offers a more complete observability platform (logs, metrics, traces) if you need that later

Weaknesses

• Smaller community — 17.8k GitHub stars vs OpenSearch's 18k+ and Elastic's 37k+. Fewer tutorials and examples online compared to the Elastic ecosystem, but active development with 553+ commits.
• Embedded mode is less documented — running as a library inside your application works but is not the primary use case they market
• Aggregation support is limited — it handles basic aggregations but not the deep analytics queries OpenSearch supports (which is fine for our light edge analytics use case)
• Single-node only — it does not support clustering (though our use case doesn't need it)
• Active development but early — the project is maintained and growing, but fewer months of real-world usage than mature incumbents

2. Meilisearch

Rust Fast search Good UX

Meilisearch is an ultra-fast, typo-tolerant full-text search engine written in Rust (the language we're evaluating for a custom build). It's designed for search-in-application use cases — product search, site search, internal tools — rather than log aggregation or observability.

Strengths

• Written in Rust — compiled to a single binary, no runtime dependencies. Zero JVM overhead.
• Extremely fast search — one of the fastest full-text search engines available
• Typo-tolerant search — excellent for user-facing search, good for log keyword matching where typos appear
• Simple to deploy — single binary, single config file, zero dependencies
• REST API with SDKs — HTTP API + SDKs for many languages
• Low memory footprint — typically 50–150MB depending on index size (comparable to Zinc); 57.7k GitHub stars, active daily with 13,900+ commits
• Excellent developer experience — great docs, good UI, active community

Weaknesses

• Not designed for logs — it's built for search over structured documents (products, articles, etc.), not for streaming log ingestion or time-series data. No native time-field optimization.
• No built-in retention/TTL management — logs need automatic expiration. Meilisearch doesn't have this natively.
• Not Elasticsearch-compatible — API is completely different from OpenSearch/Elasticsearch's. Migration would require rewriting all query logic.
• No native aggregation — no bucket aggregations, no aggregations API. For our light analytics this may not be a blocker, but it limits ad-hoc analysis capabilities.
• Not designed for high ingest volume — built for indexing documents, not for continuous log streaming ingestion at scale.

3. SQLite with FTS5

Zero additional install ~0MB extra RAM

SQLite is the most-deployed database in the world, embedded in virtually every computing device. SQLite's FTS5 (Full-Text Search 5) extension provides full-text indexing and searching capability. Since SQLite is already installed on virtually every Linux/Unix system as a dependency of other tools, FTS5 adds near-zero additional footprint.

Strengths

• Zero additional installs — SQLite is everywhere, often already included. FTS5 is built into the library.
• Negligible memory footprint — no separate service to run, no daemon. SQLite operates within the process that queries it.
• Semantic SQL queries — can combine full-text search with structured queries (WHERE clauses, joins) — ideal for log filtering by time, severity, source, etc. plus text search
• Mature, battle-tested — decades of real-world usage, excellent stability, no project risk
• Single file — the entire database is a single file on disk. Easy backups, easy replication.
• Rich ecosystem — language bindings for every language, ORMs, migration tools, admin UIs
• FTS5 is fast — BM25 ranking, phrase searching, proximity searches, highlighting

Weaknesses

• No native UI — you'd need to build or integrate a web UI for remote debugging sessions
• No native time-series optimization — FTS5 doesn't have native time-field features. You'd need to structure your schema with timestamp columns and query them separately.
• No real-time streaming ingest — SQLite is a file-based database, not a log ingestion engine. High-volume streaming would need a wrapper/ingestor service built on top.
• No native retention/TTL — would need a scheduled cleanup job to expire old logs
• No Elasticsearch API compatibility — complete API rewrite for migration.
• Concurrency limits — SQLite uses file locking for writes. High concurrency write loads are limited (for log ingestion this may be acceptable if ingestion is sequential, not highly concurrent).

4. OpenObserve

Go + Rust Full observability

OpenObserve (by the ZincSearch team) is a full observability platform — logs, metrics, traces, alerts — built as a more feature-complete but still lightweight alternative to ELK stack.

Strengths

• Purpose-built for observability — this is its actual domain, unlike Meilisearch or SQLite which need adaptation
• Lightweight — significantly lighter than OpenSearch, though heavier than ZincSearch
• Complete feature set — logs, metrics, dashboards, alerts, retention policies, multi-tenancy
• Elasticsearch-like API — supports OpenSearch-compatible API for query, search, and index management
• Built by the ZincSearch team — same engineering team, related ecosystem
• Native time-series support — first-class time-based queries and aggregation

Weaknesses

• Heavier than ZincSearch — a full observability platform naturally uses more resources than a bare search engine. Expect 200–500MB RAM under normal operation.
• More complex to deploy — more moving parts, more config, longer startup time
• May still be too heavy for very constrained edge devices — if RAM is under 1GB available, this could be a tight fit

5. Fluent Bit + Loki (Off-Board Strategy)

Rather than a local index, this approach moves the storage requirement entirely off-device: Fluent Bit (a lightweight log shipper) collects logs from the edge and streams them to a local Grafana Loki instance, or directly to cloud-based Loki/OpenSearch in the back office. Queries happen remotely, not locally.

Strengths

• Minimal edge footprint — Fluent Bit uses ~5–15MB RAM at runtime (~450KB binary, per fluentbit.io. The edge device becomes a collector, not a storage engine, removing all the indexing/search burden.
• Loki is lightweight — only indexes metadata (labels like severity, service, hostname), not full text. Stores raw log lines compressed in objects.
• Scales naturally — centralize everything in the cloud back-office where resources aren't constrained.
• Built for log pipelines — this is exactly what these tools were designed for.

Weaknesses

• No local search — if the network to the cloud is slow or unavailable, remote debugging is impossible. This matters for field engineers.
• Network dependency — no local UI, no local API, no local analysis. Everything requires connectivity.
• Not a drop-in replacement — would require architectural changes to our remote debugging workflow.

6. Build a Rust Replacement from Scratch

Rust Minimal footprint Maximum control

Instead of adopting an existing search engine, build a purpose-built log storage and search engine in Rust tailored to our exact constraints. This is the option we should seriously consider given our interest in Rust.

Why Consider This

• Built for our exact use case — no adapting a general-purpose tool. The design decisions match our requirements: log streaming, time-series awareness, retention/TTL, lightweight indexing, API access.
• Rust is ideal for this — Rust provides memory safety without GC overhead, near-C performance, compiled to a single binary with zero dependencies. No JVM, no Python runtime, no Go runtime. Just the binary.
• Zero runtime overhead — a bare-minimum Rust binary can run on 10–30MB RAM for basic indexing and search. As the index grows, memory usage scales linearly and predictably.
• Full control over data model — design the schema for log data specifically: timestamp indexing, severity fields, structured JSON log parsing, source/host tagging.
• Performance predictability — no garbage collection pauses, deterministic memory allocation, no tuning needed.
• Long-term strategic value — if Rust is part of our technology stack moving forward, building this in Rust gives us confidence and experience in the language through a concrete, valuable project.

Why It's Complex

• It's non-trivial engineering — building FTS with BM25 ranking, inverted indexes, time-range queries, and retention management is 6–12 months of focused development for a small team, not a weekend project.
• No existing UI — you'd build a web UI from scratch.
• Testing and reliability — you'd be maintaining your own infrastructure, including all edge-case handling, indexing bugs, and performance regression testing.
• Opportunity cost — six months of engineering time on this is six months not spent on core product work.

Suggested Rust Stack

// Core stack for a Rust log search engine
// ================================

// Storage - choose one based on scale
let // Option A: SQLite via rusqlite + rusqlite-fts5 extension
// Option B: LightSQL/lightstore — purpose-built log storage
// Option C: RocksDB (via rocksdb crate) — embedded KV store

// Indexing
let // tantivy — Rust full-text search engine, BM25 scoring, inverted indexes
// Tantivy is to Rust what Lucene is to Java — but much lighter

// Async runtime
let // tokio — the standard async runtime in Rust

// HTTP API
let // axum or actix — both are lightweight, production-grade web frameworks

// Structured log parsing
let // serde + serde_json — JSON parsing and serialization

// Date/time handling
let // chrono — Rust's standard date/time library

Recommended Approach: Tantivy as Index Engine

Tantivy is the closest Rust equivalent to Lucene (which Elasticsearch/OpenSearch use under the hood). It provides:

• Built-in BM25 ranking algorithm
• Inverted indexes for fast text search
• Numeric, date, and faceted field types — perfect for log timestamps and severity
• Single-binary deployment, no external dependencies
• 0-index overhead (index lives in files on disk, loaded lazily)

A Rust/Tantivy-based logger could be started as:

// Example concept — not actual code, just the idea
use tantivy::Index;
use chrono::prelude::*;

// Index structure for logs:
//   - timestamp: Date field (for time-range queries)
//   - severity:   Keyword field (INFO, WARN, ERROR, etc.)
//   - source:     Text field (service name, hostname tags)
//   - message:    Text field (full log message for FTS)

// Retention: scheduled cleanup job based on timestamp field
// API: axum endpoints matching OpenSearch query DSL for easy migration

Comparative Analysis

Criterion	ZincSearch	Meilisearch	SQLite FTS5	OpenObserve	Fluent + Loki	Rust/Tantivy
Memory footprint	<100MB	50–150MB	~0MB (in-process)	200–500MB	~5–10MB	50–100MB
Deployment complexity	Single binary	Single binary	In-process library	Complex	2+ components	Custom build — weeks/months
ES API compatible	Yes	No	No	Partial	No	Can match
Built-in UI	Yes	Yes	No	Yes	Yes (Loki)	No — must build
Native time-series	Basic	No	Custom fields	Yes	Yes	Custom fields
Retention/TTL	Yes	Partial	Manual needed	Yes	Yes	Must be built
Migration effort from OpenSearch	Low (API compatible)	High (new API)	High (new API)	Medium	Architectural change	Medium
Edge device suitability	⭐ Excellent	Good, but mismatched	⭐ Excellent	Marginal for tight RAM	Excellent (collector only)	⭐ Excellent
Time to implement	Days	Days	Days–weeks	Weeks	Weeks	Months

Recommendation

Based on our edge device constraints and use case, here's my recommendation in order:

Phase 1: Immediate — ZincSearch (Recommended Now)

• Why: Closest drop-in replacement. Elasticsearch-compatible API means migration is minimal. <100MB RAM is a 20–30x reduction from OpenSearch. Single binary for simple deployment. Built-in UI for remote debugging.
• Timeline: 1–2 weeks including testing and validation.
• Risk: Low. It's a direct feature match for 95% of our OpenSearch usage.

Phase 2: Hybrid — Local index + Cloud shipping (Recommended)

• Why: Combine ZigSearch locally (for fast on-device search) with Fluent Bit shipping logs to cloud Loki for remote back-office access. This gives you lightweight local search plus cloud-scale remote debugging.
• Timeline: 2–3 weeks after Phase 1 is stable.

Phase 3: Strategic — Rust/Tantivy Custom Build (Long-term)

• Why: If Rust is becoming a strategic language for our infrastructure, building a purpose-built log search engine in Tantivy would be the most elegant long-term solution. But defer until after Phase 1 is stable.
• Timeline: 6–12 months. Not an immediate priority.
• Approach: Use Tantivy for indexing/search + tokio for async I/O + axum for HTTP API + chrono for time-series queries. Design as a single binary with an optional embedded web UI.

Next Steps

1. Spike ZincSearch on the edge device — install the binary, load a week's worth of logs, measure RAM/CPU, validate query performance against current OpenSearch.
2. Benchmark search response times — compare latency and throughput for key queries (log search by keyword, by time range, by severity filter).
3. Validate retention policies — test ZincSearch's TTL and compare against our current log retention practices.
4. Prototype the Rust/Tantivy approach — if Phase 1 goes well, build a minimal PoC: ingest logs, index them with Tantivy, serve a simple search API. This will demonstrate feasibility and inform the long-term architecture decision.

Fact Check Report

Sources & References

• ZincSearch — lightweight Elasticsearch alternative (redirects to openobserve.ai, same team)
• ZincSearch on GitHub — 17.8k stars, 553 commits, Go
• Meilisearch Typo Tolerance Docs — official documentation
• Meilisearch on GitHub — 57.7k stars, 13,941 commits, Rust
• Tantivy Rust Docs — "Lucene, but in Rust"
• Tantivy Architecture Overview — docs.rs architecture documentation
• Fluent Bit — CNCF graduated project, >15B deployments, v5.0.5 (May 2026), 7,865 stars
• Grafana Loki Docs — metadata-only indexing, compressed raw storage in S3/object stores
• OpenObserve — $10M Series A announced, 18,904 stars, Go + Rust
• SQLite FTS5 Documentation — full-text search with BM25 ranking