Skip to content
TxPipe Docs

Elasticsearch

Indexes each event as a json document into an Elasticsearch index or data stream. Reach for it when you want chain data searchable and ready for Kibana dashboards.

Configuration

daemon.toml
[sink]
type = "ElasticSearch"
url = "https://localhost:9200"
index = "oura.sink.v0.mainnet"


[sink.credentials]
type = "Basic"
username = "oura123"
password = "my very secret stuff"
  • type (required): the literal value ElasticSearch.
  • url (required): the Elasticsearch API endpoint.
  • index (required): the index (or data stream) to store documents in.
  • idempotency (optional): when enabled, forces idempotent writes to avoid duplicates (see below).

Section: sink.credentials

Configures authentication against Elasticsearch. Omit the whole section to skip auth (e.g. a private cluster without authentication).

  • type (required): the auth mechanism. Only Basic is currently implemented.
  • username (required): the Elasticsearch user.
  • password (required): that user’s password.

Idempotency

An idempotent write can run multiple times with the same effect as running it once. With idempotency enabled, the sink derives each document’s ID deterministically from the event, so a block that gets reprocessed — for example after a restart without a cursor — produces the same IDs and Elasticsearch rejects the duplicates. Oura then continues with the next event. This guarantees the index won’t accumulate duplicate data.

If your pipeline can’t guarantee exactly-once delivery (no cursor, at-least-once source, etc.), enable idempotency. With it disabled, each document gets a random ID and is always indexed — so reprocessing a block creates duplicates.