Skip to content

Pipeline Metrics

The [metrics] feature lets operators track the progress and performance of long-running Oura sessions over a Prometheus endpoint.

Context

Some use-cases keep Oura running continuously or for long stretches: keeping a sink updated in real time means 24/7 operation, and dumping historical chain data from origin can take hours.

These scenarios call for visibility into the pipeline’s internal state to make monitoring and troubleshooting possible. Two aspects matter most: progress and performance.

Feature

Oura provides an optional /metrics HTTP endpoint that exposes real-time operational metrics in Prometheus format. Each stage (source / sink) reports its progress as it processes events; these notifications are aggregated into counters and gauges and served over HTTP using the well-known Prometheus encoding.

The following metrics are available:

  • chain_tip: the last detected tip of the chain (height)
  • rollback_count: number of rollback events occurred
  • source_current_slot: last slot processed by the source of the pipeline
  • source_current_height: last height (block #) processed by the source of the pipeline
  • source_event_count: number of events processed by the source of the pipeline
  • sink_current_slot: last slot processed by the sink of the pipeline
  • sink_event_count: number of events processed by the sink of the pipeline

Configuration

The metrics feature is a configurable setting available when running in daemon mode. A top level [metrics] section of the daemon toml file controls the feature:

daemon.toml
[metrics]
address = "0.0.0.0:9186"
  • [metrics] section needs to be present to enable the feature. Absence of the section will not expose any HTTP endpoints.
  • address (optional): The address at which the HTTP server will be listening for request. Expected format is <ip>:<port>. Use the IP value 0.0.0.0 to allow connections on any of the available IP address of the network interface. Default value is 0.0.0.0:9186.

Usage

Once enabled, a quick method to check the metrics output is to navigate to the HTTP endpoint using any common browser. A local instance of Oura with metrics enabled on port 9186 can be accessed by opening the URL http://localhost:9186

An output similar to the following should be shown by the browser:

# HELP chain_tip the last detected tip of the chain (height)
# TYPE chain_tip gauge
chain_tip 6935733
# HELP rollback_count number of rollback events occurred
# TYPE rollback_count counter
rollback_count 1
# HELP sink_current_slot last slot processed by the sink of the pipeline
# TYPE sink_current_slot gauge
sink_current_slot 2839340
# HELP sink_event_count number of events processed by the sink of the pipeline
# TYPE sink_event_count counter
sink_event_count 2277714
# HELP source_current_height last height (block #) processed by the source of the pipeline
# TYPE source_current_height gauge
source_current_height 2837810
# HELP source_current_slot last slot processed by the source of the pipeline
# TYPE source_current_slot gauge
source_current_slot 2839340
# HELP source_event_count number of events processed by the source of the pipeline
# TYPE source_event_count counter
source_event_count 2277715

The browser check above is handy, but the intended approach is to point a Prometheus-compatible monitoring stack at the endpoint. Setting up and running that stack is outside Oura’s scope; if you don’t already have one, these are common choices:

  • Prometheus Server + Grafana
  • Metricbeat + Elasticsearch + Kibana
  • Telegraf + InfluxDB + Chronograf