Monitoring Edit

OpenTelemetry

When run as a server and configured accordingly, OPA will emit spans to an OpenTelemetry collector via gRPC.

Each REST API request sent to the server will start a span. If processing the request involves policy evaluation, and that in turn uses http.send, those HTTP clients will emit descendant spans.

Furthermore, spans exported for policy evaluation requests will contain an attribute opa.decision_id of the evaluation’s decision ID if the server has decision logging enabled.

See the configuration documentation for all OpenTelemetry-related configurables.

Prometheus

OPA exposes an HTTP endpoint that can be used to collect performance metrics for all API calls. The Prometheus endpoint is enabled by default when you run OPA as a server.

You can enable metric collection from OPA with the following prometheus.yml config:

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: "opa"
    metrics_path: "/metrics"
    static_configs:
    - targets:
      - "localhost:8181"

The Prometheus endpoint exports Go runtime metrics as well as HTTP request latency metrics for all handlers (e.g., v1/data).

Metric nameMetric typeDescription
go_gc_duration_secondssummaryA summary of the GC invocation durations.
go_goroutinesgaugeNumber of goroutines that currently exist.
go_infogaugeInformation about the Go environment.
go_memstats_alloc_bytesgaugeNumber of bytes allocated and still in use.
go_memstats_alloc_bytes_totalcounterTotal number of bytes allocated, even if freed.
go_memstats_buck_hash_sys_bytesgaugeNumber of bytes used by the profiling bucket hash table.
go_memstats_frees_totalcounterTotal number of frees.
go_memstats_gc_cpu_fractiongaugeThe fraction of this program’s available CPU time used by the GC since the program started.
go_memstats_gc_sys_bytesgaugeNumber of bytes used for garbage collection system metadata.
go_memstats_heap_alloc_bytesgaugeNumber of heap bytes allocated and still in use.
go_memstats_heap_idle_bytesgaugeNumber of heap bytes waiting to be used.
go_memstats_heap_inuse_bytesgaugeNumber of heap bytes that are in use.
go_memstats_heap_objectsgaugeNumber of allocated objects.
go_memstats_heap_released_bytesgaugeNumber of heap bytes released to OS.
go_memstats_heap_sys_bytesgaugeNumber of heap bytes obtained from system.
go_memstats_last_gc_time_secondsgaugeNumber of seconds since 1970 of last garbage collection.
go_memstats_lookups_totalcounterTotal number of pointer lookups.
go_memstats_mallocs_totalcounterTotal number of mallocs.
go_memstats_mcache_inuse_bytesgaugeNumber of bytes in use by mcache structures.
go_memstats_mcache_sys_bytesgaugeNumber of bytes used for mcache structures obtained from system.
go_memstats_mspan_inuse_bytesgaugeNumber of bytes in use by mspan structures.
go_memstats_mspan_sys_bytesgaugeNumber of bytes used for mspan structures obtained from system.
go_memstats_next_gc_bytesgaugeNumber of heap bytes when next garbage collection will take place.
go_memstats_other_sys_bytesgaugeNumber of bytes used for other system allocations.
go_memstats_stack_inuse_bytesgaugeNumber of bytes in use by the stack allocator.
go_memstats_stack_sys_bytesgaugeNumber of bytes obtained from system for stack allocator.
go_memstats_sys_bytesgaugeNumber of bytes obtained from system.
go_threadsgaugeNumber of OS threads created.
http_request_duration_secondshistogramA histogram of duration for requests.

Health Checks

OPA exposes a /health API endpoint that can be used to perform health checks. See Health API for details.

Status API

OPA provides a plugin which can push status to a remote service. See Status API for details.