From 038be889e2cb58e8a80242cde466dc4f513fb61a Mon Sep 17 00:00:00 2001 From: "Han Verstraete (OpenFaaS Ltd)" Date: Mon, 22 Jun 2026 18:29:33 +0200 Subject: [PATCH] Document source label on gateway_function_invocation_inflight Add the new source label and its values to the inflight metric, correct the metric names for the started and inflight gauges, add a migration note for the increased cardinality, and clarify the difference between gateway-side and worker-side inflight metrics. Signed-off-by: Han Verstraete (OpenFaaS Ltd) --- docs/architecture/metrics.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/architecture/metrics.md b/docs/architecture/metrics.md index db9ebb4c..0386fc7f 100644 --- a/docs/architecture/metrics.md +++ b/docs/architecture/metrics.md @@ -42,8 +42,8 @@ Advanced metrics for OpenFaaS Pro users: | Metric | Type | Description | Labels | Edition | | ----------------------------------- | ---------- | ----------------------------------- | -------------------------- |--------------------| -| `gateway_invocation_function_started` | counter | Invocations started, including async | `function_name` | Pro Edition | -| `gateway_invocation_function_invocation_inflight` | gauge | Total connections inflight for function invocations | `function_name` | Pro Edition | +| `gateway_function_invocation_started` | counter | Invocations started, including async | `function_name` | Pro Edition | +| `gateway_function_invocation_inflight` | gauge | Total connections inflight for function invocations | `function_name`, [`source`](#the-source-label) | Pro Edition | | `gateway_service_ready_count` | gauge | Number of function replicas which are in a ready state | `function_name` | Pro Edition | | `gateway_service_target` | gauge | Target load for the function | `function_name` | Pro Edition | | `gateway_service_min` | gauge | Min number of function replicas | `function_name` | Pro Edition | @@ -54,6 +54,23 @@ Advanced metrics for OpenFaaS Pro users: The `http_request*` metrics record the latency and statistics of `/system/*` routes to monitor the OpenFaaS gateway and its provider. The `/async-function` route is also recorded in these metrics to observe asynchronous ingestion rate and latency. +### The `source` label + +The `source` label identifies which OpenFaaS component initiated an invocation, so that inflight requests can be broken down by traffic source (for example, when writing [custom autoscaling rules](/architecture/autoscaling/#custom-autoscaling-rules)). + +Currently the only source that is identified is the async queue-worker, labelled `queue-worker`. All other traffic, including direct synchronous calls, is recorded as `unknown`. + +| Source | Description | +| -------------- | ------------------------------------ | +| `queue-worker` | Asynchronous invocation dispatched by the NATS queue-worker | +| `unknown` | Any other traffic that is not attributed to a known source, such as a direct synchronous call | + +!!! info "Migration note when upgrading to gateway 0.5.22 or later" + The `source` label was added to `gateway_function_invocation_inflight` in gateway version `0.5.22`, which increases the metric's cardinality. Existing queries, dashboard panels and alerts that select this metric without aggregating away the new label will now return one series per source where they previously returned a single series per function. To keep the previous behaviour, aggregate the `source` label away, for example with `sum by (function_name) (gateway_function_invocation_inflight)`. + +!!! note "Gateway and queue-worker inflight counts can differ" + `gateway_function_invocation_inflight{source="queue-worker"}` is measured by the gateway, while `queue_worker_function_invocation_inflight` (see [JetStream for OpenFaaS](#jetstream-for-openfaas)) is measured by the queue-worker itself. The two count the same traffic at different points and will not always match exactly. + Additional metrics from the Operator: | Metric | Type | Description | Labels | Edition |