diff --git a/content/en/docs/next/networking/cloudflare-tunnel.md b/content/en/docs/next/networking/cloudflare-tunnel.md new file mode 100644 index 00000000..5f692f44 --- /dev/null +++ b/content/en/docs/next/networking/cloudflare-tunnel.md @@ -0,0 +1,227 @@ +--- +title: "Publishing through a Cloudflare Tunnel" +linkTitle: "Cloudflare Tunnel" +description: "Optional system package registering a second Gateway API implementation whose Gateways are published by Cloudflare over an outbound tunnel, for clusters with no routable inbound address." +weight: 16 +--- + +## What this page covers + +What the `cloudflare-tunnel-gateway-controller` package is and when it is the right tool, what has to exist on the Cloudflare side before enabling it, the credentials Secret, the two-step enable, how a Gateway joins the class, the limitations that should decide whether to enable it at all, and the removal path. + +## What it is + +`cloudflare-tunnel-gateway-controller` is an optional system package that registers a second Gateway API implementation alongside the Cilium one. A Gateway on its class is published by Cloudflare: clients reach Cloudflare's edge, and the edge carries the request into the cluster over a Cloudflare Tunnel that the in-cluster data plane opens outbound. The Services the chart renders are `ClusterIP` — the two proxy Services unconditionally, the controller's own by a `service.type` value that defaults to it — so nothing on this path asks the cluster for an address reachable from outside. + +GatewayClasses are cluster-scoped and each Gateway names the one it wants, so enabling this package does not move anything already published through Cilium. + +## When to reach for it + +The [Gateway a tenant publishes through]({{% ref "/docs/next/networking/gateway-api" %}}) is backed by Cilium unless the tenant is moved onto another class, and that path wants two things from the outside world — an address clients can open a connection to, and an ACME challenge that completes against the cluster. The tunnel class drops both for the hostnames it serves: inbound connections are replaced by an outbound connection from the proxy pods to Cloudflare's edge, and TLS is terminated at the edge rather than on a Gateway listener. + +It fits a cluster behind NAT or CGNAT, a lab or home cluster with no routable prefix, a site whose firewall will not forward ports, or a deployment that wants Cloudflare's edge in front of a subset of hostnames. + +It is the wrong tool when you need TLS passthrough or any non-HTTP protocol, when the certificate a client sees has to be one the cluster controls, or when a third party in the request path is unacceptable. + +## What has to exist on the Cloudflare side + +1. A zone whose DNS is served by Cloudflare, covering the hostnames you intend to publish. +2. A Tunnel, created under **Zero Trust → Networks → Tunnels** with the `cloudflared` connector type. Keep both its **Tunnel ID** and its **tunnel token**. +3. An API token with the **Account → Cloudflare Tunnel → Edit** permission. + +{{% alert title="Important" color="warning" %}} + +Give the controller a tunnel of its own. Its upstream documentation states that it assumes exclusive ownership of the tunnel configuration, performs a full synchronization on startup, and removes ingress rules that do not come from routes it manages — so a tunnel that also carries hand-written public hostnames, or one shared with another system, loses them. + +{{% /alert %}} + +The Cloudflare account ID is auto-detected when the API token has access to a single account. When it does not, supply it explicitly, either as an `account-id` key in the Secret below or through the chart's `gatewayClassConfig.accountId` value. + +## The credentials Secret + +Both planes read one Secret, named `cloudflare-tunnel-credentials`, in the package namespace `cozy-cloudflare-tunnel-gateway-controller`: + +| Key | Read by | Contents | +| --- | --- | --- | +| `api-token` | the controller | the Cloudflare API token | +| `tunnel-token` | the proxy, as the `TUNNEL_TOKEN` environment variable | the tunnel's connector token | +| `account-id` | the controller, optional | account ID, when auto-detection cannot pick one | + +This is a different credential from the Cloudflare API token used by the DNS-01 ACME solver (`publishing.certificates.dns01.cloudflare.secretName`, default `cloudflare-api-token-secret`) — that one needs zone DNS permissions, this one needs tunnel permissions. Do not assume one Secret can serve both. + +Create it before or shortly after enabling the package, because the proxy pod cannot start without it: + +```bash +kubectl create namespace cozy-cloudflare-tunnel-gateway-controller \ + --dry-run=client --output yaml | kubectl apply --filename - + +kubectl --namespace cozy-cloudflare-tunnel-gateway-controller \ + create secret generic cloudflare-tunnel-credentials \ + --from-literal=api-token="$CF_API_TOKEN" \ + --from-literal=tunnel-token="$CF_TUNNEL_TOKEN" +``` + +## Enabling the package + +The package is in no bundle by default, because it cannot become Ready without operator input. Enabling it takes two edits, and the second lands on an object that exists only after the first. + +First, list it in `bundles.enabledPackages` on the platform values: + +```yaml +apiVersion: cozystack.io/v1alpha1 +kind: Package +metadata: + name: cozystack.cozystack-platform +spec: + components: + platform: + values: + bundles: + enabledPackages: + - cozystack.cloudflare-tunnel-gateway-controller +``` + +The name also has to stay out of `bundles.disabledPackages`. The two lists are checked in different places — the bundle tests `enabledPackages` before it calls the helper that renders the Package, and that helper then vetoes on `disabledPackages` — so a name present in both is not emitted, whichever order you think of them in. + +That render creates a `Package` named `cozystack.cloudflare-tunnel-gateway-controller`. Second, set the tunnel ID on it: + +```yaml +apiVersion: cozystack.io/v1alpha1 +kind: Package +metadata: + name: cozystack.cloudflare-tunnel-gateway-controller +spec: + components: + cloudflare-tunnel-gateway-controller: + values: + cloudflare-tunnel-gateway-controller: + gatewayClassConfig: + tunnelID: "00000000-0000-4000-8000-000000000000" +``` + +Until that value is set the chart refuses to render and the HelmRelease reports `gatewayClassConfig.tunnelID is required`. Because the Package appears only once the name is in `enabledPackages`, the first reconcile after enabling always fails this way; setting the tunnel ID changes the values and the install is retried. While that window is open the cluster carries a HelmRelease that is not Ready. Nothing in the platform gates on that, but the platform does ship the instrument most likely to notice: `check-readiness` counts `helmreleases.helm.toolkit.fluxcd.io` among the resources whose `Ready` condition it requires, so it reports the cluster as not ready until the tunnel ID is in place, and `check-readiness --wait` blocks until everything is ready or exits non-zero at its timeout (30 minutes by default). Have the tunnel ID ready before enabling if anything in your automation waits on that. + +The platform writes one key of its own inside that same component values block, `controller.clusterDomain`, taken from `networking.clusterDomain`, because the vendored chart bakes the cluster domain into the proxy's config-endpoint URL and falls back to `cluster.local` when it is empty — wrong on a Cozystack default of `cozy.local`. + +Editing a Package the platform renders sounds like it should be undone by the next platform reconcile, and it is not. The tunnel ID is a sibling key of the one the platform writes, and the two have different owners: helm-controller applies Packages server-side, where field ownership is tracked per field and the platform owns only what it renders, and a release still on client-side apply patches only its own rendered fields for the same result. A platform values edit or version bump therefore leaves `gatewayClassConfig.tunnelID` alone. There is no platform-values path for it — the Package is where it lives, which is why enabling takes two edits rather than one. + +## The GatewayClass, and attaching a Gateway + +The package creates one `GatewayClass` named `cloudflare-tunnel`, with `spec.controllerName: cf.k8s.lex.la/tunnel-controller` and a `parametersRef` to the cluster-scoped `GatewayClassConfig` where the tunnel ID and the credentials reference land. The class name is cosmetic **to the controller**, which binds its GatewayClasses by `controllerName` — but not to Cozystack: `gateway.className`, `gateway.tenantSelectableClasses` and `gateway.edgeTerminatedClasses` all match on the name, trimmed but never case-folded. Renaming the class means updating those lists, and how that failure surfaces depends on which list is stale — a `className` no controller claims leaves the Gateway unprogrammed, a dropped `tenantSelectableClasses` entry fails the tenant's release with a message naming the class, and only a stale `edgeTerminatedClasses` entry is genuinely silent. See [GatewayClass names that match no installed class]({{% ref "/docs/next/networking/gateway-api#gatewayclass-names-that-match-no-installed-class" %}}). + +A Gateway joins the class by naming it, and routes attach to that Gateway the usual way: + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: tunnel + namespace: tenant-example +spec: + gatewayClassName: cloudflare-tunnel + listeners: + - name: https + protocol: HTTPS + port: 443 +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: app + namespace: tenant-example +spec: + parentRefs: + - name: tunnel + hostnames: + - app.example.com + rules: + - backendRefs: + - name: app + port: 80 +``` + +The listener also declares no `hostname`, which is deliberate: the Gateway serves whatever its routes claim, and the platform's Gateway-listener admission policy allows a hostname-less listener rather than requiring one. Give it a hostname and that hostname must fall within the namespace's `namespace.cozystack.io/host` apex. + +The listener is `protocol: HTTPS` and carries no `tls` block, which is deliberate and is the shape the chart's own installation notes prescribe: TLS ends at Cloudflare's edge, so there is no certificate for this Gateway to present. Gateway API permits it — `tls` is not a required listener field, and the CRD's only rule tying the two together constrains `tls.mode` for an HTTPS listener that *has* a `tls` block, so one without it is admitted. What the listener declares here is which routes bind to it — the port and protocol are route-binding inputs on this class, not a statement about what Cloudflare's edge serves. That is why the port-80 `HTTP` shape an edge-terminated `TenantGateway` renders works on this class too. + +Routes are `HTTPRoute` and `GRPCRoute`. A backend is normally a Service, and a Service of type `ExternalName` works for an out-of-cluster origin — the proxy infers the scheme from the backendRef port. The chart also installs an `ExternalBackend` CRD for the cases that shape cannot cover: it makes the scheme explicit rather than inferred, lets the host be an address that is not a valid Service name, and can prepend a base path to the request. + +The controller writes the tunnel's CNAME target, `.cfargotunnel.com`, into the Gateway's `status.addresses`. Turning that into a DNS record needs an `external-dns` that watches Gateway API, which is not how Cozystack ships either of them: the system package `cozystack.external-dns` runs the upstream chart's default sources, `service` and `ingress`, and surfaces no value of its own for adding more — a Package's component values do pass through untyped, so `sources` can still be set that way, but it is undocumented rather than supported. The per-tenant `external-dns` application does have the switch — `gatewayAPI: true` adds the `gateway-httproute` and `gateway-tlsroute` sources — and it is `false` by default. Neither path adds `gateway-grpcroute`, so a hostname published only by a `GRPCRoute` gets no record from either. Failing all that, create the CNAME in the Cloudflare zone yourself, proxied. + +## Which Gateway to attach + +On its default settings the per-tenant Gateway that `tenant.spec.gateway: true` renders is not a drop-in for this class. Its `tlsPassthroughServices` become `protocol: TLS`, `mode: Passthrough` listeners, a shape this controller has nothing to map onto — and a listener it does not accept holds the whole object at `Ready=False`, because cozystack-controller marks a `TenantGateway` Ready only when every listener reports both `Accepted` and `Programmed`. The remaining port-443 listeners are rendered `mode: Terminate` with a `certificateRefs` entry — cert-manager's under the ACME modes, the operator's Secret under `existingSecret` — which either way is the shape this class does not use, because the edge already terminates. + +Putting the class into `gateway.edgeTerminatedClasses` resolves all of that. A tenant Gateway on an edge-terminated class renders port-80 `HTTP` listeners only, with no `tls` block, no `certificateRefs` and no passthrough listeners — see [`edge` cert mode]({{% ref "/docs/next/networking/gateway-api#edge-tls-terminated-by-the-class-provider" %}}) for the full shape and for the platform values that select it. That is a Gateway this controller can serve, so the combination works; the standalone Gateway shown above is simply the path that needs no platform configuration. + +Two caveats survive the switch. Edge listeners pin `allowedRoutes.kinds` to `HTTPRoute` alone, so a `GRPCRoute` cannot attach to an edge `TenantGateway` — a route of that kind needs the standalone Gateway. And selecting the class through `gateway.className` rather than per-tenant `tenant.spec.gatewayClass` puts the publishing tenant on it too. Where the platform's TLS-passthrough endpoints are published at all — each renders its `TLSRoute` only when `gateway.enabled` is on and its name is in `publishing.exposedServices` — that unpublishes them, with nothing on the Gateway or the `TenantGateway` reporting it; put child tenants on the class instead. + +Two platform values are needed for that, not one. `gateway.edgeTerminatedClasses` is what selects the certificate behaviour, and `gateway.tenantSelectableClasses` is what lets a tenant name the class at all — a tenant may only name the current `gateway.className` or something on that list, and anything else fails that tenant's own gateway release at render time. See [picking a GatewayClass]({{% ref "/docs/next/networking/gateway-api#picking-a-gatewayclass" %}}). Bear in mind that a child tenant's apex is one label below its parent's, so its application hostnames sit two labels below the zone — past what Cloudflare's Universal SSL covers, per the certificate limitation below. + +## Hostname ownership between tenants + +By default every Gateway of this class shares one tunnel and one proxy pool, so the data plane picks a backend by hostname across all of them — a route claiming another tenant's hostname would be answered rather than ignored, which is not the case behind a per-tenant Cilium address. + +Cozystack's own hostname policies do not close that on its own: the pair — `cozystack-route-hostname-policy` for `HTTPRoute` and `cozystack-route-hostname-policy-tls` for `TLSRoute` — binds route hostnames to the namespace's `namespace.cozystack.io/host` label, but between them they cover only those two kinds in namespaces whose name starts with `tenant-`, neither sees `GRPCRoute`, and both admit a route that declares no hostnames at all. + +So the package turns on the chart's own hostname-ownership layer, keyed on the same `namespace.cozystack.io/host` label and scoped to every namespace that carries it. On this class, a route in such a namespace is rejected by the controller unless it declares hostnames explicitly and each one equals the label value or is a subdomain of it. A leading `*.` is stripped before that comparison, so a wildcard hostname is allowed on the same terms — `*.