Skip to content

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435

Open
potiuk wants to merge 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup
Open

Docs: refresh JWT and security model for v3.2 with mermaid diagrams#67435
potiuk wants to merge 3 commits into
apache:mainfrom
potiuk:docs/security-model-v3.2-catchup

Conversation

@potiuk
Copy link
Copy Markdown
Member

@potiuk potiuk commented May 24, 2026

Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
and re-aligns prose with the current code.

What changed in the docs

airflow-core/docs/security/jwt_token_authentication.rst:

airflow-core/docs/security/security_model.rst:

Tooling change

Registers sphinxcontrib-mermaid>=1.0.0 as a new dependency in
devel-common[docs] and adds sphinxcontrib.mermaid to
BASIC_SPHINX_EXTENSIONS so every Airflow Sphinx build (core,
providers, chart, docker-stack) can use .. mermaid:: directives.
uv.lock is regenerated.

Verification

  • breeze build-docs --package-filter apache-airflow
    "Documentation build is successful", 0 build errors, 0 spelling errors.
  • All six mermaid containers render in the produced HTML.
  • prek run --from-ref upstream/main --stage pre-commit and
    --stage manual both pass.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented May 24, 2026

@potiuk potiuk added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label May 24, 2026
@potiuk potiuk added this to the Airflow 3.2.2 milestone May 24, 2026
@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented May 24, 2026

Also cc: @vatsrahul1001 -> we should include it in rc2

@potiuk
Copy link
Copy Markdown
Member Author

potiuk commented May 24, 2026

Below are the six mermaid diagrams introduced in this PR (updated for higher contrast and an easier-to-read credential matrix), rendered inline via GitHub's native mermaid support. They are identical to what breeze build-docs produces in the published HTML.


jwt_token_authentication.rst — overview of components and flows

flowchart LR
    subgraph Clients
        UI[UI / browser]
        CLI[CLI]
        EXT[External REST clients]
    end
    subgraph Internal["Internal Airflow components"]
        WORKER[Worker / Task]
        DFP[Dag File Processor]
        TRG[Triggerer]
    end
    APISVR[API Server]
    EXECAPI[Execution API]
    UI -->|JWT cookie / Bearer| APISVR
    CLI -->|Bearer| APISVR
    EXT -->|Bearer| APISVR
    WORKER -->|Bearer<br/>workload &rarr; execution| EXECAPI
    DFP -. in-process<br/>JWT bypassed .-> EXECAPI
    TRG -. in-process<br/>JWT bypassed .-> EXECAPI

    classDef internal fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
    class WORKER,DFP,TRG internal
Loading

jwt_token_authentication.rst — symmetric vs asymmetric signing

flowchart TB
    subgraph Sym["Symmetric (HS512)"]
        direction LR
        S1[Scheduler / API Server]
        S2[Shared secret<br/>jwt_secret]
        S3[Token validator]
        S1 -->|sign| S2 -->|same secret<br/>also validates| S3
    end
    subgraph Asym["Asymmetric (RS256 / EdDSA)"]
        direction LR
        A1[Scheduler / API Server]
        A2[Private key<br/>jwt_private_key_path]
        A3[Public key /<br/>JWKS endpoint]
        A4[Token validator]
        A1 -->|sign| A2
        A2 -. derives or<br/>publishes .-> A3
        A3 -->|verify only| A4
    end

    classDef secret fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
    classDef pub fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
    class S2 secret
    class A2 secret
    class A3 pub
Loading

jwt_token_authentication.rst — two-token sequence (workload → execution)

sequenceDiagram
    autonumber
    participant SCH as Scheduler
    participant EXE as Executor<br/>(Celery / K8s / Local)
    participant WRK as Worker
    participant API as Execution API

    Note over SCH: Task ready to dispatch
    SCH->>SCH: generate workload token<br/>scope=workload<br/>exp = task_queued_timeout
    SCH->>EXE: workload JSON<br/>(includes token)
    Note over EXE: Task waits in queue<br/>(can be minutes)
    EXE->>WRK: dispatch (workload JSON)
    WRK->>API: POST /run<br/>Bearer: workload token
    Note over API: validates workload scope<br/>checks TI in QUEUED/RESTARTING<br/>409 if not
    API-->>WRK: 200 OK<br/>Refreshed-API-Token: execution token<br/>(scope=execution, ~10 min)
    WRK->>WRK: BearerAuth swaps to<br/>execution token
    loop For all subsequent calls (heartbeats, XComs, ...)
        WRK->>API: Bearer: execution token
        alt token expiring (less than 20% left)
            API-->>WRK: 200 OK<br/>Refreshed-API-Token: new execution token
            WRK->>WRK: BearerAuth swaps again
        end
    end
Loading

jwt_token_authentication.rst — Execution API request-time validation pipeline

flowchart TD
    REQ([Incoming request<br/>Authorization: Bearer ...])
    REQ --> CACHE{Cached on<br/>request.scope?}
    CACHE -->|yes| RET([Return cached TIToken])
    CACHE -->|no| SIG[JWTValidator:<br/>verify signature]
    SIG -->|fail| F1([403 Forbidden])
    SIG -->|ok| STD[Verify exp / iat / nbf<br/>aud / iss]
    STD -->|fail| F1
    STD -->|ok| SCOPE[Default scope to<br/>'execution' if absent]
    SCOPE --> SCHEMA[TIClaims:<br/>typed Pydantic schema]
    SCHEMA -->|ValidationError| F1
    SCHEMA -->|ok| TYP{require_auth:<br/>scope in<br/>route.allowed_token_types?}
    TYP -->|no| F1
    TYP -->|yes| SELF{ti:self scope<br/>declared?}
    SELF -->|no| OK([Return TIToken])
    SELF -->|yes| MATCH{token.sub ==<br/>task_instance_id?}
    MATCH -->|no| F1
    MATCH -->|yes| OK

    classDef fail fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
    classDef pass fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
    class F1 fail
    class OK,RET pass
Loading

security_model.rst — component trust boundaries

flowchart LR
    subgraph users["Users (untrusted by default)"]
        UI[UI / browser]
        CLI[CLI]
        EXT[External REST clients]
    end

    subgraph dataplane["Worker plane (no metadata DB access)"]
        WRK[Worker / Task]
    end

    subgraph controlplane["Control plane (metadata DB access)"]
        APISVR[API Server]
        SCH[Scheduler]
        DFP[Dag File Processor]
        TRG[Triggerer]
    end

    DB[(Metadata DB)]

    UI -->|JWT| APISVR
    CLI -->|JWT| APISVR
    EXT -->|JWT| APISVR
    WRK -->|JWT<br/>Execution API| APISVR

    APISVR -. SQL .-> DB
    SCH -. SQL .-> DB
    DFP -. SQL .-> DB
    TRG -. SQL .-> DB

    DFP -. in-process<br/>JWT bypassed .-> APISVR
    TRG -. in-process<br/>JWT bypassed .-> APISVR

    classDef untrusted fill:#ffcdd2,stroke:#c62828,stroke-width:2px,color:#000
    classDef trusted fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
    classDef data fill:#fff9c4,stroke:#f57f17,stroke-width:2px,color:#000
    class UI,CLI,EXT,WRK untrusted
    class APISVR,SCH,DFP,TRG trusted
    class DB data
Loading

security_model.rst — credential-distribution (least → most privileged)

flowchart LR
    subgraph WRK["Worker (least privileged)"]
        direction TB
        W1[Fernet key]
        W2[Worker secrets backend credentials]
        W3[Remote log handler kwargs]
    end

    subgraph TRG["Triggerer"]
        direction TB
        T1[DB connection]
        T2[Fernet key]
        T3[Non-worker secrets backend credentials]
        T4[Remote log handler kwargs]
    end

    subgraph DFP["Dag File Processor"]
        direction TB
        D1[DB connection]
        D2[Fernet key]
        D3[Non-worker secrets backend credentials]
    end

    subgraph SCH["Scheduler"]
        direction TB
        S1[DB connection]
        S2[JWT signing key]
        S3[Fernet key]
        S4[Non-worker secrets backend credentials]
        S5[Remote log handler kwargs]
    end

    subgraph API["API Server (most privileged)"]
        direction TB
        A1[DB connection]
        A2[JWT signing key]
        A3[Fernet key]
    end

    classDef wrk fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#000
    classDef ctrl fill:#bbdefb,stroke:#1565c0,stroke-width:2px,color:#000
    class WRK wrk
    class API,SCH,DFP,TRG ctrl
Loading

The doc also includes an explicit ✓/— table for true matrix lookup (which secret does which component need?) — see the rst source.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

Comment thread airflow-core/docs/security/jwt_token_authentication.rst Outdated
potiuk added 3 commits May 25, 2026 08:40
Catch up the public security documentation to match the security-relevant
changes flowing into the 3.2 release branch. Adds six mermaid diagrams
(four in jwt_token_authentication.rst, two in security_model.rst) and
documents:

- Typed TIClaims Pydantic schema validation of Execution API tokens.
- Unconditional revoke_token() on /auth/logout so external IdP redirects
  no longer leave the Airflow JWT valid.
- Router-level Depends(get_user) as a defense-in-depth backstop on
  /api/v2 and /ui.
- ExecutionAPISecretsBackend raising PermissionError on 401/403 so a
  deny no longer falls through to less-restrictive backends.
- Tightened deserialization allowlist regex (full-string match).

Registers sphinxcontrib-mermaid as a new docs dependency in
devel-common and BASIC_SPHINX_EXTENSIONS.
- Replace the arrow-spaghetti credential-distribution mermaid with a
  component-grouped layout (least- to most-privileged left-to-right)
  plus an explicit RST table for true matrix lookup.
- Bump all six security-diagram color palettes from very-pale tints to
  medium-saturation fills with explicit black text and 2px strokes, so
  labels stay readable in both light and dark mode renderers.
The /run endpoint is PATCH /{task_instance_id}/run, not POST.
Spotted in review of apache#67435.
@potiuk potiuk force-pushed the docs/security-model-v3.2-catchup branch from e6a0a7c to 5c989d2 Compare May 25, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants