merge: bring feat-microsoft-provider-v2 up to main

Author: afourniernv

.agents/skills/build-from-issue/SKILL.md

@@ -148,7 +148,8 @@ In the prompt, instruct the reviewer to:
    - **Medium**: Multiple files/components, some design decisions, but well-scoped
    - **High**: Cross-cutting changes, architectural decisions needed, significant unknowns
 8. Call out risks, unknowns, and decisions that need stakeholder input.
-9. Assess **LSM compatibility** — if the change touches process identity, `/proc` filesystem access, binary execution, or inter-process visibility, flag whether it will behave differently on hosts running SELinux (enforcing) or AppArmor. In particular, tests that fork+exec into system binaries will fail on SELinux-enforcing hosts due to cross-label `/proc/<pid>/exe` access restrictions.
+9. Assess **gateway config documentation impact** — if the change adds, removes, renames, or changes defaults for gateway TOML keys or driver-specific config options, the plan must include an update to `docs/reference/gateway-config.mdx`. If the change is surfaced through Helm or a compute-driver overview, also include `docs/reference/sandbox-compute-drivers.mdx` or the relevant deployment docs.
+10. Assess **LSM compatibility** — if the change touches process identity, `/proc` filesystem access, binary execution, or inter-process visibility, flag whether it will behave differently on hosts running SELinux (enforcing) or AppArmor. In particular, tests that fork+exec into system binaries will fail on SELinux-enforcing hosts due to cross-label `/proc/<pid>/exe` access restrictions.
 
 ### A2: Post the Plan Comment
 
@@ -436,6 +437,13 @@ Review the documentation requirements in `AGENTS.md` and update any affected
 docs as part of the implementation. Keep documentation changes scoped to the
 behavior or subsystem that changed.
 
+If the implementation changes gateway TOML parsing, `[openshell.gateway]`
+fields, `[openshell.drivers.<name>]` fields, driver config defaults, or Helm
+rendering of `gateway.toml`, update `docs/reference/gateway-config.mdx` in the
+same branch. If the change affects user-facing compute-driver setup, also
+update `docs/reference/sandbox-compute-drivers.mdx` or the relevant deployment
+page.
+
 ### Step 12: Commit and Push
 
 Commit all changes using conventional commit format. The `<type>` comes from the issue type in the plan:

.agents/skills/create-github-pr/SKILL.md

@@ -15,6 +15,15 @@ Create pull requests on GitHub using the `gh` CLI.
 
 ## Before Creating a PR
 
+### Check Config Documentation
+
+If the branch changes gateway TOML parsing, `[openshell.gateway]` fields,
+`[openshell.drivers.<name>]` fields, driver config defaults, or Helm rendering
+of `gateway.toml`, verify that `docs/reference/gateway-config.mdx` is updated
+in the same branch. If the change affects user-facing compute-driver setup,
+also update `docs/reference/sandbox-compute-drivers.mdx` or the relevant
+deployment docs.
+
 ### Run Pre-commit Checks
 
 Run the local pre-commit task before opening a PR:

.agents/skills/create-spike/SKILL.md

@@ -91,9 +91,11 @@ The prompt to the reviewer **must** instruct it to:
 
 9. **Check architecture docs** in the `architecture/` directory for relevant documentation about the affected subsystems.
 
-10. **Assess Linux Security Module (LSM) impact.** If the change involves process identity, `/proc` filesystem access, file labeling, binary execution, or inter-process visibility, call out whether it will behave differently on hosts running SELinux (enforcing) or AppArmor. For example: reading `/proc/<pid>/exe` across an SELinux domain boundary returns ENOENT, not EACCES. Tests that fork+exec into system binaries (different SELinux label) will fail on enforcing hosts. Flag any LSM-sensitive code paths and recommend mitigations.
+10. **Assess gateway config documentation impact.** If the change would add, remove, rename, or change defaults for gateway TOML keys or driver-specific config options, call out that `docs/reference/gateway-config.mdx` must be updated. If the change is surfaced through Helm or compute-driver setup docs, call out the relevant deployment or compute-driver docs too.
 
-11. **Determine the issue type:** `feat`, `fix`, `refactor`, `chore`, `perf`, or `docs`.
+11. **Assess Linux Security Module (LSM) impact.** If the change involves process identity, `/proc` filesystem access, file labeling, binary execution, or inter-process visibility, call out whether it will behave differently on hosts running SELinux (enforcing) or AppArmor. For example: reading `/proc/<pid>/exe` across an SELinux domain boundary returns ENOENT, not EACCES. Tests that fork+exec into system binaries (different SELinux label) will fail on enforcing hosts. Flag any LSM-sensitive code paths and recommend mitigations.
+
+12. **Determine the issue type:** `feat`, `fix`, `refactor`, `chore`, `perf`, or `docs`.
 
 ### What makes a good investigation prompt
 

.agents/skills/debug-openshell-cluster/SKILL.md

@@ -138,6 +138,15 @@ kubectl -n openshell rollout status statefulset/openshell
 
 Look for failed installs, unexpected values, missing namespace, wrong image tag, TLS settings that do not match the registered endpoint, and scheduling failures.
 
+For HA or PostgreSQL-backed installs, also check the service-binding Secret and
+bundled PostgreSQL workload:
+
+```bash
+kubectl -n openshell get secret -l app.kubernetes.io/instance=openshell
+kubectl -n openshell get statefulset,pod,pvc -l app.kubernetes.io/instance=openshell
+kubectl -n openshell logs statefulset/openshell-postgres --tail=200
+```
+
 Check required Helm deployment secrets:
 
 ```bash

.agents/skills/helm-dev-environment/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: helm-dev-environment
-description: Start up, tear down, and configure the local Kubernetes development environment for OpenShell. Uses k3d (Docker-backed k3s) + Skaffold + Helm. Covers cluster lifecycle, optional add-ons (Keycloak OIDC, Envoy Gateway), and port mappings. Trigger keywords - local k8s, local cluster, k3d, skaffold, helm dev, start cluster, stop cluster, tear down cluster, delete cluster, create cluster, helm:k3s, helm:skaffold, local dev environment, dev cluster, k8s dev, envoy gateway local, keycloak local.
+description: Start up, tear down, and configure the local Kubernetes development environment for OpenShell. Uses k3d (Docker-backed k3s) + Skaffold + Helm. Covers cluster lifecycle, optional add-ons (Keycloak OIDC, Envoy Gateway), HA testing, and port mappings. Trigger keywords - local k8s, local cluster, k3d, skaffold, helm dev, start cluster, stop cluster, tear down cluster, delete cluster, create cluster, helm:k3s, helm:skaffold, local dev environment, dev cluster, k8s dev, envoy gateway local, keycloak local, high availability, HA.
 ---
 
 # Helm Dev Environment
@@ -65,6 +65,10 @@ generates mTLS secrets on first install. Envoy Gateway opt-in; see the Optional
 
 The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or `kubectl port-forward`.
 
+**HA test deploy** (two gateway replicas + bundled PostgreSQL): uncomment
+`#- ci/values-high-availability.yaml` in `deploy/helm/openshell/skaffold.yaml`,
+then run `mise run helm:skaffold:run` or `mise run helm:skaffold:dev`.
+
 ### TLS behaviour
 
 `ci/values-skaffold.yaml` sets `server.disableTls: true`, so Skaffold-based deploys run
@@ -198,6 +202,7 @@ mise run helm:k3s:status
 | `deploy/helm/openshell/ci/values-skaffold.yaml` | Dev overrides (image pull policy, TLS disabled for local Skaffold) |
 | `deploy/helm/openshell/ci/values-cert-manager.yaml` | cert-manager PKI overlay (opt-in; disables pkiInitJob) |
 | `deploy/helm/openshell/ci/values-gateway.yaml` | Envoy Gateway GRPCRoute + Gateway overlay |
+| `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with bundled PostgreSQL) |
 | `deploy/helm/openshell/ci/values-keycloak.yaml` | Keycloak OIDC overlay |
 | `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) |
 | `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass for Envoy Gateway (`mise run helm:gateway:apply`) |

.agents/skills/update-docs/SKILL.md

@@ -34,7 +34,7 @@ git log -50 --oneline --no-merges
 Filter to commits that are likely to affect docs. Look for these signals:
 
 1. **Commit type**: `feat`, `fix`, `refactor`, `perf` commits often change behavior. `docs` commits are already doc changes. `chore`, `ci`, `test` commits rarely need doc updates.
-2. **Files changed**: Changes to `crates/openshell-cli/`, `python/`, `proto/`, `deploy/`, or policy-related code are high-signal.
+2. **Files changed**: Changes to `crates/openshell-cli/`, `python/`, `proto/`, `deploy/`, gateway config parsing, driver config structs, or policy-related code are high-signal.
 3. **Ignore**: Changes limited to `tests/`, `e2e/`, `.github/`, `tasks/`, or internal-only modules.
 
 ```bash
@@ -52,6 +52,10 @@ For each relevant commit, determine which doc page(s) it affects. Use this mappi
 | `crates/openshell-cli/` (sandbox commands) | `docs/sandboxes/manage-sandboxes.mdx` |
 | `crates/openshell-cli/` (provider commands) | `docs/sandboxes/manage-providers.mdx` |
 | `crates/openshell-cli/` (new top-level command) | May need a new page or `docs/reference/` entry |
+| `crates/openshell-server/src/config_file.rs` or gateway TOML parsing | `docs/reference/gateway-config.mdx` |
+| `crates/openshell-server/src/cli.rs` gateway config merge/default behavior | `docs/reference/gateway-config.mdx` |
+| `crates/openshell-driver-*/` config structs or driver defaults | `docs/reference/gateway-config.mdx`, `docs/reference/sandbox-compute-drivers.mdx` |
+| `deploy/helm/openshell/templates/gateway-config.yaml` | `docs/reference/gateway-config.mdx`, `docs/reference/sandbox-compute-drivers.mdx`, Helm docs if values change |
 | Proxy or policy code | `docs/sandboxes/policies.mdx`, `docs/reference/policy-schema.mdx` |
 | Inference code | `docs/inference/configure.mdx` |
 | `python/` (SDK changes) | `docs/reference/` or `docs/get-started/quickstart.mdx` |

.github/actions/release-helm-oci/action.yml

@@ -71,6 +71,14 @@ runs:
           exit 1
         fi
 
+    - name: Build chart dependencies
+      env:
+        CHART_DIR: ${{ steps.prep.outputs.chart_dir }}
+      shell: bash
+      run: |
+        set -euo pipefail
+        helm dependency build "${CHART_DIR}"
+
     - name: Package Helm chart
       env:
         CHART_DIR: ${{ steps.prep.outputs.chart_dir }}

.github/workflows/branch-e2e.yml

@@ -23,6 +23,7 @@ jobs:
       should_run: ${{ steps.gate.outputs.should_run }}
       run_core_e2e: ${{ steps.labels.outputs.run_core_e2e }}
       run_gpu_e2e: ${{ steps.labels.outputs.run_gpu_e2e }}
+      run_kubernetes_ha_e2e: ${{ steps.labels.outputs.run_kubernetes_ha_e2e }}
       run_any_e2e: ${{ steps.labels.outputs.run_any_e2e }}
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -39,24 +40,27 @@ jobs:
           if [ "$EVENT_NAME" != "push" ]; then
             run_core_e2e=true
             run_gpu_e2e=true
+            run_kubernetes_ha_e2e=true
           else
             run_core_e2e="$(jq -r 'index("test:e2e") != null' <<< "$LABELS_JSON")"
             run_gpu_e2e="$(jq -r 'index("test:e2e-gpu") != null' <<< "$LABELS_JSON")"
+            run_kubernetes_ha_e2e="$(jq -r 'index("test:e2e-kubernetes") != null' <<< "$LABELS_JSON")"
           fi
-          if [ "$run_core_e2e" = "true" ] || [ "$run_gpu_e2e" = "true" ]; then
+          if [ "$run_core_e2e" = "true" ] || [ "$run_gpu_e2e" = "true" ] || [ "$run_kubernetes_ha_e2e" = "true" ]; then
             run_any_e2e=true
           else
             run_any_e2e=false
           fi
           {
             echo "run_core_e2e=$run_core_e2e"
             echo "run_gpu_e2e=$run_gpu_e2e"
+            echo "run_kubernetes_ha_e2e=$run_kubernetes_ha_e2e"
             echo "run_any_e2e=$run_any_e2e"
           } >> "$GITHUB_OUTPUT"
 
   build-gateway:
     needs: [pr_metadata]
-    if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
+    if: needs.pr_metadata.outputs.should_run == 'true' && (needs.pr_metadata.outputs.run_core_e2e == 'true' || needs.pr_metadata.outputs.run_kubernetes_ha_e2e == 'true')
     permissions:
       contents: read
       packages: write
@@ -107,6 +111,18 @@ jobs:
     with:
       image-tag: ${{ github.sha }}
 
+  kubernetes-ha-e2e:
+    needs: [pr_metadata, build-gateway, build-supervisor]
+    if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_kubernetes_ha_e2e == 'true'
+    permissions:
+      contents: read
+      packages: read
+    uses: ./.github/workflows/e2e-kubernetes-test.yml
+    with:
+      image-tag: ${{ github.sha }}
+      job-name: Kubernetes HA E2E (Rust smoke)
+      extra-helm-values: deploy/helm/openshell/ci/values-high-availability.yaml
+
   core-e2e-result:
     name: Core E2E result
     needs: [pr_metadata, build-gateway, build-supervisor, e2e, kubernetes-e2e]
@@ -160,3 +176,30 @@ jobs:
             fi
           done
           exit "$failed"
+
+  kubernetes-ha-e2e-result:
+    name: Kubernetes HA E2E result
+    needs: [pr_metadata, build-gateway, build-supervisor, kubernetes-ha-e2e]
+    if: always() && needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_kubernetes_ha_e2e == 'true'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Verify Kubernetes HA E2E jobs
+        env:
+          BUILD_GATEWAY_RESULT: ${{ needs.build-gateway.result }}
+          BUILD_SUPERVISOR_RESULT: ${{ needs.build-supervisor.result }}
+          KUBERNETES_HA_E2E_RESULT: ${{ needs.kubernetes-ha-e2e.result }}
+        run: |
+          set -euo pipefail
+          failed=0
+          for item in \
+            "build-gateway:$BUILD_GATEWAY_RESULT" \
+            "build-supervisor:$BUILD_SUPERVISOR_RESULT" \
+            "kubernetes-ha-e2e:$KUBERNETES_HA_E2E_RESULT"; do
+            name="${item%%:*}"
+            result="${item#*:}"
+            if [ "$result" != "success" ]; then
+              echo "::error::$name concluded $result"
+              failed=1
+            fi
+          done
+          exit "$failed"

.github/workflows/docker-build.yml

@@ -162,7 +162,7 @@ jobs:
       cargo-version: ${{ inputs['cargo-version'] }}
       image-tag: ${{ needs.resolve.outputs.image_tag_base }}
       checkout-ref: ${{ inputs['checkout-ref'] }}
-      features: openshell-core/dev-settings
+      features: ${{ inputs.component == 'gateway' && 'openshell-core/dev-settings bundled-z3' || 'openshell-core/dev-settings' }}
       artifact-name: ${{ needs.resolve.outputs.artifact_prefix }}-linux-${{ matrix.arch }}
     secrets: inherit
 

.github/workflows/e2e-kubernetes-test.yml

@@ -17,14 +17,29 @@ on:
         required: false
         type: string
         default: ""
+      job-name:
+        description: "Display name for the Kubernetes e2e job"
+        required: false
+        type: string
+        default: "Kubernetes E2E (Rust smoke)"
+      extra-helm-values:
+        description: "Colon-separated Helm values files to layer on the Kubernetes e2e chart install"
+        required: false
+        type: string
+        default: ""
+      mise-version:
+        description: "mise version to install on the bare Kubernetes e2e runner"
+        required: false
+        type: string
+        default: "v2026.4.25"
 
 permissions:
   contents: read
   packages: read
 
 jobs:
   e2e-kubernetes:
-    name: Kubernetes E2E (Rust smoke)
+    name: ${{ inputs.job-name }}
     # Bare runner: running kind-in-container hits nested-Docker / kubeconfig
     # complications. The runner has Docker; mise installs helm, kubectl, and
     # the Rust toolchain.
@@ -35,6 +50,8 @@ jobs:
       packages: read
     env:
       MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      # Keep bare-runner installs aligned with the project CI image.
+      MISE_VERSION: ${{ inputs.mise-version }}
       KIND_CLUSTER_NAME: kube-e2e-${{ github.run_id }}
     steps:
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
@@ -43,7 +60,7 @@ jobs:
 
       - name: Install mise
         run: |
-          curl https://mise.run | sh
+          curl https://mise.run | MISE_VERSION=v2026.4.25 sh
           echo "$HOME/.local/bin" >> "$GITHUB_PATH"
           echo "$HOME/.local/share/mise/shims" >> "$GITHUB_PATH"
 
@@ -93,6 +110,7 @@ jobs:
       - name: Run Kubernetes E2E (Rust smoke)
         env:
           OPENSHELL_E2E_KUBE_CONTEXT: kind-${{ env.KIND_CLUSTER_NAME }}
+          OPENSHELL_E2E_KUBE_EXTRA_VALUES: ${{ inputs.extra-helm-values }}
           IMAGE_TAG: ${{ inputs.image-tag }}
           OPENSHELL_REGISTRY: ghcr.io/nvidia/openshell
         run: mise run --no-deps --skip-deps e2e:kubernetes