Core 5.0

.claude/.gitignore

@@ -0,0 +1 @@
+*.local.json

.env.example

@@ -4,7 +4,5 @@ export OAUTH2_ACTIVE=0
 export LOG_LEVEL=DEBUG
 export WEBSOCKET_BROADCASTER_URL="redis://localhost:6379"
 
-export OPENAI_API_KEY=None
-export AGENT_ENABLED=False
-export SEARCH_ENABLED=False
+export EMBEDDING_API_KEY=None
 # export COMPOSE_PROFILES=lso # unhash the start of this line to also start up lso container

.pre-commit-config.yaml

@@ -1,5 +1,10 @@
 # .pre-commit-config.yaml
 repos:
+  - repo: https://github.com/adrienverge/yamllint
+    rev: v1.37.0
+    hooks:
+      - id: yamllint
+
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.15.0   # pin to a specific Ruff version
     hooks:

.yamllint

@@ -0,0 +1,23 @@
+---
+extends: default
+rules:
+  braces: disable
+  brackets: disable
+  colons: disable
+  commas: disable
+  comments: disable
+  comments-indentation: disable
+  document-end: disable
+  document-start: disable
+  empty-lines: disable
+  empty-values: disable
+  float-values: disable
+  hyphens: disable
+  indentation: disable
+  key-duplicates: disable
+  line-length: disable
+  new-line-at-end-of-file: disable
+  new-lines: disable
+  octal-values: disable
+  trailing-spaces: disable
+  truthy: disable

README.md

@@ -353,8 +353,8 @@ The `wsgi.py` can be as simple as shown below, and can be deployed by a
 ASGI server like Uvicorn[^5].
 
 ```python
-from orchestrator import OrchestratorCore
-from orchestrator.settings import AppSettings
+from orchestrator.core import OrchestratorCore
+from orchestrator.core.settings import AppSettings
 
 import products
 import workflows
@@ -373,7 +373,7 @@ uvicorn --host localhost --port 8080 wsgi:app
 To use the orchestrator command line interface create a `main.py` like below:
 
 ```python
-from orchestrator.cli.main import app as core_cli
+from orchestrator.core.cli.main import app as core_cli
 
 if __name__ == "__main__":
     core_cli()
@@ -658,7 +658,7 @@ to be added to the `SUBSCRIPTION_MODEL_REGISTRY`, in
 `products/__init__.py`, as shown below.
 
 ```python
-from orchestrator.domain import SUBSCRIPTION_MODEL_REGISTRY
+from orchestrator.core.domain import SUBSCRIPTION_MODEL_REGISTRY
 from products.product_types.core_link import CoreLink
 
 SUBSCRIPTION_MODEL_REGISTRY.update(
@@ -676,7 +676,7 @@ an argument, see below. Notice that the name of the product and the
 product type need to match with the subscription model registry.
 
 ```python
-from orchestrator.migrations.helpers import create
+from orchestrator.core.migrations.helpers import create
 
 new_products = {
     "products": {
@@ -1105,14 +1105,14 @@ Like a create workflow, the modify workflow also uses an initial input
 form but this time to only collect the values from the user that need to
 be changed. Usually, only a subset of the values may be changed. To
 assist the user, additional values can be shown in the input form using
-`ReadOnlyField`. In the example below, the name of the node is shown but
+`read_only_field`. In the example below, the name of the node is shown but
 cannot be changed, the node status can be changed and the dropdown is
 set to the current node status, and the node description is still
 optional.
 
 ```python
 class ModifyNodeForm(FormPage):
-    node_name: ReadOnlyField(port.node.node_name)
+    node_name: read_only_field(port.node.node_name)
     node_status: NodeStatusChoice = node.node_status
     node_description: str | None = node.node_description
 ```
@@ -1500,7 +1500,7 @@ WFO and NetBox both use the GraphQL framework Strawberry[^9] which supports Apol
 
 The following is required to facilitate GraphQL federation on top of WFO and other GraphQL backend(s):
 
-- WFO must be configured with `FEDERATION_ENABLED=True`
+- WFO must be configured with `FEDERATION_VERSION=2.11`
   - [`docker/orchestrator/orchestrator.env`](docker/orchestrator/orchestrator.env)
 - The other backend must also enable federation
   - NetBox: [`docker/netbox/Dockerfile`](docker/netbox/Dockerfile)

docker-compose.yml

@@ -9,7 +9,7 @@ x-netbox: &netbox
     - ./docker/netbox/netbox.env
     - path: ./docker/overrides/netbox/netbox.env
       required: false
-  user: "unit:root"
+  user: "1000:0"
   healthcheck:
     start_period: 360s
     timeout: 3s
@@ -73,6 +73,7 @@ services:
 
   embeddings:
     container_name: embeddings
+    platform: linux/amd64
     image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.8
     command: --model-id sentence-transformers/all-MiniLM-L6-v2
     profiles:
@@ -109,7 +110,7 @@ services:
 
   federation:
     container_name: federation
-    image: ghcr.io/apollographql/router:v2.6.2
+    image: ghcr.io/apollographql/router:v2.11.0
     ports:
       - "4000:4000"
     depends_on:
@@ -165,13 +166,10 @@ services:
     image: ${ORCH_BACKEND_TAG:-ghcr.io/workfloworchestrator/orchestrator-core:latest}
     env_file:
       - ./docker/orchestrator/orchestrator.env
-      - ./docker/embeddings/embeddings.env
       - path: ./docker/overrides/orchestrator/orchestrator.env
         required: false
     environment:
       LSO_ENABLED: ${COMPOSE_PROFILES:+True}
-      AGENT_ENABLED: ${AGENT_ENABLED:-False}
-      SEARCH_ENABLED: ${SEARCH_ENABLED:-False}
     ports:
       - "8080:8080"
       - "5678:5678" #Enable Python debugger

docker/embeddings/README.md

@@ -12,48 +12,61 @@ The embeddings service provides an OpenAI-compatible API for generating text emb
 
 ## Prerequisites
 
-To use embeddings for search and agent features, add these to your `.env` file (see `.env.example`):
+First, please familiarize yourself with these pages:
+- `./docker/overrides/configuration.md`: how to configure variables per docker compose service
+- `./docker/orchestrator/README.md`: the section about Search
 
-```env
-AGENT_ENABLED=True
-SEARCH_ENABLED=True
-OPENAI_API_KEY=your-api-key-here  # Optional: only needed for agent features or when using OpenAI embeddings
+To enable embeddings for search and agent features, set the following `orchestrator` variables:
+
+```dotenv
+EMBEDDING_API_ENABLED=True
+
+# Optional: only needed for agent features or when using OpenAI embeddings
+EMBEDDING_API_KEY=your-api-key-here
 ```
 
 ## Local Embeddings (Default)
 
-This setup uses a local embedding service with no external API required. The default configuration in `embeddings.env` is:
+This setup uses a local embedding service with no external API required.
+You can inspect the default configuration in `orchestrator.env`.
+
+### 1. Configuration
 
-- `OPENAI_BASE_URL=http://embeddings:80/v1`
-- `EMBEDDING_DIMENSION=384`
+Set the following `orchestrator` variables:
+
+```dotenv
+EMBEDDING_API_ENABLED=True
+```
 
-### Start the orchestrator
+### 2. Start services
 
-Start the orchestrator with the local embeddings service:
+Start the docker compose stack with the embeddings profile:
 
 ```bash
-docker compose --profile embeddings up orchestrator
+docker compose --profile embeddings up
 ```
 
 ## Alternative: Using OpenAI Embeddings
 
 If you prefer to use OpenAI's embedding service instead of running a local model:
 
-### Configuration
+### 1. Configuration
 
-Override the embedding settings by editing `docker/overrides/embeddings/embeddings.env`:
+Set the following `orchestrator` variables:
 
-```env
-OPENAI_BASE_URL=https://api.openai.com/v1
+```dotenv
+EMBEDDING_API_ENABLED=True
+EMBEDDING_API_BASE=https://api.openai.com/v1
+EMBEDDING_API_KEY=your-api-key-here
 EMBEDDING_DIMENSION=1536
 ```
 
-### Start the orchestrator
+### 2. Start services
 
-Start only the orchestrator (skips the local embeddings service):
+Start the docker compose stack as normal (skips the local embeddings service):
 
 ```bash
-docker compose up orchestrator
+docker compose up
 ```
 
 ## Post-Setup Steps
@@ -80,9 +93,11 @@ docker compose exec orchestrator /home/orchestrator/.venv/bin/python main.py ind
 
 ## Advanced Configuration
 
-The following configurations use conservative defaults for local/unknown models:
+The following `orchestrator` variables are configured with conservative defaults for local/unknown models:
 
 - `EMBEDDING_FALLBACK_MAX_TOKENS=512`: Maximum tokens per embedding request
 - `EMBEDDING_MAX_BATCH_SIZE=32`: Maximum batch size for embedding requests
 
-**Note**: These settings are only used as fallbacks for local or unknown models (like the example in this setup). For known providers and models, the system automatically retrieves the correct values via LiteLLM. The fallback values are already configured safely for local models, but can be adjusted if needed in `docker/overrides/embeddings/embeddings.env`.
+**Note**: These settings are only used as fallbacks for local or unknown models (like the example in this setup). 
+For known providers and models, the system automatically retrieves the correct values via LiteLLM. 
+The fallback values are already configured safely for local models, but can be adjusted in the `orchestrator` variables.

docker/embeddings/embeddings.env

@@ -1,11 +1 @@
-# Default: Local embeddings service using sentence-transformers/all-MiniLM-L6-v2
-# Provides OpenAI-compatible embeddings API locally with no external dependencies
-
-OPENAI_BASE_URL=http://embeddings:80/v1
-EMBEDDING_DIMENSION=384
-EMBEDDING_MAX_BATCH_SIZE=32 # Not required when using OpenAI
-
-# Alternative: Uncomment below to use OpenAI embeddings instead
-# (Requires OPENAI_API_KEY in your .env file)
-# OPENAI_BASE_URL=https://api.openai.com/v1
-# EMBEDDING_DIMENSION=1536
+# Please refer to the documentation in `./docker/embeddings/README.md`.

docker/federation/rover.Dockerfile

@@ -5,7 +5,7 @@ RUN apt update && apt install curl -y
 RUN useradd --create-home --shell /bin/bash rover-user
 
 USER rover-user
-RUN curl -sSL https://rover.apollo.dev/nix/v0.35.0 | sh
+RUN curl -sSL https://rover.apollo.dev/nix/v0.38.1 | sh
 
 USER root
 RUN apt remove curl -y

docker/federation/supergraph-config.yaml

@@ -1,4 +1,4 @@
-federation_version: =2.9.3
+federation_version: =2.11.0
 subgraphs:
   orchestrator:
     routing_url: http://orchestrator:8080/api/graphql

docker/netbox/Dockerfile

@@ -1,9 +1,9 @@
-FROM netboxcommunity/netbox:v4.4.1
+FROM netboxcommunity/netbox:v4.5-4.0.2
 
 # NOTE: when updating the Netbox version, remember to update the database snapshot. See docker/postgresql/README.md for details
 
 # Patch strawberry schema to enable federation
-RUN sed -i 's/= strawberry.Schema(/= strawberry.federation.Schema(enable_federation_2=True,/g' /opt/netbox/netbox/netbox/graphql/schema.py
+RUN sed -i 's/= strawberry.Schema(/= strawberry.federation.Schema(/g' /opt/netbox/netbox/netbox/graphql/schema.py
 
 # Patch strawberry types to allow joins
 ADD patch_federation.py .

docker/netbox/configuration/configuration.py

@@ -108,6 +108,7 @@ def _environ_get_and_map(variable_name: str, default: str | None = None, map_fn:
 # symbols. NetBox will not run without this defined. For more information, see
 # https://docs.djangoproject.com/en/stable/ref/settings/#std:setting-SECRET_KEY
 SECRET_KEY = _read_secret('secret_key', environ.get('SECRET_KEY', ''))
+API_TOKEN_PEPPERS = { 1: _read_secret('api_token_pepper', environ.get('API_TOKEN_PEPPER', ''))}
 
 
 #########################

docker/netbox/data.json

@@ -10,7 +10,6 @@
       "first_name": "",
       "last_name": "",
       "email": "admin@admin.admin",
-      "is_staff": true,
       "is_active": true,
       "date_joined": "2023-04-03T20:31:46.411Z",
       "groups": [],
@@ -44,13 +43,18 @@
     "model": "users.token",
     "pk": 1,
     "fields": {
+      "version": 2,
       "user": 1,
-      "created": "2023-04-04T20:50:21.369Z",
+      "description": "",
+      "created": "2026-05-01T16:46:57.511Z",
       "expires": null,
-      "last_used": "2023-04-04T20:51:55.276Z",
-      "key": "e744057d755255a31818bf74df2350c26eeabe54",
+      "last_used": null,
+      "enabled": true,
       "write_enabled": true,
-      "description": "",
+      "plaintext": null,
+      "key": "sDx3hakFeTCZ",
+      "pepper_id": 1,
+      "hmac_digest": "f36541405acbc41c5b26209011628b3bff980d390b103dae8c43e848168e0a6d",
       "allowed_ips": []
     }
   }

docker/netbox/netbox.env

@@ -31,6 +31,7 @@ REDIS_PASSWORD=nwa
 REDIS_SSL=false
 RELEASE_CHECK_URL=https://api.github.com/repos/netbox-community/netbox/releases
 SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNjN
+API_TOKEN_PEPPER=xslRlHpCH)w53Umz*t0*tqLLe9iE3_uhhUuu*ryiCgmN5KNiiH
 SKIP_SUPERUSER=false
 SUPERUSER_NAME=admin
 SUPERUSER_PASSWORD=admin

docker/orchestrator/README.md

@@ -0,0 +1,66 @@
+# Orchestrator
+
+This directory configures the orchestrator service through:
+- `entrypoint.sh`: the container entrypoint
+- `orchestrator.env`: default environment variables to set
+- `.bashrc`: configuration for opening a shell
+- `is_healthy.py`: defines a docker compose health check
+
+## Configuration options
+
+### Orchestrator
+
+You can override the orchestrator's environment variables as described in `./docker/overrides/configuration.md`.
+
+### Search and Embeddings
+
+The default configuration is set up to provide structured/fuzzy search without the need for embeddings.
+
+For more powerful search capabilities, consult `./docker/embeddings/README.md` on how to set up embeddings.
+
+## Development
+
+### Testing orchestrator-core upgrades
+
+The below instructions are for testing changes that require recreating the orchestrator docker compose service, such as:
+- updates to the `ghcr.io/workfloworchestrator/orchestrator-core` image
+- changes to the entrypoint.sh
+- changes to pyproject.toml
+
+#### 1. Start clean and up to date
+
+Stop containers and remove all volumes:
+
+```sh
+docker compose down --volumes
+```
+
+Pull the latest images, ensure all of them are built, and start the containers:
+
+```sh
+docker compose pull && docker compose build && docker compose up
+```
+
+So far everything should work out of the box, if not then there are pre-existing issues that should be resolved.
+
+#### 2. Make your changes
+
+Now you can make your changes, for example temporarily using a pre-release of the orchestrator-core by changing `.env` to:
+
+```
+ORCH_BACKEND_TAG=ghcr.io/workfloworchestrator/orchestrator-core:edge
+```
+
+You can also make changes to pyproject.toml/uv.lock or entrypoint.sh (in this folder)
+
+#### 3. Recreate the container
+
+To test the changes from step 2, run:
+
+```
+docker compose up -d --force-recreate orchestrator
+```
+
+Inspect the results in your other terminal where `docker compose up` is running.
+
+If needed, make changes or fixes and re-run the above command.