docs: add versioned documentation site built with Zensical

.github/workflows/docs-deploy.yml

@@ -0,0 +1,77 @@
+name: Deploy documentation
+
+# Publishes versioned docs to the `gh-pages` branch using the Zensical-compatible
+# fork of `mike` (https://github.com/squidfunk/mike): each version is committed
+# as a subdirectory of `site_url` (see zensical.toml), with a `latest` alias and
+# a root redirect pointing at the newest release.
+#
+# Runs automatically on release tags (semantic-release tags the bare version,
+# e.g. 0.95.1) and can be triggered manually. Release tags publish the minor
+# series (e.g. 0.95) so that patch releases update the same series page.
+
+on:
+  push:
+    tags: ["[0-9]+.[0-9]+.[0-9]+"]
+  workflow_dispatch:
+    inputs:
+      version:
+        description: "Version label to publish (e.g. 0.95 or dev)"
+        required: true
+        default: "dev"
+      alias:
+        description: "Alias to update"
+        required: false
+        default: "latest"
+      set_default:
+        description: "Point the site root at this alias"
+        type: boolean
+        default: true
+
+permissions:
+  contents: write # mike pushes the built docs to the gh-pages branch
+
+concurrency:
+  group: docs-deploy
+  cancel-in-progress: false
+
+jobs:
+  deploy:
+    name: Deploy docs with mike
+    runs-on: ubuntu-latest
+    # Auto-deploy only on the canonical repo; manual runs are allowed anywhere
+    # (e.g. to bootstrap gh-pages on a fork while testing).
+    if: ${{ github.event_name == 'workflow_dispatch' || github.repository == 'substrait-io/substrait-java' }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+        with:
+          fetch-depth: 0 # mike reads/writes the full gh-pages history
+      - name: Set up pixi
+        uses: prefix-dev/setup-pixi@v0.10.0
+        with:
+          environments: docs
+      - name: Configure git identity
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+      - name: Resolve version and alias
+        id: v
+        run: |
+          if [ "${{ github.event_name }}" = "push" ]; then
+            full="${GITHUB_REF_NAME}"                        # 0.95.1
+            echo "version=${full%.*}" >> "$GITHUB_OUTPUT"    # 0.95 (minor series)
+            echo "alias=latest" >> "$GITHUB_OUTPUT"
+            echo "set_default=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "version=${{ inputs.version }}" >> "$GITHUB_OUTPUT"
+            echo "alias=${{ inputs.alias }}" >> "$GITHUB_OUTPUT"
+            echo "set_default=${{ inputs.set_default }}" >> "$GITHUB_OUTPUT"
+          fi
+      - name: Deploy version
+        run: |
+          pixi run -e docs mike deploy --push --update-aliases \
+            "${{ steps.v.outputs.version }}" "${{ steps.v.outputs.alias }}"
+      - name: Set default version
+        if: ${{ steps.v.outputs.set_default == 'true' }}
+        run: |
+          pixi run -e docs mike set-default --push "${{ steps.v.outputs.alias }}"

.github/workflows/docs.yml

@@ -0,0 +1,28 @@
+name: Build documentation
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+# Build-check only: this verifies the docs compile on every PR so broken pages
+# or a broken link are caught early. It does NOT publish. Versioned publishing
+# to GitHub Pages is handled separately by .github/workflows/docs-deploy.yml
+# (mike -> gh-pages branch).
+
+jobs:
+  build:
+    name: Build docs
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+      - name: Set up pixi
+        uses: prefix-dev/setup-pixi@v0.10.0
+        with:
+          environments: docs
+      - name: Build the documentation site
+        run: pixi run docs-build

.gitignore

@@ -20,3 +20,7 @@ bin
 .classpath
 .settings
 bin/
+
+# Documentation site (Zensical + pixi)
+/site
+.pixi

AGENTS.md

@@ -192,6 +192,28 @@ compile — they have their own visitor implementors:
   fidelity. See `core/src/test/java/io/substrait/type/proto/DynamicParameterRoundtripTest.java`
   for the canonical pattern.
 
+## Documentation
+
+User-facing documentation lives in `docs/` (Markdown, one file per page) and is built with
+**Zensical** (config in `zensical.toml`). Python and the doc tooling are managed with **pixi**,
+independent of the Gradle build — the only prerequisite is a `pixi` install.
+
+- Preview locally with `pixi run docs-serve` (live reload); build the static site with
+  `pixi run docs-build` (output to `site/`, gitignored). `docs-build` validates internal links,
+  so run it before pushing doc changes.
+- Pages are grouped under `docs/{core,isthmus,isthmus-cli,spark}/` plus a landing page and
+  getting-started. Navigation is **explicit** in `zensical.toml` (`nav`) — when you add a page,
+  add it to `nav`.
+- **Keep the guide in sync with the code as substrait-java evolves.** When you add or change
+  user-facing behavior — a new expression/relation type, a `SubstraitBuilder`/`ExpressionCreator`
+  factory, a newly supported function, an isthmus/spark capability, or a CLI flag — update the
+  matching `docs/` page in the same PR. Ground snippets in real APIs/tests, and (as in source)
+  keep GitHub issue/PR numbers out of the docs.
+- CI: `.github/workflows/docs.yml` build-checks docs on every PR and push to `main`;
+  `.github/workflows/docs-deploy.yml` publishes versioned docs to the `gh-pages` branch via the
+  Zensical-compatible `mike` fork on release tags (the bare `X.Y.Z` tag publishes the minor
+  series `X.Y` and updates the `latest` alias). Site: <https://substrait-io.github.io/substrait-java/>.
+
 ## Conventions & workflow
 
 - **Conventional commits** are required (CI lints them, and PR title + body must form a
@@ -204,7 +226,8 @@ compile — they have their own visitor implementors:
 - Many features track upstream Substrait spec releases (see epic-style issues); a new
   proto message usually needs: POJO + visitor wiring + both proto converters + a
   round-trip test, and often `ExpressionCreator` factories and `dsl/SubstraitBuilder`
-  helpers for ergonomics.
+  helpers for ergonomics — plus a matching update to the user guide under `docs/`
+  (see [Documentation](#documentation)).
 - When monitoring PR checks, budget for a long tail: the **macOS `Build Isthmus Native
   Image`** job is the long pole — it `needs:` the `java` + `integration` jobs (so it
   starts late), then AOT-compiles for ~15–20 min on a slower macOS runner. A PR staying

CONTRIBUTING.md

@@ -4,6 +4,7 @@ This page provides some orientation and recommendations on how to get the best r
 
 1. [Commit conventions](#commit-conventions)
 2. [Style Guide](#style-guide)
+3. [Documentation](#documentation)
 
 ## Commit Conventions
 
@@ -48,3 +49,29 @@ org.gradle.jvmargs=--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAME
   --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
   --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
 ```
+
+## Documentation
+
+The user-facing documentation site lives under [`docs/`](docs) and is built with
+[Zensical](https://zensical.org). Python and the documentation dependencies are managed with
+[pixi](https://pixi.sh), so the only prerequisite is a pixi installation.
+
+Preview your changes locally with a live-reloading dev server, or produce the static site:
+
+```bash
+pixi run docs-serve   # live-reloading preview at http://localhost:8000
+pixi run docs-build   # build the static site into ./site
+```
+
+Guidelines:
+
+* Every page is a Markdown file under `docs/`; the navigation is defined explicitly in
+  [`zensical.toml`](zensical.toml). When you add a page, add it to the `nav`.
+* Keep code samples accurate — base them on the real APIs and tests rather than inventing method
+  names. `pixi run docs-build` validates internal links.
+* As elsewhere in the codebase, do not reference GitHub issue or PR numbers in the docs.
+
+Documentation is built on every pull request by the `Build documentation` workflow. On each
+release, the `Deploy documentation` workflow publishes a versioned copy to
+<https://substrait-io.github.io/substrait-java/> (via the `mike` version manager, on the
+`gh-pages` branch).

docs/core/building-plans.md

@@ -0,0 +1,124 @@
+# Building plans
+
+`SubstraitBuilder` (in `io.substrait.dsl`) is the recommended high-level way to
+construct a Substrait `Plan`. It wraps the immutable POJO builders with concise,
+type-aware helpers for relations (scan, filter, project, join, aggregate, ...),
+expressions (literals, field references, casts, functions), and the plan root.
+
+## Creating a builder
+
+```java
+import io.substrait.dsl.SubstraitBuilder;
+
+SubstraitBuilder b = new SubstraitBuilder();
+```
+
+The no-arg constructor uses `DefaultExtensionCatalog.DEFAULT_COLLECTION`, which
+loads the standard Substrait function extensions (arithmetic, comparison,
+boolean, aggregate, string, datetime, and more). To resolve custom functions,
+pass your own `SimpleExtension.ExtensionCollection` — see
+[Function & type extensions](extensions.md).
+
+```java
+SubstraitBuilder b = new SubstraitBuilder(myExtensionCollection);
+```
+
+## Type shortcuts
+
+Types are created with `TypeCreator`. Throughout these examples we use two
+aliases, matching the convention used across the codebase and tests:
+
+```java
+import io.substrait.type.TypeCreator;
+
+TypeCreator R = TypeCreator.REQUIRED; // non-nullable types
+TypeCreator N = TypeCreator.NULLABLE; // nullable types
+```
+
+So `R.I32` is a non-nullable 32-bit integer and `N.STRING` is a nullable string.
+See [Types](types.md) for the full catalog.
+
+## Relation helpers take lambdas
+
+Most relation helpers accept a function from the input relation to the
+expression(s) they need. This lets the builder resolve field references against
+the input's schema for you. For example, `filter` takes a
+`Function<Rel, Expression>` that produces the condition:
+
+```java
+NamedScan scan =
+    b.namedScan(List.of("t"), List.of("a", "b"), List.of(R.I32, R.STRING));
+
+Filter filter =
+    b.filter(rel -> b.equal(b.fieldReference(rel, 0), b.i32(10)), scan);
+```
+
+`project`, `aggregate`, `sort`, and `join` follow the same pattern: the lambda
+receives the input relation (or, for joins, a `JoinInput` exposing `left()` and
+`right()`).
+
+## End-to-end example
+
+The following assembles a plan that scans a table, filters it, projects two
+columns, and wraps the result in a plan root with output names:
+
+```java
+import io.substrait.dsl.SubstraitBuilder;
+import io.substrait.plan.Plan;
+import io.substrait.relation.Filter;
+import io.substrait.relation.NamedScan;
+import io.substrait.relation.Project;
+import io.substrait.type.TypeCreator;
+import java.util.List;
+
+TypeCreator R = TypeCreator.REQUIRED;
+SubstraitBuilder b = new SubstraitBuilder();
+
+// 1. scan: table "t" with columns a (i32) and b (string)
+NamedScan scan =
+    b.namedScan(List.of("t"), List.of("a", "b"), List.of(R.I32, R.STRING));
+
+// 2. filter: keep rows where a == 10
+Filter filter =
+    b.filter(rel -> b.equal(b.fieldReference(rel, 0), b.i32(10)), scan);
+
+// 3. project: output columns a and b
+Project project =
+    b.project(
+        rel -> List.of(b.fieldReference(rel, 0), b.fieldReference(rel, 1)),
+        filter);
+
+// 4. root: name the plan's output columns
+Plan.Root root = b.root(project, List.of("a", "b"));
+
+// 5. plan: wrap the root
+Plan plan = b.plan(root);
+```
+
+!!! note
+    `root(input, names)` validates that the number of output names matches the
+    field count of the input relation, so keep the names list in sync with the
+    final projection.
+
+`b.plan(root)` uses a default execution behavior of
+`VariableEvaluationMode.PER_PLAN`. Overloads accept a custom
+`Plan.ExecutionBehavior` and/or multiple roots.
+
+## What the builder covers
+
+- **Relations** — `namedScan`, `filter`, `project`, `innerJoin` / `join`,
+  `aggregate`, `sort`, `fetch` / `limit` / `offset`, `cross`, `set`,
+  `emptyVirtualTableScan`, `namedWrite`, `namedUpdate`, `expand`, and the
+  physical joins `hashJoin` / `mergeJoin` / `nestedLoopJoin`. See
+  [Relations](relations.md).
+- **Expressions** — literal helpers (`bool`, `i8`..`i64`, `fp32`, `fp64`, `str`),
+  `fieldReference`(s), `cast`, arithmetic (`add`, `subtract`, `multiply`,
+  `divide`, `negate`), comparison/boolean (`equal`, `and`, `or`, `not`,
+  `isNull`), `ifThen`, `switchExpression`, and the generic `scalarFn` /
+  `aggregateFn` / `windowFn`. See [Expressions & literals](expressions.md).
+- **Aggregates** — `grouping`, `measure`, and the shortcuts `count`,
+  `countStar`, `sum`, `sum0`, `min`, `max`, `avg`.
+- **Plan assembly** — `root`, `plan`, and `remap` for output field remapping.
+
+Once you have a `Plan`, convert it to protobuf for storage or exchange as
+described in [Serialization](serialization.md).