Persistent workers + rules_kotlin KotlinBuilder drop-in (Elide 1.3.1) by sgammon · Pull Request #5 · elide-dev/rules_elide

Sam Gammon (sgammon) · 2026-06-14T19:35:06Z

Combines two features and brings the toolchain up to Elide 1.3.1+20260614.

1. Persistent worker compilation

elide javac/kotlinc compile actions run as Bazel persistent workers (proto protocol), enabled by default; --@rules_elide//elide:use_workers=false falls back to one-shot (same elide <tool> -- <args> arg form).
Single-action compile via --jar (Elide #993): collapses the old javac → elide jar two-action flow into one elide javac --jar <out> -- <args>.
Multiplex workers enabled (supports-multiplex-workers): verified on 1.3.1 to serve concurrent requests from one warm process — lifts the singleplex --worker_max_instances concurrency cap.
The 1.3.1 worker accepts the -- separator (Elide #994), so worker and one-shot share one arg form and the Bazel-native worker-off path (--strategy=local) works.

2. `rules_kotlin` KotlinBuilder drop-in

register_elide_kotlin_toolchain (from @rules_elide//elide/kotlin:toolchain.bzl) lets existing kt_jvm_library/_binary/_test targets compile through Elide with no BUILD migration — just a toolchain swap.
Plain Kotlin/mixed Java → Elide fast path; KAPT/KSP → transparent fallback to the stock rules_kotlin builder.
In-repo Kotlin shim (built with elide_kotlin_binary, dogfood) speaking the Bazel worker protocol via Java bindings generated from vendored Bazel worker_protocol.proto/deps.proto (rules_proto + protobuf); a wrapping kt_toolchain rule swaps only kotlinbuilder.
ABI jars via Elide's embedded jvm-abi-gen.

3. Benchmarks

bench_suite.sh (hyperfine): baseline vs elide vs elide+worker, cold/warm. RESULTS.md documents the honest finding — elide is 3.5–5.5× faster than the JVM baselines; persistent workers are ~wall-clock-neutral for a native-image tool.

Deferred

jdeps / strict-deps: 1.3.1's --report-used-deps is wired but broken (empty report — jdeps resource bundle missing in the native image, WHIPLASH #1002). The shim keeps a valid stub .jdeps and strict_kotlin_deps stays off until #1002 lands.

Upstream issues filed

WHIPLASH #993 (--jar, fixed), #994 (worker --, fixed), #998 (used-deps reporting), #1002 (used-deps empty report), #1004 (officially support multiplex workers).

Notable

Adds rules_kotlin, rules_proto, protobuf as deps (needed to build the shim consumers use); JUnit/rules_jvm_external are dev-only.

Test plan

bazelisk test //tests/... → 31/31 pass
e2e/integration builds on 1.3.1 (worker, multiplex, one-shot, --strategy=local)
e2e/kotlin_builder fast path (//:greeter) + KAPT fallback (//:annotated) build on 1.3.1
buildifier clean
//:native_app native-image — pre-existing env gap (JAVA_HOME), unrelated

Signed-off-by: Sam Gammon <sam@elide.dev>

…der shim Plan for an in-repo, rules_kotlin-compatible KotlinBuilder that compiles plain kt_jvm_library targets through Elide and falls back to the stock builder for KAPT/KSP. Worker + jdeps protos generated from Bazel's vendored protos via rules_proto + java_proto_library (rules_buf for lint).

…e test Adds a dev-scoped Maven dependency set (rules_jvm_external 6.7, dev_dependency=True) with kotlin-test 2.4.0, kotlin-test-junit5, JUnit Jupiter 5.11.3, and junit-platform-console-standalone 1.11.3. Also installs the real elide toolchain (registered before the stub) so tests/kotlin_builder can actually compile and run. Smoke test SmokeTest.kt verifies the harness end-to-end: PASS.

Introduces `elide/kotlin/builder/Flagfile.kt` with `CompileRequest` (typed data class) and `Flagfile` object providing `parse(tokens)` + `readTokens(path)`. Covered by `FlagfileTest` (two test methods: scalar/repeated flags and KAPT processor capture). TDD: test written first; build failed on missing impl, then passed after impl added.

Adds ElideCompile.plan() (pure, TDD-covered) that constructs the elide kotlinc one-shot argv following the same -d/-classpath/-module-name/-Xfriend-paths form as run_kotlinc() in compile_common.bzl, plus an optional elide jar srcjar command. ElideCompile.run() executes the plan sequentially. Leaves a TODO seam for the verified jvm-abi-gen ABI jar invocation (Step 4 de-risk).

Adds ElideCompile.plan() (pure, TDD-covered) that constructs the elide kotlinc one-shot argv following the same -d/-classpath/-module-name/-Xfriend-paths form as run_kotlinc() in compile_common.bzl, plus an optional elide jar srcjar command. ElideCompile.run() executes the plan sequentially. De-risk step 4 complete: jvm-abi-gen ABI jar invocation verified against Elide 1.3.0 / Kotlin 2.4.0. The plugin jar lives at lib/resources/kotlin/<ver>/lib/jvm-abi-gen.jar in the Elide distribution (plugin ID: org.jetbrains.kotlin.jvm.abi). plan() now injects -Xplugin=<jar> -P plugin:org.jetbrains.kotlin.jvm.abi:outputDir=<abi.jar> into the kotlinc command when abiJar is set and the jar is discoverable. Degrades gracefully when the jar is absent.

…stdout deadlock When `abiJar` is requested but `findJvmAbiGenJar()` returns null, `plan()` now throws `IllegalStateException` with a clear message instead of silently omitting the plugin flags (which previously caused an opaque Bazel missing-output error downstream). `run()` now redirects subprocess stdout+stderr to a temp file instead of reading from the pipe inline, eliminating the potential deadlock when subprocess output exceeds the ~64KB OS pipe buffer. Output on failure is forwarded to stderr, never stdout, preserving the Bazel persistent-worker WorkResponse protocol on the shim's own stdout.

…ibrary bindings - Vendor worker_protocol.proto and deps.proto from Bazel 7.4.1 verbatim - Add rules_proto@7.1.0 and protobuf@33.4 as regular (non-dev) bazel_deps - Generate java_proto_library bindings for both protos - Add minimal buf.yaml (buf lint deferred; rules_buf 0.5.x pulls rules_go which is incompatible with Bazel 9.1 CcInfo removal)

…ings Add Worker object wrapping length-delimited read/write of WorkRequest/WorkResponse proto messages; wire worker_protocol_java_proto into lib and worker_test deps.

…k + jdeps

…auncher

…vm_library Wire the Elide KotlinBuilder shim through real rules_kotlin end-to-end. PART 1 (fast path): make elide_kotlin_binary's launcher work as a persistent worker. Bazel runs workers from the execroot with a scrubbed env, so the old short_path launcher could not resolve the elide binary or classpath jars. Rewrite build_launcher's SH template to use the bash runfiles library (rlocation), which works both via `bazel run` and as a worker. The toolchain launcher additionally exports JAVA_HOME (derived from the Bazel JDK's bin/java rlocation) so `elide java` finds a JVM under the scrubbed worker env, and propagates RUNFILES_DIR/JAVA_RUNFILES so the fallback java_stub locates its runfiles. PART 2 (flagfile schema): captured the real rules_kotlin KotlinBuilder flagfile. The module-name flag is --kotlin_module_name, not --module_name; fix Flagfile.kt and its unit test accordingly. All other parsed flags (--output, --kotlin_output_jdeps, --sources, --classpath, --kotlin_passthrough_flags, --direct_dependencies, --strict_kotlin_deps, --abi_jar, --kotlin_friend_paths, --processors, --processorpath) match. PART 3 (KAPT fallback): add a trivial Java annotation processor and a kt_jvm_library that uses it. rules_kotlin splits this into a KotlinKapt gensrc action (flagfile carries --processors -> Router delegates to the stock @rules_kotlin//src/main/kotlin:build) and a post-KAPT KotlinCompile (no --processors -> Elide fast path). Both go through our launcher; the generated class lands in the output jar.

…e job

…nal review Apply final-polish fixes: - Fix out-of-order-load in elide/kotlin/builder/proto/BUILD.bazel - Fix unsorted-dict-items in elide/kotlin/toolchain.bzl attrs - Fix unsorted-dict-items in elide/private/compile_common.bzl attrs - Add clarifying comment in Main.kt about singleplex worker assumption - Add clarifying comment in compile_common.bzl about test launcher asymmetry All 30/30 tests pass. No semantic changes.

…n, jdeps framing)

Run the elide javac/kotlinc compile actions as Bazel persistent workers (singleplex, proto protocol) by default: each action advertises supports-workers and delivers per-request TOOL_ARGS as a multiline params-file WorkRequest, with Bazel injecting the --persistent_worker startup flag itself. Add a --@rules_elide//elide:use_workers flag (default true). When false, compile each target as a one-shot `elide <tool> -- <args>` process. This is the supported way to run without workers, since the Bazel-native worker-off path (--worker_max_instances=0, --strategy=local) hits a broken upstream standalone mode (WHIPLASH #994). Update the analysis tests for the worker arg form and add a workers-off variant. Docs note that workers are wall-clock-neutral for elide (a native image with ~12ms startup); the real win is elide vs the JVM baselines.

…ker) Add bench_suite.sh: a hyperfine-driven comparison of vanilla rules_java/ rules_kotlin vs elide (workers off) vs elide+worker, for Java and Kotlin across cold (clean) and warm (incremental) regimes, exporting Markdown + JSON under benchmarks/results/ for screenshots. Extend bench.sh with cold/warm regimes and rewrite RESULTS.md with the measured, honest finding: elide is ~3.5-5.5x faster than the baselines, while persistent workers are wall-clock-neutral for elide (native binary, ~12ms startup, nothing to amortize). Document the singleplex parallelism caveat and the use_workers=false non-worker path.

Brings --jar fix (#993), --report-used-deps for jdeps (#998), and flag parsing fixes (#994). Hashes verified against the GitHub release.

…s broken, #1002) Elide 1.3.1 added --report-used-deps but it writes an empty report (jdeps resource bundle missing in the native image, WHIPLASH #1002), so the shim keeps its stub .jdeps and strict_kotlin_deps stays off. Recorded at the code, docs, and changelog sites.

…kers (1.3.1)

…#994 resolved)

…@@ ref The no-worker analysistest must reference the use_workers flag as @@//elide:... (canonical root) because analysis_test_transition resolves config_settings labels from bazel_skylib's context. buildifier 8.x flags @@ as canonical-repo; suppress it inline since the @@ form is required here.

) #6: ElideCompile.plan dropped CompileRequest.sourceJars, so a post-KAPT fast-path compile could not resolve references to generated symbols (e.g. Truffle DSL *NodeGen). Now Main.handle unpacks --source_jars into a temp dir and adds the extracted .kt/.java to the kotlinc source set for resolution (generated classes still come from the KAPT generated_class_jar). The e2e //:annotated target now references its generated class to guard the regression. #7: ElideCompile.run swallowed the subprocess's merged stdout+stderr to System.err, which the worker protocol cannot carry, so fast-path compile failures surfaced with no output under workers/RBE. run() now returns the captured output (like Fallback.run); Main.handle puts it in the WorkResponse. Adds unit tests for source-jar extraction, extra-source wiring, and run() output capture.

extractSourceJars wrote each entry to File(into, entry.name) without validating containment, so a crafted source jar (e.g. from a malicious processor/dependency) with '../' entries could write outside the temp dir (arbitrary file write during the build). Canonicalize the destination and verify it stays within 'into', rejecting escaping entries. Adds a regression test.

Copilot

Pull request overview

This PR upgrades the Elide toolchain to 1.3.1+20260614 and introduces two major build-speed/interop capabilities: Bazel persistent worker execution for elide javac/kotlinc, and a rules_kotlin KotlinBuilder shim + toolchain wrapper that allows kt_jvm_* targets to compile via Elide with only a toolchain swap.

Changes:

Add worker-enabled compile actions for Elide Java/Kotlin compilation (including the elide javac --jar single-action flow) plus a //elide:use_workers build setting toggle.
Add an in-repo KotlinBuilder shim (worker protocol + routing + fallback) and register_elide_kotlin_toolchain wrapper to swap only kotlinbuilder in a rules_kotlin toolchain.
Add tests, docs, benchmarks updates, and CI wiring for the new KotlinBuilder e2e workspace.

Reviewed changes

Copilot reviewed 48 out of 49 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/kotlin_rule_test.bzl	Updates analysis tests for unified `--` arg form and ensures rules don’t pass `--persistent_worker`.
tests/kotlin_builder/WorkerTest.kt	Adds unit test coverage for WorkRequest/WorkResponse delimited protobuf round-tripping.
tests/kotlin_builder/SmokeTest.kt	Adds minimal test harness validation for Kotlin/JUnit wiring.
tests/kotlin_builder/RouterTest.kt	Tests routing decisions for fast-path vs fallback compilation.
tests/kotlin_builder/MainTest.kt	Tests CLI config splitting for the KotlinBuilder shim.
tests/kotlin_builder/JdepsTest.kt	Validates `.jdeps` stub output is a parsable `Deps.Dependencies` proto.
tests/kotlin_builder/FlagfileTest.kt	Tests parsing of rules_kotlin-style flagfiles into typed request fields.
tests/kotlin_builder/FallbackTest.kt	Tests the fallback command-line is passed through verbatim.
tests/kotlin_builder/ElideCompileTest.kt	Tests Elide command planning, ABI plugin wiring, srcjar, source jar extraction, and output capture behavior.
tests/kotlin_builder/BUILD.bazel	Adds Bazel test targets for KotlinBuilder shim unit tests.
tests/java_rule_test.bzl	Updates tests for `--jar` single-action javac flow and adds a no-worker analysis test.
README.md	Documents the new Kotlin toolchain swap entry and updates roadmap/compatibility notes.
MODULE.bazel	Adds `rules_kotlin`, `rules_proto`, `protobuf`, and dev-only JVM test deps via `rules_jvm_external`.
elide/private/versions.bzl	Adds Elide 1.3.0/1.3.1 entries and updates `DEFAULT_VERSION` to 1.3.1+20260614.
elide/private/compile_common.bzl	Implements worker execution requirements, shared compile runner, `--jar` javac flow, and runfiles-based launchers.
elide/private/BUILD.bazel	Adds skylib common_settings dependency needed for BuildSettingInfo usage.
elide/kotlin/toolchain.bzl	Adds `register_elide_kotlin_toolchain` wrapping `define_kt_toolchain` and swapping `kotlinbuilder` via a launcher.
elide/kotlin/builder/Worker.kt	Adds Worker protocol IO helpers around generated proto bindings.
elide/kotlin/builder/Router.kt	Implements fast-path vs fallback routing for processors/KSP detection.
elide/kotlin/builder/proto/worker_protocol.proto	Vendors Bazel worker protocol proto for generated Java bindings.
elide/kotlin/builder/proto/deps.proto	Vendors Bazel deps proto for `.jdeps` stub writing.
elide/kotlin/builder/proto/BUILD.bazel	Defines proto_library + java_proto_library targets for worker/deps protos.
elide/kotlin/builder/proto/buf.yaml	Adds buf lint configuration for vendored protos.
elide/kotlin/builder/Main.kt	Adds worker/one-shot entrypoint, routing, source-jar extraction, and compilation execution.
elide/kotlin/builder/Jdeps.kt	Implements `.jdeps` stub writing using generated `Deps.Dependencies`.
elide/kotlin/builder/Flagfile.kt	Implements tokenization and typed parsing of rules_kotlin flagfile inputs.
elide/kotlin/builder/Fallback.kt	Implements fallback delegation to stock KotlinBuilder with captured output.
elide/kotlin/builder/ElideCompile.kt	Implements command planning/execution, ABI plugin wiring, source jar extraction, and output capture.
elide/kotlin/builder/BUILD.bazel	Builds the KotlinBuilder shim (dogfooding elide_kotlin_binary/library).
elide/kotlin/BUILD.bazel	Exposes and packages the Kotlin toolchain Starlark module.
elide/BUILD.bazel	Adds `//elide:use_workers` bool build setting.
e2e/kotlin_builder/sample/Greeter.kt	Adds a plain Kotlin target for toolchain-swap fast-path validation.
e2e/kotlin_builder/sample/Annotated.kt	Adds a KAPT-driven sample to validate transparent fallback behavior.
e2e/kotlin_builder/proc/MarkerProcessor.java	Adds a minimal annotation processor for KAPT fallback e2e coverage.
e2e/kotlin_builder/proc/Marker.java	Adds marker annotation used by the e2e processor.
e2e/kotlin_builder/MODULE.bazel	Creates standalone bzlmod workspace to test toolchain swap end-to-end.
e2e/kotlin_builder/BUILD.bazel	Registers Elide-backed Kotlin toolchain and defines fast-path + KAPT targets.
e2e/kotlin_builder/.bazelversion	Pins Bazel version for the e2e workspace.
e2e/kotlin_builder/.bazelrc	Pins hermetic JDK 21 and strict action env for the e2e workspace.
docs/superpowers/plans/2026-06-13-elide-kotlin-builder.md	Adds an implementation plan document for the KotlinBuilder shim.
docs/kotlin_builder.md	Documents how to use the Kotlin toolchain swap and current limitations.
docs/extensions.md	Updates docs for default Elide version in `elide.install`.
CHANGELOG.md	Documents KotlinBuilder interop and persistent worker/javac --jar updates.
benchmarks/RESULTS.md	Updates benchmark write-up to include worker on/off regimes and suite tooling.
benchmarks/BUILD.bazel	Refactors benchmark source globbing into named constants.
benchmarks/bench.sh	Updates benchmark script to measure cold vs warm rebuild regimes.
benchmarks/bench_suite.sh	Adds hyperfine-based 3-way benchmark suite (baseline vs elide vs elide+worker).
.gitignore	Ignores benchmark results export directory.
.github/workflows/ci.yml	Adds a dedicated kotlin-builder e2e CI job (non-PR).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

        providers = [[JavaInfo]],
    ),
+    "_use_workers": attr.label(
+        default = "@rules_elide//elide:use_workers",


+ * The command form mirrors how rules_elide invokes elide in compile_common.bzl:
+ *   elide kotlinc -- -d <out> -classpath <cp> [-module-name <m>] [-Xfriend-paths=...] <passthrough> <srcs>
+ *
+ * The `--` separator is required for one-shot invocations (worker mode omits it).


+        try {
+            val extraSources = srcJarDir?.let { ElideCompile.extractSourceJars(req.sourceJars, it) } ?: emptyList()
+            val (code, out) = ElideCompile.run(ElideCompile.plan(req, cfg.elide, extraSources), wd)
+            if (code == 0) req.jdeps?.let { Jdeps.writeStub(it, req.moduleName ?: "") }


Brings the jdeps used-deps fix (#1002/#1005) and milestone-6 changes. Hashes verified against the GitHub release.

…options/kotlin version (#8) jdeps (WHIPLASH #1002/#1005, Elide 1.3.2): the shim now passes --report-used-deps and writes a real Deps proto classifying each classpath entry EXPLICIT/IMPLICIT/UNUSED (was a stub); falls back to empty when there is no classpath. Enables unused_deps / reduced-classpath downstream. #8: Flagfile now parses --compiler_plugin_options / --stubs_plugin_options / --kotlin_api_version / --kotlin_language_version; ElideCompile.plan forwards compiler-plugin -Xplugin/-P options (deduped against passthrough) and -api-version / -language-version. Fixes optioned plugins (Metro) and the silently-relaxed Kotlin version on the fast path. Adds unit tests for plugin-option/version forwarding, the --report-used-deps flag placement, and Deps classification.

Adds an elide.use module-extension tag (precedence over elide.install) so a consumer can test against a custom or local Elide without a versions.bzl entry: - BYO release: version + url_template + integrity (per-platform <os>_<cpu> -> SRI). integrity is authoritative (no versions.bzl lookup; unknown version no longer fails) and only its platforms get a toolchain; hashes still enforced. - Local: local_path points at an already-extracted distribution; a new elide_local repo rule symlinks it and the host-platform toolchain uses it with no download (non-reproducible). hub now registers only the platforms that have a repo (new platforms attr; defaults to all, so the install path is unchanged). Verified end-to-end: local compile via a real dir, BYO download with enforced/rejected hashes; install path still builds.

Replaces the env-var wrapper + --config=dev action_env hack with the first-class elide.use(local_path=...) override. The smoke workspace keeps a plain no-op stub toolchain for the analysis-only default; real execution now uses elide.use(local_path) (verified: real Java+Kotlin compile). Drops the :dev bazelrc config and the ELIDE_DEV_BIN wrapper script.

Re-tested on 1.3.3: --jar (#993), jdeps --report-used-deps (#1002), and the KotlinBuilder shim + KAPT fallback all still work; full suite + both e2e workspaces green. native-image still requires external JAVA_HOME (#1016/#1042 not yet effective). Hashes verified against the release.

Copilot

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 3 comments.

+            if (code == 0) req.jdeps?.let { jdeps ->
+                if (usedReport != null && usedReport.exists()) {
+                    Jdeps.write(jdeps, req.moduleName ?: "", req.classpath, req.directDependencies, usedReport)
+                } else {
+                    Jdeps.writeStub(jdeps, req.moduleName ?: "")
+                }
+            }


+ * The command form mirrors how rules_elide invokes elide in compile_common.bzl:
+ *   elide kotlinc -- -d <out> -classpath <cp> [-module-name <m>] [-Xfriend-paths=...] <passthrough> <srcs>
+ *
+ * The `--` separator is required for one-shot invocations (worker mode omits it).
+ */


+        return versionDir.resolve("lib/jvm-abi-gen.jar").takeIf { it.isFile }
+    }
+
+    /**


…-dir) Gate behind //config/kotlinc:incremental (default off). When enabled, elide_kotlin_* compiles to a classes directory with --incremental --incremental-cache-dir <per-target> and packs the result into the output jar, so a warm rebuild after a small edit recompiles only the dirty subset (kotlinc IC requires a directory output, not a jar). The cache is undeclared worker-scoped scratch under bazel-out: it persists across persistent-worker requests when unsandboxed, and degrades to a correct cold compile under --worker_sandboxing/RBE. javac is unaffected (no upstream IC). Verified: IC off is unchanged (31/31 tests, native rules build); IC on carries the right argv, persists per-target build-history.bin, and a body edit rebuilds incrementally with correct output. Elide-side engine tracked in WHIPLASH#1112.

run.sh walks a sequence of module snapshots (body edit, const/inline inlined into untouched callers, signature change + caller update, add/delete file) and asserts the IC-reused output is byte-identical to a fresh-cache build of the same sources — catching silent staleness. It runs elide from an ancestor of the sources (as Bazel does from the exec root), sidestepping WHIPLASH#1113. wl1113_regression.sh guards #1113 itself: one-shot --incremental reuse looped when the source was outside the CWD. Runs elide from a non-ancestor dir under a hard timeout; green when fixed (verified on elide 1.3.3+15c568841), red if it regresses. Both drive a binary via $ELIDE; no toolchain wiring required.

Add a builtin_plugins string_list on elide_kotlin_library/_binary/_test that forwards `--plugins=<csv>` to elide kotlinc (an Elide option, before the `--` separator) to force on builtin compiler plugins by name (serialization, metro, atomicfu, power-assert). Robust for plugins the classpath heuristic may miss, notably Metro under a worker flagfile. Analysis test asserts the action argv carries `--plugins=serialization,metro` before `--`. Verified end-to-end on a dev build: a builtin_plugins=[serialization] target builds via the worker and emits the generated serializer. Note: an explicit suite does not yet fully disable heuristic-detected plugins it omits (WHIPLASH#1119).

Add ElideWorker: a long-lived `elide kotlinc --persistent_worker` subprocess the builder forwards compiles to (length-delimited WorkRequest/WorkResponse over stdin/stdout, the same proto the builder speaks to Bazel), instead of spawning elide one-shot per compile. This keeps the native image and Elide's digest-driven incremental caches warm across requests — the persistent-worker caching win for rules_kotlin consumers, who previously paid full native-image startup per compile. - ElideCompile.kotlincWorkerArgs strips the `elide kotlinc` prefix but KEEPS the `--` (the resident worker uses it to split elide-options from tool-args, as the native _run_elide_compile worker path delivers them). - Main forwards the kotlinc compile and per-request inputs (path+digest, so Elide's digest IC engages); the jar/srcjar step stays one-shot; jdeps unchanged. - Any resident failure falls back to a one-shot invocation and restarts the resident next request — a dead pipe never wedges. close() shuts the resident's stdin (clean EOF) so an instrumented binary flushes its PGO profile. - ELIDE_WORKER_RECORD tees forwarded requests for offline PGO replay. Validated end-to-end on a dev build: forwarding engages (resident elide child of the builder JVM), greeter + annotated build correctly, and annotated.jdeps stays a real Deps proto through the forwarded path. Tracks the persistent-worker PGO spec; WHIPLASH#1107/#1112.

…oolchain flag Bazel launches workers with a scrubbed environment, so an env toggle never reaches the shim. Gate resident forwarding on a startup flag instead: the toolchain launcher injects `--resident_worker` when register_elide_kotlin_toolchain(resident_worker = True), and Main.splitConfig parses it into Config.resident. Forwarding is off by default; opt in per toolchain. MainTest covers the flag parsing. Validated: with resident_worker = True the generated launcher carries --resident_worker and greeter builds through the forwarded resident worker.

…ent to IC flag - --worker_record=<path> startup flag tees the forwarded WorkRequest stream for offline PGO replay (launcher injects it; env ELIDE_WORKER_RECORD kept as a fallback for direct/replay runs). Startup flag, not env: Bazel scrubs worker env. - ElideWorker always spawns `elide --safe-close kotlinc --persistent_worker` so a --pgo-instrument binary flushes default.iprof on clean stdin-EOF shutdown; harmless otherwise. - register_elide_kotlin_toolchain(resident_worker) now defaults to the value of //config/kotlinc:incremental (new :incremental_enabled config_setting): IC is only useful with a warm resident worker, so enabling IC turns forwarding on. Verified end-to-end — building with --@rules_elide//config/kotlinc:incremental=True flips the generated launcher to inject --resident_worker through exec config. MainTest covers --resident_worker and --worker_record parsing; 32/32 tests pass.

benchmarks/pgo/ — collect.sh drives a PGO-instrumented elide over the 50-file self-contained sources/kotlin/sample in four modes (one-shot, jvm-abi-gen, karbine/--abi-only, persistent worker), collecting each flushed default.iprof to profiles/<case>.iprof (set WHIPLASH_PROFILES to also copy as kotlinc-<case>.iprof for a --pgo rebuild). gen_workrequest.py hand-encodes Bazel WorkRequests (no protobuf dep) to drive the worker. --safe-close is always passed — it is what flushes the profile on clean shutdown in every mode. benchmarks/compile_modes.sh — hyperfine microbenchmark isolating the compiler (vs bench_suite.sh's whole-build wall-clock): full compile and Karbine, each cold (fresh process) and warm (marginal per-compile on a warm persistent worker, (t[K]-t[1])/(K-1)). profiles/ is gitignored (~50MB each).

Rename the WHIPLASH_PROFILES env var to ELIDE_PROFILES and replace codename prose ('WHIPLASH side/tree/checkout') with 'Elide' in the PGO harness. Issue references (WHIPLASH#NNNN, which point at the tracker) are left intact.

The native elide_java_* rules already run elide javac as a Bazel persistent worker (ElideJavac/ElideKotlinJavac advertise supports-workers). Add the cache half: //config/javac:classpath_cache (opt-in, default off, matching upstream's correctness-first stance) makes run_javac and _compile_java_aux pass `--classpath-cache` (before `--`) so the worker reuses parsed classpath state across warm compiles, keyed by input digests. Analysis tests cover both the default-off and flag-on argv. benchmarks/javac_cache.sh measures the delta over the elide dist's own lib jars. Honest finding: the cache's saving is dominated by cold classpath I/O; once the OS page cache holds the jars (normal repeated build), warm re-indexing is cheap and the steady-state delta is ~parity. The win is on cold / page-cache-evicted classpaths (large dep sets, memory-pressured CI). javac persistent worker + --classpath-cache verified present in elide 1.3.4+d0a074f2d.

gen.py emits a self-contained Bazel workspace (gitignored ws/) of a layered DAG on the native elide_{kotlin,java} rules: a core module every layer depends on (ABI/IC traps: inline fun + const / static final), a chain of L modules x W files (depth + width), and an app sink. Built to surface work-avoidance effects a trivial fixture can't. Findings at 16x32 (~545 files/lang) on elide 1.3.4: a core method-BODY edit re-runs all 18 kotlin compiles (no compile-avoidance) vs 3 for the same Java edit (Java prunes correctly); within-target IC is net-negative (declared classes-dir is wiped each run, forcing full re-emit). These are the next 10x levers. README documents topology + experiments + findings.

Clean controlled experiment confirms: a core method-body edit changes the kotlin compile_jar because it is derived with java_common.run_ijar, which does not produce a body-stable ABI for Kotlin bytecode — so all 18 dependents recompile (Java's ijar is body-stable -> prunes to 3). Verified that jvm-abi-gen emits a byte-identical ABI across the same body edit. Fix: use jvm-abi-gen for the kotlin compile_jar.

…l wiring Pursued the IC fix; measurement refutes the wiring hypothesis. Standalone (no Bazel), elide kotlinc --incremental on a 1-file edit in an inference-heavy module is ~2x SLOWER than a full compile (~5.1s vs ~2.6s) with EITHER a persistent or wiped -d plus a persistent cache — so IC does full-compile work plus overhead and does not prune to the changed file. The earlier apparent IC win was a process/page-cache warming artifact on trivial files. No rules_elide fix applies; keep IC off-by-default until the Elide engine beats a full compile. A persistent worker dir does not help (verified).

run_all.sh drives the full suite (delegating to the existing per-benchmark scripts, section-selectable): compile modes, incremental, javac cache, deep graph, Bazel 3-way, pgo. Two new pieces: - incremental.sh: kotlinc IC benchmark — full compile vs a 1-file-edit rebuild on an inference-heavy module, process/page-cache pre-warmed, with each edit VERIFIED real (changed class bytecode must differ) so a no-op can't fake a win. - deepgraph/bench.sh: generates the layered fixture (gen.py) and measures clean-build depth + cross-target ABI compile-avoidance (kotlin body-edit cascade vs java prune — Bug 1).

Latest nightly from elide-dev/elide (was 1.3.3+20260619). All 4 platform SRI hashes verified against the published .sha256 sidecars. Regenerated docs; versions_test + DEFAULT_VERSION resolution pass (33/33).

It was an untracked local design note pulled in by a broad 'git add docs/'; it also still carries WHIPLASH codename references. Keep it local/untracked until it's intentionally committed (and scrubbed).

Sam Gammon (sgammon) added 28 commits June 13, 2026 13:15

chore: update nightly pins

f82d464

Signed-off-by: Sam Gammon <sam@elide.dev>

test(kotlin-builder): e2e workspace with stock rules_kotlin control

43751f4

feat(kotlin-builder): route KAPT/KSP requests to stock-builder fallback

0dcd0d9

feat(kotlin-builder): fallback delegates flagfile to stock KotlinBuilder

35bc9b8

feat(kotlin-builder): worker codec over generated WorkerProtocol bind…

28199c6

…ings Add Worker object wrapping length-delimited read/write of WorkRequest/WorkResponse proto messages; wire worker_protocol_java_proto into lib and worker_test deps.

feat(kotlin-builder): write .jdeps via generated Deps proto (v1 stub)

65c253c

feat(kotlin-builder): worker/one-shot dispatch wiring elide + fallbac…

2456e6a

…k + jdeps

feat(kotlin-builder): package shim as elide_kotlin_binary

2544fec

feat(kotlin-builder): wrapping kt_toolchain rule + config-injecting l…

2816ebc

…auncher

docs(kotlin-builder): document rules_kotlin drop-in toolchain + CI e2…

1f6023d

…e job

style(kotlin-builder): sort dict items to satisfy buildifier

55a09b9

docs(kotlin-builder): plan updates (proto approach, wrapping toolchai…

5a214f4

…n, jdeps framing)

chore: bump Elide nightly to 1.3.1+20260614

ee654f5

Brings --jar fix (#993), --report-used-deps for jdeps (#998), and flag parsing fixes (#994). Hashes verified against the GitHub release.

feat(workers): single-action elide javac --jar + enable multiplex wor…

f5f44ea

…kers (1.3.1)

docs: record 1.3.1 worker adoptions (--jar single-action, multiplex, …

1bc71bd

…#994 resolved)

Sam Gammon (sgammon) force-pushed the feat/kotlin-builder branch from 57d8ac4 to 690150d Compare June 14, 2026 22:16

This was linked to issues Jun 15, 2026

kt-builder: ElideCompile drops --source_jars (KAPT/generated sources) → unresolved references on annotation-processed kt_jvm_library #6

Closed

kt-builder: ElideCompile.run() swallows compiler stderr under persistent worker (silent exit 1) #7

Closed

Sam Gammon (sgammon) requested a review from Copilot June 15, 2026 00:16

Copilot started reviewing on behalf of Sam Gammon (sgammon) June 15, 2026 00:16 View session

Copilot AI reviewed Jun 15, 2026

View reviewed changes

Sam Gammon (sgammon) added 5 commits June 15, 2026 12:48

chore: bump Elide nightly to 1.3.2+20260615

4b42cfd

Brings the jdeps used-deps fix (#1002/#1005) and milestone-6 changes. Hashes verified against the GitHub release.

Sam Gammon (sgammon) requested a review from Copilot June 19, 2026 18:10

Copilot started reviewing on behalf of Sam Gammon (sgammon) June 19, 2026 18:10 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Sam Gammon (sgammon) added 15 commits June 20, 2026 13:59

chore: bump Elide nightly to 1.3.4+20260622

dfa884d

Latest nightly from elide-dev/elide (was 1.3.3+20260619). All 4 platform SRI hashes verified against the published .sha256 sidecars. Regenerated docs; versions_test + DEFAULT_VERSION resolution pass (33/33).

chore: untrack PGO spec doc accidentally swept into the nightly bump

3c2b5a3

It was an untracked local design note pulled in by a broad 'git add docs/'; it also still carries WHIPLASH codename references. Keep it local/untracked until it's intentionally committed (and scrubbed).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persistent workers + rules_kotlin KotlinBuilder drop-in (Elide 1.3.1)#5

Persistent workers + rules_kotlin KotlinBuilder drop-in (Elide 1.3.1)#5
Sam Gammon (sgammon) wants to merge 50 commits into
mainfrom
feat/kotlin-builder

Sam Gammon (sgammon) commented Jun 14, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Sam Gammon (sgammon) commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. Persistent worker compilation

2. rules_kotlin KotlinBuilder drop-in

3. Benchmarks

Deferred

Upstream issues filed

Notable

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sam Gammon (sgammon) commented Jun 14, 2026 •

edited

Loading

2. `rules_kotlin` KotlinBuilder drop-in