Skip to content

Migrate TensorBoard builds to Bzlmod on Bazel 7.7.0 - #7145

Open
psamanoelton wants to merge 4 commits into
tensorflow:masterfrom
psamanoelton:workspace_upgrade
Open

Migrate TensorBoard builds to Bzlmod on Bazel 7.7.0#7145
psamanoelton wants to merge 4 commits into
tensorflow:masterfrom
psamanoelton:workspace_upgrade

Conversation

@psamanoelton

Copy link
Copy Markdown
Contributor

Summary

This change migrates TensorBoard’s build to Bazel Modules (Bzlmod) while keeping Bazel pinned to 7.7.0.

The migration preserves compatibility with TensorFlow 2.21.0 and Protobuf 6.31.1, makes Bzlmod the only supported dependency-resolution mode, and adds a hermetic Bazel-side Python environment.

This is intentionally not a Bazel 8 upgrade.

Motivation

TensorBoard’s TensorFlow 2.21.0 release required several non-trivial dependency updates and compatibility workarounds. Continuing to manage the dependency graph primarily through WORKSPACE makes future upgrades difficult because versions and transitive dependencies are not resolved consistently.

Moving the supported build to Bzlmod provides:

  • Explicit module versions and dependency resolution
  • A checked-in module lockfile
  • Better isolation from host Python installations
  • Fewer manually initialized transitive repositories
  • A clearer path for future dependency upgrades

Main changes

Enable Bzlmod on Bazel 7.7.0

  • Add MODULE.bazel and MODULE.bazel.lock.
  • Pin Bazel to 7.7.0.
  • Enable Bzlmod by default in .bazelrc.
  • Mark legacy --noenable_bzlmod builds as unsupported.
  • Explicitly constrain the module graph to Bazel 7.x.
  • Update CI configuration to use Bazel 7.7.0.

Move core dependencies to modules

The module graph now directly manages:

  • bazel_skylib 1.7.1
  • rules_cc 0.1.1
  • rules_java 8.6.1
  • rules_python 1.0.0
  • Protobuf 31.1 / release 6.31.1
  • gRPC 1.74.0
  • grpc-java 1.69.0
  • aspect_rules_js 2.1.0
  • rules_webtesting 0.4.1
  • rules_web_testing_python 0.4.1

Migrating rules_webtesting also removes the old explicit setup for rules_go, Gazelle, legacy rules_python, Bazel Skylib, and browser repositories.

Add a hermetic Python environment

  • Configure a downloaded, checksummed Python 3.10 toolchain.
  • Add a locked pip dependency graph for Bazel-built Python targets.
  • Prevent host PYTHONPATH, user-site packages, and virtualenv state from
    leaking into Bazel tests.
  • Align the Python web-testing dependencies with Python 3.10.
  • Move TensorBoard’s Markdown, Bleach, and Webencodings source repositories
    behind a local module extension.
  • Replace the legacy Werkzeug repository with the locked pip dependency.
  • Remove the unused standalone urllib3 repository.

The pip-package smoke test continues to create its own isolated environment because it validates the produced TensorBoard wheel rather than Bazel targets.

Preserve TensorFlow 2.21 / Protobuf compatibility

  • Consume Protobuf 6.31.1 through a module source override.
  • Preserve the required Protobuf Java and Python compatibility patches.
  • Use rules_cc 0.1.1, which includes the required cc_proto_library
    correction.
  • Correct the grpc-java module dependency graph.
  • Regenerate the Rust data-server descriptor with the current Protobuf schema.
  • Update affected BUILD dependencies and compatibility targets.

All compatibility patches and their expected removal conditions are documented in patches/README.md.

Update Bzlmod runfiles handling

Bzlmod changes the runfiles layout for the root module and module-extension repositories.

The affected shell utilities now support the _main layout, including:

  • Pip package building and extraction
  • Example plugin smoke tests
  • Logo generation
  • HParams utilities

The wheel builder resolves module-extension repositories through Bazel’s _repo_mapping file. This allows vendored Bleach and Webencodings sources to work without hardcoding their canonical Bzlmod repository names.

Keep a transitional WORKSPACE.bzlmod

Some dependencies cannot be migrated safely without substantially expanding the scope of this change. WORKSPACE.bzlmod remains as a compatibility bridge for:

  • The patched Closure/Soy stack
  • Legacy rules_nodejs 5.8.1, yarn_install, and concatjs
  • rules_sass
  • The existing rules_rust and cargo-raze graph
  • Closure-dependent font and JavaScript repositories
  • Remaining Java artifacts and local compatibility repositories

The original WORKSPACE remains for repository compatibility and external tooling, but it is not a supported build mode.

Validation

The following workflows pass locally on Linux:

pip install "tensorflow==2.21.0"
pip install \
  -r ./tensorboard/pip_package/requirements.txt \
  -r ./tensorboard/pip_package/requirements_dev.txt

pip install --force-reinstall "urllib3==1.26.20" "six>=1.12,<2"

bazel build //tensorboard/... --keep_going

bazel test //tensorboard/... --test_output=errors

bazel run //tensorboard/pip_package:test_pip_package -- \
  --tf-version "tensorflow==2.21"

bazel build //tensorboard/pip_package:build_pip_package

mkdir -p /tmp/tb_wheel
./bazel-bin/tensorboard/pip_package/build_pip_package /tmp/tb_wheel
pip install /tmp/tb_wheel/tensorboard-*.whl

Additional validation included:

  • bazel mod tidy
  • Full //tensorboard/... target analysis
  • Functional web-test target analysis with Python 3.10
  • Frontend target analysis
  • Representative Python tests
  • Shell syntax and whitespace checks

Scope and follow-up work

This PR intentionally does not include:

  • Migration from Yarn/concatjs to modern rules_js package management
  • Replacement of rules_sass
  • Migration from cargo-raze to crate-universe
  • Complete removal of WORKSPACE.bzlmod

Known limitation

TensorBoard uses Bzlmod as its supported dependency-resolution entry point, but some dependencies are still provided by the transitional WORKSPACE.bzlmod.

Consequently, builds using --noenable_workspace are not supported yet and are expected to fail, currently beginning with the legacy Closure/Soy repository.

Removing WORKSPACE.bzlmod should be handled as follow-up work alongside the Closure, Node/Yarn, Sass, and Rust migrations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant