Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .bazelrc
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
common --noenable_bzlmod
# Bazel 7.7.0 remains pinned in .bazelversion. Bzlmod is TensorBoard's only
# supported dependency-resolution entry point. WORKSPACE.bzlmod is an internal
# compatibility bridge for dependencies that do not yet expose module APIs.
common --enable_bzlmod
# Let the OS-specific sections below (`common:linux`, `common:macos`, etc.)
# take effect so one `.bazelrc` can carry the required per-platform C++ flags.
common --enable_platform_specific_config
Expand Down
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ BUILD text eol=lf
BUILD.bazel text eol=lf
WORKSPACE text eol=lf
MODULE.bazel text eol=lf
tensorboard/data/server/*.pb.rs text eol=lf

third_party/rust/** -diff -merge linguist-generated=true
13 changes: 7 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ permissions:
contents: read

env:
# Keep this Bazel version in sync with the `versions.check` directive
# in our WORKSPACE file.
# Keep this Bazel version in sync with .bazelversion and MODULE.bazel's
# bazel_compatibility declaration. Bazel 8 is intentionally unsupported.
BAZEL_VERSION: '7.7.0'
BAZEL_SHA256SUM: 'fe7e799cbc9140f986b063e06800a3d4c790525075c877d00a7112669824acbf'
BUILDTOOLS_VERSION: '3.0.0'
Expand All @@ -41,6 +41,7 @@ jobs:
# Self-hosted runner since the ones provided by GH were running out of space.
# Other workflows are likely ok to use standard GH runners for now.
# Googlers, see b/460874304.
# Bazel build and test support is currently validated on Linux only.
runs-on: linux-x86-n2-32
# Pinned to a digest because zizmor's unpinned-images check rejects tags.
# To refresh: crane digest <image>:latest
Expand All @@ -53,9 +54,9 @@ jobs:
python_version: ['3.10']
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
# Use the container's system Python installation path instead of
# actions/setup-python so Bazel's @system_python repo sees a standard
# interpreter/header layout.
# The host virtualenv is used by pip-package tests. Protobuf also reads
# its matching system headers while Bazel Python actions use the
# checksummed rules_python 3.10 runtime from MODULE.bazel.
- name: 'Set up Python and system dependencies'
run: |
py_abi="python${{ matrix.python_version }}"
Expand All @@ -71,7 +72,7 @@ jobs:
echo "/tmp/tb-build-venv/bin" >> "${GITHUB_PATH}"
echo "VIRTUAL_ENV=/tmp/tb-build-venv" >> "${GITHUB_ENV}"
/tmp/tb-build-venv/bin/python -m pip install -U pip setuptools wheel virtualenv
- name: 'Check Python toolchain'
- name: 'Check host Python environment'
run: |
python --version
python - <<'PY'
Expand Down
82 changes: 72 additions & 10 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,65 @@ $ source tf/bin/activate
(tf)$ pip uninstall -y tb-nightly
```

TensorBoard builds are done with [Bazel](https://bazel.build), so you may need to [install Bazel](https://docs.bazel.build/versions/master/install.html). The Bazel build will automatically "vulcanize" all the HTML files and generate a "binary" launcher script. When HTML is vulcanized, it means all the script tags and HTML imports are inlined into one big HTML file. Then the Bazel build puts that index.html file inside a static assets zip. The python HTTP server then reads static assets from that zip while serving.
TensorBoard builds are done with [Bazel](https://bazel.build). The supported
version is pinned in `.bazelversion` (currently Bazel 7.7.0), and Bazel 8 is
intentionally unsupported. TensorBoard uses Bzlmod for dependency resolution;
legacy builds with `--noenable_bzlmod` are not supported. Some dependencies
still use legacy repository macros through the transitional `WORKSPACE.bzlmod`
file. Normal development must not add a second dependency graph to `WORKSPACE`;
new dependencies should use a BCR module, a module extension, or
`use_repo_rule` in that order.

The remaining `WORKSPACE.bzlmod` entries are intentional migration exceptions:

- The pinned Closure snapshot requires TensorBoard's Protobuf 6/Soy patches.
The module-published Closure release is currently validated upstream with
Bazel 8 rather than TensorBoard's supported Bazel 7.7.0.
- rules_nodejs 5.8.1 supplies the legacy `yarn_install`, concatjs, TypeScript,
and esbuild APIs used by the frontend build. The module-native rules_nodejs
dependency used by `aspect_rules_js` does not provide those removed APIs.
- rules_sass has no BCR module, and replacing it means moving Sass compilation
into the modern JavaScript rule stack.
- The Rust data server uses an old rules_rust toolchain and a checked-in
cargo-raze graph. Moving it requires a coordinated Rust toolchain and
crate-universe migration.
- TensorBoard's font/JavaScript archives, Closure Java artifacts, and local
compatibility repositories still depend on the exceptions above.

`WORKSPACE.bzlmod` should shrink as those projects are completed. A completely
WORKSPACE-independent build is reached when the following command succeeds;
it is a migration diagnostic, not yet a supported presubmit command:

```sh
(tf)$ bazel test //tensorboard/... --enable_bzlmod --noenable_workspace
```

The Bazel build and test configuration is currently validated on Linux only.
Bazel downloads a checksummed Python 3.10 runtime for its actions, so the
interpreter used by Bazel does not depend on the host `python3`. Continue to use
the activated virtual environment for package-management commands and for the
pip-package tests described below.

Python packages imported by Bazel targets are resolved from
`tensorboard/pip_package/requirements_bazel_lock.txt`. After changing either
pip requirements file or the TensorFlow compatibility baseline, regenerate the
lock on Linux with Python 3.10 and review the resulting version changes. Bazel
tests use the stable TensorFlow version in `requirements_bazel.in`; the
pip-package smoke test separately exercises the CI-selected version, currently
`tf-nightly`.

```sh
(tf)$ python -m pip install "uv==0.5.31"
(tf)$ uv pip compile --python-version 3.10 --generate-hashes \
--output-file tensorboard/pip_package/requirements_bazel_lock.txt \
tensorboard/pip_package/requirements_bazel.in
```

The Bazel build will automatically "vulcanize" all the HTML files and generate
a "binary" launcher script. When HTML is vulcanized, it means all the script
tags and HTML imports are inlined into one big HTML file. Then the Bazel build
puts that index.html file inside a static assets zip. The Python HTTP server
then reads static assets from that zip while serving.

You can build and run TensorBoard via Bazel (from within the TensorFlow nightly virtualenv) as follows:

Expand Down Expand Up @@ -270,26 +328,29 @@ Sample upgrade: https://github.com/tensorflow/tensorboard/pull/5977
there are occasionally special instructions, especially for major releases.
Make a mental note of any of these special instructions.

2. Update the build_bazel_rules_nodejs target in the WORKSPACE file as described
in the rules_nodejs release notes. Also remove or comment out the now-stale
links to "http://mirror.tensorflow.org/".
2. Update the `build_bazel_rules_nodejs` target in `WORKSPACE.bzlmod` as
described in the rules_nodejs release notes. This dependency still uses a
legacy repository macro behind the Bzlmod entry point. Also remove or
comment out now-stale links to "http://mirror.tensorflow.org/".

3. Update npm packages scoped with `@bazel` in package.json using yarn.
* Use the same version as the rules_nodejs version.
* See the previous section for instructions on how to use yarn.

4. Update the rules_sass target in the WORKSPACE file.
4. Update the `rules_sass` target in `WORKSPACE.bzlmod`.
* Examine https://github.com/bazelbuild/rules_sass/tags to see the list
of rules_sass releases.
* Pick a tag (the most recent is likely good enough) and use that version to
modify the rules_sass target in the WORKSPACE file.
modify the `rules_sass` target in `WORKSPACE.bzlmod`.

5. Update the minimum bazel version to match the one supported by rules_nodejs:
* Examine https://github.com/bazelbuild/rules_nodejs/blob/stable/index.bzl
and find the SUPPORTED_BAZEL_VERSIONS constant.
* Compare the minimum bazel version supported by rules_nodejs to the one we
have specified in the WORKSPACE file. Modify the WORKSPACE file and
ci.yml with a new minimum version, if necessary.
* Compare the supported Bazel version from rules_nodejs with the version
pinned in `.bazelversion` and constrained by `MODULE.bazel`. Update those
files and `.github/workflows/ci.yml` together if the project deliberately
changes Bazel versions. Do not move to Bazel 8 without a separate
compatibility migration.

6. Attempt to rebuild and test TensorBoard to make sure it works:
* `rm -rf node_modules; bazel clean --expunge; yarn`
Expand All @@ -301,7 +362,8 @@ Sample upgrade: https://github.com/tensorflow/tensorboard/pull/5977
the rules_nodejs release notes (from Step 1) might be helpful.

8. Generate mirrors for the new versions of rules_nodejs and rules_sass and
update the WORKSPACE file with the new "http://mirror.tensorflow.org/" URLs.
update `WORKSPACE.bzlmod` with the new "http://mirror.tensorflow.org/"
URLs.
Googlers, see information at go/tensorboard-tf-mirror.

## Updating Angular
Expand Down
155 changes: 155 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
module(
name = "tensorboard",
bazel_compatibility = [
">=7.7.0",
"<8.0.0",
],
repo_name = "org_tensorflow_tensorboard",
)

# Keep Bazel itself pinned in .bazelversion. This module graph is intentionally
# constrained to dependencies that support Bazel 7.7.0.
bazel_dep(name = "bazel_skylib", version = "1.7.1")
bazel_dep(name = "rules_cc", version = "0.1.1")
bazel_dep(name = "rules_java", version = "8.6.1")
bazel_dep(name = "rules_python", version = "1.0.0")

# Functional browser tests use rules_webtesting. Its module owns the Go,
# Gazelle, and Python support repositories that the legacy WORKSPACE setup had
# to initialize explicitly.
bazel_dep(
name = "rules_webtesting",
version = "0.4.1",
repo_name = "io_bazel_rules_webtesting",
)
bazel_dep(name = "rules_web_testing_python", version = "0.4.1")
archive_override(
module_name = "rules_web_testing_python",
integrity = "sha256-V08cCqBywYcZTWC+2n9b4V4Tml4AlgiadxCBjuw6T2I=",
patch_strip = 1,
patches = ["//patches:rules_web_testing_python_py310.patch"],
strip_prefix = "rules_webtesting-0.4.1/web_testing_python",
urls = [
"https://github.com/bazelbuild/rules_webtesting/releases/download/0.4.1/rules_webtesting-0.4.1.tar.gz",
],
)

# Keep the browser repository snapshot used by TensorBoard's existing test
# definitions. These repositories are imported explicitly because module
# extension repositories are visible only when requested with use_repo.
browser_repositories = use_extension(
"@io_bazel_rules_webtesting//web:extension.bzl",
"browser_repositories_extension",
)
browser_repositories.override_version(version = "0.3.4")
use_repo(
browser_repositories,
"com_saucelabs_sauce_connect_linux_x64",
"com_saucelabs_sauce_connect_macos_x64",
"com_saucelabs_sauce_connect_windows_x64",
"org_chromium_chromedriver_linux_x64",
"org_chromium_chromedriver_macos_arm64",
"org_chromium_chromedriver_macos_x64",
"org_chromium_chromedriver_windows_x64",
"org_chromium_chromium_linux_x64",
"org_chromium_chromium_macos_arm64",
"org_chromium_chromium_macos_x64",
"org_chromium_chromium_windows_x64",
"org_mozilla_firefox_linux_x64",
"org_mozilla_firefox_macos_arm64",
"org_mozilla_firefox_macos_x64",
"org_mozilla_geckodriver_linux_x64",
"org_mozilla_geckodriver_macos_arm64",
"org_mozilla_geckodriver_macos_x64",
)

# Use a downloaded, checksummed Python runtime so Bazel actions do not depend
# on whichever interpreter happens to be first on the host PATH. TensorBoard's
# Bazel build and test jobs currently standardize on Python 3.10 on Linux.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
# TensorBoard's Linux CI builds execute inside a root-owned container.
ignore_root_user_error = True,
is_default = True,
python_version = "3.10",
)
use_repo(python, "python_3_10")

# Resolve packages imported by Bazel-built Python targets into checksummed
# wheel repositories. Pip-package smoke tests continue to create an isolated
# virtualenv because they validate the built wheel rather than Bazel targets.
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
extra_pip_args = ["--only-binary=:all:"],
hub_name = "tensorboard_pip_deps",
python_version = "3.10",
requirements_lock = "//tensorboard/pip_package:requirements_bazel_lock.txt",
)
use_repo(pip, "tensorboard_pip_deps")

# Bleach and Webencodings are copied into the TensorBoard wheel, while the
# pinned Markdown source preserves the behavior expected by those vendored
# paths. Keep these TensorBoard-owned source repositories module-managed even
# though they are not published as independent BCR modules.
tensorboard_python = use_extension(
"//third_party:extensions.bzl",
"tensorboard_python_dependencies",
)
use_repo(
tensorboard_python,
"org_mozilla_bleach",
"org_pythonhosted_markdown",
"org_pythonhosted_webencodings",
)

single_version_override(
module_name = "rules_cc",
# 0.1.0 is yanked in the BCR because it removed cc_proto_library.
# TensorBoard patched that API back in WORKSPACE mode; 0.1.1 contains the
# upstream fix and is the smallest usable module version.
version = "0.1.1",
)

# Protobuf 31.1 is the Bazel module version for the protobuf 6.31.1 release
# required by TensorFlow 2.21. Keep the historical apparent repository names
# used throughout TensorBoard's BUILD files.
bazel_dep(
name = "protobuf",
version = "31.1",
repo_name = "com_google_protobuf",
)
archive_override(
module_name = "protobuf",
integrity = "sha256-bgm7yVC6YMOnswKAIQzSha+NfY7V4KbtEBxyr/IujYg=",
patch_strip = 1,
patches = [
"//patches:protobuf_6_31_1_bzlmod.patch",
"//patches:protobuf_6_31_1_java_export.patch",
],
strip_prefix = "protobuf-6.31.1",
urls = [
"https://storage.googleapis.com/mirror.tensorflow.org/github.com/protocolbuffers/protobuf/archive/refs/tags/v6.31.1.zip",
"https://github.com/protocolbuffers/protobuf/archive/refs/tags/v6.31.1.zip",
],
)

bazel_dep(
name = "grpc",
version = "1.74.0",
repo_name = "com_github_grpc_grpc",
)

# A transitive grpc-java 1.66 module imports repositories that gRPC 1.74 no
# longer generates. grpc-java 1.69 moved those repositories to normal module
# dependencies and is the smallest compatible graph correction.
bazel_dep(
name = "grpc-java",
version = "1.69.0",
repo_name = "io_grpc_grpc_java",
)

# Angular's build tooling only consumes JsInfo/js_info from rules_js. This also
# resolves the module-native rules_nodejs toolchain used by rules_js itself.
# TensorBoard's legacy Yarn/concatjs integration still requires the separate
# rules_nodejs 5.8.1 compatibility repository in WORKSPACE.bzlmod.
bazel_dep(name = "aspect_rules_js", version = "2.1.0")
Loading
Loading