From 0d595f938a0a1649e0dc07204b64ae35f43fd213 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Tue, 5 May 2026 12:52:42 -0700 Subject: [PATCH 1/8] add desktop nightly release info and some refactor --- .../android.md} | 0 .../desktop.md} | 0 processes/releases/nightly/desktop.md | 71 +++++++++++++++++++ processes/{ => releases}/yearly_esr.md | 0 4 files changed, 71 insertions(+) rename processes/{android_release.md => releases/android.md} (100%) rename processes/{desktop_release.md => releases/desktop.md} (100%) create mode 100644 processes/releases/nightly/desktop.md rename processes/{ => releases}/yearly_esr.md (100%) diff --git a/processes/android_release.md b/processes/releases/android.md similarity index 100% rename from processes/android_release.md rename to processes/releases/android.md diff --git a/processes/desktop_release.md b/processes/releases/desktop.md similarity index 100% rename from processes/desktop_release.md rename to processes/releases/desktop.md diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md new file mode 100644 index 0000000..dde78ca --- /dev/null +++ b/processes/releases/nightly/desktop.md @@ -0,0 +1,71 @@ +# Thunderbird Desktop's nightly releases + +This process is fairly well documented in general, however this doc attempts to shed some light into some of the intricacies of it. + +## Overview + +In summary, nightly build jobs are triggered based on the [.cron.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.cron.yml) file (whose schema can be found [here](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cron/schema.yml)) defined in the root directory of the [comm-central](https://hg-edge.mozilla.org/comm-central/file) repository. As described at [taskcluster/cron.html](https://firefox-source-docs.mozilla.org/taskcluster/cron.html), the [TaskCluster Hooks Service](https://firefox-ci-tc.services.mozilla.com/hooks) has a hook for each repository that runs every 15 minutes which reads `.cron.yml`, consults the time the cron task was created, rounded down to the nearest 15 minutes, and creates the tasks for the specified cron jobs. + +### Cron job definition + +``` +jobs: + - name: nightly-desktop + job: + type: decision-task + treeherder-symbol: Nd + target-tasks-method: nightly_desktop + run-on-projects: + - comm-central + when: [{hour: 11, minute: 0}] +``` + +## Task decision + +As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main.py](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: + +``` +build-decision cron \ + --repo-url https://hg.mozilla.org/comm-central \ + --project comm-central \ + --level 3 \ + --repository-type hg \ + --trust-domain comm \ + --branch default +``` + +## Locating tasks + +Jobs with type `decision-task` app creates tasks based on the [.taskcluster.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml) file. This file has a route defined for cron tasks at line [#145](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml#l145) that points to where the resulting task in TaskCluster will be. After replacing all the variables we get the following URL: + +``` +https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop +``` + +> [!NOTE] +> Click `View Task` -> `Full Task Definition` and you'll see a reference to the actual task created by `build-decision`: +> +> ``` +> description: >- +> Created by a [cron +> task](https://firefox-ci-tc.services.mozilla.com/tasks/V12RpoRORi6MsIC507wFbg) +> ([Treeherder +> job](https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=8746fe1c7efe0d0f92f6dcf73ea6d6b65d367c39&selectedTaskRun=WQc5nWWfSNGu1I48hN_vSg)) +> ``` +> +> Go into that task and you'll see the exact arguments that were passed on the `build-decision` binary. + +We can also copy the `Task ID` from there and look up the task in TreeHerder: + +``` +https://treeherder.mozilla.org/#/jobs?repo=comm-central&selectedTaskRun= +``` + +Alternatively, we can also filter by all `cron` tasks in TreeHerder as follows: + +``` +https://treeherder.mozilla.org/jobs?repo=comm-central&job_type_name=cron +``` + +Look for tasks that show `Cron(Nd)`. You may need to expand cron tasks that have been grouped (example the ones showing `Cron(+X)`), where `X` is the number grouped cron tasks. diff --git a/processes/yearly_esr.md b/processes/releases/yearly_esr.md similarity index 100% rename from processes/yearly_esr.md rename to processes/releases/yearly_esr.md From 622218f7459500df145e6f9c823bac1bdf567958 Mon Sep 17 00:00:00 2001 From: Kulana Kryoseu Date: Tue, 5 May 2026 12:55:40 -0700 Subject: [PATCH 2/8] fix: add missing parenthesis --- processes/releases/nightly/desktop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index dde78ca..46063ca 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -23,7 +23,7 @@ jobs: ## Task decision As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. -In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main.py](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main.py](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` build-decision cron \ From b210abd77ea7de22cfcc8f75d0f4ecb5ba468a97 Mon Sep 17 00:00:00 2001 From: Kulana Kryoseu Date: Tue, 5 May 2026 12:57:01 -0700 Subject: [PATCH 3/8] fix: link to main function --- processes/releases/nightly/desktop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index 46063ca..1493f61 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -23,7 +23,7 @@ jobs: ## Task decision As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. -In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main.py](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs the [main](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63) function at `cli.py`. This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` build-decision cron \ From aa49052379aea1f45dcf1aa5dfe581260ff1b2d2 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Wed, 6 May 2026 15:06:51 -0700 Subject: [PATCH 4/8] add info on mach taskgraph decision, task selection and refactor --- processes/releases/nightly/desktop.md | 33 +++++++++++++++++++++------ 1 file changed, 26 insertions(+), 7 deletions(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index 1493f61..1ea9792 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -4,9 +4,9 @@ This process is fairly well documented in general, however this doc attempts to ## Overview -In summary, nightly build jobs are triggered based on the [.cron.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.cron.yml) file (whose schema can be found [here](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cron/schema.yml)) defined in the root directory of the [comm-central](https://hg-edge.mozilla.org/comm-central/file) repository. As described at [taskcluster/cron.html](https://firefox-source-docs.mozilla.org/taskcluster/cron.html), the [TaskCluster Hooks Service](https://firefox-ci-tc.services.mozilla.com/hooks) has a hook for each repository that runs every 15 minutes which reads `.cron.yml`, consults the time the cron task was created, rounded down to the nearest 15 minutes, and creates the tasks for the specified cron jobs. +In summary, nightly build jobs are triggered based on the [.cron.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.cron.yml) file (whose schema can be found [here](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cron/schema.yml)) defined in the root directory of the [comm-central](https://hg-edge.mozilla.org/comm-central/file) repository. As described at [taskcluster/cron.html](https://firefox-source-docs.mozilla.org/taskcluster/cron.html), the [TaskCluster Hooks Service](https://firefox-ci-tc.services.mozilla.com/hooks) has a hook for each repository that runs every 15 minutes which reads `.cron.yml`, consults the time the cron task was created, rounded down to the nearest 15 minutes, and creates the tasks for the specified cron jobs based on the template [cron-task-template.yml](https://github.com/mozilla-releng/fxci-config/blob/main/cron-task-template.yml). -### Cron job definition +## Cron job definition ``` jobs: @@ -20,10 +20,11 @@ jobs: when: [{hour: 11, minute: 0}] ``` -## Task decision +## Decision Task As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. -In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.yoml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs the [main](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63) function at `cli.py`. This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: + +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` build-decision cron \ @@ -35,9 +36,7 @@ build-decision cron \ --branch default ``` -## Locating tasks - -Jobs with type `decision-task` app creates tasks based on the [.taskcluster.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml) file. This file has a route defined for cron tasks at line [#145](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml#l145) that points to where the resulting task in TaskCluster will be. After replacing all the variables we get the following URL: +Jobs with type `decision-task` create tasks based on the [.taskcluster.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml) file. This file has a route defined for cron tasks at line [#145](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml#l145) that points to where the resulting task in TaskCluster will be. After replacing all the variables we get the following URL: ``` https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop @@ -69,3 +68,23 @@ https://treeherder.mozilla.org/jobs?repo=comm-central&job_type_name=cron ``` Look for tasks that show `Cron(Nd)`. You may need to expand cron tasks that have been grouped (example the ones showing `Cron(+X)`), where `X` is the number grouped cron tasks. + +When viewing the artifacts of the nightly decision task, you will notice a set of `docker-contexts/.tar.gz` files. These are Docker build contexts — tarballs containing the Dockerfile and supporting files for each Docker image used by tasks in the graph. They are produced by the decision task itself and consumed by docker-image build tasks, which fetch the appropriate tarball, build the image, and upload the result for downstream tasks to use. + +## mach taskgraph decision + +By going into the TaskCluster nightly build decision task URL [here](https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop) and viewing the full task definition we see the task executes a `mach taskgraph decision` (see implementation [here](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#260-328)) command with a bunch of parameters pertaining to Thunderbird's repository location in addition to `--target-tasks-method=nightly_desktop`, which specifies the registered method responsible for building and returning the full graph of tasks pertaining to nightly builds and releases (see definition [here](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015)). + +The [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#266) method is the entry point for the `mach taskgraph decision` command that executes [taskgraph_commands["decision"].func(options)](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#287) which is an alias for [gecko_taskgraph.main.decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/main.py#538-598), which in turn calls [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#171) in `decision.py`. The detailed flow is as follows: + +1. [gecko_taskgraph.decision.get_decision_parameters()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#277) builds the initial parameters from the CLI options, including setting `target_tasks_method: nightly_desktop`. +2. Because `comm/taskcluster/config.yml` sets `taskgraph.decision-parameters` to `comm_taskgraph.decision:get_decision_parameters` ([here](https://searchfox.org/comm-central/source/taskcluster/config.yml#132-137)), gecko's `get_decision_parameters()` [calls into comm's version](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#440-443), which: + - Sets project defaults from `PER_PROJECT_PARAMETERS` — for `comm-central` this would default `target_tasks_method` to `comm_central_tasks`. + - Overrides `target_tasks_method` back to `nightly_desktop` from the CLI arg (since CLI args take precedence). + - Since `tasks_for == "cron"`, looks up `nightly_desktop` in `CRON_OPTIONS` and populates `existing_tasks` (a mapping of task labels to IDs from the on-push decision task, allowing artifacts already built when the push landed to be reused) and `release_history` (queried from Balrog, used to generate partial MARs). +3. Back in gecko's `decision.py`, a `TaskGraphGenerator` is created with the finalized parameters. When the [target_tasks_method](https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/parameters.html#target-tasks-method) is resolved, it calls [nightly_desktop()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015) to determine the set of tasks to schedule. +4. The resulting task graph is optimized, written out as artifacts, and tasks are created via the TaskCluster API. + +### Task selection + +`nightly_desktop` uses [make_desktop_nightly_filter()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#892-905) to select tasks across all desktop platforms. The filter picks tasks that have the `shippable: true` attribute set, which in `comm/taskcluster/kinds/build/` corresponds to the shippable build variants: `linux64-shippable/opt`, `linux64-aarch64-shippable/opt`, `win32-shippable/opt`, `win64-shippable/opt`, `macosx64-shippable/opt`, and `macosx64-aarch64-shippable/opt`. The results for each platform are unioned together along with any platform-agnostic release tasks to produce the final task list. From 2cf5a502dfff3542d24d18d14c347ed63191ccd3 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Wed, 6 May 2026 18:54:30 -0700 Subject: [PATCH 5/8] clarify some points in detailed flow and some refactor --- processes/releases/nightly/desktop.md | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index 1ea9792..bd289d7 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -23,7 +23,6 @@ jobs: ## Decision Task As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. - In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` @@ -75,16 +74,26 @@ When viewing the artifacts of the nightly decision task, you will notice a set o By going into the TaskCluster nightly build decision task URL [here](https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop) and viewing the full task definition we see the task executes a `mach taskgraph decision` (see implementation [here](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#260-328)) command with a bunch of parameters pertaining to Thunderbird's repository location in addition to `--target-tasks-method=nightly_desktop`, which specifies the registered method responsible for building and returning the full graph of tasks pertaining to nightly builds and releases (see definition [here](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015)). -The [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#266) method is the entry point for the `mach taskgraph decision` command that executes [taskgraph_commands["decision"].func(options)](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#287) which is an alias for [gecko_taskgraph.main.decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/main.py#538-598), which in turn calls [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#171) in `decision.py`. The detailed flow is as follows: +The [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#266) method is the entry point for the `mach taskgraph decision` command and ultimately executes [taskgraph_commands["decision"].func(options)](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#287) which is an alias for [gecko_taskgraph.main.decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/main.py#538-598), which in turn calls [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#171) in `decision.py`. The detailed flow is as follows: -1. [gecko_taskgraph.decision.get_decision_parameters()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#277) builds the initial parameters from the CLI options, including setting `target_tasks_method: nightly_desktop`. -2. Because `comm/taskcluster/config.yml` sets `taskgraph.decision-parameters` to `comm_taskgraph.decision:get_decision_parameters` ([here](https://searchfox.org/comm-central/source/taskcluster/config.yml#132-137)), gecko's `get_decision_parameters()` [calls into comm's version](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#440-443), which: - - Sets project defaults from `PER_PROJECT_PARAMETERS` — for `comm-central` this would default `target_tasks_method` to `comm_central_tasks`. +1. `taskgraph_decision()` creates a `TaskGraphGenerator` with `root_dir` set to `comm/taskcluster` (from the `--root` CLI arg). When `TaskGraphGenerator` gets instantiated it calls [self_run()](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/generator.py#L164), which calls [load_graph_config(root_dir)](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L164-L173) that reads [comm/taskcluster/config.yml](https://searchfox.org/comm-central/source/taskcluster/config.yml) into a `GraphConfig` object and then calls `graph_config.register()`. This adds `comm/taskcluster` to the Python path (making `comm_taskgraph` importable - see [here](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L127)) and calls the `comm_taskgraph:register` hook. +2. The `graph_config` object now holds the contents of `comm/taskcluster/config.yml`, including the `taskgraph.decision-parameters` entry pointing to `comm_taskgraph.decision:get_decision_parameters`. +3. [gecko_taskgraph.decision.get_decision_parameters()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#277) builds the initial parameters from the CLI options, including setting `target_tasks_method: nightly_desktop`. At the end, it [checks](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#440-443) whether `graph_config["taskgraph"]` has a `decision-parameters` key, finds it, and calls [comm_taskgraph.decision.get_decision_parameters()](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#134), which: + - Sets project defaults from [PER_PROJECT_PARAMETERS](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#38-70) — for `comm-central` this would default `target_tasks_method` to `comm_central_tasks`. - Overrides `target_tasks_method` back to `nightly_desktop` from the CLI arg (since CLI args take precedence). - - Since `tasks_for == "cron"`, looks up `nightly_desktop` in `CRON_OPTIONS` and populates `existing_tasks` (a mapping of task labels to IDs from the on-push decision task, allowing artifacts already built when the push landed to be reused) and `release_history` (queried from Balrog, used to generate partial MARs). -3. Back in gecko's `decision.py`, a `TaskGraphGenerator` is created with the finalized parameters. When the [target_tasks_method](https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/parameters.html#target-tasks-method) is resolved, it calls [nightly_desktop()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015) to determine the set of tasks to schedule. -4. The resulting task graph is optimized, written out as artifacts, and tasks are created via the TaskCluster API. + - Since `tasks_for == "cron"`, looks up `nightly_desktop` in `CRON_OPTIONS` and populates two additional parameters: `existing_tasks` and `release_history`. `existing_tasks` is built by fetching the on-push decision task's `full-task-graph.json` artifact for the same revision, which contains a mapping of task labels (e.g. `build-linux64-shippable/opt`) to their TaskCluster task IDs. During the optimization phase, the nightly graph uses this to avoid re-running tasks that already completed during the on-push run — reusing their artifacts instead and only scheduling nightly-specific steps such as signing, uploads to Balrog, and beetmover. `release_history` is queried from Balrog and contains the history of previous Thunderbird releases, used to generate partial MARs (incremental updates). +4. With the finalized parameters, when the [target_tasks_method](https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/parameters.html#target-tasks-method) is resolved, it calls [nightly_desktop()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015) to determine the set of tasks to schedule. +5. The resulting task graph is optimized, written out as artifacts, and tasks are created via the TaskCluster API. ### Task selection -`nightly_desktop` uses [make_desktop_nightly_filter()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#892-905) to select tasks across all desktop platforms. The filter picks tasks that have the `shippable: true` attribute set, which in `comm/taskcluster/kinds/build/` corresponds to the shippable build variants: `linux64-shippable/opt`, `linux64-aarch64-shippable/opt`, `win32-shippable/opt`, `win64-shippable/opt`, `macosx64-shippable/opt`, and `macosx64-aarch64-shippable/opt`. The results for each platform are unioned together along with any platform-agnostic release tasks to produce the final task list. +`nightly_desktop` uses [make_desktop_nightly_filter()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#892-905) to select tasks across all desktop platforms. The filter picks tasks that have the `shippable: true` attribute set, which in `comm/taskcluster/kinds/build/` corresponds to the shippable build variants: + +- `linux64-shippable/opt` +- `linux64-aarch64-shippable/opt` +- `win32-shippable/opt` +- `win64-shippable/opt` +- `macosx64-shippable/opt` +- `macosx64-aarch64-shippable/opt` + +The results for each platform are unioned together along with any platform-agnostic release tasks to produce the final task list. From 7f47b363a918508edb377a01c13a9ef0231e5bb8 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Wed, 6 May 2026 18:56:38 -0700 Subject: [PATCH 6/8] fix: add missing closing parenthesis --- processes/releases/nightly/desktop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index bd289d7..3e38b34 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -23,7 +23,7 @@ jobs: ## Decision Task As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. -In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` build-decision cron \ From e845528697d7ec4aa7d7de8f9df4bd7d7d9e10c5 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Wed, 6 May 2026 19:02:55 -0700 Subject: [PATCH 7/8] make taskcluster/config.yml a hyperlink --- processes/releases/nightly/desktop.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index 3e38b34..9be8603 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -77,7 +77,7 @@ By going into the TaskCluster nightly build decision task URL [here](https://fir The [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#266) method is the entry point for the `mach taskgraph decision` command and ultimately executes [taskgraph_commands["decision"].func(options)](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#287) which is an alias for [gecko_taskgraph.main.decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/main.py#538-598), which in turn calls [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#171) in `decision.py`. The detailed flow is as follows: 1. `taskgraph_decision()` creates a `TaskGraphGenerator` with `root_dir` set to `comm/taskcluster` (from the `--root` CLI arg). When `TaskGraphGenerator` gets instantiated it calls [self_run()](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/generator.py#L164), which calls [load_graph_config(root_dir)](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L164-L173) that reads [comm/taskcluster/config.yml](https://searchfox.org/comm-central/source/taskcluster/config.yml) into a `GraphConfig` object and then calls `graph_config.register()`. This adds `comm/taskcluster` to the Python path (making `comm_taskgraph` importable - see [here](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L127)) and calls the `comm_taskgraph:register` hook. -2. The `graph_config` object now holds the contents of `comm/taskcluster/config.yml`, including the `taskgraph.decision-parameters` entry pointing to `comm_taskgraph.decision:get_decision_parameters`. +2. The `graph_config` object now holds the contents of [comm/taskcluster/config.yml](https://searchfox.org/comm-central/source/taskcluster/config.yml), including the `taskgraph.decision-parameters` entry pointing to `comm_taskgraph.decision:get_decision_parameters`. 3. [gecko_taskgraph.decision.get_decision_parameters()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#277) builds the initial parameters from the CLI options, including setting `target_tasks_method: nightly_desktop`. At the end, it [checks](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#440-443) whether `graph_config["taskgraph"]` has a `decision-parameters` key, finds it, and calls [comm_taskgraph.decision.get_decision_parameters()](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#134), which: - Sets project defaults from [PER_PROJECT_PARAMETERS](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#38-70) — for `comm-central` this would default `target_tasks_method` to `comm_central_tasks`. - Overrides `target_tasks_method` back to `nightly_desktop` from the CLI arg (since CLI args take precedence). From 0e23aece02f635f8a86d4fbd53b6c5a464a3b257 Mon Sep 17 00:00:00 2001 From: kryoseu Date: Thu, 7 May 2026 11:43:40 -0700 Subject: [PATCH 8/8] add further clarification on detailed flow --- processes/releases/nightly/desktop.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/processes/releases/nightly/desktop.md b/processes/releases/nightly/desktop.md index 9be8603..be20593 100644 --- a/processes/releases/nightly/desktop.md +++ b/processes/releases/nightly/desktop.md @@ -23,7 +23,7 @@ jobs: ## Decision Task As seen above, the cron job specifies a `job.type`, which corresponds to the function responsible for creating TaskCluster tasks when the job runs. -In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile)). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: +In this case, the job type is `decision-task`, which results in a task that runs the [build-decision](https://github.com/mozilla-releng/fxci-config/tree/main/build-decision) docker image (see [Dockerfile](https://github.com/mozilla-releng/fxci-config/blob/main/taskcluster/docker/build-decision/Dockerfile). Specifically, the entry point for this image is a binary called `build-decision` located at `/build/.venv/bin/build-decision` which gets generated by `uv` based on [pyproject.toml](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/pyproject.toml#L28) and ultimately runs [main()](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/util/cli.py#L53-L63). This method, in turn, parses the incoming arguments (e.g `cron` for cron tasks) and invokes [cron(options)](https://github.com/mozilla-releng/fxci-config/blob/main/build-decision/src/build_decision/cli.py#L79-L92) method which gets the job created. An example execution of this binary and arguments passed by TaskCluster is as follows: ``` build-decision cron \ @@ -35,11 +35,7 @@ build-decision cron \ --branch default ``` -Jobs with type `decision-task` create tasks based on the [.taskcluster.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml) file. This file has a route defined for cron tasks at line [#145](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml#l145) that points to where the resulting task in TaskCluster will be. After replacing all the variables we get the following URL: - -``` -https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop -``` +Jobs with type `decision-task` create tasks based on the [.taskcluster.yml](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml) file. This file has a route defined for cron tasks at [line 145](https://hg-edge.mozilla.org/comm-central/file/tip/.taskcluster.yml#l145) that points to where the resulting task in TaskCluster will be. After replacing all the variables we get the nightly desktop decision task [URL](https://firefox-ci-tc.services.mozilla.com/tasks/index/comm.v2.comm-central.latest.taskgraph/decision-nightly-desktop). > [!NOTE] > Click `View Task` -> `Full Task Definition` and you'll see a reference to the actual task created by `build-decision`: @@ -60,11 +56,7 @@ We can also copy the `Task ID` from there and look up the task in TreeHerder: https://treeherder.mozilla.org/#/jobs?repo=comm-central&selectedTaskRun= ``` -Alternatively, we can also filter by all `cron` tasks in TreeHerder as follows: - -``` -https://treeherder.mozilla.org/jobs?repo=comm-central&job_type_name=cron -``` +Alternatively, we can also filter by all `cron` tasks in TreeHerder, like [this](https://treeherder.mozilla.org/jobs?repo=comm-central&job_type_name=cron). Look for tasks that show `Cron(Nd)`. You may need to expand cron tasks that have been grouped (example the ones showing `Cron(+X)`), where `X` is the number grouped cron tasks. @@ -77,11 +69,11 @@ By going into the TaskCluster nightly build decision task URL [here](https://fir The [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#266) method is the entry point for the `mach taskgraph decision` command and ultimately executes [taskgraph_commands["decision"].func(options)](https://searchfox.org/firefox-main/source/taskcluster/mach_commands.py#287) which is an alias for [gecko_taskgraph.main.decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/main.py#538-598), which in turn calls [taskgraph_decision()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#171) in `decision.py`. The detailed flow is as follows: 1. `taskgraph_decision()` creates a `TaskGraphGenerator` with `root_dir` set to `comm/taskcluster` (from the `--root` CLI arg). When `TaskGraphGenerator` gets instantiated it calls [self_run()](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/generator.py#L164), which calls [load_graph_config(root_dir)](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L164-L173) that reads [comm/taskcluster/config.yml](https://searchfox.org/comm-central/source/taskcluster/config.yml) into a `GraphConfig` object and then calls `graph_config.register()`. This adds `comm/taskcluster` to the Python path (making `comm_taskgraph` importable - see [here](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/config.py#L127)) and calls the `comm_taskgraph:register` hook. -2. The `graph_config` object now holds the contents of [comm/taskcluster/config.yml](https://searchfox.org/comm-central/source/taskcluster/config.yml), including the `taskgraph.decision-parameters` entry pointing to `comm_taskgraph.decision:get_decision_parameters`. +2. The `graph_config` object now holds the contents of `comm/taskcluster/config.yml`, including the `taskgraph.decision-parameters` entry pointing to `comm_taskgraph.decision:get_decision_parameters`. 3. [gecko_taskgraph.decision.get_decision_parameters()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#277) builds the initial parameters from the CLI options, including setting `target_tasks_method: nightly_desktop`. At the end, it [checks](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/decision.py#440-443) whether `graph_config["taskgraph"]` has a `decision-parameters` key, finds it, and calls [comm_taskgraph.decision.get_decision_parameters()](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#134), which: - Sets project defaults from [PER_PROJECT_PARAMETERS](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#38-70) — for `comm-central` this would default `target_tasks_method` to `comm_central_tasks`. - Overrides `target_tasks_method` back to `nightly_desktop` from the CLI arg (since CLI args take precedence). - - Since `tasks_for == "cron"`, looks up `nightly_desktop` in `CRON_OPTIONS` and populates two additional parameters: `existing_tasks` and `release_history`. `existing_tasks` is built by fetching the on-push decision task's `full-task-graph.json` artifact for the same revision, which contains a mapping of task labels (e.g. `build-linux64-shippable/opt`) to their TaskCluster task IDs. During the optimization phase, the nightly graph uses this to avoid re-running tasks that already completed during the on-push run — reusing their artifacts instead and only scheduling nightly-specific steps such as signing, uploads to Balrog, and beetmover. `release_history` is queried from Balrog and contains the history of previous Thunderbird releases, used to generate partial MARs (incremental updates). + - `tasks_for` was set to `"cron"` by gecko's `get_decision_parameters()`, which reads it directly from the `--tasks-for=cron` CLI argument passed by `build-decision` when it created the decision task. Comm's function checks this value and, since it is `"cron"`, looks up `nightly_desktop` in [`CRON_OPTIONS`](https://searchfox.org/comm-central/source/taskcluster/comm_taskgraph/decision.py#72-81) to populate two additional parameters. `existing_tasks` is populated by finding the on-push decision task for the same revision and fetching its `full-task-graph.json` artifact, which contains a mapping of task labels (e.g. `build-linux64-shippable/opt`) to the TaskCluster task IDs that were scheduled during that push. `existing_tasks` is then consumed by the optimization phase — a step inside the base `taskgraph` library's `TaskGraphGenerator._run()` that runs after `nightly_desktop()` has determined the target task set. It calls [`optimize_task_graph()`](https://github.com/taskcluster/taskgraph/blob/main/src/taskgraph/optimize/base.py) which, for each task in the nightly graph, looks up its label in `existing_tasks`: if a match is found the task is not scheduled again and its existing task ID is recorded for downstream tasks to reference. Tasks that are nightly-specific (signing, beetmover uploads, Balrog submission) have no entry in `existing_tasks` since they were never part of the on-push graph, so they get scheduled fresh. `release_history` is populated synchronously during `comm_taskgraph.decision.get_decision_parameters()` via [`populate_release_history()`](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/util/partials.py#178), which queries the Balrog API for the URLs of previous nightly complete MARs. Partial MAR generation tasks use these to produce binary diffs against the current nightly, giving users on a recent version a smaller incremental update to download. 4. With the finalized parameters, when the [target_tasks_method](https://taskcluster-taskgraph.readthedocs.io/en/latest/reference/parameters.html#target-tasks-method) is resolved, it calls [nightly_desktop()](https://searchfox.org/firefox-main/source/taskcluster/gecko_taskgraph/target_tasks.py#999-1015) to determine the set of tasks to schedule. 5. The resulting task graph is optimized, written out as artifacts, and tasks are created via the TaskCluster API.