docs: add bootstrapper architecture overview

AGENTS.md

@@ -102,6 +102,12 @@ Look at these before writing code:
 
 - Use single backticks around function and class names in markdown (e.g. `req_ctxvar_context()`, `WorkContext`), double backticks in .rst (reStructuredText)
 - Use Sphinx `versionadded`, `versionremoved`, `versionchanged` directives for user-facing changes (get next version from last git tag)
+- When code changes affect the architecture described in `docs/concepts/`, update the relevant doc:
+  - `architecture-overview.rst` — major subsystems, data flow, extension points, key data structures
+  - `bootstrapper-architecture.rst` — phase pipeline, class hierarchy, bootstrapper-phase interaction
+  - `resolver-architecture.rst` — resolution strategies, provider hierarchy, version filtering
+  - `hooks-and-overrides.rst` — override hook points, global hooks, plugin discovery
+  - `package-settings.rst` — settings loading flow, merge order, PackageBuildInfo facade
 
 ## Commit Messages
 

docs/concepts/architecture-overview.rst

@@ -0,0 +1,141 @@
+Architecture Overview
+=====================
+
+Fromager rebuilds complete dependency trees of Python wheels from
+source.  This document maps the major subsystems and how they connect.
+
+.. seealso::
+
+   :doc:`bootstrap-vs-build` explains the two operating modes.
+   :doc:`bootstrapper-architecture` details the bootstrap engine.
+   :doc:`resolver-architecture` covers version resolution.
+   :doc:`hooks-and-overrides` describes the two plugin systems.
+   :doc:`package-settings` explains the settings layering.
+
+Major Subsystems
+----------------
+
+.. code-block:: text
+
+   ┌───────────────────────────────────────────────────────┐
+   │                    CLI & Context                      │
+   │         command parsing, WorkContext, settings        │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+   ┌───────────────────────────────────────────────────────┐
+   │                  Bootstrap Engine                     │
+   │        iterative DFS loop over phase pipeline         │
+   │                                                       │
+   │   ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │
+   │   │ Resolution  │  │    Source   │  │     Build   │   │
+   │   │ PyPI, graph │  │ Acquisition │  │   System    │   │
+   │   │  git URLs   │  │  download,  │  │  isolated   │   │
+   │   │             │  │ patch, Rust │  │   envs,     │   │
+   │   │             │  │   vendor    │  │   wheels    │   │
+   │   └─────────────┘  └─────────────┘  └─────────────┘   │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+              ┌────────────────┴────────────────┐
+              ▼                                 ▼
+   ┌──────────────────┐              ┌──────────────────┐
+   │  Per-Package     │              │   Global Hooks   │
+   │  Overrides       │              │ post_bootstrap,  │
+   │  (stevedore)     │              │    post_build    │
+   └──────────────────┘              └──────────────────┘
+
+The **bootstrap engine** orchestrates the other subsystems.  For each
+package it resolves a version, downloads the source, builds a wheel,
+and recurses into dependencies.  **Resolution**, **source acquisition**,
+and **build** are called as needed during each phase of the pipeline.
+
+**Extension points** intercept the pipeline at specific steps, allowing
+per-package overrides (e.g. custom download logic) and global hooks
+(e.g. post-build notifications).
+
+Data Flow
+---------
+
+A ``fromager bootstrap`` invocation flows through the subsystems in
+this order:
+
+.. code-block:: text
+
+   requirements.txt
+                               │
+                               ▼
+   ┌─ CLI & Context ───────────────────────────────────────┐
+   │  parse args, load settings, create WorkContext        │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+   ┌─ Bootstrap Engine ────────────────────────────────────┐
+   │  for each package (iterative DFS):                    │
+   │    resolve version ──► download source                │
+   │    ──► build wheel ──► extract deps                   │
+   │    ──► recurse into dependencies                      │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+                        ┌── Outputs ──┐
+                        │ graph.json  │
+                        │ build-order │
+                        │ wheels/     │
+                        │ constraints │
+                        └─────────────┘
+
+The ``build`` command uses the same source acquisition and build
+subsystems but skips resolution and recursion -- it compiles a
+single package given a name, version, and source URL.
+
+Extension Points
+----------------
+
+Fromager has two plugin systems, both using stevedore:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 35 40
+
+   * - System
+     - Scope
+     - When it fires
+   * - **Per-package overrides**
+     - One package
+     - At each pipeline step (download, build sdist, build wheel,
+       dependency extraction, etc.)
+   * - **Global hooks**
+     - All packages
+     - After specific events (``post_bootstrap``, ``post_build``,
+       ``prebuilt_wheel``)
+
+Per-package overrides replace the default implementation of a step for
+a specific package.  Global hooks run in addition to the default logic
+for every package.  See :doc:`hooks-and-overrides` for the full
+breakdown.
+
+Key Data Structures
+-------------------
+
+Four data structures flow between subsystems:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Structure
+     - Role
+   * - ``WorkContext``
+     - Central configuration and state: directory paths, constraints,
+       dependency graph, settings, and variant info.  Passed to every
+       subsystem.
+   * - ``DependencyGraph``
+     - In-memory directed graph of all resolved dependencies.
+       Serialized to ``graph.json``.  Thread-safe.
+   * - ``PackageBuildInfo``
+     - Per-package build configuration: environment variables, patches,
+       resolver settings, and build options.  Derived from settings
+       files.
+   * - ``BuildEnvironment``
+     - Isolated virtual environment for building one package.  Created
+       per source build, cleaned up after completion.

docs/concepts/bootstrapper-architecture.rst

@@ -0,0 +1,109 @@
+Bootstrapper Architecture
+=========================
+
+The bootstrap command uses an iterative depth-first loop to resolve and
+build an entire dependency tree.  This document describes the phase
+pipeline, class hierarchy, and interaction model at a high level.
+
+.. seealso::
+
+   :doc:`architecture-overview` maps the major subsystems.
+   :doc:`bootstrap-vs-build` covers the difference between bootstrap and
+   build modes.
+
+Bootstrap Phase Flow
+--------------------
+
+Every package passes through a sequence of phases.  Source packages
+traverse the full pipeline; prebuilt wheels skip the build phases.
+
+.. code-block:: text
+
+   RESOLVE ──► START ──► PREPARE_SOURCE ─┬─► PREPARE_BUILD ──► BUILD ─┐
+                  │                      │                            │
+                  │            (prebuilt)└────────────────────────────┤
+                  │                                                   │
+                  │                  ┌────────────────────────────────┘
+                  │                  ▼
+                  │          PROCESS_INSTALL_DEPS ──► COMPLETE
+                  │
+             (already seen) ──► drop
+
+Each phase returns a list of new items to push onto the work stack.
+Dependency-discovery phases (``PREPARE_SOURCE``, ``PREPARE_BUILD``,
+``PROCESS_INSTALL_DEPS``) also emit ``RESOLVE`` items for newly
+discovered dependencies, which drives the recursive traversal.
+
+Phase Class Hierarchy
+---------------------
+
+All phases inherit from the ``Phase`` abstract base class, which defines
+the contract every phase must satisfy:
+
+- ``run(bt)`` — execute the phase logic; return new items for the stack.
+- ``background_work(bt)`` — optionally return a callable for background
+  I/O (used by ``Resolve`` and ``PrepareSource``).
+- ``requires_exclusive_run`` — return ``True`` to drain the thread pool
+  before running (used by ``Build`` for exclusive builds).
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 75
+
+   * - Phase
+     - Role
+   * - ``Resolve``
+     - Resolve available versions; fan out one ``Start`` per version
+   * - ``Start``
+     - Add to dependency graph; deduplicate already-seen packages
+   * - ``PrepareSource``
+     - Download source or prebuilt wheel; read build-system deps
+   * - ``PrepareBuild``
+     - Install build-system deps; discover backend and sdist deps
+   * - ``Build``
+     - Build sdist and/or wheel; update the local mirror
+   * - ``ProcessInstallDeps``
+     - Run post-bootstrap hooks; extract install deps; record build order
+   * - ``Complete``
+     - Clean up build directories (terminal phase)
+
+Each phase wraps a ``WorkItem`` dataclass that accumulates per-package
+state (resolved version, build environment, build result, etc.) as it
+moves through the pipeline.
+
+Bootstrapper and Phase Interaction
+----------------------------------
+
+The ``Bootstrapper`` class owns the work stack and thread pool.  It runs
+an iterative DFS loop:
+
+.. code-block:: text
+
+   ┌─────────────────────────────────────────────────────┐
+   │                  Bootstrapper Loop                  │
+   │                                                     │
+   │  while stack:                                       │
+   │      item = stack.pop()                             │
+   │                                                     │
+   │      if item.requires_exclusive_run:                │
+   │          drain thread pool                          │
+   │                                                     │
+   │      new_items = item.run(self)                     │
+   │        ↑ phase blocks on its own bg_future          │
+   │          inside run() if it has one                 │
+   │                                                     │
+   │      for each new_item in new_items:                │
+   │          submit background_work() to thread pool    │
+   │          stack.push(new_item)                       │
+   └─────────────────────────────────────────────────────┘
+
+The ``Bootstrapper`` provides services that phases call back into during
+``run()``: dependency graph updates, seen-set tracking, build-order
+recording, and work-item creation for discovered dependencies.
+
+Background I/O (version resolution, source downloads) runs in a thread
+pool.  When new items are pushed onto the stack, the ``Bootstrapper``
+submits their ``background_work()`` immediately so I/O overlaps with
+the main thread processing other items.  Each phase is responsible for
+blocking on its own ``bg_future`` inside ``run()`` when it needs the
+result.

docs/concepts/hooks-and-overrides.rst

@@ -0,0 +1,116 @@
+Hooks and Overrides
+===================
+
+Fromager has two stevedore-based plugin systems that serve different
+purposes: **per-package overrides** replace default behavior for a
+specific package, while **global hooks** broadcast notifications after
+events for every package.
+
+.. seealso::
+
+   :doc:`architecture-overview` maps the major subsystems.
+   :doc:`/reference/hooks` documents each override hook's arguments and
+   return values.  :doc:`/customization` covers global hooks with code
+   examples.
+
+Comparison
+----------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 20 40 40
+
+   * - Aspect
+     - Per-Package Overrides
+     - Global Hooks
+   * - **Scope**
+     - One named package
+     - Every package
+   * - **Cardinality**
+     - At most one override module per package
+     - Multiple plugins per hook
+   * - **Semantics**
+     - Replaces the default implementation
+     - Runs in addition to the default
+   * - **Return value**
+     - Used by the caller
+     - None (fire-and-forget)
+   * - **Typical use**
+     - Custom download, build, or resolution logic
+     - Publishing wheels, validation, logging
+
+Per-Package Overrides
+---------------------
+
+An override module provides alternative implementations for specific
+build steps.  Fromager looks up the module by the canonicalized package
+name.  If the module defines the requested method, it is called instead
+of the default; otherwise the default runs.
+
+Override hooks fire at these points in the pipeline:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Category
+     - Hook points
+   * - **Resolution**
+     - ``get_resolver_provider`` — provide a custom resolvelib provider
+   * - **Source**
+     - ``download_source`` — download the source archive
+
+       ``prepare_source`` — unpack and patch source
+
+       ``expected_source_archive_name`` — filename for re-discovery
+
+       ``expected_source_directory_name`` — directory for re-discovery
+   * - **Build**
+     - ``build_sdist`` — build a source distribution
+
+       ``build_wheel`` — build a wheel
+
+       ``add_extra_metadata_to_wheels`` — inject extra metadata
+   * - **Dependencies**
+     - ``get_build_system_dependencies`` — PEP 517 build-system deps
+
+       ``get_build_backend_dependencies`` — build-backend deps
+
+       ``get_build_sdist_dependencies`` — sdist build deps
+
+       ``get_install_dependencies_of_sdist`` — install deps from sdist
+   * - **Environment**
+     - ``update_extra_environ`` — customize build environment variables
+
+Third-party packages register overrides via the
+``fromager.project_overrides`` entry-point group in their
+``pyproject.toml``, mapping a package name to a Python module.
+
+See :doc:`/reference/hooks` for the full API of each hook.
+
+Global Hooks
+------------
+
+Global hooks are event callbacks that fire for every package.  All
+registered plugins for a hook are called sequentially.  They cannot
+alter the build result — they are purely side-effecting notifications.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 75
+
+   * - Hook
+     - When it fires
+   * - ``post_build``
+     - After a wheel is built from source (not for prebuilt wheels)
+   * - ``prebuilt_wheel``
+     - After a prebuilt wheel is downloaded (not built from source)
+   * - ``post_bootstrap``
+     - After a package is bootstrapped, before its install dependencies
+       are processed
+
+Third-party packages register hooks via the ``fromager.hooks``
+entry-point group in their ``pyproject.toml``, mapping a hook name to
+a callable.
+
+See :doc:`/customization` for examples and argument details.