diff --git a/docs/book/dist/firefly-rust-by-example.epub b/docs/book/dist/firefly-rust-by-example.epub index a24e33b..fc4717e 100644 Binary files a/docs/book/dist/firefly-rust-by-example.epub and b/docs/book/dist/firefly-rust-by-example.epub differ diff --git a/docs/book/dist/firefly-rust-by-example.pdf b/docs/book/dist/firefly-rust-by-example.pdf index 9cdb6de..9c3bf4c 100644 Binary files a/docs/book/dist/firefly-rust-by-example.pdf and b/docs/book/dist/firefly-rust-by-example.pdf differ diff --git a/docs/book/src/01-why-firefly.md b/docs/book/src/01-why-firefly.md index 582e5f0..57b8fa6 100644 --- a/docs/book/src/01-why-firefly.md +++ b/docs/book/src/01-why-firefly.md @@ -1,12 +1,67 @@ # Why Firefly for Rust -By the end of this chapter you will understand the problem Firefly solves, how -its tiers fit together behind a single facade crate, and why Lumen — the -digital-wallet service you grow over the rest of the book — depends on exactly -**one** Firefly crate to get all of it. No code lands in Lumen yet; this chapter -sets the stage and the philosophy. The next one boots the scaffold. - -## The cohesion problem +Every service in this book is **Lumen** — the digital-wallet and ledger service +you will grow, chapter by chapter, into the complete +[`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen) +crate. Before you scaffold it in the next chapter, this one answers the question +underneath the whole project: *why does a Rust service need a framework at all, +and why this one?* No code lands in Lumen yet. By the end you will understand the +problem Firefly solves, the single dependency it arrives through, the tiers that +live behind that dependency, and the one design choice — the in-memory-to- +production adapter swap — that the rest of the book is built around. + +This is a read-and-orient chapter, not a type-along one, but it is not hand-wavy: +every term you will meet for the next nineteen chapters is defined here from first +principles, and every claim is something you can verify against the real +`samples/lumen` crate in the closing exercises. + +By the end of this chapter you will: + +- Explain the **cohesion problem** that an opinionated framework exists to solve, + and why Rust in particular feels its absence. +- Describe what Firefly *is* — a cohesive, reactive, async-native framework — and + name the battle-tested libraries it delegates to underneath. +- Read Lumen's real `Cargo.toml` and explain why a service this rich depends on + exactly **one** Firefly crate, the `firefly` facade. +- Map the four **tiers** behind the facade (foundational → platform → adapters → + starters) and say where each capability lives. +- Describe the **adapter swap** — how Lumen moves from an in-memory baseline to a + production deployment by changing wiring, not business logic. + +## Concepts you will meet + +Before the prose, here are the four ideas this chapter leans on. Each is +reintroduced in context where it first appears; this is the short version, so the +later sections read fast. + +> **Note** **Key term — framework vs. library.** A *library* is code you call: you +> stay in charge of control flow and reach into the library when you need it. A +> *framework* is code that calls you: it owns the lifecycle — startup, request +> dispatch, shutdown — and invokes the small pieces you supply. This inversion is +> the whole point of Firefly, and it is exactly the relationship Spring Boot has +> with a Java service. + +> **Note** **Key term — facade crate.** A *facade* is a single crate that +> re-exports a whole family of crates (and their macros) so that you depend on one +> name instead of many. Firefly ships its entire framework behind the `firefly` +> facade. The Spring analog is a Spring Boot *starter* — except here there is +> essentially one front door that covers everything. + +> **Note** **Key term — port and adapter.** A *port* is an abstract capability +> expressed as a trait — "something that stores events," "something that publishes +> messages" — with no implementation. An *adapter* is a concrete implementation of +> that port — an in-memory store, a PostgreSQL store, a Kafka broker. You write +> your code against the port; you pick the adapter at wiring time. This is the +> hexagonal-architecture vocabulary and it maps to Spring's interface-plus-bean +> idiom. + +> **Note** **Key term — bean and wiring.** A *bean* is an object the framework +> constructs and manages for you, then hands to whoever needs it. *Wiring* is the +> act of connecting beans together — giving each one the collaborators it +> depends on. You declare beans; the framework discovers and wires them at +> startup. This is exactly Spring's notion of a bean in an application context. + +## Step 1 — Recognize the cohesion problem Picture your first day on a new Rust microservice. Before you write one line of business logic, you face a cascade of choices. Which HTTP layer — axum, actix, @@ -23,73 +78,118 @@ understanding of how anything works. **Rust gives you infinite choice. What it does not give you is cohesion.** -The stack-assembly problem is not a skills failure — it is a tooling gap. Mature -ecosystems closed it with a single opinionated, batteries-included framework that -makes sensible choices, lets you override what matters, and enforces a consistent -idiom across every service. Firefly is that framework for Rust: it makes the -cross-cutting decisions once, so every service shares one idiom. +What just happened: you named the problem. The stack-assembly tax is not a skills +failure — it is a tooling gap. Mature ecosystems closed it with a single +opinionated, batteries-included framework that makes sensible choices, lets you +override what matters, and enforces a consistent idiom across every service. + +> **Design note.** This is the framework-vs-library inversion in practice. A pile +> of libraries leaves *you* holding the lifecycle: you decide when the HTTP server +> binds, how configuration loads, where errors turn into responses. A framework +> makes those cross-cutting decisions once, so every service that uses it shares +> one idiom — and an operator who learns one Firefly service can read all of them. + +Firefly is that framework for Rust. It makes the cross-cutting decisions once, so +every service shares one idiom — and the cost of starting service number two is +no longer a fresh round of architecture debates. + +> **Tip** **Checkpoint.** You can state the problem in one sentence: *Rust offers +> infinite choice but no built-in cohesion, and an opinionated framework supplies +> the missing cohesion.* If that sentence feels obvious, the rest of the book will +> read as "here is how Firefly supplies it." -## What is Firefly? +## Step 2 — Understand what Firefly is (and delegates to) Firefly is a **cohesive, reactive, async-native framework** for building production-grade Rust services. It makes the cross-cutting decisions for you — -HTTP middleware, configuration, caching, CQRS, messaging, security, -observability — all integrated, all consistent, with production-ready defaults -from the very first `cargo run`. - -Under the hood Firefly delegates to battle-tested libraries — `tokio` for the -runtime, `axum`/`tower` for HTTP, `serde` for serialization, `tracing` for -logging, RustCrypto for crypto — but you depend on **Firefly's ports** -(object-safe `async_trait` traits), and you select concrete adapters at wiring -time. Swap an in-memory event store for PostgreSQL, or the in-process broker for -Kafka, without touching a single line of business logic — exactly the swap Lumen -is structured to make. - -Firefly's defining principles: - -- **Composed, not constructed.** One line boots the whole service. `FireflyApplication::new("lumen").run()` - component-scans your beans, auto-wires and auto-mounts the controllers, - handlers, listeners, and scheduled tasks, self-hosts an admin dashboard, and - serves the public + management ports with graceful shutdown — the framework - assembles the object graph instead of you spelling it out in a composition - root. You write commands, queries, handlers, and routes; nothing more. +HTTP middleware, configuration, caching, Command/Query Responsibility +Segregation, messaging, security, observability — all integrated, all consistent, +with production-ready defaults from the very first `cargo run`. + +> **Note** **Key term — reactive (`Mono` / `Flux`).** *Reactive* here means a +> lazy, composable, backpressure-aware streaming model. A `Mono` is an async +> computation that yields *at most one* value; a `Flux` yields *zero or more* +> over time. They are built natively on Tokio and run end to end — from reactive +> endpoints through reactive repositories, the reactive HTTP client, and reactive +> messaging. If you have used Project Reactor in the Spring world, these are the +> same two types by the same names. You will master them in +> [the reactive model](./05-reactive-model.md). + +Firefly does not reinvent the wheel underneath. It **delegates to battle-tested +libraries** — `tokio` for the runtime, `axum`/`tower` for HTTP, `serde` for +serialization, `tracing` for structured logging, RustCrypto for cryptography. The +twist is the direction you depend on them: + +- You depend on **Firefly's ports** — object-safe `async_trait` traits — for + cross-cutting capabilities like event storage and messaging. +- You select **concrete adapters** at wiring time, as an `Arc`. + +Because of that indirection you can swap an in-memory event store for PostgreSQL, +or the in-process broker for Kafka, without touching a single line of business +logic — exactly the swap Lumen is structured to make and that Step 5 returns to. + +Firefly's defining principles, each of which a later chapter makes concrete: + +- **Composed, not constructed.** One line boots the whole service. + `FireflyApplication::new("lumen").run()` component-scans your beans, auto-wires + and auto-mounts the controllers, handlers, listeners, and scheduled tasks, + self-hosts an admin dashboard, and serves the public + management ports with + graceful shutdown — the framework assembles the object graph instead of you + spelling it out by hand. You write commands, queries, handlers, and routes; + nothing more. [Quickstart](./02-quickstart.md) walks this line stage by stage. - **Contract-first and interoperable.** The wire contract — the - `application/problem+json` shape, the `Idempotency-Key` semantics, the saga - step definitions, the event envelopes — is a stable, versioned, language-neutral - specification. Any service that honors it interoperates with a Firefly service - byte-for-byte, so Firefly slots into a polyglot fleet without bespoke glue. + `application/problem+json` error shape (RFC 9457), the `Idempotency-Key` + semantics, the saga step definitions, the event envelopes — is a stable, + versioned, language-neutral specification. Any service that honors it + interoperates with a Firefly service byte-for-byte, so Firefly slots into a + polyglot fleet without bespoke glue. - **Pluggable at the adapter layer.** Each integration point (cache, broker, - IDP, ECM, notification channel) is an object-safe port with multiple adapter - implementations selected at wiring time as an `Arc`. + identity provider, content store, notification channel) is a port with multiple + adapter implementations, selected at wiring time as an `Arc`. - **Observable by default.** `tracing` structured logging with correlation-id - enrichment, actuator health/metrics endpoints, RFC 9457 error envelopes, and a - startup banner are all on out of the box. -- **Reactive to the core.** A first-class `Mono`/`Flux` reactive surface runs - from reactive endpoints through reactive repositories, the reactive HTTP - client, and reactive EDA/CQRS — a lazy, composable, backpressure-aware - streaming model built natively on tokio. + enrichment, actuator health and metrics endpoints, RFC 9457 error envelopes, and + a startup banner are all on out of the box. +- **Reactive to the core.** The `Mono`/`Flux` surface runs from endpoints to + repositories to the HTTP client to messaging — lazy, composable, and + backpressure-aware. + +> **Note** **Key term — RFC 9457 problem responses.** RFC 9457 (which obsoletes +> RFC 7807) defines `application/problem+json` — a standard JSON shape for HTTP +> errors with a `type`, `title`, `status`, and `detail`. Firefly renders every +> handler error in this shape automatically, so your API speaks one error dialect +> from the first endpoint. You meet it for real in +> [Your First HTTP API](./06-first-http-api.md). > **Design note.** `FireflyApplication::new(name).run()` is Firefly's composition -> root — the Rust analog of Spring Boot's `SpringApplication.run`. It stands up -> the middleware, the bus, the broker, health, and metrics, then component-scans -> and wires your beans, all from one line. Configuration layers defaults → -> profile → environment, and any handler can return a `Mono` / `Flux`. If -> you have used a batteries-included framework before, this will feel familiar. +> root — the Rust analog of Spring Boot's `SpringApplication.run(App.class, args)`. +> It stands up the middleware, the bus, the broker, health, and metrics, then +> component-scans and wires your beans, all from one line. Configuration layers +> defaults → profile → environment, and any handler can return a `Mono` / +> `Flux`. If you have used a batteries-included framework before, this will feel +> familiar. -## The one-dependency facade +> **Tip** **Checkpoint.** You can name two things at once: *what* Firefly gives you +> (one cohesive, reactive, observable stack) and *what it stands on* (tokio, axum, +> serde, tracing, RustCrypto). Firefly is the cohesion layer, not a from-scratch +> reimplementation. -Here is the part that surprises people. Lumen — a service with CQRS, event -sourcing, a saga, JWT security, scheduling, and an actuator surface — declares -exactly one Firefly dependency. This is its real `Cargo.toml`: +## Step 3 — Read the one-dependency facade + +Here is the part that surprises people. Lumen — a service with Command/Query +Responsibility Segregation, event sourcing, a saga, JWT security, scheduling, and +an actuator surface — declares exactly one Firefly dependency. This is the shape +of its real `Cargo.toml`: ```toml [dependencies] -# The whole framework AND every `#[derive(...)]` / `#[...]` macro. -firefly = { version = "26.6.24" } +# The whole framework AND every `#[derive(...)]` / `#[...]` macro. The `admin` +# feature pulls in the self-hosted admin dashboard the management port mounts. +firefly = { version = "26.6.28", features = ["admin"] } # The two ecosystem crates a Firefly service still writes against directly: # axum (you author the controller handlers) and serde (your messages and -# event payloads are Serialize/Deserialize). +# event payloads are Serialize/Deserialize); serde_json encodes the event +# payloads. axum = { version = "0.7" } serde = { version = "1", features = ["derive"] } serde_json = "1" @@ -103,200 +203,147 @@ chrono = { version = "0.4", features = ["serde"] } async-trait = { version = "0.1" } ``` -The `firefly` crate is a **facade**: it re-exports every `firefly-*` crate behind -a clean path (`firefly::cqrs`, `firefly::eventsourcing`, `firefly::reactive`, -`firefly::security`, …) and re-exports every macro at the crate root. The -high-frequency surface — plus all the macros — comes in through a single glob: - -```rust -use firefly::prelude::*; -``` - -That one line gives Lumen the CQRS `Bus`, the dependency-injection `Container`, -the `Scheduler`, the saga `Saga`/`Step`, the lifecycle `Application`, the -reactive `Mono`/`Flux`, the `WebResult`/`WebError` web types, the `FireflyError` -kernel error, and every `#[derive(...)]` / `#[...]` macro the service uses. - -Lumen takes the discipline one step further: even its typed error enums — -`MoneyError`, `DomainError`, `CqrsError` mapping — hand-write `Display` and -`std::error::Error` instead of reaching for `thiserror`. The one-dependency +What just happened, block by block: + +- The first line — `firefly = { version = "26.6.28", features = ["admin"] }` — is + the *entire framework*. Every capability and every macro arrives through it. +- The `axum` / `serde` / `serde_json` block is the small surface you still write + *against directly*: you author the controller handler functions on `axum`, and + your messages and event payloads derive `serde`'s `Serialize`/`Deserialize`. +- The `tokio` / `uuid` / `chrono` block is the runtime and the id/clock crates the + wallet domain reaches for — wallet ids and event timestamps. +- `async-trait` backs the `async fn` methods on the domain's port traits. + +Notice what is *not* there: no `firefly-web`, no `firefly-cqrs`, no +`firefly-security`. You never list a `firefly-*` sub-crate by hand. + +> **Note** **Key term — prelude glob.** A *prelude* is a module of the most-used +> items that a crate invites you to import all at once with a glob (`use … ::*`). +> Firefly's high-frequency surface — plus every macro — comes in through a single +> line: +> +> ```rust,ignore +> use firefly::prelude::*; +> ``` +> +> That one import gives Lumen the CQRS `Bus`, the dependency-injection `Container`, +> the `Scheduler`, the `Saga`/`Step` orchestration types, the lifecycle +> `Application`, the reactive `Mono`/`Flux`, the `WebResult`/`WebError` web types, +> the `FireflyError` kernel error, and every `#[derive(...)]` / `#[...]` macro the +> service uses. Spring developers will recognize the move: one import instead of a +> page of them. + +Lumen takes the discipline one step further. Even its typed error enums — +`MoneyError`, `DomainError`, and the `CqrsError` mapping — hand-write `Display` +and `std::error::Error` instead of reaching for `thiserror`. The one-dependency promise holds end to end, and the chapters point it out where it matters. > **Design note.** The `firefly` facade is a single front-door crate: one > coordinate on your dependency list pulls in a curated, calendar-version-aligned -> stack, and `use firefly::prelude::*;` brings the whole high-frequency surface -> and every macro into scope at once. One dependency, no version skew to manage. +> stack, and `use firefly::prelude::*;` brings the whole high-frequency surface and +> every macro into scope at once. Many frameworks make you assemble a constellation +> of starter or plugin artifacts and keep their versions aligned by hand. Firefly +> collapses all of that into one line: there is no starter to forget and no version +> skew between subsystems like `firefly-web` and `firefly-cqrs`, because every +> `firefly-*` crate ships as one calendar-versioned release — here `26.6.28` — and +> you depend on the facade. + +> **Tip** **Checkpoint.** You can point at the single Firefly line in a real +> `Cargo.toml` and explain the other entries as the handful of ecosystem crates a +> Firefly service writes against directly. Exercise 1 has you confirm this against +> `samples/lumen/Cargo.toml` yourself. -## The tiers behind the facade +## Step 4 — Map the tiers behind the facade Behind that single crate the framework is organized into strictly-layered tiers, with a left-to-right dependency direction. Each tier may depend on the tiers to its left, never to its right; the Cargo crate graph enforces the layering. You rarely name these crates directly — the facade re-exports them — but knowing the -shape tells you where each capability lives. - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Architecture at a glance - fireflyframework-rust - - One dependency in; four strictly-layered tiers building left to right; a reactive core at the base. - - - - - - - THE FRONT DOOR - firefly + firefly-macros - one dependency · use firefly::prelude::*; - declarative macros · stable __rt contract - - - - - - - - - - - - 1 - FOUNDATIONAL - reactive base · cross-cutting - - kernelweb - configvalidators - containeri18n - + utils, session - - - - - - - 2 - PLATFORM - engines · defines ports - - cqrseda - eventsourcingorchestration - cachesecurity - + observability, … - - - - - - - 3 - ADAPTERS - implement the ports - - data-sqlxdata-mongodb - eda-kafkacache-redis - idp-* · ecm-*notifications-* - + client, webhooks - - - - - - - 4 - STARTERS - compose · ship - - starter-corestarter-web - starter-domainstarter-data - admincli - + backoffice - - - - - - - - - - - - - firefly-reactive - the Mono / Flux reactive core every tier is built on - tokio · axum · async-native - -
Firefly at a glance: a service depends only on the firefly facade (the front door); the four tiers build left to right, each depending only on the tiers to its left, all on the firefly-reactive Mono / Flux core.
-
+shape tells you where each capability lives, and *which* book chapter unlocks it. + +```text + ┌──────────────────────────────────────────────┐ + THE FRONT DOOR → │ firefly + firefly-macros │ + │ one dependency · use firefly::prelude::*; │ + └───┬───────────┬───────────┬───────────┬───────┘ + │ │ │ │ + ┌───────────────┼───────────┼───────────┼───────────┼──────────────┐ + │ Tier 1 │ Tier 2 │ Tier 3 │ Tier 4 │ builds │ + │ FOUNDATIONAL │ PLATFORM │ ADAPTERS │ STARTERS │ left→right │ + │ reactive base│ engines │ implement│ compose │ │ + │ cross-cutting│ define │ the ports│ & ship │ │ + │ │ ports │ │ │ │ + │ kernel │ cqrs │ data-sqlx│ starter-core │ + │ web │ eda │ data- │ starter-web │ + │ config │ event- │ mongodb │ starter-domain │ + │ validators │ sourcing│ eda-kafka│ starter-data │ + │ container │ orchestr.│ cache- │ admin │ + │ i18n │ cache │ redis │ cli │ + │ + utils, │ security │ idp-* · │ + backoffice │ + │ session │ + observ.│ ecm-* · │ │ + │ │ │ notif-* │ │ + └───────────────┴───────────┴───────────┴───────────┴──────────────┘ + firefly-reactive + the Mono / Flux reactive core every tier is built on + (tokio · axum · async-native) +``` + +A service depends only on the `firefly` facade (the front door). The four tiers +build left to right — each depending only on the tiers to its left — all resting +on the `firefly-reactive` `Mono`/`Flux` core. - **Foundational** crates are the vocabulary: `firefly-kernel` (errors, clock, - correlation scopes, DDD kit), `firefly-reactive` (`Mono`/`Flux`), + correlation scopes, the DDD kit), `firefly-reactive` (`Mono`/`Flux`), `firefly-web` (middleware), `firefly-config`, `firefly-validators`, `firefly-i18n`, and `firefly-container` — a full dependency-injection engine - with component scanning and stereotype derives, covered in depth in Chapter 4. -- **Platform** crates are the capabilities: caching, CQRS, EDA, event sourcing, - orchestration, scheduling, resilience, security, observability. Lumen reaches - for `firefly::cqrs`, `firefly::eventsourcing`, `firefly::orchestration`, - `firefly::scheduling`, and `firefly::security` here. + with component scanning and stereotype derives, covered in depth in + [Dependency Wiring](./04-dependency-wiring.md). +- **Platform** crates are the capabilities: caching, Command/Query Responsibility + Segregation, event-driven architecture, event sourcing, orchestration, + scheduling, resilience, security, observability. Lumen reaches for + `firefly::cqrs`, `firefly::eventsourcing`, `firefly::orchestration`, + `firefly::scheduling`, and `firefly::security`. Crucially, this tier *defines + the ports* — the `EventStore`, `Broker`, `cache::Adapter`, and + `security::Verifier` traits — that the next tier implements. - **Adapters** are the concrete integrations: the REST/reactive HTTP client, the - IDP vendors, ECM, notifications, the event transports (Kafka, RabbitMQ, - Postgres outbox, Redis Streams), and the persistence adapters (`firefly-data-sqlx` - for relational stores, `firefly-data-mongodb` for documents) — a pluggable - multi-database story that Chapter 7 builds on. Lumen ships on the in-memory + identity-provider vendors, content stores, notifications, the event transports + (Kafka, RabbitMQ, Postgres outbox, Redis Streams), and the persistence adapters + — `firefly-data-sqlx` for relational stores, `firefly-data-mongodb` for + documents. This is a pluggable multi-database story that + [Persistence](./07-persistence.md) builds on. Lumen ships on the in-memory adapters and points at the production swaps in callouts. - **Starters** bundle a sensible default stack so a service depends on one crate. Lumen's web tier is `firefly::starter_web::WebStack`, which wires the core - (`firefly::starter_core`) plus the web middleware — the stack - `FireflyApplication` builds for you at boot. + (`firefly::starter_core`) plus the web middleware — the stack `FireflyApplication` + builds for you at boot. + +> **Note** **Key term — actuator / management surface.** The *management surface* +> is a set of operational HTTP endpoints — health checks, build info, metrics, +> configuration and bean introspection — that exist for operators and tooling, not +> for end users. Firefly serves them on a *separate* port from your business API, +> so operational endpoints never leak onto the public network. This mirrors Spring +> Boot Actuator, and you reach it for the first time in +> [Quickstart](./02-quickstart.md). + +For the full per-crate catalogue see the +[Module Index](./91-appendix-modules.md). + +> **Tip** **Checkpoint.** Given a capability — "where does event storage live?" — +> you can place it in a tier (it is a *platform* port, implemented by an *adapter*) +> and name the chapter that introduces it. The tiers are a map; the rest of the +> book is a tour of it. -For the full per-crate catalogue see the [Module Index](./91-appendix-modules.md). +## Step 5 — Understand the adapter swap -## Choosing your adapters +This is the single design choice that the whole book turns on, so it earns its own +step. Lumen runs with **zero external infrastructure** — that is what makes it a +good teaching baseline and a fast test target. It boots on the in-process +`MemoryEventStore` and the in-process broker, so `cargo run` and `cargo test` need +nothing but the crate. No Postgres to start, no Kafka to provision. -Lumen runs with **zero external infrastructure** — that is what makes it a good -teaching baseline and a fast test target. It boots on the in-process -`MemoryEventStore` and the in-process broker, so `cargo run` and `cargo test` -need nothing but the crate. When you are ready for production, you change the -*wiring*, not the handlers: +When you are ready for production, you change the *wiring*, not the handlers. Each +of the swaps below is a one-place edit at the seam where the `Arc` is +constructed: - **Event store.** Swap `MemoryEventStore` for a durable adapter where the `Arc` is constructed; the `Ledger`, the projection, and every @@ -309,62 +356,96 @@ need nothing but the crate. When you are ready for production, you change the the concrete adapter crate at wiring time, so heavy SDKs stay out of services that do not use them. -This is the thread that runs through the whole book: Lumen is written so the -in-memory baseline and the production deployment differ only in a `#[bean]` -factory — the wiring the framework scans, not the business code. +> **Note** **Key term — `#[bean]` factory.** A `#[bean]` factory is a function the +> framework calls at startup to *construct* a bean — and where you decide which +> concrete adapter satisfies a port. It is the single place the swap above happens: +> the function body returns `Arc::new(MemoryEventStore::new())` in development and +> `Arc::new(SqlEventStore::new(pool))` in production, and nothing downstream +> notices. The Spring analog is an `@Bean` method on a `@Configuration` class. You +> write your first one in [Dependency Wiring](./04-dependency-wiring.md). + +What just happened: you saw why the in-memory baseline is not a toy. Because Lumen +codes against ports, the in-memory build and the production deployment differ +*only* in a `#[bean]` factory — the wiring the framework scans, not the business +code. This is the thread that runs through the whole book. + +> **Tip** **Checkpoint.** You can finish this sentence: *to take Lumen to +> production you change a `#[bean]` factory, not a handler.* If that lands, the +> book's "Lumen ships on in-memory; here is the production swap" callouts will read +> as routine rather than magical. Exercise 4 has you locate the three port traits +> behind these swaps. ## The road ahead: Lumen, chapter by chapter -The rest of the book is Lumen's growth, additive and in order. The early -chapters introduce the framework with small standalone snippets; **Lumen proper -begins in [Chapter 6](./06-first-http-api.md)**: +The rest of the book is Lumen's growth, additive and in order. The early chapters +introduce the framework with small standalone snippets; **Lumen proper begins in +[Your First HTTP API](./06-first-http-api.md)**. - **Foundations** — scaffold and boot Lumen, bind its configuration and profiles, understand how `FireflyApplication` wires the beans it scans, master `Mono`/`Flux`, and expose the first validated REST endpoints. -- **Modeling & persisting** — a read model behind a repository, the `Money` +- **Modeling and persisting** — a read model behind a repository, the `Money` value object and the `Wallet` aggregate, and the CQRS command/query split on a bus. - **Event-driven** — domain events, a projection that keeps the read model current, and the event-sourced ledger that folds its stream. - **Into microservices** — an HTTP-client sketch and the compensating transfer saga. -- **Secure · observe · ship** — JWT bearer auth and RBAC, the actuator surface, - caching, a scheduled task, the test suite, and the production entry point with - graceful shutdown and a reactive streaming endpoint. +- **Secure, observe, ship** — JWT bearer auth and role-based access control, the + actuator surface, caching, a scheduled task, the test suite, and the production + entry point with graceful shutdown and a reactive streaming endpoint. By the last page, Lumen is the complete `samples/lumen` crate — and you have written every line of it. ## Recap — what changed in Lumen -Nothing in code yet. This chapter framed the journey: +Nothing in code yet. This chapter framed the journey and stocked your vocabulary: -- The **cohesion problem** Firefly exists to solve, and the opinionated, - one-framework answer it brings to Rust. -- The **one-dependency facade** — Lumen depends on a single `firefly` crate, and +- The **cohesion problem** Firefly exists to solve — Rust offers infinite choice + but no built-in cohesion — and the framework-vs-library inversion that lets one + opinionated framework supply it. +- What **Firefly is** (a cohesive, reactive, async-native framework) and what it + *delegates to* (tokio, axum/tower, serde, tracing, RustCrypto), with you + depending on its **ports** and selecting **adapters** at wiring time. +- The **one-dependency facade** — Lumen depends on a single + `firefly = { version = "26.6.28", features = ["admin"] }`, and `use firefly::prelude::*;` brings in the whole high-frequency surface and every - macro. Even the typed errors avoid `thiserror`, so the promise holds end to - end. -- The **tiers** behind that facade (foundational → platform → adapters → - starters) and the in-memory-to-production **adapter swap** that Lumen is built - to make by changing a single `#[bean]` factory. + macro. Even the typed errors avoid `thiserror`, so the promise holds end to end. +- The **four tiers** behind that facade (foundational → platform → adapters → + starters) resting on the `firefly-reactive` core, and where each capability + lives. +- The **adapter swap** that Lumen is built to make — moving from the in-memory + baseline to production by changing a single `#[bean]` factory, never a handler. ## Exercises -1. Open `samples/lumen/Cargo.toml` and confirm the dependency list: one - `firefly`, plus `axum`/`serde`/`serde_json`/`tokio`/`uuid`/`chrono`/`async-trait`. - Note that no `firefly-*` sub-crate is listed directly. -2. Skim `samples/lumen/src/main.rs` — the single-binary crate root. List the ten - modules it declares - (`money`, `domain`, `ledger`, `commands`, `transfer`, `tcc_transfer`, - `security`, `compliance`, `web`, `housekeeping`) and predict which book part - introduces each. -3. Run `cargo doc -p firefly-sample-lumen --open` and read the crate-level - documentation. It contains the same "building block → module → Firefly - surface" table the book is organized around. -4. For each of these production swaps, find the port trait it would implement in - the facade: a Postgres event store, a Kafka broker, a Redis cache. (Hint: - `firefly::eventsourcing`, `firefly::eda`, `firefly::cache`.) - -The next chapter gets Lumen running. Turn to the [Quickstart](./02-quickstart.md). +1. **Confirm the one dependency.** Open `samples/lumen/Cargo.toml` and confirm the + dependency list: one `firefly` (with the `admin` feature), plus + `axum`/`serde`/`serde_json`/`tokio`/`uuid`/`chrono`/`async-trait`. Note that no + `firefly-*` sub-crate is listed directly. +2. **Find the single-line `main`.** Skim `samples/lumen/src/main.rs` — the + single-binary crate root. List the ten modules it declares (`commands`, + `compliance`, `domain`, `housekeeping`, `ledger`, `money`, `security`, + `tcc_transfer`, `transfer`, `web`) and predict which book part introduces each. + Confirm that `main` is genuinely one line over `FireflyApplication::new("lumen")`. +3. **Read the crate docs.** Run `cargo doc -p firefly-sample-lumen --open` and read + the crate-level documentation. It contains the same "building block → module → + Firefly surface" table the book is organized around. +4. **Locate the port traits.** For each of these production swaps, find the port + trait it would implement in the facade: a Postgres event store, a Kafka broker, + a Redis cache. (Hint: `firefly::eventsourcing::EventStore`, `firefly::eda::Broker`, + `firefly::cache::Adapter`.) These are the seams Step 5 described. +5. **Trace the prelude.** Open the `firefly` facade's `prelude` module (or its + docs) and find five types you will use repeatedly: the CQRS `Bus`, the + `Container`, `Mono`/`Flux`, `WebResult`, and `FireflyError`. Confirm they all + arrive through the single `use firefly::prelude::*;` glob. + +## Where to go next + +- Get Lumen running for the first time in **[Quickstart](./02-quickstart.md)** — + scaffold the crate, write the one-line `main`, and reach its two ports. +- Add typed, layered, profile-aware configuration in + **[Configuration](./03-configuration.md)**. +- Learn how the framework wires the object graph it scans — including your first + `#[bean]` factory — in **[Dependency Wiring](./04-dependency-wiring.md)**. diff --git a/docs/book/src/02-quickstart.md b/docs/book/src/02-quickstart.md index 864b590..02292ee 100644 --- a/docs/book/src/02-quickstart.md +++ b/docs/book/src/02-quickstart.md @@ -1,87 +1,175 @@ # Quickstart -> By the end of this chapter **Lumen** — the digital-wallet and ledger service -> you will grow across the rest of the book — exists as a real crate: it -> compiles, prints a banner, serves a live actuator, and shuts down gracefully. -> It does almost nothing yet. That is the point. Everything from here on is -> *additive*: every later chapter slices a little more out of the finished -> [`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen) -> crate and folds it back into the story, and nothing you write now gets thrown -> away. - -This chapter takes you from an empty directory to a running Lumen process in a -few minutes. Two paths get you there: the `firefly` CLI (fastest) or plain -`cargo`. Either way, the destination is the same — a binary crate whose *only* -Firefly dependency is the [`firefly`](./21-declarative-macros.md) facade. - -## Prerequisites +This is where **Lumen** — the digital-wallet and ledger service you will grow +across the rest of the book — first comes to life. By the end of this chapter +Lumen exists as a real crate: it compiles, prints a banner, serves a live +management surface, and shuts down gracefully. It does almost nothing else yet, +and that is deliberate. Everything from here on is *additive* — every later +chapter slices a little more out of the finished +[`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen) +crate and folds it back into the story, and nothing you write now gets thrown +away. + +We will take two passes at the same goal. First we will scaffold the crate with +the `firefly` CLI (the fast path), then we will build the identical crate by +hand so that every line is something you typed and understand. Both land on the +same single-binary shape that the rest of the book assumes. + +By the end of this chapter you will: + +- Scaffold a Firefly project two ways — with the `firefly new` CLI and from a + bare `cargo new`. +- Understand why a Firefly service depends on a *single* crate, the `firefly` + facade, instead of a constellation of starter artifacts. +- Write the one-line `main` that boots and serves the whole service, and explain + what each stage of `run()` does. +- Run Lumen and reach its two ports — the public API on `8080` and the + management surface (actuator, admin dashboard, API docs) on `8081`. +- Read the startup report and confirm Lumen's health and build metadata with + `curl`. + +## Concepts you will meet + +Before the first command, here are the three ideas this chapter leans on. Each +is reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — facade crate.** A *facade* is a single crate that +> re-exports a whole family of crates (and their macros) so that you depend on +> one name instead of many. Firefly ships its entire framework behind the +> `firefly` facade. The Spring analog is a Spring Boot *starter* — except here +> there is exactly one, and it covers everything. + +> **Note** **Key term — bean.** A *bean* is an object the framework constructs +> and manages for you, then hands to whoever needs it. You declare beans; the +> framework discovers them at startup and wires them together. This is exactly +> Spring's notion of a bean managed by the application context. + +> **Note** **Key term — actuator / management surface.** The *management +> surface* is a set of operational HTTP endpoints — health checks, build info, +> metrics, configuration introspection — that exist for operators and tooling, +> not for end users. Firefly serves them on a separate port from your business +> API. This mirrors Spring Boot Actuator. + +## Step 1 — Check your toolchain + +You need a recent stable Rust toolchain and nothing else. Lumen's default stack +requires **no external infrastructure** — its event store, event broker, and +read model are all pure Rust running in-process. ```bash rustc --version # 1.88 or later cargo --version ``` -That is all you need. Lumen's default stack requires **no external -infrastructure** — the event store, the event broker, and the read model are -all pure Rust, in-process. You will swap each of them for real infrastructure -(Postgres, Kafka) in [Production & Deployment](./20-production.md), but never -before you are ready. +> **Tip** **Checkpoint.** Both commands print a version. If `rustc` reports +> anything below 1.88, update with `rustup update stable` before continuing. -## Path A — scaffold with the `firefly` CLI +You will swap the in-process pieces for real infrastructure (Postgres, Kafka) in +[Production & Deployment](./20-production.md), but never before you are ready — +the whole book runs against the in-process defaults. -Install the developer CLI once, then scaffold the project: +## Step 2 — Scaffold with the `firefly` CLI (Path A) + +The fastest way to a running service is the developer CLI. Install it once, then +ask it to generate the project. + +> **Note** **Key term — archetype.** An *archetype* is a project template that +> decides the starting shape of your crate — which modules exist, which Firefly +> features are switched on, and what the example code looks like. The CLI ships +> several (`core`, `web-api`, `web`, `hexagonal`, `library`, `cli`). The Spring +> analog is a Spring Initializr "project type" plus its preselected +> dependencies. ```bash cargo install --path crates/cli # from a checkout of the framework -# or: cargo install firefly-cli +# or, once published: cargo install firefly-cli firefly new lumen --archetype web-api --features web,cqrs --git cd lumen cargo run ``` -`firefly new` generates a Cargo crate with a `src/` tree, a `firefly.yaml`, a -`.gitignore`, a `README.md`, a `Dockerfile`, and a `tests/` directory. The -`web-api` archetype is the right starting shape for Lumen: a web service with -the CQRS bus already wired. See [The CLI](./19-cli.md) for every archetype and -generator. +What just happened: `firefly new` wrote a Cargo crate with a `src/` tree, a +`firefly.yaml`, a `.gitignore`, a `README.md`, a `Dockerfile`, and a `tests/` +directory, then (because of `--git`) initialized a Git repository with a first +commit. The `web-api` archetype is the right starting shape for Lumen — a web +service with the CQRS bus already wired — and `--features web,cqrs` switches on +exactly those two subsystems. `cargo run` compiles and boots the service. + +> **Note** **Key term — CQRS.** *Command/Query Responsibility Segregation* is a +> pattern that routes state-changing **commands** and read-only **queries** +> through separate handlers on a shared *bus*. You will build Lumen's command +> and query handlers in later chapters; for now it is enough that the `cqrs` +> feature reserves the wiring. + +> **Tip** Run `firefly new --list` to print every archetype and feature flag, or +> `firefly new lumen --dry-run` to preview the exact file plan without writing a +> single file. See [The CLI](./19-cli.md) for the full generator catalogue. -> **Tip** — Run `firefly new --list` to see every archetype (`core`, `web-api`, -> `web`, `hexagonal`, `library`, `cli`) and feature flag, or -> `firefly new lumen --dry-run` to preview the plan without writing files. +> **Tip** **Checkpoint.** After `cargo run` you should see the Firefly banner +> followed by a `::`-prefixed startup report and two URLs (the admin dashboard +> and the API docs). If you got that far, skip to [Step 7](#step-7--run-it). If +> you want to understand every generated line, do Steps 3–6 by hand instead. -## Path B — start from cargo +## Step 3 — Build the crate by hand (Path B) -If you would rather see every line yourself, create the crate by hand. This is -exactly the shape `samples/lumen` has, so the rest of the book lines up with it -listing for listing. +The CLI is convenient, but the rest of the book lines up with `samples/lumen` +listing for listing, and the surest way to follow along is to type the crate +yourself. Start from a bare Cargo binary. ```bash cargo new lumen cd lumen ``` -Lumen's `Cargo.toml` makes the one-dependency story concrete. The whole -framework — CQRS, dependency injection, the reactive web stack, event sourcing, -saga orchestration, scheduling, security, observability — and *every* -`#[derive(...)]` / `#[...]` macro arrive through a single crate: +What just happened: `cargo new` created a binary crate — a `Cargo.toml` and a +placeholder `src/main.rs`. Over the next three steps you will replace both with +Lumen's real contents. + +> **Tip** **Checkpoint.** `ls` shows a `Cargo.toml` and a `src/` directory. +> `cargo run` prints `Hello, world!`. That placeholder is the last code in this +> book that Firefly does *not* manage for you. + +## Step 4 — Depend on the one crate that is the framework + +Open `Cargo.toml`. This is where the one-dependency story becomes concrete. The +whole framework — CQRS, dependency injection, the reactive web stack, event +sourcing, saga orchestration, scheduling, security, observability — and *every* +`#[derive(...)]` / `#[...]` macro arrive through a single crate. ```toml # Cargo.toml [dependencies] # The one-dependency front door: the `firefly` facade re-exports the whole # framework AND every macro. Generated code resolves runtime types through the -# facade, so Lumen never lists the underlying `firefly-*` crates. -firefly = "26.6.24" +# facade, so Lumen never lists the underlying `firefly-*` crates. The `admin` +# feature pulls in the self-hosted admin dashboard the management port mounts. +firefly = { version = "26.6.28", features = ["admin"] } +``` -# The two ecosystem crates a Firefly service still writes against directly: -# axum (you author the controller handlers) and serde (your messages and event -# payloads are Serialize/Deserialize). serde_json encodes the event payloads. +What just happened: that one line is the entire framework. Every later chapter +adds *code*, not dependencies — you will not edit this `firefly` line again. + +> **Design note.** Many frameworks make you assemble a constellation of starter +> or plugin artifacts and keep their versions aligned by hand. Firefly collapses +> all of that into one `firefly` line: there is no starter to forget and no +> version skew between subsystems like `firefly-web` and `firefly-cqrs` — every +> `firefly-*` crate ships as one calendar-versioned release (here `26.6.28`), +> and you depend on the facade. + +A Firefly service still writes directly against a few ecosystem crates: `axum` +(you author the controller handlers), `serde` / `serde_json` (your messages and +event payloads are serializable), the async runtime, and the id/clock crates the +domain uses. Add them, plus the feature flag that gates the streaming endpoint: + +```toml +# The ecosystem crates a Firefly service still uses directly. axum = "0.7" serde = { version = "1", features = ["derive"] } serde_json = "1" -# The async runtime, and the id/clock crates the domain uses later. +# The async runtime for `#[tokio::main]`, and the id/clock crates the domain +# uses for wallet ids and event timestamps. tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "signal"] } uuid = { version = "1", features = ["v4"] } chrono = "0.4" @@ -89,24 +177,34 @@ async-trait = "0.1" [features] # The reactive streaming endpoint is feature-gated so the teaching baseline -# stays lean; chapter 20 turns it on. It needs nothing beyond `firefly`. +# stays lean; the production chapter turns it on. It needs nothing beyond the +# `firefly` facade. default = [] streaming = [] ``` -> **Design note.** Many frameworks make you assemble a constellation of -> starter or plugin artifacts and keep their versions aligned by hand. Firefly -> collapses all of that into one `firefly` line: there is no starter to forget -> and no version skew between subsystems like `firefly-web` and `firefly-cqrs` — -> every `firefly-*` crate ships as one calendar-versioned release, and you -> depend on the facade. +What just happened: you declared the handful of crates you will write code +against directly, and a `streaming` feature flag that stays off by default. +Everything else flows in through `firefly`. -## A one-line `main` +> **Tip** **Checkpoint.** Run `cargo build`. It downloads and compiles the +> framework (the first build is the slow one). A clean compile here means the +> facade and your direct dependencies all resolve. -A Firefly service has one entry point: `main`. There is **no composition root, -no `build_app`, and no application struct** to assemble by hand. Lumen is a -single-binary crate — `src/main.rs` is the crate root: a few `mod` declarations -and a `main` that hands the whole service to the framework in one line: +## Step 5 — Write the one-line `main` + +A Firefly service has exactly one entry point: `main`. There is **no composition +root, no `build_app`, and no application struct** to assemble by hand. Lumen is a +single-binary crate, so `src/main.rs` is the crate root — a few `mod` +declarations and a `main` that hands the whole service to the framework. + +> **Note** **Key term — composition root.** The *composition root* is the one +> place in a program where the object graph is assembled — where every component +> is constructed and connected. In many frameworks you write this by hand. In +> Firefly the framework *is* the composition root: it scans your beans and wires +> them, so you never spell out the graph in a function. + +Replace the contents of `src/main.rs` with the module list and entry point: ```rust,ignore // src/main.rs @@ -129,89 +227,114 @@ async fn main() -> Result<(), firefly::BoxError> { } ``` -`FireflyApplication::new("lumen").run()` is the Rust analog of Spring Boot's -`SpringApplication.run(App.class, args)`. That single call boots and serves the -whole service. Everything else in the crate is *declarative app code* the -framework discovers — there is nothing to wire by hand. (The two service -constants live next to the HTTP surface in `src/web.rs`:) - -```rust,ignore -// src/web.rs -/// Lumen's application name (banner + `/actuator/info`). -pub const APP_NAME: &str = "lumen"; - -/// The released framework version, surfaced in the banner. -pub const VERSION: &str = firefly::VERSION; -``` - -### What `run()` does, step by step - -When `run()` is called, `FireflyApplication` performs the entire boot pipeline a -service used to hand-roll in a composition root: +What just happened, line by line: + +- The `mod` declarations name the modules Lumen will grow into. They are listed + now so `main.rs` never changes again; you will fill each in across the book. + Until a module file exists this list will not compile, so when you follow along + for real you add the `mod` line in the same chapter that adds the module. For + this quickstart, the only one you need is whatever you choose to keep — the + point is the shape of `main`. +- `#[tokio::main]` turns `async fn main` into a normal `main` backed by the Tokio + runtime, which Firefly needs because the whole stack is asynchronous. +- `Result<(), firefly::BoxError>` is the return type. `BoxError` is Firefly's + boxed error type (`Box`); returning it lets + you use `?` on the bootstrap and lets a startup failure surface as a non-zero + exit. +- `firefly::FireflyApplication::new("lumen").run().await` is the whole service. + `new("lumen")` names the application (the name shows up in the banner and in + `/actuator/info`); `.run().await` boots and serves it. + +> **Design note.** `FireflyApplication::new(name).run()` is the Rust analog of +> Spring Boot's `SpringApplication.run(App.class, args)`. That single call *is* +> the composition root — the framework assembles the object graph from the beans +> it scans rather than you spelling it out in a function. Nothing is reflective +> or hidden: the startup report (Step 7) logs exactly what was wired, so "what is +> running" is printed line-by-line at boot. + +If you want to follow along with the smallest thing that compiles, drop the `mod` +lines and keep just the `main` function and the `#![allow(dead_code)]` attribute. +The full module list above is the real Lumen shape the rest of the book assumes. + +> **Note** **Key term — application name and version.** Lumen keeps its name and +> version in two constants next to its HTTP surface, in `src/web.rs`. The version +> is sourced from the framework itself, so it tracks the release you depend on: +> +> ```rust,ignore +> // src/web.rs +> /// Lumen's application name (banner + `/actuator/info`). +> pub const APP_NAME: &str = "lumen"; +> +> /// The released framework version, surfaced in the banner. +> pub const VERSION: &str = firefly::VERSION; +> ``` + +## Step 6 — Understand what `run()` does + +`run()` is one line in your code and an entire boot pipeline underneath — the +work a service used to hand-roll in a composition root. Knowing the stages pays +off in every later chapter, because each chapter adds a bean that one of these +stages discovers. In order, `run()`: - **Builds the web stack** — the RFC 9457 problem renderer, correlation-id propagation, idempotency replay, the in-process cache, the CQRS bus, the event - broker, the health and metrics registries, the scheduler, plus the web + broker, the health and metrics registries, the scheduler, and the web batteries (CORS, security headers, request metrics, the access log). - **Component-scans the DI container** — it auto-registers the framework's - infrastructure beans, then discovers and wires every app bean: your + infrastructure beans, then discovers and wires every app bean you declared: `#[derive(Configuration)]` + `#[bean]` factories, `#[derive(Controller)]` - controllers, and `#[autowired]` fields. + controllers, and `#[autowired]` fields. Any `async fn` bean factory (a DB pool, + a broker dial) is awaited here so async beans are live before anything resolves + them — and a construction error aborts startup (fail-fast). - **Auto-configures the CQRS bus** — correlation propagation always; the read-cache middleware whenever a `QueryCache` bean is present. - **Auto-discovers security** — the `FilterChain` and `BearerLayer` DI beans (Spring's `SecurityFilterChain`), layered onto the API with no `.security(...)` - call. + call needed. - **Auto-mounts every controller** — each `#[rest_controller]` is mounted from - the container (state resolved automatically), and every `RouteContributor` - bean's routes are merged in. -- **Drains the discovered handlers** — the inventory-registered CQRS command / - query handlers, EDA event listeners, and `#[scheduled]` tasks. -- **Self-hosts the admin dashboard** on the management port, wired to the live - components with real env / config / mappings data, and auto-serves the - generated OpenAPI docs (Swagger UI + ReDoc). -- **Prints a pyfly/Spring-style startup report** — a line-by-line log of the - active profiles, every discovered bean, the mounted route table, and the - handler/listener/scheduled counts — then **serves the public + management - ports with graceful shutdown**. - -A few things to notice, because they recur in every chapter: + the container with its state resolved automatically, and every + `RouteContributor` bean's routes are merged in. +- **Drains the discovered handlers** — the inventory-registered CQRS command and + query handlers, EDA event listeners, and `#[scheduled]` tasks, including the + ones declared as bean methods that autowire their collaborators. +- **Builds the OpenAPI docs** from the live inventory and self-hosts the admin + dashboard, both on the management port, wired to the real components. +- **Prints a Spring-style startup report** — the active profiles, every + discovered bean, the mounted route table, and the handler/listener/scheduled + counts — then **serves the public + management ports with graceful shutdown**. + +A few properties recur in every chapter, so notice them now: - **No `main` churn.** As Lumen grows a controller, a CQRS bus, an event-sourced ledger, and a security chain, `main` never changes — the new beans are *discovered*, not threaded through an entry point. -- **Two ports.** The public API serves on `8080` and the management surface - (`/actuator/*` + the self-hosted `/admin` dashboard) on `8081` by default, so - management never leaks onto the public network. +- **Two ports.** The public API serves on `8080`; the management surface + (`/actuator/*` plus the self-hosted `/admin` dashboard plus the API docs) on + `8081` by default — so operational endpoints never leak onto the public + network. - **`FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`** override the bind - addresses from the environment (defaulting to `0.0.0.0:8080` / `0.0.0.0:8081`) - — your first taste of the typed configuration story in + addresses from the environment (defaulting to `0.0.0.0:8080` / + `0.0.0.0:8081`). That is your first taste of the typed configuration story in [Configuration](./03-configuration.md). - **Graceful shutdown is built in.** `run()` traps SIGINT/SIGTERM and drains in-flight requests before exiting; a cancelled run is a clean shutdown, not an error. -> **Design note.** `FireflyApplication::new(name).run()` *is* the composition -> root — the framework assembles the object graph from the beans it scans, -> rather than you spelling it out in a function. Nothing is reflective or -> hidden: the startup report logs exactly what was wired (every bean, every -> route, every handler), so "what is running" is printed line-by-line at boot. - -> **Testing seam.** `bootstrap()` is the sibling of `run()`: it assembles the -> same app but returns it *without serving*, so the tests can drive the fully -> wired public router in-process with no socket bound. You will lean on that -> hard in [Your First HTTP API](./06-first-http-api.md) and -> [Testing](./18-testing.md). +> **Note** **Testing seam.** `bootstrap()` is the sibling of `run()`: it +> assembles the same app but returns a `Bootstrapped` value *without serving*, so +> tests can drive the fully wired public router (`Bootstrapped::api_router`) +> in-process with no socket bound. You will lean on that hard in +> [Your First HTTP API](./06-first-http-api.md) and [Testing](./18-testing.md). -## Run it +## Step 7 — Run it ```bash cargo run ``` -You will see the Firefly banner, then the line-by-line startup report — the -active profiles, the discovered beans, the auto-mounted routes, and the -handler/listener/scheduled counts — followed by the admin and API-docs URLs: +You will see the Firefly banner (ASCII art plus the framework version, your app +name, and the active profile), then the line-by-line startup report, followed by +the admin and API-docs URLs: ```text :: admin dashboard :: http://0.0.0.0:8081/admin/ @@ -219,46 +342,78 @@ handler/listener/scheduled counts — followed by the admin and API-docs URLs: :: active profiles :: default :: beans (…) :: :: routes (…) :: +:: cqrs handlers: … | event listeners: … | scheduled tasks: … | controllers: … :: +:: openapi :: … operations | … component schemas (served at /v3/api-docs) :: ``` -Even with no business routes of your own yet, the actuator is live on the -management port: +What just happened: the framework booted the whole pipeline from Step 6 and is +now serving both ports. The `:: beans ::`, `:: routes ::`, and counts lines are +the inventory the framework wired — right now they are small because Lumen has no +business logic yet, and they grow as you add chapters. + +> **Tip** **Checkpoint.** The process stays running and the last lines show the +> two URLs above. Open `http://localhost:8081/admin/` in a browser to see the +> self-hosted dashboard. Leave `cargo run` running in this terminal and use a +> second terminal for the `curl` checks below. + +## Step 8 — Confirm health and build metadata + +Even with no business routes of your own, the actuator is live on the management +port. From a second terminal: ```bash # Liveness / readiness — on the management port, never the public one. curl localhost:8081/actuator/health # {"status":"UP", ...} +``` +What just happened: `/actuator/health` aggregates every health indicator the +framework registered and reports the overall `status`. With the in-process +defaults everything is `"UP"`. + +```bash # Build metadata — the app name and version flow straight from -# `FireflyApplication::new(...).version(...)`. +# `FireflyApplication::new("lumen")` and the framework version. curl localhost:8081/actuator/info -# {"app":{"name":"lumen","version":"26.6.24"}, ...} +# {"app":{"name":"lumen","version":"26.6.28"},"runtime":{...},"build":{...}} ``` +What just happened: `/actuator/info` echoes the application name you passed to +`new(...)` and the version, alongside runtime and build details. Change the name +in `main` and this endpoint follows on the next run. + +> **Tip** **Checkpoint.** Both `curl`s return JSON: health reports +> `"status":"UP"` and info reports `"app":{"name":"lumen", ...}`. If `curl` can +> connect but to neither path, confirm you are hitting `8081` (management), not +> `8080` (public). The public port has no `/actuator/*`. + ## What you got for free Without writing any of it yourself, Lumen already has: - **RFC 9457 problem responses.** Any handler error renders as - `application/problem+json`, and a panic is caught and rendered as a 500 - problem. (You will use this from the very first endpoint in chapter 6.) + `application/problem+json`, an unmatched route returns a proper 404 problem + document (not a blank body), and a panic is caught and rendered as a 500 + problem. You will use this from the very first endpoint in chapter 6. - **Correlation IDs.** Every response echoes an `X-Correlation-Id`; an incoming one is honored and scoped through the whole request. - **Idempotency.** Every `POST`/`PUT`/`PATCH` carrying an `Idempotency-Key` header is recorded; repeating the request replays the stored response, and reusing the key with a different body is a `409`. - **A management surface.** `/actuator/{health,info,metrics,env,beans,mappings, - conditions,...}` (the `beans` / `mappings` / `conditions` DI-introspection - reports mirror Spring Boot Actuator's) plus a self-hosted `/admin` dashboard, - on a separate listener. + conditions,...}` (the `beans` / `mappings` / `conditions` reports mirror Spring + Boot Actuator's DI introspection) plus a self-hosted `/admin` dashboard, on a + separate listener. - **Auto-generated API docs.** Swagger UI (`/swagger-ui`), ReDoc (`/redoc`), and the OpenAPI 3.1 spec (`/v3/api-docs`) are served automatically on the - **management** port (beside actuator + admin, not the public API) — zero app code. -- **Graceful shutdown.** `run()` traps SIGINT/SIGTERM and drains. + **management** port (beside actuator and admin, not the public API) — zero app + code. +- **Graceful shutdown.** `run()` traps SIGINT/SIGTERM and drains in-flight + requests. > **Design note.** Health, info, and metrics on a dedicated management port, a > self-hosted admin dashboard, auto-generated API docs, and production-grade -> request middleware — all stood up by a single `FireflyApplication::new(...).run()` +> request middleware — all stood up by a single `FireflyApplication::new(...).run()`, > with no config file to author first and no annotations to remember. This is > Firefly's actuator surface, on by default. @@ -266,11 +421,20 @@ Without writing any of it yourself, Lumen already has: | Before | After this chapter | |--------|--------------------| -| empty directory | a compiling `firefly-sample-lumen` crate with one Firefly dependency | +| empty directory | a compiling crate whose only Firefly dependency is the `firefly` facade | | no entry point | a one-line `main` over `FireflyApplication::new("lumen").run()` | | nothing to run | a live actuator + admin on `:8081`, a public API on `:8080`, auto-generated docs, graceful shutdown | | — | `APP_NAME` / `VERSION` constants that name the service and feed `/actuator/info` | +You also now know: + +- Why a Firefly service depends on one crate — the `firefly` facade — instead of + many starters, and how that avoids version skew. +- That `run()` is a full boot pipeline: build the web stack, component-scan the + DI container, auto-configure CQRS, auto-discover security, auto-mount + controllers, drain handlers, self-host admin and docs, then serve two ports. +- That `bootstrap()` is the test seam that returns the wired app without serving. + Lumen is now a real, runnable service that happens to have no business logic. Every subsequent chapter fills that emptiness in — never by rewriting `main`, only by declaring more beans for the framework to discover. @@ -278,29 +442,30 @@ only by declaring more beans for the framework to discover. ## Exercises 1. **Move the ports.** Start Lumen with `FIREFLY_SERVER_ADDR=127.0.0.1:9090 - FIREFLY_MANAGEMENT_ADDR=127.0.0.1:9091 cargo run`, then `curl - localhost:9091/actuator/health`. Confirm the public and management surfaces - moved independently — this is the seam [Configuration](./03-configuration.md) - builds on. + FIREFLY_MANAGEMENT_ADDR=127.0.0.1:9091 cargo run`, then + `curl localhost:9091/actuator/health`. Confirm the public and management + surfaces moved independently — this is the seam + [Configuration](./03-configuration.md) builds on. 2. **Read your own metadata.** `curl localhost:8081/actuator/info` and find the `app.name` / `app.version` values. Change the name passed to `FireflyApplication::new(...)`, re-run, and watch the banner and `/actuator/info` both follow. -3. **Read the startup report.** Run Lumen and read the line-by-line boot log: - the active profiles, the discovered beans, the auto-mounted routes, and the - handler/listener/scheduled counts. This is the inventory the framework wired. +3. **Read the startup report.** Run Lumen and read the line-by-line boot log: the + active profiles, the discovered beans, the auto-mounted routes, and the + handler/listener/scheduled counts. This is the inventory the framework wired — + note how short it is today, then revisit it after a later chapter. 4. **Provoke graceful shutdown.** Run Lumen, then press `Ctrl-C`. Notice the - process exits cleanly (no stack trace): `run()` treated the signal as a + process exits cleanly with no stack trace: `run()` treated the signal as a shutdown, not a failure. 5. **Preview the scaffold.** Even if you took Path B, run `firefly new lumen2 - --archetype web-api --dry-run` and compare the generated plan to the files - you wrote by hand. + --archetype web-api --features web,cqrs --dry-run` and compare the generated + plan to the `Cargo.toml` and `main.rs` you wrote by hand. ## Where to go next - Add typed, layered, profile-aware configuration in **[Configuration](./03-configuration.md)** — and replace those raw - `std::env::var` calls. + `FIREFLY_*` environment overrides with real properties. - Learn how the framework wires the object graph it scans in **[Dependency Wiring](./04-dependency-wiring.md)**. - Give Lumen its first real endpoints in diff --git a/docs/book/src/03-configuration.md b/docs/book/src/03-configuration.md index 6b113e2..5c19ede 100644 --- a/docs/book/src/03-configuration.md +++ b/docs/book/src/03-configuration.md @@ -1,58 +1,120 @@ # Configuration -> By the end of this chapter Lumen reads its identity and its bind addresses -> from configuration instead of hard-coding them: the `app_name` and -> `app_version` that flow into the banner and `/actuator/info`, and the -> `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` overrides `FireflyApplication` -> already honors. You will also see the typed, layered, profile-aware machinery -> Lumen grows into as it moves toward production. - -In the last chapter Lumen named itself with two `pub const` strings, and -`FireflyApplication` pulled its ports straight off the environment -(`FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`). That is the right starting -point — but a real wallet service runs in dev, in CI, and in prod, and each -environment wants different ports, log levels, and (eventually) database URLs. -`firefly-config` provides **typed, layered configuration binding**: you -declare a `serde`-deserializable struct, call `load`/`load_from_profile`, and the -loader merges sources in precedence order, resolves the active profile, resolves -`${...}` placeholders, and binds the flat dot-keyed map onto your struct. If -you've used a batteries-included framework before, the shape will feel familiar. +In the [Quickstart](./02-quickstart.md) Lumen named itself with two `pub const` +strings, and `FireflyApplication` pulled its bind addresses straight off the +environment (`FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`). That is the +right starting point — but a real wallet service runs in dev, in CI, and in +prod, and each environment wants different ports, log levels, and (eventually) +database URLs. Hard-coded literals do not survive that journey. + +This chapter is where those literals stop being literals and start coming from +files and the environment, in a typed, layered, profile-aware way — the same +shape Spring Boot's `@ConfigurationProperties` gives a Java service, ported to +plain `serde` structs. Everything here is *additive*: the one-line `main` from +the Quickstart does not change, and the constants you wrote keep working while +you learn the machinery that will eventually replace them. + +By the end of this chapter you will: + +- Define a configuration as a plain `serde` struct and **bind** flat, + dot-keyed values onto it with the type-driven binder. +- Load configuration from `application.yaml`, a profile-specific overlay, and + the environment, and explain the **precedence chain** that decides who wins. +- Resolve `${...}` placeholders and reason about environment-beats-config + ordering. +- Turn a config struct into an **injectable bean** with + `#[derive(ConfigProperties)]`, optionally validated at startup. +- Stand up Lumen's datasource and security layer **from `application.yaml`** + with one awaited auto-configure call each — no container, no builder chains. +- Mask secrets, reload at runtime, and pull configuration from a config server. + +## Concepts you will meet + +Before the first struct, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — configuration property.** A *configuration property* is +> a single named value your program reads at startup — `web.addr`, +> `cache.ttl`, `datasource.url`. Firefly represents the whole set as a **flat, +> dot-keyed map** of strings (`{"web.addr": "127.0.0.1:8080", ...}`) and then +> *binds* it onto your typed struct. The Spring analog is a property in +> `application.properties` / `application.yaml`. + +> **Note** **Key term — source.** A *source* is anything that produces some of +> those flat entries: a YAML file, the process environment, hard-coded +> defaults, CLI flags, a remote config server. Firefly's `Source` trait has one +> job — hand back a `HashMap`. The Spring analog is a +> `PropertySource`. + +> **Note** **Key term — profile.** A *profile* names an environment — +> `dev`, `test`, `staging`, `prod` — and selects an extra YAML overlay +> (`application-prod.yaml`) layered on top of the base file. This is exactly +> Spring's notion of an active profile, down to the `dev,cloud` comma syntax. + +> **Note** **Key term — binding.** *Binding* is the act of decoding the flat +> string map onto a typed struct: `"9090"` becomes a `u16`, `"alpha,beta"` +> becomes a `Vec`, `"true"` becomes a `bool`. The binder is +> **type-driven** — the target field's type decides how each string is parsed. +> Spring calls the same idea relaxed binding onto a `@ConfigurationProperties` +> class. > **Design note.** Firefly binds a profile-aware `application.yaml` → profile → -> environment hierarchy onto typed structs. The flattening and binding rules are -> specified precisely (see below) so the same `application.yaml` produces the -> same keys deterministically — Firefly treats this determinism as a guarantee, -> not an accident. +> environment hierarchy onto typed structs, and the flattening and binding +> rules are specified precisely so the same `application.yaml` produces the same +> keys deterministically. Firefly treats this determinism as a guarantee, not an +> accident — there is no general-purpose YAML engine deciding things behind your +> back. -## Where Lumen is today: app identity +## Step 1 — See where Lumen is today: app identity as configuration -Recall the one-line bootstrap from the Quickstart — `main()` is a single -`FireflyApplication` call that carries the app name and version: +You do not have to write any config to follow this step — you already have +config, you just spelled it as constants. Recall Lumen's bootstrap. The +Quickstart's `main` was the bare form; `src/web.rs` keeps a fuller `bootstrap` +helper that also stamps the version: ```rust,ignore -// src/main.rs -firefly::FireflyApplication::new("lumen") - .version(firefly::VERSION) +// src/web.rs — the two values that name the service +pub const APP_NAME: &str = "lumen"; +pub const VERSION: &str = firefly::VERSION; + +firefly::FireflyApplication::new(APP_NAME) + .version(VERSION) .run() .await ``` -Under the hood those two values become `CoreConfig.app_name` / `app_version` — -plain configuration: every field of `CoreConfig` is a knob, and the two -Lumen sets are exactly the values -`/actuator/info` reports and the banner prints. The remaining fields default -(in-memory cache, in-process broker, a fresh CQRS bus), which is why a bare -`cargo run` needs no infrastructure. Promoting any of those to real -infrastructure is a one-field change you will make in -[Production & Deployment](./20-production.md); the config story in this chapter -is how those values stop being literals and start coming from files and the +What just happened: those two values become `CoreConfig.app_name` / +`CoreConfig.app_version` inside the framework — plain configuration. +`FireflyApplication::new(name)` writes `app_name`; `.version(v)` writes +`app_version`. Every other field of `CoreConfig` is a knob too, and the two +Lumen sets are exactly the values `/actuator/info` reports and the banner +prints. + +> **Note** **Key term — `CoreConfig`.** `CoreConfig` is the framework's own +> configuration struct (CORS, security headers, idempotency, the app name and +> version, …). `FireflyApplication` carries one and lets you tune it with +> `.configure(|c| ...)`. The remaining fields default — an in-memory cache, an +> in-process broker, a fresh CQRS bus — which is why a bare `cargo run` needs no +> infrastructure. The Spring analog is the bundle of `server.*` / `spring.*` +> properties Spring Boot binds for you. + +Promoting any of those defaults to real infrastructure is a one-field change you +will make in [Production & Deployment](./20-production.md). The config story in +*this* chapter is the general machinery underneath: how a value like an address +stops being a literal in Rust and starts arriving from a file or the environment. -## Defining configuration +> **Tip** **Checkpoint.** You can already prove identity is configuration: +> `curl localhost:8081/actuator/info` (management port) and read back +> `"app":{"name":"lumen","version":"..."}`. Change the string passed to +> `new(...)`, re-run, and the banner and that endpoint both follow. + +## Step 2 — Define a configuration struct -A configuration struct is plain `serde`. Nested structs become nested dot-keyed -sections (`web.port`, `cache.adapter`). Here is the shape Lumen would adopt as -it outgrows the two constants: +A configuration struct is plain `serde`. There is no special base type to +inherit and no attribute to remember — nested structs simply become nested +dot-keyed sections (`web.addr`, `web.admin_addr`). Here is the shape Lumen would +adopt as it outgrows the two constants. ```rust use serde::Deserialize; @@ -73,122 +135,270 @@ struct LumenConfig { } ``` -The binder is **type-driven**: `"9090"` binds onto a `u16`, `"alpha,beta"` -splits onto a `Vec`, `"true"` parses onto a `bool`, and missing keys -produce zero values — so plain `#[derive(Deserialize)]` structs bind without -`#[serde(default)]`. +What just happened: you declared three top-level keys — `name`, the `web` +section, and a `tags` list — purely with `serde`. The binder reaches `web.addr` +by walking `LumenConfig.web` → `Web.addr`, and it reaches each element of `tags` +by splitting a comma-joined string. + +Why it matters: the binder is **type-driven**, so you rarely need +`#[serde(default)]`. A missing key produces the type's zero value — `0` for an +integer, `""` for a `String`, `false` for a `bool`, an empty `Vec` for a list — +exactly like a zero-valued struct. That is a deliberate parity choice with the +Go and pyfly ports. + +> **Note** **Key term — relaxed key.** Keys are normalized at the door: +> lower-cased, with kebab-case dashes folded to snake_case underscores. So +> `admin-addr:` written in YAML binds the `admin_addr` serde field, and +> `WEB.ADDR` from the environment lands on the same `web.addr` key as a YAML +> `web.addr`. Spring calls this relaxed binding. + +The full leaf catalogue the binder supports: `String`, `bool` (it accepts +`1`/`0`, `t`/`f`, and `true`/`false` forms), every integer width, `f32`/`f64`, +`char`, unit enums (matched by variant name), `Option` (`None` when the key +and its whole subtree are absent), sequences of scalars (comma-separated, +trimmed), and `HashMap` subtrees (every immediate child segment +becomes a map key). For a duration, bind an `i64`/`u64` of milliseconds and +convert: `Duration::from_millis(cfg.cache.ttl_ms)`. + +## Step 3 — Bind values onto the struct + +A struct alone does nothing; you bind a flat map onto it. The lowest-level +entry point is `bind`, which takes a `HashMap` and decodes it +onto a fresh `T`. + +```rust,ignore +use std::collections::HashMap; +use firefly::config::{bind, ConfigError}; + +let flat = HashMap::from([ + ("name".to_string(), "lumen".to_string()), + ("web.addr".to_string(), "127.0.0.1:8080".to_string()), + ("web.admin-addr".to_string(), "127.0.0.1:8081".to_string()), + ("tags".to_string(), "wallet, ledger, demo".to_string()), +]); + +let cfg: LumenConfig = bind(&flat)?; +assert_eq!(cfg.web.addr, "127.0.0.1:8080"); +assert_eq!(cfg.tags, vec!["wallet", "ledger", "demo"]); +# Ok::<(), ConfigError>(()) +``` + +What just happened: `bind` walked your struct's type, looked up each dotted key, +and parsed the string into the target field. Note three things the type drove on +its own — `web.admin-addr` (kebab) bound the `admin_addr` (snake) field, +`"wallet, ledger, demo"` split-and-trimmed onto a `Vec`, and nothing +required `#[serde(default)]`. + +> **Note** **Key term — facade import.** `firefly::config` is the +> `firefly-config` crate re-exported through the one-dependency facade, so you +> still depend only on `firefly`. Throughout this chapter `firefly::config::X` +> and `firefly_config::X` name the same item; the book prefers the facade path +> to keep the single-dependency story honest. + +In real code you almost never build that map by hand — sources build it for you. +The canonical loader, `load`, takes a list of sources, merges them, resolves +placeholders, and binds in one call: + +```rust,ignore +use firefly::config::{load, Source}; + +let cfg: LumenConfig = load(&sources)?; +``` + +The next step is where `sources` comes from. -## Loading with profiles +> **Tip** **Checkpoint.** Drop the `bind` example into a unit test and run it. +> A green test means your struct's shape and the dotted keys line up — this is +> the fastest way to debug a binding before YAML and the environment are in the +> mix. -The canonical helper reads `application.yaml`, then the profile-specific -`application-{profile}.yaml`, then `LUMEN_*` environment variables: +## Step 4 — Load with profiles + +The most common bootstrap is one helper call. `load_from_profile` reads +`application.yaml`, then the profile-specific `application-{profile}.yaml`, then +`FIREFLY_*` environment variables, merges them in that order, and binds the +result: ```rust,ignore -use firefly_config::{load_from_profile, ConfigError}; +use firefly::config::{load_from_profile, ConfigError}; fn main() -> Result<(), ConfigError> { - // dir, app basename, fallback profile (FIREFLY_PROFILE overrides). + // dir, app basename, fallback profile (FIREFLY_PROFILE overrides at runtime). let cfg: LumenConfig = load_from_profile("/etc/lumen", "application", "dev")?; println!("public API on {}", cfg.web.addr); Ok(()) } ``` -`FIREFLY_PROFILE` selects the profile file at runtime — `FIREFLY_PROFILE=prod` -reads `application-prod.yaml`. A comma-separated value -(`FIREFLY_PROFILE=dev,cloud`) overlays one file per profile in order. This is -how Lumen would carry an in-memory event store in `dev` and a Postgres event -store in `prod` without a single `if` in the wiring code. +What just happened, argument by argument: + +- `"/etc/lumen"` is the directory the YAML files live in. +- `"application"` is the file *basename* — so it reads `application.yaml` and + `application-{profile}.yaml`. (Pass `"lumen"` to read `lumen.yaml` instead.) +- `"dev"` is the **fallback** profile, used only when `FIREFLY_PROFILE` is unset. + +Both YAML files are tolerated absent — a service that hard-codes everything in +Rust can ship no YAML at all and this call still succeeds against the +environment alone. + +> **Note** **Key term — `FIREFLY_PROFILE`.** This environment variable selects +> the active profile(s) at runtime. `FIREFLY_PROFILE=prod` reads +> `application-prod.yaml`; a comma-separated value (`FIREFLY_PROFILE=dev,cloud`) +> overlays one file per profile, in order (`application-dev.yaml` then +> `application-cloud.yaml`, later wins). This is how Lumen would carry an +> in-memory event store in `dev` and a Postgres one in `prod` without a single +> `if` in the wiring code. -## Source precedence +> **Warning** `load_from_profile` always appends `from_env("FIREFLY")` as its +> top layer, so its environment overrides are spelled `FIREFLY_*` +> (`FIREFLY_WEB_ADDR`), *not* `LUMEN_*`. If you want a `LUMEN_`-prefixed +> environment layer, build the chain yourself (Step 6) with `from_env("LUMEN")`. -`Layered::new(vec![s1, s2, ...])` merges from left to right — **last write -wins**. The canonical chain is: +## Step 5 — Understand source precedence -| Order | Source | Beats | -|-------|-------------------------------------------------|--------------| -| 1 | Defaults — `StaticSource::new(name, entries)` | nothing | -| 2 | Base YAML — `from_optional_yaml("application.yaml")` | defaults | +The whole system rests on one rule: **`Layered::new(vec![s1, s2, ...])` merges +its sources left to right, and the last write wins.** Higher rows in the table +below sit later in the list and therefore override lower ones. + +| Order | Source | Beats | +|-------|-----------------------------------------------------|--------------| +| 1 | Defaults — `StaticSource::new(name, entries)` | nothing | +| 2 | Base YAML — `from_optional_yaml("application.yaml")` | defaults | | 3 | Profile YAML — `from_optional_yaml("application-prod.yaml")` | base | -| 4 | Environment — `from_env("LUMEN")` | YAML files | +| 4 | Environment — `from_env("FIREFLY")` | YAML files | | 5 | CLI flags — `FlagSource::new().set("web.addr", "0.0.0.0:80")` | everything | -So an environment override (`LUMEN_WEB_ADDR=0.0.0.0:80`) always beats a YAML -file, and a CLI override always beats both. That same precedence is why +So an environment override (`FIREFLY_WEB_ADDR=0.0.0.0:80`) always beats a YAML +file, and a CLI flag beats both. That same precedence is exactly why `FireflyApplication` lets `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` win -over any baked-in default bind address. Build the chain explicitly when you need -full control: +over any baked-in default bind address — the environment layer outranks the +default. + +Why it matters: precedence is what makes one artifact deployable everywhere. +You commit sensible defaults and a base `application.yaml`, ship a thin +`application-prod.yaml` overlay, and let the platform inject secrets and +last-mile overrides through the environment — each layer only states what it +needs to change. + +> **Tip** **Checkpoint.** You can reason about any value by reading the table +> top-down and taking the first source that defines it. If `web.addr` appears in +> both `application.yaml` and `FIREFLY_WEB_ADDR`, the environment wins because +> row 4 is later than row 2. + +## Step 6 — Build the source chain explicitly + +`load_from_profile` is the convenient default. When you need full control — a +different env prefix, hard-coded defaults, a remote source slotted in — assemble +the `Vec>` yourself and hand it to `load`: ```rust,ignore -use firefly_config::{from_env, from_optional_yaml, load, Source, StaticSource}; +use std::collections::HashMap; +use firefly::config::{from_env, from_optional_yaml, load, Source, StaticSource}; let sources: Vec> = vec![ - Box::new(StaticSource::new("defaults", [("web.addr".into(), "127.0.0.1:8080".into())])), + // 1. Defaults at the bottom — overridden by anything below. + Box::new(StaticSource::new( + "defaults", + HashMap::from([("web.addr".to_string(), "127.0.0.1:8080".to_string())]), + )), + // 2. Base YAML beats defaults. Box::new(from_optional_yaml("application.yaml")), + // 3. A LUMEN_*-prefixed environment layer beats the YAML. Box::new(from_env("LUMEN")), ]; let cfg: LumenConfig = load(&sources)?; ``` -## YAML subset and value rules +What just happened: you spelled out the precedence chain in list order. +`StaticSource::new` takes a name and a `HashMap` of hard-coded entries — it sits +at the bottom. `from_optional_yaml` reads a file if present (and is silently +empty if not). `from_env("LUMEN")` maps `LUMEN_WEB_ADDR` → `web.addr`. Because +the env source is *last*, `LUMEN_WEB_ADDR=0.0.0.0:80` overrides both the YAML +and the default. -Files are parsed by a line-by-line YAML-subset scanner (no general-purpose YAML -dependency), so the flattened output is deterministic and stable for any given -`application.yaml`: +> **Note** **Key term — `StaticSource` / `from_env` / `from_optional_yaml` / +> `FlagSource`.** These are the four built-in sources. `StaticSource` wraps an +> in-memory map (defaults). `from_env(prefix)` reads `PREFIX_FOO_BAR` → +> `foo.bar` from the process environment. `from_optional_yaml(path)` reads a +> YAML file, tolerating absence. `FlagSource` collects CLI overrides set with +> `.set("web.addr", "...")`. All four implement the same `Source` trait, so the +> order in the `vec!` *is* the precedence. + +## Step 7 — Write the YAML and know the value rules + +YAML files are parsed by a small line-by-line YAML-subset scanner — not a +general-purpose YAML engine — so the flattened output is deterministic and +stable for any given file: ```yaml +# application.yaml name: lumen web: addr: 127.0.0.1:8080 admin-addr: 127.0.0.1:8081 -tags: wallet, ledger, demo # sequences of scalars are comma-joined +tags: wallet, ledger, demo # a comma-joined scalar binds a Vec ``` -- nested mappings become dot-joined, lower-cased keys; -- scalar lexemes are preserved verbatim until the binder parses them against the - target field type; -- duplicate keys follow last-write-wins; -- aliases, anchors, multi-doc, tags, and flow sequences are deliberately not - interpreted. +The rules the scanner guarantees: -Supported leaf kinds: `String`, `bool` (accepts `1`/`0`, `t`/`f`, and -`true`/`false` in any case), every integer -width, `f32`/`f64`, `char`, unit enums (by variant name), `Option`, sequences -of scalars, and `HashMap` subtrees. For durations, use an `i64` field -plus a conversion: `Duration::from_millis(cfg.cache.ttl as u64)`. +- nested mappings become **dot-joined, lower-cased** keys (`web.admin-addr` → + the flat key `web.admin_addr` after relaxed normalization); +- scalar lexemes are **preserved verbatim** (`1.10` stays `"1.10"`) until the + binder parses them against the target field's type; +- duplicate keys follow **last-write-wins**; +- aliases, anchors, multi-document files, tags, and flow sequences are + **deliberately not interpreted** — bring your own parser if you need them. -> **Note** — Keys are normalized kebab ↔ snake, so `admin-addr:` in YAML binds -> an `admin_addr` serde field. +What just happened: this base file states Lumen's identity and its two bind +addresses in the typed home those addresses always wanted. The `tags` line shows +the one subtlety — a sequence is written as a comma-joined scalar, and the +binder splits it back into the `Vec` field. -## Placeholders +> **Tip** **Checkpoint.** Put this file next to a test that calls +> `load_from_profile(".", "application", "dev")` and asserts +> `cfg.web.admin_addr == "127.0.0.1:8081"`. A pass proves the kebab→snake +> normalization and the comma-split list both work end to end. -`load` / `bind` run a post-merge pass resolving `${...}` placeholders in values -(also exposed standalone as `resolve_placeholders(&flat)`): +## Step 8 — Resolve `${...}` placeholders + +`load` (and `bind`) run a post-merge pass that resolves `${...}` placeholders +inside values — the same `${...}` syntax Spring uses. It is also exposed +standalone as `resolve_placeholders(&flat)`. ```yaml name: lumen datasource: - url: ${DATABASE_URL:postgres://localhost/lumen} # env, else default + url: ${DATABASE_URL:postgres://localhost/lumen} # env var, else default pool: ${name}-pool # config reference ``` -- `${ENV_VAR}` — a literal environment variable; -- `${name}` — a config reference, resolved recursively with a depth-10 guard - against cycles; -- `${key:default}` — a fallback when neither environment nor config resolves - `key`; -- **environment beats config**: `${name}` honors `FIREFLY_NAME` before the - merged map. +The resolution order, highest priority first: + +- `${ENV_VAR}` — a literal environment variable, read as written; +- the **relaxed `FIREFLY_*` form** of a config key — `${name}` also honors + `FIREFLY_NAME` before consulting the merged map, so **environment beats + config**; +- `${name}` — a config reference into the merged map itself, resolved + recursively with a depth-10 guard against cycles; +- `${key:default}` — the text after the first `:` is a fallback when neither the + environment nor the config resolves `key`. -An unresolvable placeholder without a default raises `ConfigError::Placeholder`. +What just happened: `datasource.url` reads `DATABASE_URL` from the environment +when present and otherwise falls back to the local default — one line that is +correct in both dev and prod. `datasource.pool` interpolates another config +value (`name` → `lumen`) to produce `lumen-pool`. -## Binding config straight into a bean — `#[derive(ConfigProperties)]` +> **Warning** An unresolvable placeholder *without* a default raises +> `ConfigError::Placeholder`, and so does a circular reference (`a: ${b}` / +> `b: ${a}`) once it trips the depth-10 guard. A typo'd `${DATBASE_URL}` with no +> `:default` fails the load loudly rather than binding an empty string. -Loading a struct by hand is fine for `main`. But Lumen's services want their -configuration *injected*, not threaded through every constructor. The -`#[derive(ConfigProperties)]` macro turns a `serde` struct into a -container-managed, prefix-bound bean — the exact pattern the dependency-injection -chapter builds on: +## Step 9 — Bind config straight into a bean with `#[derive(ConfigProperties)]` + +Loading a struct by hand in `main` is fine, but Lumen's services want their +configuration *injected*, not threaded through every constructor. +`#[derive(ConfigProperties)]` turns a `serde` struct into a container-managed, +prefix-bound bean — the exact pattern the next chapter builds on. ```rust,ignore use firefly::prelude::*; @@ -204,17 +414,32 @@ pub struct WebProperties { } ``` -Any `#[derive(Service)]` bean can then `#[autowired] props: Arc` -and receive the bound values — no manual `load`, no global. You will wire one in -[Dependency Wiring](./04-dependency-wiring.md). For one-off scalars there is an -even lighter touch: a `#[firefly(value = "${lumen.web.addr:127.0.0.1:8080}")]` -field injects a single resolved value with a default. +What just happened: the derive registers `WebProperties` as a singleton whose +factory binds the `lumen.web.*` slice of the merged, profile-resolved, +placeholder-expanded config map. The container warms it eagerly at startup, so +any bean can then receive it by type. + +> **Note** **Key term — bean / autowiring.** A *bean* is an object the framework +> constructs and manages for you; *autowiring* is the framework handing a bean +> to whoever declares a field for it. A `#[derive(Service)]` bean writes +> `#[autowired] props: Arc` and receives the bound values — no +> manual `load`, no global. You will wire one in +> [Dependency Wiring](./04-dependency-wiring.md). This is Spring's +> `@ConfigurationProperties` bean injected with `@Autowired`. + +For one-off scalars there is a lighter touch — inject a single resolved value +onto a field with a default: + +```rust,ignore +#[firefly(value = "${lumen.web.addr:127.0.0.1:8080}")] +addr: String, +``` To *validate* a properties bean after binding — Spring's `@Validated` on a -`@ConfigurationProperties` — add `#[firefly(validate)]` and `#[derive(Validate)]`. -The macro runs the struct's declarative `Validate` constraints once the config is -bound, and a violation **fails the bean's creation** (it fails context refresh at -startup) with the structured per-field errors, rather than letting a malformed +`@ConfigurationProperties` class — add `#[firefly(validate)]` and +`#[derive(Validate)]`. The macro runs the struct's declarative constraints once +the config is bound, and a violation **fails the bean's creation** at context +refresh with the structured per-field errors, rather than letting a malformed configuration boot: ```rust,ignore @@ -231,16 +456,20 @@ pub struct WebProperties { } ``` -> **Design note.** Firefly offers two binding styles. A prefix-bound bean +What just happened: an empty `lumen.web.addr` now aborts startup with a clear +per-field violation (`addr: must not be empty (not_empty)`) instead of binding +`""` and failing later when something tries to bind a socket. + +> **Design note.** Firefly offers two binding styles against the *same* merged, +> profile-resolved, placeholder-expanded map. A prefix-bound bean > (`#[derive(ConfigProperties)]` + `#[firefly(prefix = "...")]`) pulls a whole -> config subtree into one injectable struct, while single-value injection -> (`#[firefly(value = "${...}")]`) wires one resolved scalar onto a field. Both -> bind against the same merged, profile-resolved, placeholder-expanded map -> described above. +> config subtree into one injectable struct; single-value injection +> (`#[firefly(value = "${...}")]`) wires one resolved scalar onto a field. Use +> the first for a cohesive settings group, the second for a stray knob. -## Config-driven auto-configuration — datasource and security from `application.yaml` +## Step 10 — Auto-configure the datasource and security from `application.yaml` -The properties-binding machinery so far hands you a typed struct. Firefly's +The properties machinery so far hands you a typed struct. Firefly's infrastructure crates take the next step: a handful of subsystems are **config-driven and DI-free** — you bind a plain `serde` struct from `application.yaml`/env, then `await` a single auto-configure call at boot, and @@ -270,52 +499,68 @@ firefly: allow-anonymous: false ``` -### Datasource — `DataSourceProperties` → pool → transaction manager +### The datasource — `DataSourceProperties` → pool → transaction manager -`DataSourceProperties` is a plain `serde` struct — `{ url, max_connections, -min_connections, acquire_timeout_ms, idle_timeout_ms, max_lifetime_ms }`. The -**URL scheme selects the backend**, each behind its own cargo feature: -`postgres://` / `postgresql://` → PostgreSQL, `mysql://` → MySQL, `sqlite:` → -SQLite. A `0` for any pool setting leaves the `sqlx` default in place. +`DataSourceProperties` is a plain `serde` struct with the fields `{ url, +max_connections, min_connections, acquire_timeout_ms, idle_timeout_ms, +max_lifetime_ms }`. The **URL scheme selects the backend**, each behind its own +cargo feature: `postgres://` / `postgresql://` → PostgreSQL, `mysql://` → +MySQL, `sqlite:` → SQLite. A `0` for any pool setting leaves the `sqlx` default +in place. -`firefly_data_sqlx::auto_configure(&props)` does the one thing you want at boot: -it builds the connection pool **and** registers a `SqlxTransactionManager` over -it, so `#[transactional]` resolves with no manual wiring. The returned `Db` is -the same pool, ready to build typed repositories. (For finer control, -`Db::connect(url)` and `Db::connect_with(&props)` build just the pool.) +`firefly::data_sqlx::auto_configure(&props)` does the one thing you want at +boot: it builds the connection pool **and** registers a +`SqlxTransactionManager` over it, so `#[transactional]` resolves later with no +manual wiring. The returned `Db` is the same pool, ready to build typed +repositories. (For finer control, `Db::connect(url)` and +`Db::connect_with(&props)` build just the pool.) ```rust,ignore -use firefly_data_sqlx::{auto_configure, DataSourceProperties}; +use firefly::data_sqlx::{auto_configure, DataSourceProperties}; // `Db` carries the pool; auto_configure also registers the tx manager. let db = auto_configure(&props).await?; // Result ``` -### Security — `SecurityProperties` → verifier → bearer layer +> **Note** **Key term — transaction manager.** A *transaction manager* opens, +> commits, and rolls back database transactions on behalf of the +> `#[transactional]` attribute. By registering one, `auto_configure` makes +> `#[transactional]` work process-wide without you constructing or threading the +> manager anywhere — the Rust analog of Spring Boot auto-configuring a +> `DataSourceTransactionManager`. You will use it in [Persistence](./07-persistence.md). + +### The security layer — `SecurityProperties` → verifier → bearer layer `SecurityProperties` nests `{ jwt: JwtProperties, bearer: BearerProperties }`. `JwtProperties` holds `{ jwk_set_uri, issuer_uri, audience, secret, algorithm, expiration_seconds }`; `BearerProperties` holds `{ header_name, allow_anonymous }`. Two functions turn that into running middleware: -- `verifier_from_config(&JwtProperties)` returns +- `verifier_from_config(&props.jwt)` returns `Result>, SecurityError>`. A non-empty `jwk_set_uri` builds a JWKS (RS256) resource-server verifier; otherwise a non-empty `secret` builds an HMAC verifier (`HS256`/`HS384`/`HS512`); otherwise `None`. -- `bearer_layer_from_config(&SecurityProperties)` returns +- `bearer_layer_from_config(&props)` returns `Result, SecurityError>` — the ready-to-mount layer with the configured header name and anonymous policy already applied, or `None` when no verifier is configured. +> **Note** **Key term — verifier / bearer layer.** A *verifier* checks an +> incoming JWT's signature and claims; a *bearer layer* is the HTTP middleware +> that pulls the token off the request header and runs the verifier. Together +> they are the Rust analog of a Spring Security resource-server filter chain. +> The full security story is [Security](./14-security.md); here you are only +> learning that both can be *configured*, not hand-built. + ### The one-call startup wiring Bind one config struct, then drive both subsystems from it. The whole wiring is a load plus two awaited calls: ```rust,ignore -use firefly_config::{load_from_profile, ConfigError}; -use firefly_data_sqlx::{auto_configure, DataSourceProperties}; -use firefly_security::{bearer_layer_from_config, SecurityProperties}; +use firefly::config::{load_from_profile, ConfigError}; +use firefly::data_sqlx::{auto_configure, DataSourceProperties}; +use firefly::security::{bearer_layer_from_config, SecurityProperties}; use serde::Deserialize; #[derive(Debug, Default, Deserialize)] @@ -344,11 +589,12 @@ async fn main() -> anyhow::Result<()> { // `db` builds typed repositories; mount `bearer` on the web stack. // ... + let _ = (db, bearer); Ok(()) } ``` -This is the same precedence chain from earlier in the chapter doing real work: +What just happened: this is the precedence chain from Step 5 doing real work. `DATABASE_URL` in the environment overrides the YAML default for the pool, and the JWKS endpoint can be re-pointed per profile without touching code. The `#[transactional]` machinery and the bearer middleware both pick up what @@ -358,18 +604,23 @@ through your constructors. > **Design note.** Firefly deliberately keeps this path DI-free: the config > structs are ordinary `serde` types and the auto-configure calls are ordinary > `async fn`s you `await` at boot. You can adopt the full -> `#[derive(ConfigProperties)]` container later without rewriting any of it — -> the same bound values flow either way. +> `#[derive(ConfigProperties)]` container style later without rewriting any of +> it — the same bound values flow either way. -## Profile expressions +> **Tip** **Checkpoint.** Even without a real database, this `main` compiles: +> `auto_configure` against `sqlite::memory:` (set `firefly.datasource.url` to +> `sqlite::memory:`) returns a live `Db` you can hold, and an empty +> `firefly.security.*` makes `bearer_layer_from_config` return `Ok(None)`. -`accepts_profiles(&active, &exprs)` evaluates a profile-expression grammar — AND -(`&`), OR (`|`), negation (`!`), and grouping with parentheses — against an -active-profile list, useful for gating a bean that should exist only in some -environments: +## Step 11 — Gate beans by profile expression + +Sometimes a value is not enough — you want a whole bean to exist only in some +environments. `accepts_profiles(&active, &exprs)` evaluates a +profile-expression grammar against an active-profile list: AND (`&`), OR (`|`), +negation (`!`), and grouping with parentheses. ```rust,ignore -use firefly_config::{accepts_profiles, active_profiles}; +use firefly::config::{accepts_profiles, active_profiles}; let active = active_profiles("dev"); // e.g. ["prod", "cloud"] accepts_profiles(&active, &["prod & cloud"]); // AND @@ -378,72 +629,67 @@ accepts_profiles(&active, &["!test"]); // negation accepts_profiles(&active, &["(prod & cloud) | qa"]); // grouping ``` -It returns `true` when any expression matches; a malformed expression evaluates -to `false` (it never panics). The dependency-injection chapter shows how a bean -declares `#[firefly(profile = "prod")]` so the container applies exactly this -rule at scan time. +What just happened: `active_profiles("dev")` reads the comma-separated +`FIREFLY_PROFILE` (falling back to `"dev"`), and `accepts_profiles` answers +whether *any* of the given expressions matches that active set. It returns +`true` on a match; a malformed expression evaluates to `false` and never panics. + +Why it matters: the next chapter shows a bean declaring +`#[firefly(profile = "prod")]`, and the container applies exactly this rule at +scan time — so a Postgres-only bean simply does not exist in the `dev` profile. + +## Step 12 — Reload at runtime and mask secrets -## Runtime reload — the `/actuator/refresh` contract +Two operational concerns round out the picture. -`ReloadableConfig` replays the full merge → placeholder-resolution → bind +**Runtime reload.** `ReloadableConfig` keeps the source chain alive after the +first bind. `reload()` replays the full merge → placeholder-resolution → bind pipeline and atomically swaps the snapshot; a failed reload keeps the previous -snapshot. This is the hook a `POST /actuator/refresh` endpoint wires up — so an +one. This is the hook a `POST /actuator/refresh` endpoint wires up — so an operator could re-point Lumen's datasource without a restart. ```rust,ignore -use firefly_config::ReloadableConfig; +use firefly::config::{ReloadableConfig, Source}; let cfg: ReloadableConfig = ReloadableConfig::load(sources)?; -let snapshot = cfg.get(); // Arc -let mut rx = cfg.subscribe(); // tokio watch receiver -let changed: Vec = cfg.reload()?; // sorted, changed top-level keys +let snapshot = cfg.get(); // Arc — read per use +let mut rx = cfg.subscribe(); // tokio watch receiver +let changed: Vec = cfg.reload()?; // sorted, changed top-level keys ``` `Arc>` coerces to `Arc` — the object-safe trait the actuator refresh endpoint depends on. -## Property-source introspection and masking +> **Note** **Key term — refresh scope.** A *refresh-scoped* reader calls +> `cfg.get()` per use instead of caching the inner value, so it always sees the +> latest snapshot after a reload. This is the Rust analog of Spring Cloud's +> `@RefreshScope` plus its `POST /actuator/refresh` contract. -`Layered::property_sources()` returns ordered, origin-attributed -`PropertySourceView`s (highest precedence first) — the data Firefly's -`/actuator/env` view renders, with secrets masked: keys naming secrets (`password`, `secret`, `token`, -`credential`, `*key`) mask as `******`, and URI userinfo passwords are redacted +**Masking secrets.** `Layered::property_sources()` returns ordered, +origin-attributed `PropertySourceView`s (highest precedence first) — the data +Firefly's `/actuator/env` view renders, with secrets masked. Keys naming secrets +(`password`, `secret`, `token`, `credential`, `*key`, …) mask as `******`, and a +password embedded in a URI's userinfo is redacted (`postgresql://user:******@host`). The `mask` module exposes `mask_value`, -`is_sensitive_key`, and `sanitize_uri` directly. This matters for Lumen the -moment it has a JWT signing key (chapter 14) and a datasource URL — neither -should ever appear in plaintext on `/actuator/env`. - -## In-process application events - -`ApplicationEventBus` is an **in-process, `TypeId`-dispatched, order-sorted, -synchronous** pub/sub for lifecycle and local notification events. This is -distinct from the asynchronous `firefly-eda` broker Lumen uses for domain events -(no transport, no topics; listeners run on the publishing thread): - -```rust,ignore -use firefly_config::{ApplicationEventBus, ApplicationReadyEvent}; - -let bus = ApplicationEventBus::new(); -bus.subscribe::(|_e| { /* on ready */ }); -bus.publish(&ApplicationReadyEvent); -``` +`is_sensitive_key`, and `sanitize_uri` directly. -Lifecycle events ship: `ContextRefreshedEvent`, `ApplicationReadyEvent`, -`ContextClosedEvent`. Any `'static` type can be published as a domain event. +Why it matters for Lumen: the moment it holds a JWT signing key (chapter 14) and +a datasource URL, neither should ever appear in plaintext on `/actuator/env` — +and with masking on by default, neither does. -> **Note** — Do not confuse this with [Event-Driven Architecture](./10-eda-messaging.md): -> the `ApplicationEventBus` is a *local* lifecycle/notification channel; Lumen's -> wallet domain events ride the `firefly-eda` `Broker` over a topic, with a real -> Kafka/RabbitMQ adapter waiting behind the in-memory default. +> **Tip** **Checkpoint.** Add a `datasource.password` key to a `StaticSource`, +> call `Layered::new(sources).property_sources()`, and confirm the rendered +> value is `******`, not the secret. -## Pulling config from a config server +## Step 13 — Pull configuration from a config server (optional) -`ConfigClient` fetches a remote configuration document (compatible with the -Spring Cloud Config server wire format) and flattens it into a `StaticSource` -you slot into your chain above the defaults: +For a fleet of services, you may centralize configuration. `ConfigClient` +fetches a remote document (compatible with the Spring Cloud Config server wire +format) and flattens it into a `StaticSource` you slot into the chain above the +defaults: ```rust,ignore -use firefly_config::ConfigClient; +use firefly::config::ConfigClient; let remote = ConfigClient::new("http://config:8888", "lumen") .with_profile("prod") @@ -454,9 +700,38 @@ let remote = ConfigClient::new("http://config:8888", "lumen") sources.insert(1, Box::new(remote)); // above defaults, below env/flags ``` -A non-2xx response logs a warning and yields an empty map (soft miss); transport -or decode failures raise `ConfigError::Remote`. The standalone config server -lives in [`firefly-config-server`](./91-appendix-modules.md). +What just happened: `ConfigClient::new(url, app)` builds a client (profile +defaults to `default`, label to `main`); the builder methods set the rest; +`fetch_source().await` queries `{url}/{app}/{profile}/{label}` and returns a +`StaticSource`. A non-2xx response logs a warning and yields an empty map (a +soft miss); transport or decode failures raise `ConfigError::Remote`. The +standalone server lives in [`firefly-config-server`](./91-appendix-modules.md). + +## In-process application events + +One more piece of the config crate is worth naming, because you will meet it at +lifecycle boundaries. `ApplicationEventBus` is an **in-process, +`TypeId`-dispatched, order-sorted, synchronous** pub/sub for lifecycle and local +notification events — distinct from the asynchronous `firefly-eda` broker Lumen +uses for domain events (no transport, no topics; listeners run on the publishing +thread): + +```rust,ignore +use firefly::config::{ApplicationEventBus, ApplicationReadyEvent}; + +let bus = ApplicationEventBus::new(); +bus.subscribe::(|_e| { /* on ready */ }); +bus.publish(&ApplicationReadyEvent); +``` + +Lifecycle events ship: `ContextRefreshedEvent`, `ApplicationReadyEvent`, +`ContextClosedEvent`, and `RefreshScopeRefreshedEvent` (fired after a successful +reload). Any `'static` type can be published as a local domain event. + +> **Note** Do not confuse this with [Event-Driven Architecture](./10-eda-messaging.md): +> the `ApplicationEventBus` is a *local* lifecycle/notification channel; Lumen's +> wallet domain events ride the `firefly-eda` `Broker` over a topic, with a real +> Kafka/RabbitMQ adapter waiting behind the in-memory default. ## Recap — what changed in Lumen @@ -465,24 +740,52 @@ lives in [`firefly-config-server`](./91-appendix-modules.md). | identity hard-coded in two `pub const` strings | the same values understood as `CoreConfig` knobs that feed the banner and `/actuator/info` | | bind addresses read by `FireflyApplication` from `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` | the typed home for those addresses, sitting at the top of a documented precedence chain | | no path to per-environment settings | profiles, placeholders, and `#[derive(ConfigProperties)]` ready for injection in the next chapter | +| datasource and security would be hand-built | both stand up from `application.yaml` with one awaited auto-configure call each | | secrets unconsidered | masking + `/actuator/env` redaction in place before Lumen ever holds a signing key | +You also now know: + +- That configuration is a **flat, dot-keyed string map** bound onto a typed + `serde` struct, with the *target type* driving every parse. +- The **precedence chain** — defaults → base YAML → profile YAML → environment → + CLI flags — and that the last source wins. +- That `load_from_profile` is the convenient default (with a `FIREFLY_*` env + layer), while an explicit `Vec>` + `load` gives full control. +- How `${...}` placeholders resolve (environment beats config, with `:default` + fallbacks and a cycle guard), how `#[derive(ConfigProperties)]` injects a + bound subtree, and how `auto_configure` / `bearer_layer_from_config` stand up + whole subsystems from YAML. + ## Exercises -1. **Promote the ports to YAML.** Write an `application.yaml` with - `web.addr` / `web.admin-addr`, load it with `load_from_profile`, and confirm - a `LUMEN_WEB_ADDR` environment variable still wins (precedence row 4 beats - row 2). +1. **Promote the ports to YAML.** Write an `application.yaml` with `web.addr` / + `web.admin-addr`, load it with `load_from_profile(".", "application", "dev")`, + and confirm a `FIREFLY_WEB_ADDR` environment variable still wins (precedence + row 4 beats row 2). Then rebuild the chain by hand with `from_env("LUMEN")` + and show `LUMEN_WEB_ADDR` winning instead. 2. **Add a profile.** Create `application-prod.yaml` that overrides `web.addr` to `0.0.0.0:80`, run with `FIREFLY_PROFILE=prod`, and verify the prod value - takes effect while `dev` keeps the localhost binding. -3. **Bind a `ConfigProperties` bean.** Define the `WebProperties` struct above, - set its keys via a `ConditionContext`, and resolve it from a `Container` - (you will recognize this pattern in the next chapter's DI tests). -4. **Mask a secret.** Add a `datasource.password` key and call - `Layered::property_sources()`; confirm the value renders as `******` rather - than in plaintext. - -Next, see how Lumen's composition root resolves its collaborators — explicitly -today, with the best-in-class DI container as the scan-driven alternative — in -[Dependency Wiring](./04-dependency-wiring.md). + takes effect while plain `dev` keeps the localhost binding. +3. **Resolve a placeholder.** Set `datasource.url: + ${DATABASE_URL:postgres://localhost/lumen}` in YAML, load once with + `DATABASE_URL` unset (assert the default) and once with it set (assert the + override). Then delete the `:default` and confirm the unset case now raises + `ConfigError::Placeholder`. +4. **Bind a `ConfigProperties` bean.** Define the `WebProperties` struct from + Step 9, set `lumen.web.addr` via a `ConditionContext::new().with_property(...)`, + and resolve `WebProperties` from a `Container` — you will recognize this + pattern in the next chapter's DI tests. +5. **Mask a secret.** Add a `datasource.password` key to a `StaticSource`, call + `Layered::new(sources).property_sources()`, and confirm the value renders as + `******` rather than in plaintext. + +## Where to go next + +- See how Lumen's composition root resolves its collaborators — and how the + best-in-class container scans and wires the beans (including the + `#[derive(ConfigProperties)]` ones you just met) — in + **[Dependency Wiring](./04-dependency-wiring.md)**. +- Turn the configured datasource into typed repositories in + **[Persistence](./07-persistence.md)**. +- Promote the in-process defaults to real Postgres and Kafka in + **[Production & Deployment](./20-production.md)**. diff --git a/docs/book/src/04-dependency-wiring.md b/docs/book/src/04-dependency-wiring.md index ee317d5..f72c3ea 100644 --- a/docs/book/src/04-dependency-wiring.md +++ b/docs/book/src/04-dependency-wiring.md @@ -1,56 +1,245 @@ # Dependency Wiring -> By the end of this chapter you will understand how Lumen is wired: it -> **declares beans** and lets the framework's dependency-injection container -> discover and connect them. There is no hand-written composition root — every -> collaborator is a bean (`#[derive(Configuration)]` + `#[bean]` factories, a -> `#[derive(Repository)]` read model, `#[derive(Controller)]` controllers), every -> dependency is `#[autowired]` by type, and `FireflyApplication` component-scans -> the whole graph at boot. This -> is the best-in-class DI container that powers the framework, told against -> Lumen's own collaborators. - -Every service has a moment where the pieces come together: the cache, the bus, -the event store, the ledger, the controller that depends on them. In a Firefly -service that moment is **not** a hand-written function — it is the framework's -component scan. You *declare* each collaborator as a bean with a stereotype -derive, mark its dependencies `#[autowired]`, and `FireflyApplication` discovers -and wires the whole object graph when it boots. This chapter walks that path -using Lumen's real beans, then surveys the full container surface -[the next chapter](./04a-dependency-injection.md) covers in depth. - -## How `FireflyApplication` wires the object graph - -When `FireflyApplication::new("lumen").run().await` boots (the one-line `main` from -the [Quickstart](./02-quickstart.md)), it does the wiring a composition root -used to do by hand: - -1. It builds the web stack and **auto-registers** the framework's infrastructure - beans — the CQRS `Bus`, the event `Broker`, the cache, the metric registry, - the scheduler — into the container so your beans can autowire them. -2. It **component-scans** the crate graph: every stereotype-derived type and - every `#[bean]` factory in Lumen is discovered, condition-checked, and - registered. -3. It **resolves** the controllers to mount them, which recursively constructs - their autowired collaborators in dependency order — exactly the graph a - hand-written root would build, but derived from the bean declarations instead - of spelled out. - -The whole graph lives in `src/web.rs` as declarations next to each type. Two -declarations carry it: a `#[derive(Configuration)]` holder whose `#[bean]` -methods declare the domain beans, and a `#[derive(Controller)]` whose -`#[autowired]` fields name what it needs. - -## Lumen's beans — `#[derive(Configuration)]` + `#[bean]` - -Not every collaborator is a type you can annotate directly: the event store, the -query cache, the JWT service, and the ledger are all built by a factory. Lumen -declares them on a `#[derive(Configuration)]` holder — the Spring -`@Configuration` + `@Bean` analog. Each `#[bean]` method is keyed by its return -type, and the container resolves each method's `Arc` arguments from the -container before calling it, so a factory can depend on other beans. (The read -model is the exception: it carries `#[derive(Repository)]` on its own struct, so -the scan registers it directly — no `#[bean]` factory, as you will see below.) +In the [Quickstart](./02-quickstart.md) you wrote a one-line `main` and watched +`FireflyApplication::new("lumen").run()` boot a whole service. One line in that +boot pipeline did something a service usually hand-rolls: it assembled the +**object graph** — it constructed the cache, the CQRS bus, the event store, the +ledger, and the controller that depends on all of them, in the right order, and +connected them. This chapter is about *how*. + +The short answer is that you never write that assembly. In a Firefly service you +**declare** each collaborator as a bean — a struct with a stereotype derive, or a +factory method — mark its dependencies `#[autowired]`, and the framework's +**component scan** discovers every declaration at boot and wires the graph for +you. There is no hand-written composition root, no `build_app`, no list of +`new(...)` calls threaded through a function. You say *what* each piece needs; +the container supplies it. + +We will learn this the way the rest of the book teaches — against Lumen's real +beans, the ones in `samples/lumen`. By the end you will be able to read Lumen's +wiring, add to it, and explain exactly what the framework did at boot. The very +next chapter, [Dependency Injection & Auto-Configuration](./04a-dependency-injection.md), +then surveys the full container surface in depth; this chapter gives you the +working mental model it builds on. + +By the end of this chapter you will: + +- Explain what a **bean**, a **stereotype**, and a **component scan** are, and how + they replace a hand-written composition root. +- Declare beans two ways — a stereotype derive on a struct you own, and a + `#[bean]` factory for things you don't — and know when to reach for each. +- Use `#[autowired]` to inject a single dependency, a whole collection, an + optional one, or a deferred `Provider`. +- Bind a trait to its implementation with `provides`, and disambiguate several + candidates with `primary` and `order`. +- Read Lumen's bean inventory in the startup report and trace how + `FireflyApplication` resolves the graph from the declarations. + +## Concepts you will meet + +Before the first declaration, here are the four ideas this chapter leans on. Each +is reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — bean.** A *bean* is an object the framework constructs and +> manages for you, then hands to whoever declares it as a dependency. You declare +> beans; the container discovers them at startup and connects them. This is +> exactly Spring's notion of a bean managed by the application context. + +> **Note** **Key term — dependency injection (DI).** *Dependency injection* means +> a component does not construct its own collaborators — it declares *what* it +> needs and the framework supplies them. The piece that does the supplying is the +> **DI container**. Firefly's container is the Rust analog of Spring's +> `ApplicationContext`. + +> **Note** **Key term — stereotype.** A *stereotype* is a derive you put on a +> struct to make it a managed bean and to record its architectural role — +> business logic, data access, HTTP layer, and so on. Firefly's five stereotypes +> (`Service`, `Component`, `Repository`, `Configuration`, `Controller`) mirror +> Spring's `@Service`, `@Component`, `@Repository`, `@Configuration`, and +> `@Controller`. + +> **Note** **Key term — component scan.** A *component scan* is the startup pass +> that finds every declared bean and registers it. Spring scans the classpath +> with reflection; Rust has no runtime reflection, so Firefly's scan is +> *link-time* — each stereotype derive emits a registration that the scan collects +> from the compiled binary. + +## Step 1 — See the wiring you no longer write + +Open `samples/lumen/src/web.rs` and read its module doc comment. It calls the +file "the **composition root**", and then immediately tells you there is no +hand-written one. + +> **Note** **Key term — composition root.** The *composition root* is the single +> place in a program where the object graph is assembled — where every component +> is constructed and connected. In many frameworks you write this function by +> hand. In Firefly the framework *is* the composition root: it scans your beans +> and wires them, so you never spell the graph out. + +Recall the one-line `main` from the Quickstart: + +```rust,ignore +// src/main.rs +#[tokio::main] +async fn main() -> Result<(), firefly::BoxError> { + firefly::FireflyApplication::new("lumen").run().await +} +``` + +That single call assembles Lumen's entire object graph. Inside `run()`, the +wiring happens in three moves: + +1. It builds the web stack and **auto-registers** the framework's own + infrastructure beans — the CQRS `Bus`, the event `Broker`, the cache, the + metric registry, the scheduler — into the container, so *your* beans can + autowire them. +2. It **component-scans** the crate graph: every stereotype-derived type and every + `#[bean]` factory in Lumen is discovered, condition-checked, and registered. +3. It **resolves** the controllers in order to mount them, which recursively + constructs each controller's autowired collaborators in dependency order — + exactly the graph a hand-written root would build, but derived from the + declarations instead of spelled out. + +What just happened: nothing in your code names the order in which the cache, bus, +store, ledger, and controller are built. You declared each one next to itself; +the container computed the order from the dependency types. That is the whole +trick, and the rest of the chapter is the mechanics behind it. + +> **Tip** **Checkpoint.** Open `samples/lumen/src/web.rs` and find the comment +> that says there is "**no hand-written composition root and no builder**." Every +> example below comes from this file (and its sibling `ledger.rs` and +> `commands.rs`). You are reading the real wiring, not a toy. + +## Step 2 — Declare a bean you own with a stereotype + +The simplest bean is a struct you can annotate directly. You make it visible to +the container by deriving a **stereotype**. Lumen's read model is exactly this +case — an in-memory map the projection writes and the `GetWallet` query reads: + +```rust,ignore +// src/ledger.rs — the CQRS query side, a scanned data-access bean. +use std::collections::HashMap; +use std::sync::Mutex; +use firefly::prelude::*; + +/// The in-memory read model — a `#[derive(Repository)]` bean (Spring's +/// `@Repository`): the projection upserts it, `GetWallet` reads it. +#[derive(Debug, Default, Repository)] +pub struct ReadModel { + rows: Mutex>, +} +``` + +What just happened, block by block: + +- `#[derive(Repository)]` is the stereotype. It declares `ReadModel` a managed + bean *and* records its role as the data-access layer. That single derive is all + the registration there is — no `register(...)` call, no entry in a list. +- `Default` lets the container construct the bean with zero arguments. A + stereotype-derived struct with no `#[autowired]` fields is built by its + `Default`, then registered as a **singleton** (one shared instance for the + process). +- The struct's own fields (`rows`) are ordinary state. Only fields you mark + `#[autowired]` — and `ReadModel` has none — are filled from the container. + +The five stereotypes differ only in the architectural role they communicate; all +five register the type as a managed bean: + +| Derive | Role | +|----------------------------|--------------------------------------------------------| +| `#[derive(Service)]` | Business-logic layer: use-case orchestration. | +| `#[derive(Component)]` | Generic managed bean with no specific role. | +| `#[derive(Repository)]` | Data-access layer: databases, external storage, ports. | +| `#[derive(Configuration)]` | A factory holder that can carry `#[bean]` methods. | +| `#[derive(Controller)]` | HTTP layer (`#[rest_controller]` builds on this). | + +> **Design note.** The role each stereotype records is not cosmetic. It is stored +> on the bean, so the admin dashboard's `/beans` view (and the startup report) can +> group beans by layer — `[repository] ReadModel`, `[service] WalletHandlers`, and +> so on — the same DI introspection Spring Boot Actuator exposes. + +> **Tip** **Checkpoint.** `ReadModel` becomes a bean from one derive and one +> `Default`. Keep that picture: *a stereotype derive is the registration.* + +## Step 3 — Inject dependencies with `#[autowired]` + +A read model has no collaborators, but most beans do. To declare what a bean +needs, mark a field `#[autowired]` and the container fills it in by type. This is +the Rust spelling of constructor injection: you declare *what*, the container +supplies it. Lumen's wallet controller is the textbook case (the real `WalletApi` +from `src/web.rs`): + +```rust,ignore +use std::sync::Arc; +use firefly::cqrs::QueryCache; +use firefly::prelude::*; + +/// The wallet HTTP surface — a `#[derive(Controller)]` DI bean. Its +/// collaborators are autowired from the container; `#[rest_controller]` +/// auto-mounts it (Chapter 6). +#[derive(Clone, Controller)] +pub struct WalletApi { + #[autowired] + pub bus: Arc, // the CQRS bus it dispatches through + #[autowired] + pub ledger: Arc, // the application service the saga + stream use + #[autowired] + pub query_cache: Arc, // invalidated after a mutation +} +``` + +> **Note** **Key term — `Arc`.** `Arc` is Rust's atomically reference-counted +> shared pointer. The container hands out shared singletons, so an injected +> dependency arrives as `Arc` — many beans can hold the same `Arc` and +> all see the one instance. Wherever you see `#[autowired] field: Arc`, read it +> as "give me the shared `T`." + +What just happened: when the container constructs `WalletApi`, it first resolves +the `Bus`, then the `Ledger` (recursively building *its* own dependencies — you +will see those in Step 5), then the `QueryCache`, and only then injects all three +and hands back the controller. You wrote no constructor; the field types *are* the +constructor signature. + +A dependency that does not exist surfaces as a clear **resolution error at +startup** — a named "no such bean" pointing at the missing type — not a panic +three frames deep at runtime. Fail-fast wiring is the whole point. + +`#[autowired]` injects more than a single `Arc`. The field's *shape* selects +the injection mode: + +- `#[autowired] widgets: Vec>` injects **every** registered `Widget`, + ordered by each bean's `order` — collection injection, the way you gather all + implementations of a port. +- `#[autowired] maybe: Option>` injects `Some` when a `Thing` is + registered and `None` when it is not — an optional dependency that does not abort + startup if absent. +- `#[autowired] tickets: Provider` injects a **deferred** handle: + `tickets.get()` resolves a fresh value on each call, the way you reach for a + transient inside a singleton. + +> **Note** **Key term — `Provider`.** A `Provider` is a lazy handle to a +> bean rather than the bean itself. Calling `tickets.get()` resolves it on demand. +> It is the Rust analog of Spring's `ObjectProvider` / `Provider`, and the way +> a long-lived singleton pulls a short-lived bean each time it needs one. + +> **Tip** **Checkpoint.** `WalletApi` names three collaborators and constructs +> none of them. If you removed the `#[autowired] ledger` line, the controller +> would no longer ask for a ledger — the field is the entire request. + +## Step 4 — Declare beans you do not own with `#[bean]` factories + +Not every collaborator is a struct you can put a derive on. The event store, the +query cache, the JWT service, and the ledger are all *built* — they take +constructor arguments, or they come from a third-party crate, or a factory is +simply the clearest way to express them. For these, you declare a +`#[derive(Configuration)]` holder and give it `#[bean]` factory methods. + +> **Note** **Key term — `#[bean]` factory.** A `#[bean]` method is a factory: the +> container calls it and registers whatever it returns as a bean, keyed by the +> method's **return type**. The holder carries `#[derive(Configuration)]`. This is +> Spring's `@Configuration` class with `@Bean` methods, one-for-one. + +Here is Lumen's whole `LumenBeans` holder from `src/web.rs`: ```rust,ignore // src/web.rs @@ -109,189 +298,176 @@ impl LumenBeans { } ``` -The read model is **not** declared here. `ReadModel` is a `#[derive(Repository)]` -data-access bean (Spring's `@Repository`) annotated right on the struct, so -`container.scan()` registers it as a singleton directly — autowired as -`Arc` into the handler and projection beans, and shown in the startup -report as `[repository] ReadModel`: - -```rust,ignore -// src/ledger.rs — the CQRS query side, a scanned data-access bean. -use std::collections::HashMap; -use std::sync::Mutex; -use firefly::prelude::*; - -/// The in-memory read model — a `#[derive(Repository)]` bean (Spring's -/// `@Repository`): the projection upserts it, `GetWallet` reads it. -#[derive(Debug, Default, Repository)] -pub struct ReadModel { - rows: Mutex>, -} -``` - -Three ideas carry the whole design: +What just happened, block by block: + +- `#[derive(Configuration)] struct LumenBeans;` is the holder. The scan discovers + it the same way it discovers `ReadModel` — one derive. +- `#[bean] impl LumenBeans { ... }` marks the whole impl block as carrying bean + factories, and each inner `#[bean]` method is registered as its own bean. The + container keys each one by its return type: `event_store` registers a + `MemoryEventStore`, `query_cache` registers a `QueryCache`, and so on. +- `event_store`, `query_cache`, and `jwt_service` take only `&self` — they are + zero-dependency factories. The container calls them and registers the result. +- `ledger(&self, store: Arc, broker: Arc)` is the + important one: its **parameters are themselves resolved from the container** by + type before the method runs. A bean can depend on a bean. The container builds + the `MemoryEventStore` (from the `event_store` factory above) and supplies the + framework's `Broker`, then calls `ledger`. + +> **Note** **Key term — port and adapter.** A *port* is an interface (in Rust, a +> trait object like `Arc` or `Arc`); an *adapter* is a +> concrete implementation of it. "Depend on the port, inject the adapter" means a +> bean asks for the trait and the container supplies whatever implementation is +> registered. This is the hexagonal-architecture vocabulary the rest of the book +> uses. + +Notice the `ledger` factory widens `Arc` to `Arc` before handing it to `Ledger::new`. The `Ledger` stores the *port*, +not the concrete store — so swapping `MemoryEventStore` for a Postgres-backed +store is a one-line change in this factory, and the ledger, the handlers, and the +controller never notice. + +Three ideas carry the whole design. Read them slowly; everything later is a +consequence: - **The framework does the registration.** You never call `register_arc` or - `Container::bind`: `container.scan()` discovers the `LumenBeans` holder *and* - each of its `#[bean]` methods (every method submits its own link-time scan - thunk) and registers the produced values keyed by their return type. + `Container::bind`. `container.scan()` discovers the `LumenBeans` holder *and* + each `#[bean]` method and registers the produced values keyed by return type. - **Ports are resolved by type.** The `ledger` factory takes `broker: Arc` — "depend on the interface, inject the implementation" — and the - container supplies the framework's broker. Swap `MemoryEventStore` for a - Postgres-backed store by changing *only this holder*; the ledger, the - handlers, and the controller never notice. + Broker>` and the container supplies the framework's broker — "depend on the + interface, inject the implementation." - **A bean can depend on a bean.** `ledger(&self, store, broker)` pulls in two - other beans by type. The container builds them first, then calls the factory — + other beans by type; the container builds them first, then calls the factory — the same dependency-ordered construction a hand-written root would do, derived from the parameter types. -> **Tip** — Declarative wiring keeps each collaborator's dependencies *next to -> the collaborator*, and the container resolves them by type. A missing -> dependency is a clear resolution error at startup — not a panic three frames -> deep at runtime. The startup report logs every bean it registered, so "what is -> wired" is printed line-by-line at boot. - > **Design note.** A `#[derive(Configuration)]` holder with `#[bean]` methods is > the Spring `@Configuration` + `@Bean` analog: a factory whose methods produce > beans keyed by return type, resolving their own arguments from the container. -> Lumen declares its whole domain graph this way, and the framework's component -> scan turns the declarations into the wired object graph. - -## The container as a composition root - -The container *is* Lumen's composition root: `FireflyApplication` scans it, and -the framework's own infrastructure beans — the bus, broker, cache, scheduler, -and metric/health registries — are pre-registered into it before the scan, so -any bean can autowire them. The infrastructure surface available by type: - -| Bean (resolve by type) | Type | -|---------------------------|----------------------------------------| -| `Bus` | `Arc` (validation pre-installed) | -| cache adapter | `Arc` (Memory by default) | -| broker | `Arc` (InMemory by default) | -| scheduler | `Arc` | -| metric registry | `Arc` | -| health composite | `Arc` | - -Tune the underlying `WebStack`/`Core` knobs through -`FireflyApplication::configure(|cfg: &mut CoreConfig| { … })` — but the -collaborators themselves you reach by autowiring them into a bean. - -## The DI container under the hood — `firefly-container` - -The container the scan drives is a full dependency-injection engine with -**component scanning**, stereotype derives, constructor-style `#[autowired]` -injection, qualifier/primary/order disambiguation, `Vec` and `Provider` -injection, `#[bean]` factories, lifecycle hooks, and conditional/profile gating. -It is `TypeId`-keyed, `Send + Sync`, and shareable as `Arc`. Beans -default to singleton lifetime; the container also supports transient, request, -and session scopes, which [Dependency -Injection](./04a-dependency-injection.md) covers in full. - -### Stereotypes — declaring your beans - -You make a type visible to the container by deriving a **stereotype**. All five -register the type as a managed bean; the difference is the architectural role -each name communicates (and that the web layer uses to find controllers): - -| Derive | Role | -|----------------------------|--------------------------------------------------------| -| `#[derive(Service)]` | Business-logic layer: use-case orchestration. | -| `#[derive(Component)]` | Generic managed bean with no specific role. | -| `#[derive(Repository)]` | Data-access layer: databases, external storage, ports. | -| `#[derive(Configuration)]` | A factory holder that can carry `#[bean]` methods. | -| `#[derive(Controller)]` | HTTP layer (`#[rest_controller]` builds on this). | - -> **Design note.** The five stereotype derives document an architectural role — -> business logic (`Service`), generic managed bean (`Component`), data access -> (`Repository`), factory configuration (`Configuration`), or HTTP layer -> (`Controller`). The role is recorded on each bean, so the admin dashboard's -> `/beans` view can group beans by layer. - -### `#[autowired]` — constructor injection without a constructor - -Mark a field `#[autowired]` and the container fills it in by type. This is the -Rust spelling of constructor injection — you declare *what* a bean needs; the -container supplies it. This is exactly how Lumen's wallet controller names its -collaborators (the real `WalletApi` from `src/web.rs`): +> Lumen declares its whole domain graph this way, and the component scan turns the +> declarations into the wired object graph. + +> **Tip** **Checkpoint.** You now have both ways to declare a bean: a stereotype +> derive on a struct you own (`ReadModel`), and a `#[bean]` factory for things you +> build (`event_store`, `ledger`). Lumen uses a derive when it can annotate the +> type and a factory when it cannot. + +## Step 5 — Trace one resolution end to end + +Put Steps 2–4 together by following a single resolution: how does +`FireflyApplication` construct `WalletApi`? + +1. The scan has already registered every bean: `LumenBeans` and its five + factories, the `ReadModel`, the `WalletHandlers` and `WalletProjection` service + beans, and `WalletApi` itself — plus the framework's own `Bus`, `Broker`, + cache, scheduler, and registries. +2. To mount the controller, the container calls `resolve::()`. The + field types say it needs `Arc`, `Arc`, and `Arc`. +3. `Arc` and `Arc` already exist (the framework pre-registered + the bus; the `query_cache` factory produced the cache). They are handed over + directly. +4. `Arc` does not exist yet, so the container calls the `ledger` factory. + That factory needs `Arc` and `Arc`. The container + builds the store from the `event_store` factory, supplies the framework broker, + and calls `ledger` — producing the `Ledger`. +5. With all three collaborators in hand, the container constructs `WalletApi` and + caches it as a singleton. + +What just happened: the container built the graph **leaves-first** — store and +broker before ledger, ledger before controller — purely from the dependency +types. That ordering is the work a composition root used to do by hand. Here it is +*derived*, and it is recomputed correctly the moment you add or remove a +dependency. + +The same recursion wires the rest of Lumen. The CQRS handler bean autowires the +ledger and the read model the same way (the real `WalletHandlers` from +`src/commands.rs`): ```rust,ignore -use std::sync::Arc; -use firefly::cqrs::QueryCache; -use firefly::prelude::*; - -/// The wallet HTTP surface — a `#[derive(Controller)]` DI bean. Its -/// collaborators are autowired from the container; `#[rest_controller]` -/// auto-mounts it (Chapter 6). -#[derive(Clone, Controller)] -pub struct WalletApi { - #[autowired] - pub bus: Arc, // the CQRS bus it dispatches through +/// The CQRS handler bean — Spring's `@Component` command/query handler. Its +/// collaborators are `#[autowired]` from the DI container. +#[derive(Service)] +struct WalletHandlers { #[autowired] - pub ledger: Arc, // the application service the saga + stream use + ledger: Arc, #[autowired] - pub query_cache: Arc, // invalidated after a mutation + read_model: Arc, } -``` - -When the container constructs `WalletApi` it first resolves the `Bus`, the -`Ledger` (recursively building *its* store and broker from the `#[bean]` -factories above), and the `QueryCache`, then injects all three. A dependency -that does not exist surfaces as a clear resolution error at startup — not a panic -three frames deep at runtime. - -`#[autowired]` injects more than a single `Arc`: -- `#[autowired] widgets: Vec>` injects **every** registered `Widget`, - ordered by each bean's `order` (collection injection). -- `#[autowired] maybe: Option>` injects `Some` when a `Thing` is - registered and `None` when it is not — an optional dependency. -- `#[autowired] tickets: Provider` injects a **deferred** handle: - `tickets.get()` resolves a fresh instance on each call, the way you would - reach for a transient bean inside a singleton. - -### Component scanning — `firefly::scan` +#[handlers] +impl WalletHandlers { + #[command_handler] + async fn deposit(&self, cmd: Deposit) -> Result { + self.ledger + .deposit(&cmd.wallet_id, Money::cents(cmd.amount)) + .await + .map_err(to_cqrs) + } + // ... open_wallet, withdraw, get_wallet ... +} +``` -Rust has no runtime package introspection, so discovery is **link-time**: every -stereotype derive emits an `inventory` thunk, and `firefly::scan(&container)` -(equivalently `container.scan()`) collects every submitted thunk across the -whole crate graph and registers them — honoring conditionals and profiles as it -goes. **`FireflyApplication` runs this scan for you at boot**; the lower-level -entry point is the `ApplicationContext`, which wraps the container with the full -startup sequence and is handy in a focused test: +The same `Arc` and `Arc` singletons are injected here that the +controller and projection also hold — one instance each, shared by every bean that +asks for it. (The `#[handlers]` / `#[command_handler]` machinery that puts these +methods on the bus is the subject of [CQRS & Messaging](./09-cqrs.md); for now, +notice only that a handler is a bean and gets its collaborators by autowiring.) + +> **Tip** **Checkpoint.** Trace it in your head once more: `WalletApi` → +> `Ledger` → `MemoryEventStore` + `Broker`. If you can name that chain, you +> understand the resolver. + +## Step 6 — Use the framework's infrastructure beans by type + +You may have noticed `WalletApi` autowires `Arc` and the `ledger` factory +autowires `Arc`, yet Lumen never *declares* a bus or a broker. They +come from the framework. Before the scan runs, `FireflyApplication` pre-registers +its own infrastructure beans into the container, so any of your beans can autowire +them by type: + +| Bean (resolve by type) | Type | +|---------------------------|--------------------------------------------------| +| `Bus` | `Arc` (validation pre-installed) | +| cache adapter | `Arc` (Memory by default) | +| broker | `Arc` (InMemory by default) | +| scheduler | `Arc` | +| metric registry | `Arc` | +| health composite | `Arc` | + +What just happened: the container *is* Lumen's composition root, and the +framework seeds it with these collaborators first. That is why a `#[bean]` factory +can take `broker: Arc` and just receive one — the broker was already +registered. You reach any of these by autowiring it into a bean; you tune the +*configuration* knobs underneath them (CORS, idempotency, security headers, bind +addresses) through `FireflyApplication::configure`: ```rust,ignore -use firefly::prelude::*; +firefly::FireflyApplication::new("lumen") + .configure(|cfg: &mut CoreConfig| { + // adjust CoreConfig / WebStack knobs here + }) + .run() + .await +``` -let ctx = ApplicationContext::builder() - .profiles(["test"]) - .property("feature.audit", "on") - .build(); -let c = ctx.container(); +> **Note** **Key term — auto-configuration.** *Auto-configuration* is the +> framework pre-registering sensible infrastructure beans (an in-memory broker, an +> in-memory cache, the metric registry) so your app works with zero wiring, while +> still letting you override any of them. It is the mechanism behind Spring Boot's +> "it just works" defaults, covered in full in [Dependency Injection & +> Auto-Configuration](./04a-dependency-injection.md). -// Every stereotype-derived bean in the crate graph is discovered and wired. -let api = c.resolve::().expect("scan registered the controller"); -``` +> **Tip** **Checkpoint.** No bean in `samples/lumen` constructs a `Bus`, a +> `Broker`, or a cache — they autowire them. Grep `samples/lumen/src` for +> `Arc` and `Arc` and confirm every use is a consumer, never a +> producer. + +## Step 7 — Bind a trait to its implementation -> **Design note.** `firefly::scan` / `ApplicationContext::builder()` discover -> every stereotype-derived type in the linked crate graph and register it, -> subject to its conditions and the active profiles. Because Rust has no runtime -> reflection, discovery is link-time (via `inventory`): a bean is discoverable -> exactly when its crate's registrations are **linked into the binary**. For a -> crate the binary actually names (a single-crate app, or a library whose types -> it `use`s), that is automatic. But a *layer* crate the binary only depends on -> transitively — e.g. a `-models` or `-core` crate in a multi-crate service whose -> beans are never named directly — can be **dead-stripped** by the linker, beans -> and all. Force-link those with [`firefly::link!`](./22-layered-microservices.md) -> at the binary's crate root (`firefly::link!(my_core, my_models);`) and guard the -> result with `firefly::assert_discovered(...)`. The other Rust-specific note: -> generic types can't be inventoried (the monomorphization is chosen at the use -> site), so you register those with the explicit `register_all!` fallback below. - -### Interface auto-binding, `primary`, `order`, and qualifiers - -Bind a trait object to an implementation right on the derive, and resolve the -trait afterward — "depend on the port, get the adapter": +So far every autowired dependency has been a concrete type or a framework port. +When *you* own both a trait (a port) and its implementation (an adapter), you bind +them on the derive with `provides`, then resolve the trait — "depend on the port, +get the adapter": ```rust,ignore trait Clock: Send + Sync { fn now(&self) -> u64; } @@ -304,51 +480,95 @@ impl Clock for SystemClock { fn now(&self) -> u64 { 42 } } // elsewhere: c.resolve::() yields the SystemClock instance. ``` -When several beans satisfy the same interface, `#[firefly(... primary)]` picks -the default for `resolve`, and `resolve_all::()` returns *all* of -them ordered by `order`. For the rare case -where you need a *specific* named instance rather than any satisfying one, the -container supports qualifier-by-name resolution. - -Declaring `provides =` on the derive is the scan-friendly way to bind a trait to -an implementation. When you are wiring a container by hand instead, the -equivalent move is an explicit `Container::bind::()` call; -both register the same trait-to-adapter mapping. [Dependency -Injection](./04a-dependency-injection.md) covers `bind`, named beans, and the -full disambiguation surface in depth. - -### `#[bean]` factories — wiring things you do not own - -Not every dependency is a type you can annotate. Third-party clients need -constructor arguments; some beans are clearest as a factory. This is the -`LumenBeans` holder you already saw — a `#[derive(Configuration)]` with `#[bean]` -methods that produce beans keyed by their **return type** — the way Lumen wires -its event store, query cache, JWT service, and ledger. (The read model is the -counter-example: a type you *can* annotate directly, so it carries -`#[derive(Repository)]` instead of being produced by a factory.) A single factory -can swap an implementation in one place: +What just happened, block by block: + +- `#[derive(Component, Default)]` registers `SystemClock` as a managed bean, as + usual. +- `#[firefly(provides = "dyn Clock")]` *additionally* binds the `dyn Clock` trait + object to this implementation. Now a bean can autowire `Arc` and the + container hands it the `SystemClock`. +- `primary` marks this the default when several beans satisfy the same trait. + +> **Note** **Key term — `primary` and `order`.** When several beans satisfy one +> trait, `#[firefly(... primary)]` picks the one that plain `resolve:: Trait>()` returns (Spring's `@Primary`), and `#[firefly(order = N)]` sets the +> position a bean takes when *all* of them are collected — by `resolve_all:: Trait>()` or by a `Vec>` autowired field (Spring's `@Order`). + +`provides` on the derive is the **scan-friendly** way to bind a trait. When you +are assembling a container by hand instead (in a focused test, say), the equivalent +move is an explicit `Container::bind::()` call; both register +the same trait-to-adapter mapping. For the rare case where you need a *specific* +named instance rather than any satisfying one, the container also supports +qualifier-by-name resolution. All three — `bind`, named beans, and the full +disambiguation surface — are covered in +[Dependency Injection & Auto-Configuration](./04a-dependency-injection.md). + +Lumen itself uses `provides` for its feature-gated streaming endpoint, which it +registers as a `RouteContributor` port the framework discovers and merges: ```rust,ignore -#[bean] -impl LumenBeans { - // Swap MemoryEventStore for a Postgres store here and nothing else in - // Lumen changes — the `ledger` factory depends on the EventStore *port*. - #[bean] - fn event_store(&self) -> MemoryEventStore { - MemoryEventStore::new() - } +#[cfg(feature = "streaming")] +#[derive(Service)] +#[firefly(provides = "dyn firefly::web::RouteContributor")] +struct StreamingRoutes { + #[autowired] + api: Arc, } ``` -`#[bean]` methods may declare parameters; the container resolves each by type -before the method runs (Lumen's `ledger` factory does exactly that). A -`#[bean(profile = "prod")]` method registers only when the `prod` profile is -active — the factory-level twin of the conditional gating below. +> **Tip** **Checkpoint.** You can now bind a port to an adapter without a +> composition root: `provides` on the derive, then `resolve::()`. Add a +> second implementation without `primary` and resolving the trait becomes a +> *non-unique-bean* error — the container refusing to guess. + +## Step 8 — Gate beans by condition and profile + +One codebase has to run with cheap in-memory adapters in development and real +infrastructure in production. The mechanism is **conditional registration**: a +bean can declare the circumstances under which it should exist at all, and the +scan honors that as it collects each registration. + +```rust,ignore +// Registered only when the property is present and not false. +#[derive(Service, Default)] +#[firefly(condition_on_property = "feature.audit=on")] +struct AuditService; + +// Registered only under the named profile. +#[derive(Service, Default)] +#[firefly(profile = "prod")] +struct PostgresHealthCheck; +``` + +What just happened: `condition_on_property = "feature.audit=on"` registers +`AuditService` only when that config property is set; `profile = "prod"` registers +`PostgresHealthCheck` only when the `prod` profile is active. The scan evaluates +these as it discovers each bean, so the container ends up holding *exactly* the +beans the environment calls for — no `if` in your service code. + +> **Note** **Key term — profile.** A *profile* is a named environment slice — +> `dev`, `test`, `prod` — that toggles which beans and which configuration are +> active. Firefly reads the active profiles from configuration (the +> `FIREFLY_PROFILE` environment variable by default); profiles are introduced in +> [Configuration](./03-configuration.md) and used here to gate beans, exactly as +> Spring's `@Profile` does. -### Lifecycle hooks — `#[post_construct]` / `#[pre_destroy]` +The same gating works on `#[bean]` factories — `#[bean(profile = "prod")]` +registers a factory only under the `prod` profile — and it is the engine behind +every "swap the adapter for production" callout in this book. Lumen can stay +in-memory for teaching while a production deployment flips to real infrastructure +through configuration alone. -Real infrastructure beans need to *act* once wired (open a pool, subscribe to a -topic) and undo it on shutdown. Name the methods on the derive: +> **Tip** **Checkpoint.** Add `#[firefly(condition_on_property = +> "wallet.enabled=true")]` to a throwaway `#[derive(Service)]` bean, run Lumen, and +> watch it *not* appear in the startup report until you set the property. The +> condition decided the bean's existence before construction. + +## Step 9 — Hook into a bean's lifecycle + +Real infrastructure beans need to *act* once wired — open a pool, subscribe to a +topic — and undo it on shutdown. Name the methods on the derive: ```rust,ignore #[derive(Service, Default)] @@ -361,45 +581,68 @@ impl ProjectionSubscriber { } ``` -`post_construct` runs after construction and injection complete; `pre_destroy` -runs on `container.destroy()` in **reverse** construction order, so a subscriber -started after the store is torn down before it — the container-managed form of -the kind of one-time wiring Lumen's `ledger` `#[bean]` does when it seeds the -event-sourcing projection on construction. +What just happened: `post_construct = "started"` names a method to run *after* +the bean is built and its `#[autowired]` fields are injected; `pre_destroy = +"stopped"` names a method to run on `container.destroy()`. Destruction happens in +**reverse construction order**, so a subscriber started after the store is torn +down before it — clean teardown without a hand-written shutdown sequence. -> **Design note.** `#[firefly(post_construct = "...", pre_destroy = "...")]` name -> a method to run after a bean's dependencies are injected and a method to run on -> shutdown, with a "destroy in reverse construction order" guarantee. +> **Note** **Key term — `post_construct` / `pre_destroy`.** These are the Rust +> analogs of Spring's `@PostConstruct` and `@PreDestroy` (and JSR-250's lifecycle +> callbacks): a method to run once after wiring completes, and a method to run on +> shutdown, with the reverse-order guarantee. -### Conditional and profile gating — the same codebase in every environment +> **Tip** **Checkpoint.** Lifecycle hooks are how a bean does its one-time wiring +> *itself*, instead of a composition root doing it after construction. The +> container owns the ordering. -Conditions answer "should this bean exist at all, given the environment?" — how -one codebase runs with cheap in-memory adapters in dev and real infrastructure -in prod, without an `if` in your service code: +## Step 10 — Read the bean inventory at boot -```rust,ignore -// Registered only when the property is present and not false. -#[derive(Service, Default)] -#[firefly(condition_on_property = "feature.audit=on")] -struct AuditService; +You have declared beans, autowired them, bound a port, gated some, and given one +lifecycle hooks. The framework prints exactly what it wired. Run Lumen and read +the startup report: -// Registered only under the named profile. -#[derive(Service, Default)] -#[firefly(profile = "prod")] -struct PostgresHealthCheck; +```bash +cargo run ``` -`firefly::scan` evaluates these as it collects each thunk, so the resolved -container holds exactly the beans the environment calls for. This is the -mechanism behind the "swap the adapter" callouts throughout the book — and the -reason Lumen can stay in-memory for teaching while a production deployment flips -to real infrastructure through configuration alone. - -### `register_all!` — the explicit fallback - -When you want an explicit list (for generics that can't be scanned, or simply to -keep wiring local to one test), `register_all!` registers a known set on a -container: +The `:: beans (…) ::` block lists every registered bean grouped by stereotype: +`LumenBeans` and its factories, `WalletApi`, the `ledger` / `event_store` beans, +the `[repository] ReadModel`, the `[service] WalletHandlers` and +`WalletProjection`. This is the same data the admin dashboard's `/beans` view +renders, on the management port at `http://localhost:8081/admin/`. + +> **Note** **Key term — component scan (link-time).** Because Rust has no runtime +> reflection, each stereotype derive emits an `inventory` registration at compile +> time, and `firefly::scan(&container)` (equivalently `container.scan()`) collects +> every one linked into the binary and registers them — honoring conditions and +> profiles as it goes. `FireflyApplication` runs this scan for you at boot. + +What just happened: the report *is* the inventory the scan produced. Nothing is +reflective or hidden — "what is wired" is printed line-by-line. A missing +dependency would have aborted the boot with a named resolution error before this +report ever printed. + +> **Warning** Link-time discovery has one Rust-specific wrinkle. A bean is +> discoverable only when its crate's registrations are **linked into the binary**. +> For a single-crate app like Lumen that is automatic. But in a multi-crate +> service, a *layer* crate the binary only depends on transitively — a `-models` or +> `-core` crate whose beans are never named directly — can be **dead-stripped** by +> the linker, beans and all. Force-link those with +> [`firefly::link!`](./22-layered-microservices.md) at the binary's crate root +> (`firefly::link!(my_core, my_models);`) and guard the result with +> `firefly::assert_discovered(...)`. Single-crate Lumen never needs this; the +> note is here so the report's emptiness for a stripped crate is never a mystery. + +> **Tip** **Checkpoint.** The `:: beans ::` block names every bean you declared in +> this chapter, with no registration call anywhere in your code. That is the +> payoff: you wrote declarations, the framework wrote the graph. + +## The one escape hatch — `register_all!` + +Component scanning is the path Lumen and the framework take, and it is the default +for everything in `samples/lumen`. There is exactly one explicit fallback, for the +two cases the scan cannot reach: ```rust,ignore let c = Container::new(); @@ -407,39 +650,54 @@ firefly::register_all!(&c, [ReadModel, Ledger, WalletApi]); let api = c.resolve::().expect("controller resolves"); ``` -### Errors and introspection +Reach for `register_all!` for **generic beans** — a generic type's +monomorphization is chosen at the use site, so it can't be inventoried — or simply +to keep wiring local to a single focused test. Both register the same beans +against the same container; the scan just builds the list for you from the +link-time inventory. The lower-level entry point underneath the scan is the +`ApplicationContext`, which wraps the container with the full startup sequence and +is handy in a test: -The error taxonomy is precise: a missing bean, a non-unique bean with no -`primary`, and a detected circular dependency each surface as a distinct, -named error at resolution time. For diagnostics, the container can list its -registered beans and report per-bean resolution stats — the data the admin -dashboard's `/beans` view renders. - -## Scan-driven wiring vs. an explicit list +```rust,ignore +use firefly::prelude::*; -The container is the path Lumen — and the framework — take: declare beans, let -`FireflyApplication`'s component scan discover and wire them, and read the result -off the `/beans` view. There is one explicit escape hatch, `register_all!`, for -the cases the scan can't reach: +let ctx = ApplicationContext::builder() + .profiles(["test"]) + .property("feature.audit", "on") + .build(); +let c = ctx.container(); -- **Component scanning** is the default. `#[derive(...)]` a stereotype (or add a - `#[bean]` factory) and the bean is discovered link-time, condition-checked, and - wired — no list to maintain. This is how every bean in `samples/lumen` is - registered. -- **`register_all!`** is the explicit fallback. Reach for it for **generic - beans** (which can't be inventoried, since the monomorphization is chosen at the - use site) or to keep wiring local to a single focused test. +// Every stereotype-derived bean in the crate graph is discovered and wired. +let api = c.resolve::().expect("scan registered the controller"); +``` -Both register the same beans against the same container; the scan just builds the -list for you from the link-time inventory. +The error taxonomy is precise: a missing bean, a non-unique bean with no +`primary`, and a detected circular dependency each surface as a distinct, named +error at resolution time — the data the admin `/beans` view also reports. +[Dependency Injection & Auto-Configuration](./04a-dependency-injection.md) covers +the full container surface — scopes, named beans, `bind`, `register_all!`, and the +error model — in depth. ## Recap — what changed in Lumen | Before | After this chapter | |--------|--------------------| -| wiring imagined as a hand-written function | understood as **declared beans** the framework's component scan discovers and wires — no composition root to maintain | -| ports felt abstract | seen concretely as `Arc` parameters on the `ledger` `#[bean]` — one factory to swap an adapter | -| how `FireflyApplication` resolves the graph unclear | named: register infra beans → scan → resolve controllers, constructing collaborators in dependency order | +| wiring imagined as a hand-written composition root | understood as **declared beans** the component scan discovers and wires — no root to maintain | +| a stereotype derive looked decorative | seen as **the registration itself**: one derive makes a managed bean and records its layer | +| `#[autowired]` felt like a single-value annotation | known as four injection modes — `Arc`, `Vec>`, `Option>`, `Provider` | +| ports felt abstract | seen concretely as `Arc` / `Arc` — one `#[bean]` factory to swap an adapter | +| how `FireflyApplication` resolves the graph was unclear | named: pre-register infra beans → scan → resolve controllers, building collaborators leaves-first in dependency order | + +You also now know: + +- Why a Firefly service has no `build_app` — declarations plus a component scan + replace the hand-written graph, and the framework *is* the composition root. +- That conditions and profiles gate a bean's existence, so one codebase runs + in-memory in dev and on real infrastructure in prod without an `if`. +- That `post_construct` / `pre_destroy` give a bean its own one-time wiring and + teardown, ordered by the container. +- That `register_all!` and `ApplicationContext::builder()` are the explicit + fallbacks for generics and focused tests — everything else is scanned. ## Exercises @@ -447,20 +705,33 @@ list for you from the link-time inventory. the startup report. Find `LumenBeans`, `WalletApi`, the `ledger` / `event_store` factories, and the `[repository] ReadModel` data-access bean, grouped by stereotype — the same data the admin dashboard's `/beans` view - renders. + renders at `http://localhost:8081/admin/`. 2. **Add a bean and watch it appear.** Add a small `#[derive(Service)]` to - `web.rs`, run Lumen, and confirm it shows up in the startup report — you wrote - no registration call. Then add `#[firefly(condition_on_property = + `web.rs`, run Lumen, and confirm it shows up in the report — you wrote no + registration call. Then add `#[firefly(condition_on_property = "wallet.enabled=true")]` and watch it disappear until you set the property. 3. **Auto-bind a port.** Define a `Clock` trait, give `SystemClock` `#[firefly(provides = "dyn Clock", primary)]`, and resolve `dyn Clock`. Add a - second implementation without `primary` and observe the non-unique-bean error; - move `primary` to fix it. + second implementation *without* `primary`, observe the non-unique-bean error, + then move `primary` to the one you want as the default. 4. **Swap a store from one factory.** Change the `event_store` `#[bean]` in `LumenBeans` to return a different store, and explain in one sentence why the - `ledger` factory, the handlers, and the controller need no change — they - depend on the `EventStore` *port*. - -With the wiring understood, the reactive primitives underpin everything that -follows — read [The Reactive Model](./05-reactive-model.md) next, then give -Lumen its first endpoints in [Your First HTTP API](./06-first-http-api.md). + `ledger` factory, the handlers, and the controller need no change — they depend + on the `EventStore` *port*, not the concrete store. +5. **Trace a resolution.** Pick `WalletHandlers` and write down, in order, every + bean the container must build before it can construct that handler. Check your + answer against the `#[autowired]` fields in `src/commands.rs` and the `ledger` + factory in `src/web.rs`. + +## Where to go next + +- Go deep on the container in **[Dependency Injection & + Auto-Configuration](./04a-dependency-injection.md)** — scopes, named beans and + qualifiers, `Container::bind`, the full conditional surface, and the + auto-configuration model this chapter only sketched. +- See exactly what `run()` does, stage by stage, in **[Bootstrapping with + FireflyApplication](./04b-bootstrap.md)** — the boot pipeline that drives the + scan you just learned. +- Then meet the reactive primitives every later chapter builds on in **[The + Reactive Model — Mono & Flux](./05-reactive-model.md)**, and give Lumen its + first endpoints in **[Your First HTTP API](./06-first-http-api.md)**. diff --git a/docs/book/src/04a-dependency-injection.md b/docs/book/src/04a-dependency-injection.md index 727bf1a..abaf0d9 100644 --- a/docs/book/src/04a-dependency-injection.md +++ b/docs/book/src/04a-dependency-injection.md @@ -1,54 +1,126 @@ # Dependency Injection & Auto-Configuration -The [previous chapter](./04-dependency-wiring.md) showed how Lumen is wired: it -**declares beans** and `FireflyApplication`'s component scan discovers and -connects them at boot. This chapter is the full reference for that container — -the engine that turns a crate full of `#[derive(...)]` and `#[bean]` -declarations into a wired object graph. - -By the end of it you will know how Lumen's exact collaborators — `ReadModel`, -`Ledger`, `WalletApi` — are expressed as **beans**: declared with stereotype -derives, wired by constructor injection, disambiguated with -`#[firefly(primary)]`, gated by profiles and conditions, bracketed by lifecycle -hooks, auto-configured with sensible defaults, and discovered by component -scanning. This is the full Firefly DI surface, told against Lumen's own -collaborators. +The [previous chapter](./04-dependency-wiring.md) walked Lumen's wiring at a +glance: it **declares beans** and `FireflyApplication`'s component scan discovers +and connects them at boot. This chapter is the guided, from-first-principles tour +of that container — the engine that turns a crate full of `#[derive(...)]` and +`#[bean]` declarations into a wired object graph. We will build up the whole DI +surface one idea at a time, always against Lumen's own collaborators +(`ReadModel`, `Ledger`, `WalletApi`), so by the end nothing in the container is a +black box. + +You do not need to have finished the previous chapter to follow this one — every +concept is reintroduced here in context. But you should have a Lumen crate that +compiles and runs (from [Quickstart](./02-quickstart.md)), because the examples +mirror code that already ships in +[`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen). + +By the end of this chapter you will: + +- Explain **inversion of control** the Rust way, and why Firefly's discovery is + *link-time* rather than reflective. +- Declare a bean with a **stereotype derive** and wire its dependencies with + `#[autowired]` constructor injection. +- Disambiguate competing adapters with `#[firefly(primary)]`, names, and + qualifiers, and read the four resolution rules in priority order. +- Produce beans you do not own with `#[derive(Configuration)]` + `#[bean]` + factories, including **async** factories that do I/O at startup. +- Gate beans by **profiles** and the `condition_on_*` family, and let an + **auto-configuration** back off the moment you declare your own bean. +- Bracket a bean with lifecycle hooks, choose its **scope**, and introspect the + whole graph through the `/beans` view. + +## Concepts you will meet + +Before the first line of code, here are the load-bearing terms. Each is +reintroduced in context the first time it is used; this is the short version so +the map is in your head from the start. + +> **Note** **Key term — bean.** A *bean* is any value the framework constructs, +> wires, owns, and hands to whoever needs it. You declare beans; the container +> discovers them at startup and connects them. This is exactly Spring's notion of +> a bean managed by the application context. + +> **Note** **Key term — container (DI container).** The *container* is the +> registry that holds every bean, resolves a bean's dependencies, and builds the +> object graph in dependency order. Firefly's container is the type +> `firefly::container::Container`, usually driven through an +> `ApplicationContext`. The Spring analog is the `ApplicationContext` / +> `BeanFactory`. + +> **Note** **Key term — inversion of control (IoC).** *Inversion of control* +> means the framework calls your constructors in the right order, instead of your +> code calling them by hand. You declare *what* a bean needs; the container +> decides *when* and *in what order* to build it. Dependency injection is the +> concrete mechanism that delivers IoC. + +> **Note** **Key term — port and adapter.** A *port* is an abstract capability +> your code depends on (a trait, e.g. `EventStore`); an *adapter* is a concrete +> implementation of it (e.g. `MemoryEventStore`). Depending on the port and +> selecting the adapter at wiring time is what lets one Lumen codebase run on +> in-memory infrastructure in tests and real infrastructure in production. This +> is the hexagonal-architecture vocabulary. > **Design note.** Firefly's container offers a declarative DI model — > stereotype derives, `#[autowired]`, `#[bean]` factories, `primary`, profiles, > the `condition_on_*` family, and lifecycle hooks. Because Rust has no runtime -> reflection, discovery is **link-time** (via `inventory`) and autowiring is -> generated by a derive macro rather than inferred at runtime; a bean is -> discoverable exactly when its crate is linked. The model is: declare intent, -> let the container provide the instances. The surface will feel familiar if -> you've used a batteries-included framework, but the link-time mechanism is +> reflection, discovery is **link-time** (via the `inventory` crate) and +> autowiring is generated by a derive macro rather than inferred at runtime; a +> bean is discoverable exactly when its crate is linked. The model is: declare +> intent, let the container provide the instances. The surface feels familiar if +> you have used a batteries-included framework, but the link-time mechanism is > Firefly's own. -## Inversion of control, the Rust way +## Step 1 — See the problem inversion of control solves -The container inverts the *who calls whom*. Rather than a function calling each -constructor in turn: +Start by writing the wiring *by hand*, the way you would without a container, so +the value of inverting it is concrete. Lumen's read side keeps wallet views in a +`ReadModel`; its write side is a `Ledger` over an event store and a broker; its +HTTP surface is a `WalletApi` controller that needs both the CQRS bus and the +ledger. Wired by hand, that is a small but real assembly: ```rust,ignore +let store = Arc::new(MemoryEventStore::new()); +let broker = Arc::new(InMemoryBroker::new()); let read_model = Arc::new(ReadModel::default()); -let ledger = Ledger::new(Arc::clone(&store), Arc::clone(&broker)); -// ... hand the collaborators to the controller state. +let ledger = Arc::new(Ledger::new(Arc::clone(&store), Arc::clone(&broker))); +// ... and then hand the collaborators to the controller's state, in order. ``` -every bean *declares* its dependencies and the container calls the constructors -in dependency order. The wiring lives in declarations next to each type, and -`FireflyApplication`'s component scan turns those declarations into the wired -graph at boot. The result is a central registry the admin dashboard introspects -(`/beans`) and the startup report logs line-by-line — and a failure surfaces as -a clear resolution error at startup, not a panic deep in the first request. +The trouble is not the four lines — it is that **you** are responsible for the +*order*. `Ledger` must exist before `WalletApi`; `store` and `broker` must exist +before `Ledger`. Add a fifth collaborator and you re-thread the function. The +container inverts this: every bean *declares* its dependencies, and the container +calls the constructors in dependency order for you. -## Stereotypes: declaring your beans +> **Note** **Key term — composition root.** The *composition root* is the one +> place in a program where the whole object graph is assembled. The hand-written +> block above *is* a composition root. In Firefly the framework is the composition +> root: it scans your beans and wires them, so you never spell the graph out in a +> function. + +What just happened: you saw the exact wiring the container will take over. The +payoff is not just fewer lines — it is a central registry the admin dashboard +introspects (the `/beans` page), a startup report that logs the graph +line-by-line, and a *resolution error at startup* instead of a panic deep in the +first request when something is missing. + +> **Tip** **Checkpoint.** You can name Lumen's three core collaborators — +> `ReadModel`, `Ledger`, `WalletApi` — and say which depends on which. Hold that +> graph in mind; every step below wires one edge of it. + +## Step 2 — Declare a bean with a stereotype derive A **bean** is any value the container builds, wires, and owns. You make a type a bean with a **stereotype derive** — a thin annotation that generates a -`firefly_register(&Container)` method *and* submits a link-time scan thunk. Five -stereotypes ship, all functionally equivalent, differing only in the -architectural intent they document and the label they carry into `/beans`: +`firefly_register(&Container)` method *and* submits a link-time scan thunk so the +component scan can find it. + +> **Note** **Key term — stereotype.** A *stereotype* is a derive that both makes +> a type a bean and documents its architectural role. Five ship, all functionally +> equivalent — they differ only in the intent they record and the label they +> carry into the `/beans` view. These are Spring's `@Component` / `@Service` / +> `@Repository` / `@Configuration` / `@Controller` stereotypes. | Derive | Documents | |--------|-----------| @@ -58,29 +130,51 @@ architectural intent they document and the label they carry into `/beans`: | `#[derive(Configuration)]` | a holder of `#[bean]` factories | | `#[derive(Controller)]` | a web controller bean | -Lumen's read model is a data-access bean, so it would carry `#[derive(Repository)]`: +Lumen's read model is a data-access bean, so it carries `#[derive(Repository)]`. +This is real code from `samples/lumen/src/ledger.rs`: ```rust,ignore use firefly::prelude::*; -use std::sync::Mutex; use std::collections::HashMap; +use std::sync::Mutex; -#[derive(Repository, Default)] +#[derive(Debug, Default, Repository)] pub struct ReadModel { rows: Mutex>, } // generated: ReadModel::firefly_register(&container) + a component-scan thunk ``` -> **Tip.** Choosing `Repository` over `Component` costs nothing technically but -> tells every reader exactly what `ReadModel` is for. +What just happened: deriving `Repository` registered `ReadModel` as a singleton +bean and submitted a scan thunk so `container.scan()` finds it at boot. Choosing +`Repository` over `Component` costs nothing technically but tells every reader — +and the `/beans` page — exactly what `ReadModel` is for. + +> **Note** **Key term — singleton.** A *singleton* bean has one instance, cached +> after the first time it is resolved, and shared by everything that depends on +> it. It is the default scope; you will meet the others in Step 9. Spring's +> default bean scope is the same. -## Constructor injection with `#[autowired]` +Why it matters: the projection that fills the read model and the query handler +that reads it both autowire `Arc` — and because it is a singleton, +they share *the same* map. A read-after-write sees the write. + +> **Tip** **Checkpoint.** A struct with a stereotype derive is a bean. If you ran +> Lumen and opened `http://localhost:8081/admin/`, `ReadModel` appears on the +> `/beans` page labelled `repository`. + +## Step 3 — Inject dependencies with `#[autowired]` A bean's dependencies are its `#[autowired]` fields. The container resolves each -one by type and injects it before the bean is built. The field type controls the +one by type and sets it before the bean is built. The **field type** controls the *shape* of the injection: +> **Note** **Key term — autowiring (constructor injection).** *Autowiring* means +> the container fills a field by resolving its type from the registry, rather than +> you passing the value in. Firefly does this at construction time, so a missing +> required dependency is a loud error at startup, not a `None` three frames into a +> request. This is Spring's `@Autowired` constructor injection. + | Field type | Resolves via | |------------|--------------| | `Arc` | `resolve::()` (required) | @@ -88,68 +182,95 @@ one by type and injects it before the bean is built. The field type controls the | `Vec>` | `resolve_all::()` (all implementations) | | `Provider` | a deferred handle (`container.provider::()`) | -Lumen's `Ledger` depends on an event store and a broker; as a bean it autowires -them, and a `WalletApi` controller bean autowires the `Bus` and the `Ledger`: +Lumen's read-model **projection** is a `#[derive(Service)]` bean that autowires +the `Ledger` (whose event store it replays) and the `ReadModel` it feeds. This is +real Lumen code: ```rust,ignore use firefly::prelude::*; use std::sync::Arc; #[derive(Service)] -pub struct Ledger { - #[autowired] store: Arc, - #[autowired] broker: Arc, -} - -#[derive(Controller)] -pub struct WalletApi { - #[autowired] bus: Arc, - #[autowired] ledger: Arc, +struct WalletProjection { + #[autowired] + ledger: Arc, + #[autowired] + read_model: Arc, } ``` -When the container builds `WalletApi`, it builds `Ledger` first (recursively -resolving its store and broker) — and the order is derived from the field types, -not written out. A field with no attribute is built with `Default::default()`; a -`#[firefly(value = "${...}")]` field is bound from config (see *Config-driven -injection* below). - -> **Note.** `#[autowired]` is constructor injection: a missing required -> dependency is a loud `ContainerError::NoSuchBean` at resolve time, with fuzzy -> "did you mean…" suggestions, rather than a `None` panic three frames deep. - -> **Note — the `Default` rule.** The generated factory constructs the struct as a -> literal, filling `#[autowired]`/`#[firefly(value = ...)]` fields from the +When the container builds `WalletProjection`, it resolves `Arc` and +`Arc` first — recursively building each if it has not already — then +constructs the projection with both fields set. The order is *derived from the +field types*, never written out. + +> **Note** **Key term — `Arc`.** `Arc` is Rust's atomically reference-counted +> shared pointer. Beans are shared (a singleton has many holders), so the container +> always hands out an `Arc`. Cloning an `Arc` is cheap — it bumps a counter, not +> the data — which is why bean structs are usually `#[derive(Clone)]` with `Arc` +> fields. + +What just happened: you declared *what* the projection needs, and the container +will provide it in dependency order. A field with no attribute is filled with +`Default::default()`; a `#[firefly(value = "${...}")]` field is bound from config +(Step 8). + +> **Note** A missing **required** dependency (`Arc` with no provider) is a loud +> `ContainerError::NoSuchBean` at resolve time, with fuzzy "did you mean…" +> suggestions, rather than a silent failure. Make it optional with +> `Option>` and the field becomes `None` instead of an error. + +> **Note** **The `Default` rule.** The generated factory constructs the struct as +> a literal, filling `#[autowired]` / `#[firefly(value = ...)]` fields from the > container and **every other field** from `Default::default()`. So a stereotype > struct needs `#[derive(Default)]` if — and only if — it has at least one field > that is neither `#[autowired]` nor `#[firefly(value = ...)]` (like `ReadModel`'s -> `rows` above). An all-autowired struct, or a unit/field-less holder like a +> `rows`). An all-autowired struct, or a field-less holder like a > `#[derive(Configuration)]`, compiles without it. -> **Note — `Provider` in a hand-wired container.** Autowiring a `Provider` -> field needs the container to hold a handle to itself, which only a shared -> container has. Build one with `Container::shared()` (or call -> `install_shared_handle()` on an `Arc`); an `ApplicationContext` -> already does this for you. Resolving a `Provider` field against a bare -> `Container::new()` panics with a message telling you to use `shared()`. +> **Note** Autowiring a `Provider` field needs the container to hold a handle +> to *itself*, which only a shared container has. Build one with +> `Container::shared()` (or call `install_shared_handle()` on an +> `Arc`); an `ApplicationContext` already does this for you. Resolving a +> `Provider` field against a bare `Container::new()` panics with a message +> telling you to use `shared()`. + +> **Tip** **Checkpoint.** You can predict the build order of `WalletProjection`: +> the container resolves `Ledger` and `ReadModel` first, then constructs the +> projection. Nowhere did you write that order down — the field types encode it. -## Resolution rules, `#[firefly(primary)]`, and ports +## Step 4 — Depend on a port, get the adapter -Lumen depends on *ports* — `EventStore`, `Broker` — and selects an adapter at -wiring time. The container expresses "depend on the port, get the adapter" with -a trait-object **binding**. Register the concrete type, bind the trait to it, -then resolve the trait: +Lumen depends on *ports* — `EventStore`, `Broker` — and picks an adapter at +wiring time. The container expresses "depend on the port, get the adapter" with a +trait-object **binding**: register the concrete type, bind the trait to it, then +resolve the trait. ```rust,ignore +use firefly::prelude::*; + let c = Container::new(); firefly::register_all!(&c, [MemoryEventStore]); c.bind::(|a| a); let store: Arc = c.resolve().unwrap(); ``` -When a single implementation is bound, it resolves immediately. When **several** -adapters back one port — the classic in-memory-vs-Redis split — exactly one must -be marked primary, or resolution fails loudly: +What just happened: `register_all!` registered the concrete `MemoryEventStore`; +`bind` recorded "the trait `EventStore` is satisfied by `MemoryEventStore`"; and +`resolve` then returns the adapter through the port type. Consumers autowire +`Arc` and never name the concrete adapter. + +> **Note** `bind::(|a| a)` panics if `T` is not registered first — bind is a +> view over an *existing* registration, so register the concrete type before you +> bind a trait to it. + +When **several** adapters back one port — the classic in-memory-vs-Postgres split +— exactly one must be marked **primary**, or resolution fails loudly: + +> **Note** **Key term — primary bean.** When more than one bean satisfies a type, +> the one marked `#[firefly(primary)]` is the default choice. With no primary and +> more than one candidate, resolution fails rather than guessing. This is Spring's +> `@Primary`. ```rust,ignore #[derive(Repository)] @@ -162,25 +283,34 @@ pub struct PostgresEventStore { /* … */ } // activated by profile/condition The resolution rules, in strict priority order: -1. **Direct registration** — `T` registered directly resolves to it. +1. **Direct registration** — a type `T` registered directly resolves to it. 2. **Single binding** — one implementation bound to a trait resolves to it. 3. **`#[firefly(primary)]`** — among several bindings, the primary one wins. 4. **Error** — `NoSuchBean` when nothing matches; `NoUniqueBean` (naming every competing candidate) when several match with no primary. -> **Note.** `#[firefly(primary)]` marks the default adapter among several bound -> to one port; with no primary, resolution fails loudly with a `NoUniqueBean` -> error that names every competing candidate. Moving `primary` from one adapter -> to the other is the *only* change needed to swap Lumen's backing store — -> nothing in `Ledger` changes. +What just happened: you learned the only four outcomes `resolve` can produce. +Moving `primary` from one adapter to the other is the *only* change needed to swap +Lumen's backing store — nothing in `Ledger` changes, because `Ledger` depends on +the port. + +> **Tip** **Checkpoint.** Given two adapters bound to one port with no primary, +> you can predict the error: `NoUniqueBean`, naming both candidates. Add +> `#[firefly(primary)]` to one and resolution succeeds — with no change to the +> consumer. ### `#[firefly(order)]` and named beans `#[firefly(order = N)]` controls initialization sequence and the order `resolve_all::()` returns implementations (lower runs first); the constants `HIGHEST_PRECEDENCE` / `LOWEST_PRECEDENCE` mark the extremes. When two beans share -a type — say a primary and a read-replica store — give one a name and select it -with a **qualifier**: +a type — say a primary store and a read-replica — give one a **name** and select +it with a **qualifier**: + +> **Note** **Key term — qualifier.** A *qualifier* names which of several +> same-typed beans you want at an injection site. The producing bean carries +> `#[firefly(name = "…")]`; the consuming field carries +> `#[firefly(qualifier = "…")]`. This is Spring's `@Qualifier`. ```rust,ignore #[derive(Repository)] @@ -193,102 +323,312 @@ pub struct ReportService { } ``` -> **Note.** A mistyped qualifier name pointing at the wrong type is a clear -> `NoSuchBean`, not a silently-wrong injection. +What just happened: the qualifier disambiguates by name where the type alone is +ambiguous. A mistyped qualifier pointing at the wrong type is a clear +`NoSuchBean`, not a silently-wrong injection. + +## Step 5 — Produce beans you do not own with `#[bean]` factories -## Bean factories: `#[derive(Configuration)]` + `#[bean]` +Not every dependency is a type you can annotate — a third-party client needs +constructor arguments, an interface needs a hand-built adapter. For these, a +`#[derive(Configuration)]` holder exposes `#[bean]` **factory methods**. Each +method is keyed by its **return type**; its `Arc` arguments are resolved from +the container, so a factory can depend on other beans. -Not every dependency is a type you own — a third-party client needs constructor -arguments, an interface needs a hand-built adapter. For these, a -`#[derive(Configuration)]` holder exposes `#[bean]` factory methods. Each method -is keyed by its **return type**; its `Arc` arguments are resolved from the -container (so a factory can depend on other beans). This is how Lumen would -produce its `InMemoryBroker` behind the `Broker` port: +> **Note** **Key term — bean factory.** A *bean factory* is a method whose return +> value becomes a bean, keyed by the return type. You use it when a bean cannot +> simply derive a stereotype — it needs construction logic or wraps a foreign +> type. This is Spring's `@Bean` method on a `@Configuration` class. + +This is exactly how Lumen produces its core infrastructure beans — real code from +`samples/lumen/src/web.rs`: ```rust,ignore use firefly::prelude::*; use std::sync::Arc; +use firefly::cqrs::QueryCache; +use firefly::eda::Broker; +use firefly::eventsourcing::{EventStore, MemoryEventStore}; #[derive(Configuration)] -pub struct LumenInfraConfig; +struct LumenBeans; -#[firefly::bean] -impl LumenInfraConfig { - #[bean(primary)] - fn broker(&self) -> firefly::eda::InMemoryBroker { - firefly::eda::InMemoryBroker::new() +#[bean] +impl LumenBeans { + /// The in-memory event store. + #[bean] + fn event_store(&self) -> MemoryEventStore { + MemoryEventStore::new() } + /// The read-side query cache honouring `GetWallet`'s 30s TTL. #[bean] - fn ledger(&self, store: Arc) -> Ledger { - Ledger::new(store, /* … */) + fn query_cache(&self) -> QueryCache { + QueryCache::new() + } + + /// The ledger application service — autowires the event store and the + /// framework-provided `Broker` port. + #[bean] + fn ledger(&self, store: Arc, broker: Arc) -> Ledger { + let store: Arc = store; + Ledger::new(store, broker) } } ``` -You **do not** call anything to wire these: `Container::scan()` discovers the -holder *and* its `#[bean]` methods (each submits its own `inventory` thunk) and -registers them automatically. (`LumenInfraConfig::firefly_register_beans(&container)` -is still generated for the explicit `register_all!` path and for generic -holders that can't be inventoried.) +What just happened: `LumenBeans` is a field-less `@Configuration` holder. Its +three methods produce three beans, each keyed by its return type +(`MemoryEventStore`, `QueryCache`, `Ledger`). The `ledger` method's arguments +(`Arc`, `Arc`) are resolved from the container — so +a factory can wire itself from other beans. You **do not** call anything to wire +these: `Container::scan()` discovers the holder *and* its `#[bean]` methods (each +submits its own scan thunk) and registers them automatically. + +> **Note** This is why Lumen's `Ledger` is a plain struct, not a +> `#[derive(Service)]` bean: it is *produced* by this factory rather than derived. +> A downstream `#[autowired] ledger: Arc` (in `WalletApi`, in the +> projection) finds it because the factory registered the value under the `Ledger` +> type. Swapping `MemoryEventStore` for a Postgres adapter is one line in this +> holder; the rest of Lumen is untouched. A `#[bean]` method returns a **concrete (sized) type** — that is the bean's key. To expose it behind a port, add `#[firefly(provides = "dyn Broker")]` to the -holder or call `Container::bind` after registration. Per-method options mirror -the struct-level conditionals: `#[bean(name = "...", scope = "...", primary, -order = N, profile = "...", condition_on_property = "k=v", condition_on_class = -"…", condition_on_bean = "T", condition_on_missing_bean = "T", +holder or call `Container::bind` after registration. Per-method options mirror the +struct-level ones: `#[bean(name = "...", scope = "...", primary, order = N, +profile = "...", condition_on_property = "k=v", condition_on_class = "…", +condition_on_bean = "T", condition_on_missing_bean = "T", condition_on_single_candidate = "T")]`. -> **Note.** A `#[bean]` method is keyed by its return type: the container -> registers the produced value *under that type*, so a downstream `#[autowired] -> ledger: Arc` finds it. Swapping `InMemoryBroker` for a Kafka adapter -> is one line in this holder; the rest of Lumen is untouched. +> **Tip** **Checkpoint.** You can explain why `LumenBeans` calls no +> `register_*` and no `bind`: `container.scan()` discovers each `#[bean]` and +> registers its return value under the return type. The framework does the +> registration. ### Async beans (`async fn #[bean]`) -A bean that performs I/O to construct itself — opening a database pool, dialing -a broker, warming a cache — declares its factory `async`: +A bean that performs I/O to construct itself — opening a database pool, dialing a +broker, warming a cache — declares its factory `async`. The +[`lumen-ledger`](./22-layered-microservices.md) service does exactly this; real +code from its `WalletPersistenceConfig`: ```rust,ignore +use firefly::data_sqlx::Db; +use firefly::prelude::*; + +#[derive(Configuration, Default)] +pub struct WalletPersistenceConfig; + #[firefly::bean] -impl PersistenceConfig { +impl WalletPersistenceConfig { + /// The `Db` datasource bean — an async factory that opens the connection + /// pool and applies the schema with `await`. #[bean] - async fn wallet_repository(&self) -> WalletRepository { - WalletRepository::new(connect_and_migrate().await) + async fn data_source(&self) -> Db { + connect_and_migrate().await } } ``` -The framework parks an `async` factory during the synchronous `container.scan()` -and `await`s it during `Container::init_async_beans()` — run by the bootstrap -immediately after the scan, before controllers, handlers, and eager singletons -resolve — then installs the result as a ready singleton. Async beans are -sequenced by `#[bean(order = N)]`, so one can autowire another initialised -earlier. This is Spring Boot's "a `@Bean` does blocking I/O at context-refresh -time", except the I/O is `await`ed instead of blocking a thread. A factory -failure is reported as a `BeanCreation` error naming the bean — Spring's -"Error creating bean named '…'". - -`FireflyApplication` drains async beans on its own bootstrap path. If you build -an `ApplicationContext` directly, call **`build_async().await`** instead of -`build()` — the synchronous `build()` cannot `await` a pending async bean and -**panics** rather than silently leaving it uninitialized: +What just happened: the framework parks an `async` factory during the synchronous +`container.scan()` and `await`s it during `Container::init_async_beans()` — run by +the bootstrap immediately after the scan, before controllers, handlers, and eager +singletons resolve — then installs the result as a ready singleton. Async beans +are sequenced by `#[bean(order = N)]`, so one can autowire another initialised +earlier. + +> **Note** This is Spring Boot's "a `@Bean` does blocking I/O at context-refresh +> time", except the I/O is `await`ed instead of blocking a thread. A factory +> failure is reported as a `BeanCreation` error naming the bean — Spring's "Error +> creating bean named '…'". + +`FireflyApplication` drains async beans on its own bootstrap path. If you build an +`ApplicationContext` directly, call **`build_async().await`** instead of `build()` +— the synchronous `build()` cannot `await` a pending async bean and **panics** +rather than silently leaving it uninitialized: ```rust,ignore let ctx = ApplicationContext::builder().build_async().await?; // awaits async beans ``` -An async-constructed data-access bean still classifies as `@Repository` in the -admin `/beans` view with `#[bean(stereotype = "repository")]`. The -[`lumen-ledger`](./22-layered-microservices.md) service uses an `async fn #[bean]` -for exactly this — its sqlx repository opens the pool and runs the migration with -`await`. +In `lumen-ledger`, the `Db` from this async factory is what the +`#[derive(SqlxRepository)]` repository is built from — the repository opens the +pool and runs the migration with `await` before any request arrives. + +## Step 6 — Gate beans by profile and condition + +So far every bean always exists. Conditions answer a different question: "should +this bean exist *at all*, given the environment?" This is the mechanism that lets +one Lumen codebase run on in-memory infrastructure in tests and real +infrastructure in production with no `if` in the service code. + +> **Note** **Key term — profile.** A *profile* is a named environment flag +> (`dev`, `test`, `prod`) that gates which beans are active. A bean with +> `#[firefly(profile = "prod")]` exists only when `prod` is active. Profiles +> support a boolean grammar — `prod & cloud`, `dev | test`, `!staging`, +> parentheses. This is Spring's `@Profile`. + +> **Note** **Key term — conditional bean.** A *conditional bean* exists only when +> a stated condition holds — a config property is set, a feature label is present, +> another bean exists or is missing. The container evaluates these at scan time. +> This is Spring Boot's `@ConditionalOn*` family. + +Conditions are evaluated by `Container::scan` in **two passes**. + +**Pass 1** settles config/profile facts — knowable before any bean is built: + +```rust,ignore +#[derive(Repository)] +#[firefly(profile = "prod", condition_on_property = "lumen.store.postgres=true")] +pub struct PostgresEventStore { /* … */ } +``` + +**Pass 2** evaluates registry-dependent conditions — knowable only after pass 1 +settles. This enables the **default-with-override** pattern: ship a fallback that +yields to any user-provided implementation: + +```rust,ignore +#[derive(Repository)] +#[firefly(condition_on_property = "lumen.store.postgres=true")] +pub struct PostgresEventStore { /* … */ } // real store when configured + +#[derive(Repository)] +#[firefly(condition_on_missing_bean = "EventStore")] +pub struct MemoryEventStore { /* … */ } // fallback whenever none is wired +``` + +What just happened: with the property unset, `PostgresEventStore` is skipped in +pass 1, so in pass 2 there is no `EventStore` bean — and `MemoryEventStore`'s +`condition_on_missing_bean` fires, registering the fallback. Set the property and +`PostgresEventStore` registers in pass 1, so the fallback backs off. No `if` in +sight. + +The full conditional family: `condition_on_property = "key=value"`, +`condition_on_class = "label"`, `condition_on_bean = "Type"`, +`condition_on_missing_bean = "Type"`, `condition_on_single_candidate = "Type"`, +plus `profile = "expr"`. + +> **Design note.** The scan evaluates conditions in two passes precisely so the +> registry-dependent ones (`condition_on_bean`, `condition_on_missing_bean`, +> `condition_on_single_candidate`) can see the *result* of the config/profile +> pass. This two-pass scan is how *all* of Firefly's own auto-configuration backs +> off when you provide your own bean — the subject of the next step. + +> **Tip** **Checkpoint.** You can predict which store resolves: with +> `lumen.store.postgres` unset, the memory fallback; with it set to `true`, the +> Postgres store. The service code that depends on `Arc` is +> identical in both cases. + +## Step 7 — Let an auto-configuration get out of your way + +An **auto-configuration** is how a starter contributes sensible defaults that +disappear the moment you declare your own. It is the default-with-override pattern +of Step 6, packaged. + +> **Note** **Key term — auto-configuration.** An *auto-configuration* is a +> `@Configuration` holder whose `#[bean]`s are guarded by +> `condition_on_missing_bean` and registered **last** in the scan — so your own +> bean always wins and the default contributes only when you wrote nothing. This +> is Spring Boot's `@AutoConfiguration`. + +Derive `#[derive(AutoConfiguration)]` and guard each `#[bean]`: + +```rust,ignore +use firefly::prelude::*; + +#[derive(AutoConfiguration, Default)] +pub struct CacheAutoConfiguration; + +#[firefly::bean] +impl CacheAutoConfiguration { + #[bean(condition_on_missing_bean = "CacheClient", condition_on_property = "cache.type=memory")] + fn cache_client(&self) -> CacheClient { + CacheClient::in_memory() // the default — only if you didn't wire one + } +} +``` + +What just happened: `#[derive(AutoConfiguration)]` is a `#[derive(Configuration)]` +whose beans register **last** during the scan. Because its `#[bean]`s carry +`condition_on_missing_bean`, the two-pass scan registers your unconditional bean +first, then *skips* the auto-configuration default — so your bean always wins, and +you never write an `if`. Drop the starter from your dependencies and the +auto-configuration code is not linked, so it contributes nothing: discovery is +link-time, not reflective. + +> **Design note.** "Present exactly when linked" is the whole trick. An +> auto-configuration contributes its defaults precisely when its crate is compiled +> in, and contributes nothing once you drop the starter — no classpath scan, no +> reflection, no `spring.factories` registry to maintain. + +## Step 8 — Pull configuration straight into a bean + +A service should not thread config through its constructor by hand. Two stereotype +attributes pull configuration straight in. They are covered in full in +[Configuration](./03-configuration.md) §"Binding config straight into a bean"; +this is the DI-relevant summary. + +**Single value — `#[firefly(value = "${key:default}")]`** binds one resolved, +placeholder-expanded scalar onto a field (parsed via `FromStr`); the `:default` +tail supplies a fallback when the key is absent: + +> **Note** **Key term — placeholder.** A *placeholder* like `${lumen.web.addr}` +> is replaced at bind time with the resolved config value, with an optional +> `:default` tail. Only `${...}` placeholders are supported — SpEL `#{...}` +> expressions are out of scope (see Step 12). This is Spring's `@Value` in its +> placeholder form. + +```rust,ignore +#[derive(Service)] +pub struct WalletApiConfig { + #[firefly(value = "${lumen.web.addr:127.0.0.1:8080}")] addr: String, +} +``` + +**Whole subtree — `#[derive(ConfigProperties)]`** binds a `serde` struct under a +prefix and registers it as an injectable singleton; any bean can then autowire it: -## Scopes +> **Note** **Key term — configuration properties.** A *configuration-properties* +> struct binds a whole config subtree (everything under a prefix) into a typed +> `serde` struct and registers it as a bean. This is Spring's +> `@ConfigurationProperties` plus `@EnableConfigurationProperties`. + +```rust,ignore +use serde::Deserialize; +use firefly::prelude::*; +use std::sync::Arc; + +#[derive(Deserialize, ConfigProperties, Default)] +#[firefly(prefix = "lumen.web")] +pub struct WebProperties { + pub addr: String, + #[serde(default)] pub admin_addr: String, +} + +#[derive(Service)] +pub struct ReportService { + #[autowired] props: Arc, // the bound subtree, injected +} +``` + +What just happened: `ConfigProperties` is the sixth container-aware derive +alongside the five stereotypes. It generates a `firefly_register` that binds and +registers the struct, and carries a `config_properties` stereotype label into +`/beans`. Adding `#[firefly(validate)]` (with `#[derive(Validate)]` on the struct) +runs the bound struct's declarative constraints after binding, and a violation +**fails the bean's creation** at startup rather than booting a malformed +configuration — Spring's `@Validated`. + +> **Note** `#[firefly(value = "${...}")]` is `@Value` (placeholder form), and +> `#[derive(ConfigProperties)]` + `#[firefly(prefix = "...")]` is +> `@ConfigurationProperties`. Both bind against the same merged, profile-resolved, +> placeholder-expanded config map described in [Configuration](./03-configuration.md). + +## Step 9 — Choose a bean's scope Every bean has a **scope** controlling how long its instance lives. Pass it as -`#[firefly(scope = "...")]`: +`#[firefly(scope = "...")]`. Almost every Lumen bean is a singleton; you reach for +the others rarely but deliberately. | Scope | Behaviour | Use for | |-------|-----------|---------| @@ -305,20 +645,23 @@ pub struct TransferContext { // a fresh scratch pad per transfer } ``` -> **Note.** Firefly's scopes are `singleton` (default), `transient` (a fresh -> instance per resolve), `request` (one per HTTP request), and `session` (one -> per session). Almost every Lumen bean is a singleton; the `request` and -> `session` scopes resolve through a `ScopeHandler` (see below). The scope name -> is the lower-case variant: `Scope::{Singleton, Transient, Request, Session}` -> in `crates/container/src/scope.rs`. +What just happened: the scope name is the lower-case variant of +`Scope::{Singleton, Transient, Request, Session}` (defined in +`crates/container/src/scope.rs`). A `transient` bean is rebuilt on every resolve; +the `request` and `session` scopes need a handler, covered next. ### Request and session scopes: the `ScopeHandler` SPI Rust has no ambient request thread-local the way a reflective JVM container does, so the `request` and `session` scopes are **driven explicitly** by an implementation of the `ScopeHandler` SPI — the direct analog of Spring's -`org.springframework...config.Scope`. A handler caches an instance per key -(per request, per session) and evicts it when that key ends: +`org.springframework...config.Scope`. A handler caches an instance per key (per +request, per session) and evicts it when that key ends: + +> **Note** **Key term — SPI (service provider interface).** An *SPI* is a trait +> the framework defines and a *host* implements to plug behaviour in. +> `ScopeHandler` is the SPI for request/session lifecycle: you install an +> implementation and the container drives scoping through it. ```rust,ignore use firefly::prelude::*; @@ -331,27 +674,25 @@ container.register_request_scope(Arc::new(my_request_handler)); container.register_session_scope(Arc::new(my_session_handler)); ``` -`register_request_scope` / `register_session_scope` back the two built-in -scopes; arbitrary custom scopes use `register_scope("name", handler)` (rejecting -empty names and the four built-ins). All three live on `Container` -(`crates/container/src/lib.rs`), and a `ScopeHandler` is just a `get(name, -factory)` plus `remove(name)` (`crates/container/src/scope.rs`). +What just happened: `register_request_scope` / `register_session_scope` back the +two built-in scopes; arbitrary custom scopes use `register_scope("name", handler)` +(which rejects empty names and the four built-ins). All three live on `Container` +(`crates/container/src/lib.rs`); a `ScopeHandler` is just a `get(name, factory)` +plus `remove(name)` (`crates/container/src/scope.rs`). -> **Design note.** The request/session lifecycle is *installed*, not implicit: a -> host registers a `ScopeHandler` via `register_request_scope` / -> `register_session_scope`, and request/session-scoped beans resolve through it. -> A bare `Container` with no handler installed reports `NoSuchBean` for those -> scopes rather than silently leaking a singleton — the deferral is explicit, the -> same trade described under *What Rust's model changes* below. +> **Design note.** The request/session lifecycle is *installed*, not implicit. A +> bare `Container` with no handler installed reports `NoSuchBean` for those scopes +> rather than silently leaking a singleton — the deferral is explicit, the same +> trade described in Step 12. -### `RefreshScope` — Spring Cloud `@RefreshScope` parity +### `RefreshScope` — config-reload parity -A fourth, ready-made handler ships for config-reload: `RefreshScope` +A fourth, ready-made handler ships for config reload: `RefreshScope` (`crates/container/src/scope.rs`). A refresh-scoped bean is cached like a singleton, but a `refresh()` call evicts **every** refresh-scoped instance so the next resolution rebuilds it against the new config — the hook a future -`/actuator/refresh` calls on a config change. Register it under the conventional -name `REFRESH_SCOPE_NAME` (`"refresh"`): +`/actuator/refresh` would call on a config change. Register it under the +conventional name `REFRESH_SCOPE_NAME` (`"refresh"`): ```rust,ignore use firefly::prelude::*; @@ -365,11 +706,11 @@ let evicted: Vec = refresh.refresh(); // rebuild on next resolve ``` For a single singleton there is a lighter hook: `container.reset_instance::()` -drops just `T`'s cached instance so it is rebuilt on next resolve (returns +drops just `T`'s cached instance so it is rebuilt on next resolve (it returns whether an instance was actually evicted). It is the per-bean form of the same refresh-on-config-change idea. -> **Spring parity.** `RefreshScope` / `REFRESH_SCOPE_NAME` mirror Spring Cloud's +> **Note** `RefreshScope` / `REFRESH_SCOPE_NAME` mirror Spring Cloud's > `@RefreshScope`, and `reset_instance::()` is the per-bean refresh hook. Both > exist so a config reload can rebuild affected beans without restarting the > process. @@ -377,9 +718,8 @@ refresh-on-config-change idea. ### `#[firefly(lazy)]` — opting out of the eager warm pass By default singletons are built **eagerly** when an `ApplicationContext` starts -(see *Startup and eager initialization* below). `#[firefly(lazy)]` (Spring -`@Lazy`) opts a singleton **out of that eager warm pass** — it is then built on -first resolve instead: +(Step 11). `#[firefly(lazy)]` opts a singleton **out of that eager warm pass** — +it is then built on first resolve instead: ```rust,ignore #[derive(Service)] @@ -387,19 +727,22 @@ first resolve instead: pub struct ExpensiveReportEngine { /* … */ } ``` -> **Spring parity.** `#[firefly(lazy)]` is `@Lazy`: it removes the bean from the -> startup warm pass so an expensive or rarely-used singleton is constructed only -> when something resolves it. (The macro carries the flag into the scan registry -> so `warm_singletons` can skip it — `crates/macros/src/container.rs`, -> `crates/firefly/src/context.rs`.) Unlike Spring it does **not** create a proxy -> and so does not break a true construction-time dependency cycle; reach for -> `Provider` to defer a dependency (see *What Rust's model changes*). +> **Note** `#[firefly(lazy)]` is `@Lazy`: it removes the bean from the startup +> warm pass so an expensive or rarely-used singleton is constructed only when +> something resolves it. Unlike Spring it does **not** create a proxy and so does +> not break a true construction-time dependency cycle; reach for `Provider` to +> defer a dependency (Step 12). -## Lifecycle: post-construct and pre-destroy +## Step 10 — Bracket a bean with lifecycle hooks -A bean often needs one-time setup after its dependencies are injected — Lumen's -projection, for instance, **subscribes to the broker** once at startup — and a -clean teardown on shutdown. Name a method per hook on the struct attribute: +A bean often needs one-time setup after its dependencies are injected, and a clean +teardown on shutdown. Name a method per hook on the struct attribute: + +> **Note** **Key term — lifecycle hooks.** A *post-construct* hook runs once after +> the bean is built and its dependencies injected; a *pre-destroy* hook runs on +> shutdown. They are attribute keys on the struct, not standalone macros. These +> are Spring's `@PostConstruct` / `@PreDestroy` (the `InitializingBean` / +> `DisposableBean` substitute). ```rust,ignore #[derive(Service)] @@ -412,40 +755,45 @@ impl ProjectionListener { } ``` -`on_start` runs after construction (all `#[autowired]` fields set), so it can -safely query collaborators. `Container::destroy()` (the `ApplicationContext`'s -`close()`) runs every `pre_destroy` hook in **reverse** construction order, so a -listener that started after the read model is stopped before it. +What just happened: `on_start` runs after construction — all `#[autowired]` fields +set — so it can safely query collaborators. `Container::destroy()` (the +`ApplicationContext`'s `close()`) runs every `pre_destroy` hook in **reverse** +construction order, so a listener that started after the read model is stopped +before it. -> **Note.** `post_construct` runs after construction and injection, so it can -> safely query collaborators; `pre_destroy` runs on close in **reverse** -> construction order, so a listener started after the read model is stopped -> before it. (Note these are *attribute keys on the struct*, not standalone -> macros.) +> **Note** In real Lumen the read-model projection subscribes to the broker via +> `#[event_listener]` (the EDA mechanism), not a `post_construct` hook — the hook +> here is the general pattern any bean uses for one-time setup. You will see the +> EDA path in [Messaging & Event-Driven Architecture](./10-eda-messaging.md). -## Startup and eager initialization +## Step 11 — Understand eager startup and fail-fast A bare `Container` builds singletons **lazily** — the first `resolve::()` constructs `T` and caches it. The `ApplicationContext`, however, is **eager** by -default, matching Spring's fail-fast startup: `ApplicationContext::build()` scans +default, matching Spring's fail-fast startup. `ApplicationContext::build()` scans the crate graph, registers the survivors, then immediately **warms** every -non-lazy singleton by resolving it once (`crates/firefly/src/context.rs`). Two -things follow from that warm pass: +non-lazy singleton by resolving it once (`crates/firefly/src/context.rs`). + +> **Note** **Key term — eager initialization / warm pass.** The *warm pass* is +> `build()` resolving every non-lazy singleton once at startup, so they are all +> constructed before the first request. This is what gives you Spring's "validate +> the wiring at startup" guarantee. + +Two things follow from that warm pass: - **Fail-fast.** A construction error — a missing required dependency, a panic in - a `#[post_construct]` — surfaces at *startup*, not deep into the first request. -- **`#[post_construct]` runs at startup.** Because warming a bean constructs it, - each non-lazy singleton's `post_construct` hook fires during `build()`, before - the context hands you the container. (Lumen's projection subscribes to the - broker here, not on first request.) + a `post_construct` — surfaces at *startup*, not deep into the first request. +- **`post_construct` runs at startup.** Because warming a bean constructs it, each + non-lazy singleton's `post_construct` hook fires during `build()`, before the + context hands you the container. That puts the whole bean lifecycle in one line: -> **scan → register → warm non-lazy singletons → `#[post_construct]` → (serve) -> → `close()` → `#[pre_destroy]` in reverse order** +> **scan → register → warm non-lazy singletons → `post_construct` → (serve) → +> `close()` → `pre_destroy` in reverse order** -You can opt the warm pass out entirely with `.eager(false)`, or opt a single -bean out with `#[firefly(lazy)]` (above): +You can opt the warm pass out entirely with `.eager(false)`, or opt a single bean +out with `#[firefly(lazy)]`: ```rust,ignore use firefly::prelude::*; @@ -456,17 +804,16 @@ let ctx = ApplicationContext::builder() .build(); ``` -> **Design note.** Eager warm-up at `build()` is where Firefly matches Spring's -> "validate the wiring at startup" guarantee. It is a *context*-level policy: a -> bare `Container` stays lazy. `.eager(false)` turns the whole warm pass off; +> **Design note.** Eager warm-up at `build()` is a *context*-level policy: a bare +> `Container` stays lazy. `.eager(false)` turns the whole warm pass off; > `#[firefly(lazy)]` excludes one bean from it. `close()` is the symmetric -> shutdown — it runs every `#[pre_destroy]` in reverse construction order and -> evicts the cached singletons (`crates/firefly/src/context.rs`). +> shutdown — it runs every `pre_destroy` in reverse construction order and evicts +> the cached singletons. ### The `ApplicationContext` builder surface -`ApplicationContext::builder()` accepts more than `.profiles()` and -`.property()`; the full surface (`crates/firefly/src/context.rs`): +`ApplicationContext::builder()` accepts more than `.profiles()` and `.property()`; +the full surface (`crates/firefly/src/context.rs`): | Method | Purpose | |--------|---------| @@ -476,152 +823,28 @@ let ctx = ApplicationContext::builder() | `.class(label)` | mark a feature "label" present for `condition_on_class` checks | | `.eager(bool)` | warm non-lazy singletons at `build()` — default `true` | | `.build()` | construct the shared container, scan, then (by default) warm singletons | +| `.build_async().await` | the same, but `await`s pending async beans (Step 5) | -> **Note.** `.class(label)` is Firefly's stand-in for `@ConditionalOnClass`: -> Rust has no classpath to probe, so a host declares which optional features are +> **Note** `.class(label)` is Firefly's stand-in for `@ConditionalOnClass`: Rust +> has no classpath to probe, so a host declares which optional features are > "present" by label, and `condition_on_class = "label"` beans gate on it. -## Config-driven injection - -Two stereotype attributes pull configuration straight into a bean, so a service -never threads config through its constructor. They are covered in full in -[Configuration](./03-configuration.md) §"Binding config straight into a bean"; -the DI-relevant summary: - -**Single value — `#[firefly(value = "${key:default}")]`.** Binds one resolved, -placeholder-expanded scalar onto a field (parsed via `FromStr`); the `:default` -tail supplies a fallback when the key is absent: +> **Tip** **Checkpoint.** You can state when a missing dependency surfaces: +> under an eager `ApplicationContext`, at `build()`; under a bare lazy `Container`, +> on first `resolve`. Lumen boots through `FireflyApplication`, which is eager — +> so wiring mistakes fail the boot, not the first request. -```rust,ignore -#[derive(Service)] -pub struct WalletApi { - #[autowired] ledger: Arc, - #[firefly(value = "${lumen.web.addr:127.0.0.1:8080}")] addr: String, -} -``` +## Step 12 — Component scanning vs. `register_all!` -**Whole subtree — `#[derive(ConfigProperties)]`.** Binds a `serde` struct under a -prefix and registers it as an injectable singleton (the -`@EnableConfigurationProperties` equivalent); any bean can then autowire it: - -```rust,ignore -use serde::Deserialize; - -#[derive(Deserialize, ConfigProperties, Default)] -#[firefly(prefix = "lumen.web")] -pub struct WebProperties { - pub addr: String, - #[serde(default)] pub admin_addr: String, -} - -#[derive(Service)] -pub struct ReportService { - #[autowired] props: Arc, // the bound subtree, injected -} -``` - -`ConfigProperties` is the sixth container-aware derive alongside the five -stereotypes: it generates a `firefly_register` that binds and registers the -struct, and carries a `config_properties` stereotype label into `/beans`. Adding -`#[firefly(validate)]` (with `#[derive(Validate)]` on the struct) is Spring's -`@Validated`: the bound struct's declarative constraints run after binding, and a -violation **fails the bean's creation** at startup rather than booting a malformed -configuration. Single values use `${...}` placeholders only — SpEL `#{...}` is -out of scope (see *What Rust's model changes*). - -> **Spring parity.** `#[firefly(value = "${...}")]` is `@Value` (placeholder -> form), and `#[derive(ConfigProperties)]` + `#[firefly(prefix = "...")]` is -> `@ConfigurationProperties`. Both bind against the same merged, -> profile-resolved, placeholder-expanded config map. - -## Conditional beans and profiles - -Conditions answer "should this bean exist *at all*, given the environment?" — -the mechanism that lets one Lumen codebase run on in-memory infrastructure in -tests and real infrastructure in production with no `if` in the service code. -They are evaluated by `Container::scan` in two passes: - -**Pass 1** (config/profile facts, knowable before any bean is built): - -```rust,ignore -#[derive(Repository)] -#[firefly(profile = "prod", condition_on_property = "lumen.store.postgres=true")] -pub struct PostgresEventStore { /* … */ } -``` - -**Pass 2** (registry-dependent, knowable only after pass 1 settles) — the -**default-with-override** pattern: ship a fallback that yields to any -user-provided implementation: - -```rust,ignore -#[derive(Repository)] -#[firefly(condition_on_property = "lumen.store.postgres=true")] -pub struct PostgresEventStore { /* … */ } // real store when configured - -#[derive(Repository)] -#[firefly(condition_on_missing_bean = "EventStore")] -pub struct MemoryEventStore { /* … */ } // fallback whenever none is wired -``` - -The full conditional family: `condition_on_property = "key=value"`, -`condition_on_class = "label"`, `condition_on_bean = "Type"`, -`condition_on_missing_bean = "Type"`, `condition_on_single_candidate = "Type"`, -plus `profile = "expr"` (which supports a boolean grammar — `prod & cloud`, -`dev | test`, `!staging`, parentheses). - -> **Design note.** The conditional family is `condition_on_property`, -> `condition_on_class`, `condition_on_bean`, `condition_on_missing_bean`, -> `condition_on_single_candidate`, and `profile`. The scan evaluates them in two -> passes: pass-1 facts (config, feature labels) settle first, then pass-2 -> conditions evaluate against the resulting bean registry — which is how *all* -> of Firefly's own auto-configuration backs off when you provide your own. - -## Auto-configuration - -An **auto-configuration** is how a starter contributes sensible defaults that -get out of your way the moment you declare your own. Derive -`#[derive(AutoConfiguration)]` on a holder and guard each `#[bean]` with -`condition_on_missing_bean`: - -```rust,ignore -use firefly::prelude::*; - -#[derive(AutoConfiguration, Default)] -pub struct CacheAutoConfiguration; - -#[firefly::bean] -impl CacheAutoConfiguration { - #[bean(condition_on_missing_bean = "CacheClient", condition_on_property = "cache.type=memory")] - fn cache_client(&self) -> CacheClient { - CacheClient::in_memory() // the default — only if you didn't wire one - } -} -``` - -`#[derive(AutoConfiguration)]` is a `#[derive(Configuration)]` whose beans are -ordered to register **last** during the scan. Because its `#[bean]`s carry -`condition_on_missing_bean`, the two-pass scan registers your unconditional bean -first, then *skips* the auto-configuration default — so your bean always wins, -and you never write an `if`. Drop the starter from your dependencies and the -auto-configuration code isn't linked, so it contributes nothing: discovery is -link-time, not reflective. - -> **Design note.** An auto-configuration is a holder whose `#[bean]`s carry -> `condition_on_missing_bean` and register **last** in the scan: your own bean -> always wins, and you never write an `if`. Discovery is link-time, so an -> auto-configuration is "present" — and contributes its defaults — exactly when -> its crate is compiled in, and contributes nothing once you drop the starter. - -## Component scanning vs. `register_all!` - -`Container::scan()` collects every non-generic stereotype derive's `inventory` -thunk across the whole crate graph, applies conditions and profiles, and -registers the survivors. Drive it through -the `ApplicationContext`, which builds the condition context from the active -profiles and config: +You have used both discovery paths already; here is the contrast in one place. +`Container::scan()` collects every non-generic stereotype derive's scan thunk +across the whole crate graph, applies conditions and profiles, and registers the +survivors. Drive it through the `ApplicationContext`, which builds the condition +context from the active profiles and config: ```rust,ignore use firefly::prelude::*; +use std::sync::Arc; let ctx = ApplicationContext::builder() .profiles(["prod"]) @@ -636,14 +859,15 @@ To scan only part of the crate graph, pass the base module paths: ```rust,ignore let c = Container::new(); -c.scan_packages(&["lumen::domain", "lumen::adapters"]); // these modules only +c.scan_packages(&["lumen::domain", "lumen::ledger"]); // these modules only ``` -A registration matches when its defining module path equals a base package or is -a descendant of it; conditions and profiles are applied exactly as in `scan()`. +A registration matches when its defining module path equals a base package or is a +descendant of it; conditions and profiles are applied exactly as in `scan()`. -Generic beans cannot be inventoried (the monomorphization is chosen at the use -site), so register those with the explicit-list fallback: +**Generic beans** cannot be inventoried (the monomorphization is chosen at the use +site), so register those with the explicit-list fallback — the same +`register_all!` you used in Step 4: ```rust,ignore let c = Container::new(); @@ -651,103 +875,126 @@ firefly::register_all!(&c, [ReadModel, Ledger, WalletApi]); // calls each firef let api = c.resolve::().unwrap(); ``` -> **Design note.** Discovery happens at link time via `inventory`, not at -> runtime via reflection — so a bean is only discoverable if its crate is -> actually linked. `register_all!` is the explicit fallback for generics, which -> can't be inventoried. +> **Design note.** Discovery happens at link time via `inventory`, not at runtime +> via reflection — so a bean is only discoverable if its crate is actually linked. +> `register_all!` is the explicit fallback for generics, which cannot be +> inventoried. -## Introspection — the `/beans` view +### Introspection — the `/beans` view The container is observable. `container.beans()` returns a `BeanDescriptor` per registration (name, type, scope, stereotype, primary, initialized, resolution -count) and `bean_stats()` aggregates counts per stereotype — exactly what the +count), and `bean_stats()` aggregates counts per stereotype — exactly what the admin dashboard's `/beans` page renders, and what `firefly beans --url …` (the -[CLI chapter](./19-cli.md)) prints. Errors -carry diagnostics too: `fuzzy_suggestions(name)` powers the "did you mean…" -hints, and `CircularDependency` is caught with a per-thread resolution stack. +[CLI chapter](./19-cli.md)) prints. Errors carry diagnostics too: +`fuzzy_suggestions(name)` powers the "did you mean…" hints, and +`CircularDependency` is caught with a per-thread resolution stack. + +> **Tip** **Checkpoint.** With Lumen running, open +> `http://localhost:8081/admin/` and find the `/beans` page. `ReadModel` +> (`repository`), `LumenBeans` (`configuration`), `WalletHandlers` / +> `WalletProjection` (`service`), and `WalletApi` (`controller`) are all there — +> the same graph the startup report logged line-by-line. ## What Rust's model changes Firefly's container is deliberately *not* a line-for-line clone of Spring's. The single structural fact behind every difference is that **Rust has no runtime -reflection**: a bean is built by a generated factory closure that resolves its -own dependencies and constructs the struct in one shot, rather than Spring's +reflection**: a bean is built by a generated factory closure that resolves its own +dependencies and constructs the struct in one shot, rather than Spring's "instantiate → populate → call init" phases. There is no post-instantiation seam -to weave behaviour into, and there is no way to hand a bean a reflective handle -to the container. The following Spring features are therefore replaced by Rust -idioms rather than ported — each is a deliberate choice, not a missing feature: +to weave behaviour into, and no way to hand a bean a reflective handle to the +container. The following Spring features are therefore replaced by Rust idioms +rather than ported — each a deliberate choice, not a missing feature: - **No `BeanPostProcessor` / `BeanFactoryPostProcessor`.** There is no - post-instantiation interception phase (the factory closure builds the bean in - one shot), and no definition-rewriting pass. Cross-cutting behaviour is instead - composed explicitly — wrap a collaborator, or use a `#[bean]` factory to build - the wired-up instance you want. -- **No `*Aware` interfaces** (`ApplicationContextAware`, `EnvironmentAware`, …). - A bean is not handed the context by a callback. Autowire what you actually need - — `Arc` for config, `Provider` for a deferred dependency — - rather than reaching back into the container. -- **No `FactoryBean`.** A `#[bean]` factory method (above) produces any type and - is keyed by its return type; that covers the "a bean whose job is to build - another bean" case without a distinct `FactoryBean` abstraction. + post-instantiation interception phase and no definition-rewriting pass. + Cross-cutting behaviour is composed explicitly — wrap a collaborator, or use a + `#[bean]` factory to build the wired-up instance you want. +- **No `*Aware` interfaces** (`ApplicationContextAware`, `EnvironmentAware`, …). A + bean is not handed the context by a callback. Autowire what you actually need — + `Arc` for config, `Provider` for a deferred dependency — rather + than reaching back into the container. +- **No `FactoryBean`.** A `#[bean]` factory method produces any type and is keyed + by its return type; that covers "a bean whose job is to build another bean" + without a distinct `FactoryBean` abstraction. - **No `@Scope(proxyMode)` scoped proxies.** A `request`/`session`-scoped bean is resolved through its `ScopeHandler`, not injected into a singleton via a transparent proxy. To pull a shorter-scoped (or deferred) dependency into a longer-lived bean, hold a `Provider` and call `.get()` at the point of use — - that is the idiomatic substitute for both `@Lazy` proxies and scoped proxies. -- **No per-bean `SmartLifecycle` phases.** The container has `#[post_construct]` / - `#[pre_destroy]` (the `InitializingBean` / `DisposableBean` substitute), and + the idiomatic substitute for both `@Lazy` proxies and scoped proxies. +- **No per-bean `SmartLifecycle` phases.** The container has `post_construct` / + `pre_destroy` (the `InitializingBean` / `DisposableBean` substitute), and application-level start/stop ordering lives in the separate `firefly-lifecycle` crate — there is no per-bean `start`/`stop`/`isRunning`/phase negotiation. -- **No SpEL `#{...}`.** `#[firefly(value = "${...}")]` does placeholder - injection only. Expressions, method/bean references, and arithmetic in config - are intentionally out of scope for the typed-Rust idiom. +- **No SpEL `#{...}`.** `#[firefly(value = "${...}")]` does placeholder injection + only. Expressions, method/bean references, and arithmetic in config are + intentionally out of scope for the typed-Rust idiom. > **Design note.** Read this list as *substitutions*, not gaps: `Provider` -> stands in for `@Lazy`/scoped proxies/`ObjectFactory` when you need to defer or -> reach a shorter-scoped dependency; `#[post_construct]` / `#[pre_destroy]` stand -> in for `InitializingBean` / `DisposableBean`; a `#[bean]` factory stands in for +> stands in for `@Lazy` / scoped proxies / `ObjectFactory` when you need to defer +> or reach a shorter-scoped dependency; `post_construct` / `pre_destroy` stand in +> for `InitializingBean` / `DisposableBean`; a `#[bean]` factory stands in for > `FactoryBean`; and explicit composition stands in for `BeanPostProcessor` > weaving. The reflective machinery is gone, but every job it did has a typed, > compile-checked counterpart. -## What changed in Lumen - -This chapter is reference, not a code step — the wiring it documents is already -how `samples/lumen` works. It told the full DI surface against Lumen's real -collaborators: `ReadModel` as a `#[derive(Repository)]`, `Ledger`/`WalletApi` -autowiring their dependencies, `MemoryEventStore` vs. a hypothetical -`PostgresEventStore` disambiguated by `#[firefly(primary)]` and gated by -`profile` / `condition_on_missing_bean`, the `#[derive(Configuration)]` + -`#[bean]` factories that produce Lumen's beans, lifecycle hooks bracketing a -projection, eager warm-up at `ApplicationContext::build()`, and -`Container::scan()` (the scan `FireflyApplication` runs at boot) / `register_all!` -discovering it all. Between them, Lumen's beans exercise the **full stereotype -set**: `@Configuration` + `@Bean` (`LumenBeans`), `@Service` (`WalletHandlers`, -`WalletProjection`, the streaming `RouteContributor`), `@Repository` (`ReadModel`), -and `@Controller` + `@Autowired` (`WalletApi`). Every attribute shown — -`autowired`, `primary`, `order`, `qualifier`, `scope`, `lazy`, `profile`, the -`condition_on_*` family, `post_construct`, `pre_destroy`, `provides`, `value`, and -`prefix` (on `#[derive(ConfigProperties)]`) — is a real option on the stereotype -derives. +## Recap + +This chapter was a guided tour, not a code-step — the wiring it documents is +already how `samples/lumen` works. You now know how to: + +- Declare a bean with a **stereotype derive** (`Component` / `Service` / + `Repository` / `Configuration` / `Controller`) and read its role off the + `/beans` view. +- Wire dependencies with **`#[autowired]`** constructor injection, and reason + about build order from the field types alone. +- Depend on a **port** and select the **adapter** with `#[firefly(primary)]`, + names, and qualifiers — knowing the four resolution rules cold. +- Produce foreign or constructed beans with **`#[derive(Configuration)]` + + `#[bean]`** factories, including **async** factories that `await` I/O at startup + (drained by `init_async_beans()` / `build_async()`). +- Gate beans by **profile** and the **`condition_on_*`** family, and let an + **auto-configuration** back off the moment you declare your own bean. +- Choose a **scope**, install the **`ScopeHandler`** SPI for request/session/ + refresh scopes, bracket a bean with **lifecycle hooks**, and rely on **eager + fail-fast** startup. +- Discover beans by **component scan** or the **`register_all!`** fallback for + generics, and introspect the whole graph through **`/beans`**. + +Lumen's beans between them exercise the full stereotype set: `@Configuration` + +`@Bean` (`LumenBeans`), `@Service` (`WalletHandlers`, `WalletProjection`, the +streaming `RouteContributor`), `@Repository` (`ReadModel`), and `@Controller` + +`@Autowired` (`WalletApi`). Every attribute shown — `autowired`, `primary`, +`order`, `qualifier`, `scope`, `lazy`, `profile`, the `condition_on_*` family, +`post_construct`, `pre_destroy`, `provides`, `value`, and `prefix` — is a real +option on the stereotype derives. ## Exercises 1. **Resolve the graph by hand.** Take `ReadModel`, `Ledger`, and `WalletApi`, wire them with `register_all!(&c, [ReadModel, Ledger, WalletApi])` and - `resolve::()`, and confirm the container builds the graph in + `c.resolve::()`, and confirm the container builds the graph in dependency order — the same graph `FireflyApplication`'s scan builds at boot. 2. **Default-with-override.** Give `MemoryEventStore` `#[firefly(condition_on_missing_bean = "EventStore")]` and `PostgresEventStore` - `#[firefly(condition_on_property = "lumen.store.postgres=true")]`. Scan with - and without the property set; confirm which store resolves each time. + `#[firefly(condition_on_property = "lumen.store.postgres=true")]`. Scan with and + without the property set; confirm which store resolves each time. 3. **Primary swap.** Bind two adapters to one port without a primary and observe the `NoUniqueBean` error naming both candidates. Add `#[firefly(primary)]` to one and watch resolution succeed — with no change to the consuming bean. -4. **Lifecycle order.** Add `post_construct`/`pre_destroy` hooks to two beans +4. **Lifecycle order.** Add `post_construct` / `pre_destroy` hooks to two beans where one depends on the other; call `ApplicationContext::close()` and confirm the dependent is torn down first. +5. **Read the live graph.** Run Lumen, open `http://localhost:8081/admin/`, and + find the `/beans` page. Map each entry back to the code that declared it — the + `#[bean]` factories in `LumenBeans`, the `#[derive(Repository)]` `ReadModel`, + the `#[derive(Controller)]` `WalletApi` — and note the stereotype label each + carries. + +## Where to go next You now have the full Firefly DI container in hand — the engine that wires every bean in Lumen, from the `#[bean]` factories to the autowired controller. The reactive model underpins everything that follows — continue to -[The Reactive Model](./05-reactive-model.md). +**[The Reactive Model](./05-reactive-model.md)**. diff --git a/docs/book/src/04b-bootstrap.md b/docs/book/src/04b-bootstrap.md index e3e798d..c87efb2 100644 --- a/docs/book/src/04b-bootstrap.md +++ b/docs/book/src/04b-bootstrap.md @@ -1,15 +1,71 @@ # Bootstrapping with FireflyApplication -> By the end of this chapter you will understand the single line that boots -> Lumen: how `FireflyApplication::new("lumen").run()` builds the web stack, -> component-scans the container, auto-configures the CQRS bus, discovers -> security, auto-mounts every `#[rest_controller]`, drains the inventory- -> registered handlers / listeners / scheduled tasks, serves OpenAPI docs and the -> self-hosted admin dashboard, prints a Spring-Boot-style startup report, and -> serves the public + management ports with graceful shutdown — all with **no -> composition root and no bootstrap file**. - -Lumen's `main` is one line: +Lumen's `main` is a single line, and that line is the whole service. In +[Quickstart](./02-quickstart.md) you ran it and saw a banner, a startup report, +and two live ports — but you took `run()` on faith. This chapter opens the lid. +Nothing new gets *added* to Lumen here; instead you will learn exactly what +`FireflyApplication::new("lumen").run().await` does between the moment you press +Enter and the moment the two servers are accepting connections. Knowing the +pipeline pays dividends in every later chapter, because each one declares a bean, +controller, handler, listener, or scheduled task that *one of these stages* +discovers and wires for you. + +By the end of this chapter you will: + +- Explain the difference between `new`, `run`, and `bootstrap`, and know which + one your tests should call. +- Walk the twelve-stage boot pipeline `bootstrap()` runs, and name what each + stage discovers — the web stack, the DI scan, CQRS auto-configuration, + security discovery, controller auto-mounting, handler draining, OpenAPI, and + the management router. +- Use the `FireflyApplication` builder knobs (`version`, `configure`, + `security`, `on_ready`, `extra_routes`, the address overrides) and know when + the *declarative bean* path is preferred over the imperative knob. +- Override the public and management bind addresses from the environment with + `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`. +- Read the line-by-line startup report and use it as a sanity check on what the + framework wired. +- Understand graceful shutdown and the default RFC 9457 404 you get for free. + +## Concepts you will meet + +Before the pipeline walk, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — bootstrap.** *Bootstrapping* is the one-time act of +> assembling a running application from its declarations: building infrastructure, +> discovering components, wiring them together, and producing something +> serveable. In Firefly the entire bootstrap is the body of +> `FireflyApplication::bootstrap()`. The Spring analog is everything +> `SpringApplication.run(...)` does before the embedded server starts accepting +> requests. + +> **Note** **Key term — composition root.** The *composition root* is the single +> place in a program where the object graph is assembled — where every component +> is constructed and connected. Many frameworks make you write this by hand. In +> Firefly the framework *is* the composition root: it scans your beans and wires +> them, so you never spell out the graph in a function. That is why Lumen has no +> `build_app`, no hand-written router, and no `register_*` call site. + +> **Note** **Key term — inventory.** The *inventory* is a set of link-time +> registries the Firefly macros fill at compile time. When you write +> `#[command_handler]`, `#[event_listener]`, `#[scheduled]`, or +> `#[rest_controller]`, the macro registers the item into a global table that the +> framework *drains* at boot. There is no reflection and no runtime scanning of +> the filesystem: the declarations themselves *are* the registration. This is how +> `main` never changes as Lumen grows. + +> **Note** **Key term — management surface.** The *management surface* is the set +> of operational HTTP endpoints — health, info, metrics, configuration +> introspection — plus the self-hosted admin dashboard and the API docs. Firefly +> serves them on a separate port (`8081` by default) from your business API +> (`8080`), so operational endpoints never leak onto the public network. This +> mirrors Spring Boot Actuator. + +## Step 1 — Look at the one line you are about to decode + +Lumen's `main` is the same one line you wrote in the quickstart, living in +`src/main.rs` beside the crate's `mod` declarations: ```rust,ignore // src/main.rs @@ -19,24 +75,55 @@ async fn main() -> Result<(), firefly::BoxError> { } ``` -This is the Rust analog of Spring Boot's `SpringApplication.run(App.class, args)` -and pyfly's `FireflyApplication("lumen").run()`. There is no `build_app`, no -hand-written router, no `register_*`/`subscribe_*`/`schedule_*` call site — the -framework discovers every one of those from the declarations you wrote next to -the code (the beans in [chapter 4](./04a-dependency-injection.md), the -controllers in [chapter 6](./06-first-http-api.md), the handlers in -[chapter 9](./09-cqrs.md), the listeners in -[chapter 10](./10-eda-messaging.md), the scheduled tasks in -[chapter 16](./16-scheduling-notifications.md)). `main` only names the app and -hands control to the framework. - -## `new` / `run` / `bootstrap` - -`FireflyApplication::new(name)` constructs the application. `run().await` boots -and serves until the process receives `SIGINT`/`SIGTERM`. For tests, you do not -want to bind a socket — so `bootstrap().await` does everything `run` does -*except* serve, returning a `Bootstrapped` whose `api_router` you can drive -in-process. Lumen's HTTP tests use exactly that: +What just happened: there is no `build_app`, no hand-written router, and no +`register_*` / `subscribe_*` / `schedule_*` call site anywhere in Lumen. `main` +only names the application and hands control to the framework. Everything else — +the beans from [Dependency Injection](./04a-dependency-injection.md), the +controllers from [Your First HTTP API](./06-first-http-api.md), the handlers from +[CQRS](./09-cqrs.md), the listeners from +[Event-Driven Architecture](./10-eda-messaging.md), and the scheduled tasks from +[Scheduling & Notifications](./16-scheduling-notifications.md) — is discovered by +the framework from declarations sitting next to the code. + +> **Note** **Key term — `BoxError`.** `firefly::BoxError` is the framework's +> boxed error type, `Box`. Returning it from +> `main` lets you use `?` on the bootstrap and lets any startup failure surface as +> a non-zero process exit. It is re-exported from the `firefly` facade, so you +> never name the underlying crate. + +> **Design note.** `FireflyApplication::new(name).run()` is the Rust analog of +> Spring Boot's `SpringApplication.run(App.class, args)` and pyfly's +> `FireflyApplication("lumen").run()`. That single call *is* the composition root. +> Nothing is reflective or hidden: the startup report (Step 10) logs exactly what +> was wired, so "what is running" is printed line-by-line at boot. + +> **Tip** **Checkpoint.** Open `samples/lumen/src/main.rs` (or your own crate's +> `main.rs`). Confirm `main` is one statement: `new("lumen").run().await`. If you +> see a `build_app`, a router, or any `register_*` call, you are reading an older +> shape — the current framework wires all of that for you. + +## Step 2 — Tell `new`, `run`, and `bootstrap` apart + +Three methods drive the lifecycle, and choosing the right one is the difference +between a production server and a fast in-process test. + +> **Note** **Key term — `bootstrap` vs `run`.** `bootstrap()` assembles the +> entire application — every stage in Step 4 — and returns a `Bootstrapped` value +> **without binding a socket or serving**. `run()` calls `bootstrap()` and then +> `serve()`. So the two paths assemble the *identical* app; only the last move +> (bind + serve) differs. + +- **`FireflyApplication::new(name)`** constructs the builder. It reads the + default bind addresses from the environment and seeds the app name. Nothing + happens yet — no scan, no server. +- **`.run().await`** boots and serves until the process receives + `SIGINT`/`SIGTERM`. This is what `main` calls. +- **`.bootstrap().await`** does everything `run` does *except* serve, returning a + `Bootstrapped` whose `api_router` you can drive in-process. This is the test + seam. + +Lumen's HTTP tests use exactly the `bootstrap` path. Here is the real helper from +`src/web.rs` that the test modules call: ```rust,ignore // src/web.rs — the testable in-process router, no socket bound @@ -51,110 +138,198 @@ pub(crate) async fn build_router() -> axum::Router { } ``` -So the production path (`run`) and the test path (`bootstrap` → -`tower::ServiceExt::oneshot`) assemble the **identical** app — the tests in -[chapter 18](./18-testing.md) exercise the same wiring `main` serves. - -## The boot pipeline, step by step - -`bootstrap()` runs a fixed pipeline. Each step ties to something the framework -*discovers and wires* for you: - -1. **Build the web stack.** `WebStack::new(config)` brings up axum, the CQRS - `Bus`, the EDA `Broker`, the `Scheduler`, the metric registry, the health - composite, and the default middleware (correlation id, request metrics, - idempotency, CORS, security headers). Security is *not* applied here — it - comes from a bean after the scan — so the stack is built plain and mutable. - -2. **Initialise logging.** The structured-logging subscriber is installed. When - the `admin` feature is on, logs are also teed into the admin dashboard's - in-memory capture buffer so `/admin` can show a live log tail. - -3. **Component-scan the container.** The framework first **auto-registers its - own infrastructure beans** (`web.register_beans(&container)` — the `Bus`, - `Broker`, `Scheduler`, …), then `container.scan()` discovers, registers, and - autowires **your** beans — every `#[derive(Component/Service/Repository/ - Configuration/Controller)]` and every `#[bean]` factory linked into the - binary. For Lumen that is the `LumenBeans` `#[derive(Configuration)]` and its - `#[bean]` factories (the event store, read model, query cache, JWT service, - `FilterChain`, `BearerLayer`, ledger) plus the `WalletApi` controller bean. - This is the link-time DI of [chapter 4](./04a-dependency-injection.md). - -4. **Auto-configure the CQRS bus.** Correlation propagation is always layered - on. If a `QueryCache` bean is present in the container, its read-cache - middleware is layered on too — so Lumen's `GetWallet` 30s cache - ([chapter 17](./17-caching.md)) is wired with no app code, just by *declaring - the `QueryCache` bean*. (Validation middleware is already installed by the - core.) - -5. **Run the optional readiness hook.** Most apps — Lumen included — need none; - the beans and the pipeline cover the wiring. `on_ready` exists for the rare - case that wants the live collaborators (container, bus, broker, scheduler) - after the scan but before serving. - -6. **Auto-discover security.** With no explicit `.security(...)` call, the - framework resolves the `FilterChain` bean (path-based RBAC) and the - `BearerLayer` bean (token extraction) from the container and applies them — - Spring's `SecurityFilterChain` bean, discovered. Lumen declares both as - `#[bean]`s in `LumenBeans`, so the secured routes from - [chapter 14](./14-security.md) light up automatically. (If an - `ExceptionHandlerRegistry` bean is present, it is installed as the outermost - advice layer — the `@ControllerAdvice` analog.) - -7. **Auto-mount the routes.** `mount_controllers(&container)` resolves every - `#[rest_controller]` and builds its router from the controller's autowired - state bean; `mount_route_contributors(&container)` merges every - `RouteContributor` bean (this is how Lumen's feature-gated streaming endpoint - is added — by declaring a bean, not by editing a composition root). - Resolving the controllers here also constructs their collaborators, including - the `ledger` `#[bean]` that **seeds the event-sourcing projection** on - construction. - -8. **Drain the inventory.** The framework drains the link-time registries the - macros filled at compile time: - `register_discovered_handlers(&bus)` installs every `#[command_handler]` / - `#[query_handler]`, `subscribe_discovered_listeners(broker)` subscribes every - `#[event_listener]`, and `register_discovered_scheduled(&scheduler)` - schedules every `#[scheduled]` task. No `register(&bus)` / - `subscribe(&broker)` call sites — the declarations *are* the registration. - -9. **Apply the middleware chain.** The discovered middleware is applied over the - mounted routes, the bearer-auth layer is added, and (with the `admin` - feature) a W3C trace layer is added that **originates and echoes** - `traceparent` so every request is correlatable across services. - -10. **Serve OpenAPI docs.** The spec is built from the **live inventory** — - every `#[rest_controller]` route plus every `#[derive(Schema)]` DTO — and - served at `/v3/api-docs` (+ `/openapi.json`), with Swagger UI at - `/swagger-ui` and ReDoc at `/redoc` — mounted on the **management** router - (beside actuator + admin), not the public API, since they expose the whole - API surface. This is auto-wired with no app code; - [the next-but-one chapter](./06a-openapi.md) covers it in full. - -11. **Install the default 404.** An unmatched route gets a proper RFC 9457 - `application/problem+json` 404 instead of axum's bare empty body (see - [below](#the-default-404)). - -12. **Build the management router.** The actuator endpoints - (`/actuator/health|info|metrics|loggers|mappings|beans|conditions|env`) are - assembled, and — with the `admin` feature — the **self-hosted admin - dashboard** is mounted at `/admin/`, wired to the live components (health, - metrics, the bus, the scheduler, the container, the environment snapshot, - the trace buffer, the log buffer). [Chapter 15](./15-observability.md) - covers the admin surface. - -`bootstrap` returns the assembled `Bootstrapped`; `run` then calls `serve`. - -## Builder knobs - -`FireflyApplication` is a builder. Every knob is optional; Lumen uses only -`new` (in `main`) and `version` (in `build_router`). The real methods: +What just happened: `bootstrap()` returns a `Bootstrapped`, and `.api_router` is +its fully-assembled public router — controllers, middleware, and security all +applied. A test then drives that router with `tower::ServiceExt::oneshot`, sending +a request straight into the router with no TCP socket involved. Because the +production path (`run`) and the test path (`bootstrap` → `oneshot`) assemble the +**same** app, the tests in [Testing](./18-testing.md) exercise the exact wiring +that `main` serves. + +> **Note** Notice this helper calls `.version(VERSION)` while `main` does not. +> The version is purely cosmetic — it shows up on the banner and in +> `/actuator/info` — so `main` can omit it and let it default. The test sets it +> explicitly only so assertions on `/actuator/info` are stable. + +> **Tip** **Checkpoint.** You can now answer: *which method does a test call, and +> why?* A test calls `bootstrap()` because it wants the wired router without +> binding a port; `main` calls `run()` because it wants to serve. + +## Step 3 — Meet the `Bootstrapped` value + +`bootstrap()` hands back a `Bootstrapped` struct. You rarely construct one +yourself, but knowing its fields demystifies what "assembled" means. The real +shape from the framework: + +```rust,ignore +pub struct Bootstrapped { + /// The web stack (kept so `serve` can run the lifecycle). + pub web: WebStack, + /// The scanned DI container. + pub container: Arc, + /// The fully-assembled public API router (controllers + middleware + security). + pub api_router: Router, + /// The management router (`/actuator/*` + the self-hosted `/admin` dashboard). + pub management_router: Router, + /// The task scheduler (started by `serve`). + pub scheduler: Arc, + /// The public bind address. + pub api_addr: String, + /// The management bind address. + pub management_addr: String, +} +``` + +What just happened: a `Bootstrapped` carries both routers (public + management), +the scanned DI container, the scheduler that has not started yet, and the two +addresses to bind. `run()`'s only remaining job is to call `serve()` on this +value, which starts the scheduler and binds both routers. A test ignores +everything except `api_router`. + +> **Note** **Key term — DI container.** The *container* is the registry that +> holds every bean the framework constructed, keyed by type, so any component can +> ask for a collaborator by type and get the managed instance. It is the runtime +> half of the dependency injection you met in +> [Dependency Injection](./04a-dependency-injection.md). `Bootstrapped.container` +> is that registry, fully scanned. + +## Step 4 — Walk the boot pipeline, stage by stage + +This is the heart of the chapter. `bootstrap()` runs a fixed pipeline, and every +stage ties to something the framework *discovers and wires* for you. Read it once +top to bottom; you will return to individual stages as later chapters add the +beans each stage finds. The numbering below follows the framework source +(`crates/firefly/src/application.rs`). + +**1. Build the web stack.** `WebStack::new(config)` brings up axum, the CQRS +`Bus`, the EDA `Broker`, the `Scheduler`, the metric registry, the health +composite, and the default middleware (correlation id, request metrics, +idempotency, CORS, security headers). Security is *not* applied here — it comes +from a bean after the scan — so the stack is built plain and mutable. + +> **Note** **Key term — bus / broker / scheduler.** The *bus* routes CQRS +> (Command/Query Responsibility Segregation) commands and queries to their +> handlers; the *broker* delivers events to listeners; the *scheduler* runs +> `#[scheduled]` tasks on a timer. All three are framework infrastructure beans, +> constructed here and registered into the container in stage 3 so your code can +> autowire them. + +**2. Initialise logging.** The structured-logging subscriber is installed. When +the `admin` feature is on, logs are also teed into the admin dashboard's +in-memory capture buffer so `/admin` can show a live log tail. + +**3. Component-scan the container.** The framework first **auto-registers its own +infrastructure beans** (`web.register_beans(&container)` — the `Bus`, `Broker`, +`Scheduler`, the registries), then `container.scan()` discovers, registers, and +autowires **your** beans — every `#[derive(Component/Service/Repository/ +Configuration/Controller)]` and every `#[bean]` factory linked into the binary. +Immediately after the synchronous scan, `container.init_async_beans().await` +awaits every `async fn` `#[bean]` factory (a DB pool, a broker dial) so async +beans are live before anything resolves them — and a construction error aborts +startup (fail-fast). For Lumen that scan finds the `LumenBeans` +`#[derive(Configuration)]` and its `#[bean]` factories (the event store, query +cache, JWT service, `FilterChain`, `BearerLayer`, ledger) plus the `WalletApi` +controller bean. This is the link-time DI of +[Dependency Injection](./04a-dependency-injection.md). + +**4. Auto-configure the CQRS bus.** Correlation propagation is always layered on +(`bus.use_middleware(CorrelationMiddleware::new())`). If a `QueryCache` bean is +present in the container, its read-cache middleware is layered on too — so +Lumen's `GetWallet` 30-second cache ([Caching](./17-caching.md)) is wired with no +app code, just by *declaring the `QueryCache` bean*. Validation middleware is +already installed by the core. + +**5. Run the optional readiness hook.** Most apps — Lumen included — need none; +the beans and the pipeline cover the wiring. `on_ready` exists for the rare case +that wants the live collaborators (container, bus, broker, scheduler) after the +scan but before serving. We cover it in Step 5 below. + +**6. Auto-discover security.** With no explicit `.security(...)` call, the +framework resolves the `FilterChain` bean (path-based RBAC) and the `BearerLayer` +bean (token extraction) from the container and applies them — Spring's +`SecurityFilterChain` bean, discovered. Lumen declares both as `#[bean]`s in +`LumenBeans`, so the secured routes from [Security](./14-security.md) light up +automatically. (If an `ExceptionHandlerRegistry` bean is present, it is installed +as the outermost advice layer — the `@ControllerAdvice` analog.) + +**7. Auto-mount the routes.** `mount_controllers(&container)` resolves every +`#[rest_controller]` and builds its router from the controller's autowired state +bean; `mount_route_contributors(&container)` merges every `RouteContributor` +bean. This is how Lumen's feature-gated streaming endpoint is added — by +declaring a bean, not by editing a composition root. Resolving the controllers +here also constructs their collaborators, including the `ledger` `#[bean]`. + +> **Note** **Key term — `RouteContributor`.** A `RouteContributor` is a bean that +> contributes raw axum routes the framework merges into the public router. It is +> the escape hatch for endpoints that do not fit the `#[rest_controller]` shape — +> like Lumen's reactive event stream. You declare it as a bean +> (`#[firefly(provides = "dyn firefly::web::RouteContributor")]`) and the +> framework finds it; there is still no composition root to edit. + +**8. Drain the inventory.** The framework drains the link-time registries the +macros filled at compile time. `register_discovered_handlers(&bus)` plus +`register_discovered_handler_beans(&bus, &container)` install every +`#[command_handler]` / `#[query_handler]`; +`subscribe_discovered_listeners(broker)` plus the bean variant subscribe every +`#[event_listener]`; and `register_discovered_scheduled(&scheduler)` plus the +bean variant schedule every `#[scheduled]` task. There are no `register(&bus)` / +`subscribe(&broker)` call sites — the declarations *are* the registration. + +**9. Apply the middleware chain.** The discovered middleware is applied over the +mounted routes, the bearer-auth layer is added, the default 404 fallback is set, +and `web.apply_middleware(...)` wraps the whole router in the inherited +observability edge — idempotency, the access log, request metrics, correlation, +W3C trace, security headers, problem rendering, CORS, and the global exception +advice. With the `admin` feature, an outermost trace layer **originates and +echoes** `traceparent` so every request is correlatable across services. + +**10. Serve OpenAPI docs.** The spec is built from the **live inventory** — every +`#[rest_controller]` route plus every `#[derive(Schema)]` DTO — and served at +`/v3/api-docs` (plus `/openapi.json`), with Swagger UI at `/swagger-ui` and ReDoc +at `/redoc`. These are mounted on the **management** router (beside actuator and +admin), *not* the public API, since they expose the whole API surface. This is +auto-wired with no app code; [OpenAPI, Swagger UI & ReDoc](./06a-openapi.md) +covers it in full. + +> **Note** The OpenAPI spec advertises the *public* API base URL as its `server` +> even though the docs are served on the management port — so Swagger UI's "Try +> it out" sends requests to `8080`, not the `8081` origin it loaded from. +> `FIREFLY_OPENAPI_SERVER_URL` overrides that base URL (for example, a public URL +> behind a reverse proxy). + +**11. Install the default 404.** An unmatched route gets a proper RFC 9457 +`application/problem+json` 404 instead of axum's bare empty body (see +[Step 8](#step-8--understand-the-default-404)). + +**12. Build the management router.** The actuator endpoints +(`/actuator/health|info|metrics|loggers|mappings|beans|conditions|env`) are +assembled, and — with the `admin` feature — the **self-hosted admin dashboard** +is mounted at `/admin/`, wired to the live components (health, metrics, the bus, +the scheduler, the container, the environment snapshot, the trace buffer, the log +buffer). The OpenAPI docs router from stage 10 is merged in here, and a single +RFC 9457 404 fallback is set for the whole management surface. +[Observability](./15-observability.md) covers the admin surface in depth. + +`bootstrap()` returns the assembled `Bootstrapped`; `run()` then calls `serve()`. + +> **Tip** **Checkpoint.** Without re-reading, name what stage discovers a CQRS +> command handler (stage 8 — draining the inventory), a controller (stage 7 — +> auto-mounting), a security filter chain (stage 6 — security discovery), and the +> `GetWallet` read cache (stage 4 — CQRS auto-configuration, because a +> `QueryCache` bean is present). If you can place each, you understand why `main` +> never changes as Lumen grows. + +## Step 5 — Reach for a builder knob (only when a bean will not do) + +`FireflyApplication` is a builder, and every knob is optional. Lumen uses only +`new` (in `main`) and `version` (in `build_router`). Here is the full set, drawn +from the framework source so the signatures are exact: | Method | What it does | |--------|--------------| | `new(name)` | Names the app (banner + `/actuator/info`). Defaults the binds from `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`. | | `version(v)` | Sets the version (banner + `/actuator/info`). | -| `configure(\|cfg\| { … })` | Tunes the `CoreConfig` in place — CORS, security headers, idempotency, the knobs of [chapter 3](./03-configuration.md). | +| `configure(\|cfg\| { … })` | Tunes the `CoreConfig` in place — CORS, security headers, idempotency, the knobs of [Configuration](./03-configuration.md). | | `security(chain, bearer)` | Installs a `FilterChain` + `BearerLayer` **explicitly**, instead of discovering them from beans. | | `on_ready(\|ctx\| async { … })` | A readiness hook over the live `container` / `bus` / `broker` / `scheduler`, run after the scan, before serving. | | `extra_routes(\|container\| router)` | Merges extra non-`#[rest_controller]` routes built from the scanned container. | @@ -164,6 +339,9 @@ So the production path (`run`) and the test path (`bootstrap` → | `bootstrap()` | Assembles the app **without serving** (tests). | | `run()` | Bootstraps and serves. | +The knobs are chainable. A hypothetical `orders` service that wants a touch of +imperative wiring might write: + ```rust,ignore // the knobs are chainable; Lumen needs almost none of them firefly::FireflyApplication::new("orders") @@ -174,18 +352,28 @@ firefly::FireflyApplication::new("orders") .await ``` +What just happened: each knob returns `Self`, so you chain as many as you need +and finish with `run()` (or `bootstrap()`). Most services finish with a much +shorter chain than this; Lumen's is the empty chain, `new("lumen").run()`. + > **Design note.** Lumen declares security as `#[bean]`s rather than calling -> `.security(...)`, declares its streaming endpoint as a `RouteContributor` -> bean rather than calling `.extra_routes(...)`, and seeds its projection inside -> the `ledger` `#[bean]` rather than in `.on_ready(...)`. The explicit builder -> knobs exist for apps that prefer a touch of imperative wiring; the *bean* -> route is the framework's preferred, fully-declarative path — declaration next -> to the code. +> `.security(...)`, declares its streaming endpoint as a `RouteContributor` bean +> rather than calling `.extra_routes(...)`, and seeds its projection inside the +> `ledger`/projection beans rather than in `.on_ready(...)`. The explicit builder +> knobs exist for apps that prefer a touch of imperative wiring; the *bean* route +> is the framework's preferred, fully-declarative path — declaration next to the +> code, discovered at boot. Prefer a bean; reach for a knob only when no bean +> shape fits. + +> **Tip** **Checkpoint.** You can now justify why Lumen's `main` has no builder +> knobs at all: everything a knob could do, Lumen does with a bean the scan +> finds. -## Env-overridable binds +## Step 6 — Override the bind addresses from the environment By default the public API binds `0.0.0.0:8080` and the management surface binds -`0.0.0.0:8081`. Override either without touching code: +`0.0.0.0:8081`. You can move either without touching code, because `new` reads +two environment variables at construction time: ```bash FIREFLY_SERVER_ADDR=127.0.0.1:9090 \ @@ -193,15 +381,24 @@ FIREFLY_MANAGEMENT_ADDR=127.0.0.1:9091 \ cargo run --bin lumen ``` -`new` reads `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` at construction; -`api_addr(...)` / `management_addr(...)` override them in code if you need to. +What just happened: `new` read `FIREFLY_SERVER_ADDR` for the public bind and +`FIREFLY_MANAGEMENT_ADDR` for the management bind, each falling back to its +`0.0.0.0:808x` default when unset. The two surfaces move *independently* — proof +that they are genuinely separate listeners, not one server with a path prefix. +If you would rather set the addresses in code, `api_addr(...)` / +`management_addr(...)` override the environment. -## The startup report +> **Tip** **Checkpoint.** Start Lumen with the two overrides above, then in a +> second terminal run `curl localhost:9091/actuator/health`. A `{"status":"UP"}` +> from `:9091` (and nothing on `:9090`'s `/actuator/*`) confirms the management +> surface moved on its own. This is the first taste of the typed configuration +> story in [Configuration](./03-configuration.md). -Just before serving, `serve` prints the banner, the docs URLs, and then -`log_startup_report(&container)` — the Spring-Boot/pyfly-style **line-by-line -report** so a boot log reads like Spring Boot's console: you can see exactly -what the framework wired. The format is: +## Step 7 — Read the startup report + +Just before serving, `serve()` prints the banner, the docs URLs, and then +`log_startup_report(&container)` — a Spring-Boot/pyfly-style **line-by-line +report** so a boot log reads like Spring Boot's console. The format is: ```text :: active profiles :: default @@ -219,39 +416,39 @@ Reading it top to bottom: - **`:: active profiles ::`** — the active config profiles (`default` when none is set). -- **`:: beans (N) ::`** — every bean the container scanned, one per line: +- **`:: beans (N) ::`** — every bean the container scanned, one per line: the `[stereotype]` (`service`, `repository`, `controller`, `configuration`, `component`, or `bean`), the bean name, its scope, and its short type name. This is the same table the admin dashboard's `/beans` view renders. -- **`:: routes (N) ::`** — the auto-mounted route table: each - `#[rest_controller]` route as `METHOD path -> Controller::handler`. This is - drawn from the same `firefly_container::routes()` registry that feeds - `/admin/api/mappings` and the OpenAPI document, so the three never drift. -- **`:: cqrs handlers … ::`** — the *counts* drained from the inventory: how - many `#[command_handler]`/`#[query_handler]`, `#[event_listener]`, - `#[scheduled]`, and controllers were discovered. -- **`:: openapi ::`** — the operation count (one per route) and the component- - schema count (one per `#[derive(Schema)]` DTO), confirming the spec is live. - -Nothing in your app prints this — the framework does. The numbers are a quick -sanity check: if you expected four handlers and the report says three, a -`#[command_handler]` is missing or its crate is not linked. - -## Graceful shutdown - -`serve` starts the scheduler on a background task, then serves the public API on -`api_addr` and the management surface on `management_addr` through the framework -lifecycle, each with `with_graceful_shutdown`. On `SIGINT`/`SIGTERM` both -servers stop accepting connections, let in-flight requests finish, and `run` -returns `Ok(())` — a signal-triggered stop is a clean shutdown, not an error. - -## The default 404 - -Because the framework installs a fallback, an unmatched path returns a proper -RFC 9457 `application/problem+json` 404 — the same `type`/`title`/`status` -envelope and `application/problem+json` content type as every other framework -error — instead of axum's bare empty body (which a browser would offer to -download as a blank file): +- **`:: routes (N) ::`** — the auto-mounted route table: each `#[rest_controller]` + route as `METHOD path -> Controller::handler`. It is drawn from the same + `firefly_container::routes()` registry that feeds `/admin/api/mappings` and the + OpenAPI document, so the three never drift. +- **`:: cqrs handlers … ::`** — the *counts* drained from the inventory: how many + `#[command_handler]`/`#[query_handler]`, `#[event_listener]`, `#[scheduled]`, + and controllers were discovered (each count sums the free-`fn` and the bean + registrations). +- **`:: openapi ::`** — the operation count (one per route) and the + component-schema count (one per `#[derive(Schema)]` DTO), confirming the spec is + live. + +What just happened: nothing in your app printed this — the framework did, from +the live container and inventory. The numbers are a quick sanity check: if you +expected four handlers and the report says three, a `#[command_handler]` is +missing or its crate is not linked. + +> **Tip** **Checkpoint.** Run `cargo run` and read the report. Note how short the +> `beans`, `routes`, and counts lines are today — Lumen has little business logic +> yet — then revisit this report after [CQRS](./09-cqrs.md) and watch the numbers +> grow without a single edit to `main`. + +## Step 8 — Understand the default 404 + +Because the framework installs a fallback on both routers (stages 11 and 12), an +unmatched path returns a proper RFC 9457 `application/problem+json` 404 — the same +`type`/`title`/`status` envelope and `application/problem+json` content type as +every other framework error — instead of axum's bare empty body (which a browser +would offer to download as a blank file): ```text GET /api/v1/nope @@ -262,17 +459,96 @@ content-type: application/problem+json "detail": "No route matches GET /api/v1/nope" } ``` -This is the same problem-rendering you met for handler errors in -[chapter 6](./06-first-http-api.md) and security errors in -[chapter 14](./14-security.md) — uniform errors, end to end, with no per-route -work. +What just happened: the fallback is wired *inside* the observability edge, so even +an unmatched-route 404 is logged, traced, and correlated — there is no +observability gap for "the path that did not exist." This is the same +problem-rendering you meet for handler errors in +[Your First HTTP API](./06-first-http-api.md) and security errors in +[Security](./14-security.md) — uniform errors, end to end, with no per-route work. -## What changed in Lumen +> **Note** **Key term — RFC 9457.** RFC 9457 (which obsoletes RFC 7807) defines +> the `application/problem+json` media type: a small, machine-readable error +> envelope with `type`, `title`, `status`, and `detail` fields. Firefly renders +> *every* error — handler failures, validation, security, and unmatched routes — +> through this one shape, so a client parses errors exactly the same way no matter +> where they came from. -Nothing was *added* here — this chapter explains the line that was already in -`main.rs` since the start. `FireflyApplication` is the spine the rest of the -book hangs on: every chapter that declares a bean, a controller, a handler, a -listener, or a scheduled task is contributing to the pipeline above, and this -chapter is where you see how the framework finds and wires it all from a single -line. Next, [Your First HTTP API](./06-first-http-api.md) writes the -controllers the framework auto-mounts. +## Step 9 — Understand graceful shutdown + +`serve()` starts the scheduler on a background task, then serves the public API on +`api_addr` and the management surface on `management_addr` through the framework +lifecycle, each wrapped with `with_graceful_shutdown`. On `SIGINT`/`SIGTERM` both +servers stop accepting new connections, let in-flight requests finish, and `run()` +returns `Ok(())`. A signal-triggered stop is treated as a *clean shutdown, not an +error* — the lifecycle's cancelled-error case is mapped to `Ok(())`. + +What just happened: you never wrote a signal handler. The framework traps the +signal, drains both ports, and returns success, so a `Ctrl-C` at your terminal +exits with no stack trace and a zero exit code. That is the behaviour a container +orchestrator (Kubernetes sending `SIGTERM`) relies on for a rolling restart. + +> **Tip** **Checkpoint.** Run Lumen, then press `Ctrl-C`. The process exits +> cleanly with no panic and no stack trace. If you saw an error, you are on an +> older build — the current `serve()` maps the cancellation to `Ok(())`. + +## Recap + +This chapter added no code to Lumen — it decoded the line that has been in +`main.rs` since the quickstart. You now know: + +- **`new` / `run` / `bootstrap`.** `new` builds the builder; `run` bootstraps and + serves; `bootstrap` assembles the identical app *without* serving and returns a + `Bootstrapped` whose `api_router` your tests drive in-process. +- **The twelve-stage pipeline.** Build the web stack, init logging, component-scan + the container (awaiting async beans), auto-configure the CQRS bus, run the + optional readiness hook, auto-discover security, auto-mount controllers and + route contributors, drain the inventory (handlers / listeners / scheduled + tasks), apply the middleware chain, serve OpenAPI on the management port, install + the default 404, and build the management router with actuator + admin. +- **Builder knobs vs beans.** `version`, `configure`, `security`, `on_ready`, + `extra_routes`, `info_contributor`, and the address overrides exist for + imperative wiring — but Lumen prefers the declarative bean for every one of + them, so its `main` is the empty chain. +- **The operational defaults.** Two independent ports overridable by + `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`, a line-by-line startup report, + an RFC 9457 404 for unmatched paths, and graceful SIGINT/SIGTERM shutdown — all + for free. + +`FireflyApplication` is the spine the rest of the book hangs on. Every chapter +that declares a bean, a controller, a handler, a listener, or a scheduled task is +contributing to the pipeline above — and never by rewriting `main`, only by giving +the framework one more thing to discover. + +## Exercises + +1. **Trace a stage to a chapter.** For each of stages 4, 6, 7, and 8, name the + later chapter that adds the bean or declaration that stage discovers, and the + single line of Lumen code that makes it light up. (Hint: stage 4 is the + `QueryCache` bean from [Caching](./17-caching.md).) +2. **Drive the test seam.** In `samples/lumen`, read `src/http_test.rs` and find + where it calls `build_router()`. Confirm the test never binds a socket — it + drives the `bootstrap()`-assembled router directly. Then explain why a passing + HTTP test proves something about the *production* `run()` path. +3. **Move the ports independently.** Start Lumen with + `FIREFLY_SERVER_ADDR=127.0.0.1:9090 FIREFLY_MANAGEMENT_ADDR=127.0.0.1:9091 + cargo run`, then `curl localhost:9091/actuator/health` and + `curl localhost:9090/api/v1/wallets/none`. Confirm health answers on `:9091` + and the public 404 (RFC 9457 problem+json) answers on `:9090`. +4. **Read the startup report as a checklist.** Run Lumen and copy the + `:: cqrs handlers … ::` and `:: routes … ::` lines. After you finish + [CQRS](./09-cqrs.md), run it again and diff the two — every new number should + map to a `#[command_handler]`, `#[query_handler]`, or `#[rest_controller]` you + added, with `main` untouched. +5. **Provoke graceful shutdown.** Run Lumen, fire a slow request, and press + `Ctrl-C` mid-flight. Confirm the in-flight request still completes and the + process exits with code `0` and no stack trace — the signal was a shutdown, not + a failure. + +## Where to go next + +- See the beans this pipeline scans declared in + **[Dependency Injection & Auto-Configuration](./04a-dependency-injection.md)**. +- Write the `#[rest_controller]` that stage 7 auto-mounts in + **[Your First HTTP API](./06-first-http-api.md)**. +- Watch the OpenAPI spec stage 10 builds come to life in + **[OpenAPI, Swagger UI & ReDoc](./06a-openapi.md)**. diff --git a/docs/book/src/05-reactive-model.md b/docs/book/src/05-reactive-model.md index f9e41c9..c5cc887 100644 --- a/docs/book/src/05-reactive-model.md +++ b/docs/book/src/05-reactive-model.md @@ -1,27 +1,63 @@ # The Reactive Model — Mono & Flux -This is the keystone chapter. `firefly-reactive` is Firefly's -production-grade **reactive core**: two lazy, composable, backpressure-aware -publishers — `Mono` (0-or-1 value) and `Flux` (0..N values) — built natively on -tokio. Every reactive surface in the framework — reactive HTTP endpoints, -reactive repositories, the reactive `WebClient`, reactive EDA and CQRS — is -built on the two types you will learn here. Read this before the -service-building chapters. - -If you've used a reactive-streams library before, the `Mono` / `Flux` shapes -will feel familiar; everything below is Firefly's own API. - -> **By the end of this chapter, Lumen will** have the vocabulary it leans on -> twice over: the `Flux` that backs its NDJSON / SSE -> *stream-a-wallet's-events* endpoint (turned on in -> [Production & Deployment](./20-production.md)), and the lazy `Mono` that -> `Bus::send_mono` / `Bus::query_mono` return so a wallet command can be composed -> into a reactive pipeline. No Lumen file lands yet — but every reactive shape in -> the chapters ahead is one of the two publishers below. - -## Mono and Flux - -Two publishers: +This is the keystone chapter. `firefly-reactive` is Firefly's production-grade +**reactive core**: two lazy, composable, backpressure-aware publishers — `Mono` +(0-or-1 value) and `Flux` (0..N values) — built natively on Tokio. Every +reactive surface in the framework is built from these two types: reactive HTTP +endpoints, reactive repositories, the reactive `WebClient`, and the reactive +faces of EDA and CQRS. Nothing here requires infrastructure — every example runs +in-process — so you can type each one into a scratch test and watch it pass. + +No Lumen source file lands in this chapter. Instead you build the *vocabulary* +Lumen leans on twice over later: the `Flux` behind its NDJSON / SSE +*stream-a-wallet's-events* endpoint (turned on in +[Production & Deployment](./20-production.md)), and the lazy `Mono` that +`Bus::send_mono` / `Bus::query_mono` return so a wallet command composes into a +reactive pipeline. Read this before the service-building chapters; everything +after it assumes you can read a `Mono`/`Flux` pipeline at a glance. + +By the end of this chapter you will: + +- Explain what a *reactive publisher* is, why `Mono` and `Flux` are **lazy**, and + why their error channel is fixed to one type. +- Build, transform, combine, and run pipelines over both publishers, and read the + `Result, _>` that `.block().await` returns. +- Recover from errors with `on_error_*` / `retry_backoff`, and push values + imperatively into a `Flux` with a `FluxSink`. +- Move work between threads with a `Scheduler` (`subscribe_on` / `publish_on`). +- Turn a `Mono`/`Flux` into an HTTP response with Firefly's reactive responders + (`MonoJson`, `NdJson`, `Sse`), and trace how Lumen's streaming endpoint uses + them. +- See how the same two types thread through the reactive `WebClient`, + repositories, EDA, and the CQRS bus. + +## Concepts you will meet + +Before the first pipeline, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — reactive publisher.** A *publisher* is a value that +> *describes* a computation producing data over time, without running it yet. You +> chain operators onto it to build a pipeline, then *subscribe* to make it run. +> The Spring analog is a Project Reactor `Publisher` (`Mono` / `Flux`), the engine +> behind Spring WebFlux. Firefly's `Mono` and `Flux` are the Rust spelling of +> exactly those. + +> **Note** **Key term — lazy.** A publisher is *lazy* when building the pipeline +> does no work; the work runs only when you subscribe, block, or await. This is +> the opposite of an eagerly-executed `Future` that starts the moment it is +> created in some runtimes — a `Mono` you never subscribe to never runs. + +> **Note** **Key term — backpressure.** *Backpressure* is the mechanism by which +> a slow consumer throttles a fast producer so data does not pile up in memory. A +> `Flux` honors backpressure end to end: a slow HTTP client streaming an +> `NdJson` body actually slows the producer feeding it, rather than buffering the +> whole stream up front. + +## Step 1 — Meet the two publishers + +Firefly's reactive core is two types, distinguished by **cardinality** — how many +values they can emit: - **`Mono`** — a producer of *at most one* value (0-or-1, plus a terminal error). The reactive analog of "an async function that returns a `T`." @@ -30,11 +66,19 @@ Two publishers: Both are **lazy**: building a pipeline does nothing; work runs only when you subscribe, block, or await. Both are `Send + 'static`, so a `Mono` or `Flux` -drops directly into an axum handler. +drops directly into an axum handler with no wrapping. + +> **Note** **Key term — terminal signal.** A pipeline ends with exactly one +> *terminal signal*: a `Flux` completes after its last value (or with no values), +> and either publisher can end early with an **error**. In `firefly-reactive` the +> error type is fixed to `firefly_kernel::FireflyError`. Fixing the error keeps +> the operator surface ergonomic — there is no error type parameter to thread +> through every `map` — and it wires straight into the framework's RFC 9457 +> problem responses, so a failed pipeline becomes an `application/problem+json` +> body for free. -The error type is fixed to `firefly_kernel::FireflyError`. Fixing the error -keeps the operator surface ergonomic (no error type parameter) and wires -straight into the framework's RFC 9457 problem responses. +Type the following into a test (`#[tokio::test] async fn`) to see both shapes run +to completion: ```rust use firefly_reactive::{Flux, Mono}; @@ -63,47 +107,59 @@ assert_eq!(xs, vec![10, 30, 50]); # } ``` -> **Warning.** `Mono::block()` is `async`: it never parks a Tokio worker. -> It resolves the publisher in place and returns `Result, FireflyError>`, -> so `.block().await` is the idiomatic way to run a pipeline to completion. -> Despite the name, it does *not* park a thread the way a blocking call would. - -## Core concepts at a glance - -The terms you will meet throughout the chapter, in one place: - -| firefly-reactive | What it is | -|-------------------------------------------------|-----------------------------------------------------| -| `Mono` | a producer of 0-or-1 value plus a terminal error | -| `Flux` | a producer of 0..N values plus a terminal signal | -| `firefly_kernel::FireflyError` (fixed) | the single, fixed error signal | -| `Ok(None)` from a `Mono` | the empty / complete-with-no-value outcome | -| `Err(FireflyError)` (terminal) | an error signal — short-circuits the pipeline | -| `Mono::block` (async — never parks a thread) | resolve a publisher in place, then `.await` it | -| `Mono::subscribe` / `Flux::subscribe` | drive a pipeline with explicit callbacks | -| `Scheduler::Immediate` | run work inline on the current task | -| `Scheduler::Parallel` | run work on the Tokio worker pool (CPU-bound) | -| `Scheduler::BoundedElastic` | run blocking work without starving the worker pool | -| `subscribe_on` / `publish_on` | choose *where* the source / downstream runs | -| `FluxSink` / `Flux::create` | push values imperatively into a `Flux` | -| `Backoff` + `*::retry_backoff` | re-subscribe on error with a backoff schedule | -| `Mono::into_future` / `.await` | bridge a `Mono` out to a plain `Future` | -| `Flux::to_stream` / `Flux::into_stream` | bridge a `Flux` out to a plain `Stream` | - -## Creating publishers +What just happened, block by block: + +- The `Mono` pipeline starts with `Mono::just(20)`, then `map`s, `filter`s, and + supplies a `default_if_empty(0)` in case the filter rejected the value. None of + that ran until `.block().await`. The result is `Some(21)`: one value survived. +- The `Flux` pipeline ranges over `1..=5`, keeps the odd numbers, multiplies each + by ten, and `collect_list` folds the whole stream into a single `Vec`. Because + `collect_list` returns a `Mono>`, running it yields `Ok(Some(vec))`. + +> **Warning** `Mono::block()` is `async`: despite the name it never parks a Tokio +> worker. It resolves the publisher in place and returns +> `Result, FireflyError>`, so `.block().await` is the idiomatic way to +> run a pipeline to completion. The two layers it returns are deliberate — the +> outer `Result` is success-or-error, the inner `Option` is value-or-empty. + +> **Tip** **Checkpoint.** Drop both snippets into a `#[tokio::test]` and run +> `cargo test`. The two `assert_eq!`s pass: `n == Some(21)` and +> `xs == vec![10, 30, 50]`. You have run your first lazy pipelines — and you have +> seen the `Result, _>` shape `.block().await` always returns. + +### Reading the return type + +Everything that runs a `Mono` to completion returns `Result, FireflyError>`. +The three layers each carry one fact, and reading them is a skill you will use in +every later chapter: + +| Outcome | What it means | +|------------------------|----------------------------------------------------------| +| `Ok(Some(v))` | the pipeline produced the value `v` | +| `Ok(None)` | the pipeline completed **empty** (`Mono::empty`, a `filter` that rejected everything) | +| `Err(FireflyError)` | the pipeline hit a **terminal error** and short-circuited | + +A `Flux` terminal operator (`collect_list`, `reduce`, `count`, …) returns a +`Mono`, so it follows the same rule — which is why `collect_list().block().await` +unwraps twice in the example above. + +## Step 2 — Create publishers + +A pipeline starts at a *constructor*. You will reach for a handful constantly; +the rest are there when an edge case needs them. `Mono` constructors: -| Constructor | Produces | -|-----------------------------------|------------------------------------------------| -| `Mono::just(v)` | exactly `v` | -| `Mono::just_or_empty(opt)` | `v` if `Some`, empty if `None` | -| `Mono::empty()` | completes with no value (`Ok(None)`) | -| `Mono::error(e)` | terminal error | -| `Mono::from_future(fut)` | awaits a `Future` | -| `Mono::from_result_future(fut)` | awaits a `Future>` | -| `Mono::from_callable(f)` | runs a `FnOnce() -> Result, _>` on subscribe | -| `Mono::defer(factory)` | builds the `Mono` fresh per subscription | +| Constructor | Produces | +|-----------------------------------|---------------------------------------------------------| +| `Mono::just(v)` | exactly `v` | +| `Mono::just_or_empty(opt)` | `v` if `Some`, empty if `None` | +| `Mono::empty()` | completes with no value (`Ok(None)`) | +| `Mono::error(e)` | a terminal error | +| `Mono::from_future(fut)` | awaits a `Future` | +| `Mono::from_result_future(fut)` | awaits a `Future>` | +| `Mono::from_callable(f)` | runs a `FnOnce() -> Result, FireflyError>` on subscribe | +| `Mono::defer(factory)` | builds the `Mono` fresh per subscription | `Flux` constructors: @@ -113,17 +169,30 @@ The terms you will meet throughout the chapter, in one place: | `Flux::from_iter(iter)` | each element of an iterator | | `Flux::range(start, count)` | `start, start+1, …` (count items) | | `Flux::empty()` / `Flux::never()` | completes immediately / never emits | -| `Flux::error(e)` | terminal error | +| `Flux::error(e)` | a terminal error | | `Flux::from_stream(s)` | a `Stream>` | | `Flux::from_value_stream(s)` | a `Stream` | -| `Flux::create(producer)` | imperative push via a `FluxSink` (see below) | -| `Flux::interval(period)` | `0, 1, 2, …` on a timer | +| `Flux::create(producer)` | imperative push via a `FluxSink` (Step 5) | +| `Flux::interval(period)` | `0, 1, 2, …` on a timer | | `Flux::generate(seed, step)` | stateful generation | -## Operator quick reference +What just happened: `Mono::just` / `Flux::just` are the literal constructors you +will use most. `from_future` / `from_result_future` are the bridge from `async` +Rust into the reactive world — the same bridge the CQRS bus uses internally to +wrap a dispatch into a `Mono`. `defer` and `from_callable` matter for **retry**, +because they build the work *fresh on each subscription* (Step 4). + +> **Note** **Key term — cold publisher.** All of these are *cold*: the work is +> redone for each subscriber, starting at subscribe time, like calling a function +> again. (The opposite, a *hot* publisher, shares one running source among +> subscribers — `Mono::cache` turns a cold `Mono` into one that remembers its +> result.) Cold-by-default is what makes `retry` possible: a retry is just another +> subscription. -`Mono` and `Flux` share most operator names; the differences reflect -cardinality. +## Step 3 — Transform, combine, and terminate + +`Mono` and `Flux` share most operator names; the differences reflect cardinality. +This is the working set — keep it nearby, you will not memorize it in one read: | Category | Mono | Flux | |-------------|----------------------------------------------------------------------|--------------------------------------------------------------------------------------------| @@ -139,13 +208,16 @@ cardinality. | schedule | `subscribe_on` `publish_on` | `subscribe_on` `publish_on` | | cache/view | `cache` `as_flux` | — | -### Transforming and chaining +The one distinction worth internalizing now is `map` versus `flat_map`. `map` +transforms each value with a plain function (`T -> U`). `flat_map` transforms each +value into *another publisher* and flattens the result — that is how you chain a +dependent reactive step onto a previous one. ```rust use firefly_reactive::{Flux, Mono}; # async fn ex() { -// flat_map: chain a Mono onto the result of another (sequential dependency). +// flat_map: chain a Mono onto the result of another (a sequential dependency). let total = Mono::just(3) .flat_map(|seed| Mono::just(seed * 10)) .map(|x| x + 1) @@ -154,7 +226,8 @@ let total = Mono::just(3) .unwrap(); assert_eq!(total, Some(31)); -// flat_map on a Flux runs up to N inner publishers concurrently. +// flat_map on a Flux runs up to N inner publishers concurrently; the first +// argument is that concurrency bound. let doubled = Flux::range(1, 3) .flat_map(2, |n| Mono::just(n * 2).as_flux()) .collect_list() @@ -166,7 +239,17 @@ assert_eq!(doubled.len(), 3); # } ``` -### Combining +What just happened: + +- On the `Mono`, `flat_map(|seed| Mono::just(seed * 10))` takes the `3`, produces + a fresh `Mono` (`30`), and flattens it so the next `map` sees `30`. This is the + reactive spelling of "do A, then use A's result to do B." +- On the `Flux`, `flat_map(2, ..)` is the same idea fanned out: each of the three + source values becomes an inner publisher, and up to **2** of them run at once. + `.as_flux()` lifts the inner `Mono` into a `Flux` so the signatures line up. + +To run two independent pipelines and combine their results, use `zip` (the free +function) — both run, then their outputs pair into a tuple: ```rust use firefly_reactive::{zip, Mono}; @@ -181,21 +264,33 @@ assert_eq!(pair, Some(("alice", 42))); # } ``` -## Error semantics +> **Tip** **Checkpoint.** Run all three snippets in a test. You should see +> `total == Some(31)`, `doubled.len() == 3`, and `pair == Some(("alice", 42))`. +> If you reach for `flat_map` on a `Flux` and the compiler complains about +> arguments, remember the Flux form takes the concurrency bound first. + +## Step 4 — Handle errors and retry An `Err` item is **terminal** in a `Flux`: every operator short-circuits on the -first error and propagates it downstream — there is no per-element error -channel. To recover: +first error and propagates it downstream — there is no per-element error channel. +Once an error fires, no later value flows. To recover, you choose a recovery +operator: -- `Mono::on_error_return(fallback)` — substitute a value; +- `Mono::on_error_return(fallback)` — substitute a value. - `Mono::on_error_resume(f)` / `Flux::on_error_resume(f)` — switch to a fallback - publisher, keeping items emitted before the error; -- `Flux::on_error_continue(handler)` — drop the failing element and keep the - rest (for operators that re-signal per item); -- `Mono::on_error_map(f)` — translate the error. + publisher, keeping items emitted before the error. +- `Flux::on_error_continue(handler)` — drop the failing element and keep the rest + (for operators that re-signal per item). +- `Mono::on_error_map(f)` — translate the error into a different `FireflyError`. + +> **Note** **Key term — retry factory.** `retry` and `retry_backoff` cannot +> re-run an existing publisher, because a Rust stream or future is *single-use* — +> once consumed it is gone. So they take a **factory closure** that builds the +> publisher *fresh* for each attempt. Each retry is a brand-new subscription to a +> brand-new publisher. The Spring analog is Reactor's `Retry.backoff(..)`. -`retry` and `retry_backoff` re-subscribe to a **factory closure**, since Rust -streams and futures are single-use: +`Backoff::new(max_retries, base_delay)` describes the schedule. Here a flaky +source fails its first two attempts and succeeds on the third: ```rust use std::time::Duration; @@ -224,13 +319,32 @@ assert_eq!(value, Some(2)); # } ``` -`Mono::timeout` / `Flux::timeout` map a missed deadline to a 504 `FireflyError` -(code `REACTIVE_TIMEOUT`), which renders as an RFC 9457 problem response. +What just happened: the outer `move || { … }` is the factory — `retry_backoff` +calls it once per attempt. Inside, `Mono::from_callable` runs the fallible work +when subscribed. The shared `AtomicUsize` counts attempts: calls `0` and `1` +return `Err`, so `retry_backoff` waits (10 ms, then a growing backoff) and +re-subscribes; call `2` returns `Ok(Some(2))`, which becomes the result. The +`Backoff::new(5, …)` cap means it would give up after five retries. + +Deadlines are errors too. `Mono::timeout` / `Flux::timeout` map a missed deadline +to a 504 `FireflyError` (code `REACTIVE_TIMEOUT`), which renders as an RFC 9457 +problem response — the same response path as any other terminal error. + +> **Tip** **Checkpoint.** Run the retry snippet. It asserts `value == Some(2)`: +> the source failed twice and the third subscription succeeded. Change the +> threshold from `n < 2` to `n < 9` and watch the pipeline exhaust its five +> retries and surface the `Err` instead. -## Imperative emission with `FluxSink` +## Step 5 — Emit imperatively with `FluxSink` -When values arrive from a callback or an imperative loop, push them into a -`Flux` with `Flux::create`: +The constructors in Step 2 cover declarative sources. When values arrive from a +callback, a channel, or an imperative loop, push them into a `Flux` with +`Flux::create` and a `FluxSink`. + +> **Note** **Key term — `FluxSink`.** A `FluxSink` is the push handle you are +> handed inside `Flux::create`. Call `sink.next(v)` to emit a value, `sink.error(e)` +> to terminate with an error, and `sink.complete()` to finish the stream. It is +> the Rust analog of Reactor's `FluxSink` from `Flux.create(..)`. ```rust use firefly_reactive::Flux; @@ -247,10 +361,30 @@ assert_eq!(out, Some(vec![1, 2, 3])); # } ``` -## Schedulers +What just happened: `Flux::create` hands your closure a `sink`. The loop emits +`1, 2, 3` with `sink.next`, then `sink.complete()` closes the stream so +`collect_list` knows it is done. This is how you adapt a non-reactive producer — +say a database cursor or a callback-based SDK — into a `Flux` without rewriting it. + +> **Tip** **Checkpoint.** The test asserts `out == Some(vec![1, 2, 3])`. Forget +> the `sink.complete()` and the stream never terminates — `collect_list` would +> wait forever. Completion is your responsibility with `create`. + +## Step 6 — Move work between threads with a `Scheduler` + +By default a pipeline runs wherever you subscribed it. A `Scheduler` lets you move +work onto a different execution context — the Tokio worker pool, a blocking pool, +or inline — without restructuring the pipeline. + +> **Note** **Key term — `Scheduler`.** A `Scheduler` decides *where* work runs. +> `Scheduler::Immediate` runs inline on the current task (no hop); +> `Scheduler::Parallel` runs on the Tokio worker pool, for CPU-bound work; +> `Scheduler::BoundedElastic` runs blocking calls on a separate pool so they never +> starve the worker pool. These mirror Reactor's `Schedulers.immediate()`, +> `.parallel()`, and `.boundedElastic()`. -A `Scheduler` decides *where* work runs. `subscribe_on` hops the source onto a -scheduler; `publish_on` switches the thread for everything downstream. +Two operators apply a scheduler. `subscribe_on` hops the **source** onto a +scheduler: ```rust use firefly_reactive::{Flux, Scheduler}; @@ -267,10 +401,9 @@ assert_eq!(out, Some(vec![2, 4, 6])); # } ``` -Where `subscribe_on` chooses where the *source* runs, `publish_on` switches the -scheduler for everything *downstream* of it — the offload point can sit anywhere -in the chain, so a cheap source can hop onto a worker thread right before an -expensive map: +`publish_on` switches the thread for everything **downstream** of it, so the +offload point can sit anywhere in the chain — a cheap source can hop onto a worker +thread right before an expensive `map`: ```rust use firefly_reactive::{Flux, Scheduler}; @@ -288,15 +421,23 @@ assert_eq!(out, Some(vec![20, 30, 40])); # } ``` -`Scheduler::Immediate` runs inline, `Scheduler::Parallel` uses the Tokio worker -pool for CPU-bound work, and `Scheduler::BoundedElastic` is for blocking calls. +What just happened: `subscribe_on` chose where the *source* runs (the whole chain +above followed it onto `Parallel`); `publish_on` split the chain in two — the +first `map` ran at the subscribe site, the second ran on the worker pool. The +rule of thumb: reach for `subscribe_on` to place a blocking or CPU-bound *source*, +and `publish_on` to offload an expensive *downstream* stage. -## Reactive HTTP endpoints +> **Tip** **Checkpoint.** Both snippets assert their collected results +> (`[2, 4, 6]` and `[20, 30, 40]`). The values are identical to running without a +> scheduler — schedulers change *where* work runs, never *what* it computes. -`firefly-web` ships responders that turn a `Mono`/`Flux` into an axum response. -A reactive handler simply returns one of them; the responder drives the -publisher and writes the response. They use the stable `firefly-sse` wire -format, so any client that speaks NDJSON or SSE consumes them directly. +## Step 7 — Turn a publisher into an HTTP response + +This is where the reactive core meets the web layer, and where Lumen will use it. +`firefly-web` ships responders that turn a `Mono`/`Flux` into an axum response: a +reactive handler simply returns one of them, and the responder drives the +publisher and writes the response. They use the stable `firefly-sse` wire format, +so any client that speaks NDJSON or SSE consumes them directly. | Responder | Behaviour | |--------------------------|-----------------------------------------------------------| @@ -332,43 +473,49 @@ let app: Router = Router::new() .route("/orders/live", get(live_orders)); ``` -The responders, precisely: +What just happened, responder by responder: - **`MonoJson(Mono)`** resolves the `Mono`: `Ok(Some)` → `200` - `application/json`; `Ok(None)` → `404` `application/problem+json`; `Err` → - that error's problem response. -- **`NdJson(Flux)`** streams `application/x-ndjson` — one compact JSON doc + - `'\n'` per element, flushed incrementally with real backpressure. The `Flux`'s - `Stream` is bridged straight into an axum streaming `Body`; the whole stream - is **never** buffered. An `Err` item mid-stream terminates the body cleanly. -- **`Sse(Flux)`** streams `text/event-stream` — each element serialized to a + `application/json`; `Ok(None)` → `404` `application/problem+json`; `Err` → that + error's problem response. The empty `Mono` becoming a clean 404 is exactly the + three-layer `Result, _>` from Step 1 mapped onto HTTP. +- **`NdJson(Flux)`** streams `application/x-ndjson` — one compact JSON document + plus `'\n'` per element, flushed incrementally with real backpressure. The + `Flux`'s `Stream` is bridged straight into an axum streaming body; the whole + stream is **never** buffered. An `Err` item mid-stream terminates the body + cleanly. +- **`Sse(Flux)`** streams `text/event-stream` — each element serialized into a bare `data: \n\n` frame, byte-identical to the `firefly-sse` writer. -- **`SseEvents(Flux)`** streams pre-built `firefly_sse::Event` values — - use it when you need `id` / `event` / `retry` fields. +- **`SseEvents(Flux)`** streams pre-built `firefly_sse::Event` values — use + it when you need control over the `id` / `event` / `retry` fields. -> **Warning** — Backpressure is real, not cosmetic. A slow client throttles the -> producer; nothing is buffered up front. This is what lets a `NdJson` endpoint -> stream a million rows without the response landing fully in memory. +> **Warning** Backpressure here is real, not cosmetic. A slow client throttles the +> producer; nothing is buffered up front. This is what lets an `NdJson` endpoint +> stream a million rows without the response ever landing fully in memory. -### How Lumen will use this +### How Lumen uses this -Lumen's optional `GET /api/v1/wallets/:id/events` endpoint is exactly this -shape. It replays a wallet's persisted event stream as a `Flux` and -hands it to `NdJson` (or `Sse` with `?format=sse`). The whole handler — drawn -verbatim from `samples/lumen/src/web.rs`, feature-gated behind `streaming` — is -the responders above applied to the wallet domain: +Lumen's optional `GET /api/v1/wallets/:id/events` endpoint is exactly this shape. +It replays a wallet's persisted event stream as a `Flux` and hands it +to `NdJson` (or `Sse` with `?format=sse`). The whole handler — drawn verbatim from +`samples/lumen/src/web.rs`, feature-gated behind `streaming` — is the responders +above applied to the wallet domain: ```rust,ignore // samples/lumen/src/web.rs — the reactive streaming handler (feature `streaming`). -use firefly::reactive::Flux; -use firefly::web::{NdJson, Sse}; - +#[cfg(feature = "streaming")] async fn stream_events( State(api): State, Path(id): Path, axum::extract::Query(params): axum::extract::Query, ) -> Response { - // A missing wallet is decided before the streaming head is committed. + use crate::domain::WalletEvent; + use axum::response::IntoResponse; + use firefly::reactive::Flux; + use firefly::web::{NdJson, Sse}; + + // `load_events` returns `Err(NotFound)` for an absent wallet, so the 404 is + // decided before the streaming response head is committed. let events = match api.ledger.load_events(&id).await { Ok(events) => events, Err(e) => return WebError::from(domain_to_web(e)).into_response(), @@ -384,23 +531,33 @@ async fn stream_events( ``` Two details worth carrying forward. First, the *not-found* decision happens -**before** the `Flux` is built, so a 404 still renders as a clean problem -response rather than a half-open stream. Second, Lumen reaches the reactive -types through the one-dependency facade — `firefly::reactive::Flux` and -`firefly::web::{NdJson, Sse}` — never the underlying `firefly-reactive` / -`firefly-web` crates. The full endpoint, including the route wiring, returns in +**before** the `Flux` is built, so a 404 still renders as a clean problem response +rather than a half-open stream. Second, Lumen reaches the reactive types through +the one-dependency facade — `firefly::reactive::Flux` and +`firefly::web::{NdJson, Sse}`, never the underlying `firefly-reactive` / +`firefly-web` crates. The full endpoint, including route wiring, returns in [Production & Deployment](./20-production.md). -## The reactive `WebClient` +> **Note** Throughout the rest of the book Lumen reaches reactive types through +> the facade — `firefly::reactive::*` for `Mono`/`Flux` and `firefly::web::*` for +> the responders. The examples in *this* chapter import `firefly_reactive` / +> `firefly_web` directly so each snippet stands alone, but the two paths name the +> identical types: `firefly::reactive` re-exports `firefly_reactive`, and +> `firefly::web` re-exports `firefly_web`. + +## Step 8 — Trace the same two types through the rest of the framework + +`Mono` and `Flux` are not a web-only convenience; they are the spine the whole +framework hangs off. You will meet each of these in its own chapter, but seeing +the through-line now makes those chapters click. -Firefly's reactive HTTP client hands its terminal operators back as -`Mono` / `Flux`, so an outbound call drops straight into a reactive pipeline and -composes end-to-end with the `NdJson` / `Sse` responders above. Full treatment -in [HTTP Clients](./13-http-clients.md); the shape: +**The reactive `WebClient`.** Firefly's reactive HTTP client hands its terminal +operators back as `Mono` / `Flux`, so an outbound call drops straight into a +reactive pipeline and composes end-to-end with the `NdJson` / `Sse` responders +above. Full treatment in [HTTP Clients](./13-http-clients.md); the shape: ```rust,no_run use firefly_client::WebClientBuilder; -use http::Method; use serde::Deserialize; #[derive(Deserialize)] @@ -426,27 +583,24 @@ let _ticks: firefly_reactive::Flux = client # } ``` -> **Note.** The client has **no baked-in retry**. Compose `Mono::retry` / -> `Mono::retry_backoff` on the returned publisher, so retry policy lives at the -> call site where it belongs rather than hidden inside the client. +> **Note** The client has **no baked-in retry**. Compose `Mono::retry` / +> `Mono::retry_backoff` (Step 4) on the returned publisher, so retry policy lives +> at the call site where it belongs rather than hidden inside the client. -## Reactive repositories, EDA, and CQRS +**Repositories.** `ReactiveCrudRepository` returns `Mono`/`Flux`; the SQL +adapters stream rows out of `find_all()` as a `Flux` so a huge table never lands +fully in memory. See [Persistence](./07-persistence.md). -The same two types thread through the rest of the framework — and through Lumen: +**EDA.** `InMemoryBroker::subscribe_reactive(topic)` yields a `Flux` (in an +`EdaResult`), and `publish_mono(event)` is a cold reactive publish returning +`Mono<()>`. Lumen's ledger publishes every wallet event to a `Broker`; see +[EDA](./10-eda-messaging.md). -- **Repositories** — `ReactiveCrudRepository` returns `Mono`/`Flux`; the - SQL adapters stream rows out of `find_all()` as a `Flux` so a huge table - never lands fully in memory. See [Persistence](./07-persistence.md). -- **EDA** — `InMemoryBroker::subscribe_reactive(topic)` yields a `Flux`, - and `publish_mono(event)` is a cold reactive publish. Lumen's ledger publishes - every wallet event to a `Broker`; see [EDA](./10-eda-messaging.md). -- **CQRS** — `Bus::send_mono` / `Bus::query_mono` wrap the dispatch in a lazy - `Mono`, running the same handler lookup and middleware chain. Lumen's - wallet commands ride this bus; see [CQRS](./09-cqrs.md). - -A taste of the reactive bus, the shape a Lumen `OpenWallet` could take when -composed reactively (it dispatches the same handler the controller's -`bus.send(..)` does, only lazily): +**CQRS.** `Bus::send_mono` / `Bus::query_mono` wrap the dispatch in a lazy +`Mono`, running the *same* handler lookup and middleware chain as the +synchronous `Bus::send`. Lumen's wallet commands ride this bus; see +[CQRS](./09-cqrs.md). A taste — the shape a reactively-composed `GetWallet` query +takes (both methods take `&Arc` so the lazy `Mono` can own the bus): ```rust,ignore use std::sync::Arc; @@ -461,58 +615,93 @@ let balance = bus .await?; // Ok(Some()) ``` -> **Note.** Because `firefly-reactive` fixes its error channel to -> `FireflyError`, a failed dispatch is mapped from the bus's `CqrsError` into a -> status-faithful `FireflyError` (validation → 422, authorization → 403, missing -> handler → 500) with the original error preserved as `source()` — so a reactive -> command flows straight into the RFC 9457 problem stack. +> **Note** Because `firefly-reactive` fixes its error channel to `FireflyError`, a +> failed dispatch is mapped from the bus's `CqrsError` into a status-faithful +> `FireflyError` (validation → 422, authorization → 403, missing handler → 500) +> with the original error preserved as `source()` — so a reactive command flows +> straight into the RFC 9457 problem stack with no extra translation. -## Interop with raw `Stream` / `Future` +## Step 9 — Interop with raw `Stream` / `Future` -The reactive types are not a walled garden. Convert in and out at the edges: +The reactive types are not a walled garden. Convert in and out at the edges so a +`Mono`/`Flux` can wrap (or be wrapped by) ordinary async Rust: - **In:** `Flux::from_stream` (a `Stream>`), `Flux::from_value_stream` (a `Stream`), `Mono::from_future`, `Mono::from_result_future`. - **Out:** `Flux::to_stream` / `Flux::into_stream`, `Mono::into_future` (or just - `.await` the `Mono`). - -## What changed in Lumen - -No Lumen source file lands in this chapter — but Lumen now has the two -publishers every reactive surface it touches is built from: - -- **`Flux`** is the engine behind the `GET /api/v1/wallets/:id/events` - streaming endpoint: a wallet's events replay through `Flux::just(items)` and - stream out as `NdJson` (or `Sse`), with the not-found check resolved *before* - the stream head is committed. Lumen reaches it as `firefly::reactive::Flux` + - `firefly::web::{NdJson, Sse}` — one dependency, no crate sprawl. -- **`Mono`** is the lazy result of `Bus::send_mono` / `Bus::query_mono`, the - reactive face of the wallet command/query bus you wire in - [CQRS](./09-cqrs.md). Its `FireflyError` channel is the same one the RFC 9457 - problem responses render from. + `.await` the `Mono` directly — a `Mono` is itself awaitable). + +What just happened: these are the seams that let you adopt the reactive core +incrementally. An existing `Stream` becomes a `Flux` you can apply backpressure +and recovery operators to; a `Mono` becomes a plain `Future` the moment some other +API wants one. + +## Recap + +You now hold the vocabulary the rest of the book builds on: + +- **Two publishers, by cardinality.** `Mono` produces 0-or-1 value; + `Flux` produces 0..N. Both are **lazy** and **cold**: nothing runs until you + subscribe, block, or await, and each subscription redoes the work. +- **One fixed error channel.** Every terminal error is a + `firefly_kernel::FireflyError`, which is why pipelines wire straight into RFC + 9457 problem responses with no error-type plumbing. +- **`.block().await` returns `Result, FireflyError>`** — outer + success/error, inner value/empty. A `Flux` terminal operator returns a `Mono`, + so it reads the same way. +- **Recovery is explicit.** `on_error_return` / `on_error_resume` / + `on_error_continue` / `on_error_map` recover; `retry` / `retry_backoff` take a + **factory** because publishers are single-use; `timeout` maps a deadline to a + 504 `FireflyError`. +- **`Flux::create` + `FluxSink`** push values imperatively; a `Scheduler` + (`subscribe_on` / `publish_on`) moves work between inline, the worker pool, and + the blocking pool. +- **The web responders** `MonoJson`, `NdJson`, `Sse`, and `SseEvents` turn a + publisher into an HTTP response, with real backpressure on the streaming ones. +- **The same two types thread everywhere** — the reactive `WebClient`, + `ReactiveCrudRepository`, the EDA broker, and `Bus::send_mono` / `query_mono`. + +What this means for Lumen: no source file landed this chapter, but Lumen now has +the two publishers every reactive surface it touches is built from — the +`Flux` behind its streaming endpoint, and the `Mono` behind its +command/query bus. ## Exercises -1. **Map a balance.** Build `Mono::just(1_250_i64)` (a balance in cents), - `map` it to a major-unit `f64` (`cents as f64 / 100.0`), and `block().await` - it. Confirm you get `Some(12.5)`. +1. **Map a balance.** Build `Mono::just(1_250_i64)` (a balance in cents), `map` it + to a major-unit `f64` (`cents as f64 / 100.0`), and `block().await` it. Confirm + you get `Some(12.5)`. -2. **Stream wallet events as a `Flux`.** Make a `Vec` of signed - balance deltas (`[1000, 50, -25]`), wrap it with `Flux::just`, `scan` a - running balance, and `collect_list` it. Verify the running balances are - `[1000, 1050, 1025]` — a hand-rolled version of what the Lumen streaming - endpoint emits per event. +2. **Stream wallet events as a `Flux`.** Make a `Vec` of signed balance deltas + (`[1000, 50, -25]`), wrap it with `Flux::just`, `scan` a running balance, and + `collect_list` it. Verify the running balances are `[1000, 1050, 1025]` — a + hand-rolled version of what the Lumen streaming endpoint emits per event. 3. **Recover from a flaky source.** Write a `Mono::from_callable` that returns `Err(FireflyError::internal("flaky"))` the first two times and `Ok(Some(n))` afterward, then wrap it in `Mono::retry_backoff(factory, Backoff::new(5, - Duration::from_millis(10)))`. Assert it resolves to a value — the retry - factory pattern Lumen's HTTP client would use against an external FX provider. + Duration::from_millis(10)))`. Assert it resolves to a value — the retry-factory + pattern Lumen's HTTP client would use against an external FX provider. 4. **Pick a responder.** Given a `Flux`, decide which responder a real-time dashboard wants (`Sse`) versus a bulk export (`NdJson`), and explain in one sentence why backpressure matters for the export case. -You now have the vocabulary the rest of the book builds on. Next, put it to work -in [Your First HTTP API](./06-first-http-api.md). +5. **Push, then complete.** Use `Flux::create` to emit `1..=5` with `sink.next`, + but *omit* `sink.complete()`. Run it under a `Mono::timeout` of a few hundred + milliseconds and observe the 504 `FireflyError` — then add the `complete()` and + watch it pass cleanly. This is why completion is your responsibility with + `create`. + +## Where to go next + +- Put these publishers to work behind real routes in + **[Your First HTTP API](./06-first-http-api.md)** — the first chapter that + returns a `Mono`/`Flux` from a Lumen handler. +- See `Flux` stream rows out of the database in + **[Persistence](./07-persistence.md)** via `ReactiveCrudRepository`. +- Compose `Bus::send_mono` / `Bus::query_mono` into wallet pipelines in + **[CQRS](./09-cqrs.md)**. +- Subscribe to a `Flux` and `publish_mono` wallet events in + **[EDA & Messaging](./10-eda-messaging.md)**. diff --git a/docs/book/src/06-first-http-api.md b/docs/book/src/06-first-http-api.md index eaf62ae..93203e5 100644 --- a/docs/book/src/06-first-http-api.md +++ b/docs/book/src/06-first-http-api.md @@ -1,32 +1,76 @@ # Your First HTTP API -> By the end of this chapter Lumen has a real HTTP surface: a `WalletApi` -> controller declared with `#[rest_controller]`, a `POST /api/v1/wallets` -> endpoint that opens a wallet and answers `201 Created`, a `GET -> /api/v1/wallets/:id` that returns a `WalletView`, and typed errors that render -> as RFC 9457 problem documents — with an in-process test that drives the whole -> router without binding a socket. This is the chapter where Lumen stops being a -> banner and starts being a service. - -So far Lumen compiles, runs, and serves an actuator, and you know how the -framework wires the beans it scans. Now you give it endpoints. The HTTP layer is -axum; Firefly contributes the controller macro, the problem rendering, and the -correlation/idempotency middleware you met in the Quickstart, all woven in by the -framework. You write handlers; the framework supplies the wiring *and* mounts the -controller for you. - -## The `#[rest_controller]` macro - -Lumen's wallet endpoints live on one type, `WalletApi`, a `#[derive(Controller)]` -DI bean whose `impl` block carries `#[rest_controller]`. The struct's -collaborators are `#[autowired]` from the container, and the macro reads each -verb attribute and generates a `WalletApi::routes(state) -> axum::Router` -function — so a controller is just a bean plus an `impl` with annotated methods, -and the routing table is derived from the code rather than maintained beside it. +So far Lumen compiles, boots, prints a banner, and serves an actuator — but it +has no endpoints of its own. You also know, from +[Dependency Wiring](./04-dependency-wiring.md), how the framework discovers and +wires the beans it scans. This is the chapter where Lumen stops being a banner +and starts being a *service*: you give it a real HTTP surface, declared with one +macro, mounted for you, and proven by a test that drives the whole router +without ever binding a socket. + +The HTTP layer underneath is [axum](https://docs.rs/axum). Firefly does not +hide it — you write ordinary axum handlers — but it *adds* the controller macro, +the problem-rendering, and the correlation/idempotency middleware you met in the +[Quickstart](./02-quickstart.md). You write two handlers; the framework supplies +the wiring and mounts the controller. + +By the end of this chapter you will: + +- Declare a REST controller as a single DI bean whose collaborators are + autowired, using `#[derive(Controller)]` and `#[rest_controller]`. +- Map two verbs — `POST /api/v1/wallets` and `GET /api/v1/wallets/:id` — onto + handler methods, and understand how the macro composes the route paths. +- Return a plain `serde` view (`WalletView`) and turn typed errors into RFC 9457 + `application/problem+json` documents with the right HTTP status. +- Understand *why* you never call `mount` — that adding the controller bean *is* + mounting it. +- Drive the fully wired router in-process with `tower::oneshot`, with no live + server and no port to race on. + +## Concepts you will meet + +Before the first line of code, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — controller.** A *controller* is the object that owns a +> group of HTTP endpoints. Its methods are the *handlers* — one per verb-and-path +> mapping. In Firefly a controller is just a bean with an annotated `impl` block; +> the framework reads the annotations and builds the routing table. The Spring +> analog is a `@RestController`. + +> **Note** **Key term — handler / extractor.** A *handler* is the async function +> that runs for one route. An *extractor* is an argument type that pulls a piece +> of the request out for you — the path id, the JSON body, a query object. These +> are axum's own extractors (`Path`, `Json`, `State`); Firefly reuses them and +> adds a few of its own. + +> **Note** **Key term — RFC 9457 problem document.** RFC 9457 (which obsoletes +> RFC 7807) defines `application/problem+json` — a small, standard JSON envelope +> for HTTP errors with `type`, `title`, `status`, and `detail` fields. Firefly +> renders every handler error this way automatically, so all your errors speak +> one machine-readable shape. The Spring analog is `ProblemDetail`. + +> **Note** **Key term — CQRS bus.** Lumen routes state-changing **commands** and +> read-only **queries** through a shared *bus*. The controller's job is only to +> translate HTTP into a message and dispatch it; the wallet logic lives behind +> the bus. You build that machinery in [CQRS](./09-cqrs.md). For this chapter, +> treat `bus.send(...)` / `bus.query(...)` as "hand this message to the handler +> that knows how". *CQRS* expands to Command/Query Responsibility Segregation. + +## Step 1 — Declare the controller bean + +Lumen's wallet endpoints all live on one type, `WalletApi`. It is a +`#[derive(Controller)]` DI bean: a plain struct whose collaborators are +`#[autowired]` from the container. Declaring the struct is the first half of a +controller; the annotated `impl` block in [Step 2](#step-2--map-the-verbs) is the +second half. + +Open `src/web.rs` and add the imports and the struct: ```rust,ignore // src/web.rs use std::sync::Arc; + use axum::extract::{Path, State}; use axum::Json; use firefly::cqrs::QueryCache; @@ -34,7 +78,7 @@ use firefly::prelude::*; use firefly::web::{WebError, WebResult}; use crate::commands::{GetWallet, OpenWallet}; -use crate::domain::WalletView; +use crate::domain::{DomainError, WalletView}; /// The wallet HTTP surface — a `#[derive(Controller)]` DI bean. Its /// collaborators are **autowired** from the container, and `#[rest_controller]` @@ -51,16 +95,65 @@ pub struct WalletApi { #[autowired] pub query_cache: Arc, } +``` + +What just happened, block by block: + +- The imports bring in axum's extractors (`Path`, `State`, `Json`), the CQRS + `QueryCache`, the whole high-frequency surface via `firefly::prelude::*` (which + gives you `Bus`, `Controller`, `#[autowired]`, and the verb macros), and the + web result/error types (`WebResult`, `WebError`). The `DomainError` import is + used by the error mapper in [Step 5](#step-5--map-typed-errors-to-rfc-9457-problems). +- `#[derive(Controller)]` marks the struct as a controller bean. It is the same + stereotype as the other Firefly beans you have seen — the container scans it, + constructs it, and manages its lifetime. +- Each `#[autowired]` field is a *collaborator* the container resolves and injects + when it builds the bean. `bus` is the CQRS bus the handlers dispatch through; + `ledger` is the application service the later saga and streaming endpoints use; + `query_cache` is invalidated after a write so a read-after-write never serves a + stale balance. You never construct `WalletApi` yourself — the framework does. +- `Clone` is required because the macro hands a clone of the controller to axum as + per-route *state*; the struct is `Arc`-backed, so cloning is cheap. + +> **Note** **Key term — autowiring.** *Autowiring* is the framework's +> constructor-injection: a `#[autowired]` field is resolved from the container by +> type and handed to the bean at construction. It is exactly Spring's +> `@Autowired`. You declare *what* a controller needs; the container decides *how* +> to supply it. + +> **Tip** **Checkpoint.** The struct compiles once the `Bus`, `Ledger`, and +> `QueryCache` beans it autowires exist in the crate (you declare them as +> `#[bean]` factories — the `Bus` is framework-provided, `Ledger` and +> `QueryCache` are Lumen's). If `cargo build` complains that one of these types is +> unresolved, you are ahead of the narrative: the bean factories land in +> [CQRS](./09-cqrs.md). For now, focus on the controller shape. + +## Step 2 — Map the verbs + +A struct with autowired fields is just a bean. It becomes a controller when its +`impl` block carries `#[rest_controller]` and its methods carry verb attributes. +The macro reads each one and generates a `WalletApi::routes(state) -> +axum::Router` function — so the routing table is *derived from your code*, not +maintained in a separate file beside it. + +Add the `impl` block to `src/web.rs`: +```rust,ignore +// src/web.rs (continued) /// `#[rest_controller(path = "...")]` generates `WalletApi::routes(state) -> /// axum::Router`. Each method carries one verb mapping and returns /// `WebResult`, so a handler error renders as RFC 9457 /// `application/problem+json`. -#[rest_controller(path = "/api/v1")] +#[rest_controller(path = "/api/v1", tag = "Wallets")] impl WalletApi { /// `POST /api/v1/wallets` — open a wallet. Validation failures surface as /// 422 problems; success answers `201 Created` with the view. - #[post("/wallets")] + #[post( + "/wallets", + summary = "Open a wallet", + description = "Opens a new wallet for an owner with an optional opening balance.", + status = 201 + )] async fn open( State(api): State, Json(body): Json, @@ -71,7 +164,11 @@ impl WalletApi { /// `GET /api/v1/wallets/:id` — fetch the read-model view. An unknown id /// renders as a 404 problem. - #[get("/wallets/:id")] + #[get( + "/wallets/:id", + summary = "Fetch a wallet", + description = "Returns the read-model view of a wallet." + )] async fn get( State(api): State, Path(id): Path, @@ -82,75 +179,120 @@ impl WalletApi { } ``` -Three things to read here: - -- **The path is composed.** `#[rest_controller(path = "/api/v1")]` is the prefix; - `#[post("/wallets")]` and `#[get("/wallets/:id")]` are the suffixes. The macro - joins them into `/api/v1/wallets` and `/api/v1/wallets/:id`. -- **Each handler is a plain axum handler.** `State`, `Path`, and `Json` are - axum's own extractors — Firefly does not replace them, but it *adds* to them. - Beyond `Json`/`Path`/`Query`, `firefly::web` (re-exported in - `firefly::prelude`) ships validating and problem-rendering extractors that - drop into the same handler signature: `Valid` for a JSON body and - `ValidPath` / `ValidQuery` for path/query objects (a bind failure is a - **400**, a constraint failure a **422** problem), the `Multipart` / - `UploadedFile` form-upload extractor, and the `PageRequest` argument resolver - that binds Spring's `Pageable` from `?page=&size=&sort=`. The layered sample - in [Layered Microservices](./22-layered-microservices.md) uses all of them. - You write the function; the macro only registers it on the router. -- **The controller is thin.** `open` and `get` translate HTTP into a message and - hand it to the CQRS `Bus`, then translate the result (or error) back into an - HTTP response. The wallet *logic* lives behind the bus, where - [CQRS](./09-cqrs.md) puts it. Treat `api.bus.send(...)` / `api.bus.query(...)` - here as "dispatch to the handler that knows how"; the bus, the commands, and - the read model are the subjects of chapters 7 through 11. +There are three things worth reading carefully here. + +**The path is composed.** `#[rest_controller(path = "/api/v1")]` is the prefix; +`#[post("/wallets")]` and `#[get("/wallets/:id")]` are the suffixes. The macro +joins them into `/api/v1/wallets` and `/api/v1/wallets/:id`. The `tag`, +`summary`, `description`, and `status` attributes are optional metadata: `tag` +groups the endpoints in the API docs, `summary`/`description` annotate them, and +`status = 201` tells the OpenAPI generator the success status. They change the +*documentation*, not the routing. + +**Each handler is a plain axum handler.** `State`, `Path`, and `Json` are axum's +own extractors — Firefly does not replace them. `State(api): State` +hands you the controller (with its autowired collaborators already in place); +`Path(id): Path` binds the `:id` segment; `Json(body): Json` +deserializes the request body. The return type `WebResult` is what lets a +handler error render as a problem document — covered in +[Step 5](#step-5--map-typed-errors-to-rfc-9457-problems). + +**The controller is thin.** `open` and `get` translate HTTP into a message and +hand it to the CQRS `Bus`, then translate the result (or error) back into an HTTP +response. The wallet *logic* lives behind the bus, where [CQRS](./09-cqrs.md) +puts it. Read `api.bus.send(...)` (a command) and `api.bus.query(...)` (a query) +as "dispatch to the handler that knows how"; the bus, the commands, and the read +model are the subjects of chapters 7 through 11. + +> **Note** **Key term — argument resolver / validating extractor.** Beyond +> `Json`/`Path`/`Query`, `firefly::web` (re-exported in `firefly::prelude`) ships +> extractors that drop into the same handler signature: `Valid` for a JSON +> body and `ValidPath` / `ValidQuery` for path/query objects (a bind +> failure is a **400**, a constraint failure a **422** problem), the `Multipart` +> / `UploadedFile` form-upload extractor, and the `PageRequest` argument resolver +> that binds Spring's `Pageable` from `?page=&size=&sort=`. The layered sample in +> [Layered Microservices](./22-layered-microservices.md) uses all of them. Here +> the plain `Json`/`Path` extractors are enough. > **Design note.** `#[rest_controller(path = "/api/v1")]` declares a controller > and its path prefix; `#[get]` / `#[post]` declare the verb mappings. Beyond > generating the router, the macro emits a route descriptor per endpoint that > feeds the actuator `/mappings` view and the OpenAPI generator — so the routing > table is derived from your code rather than maintained beside it, and the -> documentation surfaces stay in sync with the handlers automatically. If you've -> used a batteries-included framework before, this declarative-controller style -> will feel familiar. +> documentation surfaces stay in sync with the handlers automatically. If you +> have used a batteries-included framework before, this declarative-controller +> style will feel familiar. + +> **Tip** **Checkpoint.** `WalletApi` now carries a `#[rest_controller]` `impl` +> with two annotated methods. The macro has generated a `WalletApi::routes(state)` +> function (you never call it by hand) and registered a *mount thunk* into the +> link-time inventory. You will see both pay off in +> [Step 6](#step-6--controllers-are-auto-mounted). -## The wire shape — `WalletView` +## Step 3 — Define the wire shape The view a handler returns is a plain `serde` struct. It is the *read model* projection of a wallet — flat, query-optimized, and decoupled from the internal -aggregate. The balance travels as an integer count of minor units (cents), so -`€10.00` is the JSON number `1000`: +aggregate. + +> **Note** **Key term — read model / DTO.** A *DTO* (data transfer object) is the +> on-the-wire shape a client sees, deliberately separate from your internal +> domain types. Lumen's `WalletView` is the read-model DTO: a flat projection a +> query returns. Keeping it separate from the `Wallet` aggregate means you can +> evolve the internal model without breaking the API contract. ```rust,ignore // src/domain.rs -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Schema)] pub struct WalletView { + /// The wallet id. pub id: String, + /// The owner's display name. pub owner: String, /// The current balance, in minor units (cents). pub balance: i64, - /// The aggregate version (number of events applied). + /// The aggregate version (number of events applied) — lets a client + /// detect staleness under eventual consistency. pub version: i64, } ``` +What just happened: `WalletView` derives `Serialize` / `Deserialize` so it +crosses the wire, and `Schema` so the OpenAPI generator can describe it (the +`Schema` derive is the subject of [OpenAPI](./06a-openapi.md)). The balance +travels as an integer count of *minor units* (cents), so `€10.00` is the JSON +number `1000` — money never rides as a float. + The request body Lumen accepts on `POST /api/v1/wallets` is just as ordinary — -the `OpenWallet` message, with a `#[serde(rename)]` so the JSON field is -`openingBalance` while the Rust field stays snake_case: +the `OpenWallet` command. A `#[serde(rename)]` on its balance field makes the JSON +key `openingBalance` while the Rust field stays snake_case, so the wire looks +like: ```json { "owner": "alice", "openingBalance": 1000 } ``` -## Content negotiation — one DTO, JSON or XML +> **Tip** **Checkpoint.** `WalletView` lives in `src/domain.rs` and the +> controller imports it with `use crate::domain::WalletView;`. The JSON a `GET` +> returns is exactly its four fields: `id`, `owner`, `balance`, `version`. + +## Step 4 — Let the client pick the format (optional) + +Lumen's handlers answer `application/json` because they return +`Json` — a deliberate, format-pinned contract. But a controller can +also hand the framework a DTO and let the *client* pick the wire format. This step +is optional reading; you can skip to [Step 5](#step-5--map-typed-errors-to-rfc-9457-problems) +and lose nothing of the running narrative. + +> **Note** **Key term — content negotiation.** *Content negotiation* lets one +> handler serve several wire formats: the client sends an `Accept` header and the +> framework renders the response with the matching converter. The Spring analog is +> an `HttpMessageConverter` chosen by `produces`. -Lumen's handlers answer `application/json` because they return `Json` -— a deliberate, format-pinned contract. But a controller can also hand the -framework a DTO and let the *client* pick the wire format. Wrap the return value -in `Negotiate(dto)` and the response is rendered with the converter the request's -`Accept` header selects — `JsonMessageConverter` for `application/json`, -`XmlMessageConverter` for `application/xml` / `text/xml` — while the request body -is read by its `Content-Type` the same way: +Wrap the return value in `Negotiate(dto)` and the response is rendered with the +converter the request's `Accept` header selects — `JsonMessageConverter` for +`application/json`, `XmlMessageConverter` for `application/xml` / `text/xml` — +while the request body is read by its `Content-Type` the same way: ```rust,ignore // a format-agnostic variant of the wallet GET @@ -176,51 +318,24 @@ GET /api/v1/wallets/wlt_1 Accept: application/xml → wlt_1alice1000... ``` -You wire none of this. The `ContentNegotiationLayer` is installed by default in -`WebStack::apply_middleware` — it sits closest to your routes, so a `Negotiate` -response is re-rendered to the client's `Accept` before the outer middleware edge -runs, and a plain `Json` (or any other) response passes through untouched. An -absent or empty `Accept` defaults to JSON, and an unmatched type falls back to the -first registered converter (JSON), so the negotiation never fails the request. +You wire none of this. The `ContentNegotiationLayer` is installed by default — it +sits closest to your routes, so a `Negotiate` response is re-rendered to the +client's `Accept` before the outer middleware edge runs, and a plain `Json` +(or any other) response passes through untouched. An absent or empty `Accept` +defaults to JSON, and an unmatched type falls back to the first registered +converter (JSON), so negotiation never fails the request. > **Design note.** `Negotiate(dto)` hands the framework a DTO and lets the -> request's `Accept` header pick the wire format — `JsonMessageConverter` for -> `application/json`, `XmlMessageConverter` for `application/xml` / `text/xml` — -> with no controller code. The `JsonMessageConverter` / `XmlMessageConverter` pair -> ships in the registry, and `apply_middleware` installs the -> `ContentNegotiationLayer` by default, so negotiation is on out of the box. Add a -> converter — say CBOR — by implementing `MessageConverter` and adding it; -> user converters take priority over the built-ins. - -Firefly's field-level format contract is **serde**: the rules live in derives on -the DTO and apply identically whichever converter renders the value — -`#[serde(rename_all = "camelCase")]` to fix the on-the-wire naming, -`#[serde(tag = "type")]` for polymorphic enums, `chrono` types for date/time, and -`#[serde(deny_unknown_fields)]` to reject unexpected keys on read: - -```rust,ignore -#[derive(Serialize, Deserialize)] -#[serde(rename_all = "camelCase", deny_unknown_fields)] -pub struct WalletView { - pub id: String, - pub owner: String, - pub balance: i64, - pub version: i64, -} -``` - -## JSON serialization & content negotiation +> request's `Accept` header pick the wire format, with no controller code. The +> `JsonMessageConverter` / `XmlMessageConverter` pair ships in the registry, and +> the `ContentNegotiationLayer` is installed by default, so negotiation is on out +> of the box. Add a converter — say CBOR — by implementing `MessageConverter` and +> registering it; user converters take priority over the built-ins. -The serde derives above pin the wire shape *per type* — practical when a handful -of DTOs each want their own contract. But a service usually wants **one** house -style: every response in `camelCase`, nulls dropped, the same inclusion rules -everywhere. Firefly gives you a single object to express that policy and a way to -install it across the whole service: `firefly_web`'s `ObjectMapper`. - -`ObjectMapper` is a builder. You set a property-naming convention, an inclusion -rule, and whether to pretty-print, and the resulting mapper translates between -your Rust structs and JSON on the wire — applying the policy on the way out and -its inverse on the way in: +If you want one *house style* — every response in `camelCase`, nulls dropped, the +same inclusion rules everywhere — rather than per-type serde attributes, Firefly +gives you a single object to express that policy: `ObjectMapper`. It is a builder +that sets a property-naming convention, an inclusion rule, and pretty-printing: ```rust,ignore use firefly::web::{ObjectMapper, PropertyNaming, Inclusion}; @@ -246,45 +361,16 @@ The naming and inclusion options are: | `Inclusion::NonNull` | omit `null` fields | | `Inclusion::NonEmpty` | omit `null`, empty strings, and empty collections | -The mapper serializes and deserializes through the same methods you'd reach for -on `serde_json`, but with the policy applied: - -```rust,ignore -// Serialize: a snake_case Rust struct speaks camelCase on the wire. -let json: String = mapper.to_string(&wallet)?; // -> Result -let value: serde_json::Value = mapper.to_value(&wallet)?; +The naming transform is *reversible*: a `snake_case` Rust struct speaks +`camelCase` on the wire and reads it back the same way, so the same mapper sits on +both ends of a request/response. If you need the raw transform — for instance to +post-process a `serde_json::Value` you built by hand — `apply_write(value)` +renames toward the wire and `apply_read(value)` renames back toward your structs. -// Deserialize: read camelCase back into the snake_case struct. -let wallet: WalletView = mapper.from_str(&json)?; // -> Result -let wallet: WalletView = mapper.from_value(value)?; -``` - -The naming transform is **reversible**: a `snake_case` Rust struct speaks -`camelCase` on the wire and reads it back the same way, so the same `ObjectMapper` -sits on both ends of a request/response without you tracking which direction a -field is travelling. If you need the raw transform — for instance to post-process -a `serde_json::Value` you built by hand — `apply_write(value) -> Value` renames -toward the wire and `apply_read(value) -> Value` renames back toward your structs. - -> **Note.** A renaming mapper rewrites *every* object key in the document — it -> works on the JSON tree, so it cannot tell a struct field from a key inside a -> free-form `HashMap` you carry as data. Use a global naming policy on -> **DTO-shaped** payloads; for a type whose body holds arbitrary string-keyed -> data, leave the global policy at `AsIs` and name that one type with -> `#[serde(rename_all = "camelCase")]` — that is type-aware and never touches -> data keys. (Field names with a trailing number round-trip cleanly when -> written glued, `opening_balance2`, rather than with a separating underscore, -> `opening_balance_2`, which normalises to `opening_balance2`.) - -### Installing a global policy with `MappingJsonConverter` - -A free-standing `ObjectMapper` is useful, but the point is to make the *whole -service* observe one policy without decorating every DTO. `MappingJsonConverter` -wraps a mapper and implements `firefly_web::MessageConverter` for -`application/json`. Add it to a `MessageConverterRegistry` and every -negotiated JSON request and response — the `Negotiate(dto)` responses from the -previous section, and any JSON body the framework reads — flows through your -naming and inclusion policy: +To make the *whole service* observe one policy without decorating every DTO, wrap +a mapper in `MappingJsonConverter` and register it. It implements `MessageConverter` +for `application/json`, and because it registers as a *user* converter it takes +priority over the built-in `JsonMessageConverter`: ```rust,ignore use firefly::web::{ObjectMapper, PropertyNaming, Inclusion, MappingJsonConverter}; @@ -299,26 +385,38 @@ let mapper = ObjectMapper::new() registry.add(std::sync::Arc::new(MappingJsonConverter::new(mapper))); ``` -Because it registers as a user converter, `MappingJsonConverter` takes priority -over the built-in `JsonMessageConverter` for `application/json` — so registering it -once (as a converter bean) is all it takes to apply a global JSON naming and +Registering it once (as a converter bean) applies a global JSON naming and inclusion policy to the entire HTTP surface, instead of repeating -`#[serde(rename_all = ...)]` on every DTO. Per-type serde attributes still work -and compose on top: reach for them when a specific type needs to deviate from the -house style, and let `MappingJsonConverter` carry the default everywhere else. +`#[serde(rename_all = ...)]` on every DTO. Per-type serde attributes still +compose on top: reach for them when one type needs to deviate from the house +style, and let `MappingJsonConverter` carry the default everywhere else. + +> **Warning** A renaming mapper rewrites *every* object key in the document — it +> works on the JSON tree, so it cannot tell a struct field from a key inside a +> free-form `HashMap` you carry as data. Use a global naming policy on +> *DTO-shaped* payloads; for a type whose body holds arbitrary string-keyed data, +> leave the global policy at `AsIs` and name that one type with +> `#[serde(rename_all = "camelCase")]` — that is type-aware and never touches +> data keys. -## Typed errors → RFC 9457 problems +## Step 5 — Map typed errors to RFC 9457 problems A handler that returns `WebResult` turns any error into the right `application/problem+json` response via `?`. `WebResult` is an alias whose error arm is a `WebError`, and the framework knows how to render it. Lumen's controller maps the bus's error channel onto a precise HTTP status with one -helper: +helper. -```rust,ignore -// src/web.rs -use crate::domain::DomainError; +> **Note** **Key term — `WebResult` / `WebError`.** `WebResult` is +> `Result`. A `WebError` carries a `FireflyError`, and the framework's +> problem renderer turns it into an `application/problem+json` body with the right +> status code. Returning `WebResult` and using `?` is all it takes — you never +> write the response yourself. + +Add the error mapper to `src/web.rs`: +```rust,ignore +// src/web.rs (continued) /// Maps a bus `CqrsError` onto the precise HTTP problem the domain implies: /// a validation failure → 422, a not-found detail → 404, an /// insufficient-funds / non-positive detail → 422, otherwise 500. @@ -342,6 +440,12 @@ fn cqrs_to_web(err: CqrsError) -> WebError { } ``` +What just happened: `cqrs_to_web` inspects the bus's `CqrsError` and picks the +`FireflyError` constructor that matches the failure — a validation failure becomes +a 422, a "not found" detail a 404, and an unexpected error a 500. The handlers +call it as `.map_err(cqrs_to_web)?`, so the error flows out of the handler as a +`WebError` and the framework's renderer does the rest. + The `FireflyError` constructors map straight to HTTP status — pick the one that matches the failure and the renderer does the rest: @@ -373,23 +477,36 @@ A rendered problem for an unknown wallet looks like this — note the dedicated > errors. The RFC 9457 contract is stable and language-neutral, so a Firefly 404 > presents identically to every client regardless of which service produced it. -## Controllers are auto-mounted +> **Tip** **Checkpoint.** `src/web.rs` now holds the `WalletApi` struct, its +> `#[rest_controller]` `impl`, and `cqrs_to_web`. That is a complete HTTP surface +> — two endpoints and their error mapping — without a single line that mounts a +> route or builds a router by hand. + +## Step 6 — Controllers are auto-mounted You never mount the controller. Because `WalletApi` is a `#[derive(Controller)]` -bean, the `#[rest_controller]` macro registers a *mount thunk* into the -link-time inventory alongside the generated `routes(state)` function. At boot, +bean, the `#[rest_controller]` macro registered a *mount thunk* into the link-time +inventory alongside the generated `routes(state)` function. At boot, `FireflyApplication` calls `firefly::web::mount_controllers(&container)`, which resolves each controller bean from the container (constructing its autowired -collaborators), calls its `routes(state)`, and merges the result — then wraps the -whole thing in the web middleware chain: +collaborators), calls its `routes(state)`, and merges the result — then layers on +security and wraps the whole thing in the web middleware chain: ```rust,ignore // inside FireflyApplication::bootstrap — you write none of this: let routes = firefly::web::mount_controllers(&container) // every #[rest_controller] .merge(firefly::web::mount_route_contributors(&container)); // every RouteContributor bean -let api = web.apply_middleware(routes); // + security, trace, docs, 404 +// security (the FilterChain + BearerLayer beans) is layered onto these routes, +// then the whole router is wrapped in the observability edge: +let api = web.apply_middleware(routes); // + trace, metrics, 404, problem ``` +> **Note** **Key term — link-time inventory.** The *inventory* is a registry the +> macros write into at compile time: each `#[rest_controller]`, command handler, +> event listener, and `#[scheduled]` task records itself there. At boot the +> framework reads the inventory back and wires everything — no reflection, no +> manual registration list. It is how `main` never changes as Lumen grows. + So adding the controller *is* mounting it: declare the bean, annotate the impl, and the route table grows. The macro's generated `routes(state)` is still there (it is what the mount thunk calls), and the `RouteDescriptor` it emits per @@ -397,110 +514,168 @@ endpoint feeds the actuator `/mappings` view and the OpenAPI generator — but y never call either by hand. Every request to a wallet route passes through the canonical chain you got for -free in the Quickstart — the RFC 9457 problem layer, correlation-id propagation, -and idempotency replay — before it reaches your handler. You wrote the two -handlers; the rest of the request lifecycle is the framework's. +free in the [Quickstart](./02-quickstart.md) — the RFC 9457 problem layer, +correlation-id propagation, and idempotency replay — before it reaches your +handler. You wrote the two handlers; the rest of the request lifecycle is the +framework's. -> **Note** — `main` never changes as Lumen grows. The JWT security layer is +> **Note** `main` never changes as Lumen grows. The JWT security layer is > discovered from a `FilterChain` bean in [Security](./14-security.md); the > streaming endpoint is added as a `RouteContributor` bean in > [Production](./20-production.md). Each is a *new bean the scan finds*, not a > line edited into a composition root — the framework absorbs every addition. -## Proving it works — an in-process round-trip +> **Tip** **Checkpoint.** Run `cargo run` and read the startup report's +> `:: routes ::` line — `/api/v1/wallets` and `/api/v1/wallets/:id` now appear in +> it. You added them by declaring a bean, not by touching a router. (The mutations +> will answer `401` until the security beans exist; that is expected and arrives +> in [Security](./14-security.md).) + +## Step 7 — Prove it works in-process + +Now prove the whole thing round-trips. Lumen's HTTP tests drive the *real, +fully-wired* router **in-process** with `tower::ServiceExt::oneshot` — no socket +bound, no port to race on. + +> **Note** **Key term — `bootstrap()` and `oneshot`.** `bootstrap()` is the +> sibling of `run()`: it assembles the same app — the same component scan and +> auto-mount — but returns a `Bootstrapped` value *without serving*, exposing the +> wired `api_router`. `tower::ServiceExt::oneshot` feeds one `Request` to that +> `Router` and returns the `Response`, all in the test process. Together they run +> the real request path with no live server. -Because `build_router()` returns a self-contained `axum::Router`, Lumen's tests drive it -**in-process** with `tower::ServiceExt::oneshot` — no socket bound, no port to -race on. This is the first end-to-end test, the open-then-get round-trip: +The test boot path is a small helper, `build_router()`, in `src/web.rs`. It is +gated to test builds and calls `bootstrap()`, returning the exact `axum::Router` +that `main` serves: ```rust,ignore -// tests/http.rs +// src/web.rs — the in-process router the tests drive (no socket bound). +#[cfg(test)] +pub(crate) async fn build_router() -> axum::Router { + firefly::FireflyApplication::new(APP_NAME) + .version(VERSION) + .bootstrap() + .await + .expect("lumen bootstrap") + .api_router +} +``` + +Because `bootstrap()` runs the *same* component scan and auto-mount as `run()`, +the test drives the real, fully-wired controller stack — the macro-generated +routes, the JSON contract, and the status-code mapping — the same code path a +real client hits, minus the network. `APP_NAME` and `VERSION` are the two +constants Lumen keeps beside its HTTP surface (you met them in the Quickstart). + +The tests themselves live in `src/http_test.rs`, a `#[cfg(test)] mod` compiled +into the crate so it can reach the crate-internal `build_router`. Each test boots +**one** app context and drives every request against it — Spring Boot's +`@SpringBootTest` model — so the singletons stay consistent across a test's +requests (the wallet a command opens is the wallet a later query reads). A couple +of small request helpers keep the tests readable: + +```rust,ignore +// src/http_test.rs use axum::body::Body; use axum::http::{Request, StatusCode}; -use firefly_sample_lumen::build_router; -use firefly_sample_lumen::domain::WalletView; +use axum::response::Response; +use axum::Router; use http_body_util::BodyExt; use tower::ServiceExt; +use crate::build_router; +use crate::domain::WalletView; +use crate::security::{mint_token, CUSTOMER_ROLE}; + +/// A bearer token for a customer — mutations require authentication, which the +/// framework auto-discovers from the security beans. +fn bearer() -> String { + format!("Bearer {}", mint_token("u-alice", &[CUSTOMER_ROLE])) +} + +/// Sends one request against the (cloned) shared app and returns the response. +async fn send(app: &Router, req: Request) -> Response { + app.clone().oneshot(req).await.unwrap() +} + +/// Decodes a JSON response body into a typed value. +async fn body_json(res: Response) -> T { + let bytes = res.into_body().collect().await.unwrap().to_bytes(); + serde_json::from_slice(&bytes).unwrap() +} +``` + +> **Note** Because security is auto-discovered from the `FilterChain` and +> `BearerLayer` beans (the subject of [Security](./14-security.md)), the +> mutating `POST` carries an `Authorization: Bearer …` header. The read-only `GET` +> does not need one. If you have not added the security beans yet, run the +> mutation tests without the header and expect a `401` — that *is* the framework +> enforcing the chain it discovered. + +Here is the first end-to-end test, the open-then-get round-trip. The `axum::Router` +is `Arc`-backed and cheap to clone, so each `oneshot` clones the shared app: + +```rust,ignore #[tokio::test] async fn open_then_get_round_trips_through_cqrs() { + let app = build_router().await; + // POST /api/v1/wallets → 201 Created with the opened view. - let res = build_router() - .await - .oneshot( - Request::post("/api/v1/wallets") - .header("content-type", "application/json") - .body(Body::from( - serde_json::to_vec(&serde_json::json!({ - "owner": "alice", "openingBalance": 1_000 - })) - .unwrap(), - )) + let res = send( + &app, + Request::post("/api/v1/wallets") + .header("content-type", "application/json") + .header("authorization", bearer()) + .body(Body::from( + serde_json::to_vec(&serde_json::json!({ + "owner": "alice", "openingBalance": 1_000 + })) .unwrap(), - ) - .await - .unwrap(); + )) + .unwrap(), + ) + .await; assert_eq!(res.status(), StatusCode::CREATED, "open should 201"); - - let bytes = res.into_body().collect().await.unwrap().to_bytes(); - let opened: WalletView = serde_json::from_slice(&bytes).unwrap(); + let opened: WalletView = body_json(res).await; assert_eq!(opened.owner, "alice"); assert_eq!(opened.balance, 1_000); assert_eq!(opened.version, 1); // GET /api/v1/wallets/:id → 200 OK with the same view. - let res = build_router() - .await - .oneshot( - Request::get(&format!("/api/v1/wallets/{}", opened.id)) - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); + let res = send( + &app, + Request::get(&format!("/api/v1/wallets/{}", opened.id)) + .body(Body::empty()) + .unwrap(), + ) + .await; assert_eq!(res.status(), StatusCode::OK); + let fetched: WalletView = body_json(res).await; + assert_eq!(fetched.id, opened.id); + assert_eq!(fetched.balance, 1_000); } ``` -`build_router()` is the testable boot path. It calls -`FireflyApplication::bootstrap()` — the sibling of `run()` that assembles the -app **without serving** — and returns its `api_router`, exactly the -`axum::Router` `main` serves: +What just happened: the `POST` opens a wallet and the framework answers `201` with +the `WalletView`; the `GET` reads the same wallet back and answers `200` with the +matching view. Both requests went through the entire mounted controller stack, +the CQRS dispatch, and the JSON contract — in one process, with no network. -```rust,ignore -// src/web.rs — the in-process router the tests drive (no socket bound). -pub(crate) async fn build_router() -> axum::Router { - firefly::FireflyApplication::new(APP_NAME) - .version(VERSION) - .bootstrap() - .await - .expect("lumen bootstrap") - .api_router -} -``` - -Because `bootstrap()` runs the *same* component scan and auto-mount as `run()`, -the test drives the real, fully-wired controller stack — the macro-generated -routes, the JSON contract, and the status-code mapping — the same code path a -real client hits, minus the network. - -The error paths are tested the same way. An empty `owner` is a `422` problem; an -id that was never opened is a `404` problem — and both assert the -`application/problem+json` content type, so the RFC 9457 contract is part of the -test suite, not just the prose: +The error paths are tested the same way. An id that was never opened is a `404` +problem, and the test asserts the `application/problem+json` content type — so the +RFC 9457 contract is part of the suite, not just the prose: ```rust,ignore #[tokio::test] async fn unknown_wallet_is_404_problem() { - let res = build_router() - .await - .oneshot( - Request::get("/api/v1/wallets/wlt_does_not_exist") - .body(Body::empty()) - .unwrap(), - ) - .await - .unwrap(); + let app = build_router().await; + let res = send( + &app, + Request::get("/api/v1/wallets/wlt_does_not_exist") + .body(Body::empty()) + .unwrap(), + ) + .await; assert_eq!(res.status(), StatusCode::NOT_FOUND); let ct = res.headers().get("content-type").unwrap().to_str().unwrap(); assert!(ct.contains("application/problem+json")); @@ -512,39 +687,69 @@ async fn unknown_wallet_is_404_problem() { > exercises the real request path at full speed and without port contention. > [Testing](./18-testing.md) builds this into a full strategy. +> **Tip** **Checkpoint.** Run `cargo test -p firefly-sample-lumen` and watch the +> round-trip and the 404-problem tests pass against the real, framework-assembled +> router. (The full sample also tests deposit/withdraw, the transfer saga, and the +> security chain — those rely on machinery from later chapters.) + ## Recap — what changed in Lumen | Before | After this chapter | |--------|--------------------| -| an empty public router | a `WalletApi` controller declared with `#[rest_controller]` and two real endpoints | +| an empty public router | a `WalletApi` controller declared with `#[derive(Controller)]` + `#[rest_controller]` and two real endpoints | | no client contract | `POST /api/v1/wallets` → `201` + `WalletView`, `GET /api/v1/wallets/:id` → `200`/`404`, all JSON | -| errors unconsidered | typed `FireflyError` → RFC 9457 `application/problem+json` with the right status | +| errors unconsidered | typed `FireflyError` → RFC 9457 `application/problem+json` with the right status, via `cqrs_to_web` | | nothing to test | a `tower::oneshot` round-trip that drives the full router in-process, content-type assertions included | -The controller is deliberately thin: it speaks HTTP and delegates the wallet -logic to the bus. That seam is what the next several chapters fill in — the read -model the `GET` serves, the domain that enforces the rules, and the CQRS -handlers the `POST` dispatches to. +You also now know: + +- That a controller is *just a bean plus an annotated `impl`* — `#[autowired]` + collaborators in the struct, verb attributes on the methods — and that the macro + derives the routing table from your code. +- That you never mount a controller: `mount_controllers(&container)` resolves and + merges every `#[rest_controller]` at boot, so adding the bean *is* adding the + routes, and `main` never changes. +- That `WebResult` plus a `FireflyError` constructor turns any handler error + into the right `application/problem+json`, with no response-writing by hand. +- That `bootstrap()` is the test seam: `build_router()` drives the fully-wired + router in-process with `tower::oneshot`, no socket bound. + +The controller is deliberately thin: it speaks HTTP and delegates the wallet logic +to the bus. That seam is what the next several chapters fill in — the read model +the `GET` serves, the domain that enforces the rules, and the CQRS handlers the +`POST` dispatches to. ## Exercises 1. **Add a route.** Give `WalletApi` a `#[get("/wallets")]` `list` method that - returns `WebResult>>`. Watch `WalletApi::routes` pick it - up automatically — you never touch a routing table. + returns `WebResult>>`. Run Lumen and watch the new path + appear in the startup report's `:: routes ::` line and in + `WalletApi::routes` — you never touch a routing table. 2. **Shape an error.** Make `cqrs_to_web` (or a small handler of your own) return - `FireflyError::conflict("wallet already closed")` and confirm the response is - a `409` with `application/problem+json`. Try `bad_request` and `forbidden` - too, and read the rendered `type`/`title`/`status` for each. -3. **Honor idempotency.** `POST /api/v1/wallets` twice with the same - `Idempotency-Key` header and identical body; confirm the second response - carries `Idempotent-Replay: true`. Then change the body under the same key and - observe the `409`. You wrote none of this — it came with `apply_middleware`. -4. **Write the round-trip yourself.** Copy the `open_then_get` test, change the - owner and opening balance, and assert the returned `balance` matches. Run - `cargo test -p firefly-sample-lumen` and watch it pass against the real + `FireflyError::conflict("wallet already closed")` and confirm the response is a + `409` with `application/problem+json`. Try `bad_request` and `forbidden` too, + and read the rendered `type`/`title`/`status` for each against the table in + [Step 5](#step-5--map-typed-errors-to-rfc-9457-problems). +3. **Negotiate the format.** Switch the `GET` handler's return type to + `Negotiate` (Step 4), run Lumen, and request the same wallet twice + — once with `Accept: application/json` and once with `Accept: application/xml`. + Confirm one handler serves both wire shapes. +4. **Write the round-trip yourself.** Copy `open_then_get_round_trips_through_cqrs`, + change the owner and opening balance, and assert the returned `balance` matches. + Run `cargo test -p firefly-sample-lumen` and watch it pass against the real router. - -Next, give the `GET` endpoint a real backing store with -[Persistence & Reactive Repositories](./07-persistence.md), then put the rules -behind the bus in [Domain-Driven Design](./08-domain-driven-design.md) and -[CQRS](./09-cqrs.md). +5. **Honor idempotency.** `POST /api/v1/wallets` twice with the same + `Idempotency-Key` header and identical body; confirm the second response + replays the stored result. Then change the body under the same key and observe + the `409`. You wrote none of this — it came with the middleware chain. + +## Where to go next + +- See how the macro turns your `#[rest_controller]` and `#[derive(Schema)]` types + into a live spec in **[OpenAPI & API Docs](./06a-openapi.md)**. +- Give the `GET` endpoint a real backing store with + **[Persistence & Reactive Repositories](./07-persistence.md)**. +- Put the wallet rules behind the bus in + **[Domain-Driven Design](./08-domain-driven-design.md)** and + **[CQRS](./09-cqrs.md)** — the machinery `bus.send(...)` / `bus.query(...)` + dispatch to. diff --git a/docs/book/src/06a-openapi.md b/docs/book/src/06a-openapi.md index 0823561..20d2e8d 100644 --- a/docs/book/src/06a-openapi.md +++ b/docs/book/src/06a-openapi.md @@ -1,31 +1,79 @@ # OpenAPI, Swagger UI & ReDoc -> By the end of this chapter you will know how Lumen serves a complete OpenAPI -> 3.1 document, Swagger UI, and ReDoc with **zero application code**: the spec is -> generated from the live inventory — every `#[rest_controller]` route plus -> every `#[derive(Schema)]` DTO — and `FireflyApplication` mounts the doc -> endpoints during boot. You will learn `#[derive(Schema)]` (and how it honours -> serde renaming), how request/response models are *inferred* from your handler -> signatures, the per-operation metadata attributes, and how to override the -> inference when you need to. - -A `#[rest_controller]` already emits a route descriptor per endpoint (you met -this in [chapter 6](./06-first-http-api.md), and it feeds -`/admin/api/mappings`). The OpenAPI generator reuses that same descriptor table. -The Rust analog of springdoc-openapi, it builds the spec from what is already -compiled in — there is no annotation framework to learn beyond the attributes -you write for routing, and no codegen step. - -## Served, with no app code - -During step 10 of the [boot pipeline](./04b-bootstrap.md), `FireflyApplication` -builds the document from the inventory and mounts it on the **management** -router — beside the actuator and the admin dashboard, on the management port — -**not** the public API. Swagger UI, ReDoc, and the spec expose the entire API -surface and every schema, so they belong on the control-plane surface (where -operators already reach `/actuator/*` and `/admin/`), keeping the public -data-plane port free of API-introspection endpoints. The served paths (on the -management port) default to: +In [Your First HTTP API](./06-first-http-api.md) you gave Lumen its first real +endpoints — a `#[rest_controller]` whose `#[post]` / `#[get]` methods mount +themselves at boot. This chapter shows what those same declarations *also* bought +you, for free: a complete, live **OpenAPI 3.1** document, a **Swagger UI** page, +and a **ReDoc** page, all served without one extra line of application code. The +spec is generated from the live inventory the framework already discovered — every +controller route plus every `#[derive(Schema)]` DTO — and `FireflyApplication` +mounts the doc endpoints during boot. + +Nothing in this chapter changes a single line of `samples/lumen`. The controller +you wrote already carries the summaries, tags, and `#[derive(Schema)]` DTOs the +generator reads. The point here is to *see* that those routing declarations **are** +the API documentation, and to learn how to enrich, override, and export the spec +when you need to. + +By the end of this chapter you will: + +- Reach Lumen's three documentation surfaces — the OpenAPI spec, Swagger UI, and + ReDoc — and explain why they live on the management port, not the public one. +- Derive a reusable component schema from a DTO with `#[derive(Schema)]`, and read + how it honours serde renaming, optionals, enums, and nested types. +- Trace how the request body, the response body, and the path/query/header + parameters of an operation are *inferred* from a handler's own signature. +- Attach per-operation metadata (summary, description, tags, status, `deprecated`) + and override the inference with `request = ` / `response = ` when a signature + can't express the DTO. +- Export the spec with the `firefly` CLI and generate a typed Rust client from it. + +## Concepts you will meet + +Before the first endpoint, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — OpenAPI.** *OpenAPI* (formerly Swagger) is a +> language-neutral, machine-readable description of a REST API — every path, +> operation, parameter, request body, response, and reusable schema, as one JSON +> (or YAML) document. Tooling reads it to render docs, generate clients, and run +> contract tests. Firefly emits **OpenAPI 3.1**. + +> **Note** **Key term — component schema.** A *component schema* is a named, +> reusable JSON Schema for one data type, registered under +> `#/components/schemas/{Type}` and referenced from operations by a `$ref`. The +> Java/Spring analog is a model annotated `@Schema`; in Firefly you opt a type in +> with `#[derive(Schema)]`. + +> **Note** **Key term — Swagger UI / ReDoc.** Both are browser apps that render an +> OpenAPI document into human-readable, interactive documentation. *Swagger UI* has +> a "Try it out" panel that fires live requests; *ReDoc* is a clean three-pane +> reference. Firefly serves both, each pointed at the same spec. + +> **Note** **Key term — the inventory.** Firefly's macros emit compile-time +> descriptors into an `inventory` registry — one `RouteDescriptor` per +> `#[rest_controller]` method and one `SchemaDescriptor` per `#[derive(Schema)]` +> type. The OpenAPI generator reads that registry rather than re-parsing your +> source. This is how a Rust framework gets springdoc-style "scan the application" +> behaviour without runtime reflection. + +## Step 1 — Reach the three documentation surfaces + +You do not write or register anything to get API docs. Boot Lumen exactly as in +the [Quickstart](./02-quickstart.md): + +```bash +cargo run +``` + +Among the startup lines, the framework prints the documentation URLs: + +```text +:: api docs (management) :: swagger-ui http://0.0.0.0:8081/swagger-ui | redoc http://0.0.0.0:8081/redoc | spec http://0.0.0.0:8081/v3/api-docs +``` + +Open each in a browser (or `curl` the spec). The endpoints, on the **management** +port, default to: | Path | Serves | |------|--------| @@ -34,40 +82,84 @@ management port) default to: | `/swagger-ui` and `/swagger-ui.html` | Swagger UI, pointed at the spec | | `/redoc` | ReDoc, pointed at the spec | -`serve` even prints the URLs on boot: +What just happened: during the boot pipeline (the docs-mounting stage you met in +[Bootstrap](./04b-bootstrap.md)), `FireflyApplication` built one OpenAPI document +from the live inventory and merged a small router serving these paths onto the +management surface. There is no annotation framework to learn beyond the routing +attributes from chapter 6, and no codegen step. This is the Rust counterpart of +springdoc-openapi. + +> **Tip** **Checkpoint.** With `cargo run` running, `curl +> localhost:8081/v3/api-docs` returns a JSON body beginning with +> `{"openapi":"3.1.0",...}`, and `http://localhost:8081/swagger-ui` renders the +> wallet API in a browser. If `curl` connects but 404s, confirm you are hitting +> `8081` (management), not `8080` (public). + +## Step 2 — Understand why the docs live on the management port + +Notice the URLs above are all on `:8081`, the management port — beside the actuator +and the admin dashboard — **not** on the public API at `:8080`. + +> **Note** **Key term — management surface.** The *management surface* is the set +> of operational HTTP endpoints — health, info, metrics, admin, and now the API +> docs — served on a separate port from your business API, for operators and +> tooling rather than end users. This mirrors Spring Boot Actuator's dedicated +> management port. + +Why split them: Swagger UI, ReDoc, and the raw spec expose your **entire** API +surface and every schema — a control-plane concern. They belong where operators +already reach `/actuator/*` and `/admin/`, keeping the public data-plane port free +of API-introspection endpoints. + +That split creates one wrinkle the framework solves for you. Because the docs are +*loaded* from the management origin (`:8081`) but the API *answers* on the public +port (`:8080`), the document declares the **public API base URL** as its OpenAPI +`server`. So Swagger UI's "Try it out" and ReDoc's samples target the API +(`:8080`), not the management origin they were loaded from. `FireflyApplication` +derives that URL from the API bind address — a wildcard host like `0.0.0.0` is not +client-usable, so it falls back to `localhost`: ```text -:: api docs (management) :: swagger-ui http://0.0.0.0:8081/swagger-ui | redoc http://0.0.0.0:8081/redoc | spec http://0.0.0.0:8081/v3/api-docs +http://localhost:8080 +``` + +Behind a reverse proxy you want a real public URL instead. Set +`FIREFLY_OPENAPI_SERVER_URL` and it overrides the derived value: + +```bash +FIREFLY_OPENAPI_SERVER_URL=https://api.lumen.example cargo run ``` -Internally this is -`firefly_openapi::Builder::new(info).add_server(api_url).from_inventory()` — -`from_inventory()` reads `firefly_container::routes()` (every controller route) -and `firefly_container::schemas()` (every `#[derive(Schema)]` DTO) — rendered by -`docs_router(&DocsConfig::default())`. The defaults above come from -`DocsConfig`; you would only touch them to relocate the doc endpoints. Lumen -touches nothing: the document just appears. - -Because the docs are served on the management port but the API answers on the -public port, the document declares the **API base URL** as its OpenAPI `server` -— so Swagger UI's *Try it out* and ReDoc's samples target the API (`:8080`), -not the management origin they were loaded from. `FireflyApplication` derives it -from the API bind address (a wildcard host falls back to `localhost`); -`FIREFLY_OPENAPI_SERVER_URL` overrides it for a public URL behind a reverse -proxy. An unknown path on **either** listener answers the same RFC 9457 -`application/problem+json` 404. - -## `#[derive(Schema)]` — component schemas - -A DTO becomes a reusable `#/components/schemas/{Type}` by deriving `Schema`. -Because Rust has no runtime reflection, the JSON Schema is computed **at macro- -expansion time** by walking the struct's fields. Lumen's wallet view: +What just happened: the spec's `servers[0].url` becomes the value you supplied, so +every "Try it out" call goes to your public hostname. (An unknown path on **either** +listener still answers the same RFC 9457 `application/problem+json` 404 you met in +[chapter 6](./06-first-http-api.md), so the docs surface degrades cleanly too.) + +> **Tip** **Checkpoint.** `curl -s localhost:8081/v3/api-docs | jq '.servers'` +> shows one entry whose `url` is `http://localhost:8080` by default — the public +> API, not the `:8081` origin you fetched from. + +## Step 3 — Turn a DTO into a component schema with `#[derive(Schema)]` + +A data type becomes a reusable `#/components/schemas/{Type}` by deriving `Schema`. +Because Rust has no runtime reflection, the JSON Schema is computed **at +macro-expansion time** by walking the struct's fields — so what ends up in the spec +is decided when you compile, not at boot. + +> **Note** **Key term — `#[derive(Schema)]`.** This derive is the Rust analog of a +> Spring `@Schema` model. It reads the struct (or field-less enum) at compile time, +> emits a JSON Schema fragment, and submits it to the inventory so the generator +> can register it as a named component and `$ref` it from operations. + +Here is Lumen's read-model view, exactly as you wrote it in `src/domain.rs`: ```rust,ignore // src/domain.rs #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Schema)] pub struct WalletView { + /// The wallet id. pub id: String, + /// The owner's display name. pub owner: String, /// The current balance, in minor units (cents). pub balance: i64, @@ -76,7 +168,7 @@ pub struct WalletView { } ``` -expands to a registered schema (roughly): +The derive walks the four fields and registers a schema equivalent to: ```json { "type": "object", @@ -89,26 +181,34 @@ expands to a registered schema (roughly): "required": ["id", "owner", "balance", "version"] } ``` -The field-type mapping mirrors what a Java/Spring `@Schema` model produces: +What just happened: each `String` field became `{"type":"string"}`, each `i64` +became `{"type":"integer"}`, and — because none of them is wrapped in `Option` — +all four landed in `required`. The mapping mirrors what a Java/Spring `@Schema` +model produces: -- `String`/`str`/`char` → `string`; `bool` → `boolean`; every integer type → - `integer`; `f32`/`f64` → `number`. -- `Uuid` → `string`/`format: uuid`; chrono / time date-times → `string`/`format: - date-time`; dates → `format: date`; times → `format: time`. +- `String` / `str` / `char` → `string`; `bool` → `boolean`; every integer type + (`i8`…`u128`, `usize`, …) → `integer`; `f32` / `f64` → `number`. +- `Uuid` → `string` with `format: uuid`; chrono / time date-times → `string` with + `format: date-time`; dates → `format: date`; times → `format: time`. - `Option` is a transparent wrapper: it describes `T` but makes the property **non-required** (so optionals drop out of the `required` list). -- `Box`/`Arc`/`Rc` are transparent too; `Vec`/`HashSet`/`BTreeSet`/… → an - `array` of the element schema; `HashMap`/`BTreeMap` → an open `object` with - `additionalProperties`. -- Any *other* named type is assumed to be a sibling DTO that also derives - `Schema`, and is emitted as a `$ref` — so a nested DTO is **linked**, not - inlined, and the two component schemas compose. +- `Box` / `Arc` / `Rc` are transparent too; `Vec` / `HashSet` / + `BTreeSet` / … → an `array` of the element schema; `HashMap` / `BTreeMap` → an + open `object` with `additionalProperties`. +- Any *other* named type is assumed to be a sibling DTO that also derives `Schema`, + and is emitted as a `$ref` — so a nested DTO is **linked**, not inlined, and the + two component schemas compose. + +> **Tip** **Checkpoint.** `curl -s localhost:8081/v3/api-docs | jq +> '.components.schemas.WalletView'` prints the object schema above. Every DTO that +> derives `Schema` shows up under `.components.schemas`. ### Serde renaming is honoured `#[derive(Schema)]` reads the struct's serde directives so the property names in -the schema match the JSON wire shape — `rename`, `rename_all`, and `skip`. -Lumen's `TransferResult` carries field renames, and the schema follows them: +the schema match the JSON **wire** shape — `rename`, `rename_all`, and `skip` — not +the Rust idents. Lumen's `TransferResult` carries field renames, and the schema +follows them: ```rust,ignore // src/transfer.rs @@ -126,63 +226,59 @@ pub struct TransferResult { ``` The schema names the array properties `stepsExecuted` / `stepsRolledBack` — the -exact JSON the handler serializes — not the snake_case Rust idents. A -struct-level `#[serde(rename_all = "camelCase")]` is applied to every field the -same way, and a `#[serde(skip)]` field is omitted from the schema entirely. +exact JSON the handler serialises — not the snake_case Rust idents. A struct-level +`#[serde(rename_all = "camelCase")]` is applied to every field the same way, and a +`#[serde(skip)]` field is omitted from the schema entirely. The rule of thumb: the +schema describes what goes on the wire, so it always matches your serialised JSON. -### Field-less enums → string enumerations +> **Design note.** This is why the schema is wire-accurate without you maintaining a +> second copy of the field names: the one set of serde attributes that controls +> serialisation also controls the schema. There is no separate annotation to keep in +> sync, and no way for the docs to drift from the bytes. -A field-less (unit-variant) enum that derives `Schema` emits a JSON Schema -`string` enumeration, the springdoc treatment of a Java `enum`. Serde renaming -is honoured, so the allowed values match the wire shape: +### Field-less enums become string enumerations + +A field-less (unit-variant) enum that derives `Schema` emits a JSON Schema `string` +enumeration — springdoc's treatment of a Java `enum`. Serde renaming is honoured +here too, so the allowed values match the wire shape. The layered `lumen-ledger` +sample models a wallet's lifecycle this way: ```rust,ignore -#[derive(Serialize, Deserialize, Schema)] +// lumen-ledger: interfaces/.../wallet_status.rs +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize, Schema)] #[serde(rename_all = "lowercase")] -pub enum WalletStatus { Active, Frozen, Closed } +pub enum WalletStatus { + #[default] + Active, + Frozen, + Closed, +} ``` +registers: + ```json "WalletStatus": { "type": "string", "enum": ["active", "frozen", "closed"] } ``` -A DTO field of this type then `$ref`s the registered enum component rather than -becoming an untyped string — the layered `lumen-ledger` sample uses exactly this -for `WalletResponse.status`. +What just happened: each variant became one allowed string, lowercased by the +struct-level `rename_all`. A DTO field of this type then `$ref`s the registered enum +component rather than becoming an untyped string — `lumen-ledger`'s `WalletResponse` +uses exactly this for its `status: WalletStatus` field, so the two component schemas +compose. (`#[derive(Schema)]` supports only field-less enums; an enum with data in a +variant is rejected at compile time.) -## Request/response inference from the handler signature +## Step 4 — Let the macro infer request and response models -You do **not** name request and response models on the attribute — the macro -infers them from the handler's own signature: +You do **not** name request and response models on the verb attribute. The macro +infers them from the handler's own signature, at compile time: - the **request body** is the inner type of the first `Json` *or* `Valid` parameter (so the validating extractor documents its body too), and -- the **response** is the `Json` found inside the return type, after - unwrapping `WebResult<…>` / `Result<…>` and looking through a - `(StatusCode, Json)` tuple. - -### Parameters: path, query, header +- the **response** is the `Json` found inside the return type, after unwrapping + `WebResult<…>` / `Result<…>` and looking through a `(StatusCode, Json)` tuple. -The same signature-driven inference covers operation **parameters**, so Swagger -UI / ReDoc render an input for each — no hand-written parameter list: - -- **Path** parameters come from the route template: every `:id` / `{id}` segment - becomes a required `in: path` parameter. -- **Query** parameters come from a `Query` / `ValidQuery` extractor — the - generator expands `T`'s `#[derive(Schema)]` fields into one `in: query` - parameter each (required iff the field is non-optional). A `PageRequest` - argument adds the standard `page` / `size` / `sort` query parameters. -- **Header** parameters are declared on the verb attribute: - `#[post("/wallets", header("Idempotency-Key", required, description = "…"))]` - emits an `in: header` parameter (and the handler reads it like any axum - header). `query("…")` declares an extra query parameter the same way. - -So Lumen's `GET /wallets/page` shows `status` (from its `Query`) -plus `page`/`size`/`sort` (from `PageRequest`), and `POST /wallets` shows its -`CreateWalletRequest` body plus the `Idempotency-Key` header — all in Swagger UI, -with zero parameter boilerplate. - -Take Lumen's `open` handler: +Take Lumen's `open` handler, unchanged from chapter 6: ```rust,ignore // src/web.rs @@ -201,24 +297,87 @@ async fn open( } ``` -From the signature alone the macro records `OpenWallet` as the request schema -(the `Json` parameter) and `WalletView` as the response schema (the -`Json` inside the `(StatusCode, …)` tuple inside `WebResult<…>`). -Both are `#[derive(Schema)]` types, so the operation `$ref`s -`#/components/schemas/OpenWallet` and `#/components/schemas/WalletView`. +What just happened: from the signature alone the macro recorded `OpenWallet` as the +request schema (the `Json` parameter) and `WalletView` as the response +schema (the `Json` inside the `(StatusCode, …)` tuple inside +`WebResult<…>`). Both derive `Schema`, so the operation `$ref`s +`#/components/schemas/OpenWallet` and `#/components/schemas/WalletView` — no +request/response declaration on the attribute. + +> **Note** A `$ref` is emitted **only** when the inferred type is actually a +> registered `#[derive(Schema)]` component. Lumen's `transfer_compliance` returns +> `Json`; `serde_json::Value` is not a registered schema, so the +> generator emits no request/response `$ref` for it rather than referencing a +> component that does not exist. The document stays valid no matter what your +> handlers return — there are never dangling `$ref`s. + +> **Tip** **Checkpoint.** `curl -s localhost:8081/v3/api-docs | jq +> '.paths."/api/v1/wallets".post.requestBody'` shows a `$ref` to +> `#/components/schemas/OpenWallet`, and the `201` response `$ref`s `WalletView`. + +### Path, query, and header parameters are inferred too + +The same signature-driven inference covers operation **parameters**, so Swagger UI +and ReDoc render an input for each — with no hand-written parameter list: + +- **Path** parameters come from the route template: every `:id` (axum) / `{id}` + segment becomes a required `in: path` parameter. Lumen's `GET /wallets/:id` gets + a required `id` path parameter automatically. +- **Query** parameters come from a `Query` / `ValidQuery` extractor — the + generator expands `T`'s `#[derive(Schema)]` fields into one `in: query` parameter + each (required iff the field is non-optional). A `PageRequest` argument adds the + standard Spring Data `page` / `size` / `sort` query parameters. +- **Header** parameters are declared on the verb attribute: + `header("Idempotency-Key", required, description = "…")` emits an `in: header` + parameter (and the handler reads it like any axum header). A `query("…")` + declaration adds an extra query parameter the same way. + +Lumen's `WalletApi` keeps its handlers simple — path-only — so its parameter +inference is just the `:id` segments. The richer query/header story is what the +layered `lumen-ledger` sample's `WalletController` exercises. Its paged-list +endpoint binds a filter query *and* the framework's pagination resolver: + +```rust,ignore +// lumen-ledger: web/.../wallet_controller.rs +#[get("/wallets/page", summary = "List wallets by status (paged)")] +async fn list_paged( + State(api): State, + Query(query): Query, + PageRequest(pageable): PageRequest, +) -> WebResult>> { + let page = api.service.list_by_status(query.status, pageable).await.map_err(service_to_web)?; + Ok(Json(page)) +} +``` + +What just happened: `Query` expanded the one `status` field of the +`StatusQuery` schema into an `in: query` parameter, and `PageRequest` added `page`, +`size`, and `sort` — so Swagger UI renders four query inputs for this endpoint with +zero parameter boilerplate. Its `open` handler shows the header form, declaring an +`Idempotency-Key` request header right on the verb attribute: -> **No dangling `$ref`s.** A `$ref` is emitted **only** when the inferred type -> is actually a registered `#[derive(Schema)]` component. Lumen's -> `transfer_compliance` returns `Json`; `serde_json::Value` -> is not a registered schema, so the generator emits no request/response `$ref` -> for it rather than referencing a component that does not exist. The document -> stays valid no matter what your handlers return. +```rust,ignore +// lumen-ledger: web/.../wallet_controller.rs +#[post( + "/wallets", + summary = "Open a wallet", + status = 201, + header("Idempotency-Key", description = "optional client-supplied key to make retries safe") +)] +async fn open(/* … */) -> WebResult<(StatusCode, Json)> { /* … */ } +``` + +— so callers see and can fill the header in Swagger UI, and the handler reads it +from the `HeaderMap` like any other header. -## Per-operation metadata +## Step 5 — Attach per-operation metadata Beyond the path, each verb attribute takes optional metadata that lands on the -OpenAPI operation. The full form is -`#[get("/x", summary = "…", description = "…", tags = ["A", "B"], status = 200, deprecated, request = T, response = T)]`: +OpenAPI operation. The full form is: + +```text +#[get("/x", summary = "…", description = "…", tags = ["A", "B"], status = 200, deprecated, request = T, response = T)] +``` | Argument | Effect on the operation | |----------|-------------------------| @@ -230,8 +389,8 @@ OpenAPI operation. The full form is | `request = T` | the request body schema name — overrides inference | | `response = T` | the success response schema name — overrides inference | -Lumen's `transfer` operation uses summary, description, an explicit `tags`, and -a `status`: +Lumen's `transfer` operation uses summary, description, an explicit `tags`, and a +`status`: ```rust,ignore // src/web.rs @@ -248,10 +407,16 @@ async fn transfer( ) -> WebResult> { /* … */ } ``` +What just happened: the macro stamped the summary, description, `Transfers` tag, and +`200` status onto the operation, then *still* inferred `TransferRequest` and +`TransferResult` from the signature. Metadata and inference compose — you only spell +out what the signature cannot. + ### Controller-level tags -`#[rest_controller(tag = "…")]` sets a default tag for **every** operation on -the controller — Spring's `@Tag(name = …)`. Lumen tags its whole wallet surface: +`#[rest_controller(tag = "…")]` sets a default tag for **every** operation on the +controller — the analog of Spring's `@Tag(name = …)`. Lumen tags its whole wallet +surface: ```rust,ignore // src/web.rs @@ -259,94 +424,123 @@ the controller — Spring's `@Tag(name = …)`. Lumen tags its whole wallet surf impl WalletApi { /* … */ } ``` -Tag resolution per operation is: an explicit per-method `tags = [...]` wins; -otherwise the `#[rest_controller(tag)]` applies; otherwise the generator derives -a tag from the controller type name by stripping a trailing `Api` / `Controller` -suffix (`WalletApi` → `Wallet`, `CatalogController` → `Catalog`). Lumen sets the -controller tag explicitly to `"Wallets"`, so in its spec `open`, `get`, -`deposit`, and `withdraw` carry the **Wallets** tag (the controller default), +Tag resolution per operation is layered: + +1. an explicit per-method `tags = [...]` wins; otherwise +2. the `#[rest_controller(tag)]` default applies; otherwise +3. the generator derives a tag from the controller type name by stripping a + trailing `Api` / `Controller` suffix (`WalletApi` → `Wallet`, `CatalogController` + → `Catalog`). + +Lumen sets the controller tag explicitly to `"Wallets"`, so in its spec `open`, +`get`, `deposit`, and `withdraw` carry the **Wallets** tag (the controller default), while `transfer`, `transfer_compliance`, and `transfer_2pc` carry **Transfers** -(their per-method `tags = ["Transfers"]` override). +(their per-method `tags = ["Transfers"]` override). Swagger UI groups the operations +under those two headings. ### Overriding inference with `request = ` / `response = ` When a body type cannot be read from the signature — a handler that takes a raw -`axum::body::Bytes`, returns an `impl IntoResponse`, or otherwise hides its DTO -— name it explicitly. `request = T` / `response = T` take the schema name (the -type's last path segment, matching what `#[derive(Schema)]` registers it under) -and **take precedence** over inference: +`axum::body::Bytes`, returns an `impl IntoResponse`, or otherwise hides its DTO — +name it explicitly. `request = T` / `response = T` take the schema **name** (the +type's last path segment, matching what `#[derive(Schema)]` registers it under) and +**take precedence** over inference: ```rust,ignore #[post("/import", summary = "Bulk import", request = ImportBatch, response = ImportReport)] async fn import(/* a non-Json body */) -> impl axum::response::IntoResponse { /* … */ } ``` -Lumen never needs this — every handler's body is a `Json` of a -`#[derive(Schema)]` type, so the inference covers it — but the escape hatch is -there for the cases the signature cannot express. +What just happened: even though the signature reveals no `Json`, the operation +now `$ref`s `ImportBatch` and `ImportReport` (provided both derive `Schema`). Lumen +never needs this — every handler's body is a `Json` of a `#[derive(Schema)]` +type, so the inference covers it — but the escape hatch is there for the cases a +signature cannot express. -## The worked example, end to end +## Step 6 — Read the worked example end to end Putting it together for Lumen's `WalletApi`: - The controller is `#[rest_controller(path = "/api/v1", tag = "Wallets")]`. -- The DTOs — `OpenWallet`, `WalletView`, `AmountBody`, `TransferRequest`, - `TransferResult`, `TccTransferResult` — each `#[derive(Schema)]`, so they - become `#/components/schemas/*` and are `$ref`ed by the operations. -- Each operation's request/response is inferred from its `Json` - parameter/return, its summary/description/tags/status come from the verb - attribute, and the `transfers/*` operations group under **Transfers**. -- `transfer_compliance` takes `Json` (a registered schema, so - the request `$ref`s `TransferRequest`) but returns `Json`, - so its response carries **no** `$ref` — and that is correct, not a gap. +- Its `#[derive(Schema)]` DTOs — `OpenWallet`, `WalletView`, `AmountBody`, + `TransferRequest`, `TransferResult`, `TccTransferResult` — each become a + `#/components/schemas/*` entry and are `$ref`ed by the operations that use them. +- Each operation's request/response is inferred from its `Json` parameter and + return; its summary / description / tags / status come from the verb attribute; + and the `transfers/*` operations group under **Transfers**. +- `transfer_compliance` takes `Json` (a registered schema, so its + request `$ref`s `TransferRequest`) but returns `Json`, so its + response carries **no** `$ref` — and that is correct, not a gap. Every operation also gets a `default` RFC 9457 response referencing -`#/components/schemas/ProblemDetail` (always added to the document), so the -uniform error shape from [chapter 6](./06-first-http-api.md) is documented for -every endpoint automatically. +`#/components/schemas/ProblemDetail`, which the generator always adds to the +document. So the uniform error shape from [chapter 6](./06-first-http-api.md) is +documented for every endpoint automatically — Swagger UI shows an error response on +each operation without you writing one. -## Consistency: one source of truth +> **Tip** **Checkpoint.** `curl -s localhost:8081/v3/api-docs | jq +> '.components.schemas | keys'` lists every registered schema, including +> `ProblemDetail`. `jq '.paths."/api/v1/transfers".post.responses | keys'` shows +> both the `200` and the `default` (problem) response. -The `#[rest_controller]` descriptor table is read by **three** surfaces, so they -can never drift: +## Step 7 — One descriptor table, three surfaces + +The `#[rest_controller]` descriptor table is read by **three** surfaces, so they can +never drift: - the **OpenAPI document** at `/v3/api-docs`, - the admin dashboard's **`/admin/api/mappings`** route table - ([chapter 15](./15-observability.md)), and + ([Observability & Admin](./15-observability.md)), and - the **startup report**'s `:: routes (N) ::` block - ([the bootstrap chapter](./04b-bootstrap.md)). + ([Bootstrap](./04b-bootstrap.md)). Add a route, and all three update from the same registry on the next build. The -startup report even prints the operation + component-schema counts -(`:: openapi :: N operations | K component schemas (served at /v3/api-docs) ::`) -so you can confirm the spec is live. +startup report even prints the operation and component-schema counts so you can +confirm the spec is live: + +```text +:: openapi :: N operations | K component schemas (served at /v3/api-docs) :: +``` -## Exporting the spec with the CLI +What just happened: because the document, the admin mappings view, and the boot log +all read one inventory, "what the API does" has a single source of truth. There is +no second hand-maintained spec file to fall behind your code. -The `firefly` CLI can write an OpenAPI document for tooling / CI: +## Step 8 — Export the spec with the CLI + +The `firefly` CLI can write an OpenAPI document for tooling and CI: ```bash -firefly openapi # OpenAPI 3.1 JSON to stdout +firefly openapi # OpenAPI 3.1 JSON to stdout firefly openapi --format yaml -o openapi.yaml ``` -A note on scope (covered in [chapter 19](./19-cli.md)): a *compiled* binary -cannot boot an arbitrary app to enumerate its live routes, so the CLI emits a -metadata-stamped **skeleton** (the `info` block, the `ProblemDetail` component, -empty `paths`) read from `firefly.yaml` / `Cargo.toml`. To capture Lumen's -**real** routes, run the service and fetch `/v3/api-docs` — that document, built -by `from_inventory()`, *is* the live spec: +There is a scope caveat worth understanding (covered in full in [The +CLI](./19-cli.md)). A *compiled* binary cannot boot an arbitrary application to +enumerate its live routes — the routes live in the consumer's own crate, and there +is no DI container to introspect from a generic tool. So `firefly openapi` emits a +metadata-stamped **skeleton**: the `info` block (read from `firefly.yaml` / +`Cargo.toml`), the always-present `ProblemDetail` component, and empty `paths`. The +wire shape is identical to what a live app serves — only the route list is blank. + +To capture Lumen's **real** routes, run the service and fetch `/v3/api-docs`. That +document, built by the framework's `from_inventory()`, *is* the live spec: ```bash cargo run --bin lumen & curl -s http://localhost:8081/v3/api-docs | jq . ``` -## Generating a client from the spec — `firefly openapi-client` +> **Tip** **Checkpoint.** `firefly openapi | jq '.openapi'` prints `"3.1.0"` even +> outside a running app, and `jq '.components.schemas.ProblemDetail'` is present. +> The skeleton's `paths` is `{}`; the live spec at `:8081/v3/api-docs` has your +> wallet routes filled in. + +## Step 9 — Generate a typed client from the spec -The inverse direction: given an OpenAPI document, generate a typed Rust client -over [`firefly_client::RestClient`] — the Rust analog of springdoc's -OpenAPI-generated WebClient SDK. +The inverse direction: given an OpenAPI document, generate a typed Rust client over +the framework's `RestClient` — the Rust analog of springdoc's OpenAPI-generated +WebClient SDK. ```bash # capture the live spec, then generate a client from it @@ -354,19 +548,84 @@ curl -s http://localhost:8081/v3/api-docs -o wallet-openapi.json firefly openapi-client --spec wallet-openapi.json -o src/generated.rs --client-name WalletClient ``` -It emits a model `struct` per object schema (and an `enum` per string -enumeration), with serde renames and optional fields preserved, plus one -`async fn` per operation — typed path/query parameters, a JSON request body, and -the success-response type — calling `RestClient::request`. The generated client -is the same shape you would hand-write (see the `lumen-ledger` `-sdk` crate and -[chapter 22](./22-layered-microservices.md)). - -## What changed in Lumen - -Nothing in `samples/lumen` — the controller already carried its tags, summaries, -and `#[derive(Schema)]` DTOs for routing's sake, and `FireflyApplication` -already serves the docs. This chapter is where you see that those same -declarations *are* the API documentation: write the controller, get Swagger UI, -ReDoc, and a valid OpenAPI 3.1 spec for free. Next, -[Persistence & Reactive Repositories](./07-persistence.md) builds the read model -behind the `WalletView` these docs describe. +What just happened: the generator walked the spec and emitted a model `struct` per +object schema (and an `enum` per string enumeration), with serde renames and +optional fields preserved, plus one `async fn` per operation — typed path/query +parameters, a JSON request body, and the success-response type — each calling +`RestClient` under the hood. The generated client is the same shape you would +hand-write; the layered `lumen-ledger` sample ships exactly such an SDK, which you +will meet in [Layered Microservices](./22-layered-microservices.md). + +> **Tip** **Checkpoint.** After the second command, `src/generated.rs` exists and +> contains a `pub struct WalletClient` plus `WalletResponse` / `WalletStatus` models +> mirroring the spec's component schemas. Unmapped shapes degrade to +> `serde_json::Value` rather than failing the generation. + +## Recap + +In this chapter you saw that the controller you already wrote *is* the API +documentation: + +- Firefly serves a live **OpenAPI 3.1** spec (`/v3/api-docs`, aliased + `/openapi.json`), **Swagger UI** (`/swagger-ui`), and **ReDoc** (`/redoc`) on the + **management** port, built from the inventory at boot with zero application code. +- The spec advertises the **public API base URL** as its `server` (falling back to + `localhost`, overridable with `FIREFLY_OPENAPI_SERVER_URL`), so "Try it out" + targets the API, not the docs origin. +- `#[derive(Schema)]` turns a DTO into a `#/components/schemas/{Type}` at + macro-expansion time, honouring serde `rename` / `rename_all` / `skip`, treating + `Option` / `Box` / `Arc` / `Rc` as transparent, mapping collections to arrays and + maps, `$ref`-ing nested DTOs, and rendering field-less enums as string + enumerations. +- Request bodies, response bodies, and path/query/header parameters are **inferred** + from the handler signature (`Json` / `Valid` bodies, `Query` and + `PageRequest` query params, `:id` path segments, declared `header(...)` params) — + with a `$ref` emitted only for actually-registered schemas, so the document never + dangles. +- Per-operation metadata (`summary`, `description`, `tags`, `status`, `deprecated`) + and the controller-level `tag` shape each operation; `request = ` / `response = ` + override the inference when a signature cannot express the DTO. +- Every operation carries a `default` RFC 9457 `ProblemDetail` response, so the + uniform error contract is documented automatically. +- One descriptor table feeds the spec, the `/admin/api/mappings` view, and the + startup report — a single source of truth — and the `firefly openapi` / + `openapi-client` commands export the spec and generate a typed client from it. + +Nothing in `samples/lumen` changed: the routing declarations you already wrote +produced Swagger UI, ReDoc, and a valid OpenAPI 3.1 spec for free. + +## Exercises + +1. **Read the live spec.** With `cargo run` running, `curl -s + localhost:8081/v3/api-docs | jq '.paths | keys'`. Confirm every wallet and + transfer route from chapter 6 is present, then `jq '.components.schemas | keys'` + to see each `#[derive(Schema)]` DTO plus `ProblemDetail`. +2. **Watch a rename flow through.** In `jq`, inspect + `.components.schemas.TransferResult.properties` and confirm the property keys are + `stepsExecuted` / `stepsRolledBack` (the serde wire names), not the snake_case + idents. Then temporarily remove one `#[serde(rename = "…")]` in `src/transfer.rs`, + rebuild, and watch the schema property name change. +3. **Move the server URL.** Start Lumen with + `FIREFLY_OPENAPI_SERVER_URL=https://api.lumen.example cargo run`, then + `curl -s localhost:8081/v3/api-docs | jq '.servers'`. Confirm the URL changed — + this is the value Swagger UI's "Try it out" will call. +4. **Deprecate an operation.** Add the bare `deprecated` flag to one verb attribute + in `src/web.rs` (e.g. `#[post("/wallets/:id/withdraw", summary = "Withdraw funds", + status = 200, deprecated)]`), rebuild, and confirm + `jq '.paths."/api/v1/wallets/{id}/withdraw".post.deprecated'` is `true` and + Swagger UI strikes the operation through. +5. **Export and diff.** Run `firefly openapi --format yaml -o skeleton.yaml`, then + `curl -s localhost:8081/v3/api-docs | jq . > live.json`. Note that the CLI + skeleton has empty `paths` while the live document carries your routes — and that + both share the same `info` block and `ProblemDetail` component. + +## Where to go next + +- Build the read model behind the `WalletView` these docs describe in + **[Persistence & Reactive Repositories](./07-persistence.md)**. +- See where the OpenAPI document is built and mounted in the boot pipeline in + **[Bootstrap](./04b-bootstrap.md)**, and the `/admin/api/mappings` view it shares a + source with in **[Observability & Admin](./15-observability.md)**. +- Consume an OpenAPI-generated client against a real upstream service in **[Layered + Microservices](./22-layered-microservices.md)**, building on **[HTTP + Clients](./13-http-clients.md)**. diff --git a/docs/book/src/07-persistence.md b/docs/book/src/07-persistence.md index dad2a2d..947e7a6 100644 --- a/docs/book/src/07-persistence.md +++ b/docs/book/src/07-persistence.md @@ -1,40 +1,83 @@ # Persistence & Reactive Repositories -Lumen has a wallet API that returns a `WalletView` — but where does that view -come from? At the end of [Your First HTTP API](./06-first-http-api.md) the -answer was "an in-memory map." This chapter gives Lumen a real persistence -vocabulary: the **repository pattern** as the seam between the query side and -storage, the in-memory baseline that keeps the teaching build infrastructure-free, -and the pluggable reactive-repository / SQL upgrade path that swaps that baseline -for Postgres, MySQL, SQLite, or MongoDB without touching a call site. - -> **By the end of this chapter, Lumen will** have its query-side read store — -> the `ReadModel` — framed as a repository: a map of wallet id → `WalletView` -> that the `GetWallet` query reads from and the event projection (built in -> [EDA](./10-eda-messaging.md)) writes to. You will see the in-memory baseline -> that ships in `samples/lumen`, and exactly which `firefly-data` adapter you -> would drop in to make it durable — the book's recurring *swap the adapter* -> move. - -`firefly-data` provides the framework's persistence vocabulary: a composable -`Filter` query DSL, a `Page` paged-result envelope, a blocking -`Repository` contract, and — built on `firefly-reactive` — a **reactive -CRUD surface** that returns `Mono` / `Flux` and streams rows lazily. This -chapter covers both, with a real streaming SQL repository. - -> **Design note.** Firefly's data layer is the Repository pattern, expressed as +Lumen already has a wallet API that returns a `WalletView` — but where does that +view come from? At the end of [Your First HTTP API](./06-first-http-api.md) the +honest answer was "an in-memory map." This chapter gives Lumen a real +persistence vocabulary and shows the exact upgrade path from that teaching map to +a durable database — without touching a single call site. + +The throughline is one move the book repeats: *depend on the contract, swap the +backend.* Lumen's read store is already framed as a repository; here you learn +the framework contract it is a miniature of, the reactive CRUD surface that +streams rows lazily, the relational and document adapters that implement that +surface over Postgres / MySQL / SQLite / MongoDB, and the transaction boundary +that makes a multi-write change atomic. The teaching build stays +infrastructure-free the whole way — every durable piece is exercised against an +in-memory SQLite or in-process double, so nothing here needs a running server. + +By the end of this chapter you will: + +- Explain the **repository pattern** as the seam between the query side and + storage, and recognise Lumen's `ReadModel` as a hand-rolled repository in + miniature. +- Compose a `Filter` query, render it to parameterized SQL, and read a `Page` + paged-result envelope. +- Drive the **reactive CRUD surface** — `Mono` / `Flux` repositories — against an + in-memory double and a real streaming SQLite/Postgres adapter. +- Declare a repository the Spring Data way with `#[derive(Entity)]` + + `#[derive(SqlxRepository)]`, and add derived and custom queries with + `#[firefly::repository]`. +- Turn on **optimistic locking** and build a pool from configuration with one + awaited `auto_configure` call. +- Make a multi-write change atomic with `#[firefly::transactional]` and its + ambient enlistment. + +## Concepts you will meet + +Before the first listing, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — repository.** A *repository* is an object that hides how +> entities are stored behind a small, intention-revealing set of operations — +> `find_by_id`, `save`, `delete`. Callers depend on the repository *interface*, +> not on SQL or a `HashMap`. This is exactly Spring Data's `Repository` / +> `CrudRepository`. + +> **Note** **Key term — port and adapter.** A *port* is an interface your code +> depends on; an *adapter* is a concrete implementation chosen at wiring time. +> `firefly-data` owns the ports (the repository traits, the query DSL); +> `firefly-data-sqlx` and `firefly-data-mongodb` are adapters. Swapping the +> adapter swaps the database with no change to call sites — this is *hexagonal +> architecture* (ports & adapters). + +> **Note** **Key term — `Mono` / `Flux`.** These are the reactive *publishers* +> from [The Reactive Model](./05-reactive-model.md): a `Mono` resolves to at +> most one value, a `Flux` to a lazy, backpressured stream of many. The +> reactive repository returns them so a database read can stream row-by-row to +> the client. If you have used a reactive-streams library (Project Reactor, +> RxJava), they are the same `Mono` / `Flux`. + +> **Note** **Key term — optimistic locking.** A concurrency strategy where each +> row carries a *version* number; a write succeeds only if the version it loaded +> still matches the stored one, otherwise it is rejected rather than silently +> overwriting a concurrent change. This is Spring Data's `@Version`. + +> **Design note.** Firefly's data layer is the Repository pattern expressed as > idiomatic Rust: you depend on a trait — `Repository` (blocking) or > `ReactiveCrudRepository` (reactive) — and an adapter supplies the SQL. -> Depend on the port, swap the backend. If you have used a reactive-streams -> library before, the `Mono` / `Flux` surface will feel familiar. +> Depend on the port, swap the backend. `firefly-data` itself owns no driver and +> implies no SQL engine; that is what makes the swap mechanical. -## Lumen's read store, as a repository +## Step 1 — See Lumen's read store as a repository -Lumen splits its write model from its read model (the CQRS shape you build out -in [CQRS](./09-cqrs.md)). The write side is the event-sourced `Ledger`; the read +Lumen splits its write model from its read model — the +Command/Query Responsibility Segregation (CQRS) shape you build out in +[CQRS](./09-cqrs.md). The write side is the event-sourced `Ledger`; the read side is a flat, query-optimized `ReadModel` that `GET /api/v1/wallets/:id` -serves. In `samples/lumen` the read model is an in-memory map — small, exact, -and dependency-free, the right baseline for teaching: +serves. In `samples/lumen` the read model is an in-memory map — small, exact, and +dependency-free, the right baseline for teaching. + +Open `samples/lumen/src/ledger.rs` and read the read-model type: ```rust,ignore // samples/lumen/src/ledger.rs — the CQRS query side. @@ -72,42 +115,48 @@ impl ReadModel { } ``` -Two things are deliberate. First, the surface is a **repository in miniature**: -`upsert(view)` and `find(id)` are the only operations the query side needs, so -those are the only operations it exposes. Second, the keys and values are the -plain domain types — a `WalletView` keyed by its `id` — so when you swap the map -for a database adapter, the *shape* the rest of Lumen sees does not move: a -`find` still returns `Option`, an `upsert` still takes one. - -That is the whole point of treating the read store as a repository: the -`GetWallet` handler depends on "give me the view for this id," not on a -`HashMap`. Below is the production surface you would lower this onto — same -contract, a real database behind it. - -## The `Page` envelope - -`Page` is the canonical paged-result envelope with a stable, versioned JSON -shape — any client that honors that contract deserializes it uniformly, so -generated SDK clients consume it without per-service handling: - -```rust,ignore -pub struct Page { - pub content: Vec, - pub number: usize, // zero-based page index - pub size: usize, - pub total_elements: u64, - pub total_pages: usize, // derived -} -``` - -A *list wallets* endpoint — a natural Lumen extension — returns a -`Page` so a client can page through accounts without ever loading -the whole table. - -## The `Filter` DSL - -A `Filter` composes predicates, sorts, and a page window, and renders to a -parameterized `WHERE` clause via `to_sql()`: +What just happened: two design choices are deliberate. + +- The surface is a **repository in miniature**. `upsert(view)` and `find(id)` are + the only operations the query side needs, so those are the only operations it + exposes — a hand-rolled, two-method subset of the framework repository contract + you meet in Step 4. +- The keys and values are plain domain types — a `WalletView` keyed by its `id`. + So when you later swap the map for a database adapter, the *shape* the rest of + Lumen sees does not move: `find` still returns `Option`, `upsert` + still takes one. + +> **Note** **Key term — `#[derive(Repository)]`.** This derive marks a type as a +> data-access bean — Spring's `@Repository`. The component scan registers it as a +> singleton, so it is autowired (as `Arc`) into the query handler and +> the projection that feeds it. The derive is about *wiring* the object into the +> container; the storage behind it is whatever the struct holds — here a +> `Mutex>`. + +Why it matters: the `GetWallet` handler depends on "give me the view for this +id," not on a `HashMap`. That is the whole point of treating the read store as a +repository — and the reason the rest of this chapter can replace the map with a +real database without the handler noticing. + +> **Tip** **Checkpoint.** From a checkout of the framework, run +> `cargo test -p lumen --lib ledger` and watch the read-model round-trip tests +> pass. You have confirmed the in-memory baseline works before swapping anything +> underneath it. + +## Step 2 — Compose a query with the `Filter` DSL + +Before lowering the read store onto a real database, you need a way to *ask* for +rows — a query value the adapters can render to SQL. `firefly-data` provides one: +the `Filter` DSL. + +> **Note** **Key term — `Filter` DSL.** A `Filter` is a composable value that +> bundles a list of predicates (field, operator, value), zero or more sort +> orders, and a page window. It renders to a parameterized `WHERE` clause via +> `to_sql()` — never string-interpolated, so values bind as `$1`, `$2`, … and +> SQL injection is structurally impossible. + +Build a "rich wallets" query — `balance >= 100_000`, newest first, first page of +20: ```rust use firefly_data::{Direction, Filter, Op, Predicate}; @@ -126,57 +175,117 @@ assert!(where_clause.contains("WHERE")); assert_eq!(args.len(), 2); ``` -The operators (`Op`) cover `Eq`, `Ne`, `Lt`, `Lte`, `Gt`, `Gte`, `Like`, -`ILike`, `In`, and `IsNil` — the last skips an argument slot, so a predicate -list and its argument list stay aligned. A "rich wallets" query (`balance >= -100_000`, newest first) is exactly the filter above. +What just happened, block by block: + +- `.where_eq("owner", json!("alice"))` adds an equality predicate; it is sugar + for `.add(Predicate { field, op: Op::Eq, value })`. +- `.add(Predicate { … op: Op::Gte … })` adds the `balance >= 100_000` predicate + explicitly — every operator is reachable this way. +- `.order_by("version", Direction::Desc)` appends a sort order (newest first). +- `.paged(0, 20)` sets a zero-based page window — page `0`, size `20`. +- `to_sql()` returns the `(where_clause, args)` pair. Two predicates produced two + bound arguments, which is what the `assert_eq!(args.len(), 2)` confirms. + +The operators on `Op` cover `Eq`, `Ne`, `Lt`, `Lte`, `Gt`, `Gte`, `Like`, +`ILike`, `In`, and `IsNil`. `IsNil` renders `IS NULL` and consumes **no** +argument slot, so a predicate list and its argument list always stay aligned. -## The blocking `Repository` contract +> **Note** `Filter::to_sql()` renders the PostgreSQL default (`$1` placeholders, +> `"id"` quoting). `Filter::to_sql_with(&dialect)` renders the *same* query tree +> for another backend — you meet `SqlDialect` in Step 6, where it is the seam +> that makes a single query string run on three databases. -`Repository` is the object-safe `async_trait` port; `MemoryRepository` -implements it for tests, and you back it with your driver in production: +> **Tip** **Checkpoint.** Drop that snippet into a `#[test]` and run it. Both +> assertions pass: the clause contains `WHERE`, and exactly two values were +> bound. You now have a query value the adapters in Step 6 know how to execute. + +## Step 3 — Read the `Page` envelope + +A query that pages needs a stable shape to return. `Page` is the canonical +paged-result envelope with a versioned JSON layout, so any client that honors the +contract deserializes it uniformly — a generated SDK consumes it without +per-service handling: ```rust,ignore -#[async_trait] -pub trait Repository: Send + Sync { - async fn find_by_id(&self, id: &K) -> Result; - async fn find(&self, filter: &Filter) -> Result, DataError>; - // save / delete / count ... +pub struct Page { + pub content: Vec, + pub number: usize, // zero-based page index + pub size: usize, + pub total_elements: u64, + pub total_pages: usize, // derived from total_elements / size } ``` -This is the contract Lumen's `ReadModel` is a hand-rolled, two-method subset of. -Lower it onto `Repository` and `find` / `find_by_id` / -`save` come from the framework. +What just happened: a *list wallets* endpoint — a natural Lumen extension — +returns a `Page` so a client can page through accounts without ever +loading the whole table. `content` carries this page's rows; `number` / `size` +echo the requested window; `total_elements` and `total_pages` let a UI render a +pager. -## The reactive CRUD surface +> **Note** `Page` is the *response* side of paging — what comes back. There is +> also a *request* side, `Pageable` (page number, size, sort), which you meet in +> Step 5. Keep them straight: a caller sends a `Pageable`, a count-aware +> repository returns a `Page`. -On top of the blocking contract, `firefly-data` adds a **reactive** CRUD surface -built on `Mono` / `Flux` (the publishers from -[The Reactive Model](./05-reactive-model.md)). It is purely additive: nothing -about the existing `Repository` API changes. +## Step 4 — Meet the repository contract -| Method | Returns | -|-----------------------------------|---------------------------------------------| -| `find_all()` | `Flux` | -| `find_all_by_id(ids)` | `Flux` | -| `find_by_id(id)` | `Mono` | -| `exists_by_id(id)` | `Mono` | -| `save(e)` | `Mono` | -| `save_all(es)` | `Flux` | -| `delete_by_id(id)` | `Mono<()>` | -| `delete_all()` | `Mono<()>` | -| `count()` | `Mono` | -| `Specification` + `Pageable` | `ReactiveSpecificationRepository` | +`ReadModel` is a two-method subset of a real framework contract. There are two, +sharing the same idea at different layers. -A "no row" `find_by_id` resolves to an **empty** `Mono` — the reactive -equivalent of Lumen's `ReadModel::find` returning `None`. +The **blocking** port, `Repository`, is the object-safe `async_trait` +contract; `MemoryRepository` implements it for tests, and an adapter backs it +with a driver in production: -### In-memory, for tests +```rust,ignore +#[async_trait] +pub trait Repository: Send + Sync { + async fn find_by_id(&self, id: &K) -> Result; + async fn find(&self, filter: &Filter) -> Result, DataError>; + async fn save(&self, entity: T) -> Result; + async fn delete(&self, id: &K) -> Result<(), DataError>; + // find_page(&Pageable), count, … with defaults +} +``` -`ReactiveMemoryRepository` is the reactive twin of `MemoryRepository`. Drive the -publishers with `block()` / `collect_list()`. Here it is holding wallet views — -the reactive version of Lumen's read store: +On top of it, `firefly-data` adds the **reactive** CRUD surface, built on +`Mono` / `Flux`. It is purely additive — nothing about the blocking `Repository` +API changes: + +| Method | Returns | +|------------------------------|---------------------------------------------| +| `find_all()` | `Flux` | +| `find_all_by_id(ids)` | `Flux` | +| `find_by_id(id)` | `Mono` | +| `exists_by_id(id)` | `Mono` | +| `save(e)` | `Mono` | +| `save_all(es)` | `Flux` | +| `delete_by_id(id)` | `Mono<()>` | +| `delete_all()` | `Mono<()>` | +| `count()` | `Mono` | +| `Specification` + `Pageable` | `ReactiveSpecificationRepository` (Step 5) | + +What just happened: these are Spring Data's `ReactiveCrudRepository` +methods, name for name. One detail matters for the rest of the chapter — a "no +row" `find_by_id` resolves to an **empty** `Mono`, the reactive equivalent of +Lumen's `ReadModel::find` returning `None`. + +> **Note** **Key term — `block()` / `collect_list()`.** The publishers are lazy: +> nothing runs until you drive them. In an `async` context `Mono::block().await` +> drives a `Mono` to its result, returning `Result, FireflyError>` — +> `Ok(None)` is the empty-`Mono` miss. `Flux::collect_list()` gathers a stream +> into a `Mono>`, so `flux.collect_list().block().await` returns +> `Result>, _>`. (A `Mono` also implements `IntoFuture`, so you +> can `repo.save(x).await` directly when you prefer.) + +This is the contract Lumen's `ReadModel` is a hand-rolled subset of. Lower it +onto `ReactiveCrudRepository` and `find_by_id` / `save` / +`count` come from the framework. The next step does exactly that, in memory. + +## Step 5 — Drive the reactive surface in memory + +`ReactiveMemoryRepository` is the reactive twin of `MemoryRepository` — the +no-infrastructure way to exercise the real reactive API. It is the reactive +version of Lumen's read store, holding wallet views: ```rust use firefly_data::{ReactiveCrudRepository, ReactiveMemoryRepository}; @@ -186,151 +295,59 @@ struct WalletView { id: String, owner: String, balance: i64, version: i64 } #[tokio::main] async fn main() { + // The closure tells the repository how to read an entity's id. let repo = ReactiveMemoryRepository::new(|w: &WalletView| w.id.clone()); - // save -> Mono + // save -> Mono, driven with block(). repo.save(WalletView { id: "wlt_1".into(), owner: "alice".into(), balance: 1000, version: 1 }) .block().await.unwrap(); - // find_all -> Flux, collected to a Vec + // find_all -> Flux, collected to a Vec. let all = repo.find_all().collect_list().block().await.unwrap().unwrap(); assert_eq!(all.len(), 1); - // find_by_id miss -> empty Mono (Lumen's `ReadModel::find` returning None) + // find_by_id miss -> empty Mono (Lumen's `ReadModel::find` returning None). assert_eq!(repo.find_by_id("ghost".into()).block().await.unwrap(), None); + + // count -> Mono. assert_eq!(repo.count().block().await.unwrap(), Some(1)); } ``` -Swapping `ReadModel` for this repository is mechanical: `upsert` becomes `save`, -`find` becomes `find_by_id(...).block().await`, and the query handler keeps its -`Option` shape. - -### Real SQL, streaming rows as a `Flux` - -`PostgresReactiveRepository` is a production repository over `tokio-postgres`. -Reads drive the driver's `query_raw` **row stream**, so each row is decoded by a -`RowMapper` and emitted the moment it arrives over the wire — a million-row -table never lands fully in memory. Writes use a per-entity `inserter` closure -that renders a `T` to an upsert whose `RETURNING` projects the configured -columns. Backing Lumen's read model with it looks like this: - -```rust,no_run -use std::sync::Arc; -use firefly_data::{PostgresReactiveRepository, ReactiveCrudRepository, TableConfig}; -use firefly_kernel::FireflyError; -use tokio_postgres::{Row, types::ToSql, NoTls}; - -#[derive(Clone, PartialEq, Debug)] -struct WalletView { id: String, owner: String, balance: i64, version: i64 } - -# async fn ex() -> Result<(), Box> { -let (client, conn) = - tokio_postgres::connect("postgres://localhost/lumen", NoTls).await?; -tokio::spawn(async move { let _ = conn.await; }); -let client = Arc::new(client); - -let repo: PostgresReactiveRepository = PostgresReactiveRepository::new( - Arc::clone(&client), - TableConfig::new("wallet_views", "id", ["id", "owner", "balance", "version"]), - // RowMapper: decode a WalletView from each streamed row. - |row: &Row| Ok(WalletView { - id: row.try_get("id").map_err(|e| FireflyError::internal(e.to_string()))?, - owner: row.try_get("owner").map_err(|e| FireflyError::internal(e.to_string()))?, - balance: row.try_get("balance").map_err(|e| FireflyError::internal(e.to_string()))?, - version: row.try_get("version").map_err(|e| FireflyError::internal(e.to_string()))?, - }), - // inserter: upsert RETURNING the projected columns (the projection's upsert). - |w: &WalletView| ( - "INSERT INTO \"wallet_views\" (\"id\", \"owner\", \"balance\", \"version\") \ - VALUES ($1, $2, $3, $4) \ - ON CONFLICT (\"id\") DO UPDATE SET \"owner\" = EXCLUDED.\"owner\", \ - \"balance\" = EXCLUDED.\"balance\", \"version\" = EXCLUDED.\"version\" \ - RETURNING \"id\", \"owner\", \"balance\", \"version\"".to_string(), - vec![ - Box::new(w.id.clone()) as Box, - Box::new(w.owner.clone()) as Box, - Box::new(w.balance) as Box, - Box::new(w.version) as Box, - ], - ), -); - -// Rows stream lazily out of find_all() as a Flux — a *list wallets* endpoint. -let all = repo.find_all().collect_list().block().await?.unwrap(); -# Ok(()) -# } -``` - -Use `stream_query(sql, params)` for custom derived queries: any `SELECT` -projecting the configured columns is streamed row-by-row through the same -`RowMapper`. This `Flux` plugs directly into a `NdJson` / `Sse` endpoint, so a -database read streams to the client end-to-end with backpressure — no -collect-then-emit step anywhere in the path. (That is the same `Flux → NdJson` -seam Lumen's events endpoint uses, only sourced from rows instead of an event -store.) - -### Any key type, and optimistic locking - -The repository's `ID` is unbounded, like Spring Data's `CrudRepository`: -the sqlx adapter accepts any `serde::Serialize` key through the `SqlKey` trait, -so a `Uuid`, `i64`, `String`, an enum, or a composite-key struct all work — no -newtype dance. The key binds as its serde-JSON form against the id column. - -For a versioned entity, enable **optimistic locking** (Spring's `@Version`) by -naming the version column on the sqlx repository: - -```rust,ignore -let repo = SqlxReactiveRepository::new(db, cfg, read, write) - .with_version_column("version"); // UPSERT guarded by `WHERE version = ` -``` - -A stale write — one whose loaded `version` no longer matches the row — is -rejected instead of silently overwriting a concurrent update. The blocking -`SqlxRepository::save` surfaces this as `DataError::OptimisticLock`; the reactive -`save` surfaces it through its `FireflyError` channel (a 409), which -`firefly_data_sqlx::is_optimistic_lock(&err)` detects so a service can map it to -a domain conflict. - -### Declaring a repository: `#[derive(SqlxRepository)]` - -You rarely build the repository by hand. `#[derive(Entity)]` on the entity -generates its `@Table` / `@Id` / `@Version` / `@Column` mapping from the fields, -then `#[derive(SqlxRepository)]` over a struct holding its `SqlxReactiveRepository`: - -```rust,ignore -#[derive(Entity)] -#[firefly(table = "wallets")] -pub struct Wallet { #[firefly(id)] id: Uuid, balance: i64, #[firefly(version)] version: i64, /* … */ } - -#[derive(SqlxRepository)] -pub struct WalletRepository { repo: SqlxReactiveRepository } -``` - -`#[derive(SqlxRepository)]` registers it as a `@Repository` bean **built from the -injected `Db` datasource bean** (wiring `@Version` locking + auditing from the -entity), and implements `ReactiveCrudRepository` by delegation — the Spring Data -"declare a repository, get the implementation" experience. Scalar columns map -automatically; a non-scalar field (an enum) uses -`#[firefly(with(read = "...", write = "..."))]`. The -[`lumen-ledger`](./22-layered-microservices.md) sample uses exactly this. - -### Reactive specification / paging - -`ReactiveSpecificationRepository` runs a composable `Specification` predicate -with an optional `Pageable` window and **streams** the matches as a `Flux`, with -no intermediate `Page` envelope — so it plugs straight into an NDJSON / SSE -endpoint with backpressure. - -### Sorting & paging — the `ReactiveSortingRepository` (free) - -firefly-data's `ReactiveSortingRepository` adds whole-collection sorting -and paging — `find_all_sorted(RequestSort) -> Flux` and -`find_all_paged(Pageable) -> Flux` — and you write **no** code for it: it is a -blanket `impl` over any repository that is both a `ReactiveCrudRepository` and a -`ReactiveSpecificationRepository`. The sort/page is run as a match-all -`Specification`, so every `SqlxReactiveRepository` and `ReactiveMemoryRepository` -acquires it automatically. +What just happened, line by line: + +- `ReactiveMemoryRepository::new(|w| w.id.clone())` builds an empty store whose + ids are derived by the keyer closure. +- `repo.save(...).block().await.unwrap()` drives the `save` `Mono` to completion; + the `unwrap()` discards the `Result`, the inner `Some(view)` is the persisted + value. +- `repo.find_all().collect_list().block().await.unwrap().unwrap()` chains the + three reactive operators: `find_all()` returns a `Flux`, `collect_list()` folds + it into a `Mono>`, `block().await` drives it. The first `unwrap()` + unwraps the `Result`, the second unwraps the `Option>`. +- `repo.find_by_id("ghost".into()).block().await.unwrap()` is the miss case — it + resolves to `None`, the empty-`Mono` contract from Step 4. + +Why it matters: swapping `ReadModel` for this repository is mechanical. `upsert` +becomes `save`, `find` becomes `find_by_id(...).block().await`, and the +`GetWallet` handler keeps its `Option` shape. You have just proven +the seam with no database in the loop. + +### Sorting and paging come for free + +`ReactiveSortingRepository` adds whole-collection sorting and paging — +`find_all_sorted(RequestSort) -> Flux` and `find_all_paged(Pageable) -> +Flux` — and you write **no** code for it. It is a blanket `impl` over any +repository that is both a `ReactiveCrudRepository` and a +`ReactiveSpecificationRepository`, so every `ReactiveMemoryRepository` and every +SQL repository acquires it automatically. + +> **Note** **Key term — `Pageable` / `RequestSort`.** These are the *request* +> side of paging (Spring's `Pageable` / `Sort`). `RequestSort::by(["owner"])` +> sorts ascending by a field; `RequestSort::of([Order::desc("id")])` builds an +> explicit order list. `Pageable::of(page, size, sort)` bundles them — and +> crucially, **`page` is 1-based** and the call returns a `Result` (an +> out-of-range page is an error, not a panic), so you `.unwrap()` / `?` it. ```rust use firefly_data::{ @@ -349,13 +366,13 @@ async fn main() { .block().await.unwrap(); } - // find_all(Sort) — ordered by owner ascending, streamed as a Flux. + // find_all_sorted(RequestSort) — ordered by owner ascending, streamed as a Flux. let sorted = repo .find_all_sorted(RequestSort::by(["owner"])) .collect_list().block().await.unwrap().unwrap(); assert_eq!(sorted[0].owner, "alice"); - // find_all(Pageable) — page 1 (1-based), size 2, sorted; a Flux window. + // find_all_paged(Pageable) — page 1 (1-based), size 2, sorted; a Flux window. let page = repo .find_all_paged(Pageable::of(1, 2, RequestSort::by(["owner"])).unwrap()) .collect_list().block().await.unwrap().unwrap(); @@ -363,48 +380,32 @@ async fn main() { } ``` -> **Note.** `find_all_paged` streams the page as a `Flux` window rather than -> buffering a `Page` envelope — reach for `Page` + a count query when you -> actually need totals. - -## Pluggable databases — a new DB is a new adapter +What just happened: `find_all_sorted` ran a match-all `Specification` with the +sort projected onto it; `find_all_paged` ran the same with a `LIMIT`/`OFFSET` +window. Note `Pageable::of(1, 2, …).unwrap()` — page `1` is the *first* page, and +the `unwrap()` handles the `Result`. -`firefly-data` is the **ports** crate: it owns no driver and implies no SQL -engine. The `Filter` DSL, the composable `Specification`, the repository traits, -and the auditing / soft-delete policies are all storage-agnostic. The lowering -surface is what makes this hexagonal: +> **Note** `find_all_paged` streams the page as a `Flux` window rather than +> buffering a `Page` envelope. Reach for `Page` (Step 3) plus a count query +> when you actually need totals; reach for the streaming window when you do not. -- a **`SqlDialect`** trait with three shipped impls — `PostgresDialect`, - `MySqlDialect`, `SqliteDialect` — so `Filter::to_sql_with(&dialect)` and - `Specification::to_sql_with(&dialect)` render the *same* query tree per - backend, getting placeholder style (`$1` vs `?`), identifier quoting (`"id"` - vs `` `id` ``), `IN`-list shape, and case-insensitive `LIKE` right for each. - (`Filter::to_sql` / `Specification::to_sql` stay the PostgreSQL default.) -- a **`Specification::to_mongo()`** / `Filter::to_mongo()` that lowers the same - tree to a MongoDB `$`-operator filter document. +> **Tip** **Checkpoint.** Run both `main`s above (each as a small binary or +> `#[tokio::test]`). The first asserts a save/find/count round-trip and an +> empty-`Mono` miss; the second asserts sort-then-page ordering. Every reactive +> repository in the rest of the chapter behaves identically — only the storage +> behind it changes. -Two adapter *crates* — `firefly-data-sqlx` (covering the three relational -backends) and `firefly-data-mongodb` — implement those ports so you code once -and swap backends. +## Step 6 — Lower onto a real, streaming SQL repository -> **Design note.** This is hexagonal architecture — ports & adapters. Your -> service depends on the repository *port*; the *adapter* (the relational crate, -> the Mongo crate) is chosen at wiring time. Swapping Postgres for MySQL is a -> pool change, not a code change — the call sites never move. +The in-memory repository proves the *shape*. Now make it durable. The relational +adapter, `firefly-data-sqlx`, serves PostgreSQL, MySQL, and SQLite from one +codebase, and **SQLite-in-memory is the no-infrastructure default** — the same +role the in-memory map plays in `samples/lumen`, but exercising the real adapter. -### Relational — `firefly-data-sqlx` (Postgres / MySQL / SQLite) - -`SqlxReactiveRepository` (and the blocking `SqlxRepository`) serve all three -relational backends from one codebase. A `Db` enum tags a `PgPool` / -`MySqlPool` / `SqlitePool` with its `Backend`, and the repository picks the -matching `SqlDialect` at runtime — so "new relational DB = new pool", not "new -adapter". `UPSERT` is dialect-aware (`ON CONFLICT … DO UPDATE` for -Postgres/SQLite, `ON DUPLICATE KEY UPDATE` for MySQL), reads stream off sqlx's -row stream into a `Flux`, and an optional `Auditor` / `SoftDeletePolicy` -auto-stamps and hides rows on every write/read. - -SQLite-in-memory is the **no-infrastructure default** — the same role the -in-memory map plays in `samples/lumen`, but exercising the real adapter: +> **Note** **Key term — `Db` enum.** `Db` tags a connection pool with its +> backend: `Db::Postgres(PgPool)`, `Db::MySql(MySqlPool)`, `Db::Sqlite(SqlitePool)`. +> The repository picks the matching `SqlDialect` at runtime from that tag — so +> "new relational database" is a new pool, not a new adapter. ```rust use firefly_data::{ReactiveCrudRepository, TableConfig}; @@ -422,13 +423,13 @@ sqlx::query(r#"CREATE TABLE "wallet_views" ("id" TEXT PRIMARY KEY, "owner" TEXT let repo: SqlxReactiveRepository = SqlxReactiveRepository::new( Db::Sqlite(pool), TableConfig::new("wallet_views", "id", ["id", "owner", "balance"]), - // RowMapper: decode by column name — backend-agnostic via AnyRow. + // RowMapper: decode a WalletView from each row — backend-agnostic via AnyRow. |row: &AnyRow| Ok::<_, FireflyError>(WalletView { id: row.get_str("id")?, owner: row.get_str("owner")?, balance: row.get_i64("balance")?, }), - // RowWriter: the entity's (column, value) pairs. + // RowWriter: the entity's (column, value) pairs for the upsert. |w: &WalletView| vec![ ColumnValue::new("id", w.id.clone()), ColumnValue::new("owner", w.owner.clone()), @@ -440,28 +441,290 @@ repo.save(WalletView { id: "wlt_1".into(), owner: "alice".into(), balance: 1000 # }); ``` -Switching to Postgres or MySQL is `Db::Postgres(pg_pool)` / `Db::MySql(my_pool)` -— the repository call sites do not change. That is the upgrade path the -`samples/lumen` comment promises: the in-memory `ReadModel` becomes a -`SqlxReactiveRepository`, and the `GetWallet` handler is -none the wiser. - -The constructor takes a `Db`, a `TableConfig`, a `RowMapper` (reads), and a -`RowWriter` (writes); three chainable builders add cross-cutting behaviour, each -returning a fresh repository: - -- `.with_auditor(Auditor)` — stamps `created_at` / `updated_at` / - `created_by` / `updated_by` on every write (insert vs update is decided by - whether the row already exists): automatic auditing. -- `.with_soft_delete(SoftDeletePolicy)` — hides soft-deleted rows from every - read and turns `delete_by_id` into a `deleted_at` stamp instead of a physical +What just happened, by argument: + +- `Db::Sqlite(pool)` tags the SQLite pool; the repository reads its backend from + the tag. +- `TableConfig::new("wallet_views", "id", ["id", "owner", "balance"])` names the + table, its id column, and the columns to project — the `RowMapper` must decode + rows shaped by exactly these columns. +- The **`RowMapper`** closure decodes one row. `AnyRow` is the backend-agnostic + row wrapper; `get_str` / `get_i64` read columns by name, so the same closure + works on all three relational backends. +- The **`RowWriter`** closure produces the `(column, value)` pairs the adapter + renders into a dialect-aware `UPSERT` (`ON CONFLICT … DO UPDATE` for + Postgres/SQLite, `ON DUPLICATE KEY UPDATE` for MySQL). + +Why it matters: switching to Postgres or MySQL is `Db::Postgres(pg_pool)` / +`Db::MySql(my_pool)` — the repository call sites do not change. That is the +upgrade path the `samples/lumen` comment promises: the in-memory `ReadModel` +becomes a `SqlxReactiveRepository`, and the `GetWallet` +handler is none the wiser. Reads stream off sqlx's row stream into a `Flux`, so a +million-row table never lands fully in memory. + +> **Design note.** This is hexagonal architecture — ports & adapters. Your +> service depends on the repository *port* (`ReactiveCrudRepository`); the +> *adapter* (the relational crate, the Mongo crate) is chosen at wiring time. +> Swapping Postgres for MySQL is a pool change, not a code change — the call +> sites never move. Adding a *new* database is "write a `firefly-data-` +> crate that implements the ports," not "rewrite the data layer." `firefly-data` +> ships three `SqlDialect` impls (`PostgresDialect`, `MySqlDialect`, +> `SqliteDialect`) and a `Specification::to_mongo()` lowering, so the same query +> tree renders correctly per backend. + +> **Tip** **Checkpoint.** Run that snippet as a test. It creates a `wallet_views` +> table in `sqlite::memory:`, saves a row through the real adapter, and returns +> with no error — the production read store, exercised end to end with zero +> external infrastructure. + +The constructor takes a `Db`, a `TableConfig`, a `RowMapper`, and a `RowWriter`; +three chainable builders add cross-cutting behaviour, each returning a fresh +repository: + +- `.with_auditor(Auditor)` — stamps `created_at` / `updated_at` / `created_by` / + `updated_by` on every write (insert vs update is decided by whether the row + already exists): automatic auditing. +- `.with_soft_delete(SoftDeletePolicy)` — hides soft-deleted rows from every read + and turns `delete_by_id` into a `deleted_at` stamp instead of a physical `DELETE`: logical (soft) delete. -- `.with_version_column("version")` — turns on optimistic locking - (next section). +- `.with_version_column("version")` — turns on optimistic locking (Step 8). + +## Step 7 — Declare the repository the Spring Data way + +You rarely build the repository by hand the way Step 6 did. For a typed entity, +two derives give you the Spring Data "declare a repository, get the +implementation" experience. This is exactly how the +[`lumen-ledger`](./22-layered-microservices.md) sample wires its persistence. + +> **Note** **Key term — `#[derive(Entity)]`.** This derive generates an entity's +> `@Table` / `@Id` / `@Version` / `@Column` mapping from its fields. Scalar +> columns (`String`, `i64`, `Uuid` as text, `DateTime` as text) map +> automatically; a non-scalar field (a typed enum) uses +> `#[firefly(with(read = "...", write = "..."))]` to name its converters — the +> `@Enumerated(STRING)` boundary. + +```rust,ignore +use chrono::{DateTime, Utc}; +use uuid::Uuid; + +#[derive(Debug, Clone, PartialEq, Eq, firefly::Entity)] +#[firefly(table = "wallets")] +pub struct Wallet { + #[firefly(id)] + pub id: Uuid, + pub account_number: String, + pub owner: String, + pub balance: i64, + pub currency: String, + // A typed enum maps through explicit converters — @Enumerated(STRING). + #[firefly(with(read = "WalletStatus::from_token", write = "WalletStatus::as_str"))] + pub status: WalletStatus, + // Optimistic-locking version (@Version) — bumped by the store on update. + #[firefly(version)] + pub version: i64, + // Audit stamps, managed by the store's Auditor. + pub created_at: DateTime, + pub updated_at: DateTime, +} +``` + +Then `#[derive(SqlxRepository)]` over a struct holding the entity's +`SqlxReactiveRepository`: + +```rust,ignore +use firefly::data::{DataError, Pageable}; +use firefly::data_sqlx::SqlxReactiveRepository; +use uuid::Uuid; + +#[derive(firefly::SqlxRepository)] +pub struct WalletRepository { + repo: SqlxReactiveRepository, +} +``` + +What just happened: `#[derive(SqlxRepository)]` registers `WalletRepository` as a +`@Repository` bean **built from the injected `Db` datasource** (wiring the +entity's `@Version` locking and `@CreatedDate`/`@LastModifiedDate` auditing), and +implements `ReactiveCrudRepository` by delegation. There is no `#[bean]` factory +and no hand-written CRUD — the derive builds the inner `SqlxReactiveRepository` +from the autowired `Db`, exactly like Spring Data's +`interface WalletRepository extends ReactiveCrudRepository`. + +> **Note** **Key term — `Uuid` (any) id.** The repository's `ID` is unbounded, +> like Spring Data's `CrudRepository`: the sqlx adapter accepts any +> `serde::Serialize` key through its `SqlKey` trait, so a `Uuid`, `i64`, +> `String`, an enum, or a composite-key struct all work with no newtype dance. +> The key binds as its serde-JSON form against the id column. + +### Derived and custom queries — `#[firefly::repository]` + +Beyond CRUD, the `#[firefly::repository]` macro derives a query straight from a +*method name*: `find_by_owner(&str)` becomes `WHERE owner = ?`. Apply it to an +`impl` block of typed stub methods; the macro discards the placeholder body +(`unimplemented!()`) and generates a real one that marshals the arguments and +delegates to the runtime engine. The **return type selects the operation**: + +| Return shape | Generated call | +|---------------------------------|---------------------------| +| `Result, DataError>` | `find_by_derived` | +| `Result, DataError>` | `find_by_derived` (first) | +| `Result` | `count_by_derived` | +| `Result` | `exists_by_derived` | +| `Result` | `delete_by_derived` | + +This is the actual derived-query block on `lumen-ledger`'s `WalletRepository`: + +```rust,ignore +use firefly::data::{DataError, Pageable}; + +#[firefly::repository] +impl WalletRepository { + /// `SELECT … WHERE owner = ?` — every wallet of one owner. + pub async fn find_by_owner(&self, owner: &str) -> Result, DataError> { + unimplemented!() + } + + /// `SELECT COUNT(*) WHERE status = ?`. + pub async fn count_by_status(&self, status: &str) -> Result { + unimplemented!() + } + + /// Paged `SELECT … WHERE status = ?` — a trailing `Pageable` makes it paged. + pub async fn find_by_status( + &self, + status: &str, + page: Pageable, + ) -> Result, DataError> { + unimplemented!() + } +} +``` + +What just happened: the method *names* are the grammar — prefix (`find` / `count` +/ `exists` / `delete`), then `By`, then a chain of `And` / `Or` property +conditions (`find_by_owner_and_status`). A method whose **last argument is a +`Pageable`** is a *paged* derived query: the pageable's sort and window are +appended to the generated `WHERE`. Build the page with +`Pageable::of(page, size, sort)` — remember `page` is **1-based** and the call +returns a `Result`: + +```rust,ignore +use firefly::data::{Pageable, RequestSort}; + +# async fn ex(wallets: &WalletRepository) -> Result<(), firefly::data::DataError> { +let page = Pageable::of(1, 20, RequestSort::by(["account_number"])).unwrap(); +let active = wallets.find_by_status("active", page).await?; +# Ok(()) +# } +``` + +When the name grammar can't express the query, annotate the stub with +`#[query(...)]` and write the SQL yourself. A `:name` placeholder binds the +argument named `name`, and the return type selects the operation exactly as for +derived methods — `Vec` / `Option` for a list, `i64` for a count, `bool` +for an exists, `u64` for a *modifying* statement (the affected-row count): + +```rust,ignore +use firefly::data::DataError; + +#[firefly::repository] +impl WalletRepository { + // Native SQL; :status binds to the `status` argument. + #[query("SELECT id, owner FROM wallets WHERE status = :status ORDER BY id DESC")] + async fn list_by_status(&self, status: &str) -> Result, DataError> { + unimplemented!() + } + + // u64 return -> a modifying statement; the value is the affected-row count. + #[query("UPDATE wallets SET status = :to WHERE status = :from")] + async fn retire(&self, from: &str, to: &str) -> Result { + unimplemented!() + } +} +``` + +What just happened: `#[query("…")]` is shorthand for `#[query(sql = "…")]` +(native SQL). For a portable, entity-oriented query use the JPQL-like form +`#[query(jpql = "…", entity = "Wallet")]`, whose `FROM ` is transpiled to +the configured table so the same string runs on Postgres, MySQL, or SQLite. Under +the hood, the method-name parser lowers a derived query through the active +`SqlDialect`, and `#[query]` lowers to the repository's +`query_list` / `query_count` / `query_exists` / `query_execute` helpers. + +> **Tip** Reach for the method-name grammar for simple predicates and a trailing +> `Pageable` for paged reads; use `#[query(...)]` for anything the name grammar +> can't express. A relational `Account` / `Order` service writes these; Lumen's +> own event-sourced read side stays a hand-rolled in-memory `ReadModel` because +> its query needs are exactly two methods. + +## Step 8 — Turn on optimistic locking + +A versioned entity needs lost-update protection. Naming the version column on the +repository turns a `save` into a **version-guarded conditional upsert**: every +write bumps the version and guards the conflict-update on the version the entity +was loaded with (`WHERE version = `). If a concurrent writer moved the +stored version on, the guarded update matches zero rows and the save is rejected +instead of silently overwriting the other change. + +```rust,ignore +use firefly_data::{DataError, Repository, TableConfig}; +use firefly_data_sqlx::{AnyRow, ColumnValue, Db, SqlxRepository}; +use firefly_kernel::FireflyError; + +#[derive(Debug, Clone)] +struct Account { id: String, balance: i64, version: i64 } + +# async fn ex(pool: sqlx::PgPool) -> Result<(), Box> { +let repo: SqlxRepository = SqlxRepository::new( + Db::Postgres(pool), + TableConfig::new("accounts", "id", ["id", "balance", "version"]), + |row: &AnyRow| Ok::<_, FireflyError>(Account { + id: row.get_str("id")?, + balance: row.get_i64("balance")?, + version: row.get_i64("version")?, + }), + |a: &Account| vec![ + ColumnValue::new("id", a.id.clone()), + ColumnValue::new("balance", a.balance), + // The loaded version — the conditional upsert guards on it. + ColumnValue::new("version", a.version), + ], +) +.with_version_column("version"); + +// Two callers loaded the same Account at version 1. The first save wins +// (the row is now version 2); the second save's guard (WHERE version = 1) +// matches nothing, so it fails with OptimisticLock — the caller reloads + retries. +let stale = repo.save(Account { id: "acc_1".into(), balance: 50, version: 1 }).await; +assert!(matches!(stale, Err(DataError::OptimisticLock))); +# Ok(()) +# } +``` + +What just happened: `.with_version_column("version")` made every `save` +conditional on the loaded version. The blocking `SqlxRepository::save` surfaces a +stale write as `DataError::OptimisticLock`; the reactive `save` surfaces it +through its `FireflyError` channel (a 409), which +`firefly_data_sqlx::is_optimistic_lock(&err)` detects so a service can map it to +a domain conflict. (Conflict detection is enforced on Postgres and SQLite; on +MySQL the version is bumped but the guard is not applied.) + +> **Note** A stale write fails rather than silently overwriting a concurrent +> change; the caller reloads and retries. Lumen's *write* side reaches the same +> lost-update guarantee differently — through the event store's +> optimistic-concurrency `append` (see +> [Event Sourcing](./11-event-sourcing.md)) — but a relational `Account` / +> `Order` repository uses a version column. + +> **Tip** **Checkpoint.** `lumen-ledger`'s +> `models/src/repositories/wallet/v1/wallet_repository.rs` has a test, +> `optimistic_locking_rejects_a_stale_write`, that loads a row twice, writes once +> through each handle, and asserts the second is an +> `is_optimistic_lock` conflict. Run it: `cargo test -p lumen-ledger-models`. -### Building the pool from config — `auto_configure` +## Step 9 — Build the pool from configuration -In the examples above the pool is constructed by hand. In a real service the +In every snippet so far the pool was constructed by hand. In a real service the connection settings live in configuration, and Firefly turns them into a live pool — plus a registered transaction manager — in a single awaited call at startup. There is no dependency-injection container in the loop: you load the @@ -484,16 +747,16 @@ pub struct DataSourceProperties { } ``` -The **URL scheme selects the backend** (each behind its cargo feature): -`postgres://` / `postgresql://` → PostgreSQL, `mysql://` → MySQL, `sqlite:` → -SQLite. So a config change from `postgres://…` to `mysql://…` moves the whole -service to MySQL with no code edit — the pluggable-database promise, driven from -configuration. +What just happened: the **URL scheme selects the backend** (each behind its cargo +feature): `postgres://` / `postgresql://` → PostgreSQL, `mysql://` → MySQL, +`sqlite:` → SQLite. So a config change from `postgres://…` to `mysql://…` moves +the whole service to MySQL with no code edit — the pluggable-database promise, +driven from configuration. Three entry points build the `Db`: -- `Db::connect(url).await -> Result` — a pool from a URL, - using driver defaults. +- `Db::connect(url).await -> Result` — a pool from a URL, using + driver defaults. - `Db::connect_with(&props).await` — a pool honouring the full `DataSourceProperties` (sizes, timeouts, lifetimes). - `data_sqlx::auto_configure(&props).await` — the **one-call startup path**: it @@ -502,7 +765,7 @@ Three entry points build the `Db`: returned `Db` then builds your typed repositories. The shape of a boot sequence is: load config → bind `DataSourceProperties` → -`await auto_configure` once → build repositories from the returned `Db`. +`await auto_configure` once → build repositories from the returned `Db`: ```rust,ignore use firefly_data::TableConfig; @@ -537,205 +800,163 @@ let wallets: SqlxReactiveRepository = SqlxReactiveRepository # } ``` +What just happened: `auto_configure(&props)` did the two startup jobs at once — +built the pool and registered the `SqlxTransactionManager` in the process — so +the transaction boundary in Step 10 needs no extra wiring. + > **Design note.** Configuration drives the runtime, not a container. A plain > `serde` struct is bound from `firefly.datasource.*`, and a single awaited > `auto_configure` builds the pool and registers the transaction manager — the > wiring is explicit, compiler-checked, and visible in one place rather than > assembled by reflection at runtime. -### Optimistic locking — `with_version_column` +### Schema migrations -`with_version_column("version")` makes a `save` a **version-guarded** conditional -upsert: every write bumps the version column and guards the conflict-update on -the version the entity was loaded with (`WHERE version = `). If a -concurrent writer moved the stored version on, the guarded update matches zero -rows and the save fails with `DataError::OptimisticLock` rather than silently -overwriting the other change. The entity's `RowWriter` must emit the version -column carrying the loaded value. (Conflict detection is enforced on Postgres and -SQLite; on MySQL the version is bumped but the guard is not applied.) +The table those repositories read needs to exist first. `firefly-migrations` is a +forward-only SQL migration runner. Files are named `V{version}__{description}.sql` +(e.g. `V001__init.sql`); each runs once, in version order, inside a transaction: ```rust,ignore -use firefly_data::{DataError, Repository, TableConfig}; -use firefly_data_sqlx::{AnyRow, ColumnValue, Db, SqlxRepository}; -use firefly_kernel::FireflyError; - -#[derive(Debug, Clone)] -struct Account { id: String, balance: i64, version: i64 } - -# async fn ex(pool: sqlx::PgPool) -> Result<(), Box> { -let repo: SqlxRepository = SqlxRepository::new( - Db::Postgres(pool), - TableConfig::new("accounts", "id", ["id", "balance", "version"]), - |row: &AnyRow| Ok::<_, FireflyError>(Account { - id: row.get_str("id")?, - balance: row.get_i64("balance")?, - version: row.get_i64("version")?, - }), - |a: &Account| vec![ - ColumnValue::new("id", a.id.clone()), - ColumnValue::new("balance", a.balance), - // The loaded version — the conditional upsert guards on it. - ColumnValue::new("version", a.version), - ], -) -.with_version_column("version"); +use firefly_migrations::{run, DirSource}; -// Two callers loaded the same Account at version 1. The first save wins -// (the row is now version 2); the second save's guard (WHERE version = 1) -// matches nothing, so it fails with OptimisticLock — the caller reloads + retries. -let stale = repo.save(Account { id: "acc_1".into(), balance: 50, version: 1 }).await; -assert!(matches!(stale, Err(DataError::OptimisticLock))); -# Ok(()) -# } +let src = DirSource { dir: "migrations".into() }; +run(&mut db, &src)?; // applies pending migrations in order +let status = firefly_migrations::inspect(&mut db, &src)?; // applied + pending ``` -> **Note.** A stale write fails with `DataError::OptimisticLock` rather than -> silently overwriting a concurrent change; the caller reloads and retries. -> Lumen's *write* side reaches the same "lost-update prevention" guarantee -> differently — through the event store's optimistic-concurrency `append` (see -> [Event Sourcing](./11-event-sourcing.md)) — but a relational `Account` / -> `Order` repository uses a version column. +The [CLI](./19-cli.md) wraps this: `firefly db init`, +`firefly db migrate -m "create wallet_views"`, +`firefly db upgrade --url sqlite://lumen.db`, and `firefly db status`. A +`V001__wallet_views.sql` creating the read-model table is all the schema Lumen's +durable read side needs. -### Declarative derived queries — `#[firefly::repository]` +## Step 10 — Make a multi-write change atomic -Beyond CRUD, the `#[firefly::repository]` macro derives a query straight from a -*method name*: `find_by_status(&str)` becomes `WHERE status = ?`. Apply it to an -`impl` block of **typed stub methods** named with the framework's grammar -(`find_by_…`, `count_by_…`, `exists_by_…`, `delete_by_…`); the macro discards the -placeholder body and generates a real one that marshals the arguments and -delegates to the tested runtime engine. The return type selects the operation: - -| Return shape | Generated call | -|---------------------------------------|-----------------------| -| `Result, DataError>` | `find_by_derived` | -| `Result, DataError>` | `find_by_derived` (first) | -| `Result` | `count_by_derived` | -| `Result` | `exists_by_derived` | -| `Result` | `delete_by_derived` | - -Each impl-block type exposes the backing repository via `self.repository()` -(override with `#[repository(repo = "…")]`), returning a -`SqlxReactiveRepository`: +A single `save` is atomic on its own. A *transfer* — debit one account, credit +another, write two ledger entries — must be atomic as a whole: all four writes +commit, or none do. That is what `#[firefly::transactional]` is for. + +> **Note** **Key term — `#[firefly::transactional]`.** Annotate an `async fn` +> that returns `Result<_, E>` (where `E: From`) and the body runs inside +> a transaction — **commit on `Ok`, rollback on `Err`**. This is Spring's +> `@Transactional`, made declarative in Rust. ```rust,ignore -use firefly_data::DataError; +use firefly::transactional::TxError; use firefly_data_sqlx::SqlxReactiveRepository; -struct Account { /* … */ } - -struct AccountRepo { - repo: SqlxReactiveRepository, -} - -impl AccountRepo { - // The accessor the macro calls (default name `repository`). - fn repository(&self) -> &SqlxReactiveRepository { - &self.repo - } -} +#[derive(Debug, Clone)] struct Account { id: String, balance: i64 } +#[derive(Debug, Clone)] struct Entry { id: String, account: String, delta: i64 } -#[firefly::repository] -impl AccountRepo { - async fn find_by_status(&self, status: &str) -> Result, DataError> { unimplemented!() } - async fn find_by_owner_and_status(&self, owner: &str, status: &str) -> Result, DataError> { unimplemented!() } - async fn count_by_owner(&self, owner: &str) -> Result { unimplemented!() } - async fn exists_by_email(&self, email: &str) -> Result { unimplemented!() } - async fn delete_by_status(&self, status: &str) -> Result { unimplemented!() } +#[firefly::transactional] // defaults: REQUIRED, datasource isolation, read-write +async fn transfer( + accounts: &SqlxReactiveRepository, + ledger: &SqlxReactiveRepository, + from: Account, + to: Account, + amount: i64, +) -> Result<(), TxError> { + // All four writes enlist in the same ambient transaction. If any await + // returns Err, the whole unit of work rolls back; otherwise it commits. + accounts.save(Account { balance: from.balance - amount, ..from.clone() }) + .into_future().await?; + accounts.save(Account { balance: to.balance + amount, ..to.clone() }) + .into_future().await?; + ledger.save(Entry { id: "e1".into(), account: from.id, delta: -amount }) + .into_future().await?; + ledger.save(Entry { id: "e2".into(), account: to.id, delta: amount }) + .into_future().await?; + Ok(()) } ``` -#### Paged derived queries — a trailing `Pageable` +What just happened, and why it is seamless: -A `find_by_…` method whose **last argument is a `Pageable`** and which returns -`Result, DataError>` is a *paged* derived query: the pageable's sort and -window are appended to the generated `WHERE`, and the runtime backs it with -`SqlxReactiveRepository::find_by_derived_paged(method_name, &args, &Pageable)`. -Build the page with `Pageable::of(page, size, sort)` — `page` is **1-based** — -and `RequestSort::of([Order::desc("id")])` for the ordering: +> **Note** **Key term — ambient enlistment.** While a transactional scope is +> active, the manager stows the open transaction in a task-local. Every +> `SqlxReactiveRepository` / `SqlxRepository` write *inside the fn* routes onto +> that active transaction instead of a fresh pool connection. So a plain sequence +> of `repo.save(...).await?` calls is atomic with **no change to the repository +> code** — you do not thread a connection or a `&mut Tx` through every call. -```rust,ignore -use firefly_data::{DataError, Order, Pageable, RequestSort}; +The attribute accepts the full Spring vocabulary: `propagation` (`required` / +`requires_new` / `nested` / `supports` / `not_supported` / `mandatory` / +`never`), `isolation` (`read_committed` / `repeatable_read` / `serializable` / +…), `read_only`, `timeout_ms`, and `manager = ""` — Spring's +`@Transactional("txManager")`, which runs against an explicit +`TransactionManager` (e.g. `self.tx_manager()`) instead of the process-global +registry. This is exactly what `lumen-ledger`'s `WalletServiceImpl::transfer_tx` +does: -#[firefly::repository] -impl AccountRepo { - // The trailing Pageable makes this a paged query: WHERE owner = ?, - // then the pageable's ORDER BY + LIMIT/OFFSET window. - async fn find_by_owner(&self, owner: &str, page: Pageable) -> Result, DataError> { - unimplemented!() - } +```rust,ignore +// lumen-ledger/core/src/services/wallet/v1/wallet_service_impl.rs (excerpt) +#[firefly::transactional(manager = "self.tx_manager()")] +async fn transfer_tx(&self, from: Uuid, to: Uuid, amount: i64) + -> Result +{ + // … preconditions checked before any write … + let saved_source = self.persist(source).await?; // debit + self.persist(dest).await?; // credit — if this fails, + Ok(saved_source) // the debit rolls back } - -# async fn ex(accounts: &AccountRepo) -> Result<(), DataError> { -let page = Pageable::of(1, 20, RequestSort::of([Order::desc("id")])).unwrap(); -let rows = accounts.find_by_owner("alice", page).await?; -# Ok(()) -# } ``` -#### Custom queries — `#[query(...)]` - -When the method-name grammar can't express the query you need, annotate the stub -with `#[query(...)]` and write the SQL yourself. A `:name` placeholder binds to -the argument named `name`, and the **return type selects the operation** exactly -as for derived methods — `Vec` / `Option` for a list, `i64` for a count, -`bool` for an exists, and `u64` for a *modifying* statement (`INSERT` / `UPDATE` -/ `DELETE`, returning the affected-row count): +### Rollback rules — naming a pattern, not an exception type -```rust,ignore -use firefly_data::DataError; - -#[firefly::repository] -impl AccountRepo { - // Native SQL; :status binds to the `status` argument. - #[query("SELECT id, owner FROM accounts WHERE status = :status ORDER BY id DESC")] - async fn list_by_status(&self, status: &str) -> Result, DataError> { - unimplemented!() - } +By default every `Err` rolls back. Spring names exception *types* to refine that; +because Rust's `Result` already separates failure from success, the Firefly +analog names an error **pattern** (any match pattern for the fn's error type, +alternatives `A | B` included). Then: - // i64 return -> a count query. - #[query("SELECT COUNT(*) FROM accounts WHERE owner = :owner")] - async fn tally(&self, owner: &str) -> Result { unimplemented!() } +- `no_rollback_for = "P"` — **Spring's `noRollbackFor`**: an `Err` matching `P` + **commits** instead of rolling back; +- `rollback_only_for = "P"` — roll back **only** for errors matching `P`, + committing the rest; +- with both, `no_rollback_for` wins on overlap. - // u64 return -> a modifying statement; the value is the affected-row count. - #[query("UPDATE accounts SET status = :to WHERE status = :from")] - async fn retire(&self, from: &str, to: &str) -> Result { unimplemented!() } +```rust,ignore +// Persist the audit row even when the domain rejects the charge, but still roll +// back on any infrastructure failure — @Transactional(noRollbackFor = …). +#[firefly::transactional(no_rollback_for = "BillingError::Rejected(_)")] +async fn charge(&self, req: Charge) -> Result { + self.audit.save(/* … */).await?; // committed even on a Rejected error + self.gateway.settle(req).await // a Backend error still rolls back } ``` -`#[query("…")]` is shorthand for `#[query(sql = "…")]` (native SQL). For a -portable, entity-oriented query use the JPQL-like form -`#[query(jpql = "…", entity = "Account")]`, whose `FROM ` is transpiled -to the configured table so the same string runs on Postgres, MySQL, or SQLite. - -Under the hood the runtime engine does two things. `QueryMethodParser` parses the -method name — prefix (`find` / `count` / `exists` / `delete`), `By`, then a chain -of `And` / `Or` property conditions — into a query the active `SqlDialect` lowers -to a parameterized statement, so the same `find_by_owner_and_status` runs on -Postgres, MySQL, or SQLite; a trailing `Pageable` routes to -`find_by_derived_paged`. The `#[query(...)]` attribute, in turn, lowers to the -repository's `query_list` / `query_count` / `query_exists` / `query_execute` -helpers — list, count, exists, and modifying ops respectively — binding the -`:name` placeholders and (for the JPQL form) transpiling `FROM ` to the -table. - -> **Tip.** Reach for the method-name grammar for simple predicates and a trailing -> `Pageable` for paged reads; use `#[query(...)]` for anything the name grammar -> can't express. A relational `Account` / `Order` service writes these; Lumen's -> event-sourced read side stays a hand-rolled in-memory `ReadModel` instead. +> **Warning** There is no `rollback_for`. Spring's `rollbackFor` is *additive* — +> it adds exception types to the runtime-exceptions that already roll back. Rust +> has no checked/unchecked split (every `Err` rolls back by default), so an +> additive rule would be a no-op. `rollback_only_for` is therefore named to +> signal that it *restricts* (rather than widens) the rollback set, so a Spring +> port is never silently inverted. Writing `rollback_for` is a friendly compile +> error pointing you at the two rules above. -### Document — `firefly-data-mongodb` (MongoDB) +For programmatic control there is `firefly::transactional::transactional(opts, f)` +and `transactional_on(&manager, opts, f)` for an explicit manager, with +`TxOptions`, `Propagation`, and `Isolation` builders. The sqlx adapter — +`SqlxTransactionManager`, registered once at startup (the `auto_configure` path +in Step 9 does this for you) — supplies the real behaviour: full propagation, +isolation, read-only, a statement timeout, and `SAVEPOINT`-based `NESTED` +nesting. -`MongoRepository` puts a MongoDB collection behind the **same** -`ReactiveCrudRepository` + `ReactiveSpecificationRepository` traits, lowering a -`Specification` via `Specification::to_mongo()` exactly as the relational -adapters lower it via `to_sql`. A `BaseDocument` mixin (embedded with -`#[serde(flatten)]`) carries the audit stamps and soft-delete column, and -`with_soft_delete(policy)` makes every read inject a `{"": null}` guard -while `delete_by_id` becomes a logical delete. Reads stream lazily off the -driver cursor as a `Flux`. +> **Tip** **Checkpoint.** The partial-write protection is proven end-to-end in +> `firefly-data-sqlx`'s `tests/transactional.rs` and in `lumen-ledger`'s service +> tests: a transfer whose credit fails after the debit leaves *both* accounts +> unchanged. Run `cargo test -p lumen-ledger-core` to watch it. + +### Document store — `firefly-data-mongodb` + +The same ports reach a document database. `MongoRepository` puts a MongoDB +collection behind the **same** `ReactiveCrudRepository` + +`ReactiveSpecificationRepository` traits, lowering a `Specification` via +`Specification::to_mongo()` exactly as the relational adapters lower it via +`to_sql`. A `BaseDocument` mixin (embedded with `#[serde(flatten)]`) carries the +audit stamps and soft-delete column, and reads stream lazily off the driver +cursor as a `Flux`: ```rust,no_run -use firefly_data::{ReactiveCrudRepository, Specification}; +use firefly_data::ReactiveCrudRepository; use firefly_data_mongodb::{BaseDocument, MongoRepository}; use mongodb::bson::{Bson, Document}; use serde::{Deserialize, Serialize}; @@ -761,181 +982,52 @@ repo.save(WalletDocument { # } ``` -Because all four backends sit behind the same ports, a service that codes -against `Repository` / `ReactiveCrudRepository` / `Specification` can move from -Postgres to MySQL, SQLite, or MongoDB by swapping the adapter constructor — and -adding a *new* database is "write a `firefly-data-` crate that implements -the ports", not "rewrite the data layer." Both adapters are tested against real -Postgres, MySQL, SQLite, and MongoDB. +What just happened: because all four backends sit behind the same ports, a +service that codes against `ReactiveCrudRepository` / `Specification` moves from +Postgres to MySQL, SQLite, or MongoDB by swapping the adapter constructor. -> **One-dependency note.** Lumen pulls none of these adapters in its default -> build — they are opt-in cargo features on the `firefly` facade +> **Note** Lumen pulls none of these adapters in its default build — they are +> opt-in cargo features on the `firefly` facade > (`firefly = { version = "26.6", features = ["data-sqlx"] }`), re-exported as > `firefly::data_sqlx` / `firefly::data_mongodb`. The teaching build stays lean; -> the production build adds exactly the driver it needs. - -## Schema migrations - -`firefly-migrations` is a forward-only SQL migration runner. Migration files are -named `V{version}__{description}.sql` (e.g. `V001__init.sql`); each runs once, in -version order, inside a transaction. The runner works against any store behind -the small synchronous `Database` port: - -```rust,ignore -use firefly_migrations::{run, DirSource}; - -let src = DirSource { dir: "migrations".into() }; -run(&mut db, &src)?; // applies pending migrations in order -let status = firefly_migrations::inspect(&mut db, &src)?; // applied + pending -``` - -The [CLI](./19-cli.md) wraps this: `firefly db init`, `firefly db migrate -m -"create wallet_views"`, `firefly db upgrade --url sqlite://lumen.db`, and -`firefly db status`. A `V001__wallet_views.sql` creating the read-model table is -all the schema Lumen's durable read side needs. - -## Transactions — `#[firefly::transactional]` - -`firefly-transactional` brings declarative transactions to Rust: annotate an -`async fn` that returns `Result<_, E>` (where -`E: From`) with `#[firefly::transactional]` and -the body runs inside a transaction — **commit on `Ok`, rollback on `Err`**. - -```rust,ignore -use firefly::transactional::TxError; -use firefly_data_sqlx::SqlxReactiveRepository; - -#[derive(Debug)] -enum TransferError { /* … */ Tx(TxError) } -impl From for TransferError { fn from(e: TxError) -> Self { TransferError::Tx(e) } } - -struct Accounts { repo: SqlxReactiveRepository } -struct Ledger { repo: SqlxReactiveRepository } - -#[derive(Debug, Clone)] struct Account { id: String, balance: i64 } -#[derive(Debug, Clone)] struct Entry { id: String, account: String, delta: i64 } - -#[firefly::transactional( - propagation = "requires_new", - isolation = "serializable", - read_only, - timeout_ms = 5000, -)] -async fn record(accounts: &Accounts, ledger: &Ledger) -> Result<(), TransferError> { - // … repository writes here join this transaction automatically … - Ok(()) -} -``` - -The attributes are `propagation` (`required` / `requires_new` / `nested` / -`supports` / `not_supported` / `mandatory` / `never`), `isolation` -(`read_committed` / `repeatable_read` / `serializable` / …), `read_only`, -`timeout_ms`, `manager = ""` (Spring's `@Transactional("txManager")` — run -against an explicit `TransactionManager`, e.g. `self.tx_manager()`, instead of -the process-global registry), and the **rollback rules** `no_rollback_for` / -`rollback_only_for`. - -Spring names exception *types*; because Rust's `Result` already separates -failure from success, the Firefly analog names an error **pattern** (any match -pattern for the fn's error type, no `if` guard, alternatives `A | B` included). -By default every `Err` rolls back. Then: - -- `no_rollback_for = "P"` — **Spring's `noRollbackFor`**: an `Err` matching `P` - **commits** instead of rolling back; -- `rollback_only_for = "P"` — roll back **only** for errors matching `P`, - committing the rest; -- with both, `no_rollback_for` wins on overlap. +> the production build adds exactly the driver it needs. This is the same +> one-dependency story from [Quickstart](./02-quickstart.md): no starter to +> forget, no version skew. -```rust,ignore -// Persist the audit row even when the domain rejects the charge, but still roll -// back on any infrastructure failure — @Transactional(noRollbackFor = …). -#[firefly::transactional(no_rollback_for = "BillingError::Rejected(_)")] -async fn charge(&self, req: Charge) -> Result { - self.audit.save(/* … */).await?; // committed even on a Rejected error - self.gateway.settle(req).await // a Backend error still rolls back -} -``` - -> **Not `rollback_for`.** Spring's `rollbackFor` is *additive* — it adds -> exception types to the runtime-exceptions that already roll back. Rust has no -> checked/unchecked split (every `Err` rolls back by default), so an additive -> rule would be a no-op. `rollback_only_for` is therefore deliberately named to -> signal that it *restricts* (rather than widens) the rollback set, so a Spring -> port is never silently inverted. Writing `rollback_for` is a friendly compile -> error pointing you at the two rules above. - -**Ambient enlistment** is what makes this seamless. The manager opens a sqlx -transaction and stows it in a task-local stack; while that scope is active, -every `SqlxReactiveRepository` / `SqlxRepository` write *inside the fn* routes -onto the active transaction instead of a fresh pool connection — so a plain -sequence of `repo.save(...).await?` calls is atomic with **no change to the -repository code**. An atomic two-repository money transfer: - -```rust,ignore -use firefly::transactional::TxError; - -#[firefly::transactional] // defaults: REQUIRED, datasource isolation, read-write -async fn transfer( - accounts: &SqlxReactiveRepository, - ledger: &SqlxReactiveRepository, - from: Account, - to: Account, - amount: i64, -) -> Result<(), TxError> { - // All four writes enlist in the same ambient transaction. If any await - // returns Err, the whole unit of work rolls back; otherwise it commits. - accounts.save(Account { balance: from.balance - amount, ..from.clone() }) - .into_future().await?; - accounts.save(Account { balance: to.balance + amount, ..to.clone() }) - .into_future().await?; - ledger.save(Entry { id: "e1".into(), account: from.id, delta: -amount }) - .into_future().await?; - ledger.save(Entry { id: "e2".into(), account: to.id, delta: amount }) - .into_future().await?; - Ok(()) -} -``` - -For programmatic control there is `firefly::transactional::transactional(opts, -f)` and `transactional_on(&manager, opts, f)` for an explicit manager, with -`TxOptions`, `Propagation`, and `Isolation` builders. The sqlx adapter — -`SqlxTransactionManager`, registered once at startup (the `auto_configure` path -below does this for you) — supplies the real behaviour: full propagation -(`REQUIRED` / `REQUIRES_NEW` / `NESTED` / `SUPPORTS` / `NOT_SUPPORTED` / -`MANDATORY` / `NEVER`), isolation, read-only, a statement timeout, and -`SAVEPOINT`-based `NESTED` nesting. - -> **Note.** The killer feature is **ambient enlistment**: while a transactional -> scope is active, every repository write inside the fn joins the active -> transaction automatically — a plain sequence of `repo.save(...).await?` calls -> is atomic with no change to the repository code, carried on a task-local rather -> than a thread-bound connection. (Lumen's write side reaches durability -> differently — through the event store's optimistic-concurrency `append`, -> covered in [Event Sourcing](./11-event-sourcing.md) — but a relational -> `Account` / `Order` service spanning two repositories is exactly what -> `#[transactional]` is for.) - -## What changed in Lumen +## Recap Lumen now has a clear persistence story, even though its teaching build stays infrastructure-free: -- **The read store is a repository.** `ReadModel` (in `samples/lumen/src/ledger.rs`) - is a `#[derive(Repository)]` data-access bean wrapping a - `Mutex>` and exposing exactly `upsert(view)` and - `find(id)` — the two operations the query side needs, keyed by the - `WalletView`'s own id. The `GetWallet` handler depends on the *contract*, not - the map. +- **The read store is a repository.** `ReadModel` (in + `samples/lumen/src/ledger.rs`) is a `#[derive(Repository)]` data-access bean + wrapping a `Mutex>` and exposing exactly + `upsert(view)` and `find(id)` — the two operations the query side needs. The + `GetWallet` handler depends on the *contract*, not the map. - **The baseline is in-memory by choice.** The map keeps the dependency footprint - at one Firefly crate; the comment in the source names the upgrade explicitly — - "a real service would back this with firefly's reactive repository over - Postgres." + at one Firefly crate; the source comment names the upgrade explicitly. - **The upgrade is an adapter swap.** Lowering `ReadModel` onto `ReactiveCrudRepository` — `SqlxReactiveRepository` for Postgres/MySQL/SQLite, `MongoRepository` for Mongo — turns `upsert` into `save` - and `find` into `find_by_id`, with the query handler's `Option` - shape unchanged. A new database is a new pool (relational) or a new adapter - crate, never a rewrite. + and `find` into `find_by_id`, with the handler's `Option` shape + unchanged. A new database is a new pool (relational) or a new adapter crate, + never a rewrite. + +You also now know: + +- How to compose a `Filter`, render it with `to_sql()`, and read a `Page`. +- That the reactive surface returns `Mono` / `Flux`; you drive them with + `block().await` (→ `Result, _>`) and `collect_list()`, and a miss is + an empty `Mono`. +- That `Pageable::of(page, size, sort)` is **1-based** and returns a `Result`. +- That `#[derive(Entity)]` + `#[derive(SqlxRepository)]` give you a Spring + Data-style repository, and `#[firefly::repository]` adds derived and + `#[query(...)]` queries. +- That `with_version_column` is `@Version` optimistic locking, that + `auto_configure` builds the pool and registers the transaction manager in one + awaited call, and that `#[firefly::transactional]` makes a multi-write change + atomic through ambient enlistment — with rollback *patterns*, not + `rollback_for`. ## Exercises @@ -946,20 +1038,36 @@ infrastructure-free: query side depends on the contract, not the map. 2. **Back the read model with SQLite.** Using the `SqlxReactiveRepository` - listing above, create a `wallet_views` table in `sqlite::memory:`, `save` two - views, and `find_all().collect_list()` them. Assert both come back. This is - the production read store, exercised end to end against the real adapter. + listing from Step 6, create a `wallet_views` table in `sqlite::memory:`, + `save` two views, and `find_all().collect_list()` them. Assert both come back. + This is the production read store, exercised end to end against the real + adapter with no external infrastructure. 3. **Page the wallets.** Build a `Filter` that selects wallets with `balance >= 100_000` ordered by `version` descending, page `(0, 20)`, and print its `to_sql()`. Then describe (one sentence each) how the same filter - would render under `MySqlDialect` and `SqliteDialect`. + would render under `MySqlDialect` and `SqliteDialect` via `to_sql_with`. -4. **Trace the swap.** List the exact lines in `samples/lumen/src/ledger.rs` that - would change if `ReadModel` became a `SqlxReactiveRepository`, and which lines in `samples/lumen/src/commands.rs` (the `GetWallet` - handler) would *not* — confirming the adapter boundary holds. +4. **Add a derived query.** On a `#[derive(SqlxRepository)]` repository, add a + `#[firefly::repository]` method `find_by_owner(&self, owner: &str) -> + Result, DataError>`, then a paged variant + `find_by_status(&self, status: &str, page: Pageable) -> Result, DataError>`. + Build the `Pageable` with `Pageable::of(1, 20, RequestSort::by(["id"]))` and + confirm it compiles. Note that `page` is the *first* page, not the second. -With persistence framed, the next chapter models the business itself — the -`Money` value object and the `Wallet` aggregate. See -[Domain-Driven Design](./08-domain-driven-design.md). +5. **Trace the swap.** List the exact lines in `samples/lumen/src/ledger.rs` that + would change if `ReadModel` became a `SqlxReactiveRepository`, and which lines in the `GetWallet` handler would *not* — confirming + the adapter boundary holds. + +## Where to go next + +- Model the business itself — the `Money` value object and the `Wallet` + aggregate — in **[Domain-Driven Design](./08-domain-driven-design.md)**. +- See how the read store and the write store are split, and how the query side + reads from the repository you just framed, in **[CQRS](./09-cqrs.md)**. +- Watch the projection that *writes* to `ReadModel` come alive in + **[EDA & Messaging](./10-eda-messaging.md)** and + **[Event Sourcing](./11-event-sourcing.md)**. +- See the full Spring Data-style persistence layer wired across crates in + **[Layered Microservices](./22-layered-microservices.md)**. diff --git a/docs/book/src/08-domain-driven-design.md b/docs/book/src/08-domain-driven-design.md index e9723f1..5473aa7 100644 --- a/docs/book/src/08-domain-driven-design.md +++ b/docs/book/src/08-domain-driven-design.md @@ -1,56 +1,146 @@ # Domain-Driven Design -Lumen can open wallets and read them back, and it has a place to put the read -model. But look closely and something is missing: nothing yet *owns* the rules. -Where is "you cannot withdraw more than you hold"? Where is "an amount must be -positive"? Where is "a wallet must have an owner"? Right now those would live as -scattered `if`-statements in a handler — exactly the kind of rule a future -developer can bypass by writing to the store directly. - -**Domain-Driven Design** fixes that by making the model responsible for its own -invariants. This chapter builds Lumen's domain core: the `Money` **value -object** (immutable, integer cents, exact arithmetic) and the `Wallet` -**aggregate** that guards the overdraft, positive-amount, and owner-required -rules — each command raising the domain event that records what happened. Both -files are drawn verbatim from `samples/lumen`. - -> **By the end of this chapter, Lumen will** have `src/money.rs` and the heart -> of `src/domain.rs`: a `Money` value object closed under `add` / `subtract`, a -> `Wallet` aggregate with `open` / `deposit` / `withdraw` that enforce the -> invariants and `raise` events, and a typed `DomainError` family whose `Display` -> strings surface verbatim as RFC 9457 problem details. The aggregate carries -> `#[derive(AggregateRoot)]`; nothing carries `thiserror`. - -> **Design note.** This chapter builds Lumen's domain core with the classic DDD -> building blocks — value objects, aggregates, aggregate roots, domain events — -> as idiomatic Rust. `#[derive(AggregateRoot)]` generates the `AGGREGATE_TYPE` -> constant and the `aggregate()` / `aggregate_mut()` accessors over the embedded -> `AggregateRoot` (which itself carries the uncommitted-event buffer and -> version), so your code holds only the rules. - -## The `Money` value object - -A value object is defined entirely by its attributes — it has no identity — and -it is **immutable**: every operation returns a *new* value rather than mutating -in place. Money is the textbook example, and getting it right matters more here -than almost anywhere: amounts are stored as integer **minor units** (cents), so -the arithmetic is exact. No binary floating-point drift, the classic correctness -bug a money type exists to prevent. - -Here is Lumen's `Money`, the whole core of `src/money.rs`: +Lumen can already open wallets and read them back, and it has a place to put the +read model. But look closely and something is missing: nothing yet *owns* the +rules. Where is "you cannot withdraw more than you hold"? Where is "an amount +must be positive"? Where is "a wallet must have an owner"? Right now those would +live as scattered `if`-statements in a handler — exactly the kind of rule a +future developer can bypass by writing to the store directly. + +**Domain-Driven Design (DDD)** fixes that by making the model responsible for +its own invariants. In this chapter you build Lumen's domain core from first +principles: the `Money` *value object* (immutable, integer cents, exact +arithmetic) and the `Wallet` *aggregate* that guards the overdraft, +positive-amount, and owner-required rules — each command raising the domain +event that records what happened. Both files are drawn verbatim from +[`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen), +so the crate you grow here matches the finished service line for line. + +This is pure Rust modelling: there is no HTTP, no database, and no framework +runtime in the hot path. The framework contributes exactly two derives — +`#[derive(AggregateRoot)]` and `#[derive(DomainEvent)]` — and otherwise stays +out of your way, which is the point: your rules live in plain methods you can +unit-test with no I/O at all. + +By the end of this chapter you will: + +- Tell a **value object** apart from an **entity / aggregate**, and know why + `Money` is the former and `Wallet` the latter. +- Build a `Money` value object that is immutable, stored as integer cents, and + closed under the operations a wallet needs (`add` / `subtract` / + `require_positive`). +- Build the `Wallet` aggregate so its overdraft, positive-amount, and + owner-required invariants are *physically* unreachable from outside — + validated before any event is raised. +- Use `#[derive(AggregateRoot)]` and `#[derive(DomainEvent)]` to embed the + framework's event buffer and stamp typed events, writing only the rules + yourself. +- Map domain failures to a typed `DomainError` family whose `Display` strings + surface verbatim as RFC 9457 problem details. +- Prove every invariant with plain unit tests — no database, no HTTP. + +## Concepts you will meet + +Before the first line of code, here are the DDD ideas this chapter leans on. +Each is reintroduced in context where it is first used; this is the short +version. + +> **Note** **Key term — value object.** A *value object* is a domain type +> defined entirely by its attributes — it has **no identity** — and is +> **immutable**: every operation returns a *new* value instead of mutating in +> place. Two value objects with equal attributes are equal, full stop. `Money` +> is the textbook example. The Java/DDD analog is exactly a value object (a +> JPA `@Embeddable`, or a Java `record` used as a value). + +> **Note** **Key term — entity and aggregate.** An *entity* has an **identity** +> that persists across changes (a wallet stays "the same wallet" as its balance +> moves). An *aggregate* is a cluster of entities and value objects treated as +> one unit, with a single **aggregate root** as its sole entry point — the +> consistency boundary through which every change must flow. `Wallet` is the +> aggregate root here. In Spring/JPA terms this is the `@Entity` that owns its +> children and guards its invariants. + +> **Note** **Key term — domain event.** A *domain event* is an immutable record +> of something that happened in the domain, in the past tense (`WalletOpened`, +> `MoneyDeposited`). The aggregate *raises* one whenever it changes state, so +> the change is captured as a fact rather than left implicit. This is the same +> notion as a Spring `ApplicationEvent` published from a domain method, but here +> the events are also the persisted source of truth (you will see that fully in +> [Event Sourcing](./11-event-sourcing.md)). + +> **Note** **Key term — invariant.** An *invariant* is a rule that must hold for +> the model to be valid — "balance never goes below zero", "owner is never +> blank". An aggregate's job is to make its invariants impossible to violate +> from outside. There is no Spring annotation for this; it is the discipline the +> aggregate boundary exists to enforce. + +The chapter builds two files: `src/money.rs` (the value object) and +`src/domain.rs` (the aggregate, its events, the error family, and the read-model +view). You declared both in the `mod` list back in +[Quickstart](./02-quickstart.md), so nothing in `main.rs` changes — you are +filling modules the entry point already names. + +## Step 1 — Define the `Money` value object's shape + +Start with representation. `Money` solves *how an amount is stored and +compared*; the `Wallet` aggregate (Step 5 onward) will solve *behavior*. Getting +the representation right matters more here than almost anywhere: amounts are +stored as integer **minor units** (cents), so the arithmetic is exact — no +binary floating-point drift, the classic correctness bug a money type exists to +prevent. + +Create `src/money.rs` and declare the struct and its imports: ```rust,ignore // samples/lumen/src/money.rs use std::fmt; + use serde::{Deserialize, Serialize}; /// An exact monetary amount, expressed in integer minor units (cents). #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)] #[serde(transparent)] pub struct Money { + /// The amount in minor units (cents). Kept private so the only way to a + /// `Money` is through the validating constructors. cents: i64, } +``` + +What just happened, decision by decision: +- **Integer cents, never a float.** The single field is `cents: i64`, kept + *private* so the only way to a `Money` is through a constructor. €10.00 is + `Money::cents(1_000)`; €12.50 after an addition is `Money::cents(1_250)`. The + math is exact by construction — there is no `f64` anywhere to drift. +- **A value object is compared by value.** The `PartialEq, Eq, PartialOrd, Ord` + derives make two `Money`s equal exactly when their cents are equal, and + orderable so the wallet can ask "is this amount more than the balance?". +- **`Copy`, because it is a value.** `Money` is a single `i64`, so it copies + freely; you never juggle references to it. +- **`#[serde(transparent)]` is the wire contract.** `Money` serializes as the + *bare cent integer* — a balance of €10.00 is the JSON number `1000`, not + `{ "cents": 1000 }`. That is the contract the read model and the event + payloads share, and it is why the field can stay private without hurting the + wire shape. + +> **Note** **Key term — minor units.** *Minor units* are the smallest +> indivisible unit of a currency — cents for euros and dollars. Storing money as +> an integer count of minor units (1000 cents, not 10.00 euros) keeps arithmetic +> exact. This is the same discipline as a database `BIGINT` cents column or a +> Java `long` of minor units. + +> **Tip** **Checkpoint.** `src/money.rs` exists with a `Money` struct holding one +> private `cents: i64` field. It will not compile yet — the constructors come +> next — but the shape is locked in. + +## Step 2 — Give `Money` immutable, validating operations + +A value object exposes only operations that *return new values*. Add the +constructors, the accessors, and the three operations a wallet needs. Append +this `impl` block to `src/money.rs`: + +```rust,ignore impl Money { /// A zero amount — the opening balance of a brand-new wallet. pub const ZERO: Money = Money { cents: 0 }; @@ -75,6 +165,11 @@ impl Money { self.cents > 0 } + /// Whether this amount is zero. + pub const fn is_zero(self) -> bool { + self.cents == 0 + } + /// Returns a new `Money` that is `self + other` (immutable addition). #[must_use] pub const fn add(self, other: Money) -> Money { @@ -100,24 +195,40 @@ impl Money { } ``` -Several design choices are load-bearing: +What just happened — four design choices are load-bearing here: -- **Integer cents, never a float.** The single field is `cents: i64`, kept - private so the only way to a `Money` is through a constructor. €10.00 is - `Money::cents(1_000)`; €12.50 after an addition is `Money::cents(1_250)`. The - math is exact by construction. -- **Immutable.** `add` is `#[must_use]` and `const`; it returns a fresh `Money` - and leaves the operands untouched. So does `subtract`. There is no - `add_assign` — a value object is replaced, not edited. +- **Immutable.** `add` is `#[must_use]` and `const`; it returns a *fresh* + `Money` and leaves the operands untouched. So does `subtract`. There is no + `add_assign` — a value object is replaced, not edited. `#[must_use]` makes the + compiler warn if you call `add` and forget to use the result, catching the + "I thought this mutated in place" bug at compile time. - **Closed under the wallet's operations.** `add` for credits, `subtract` (fallible, guarding against overdraw) for debits, and `require_positive` for - the guard every mutating command runs before raising an event. -- **`#[serde(transparent)]`.** `Money` serializes as the bare cent integer — a - balance of €10.00 is the JSON number `1000`, the contract the read model and - the event payloads share. No `{ "cents": 1000 }` wrapper. - -`Display` renders the human form (`1250` cents → `"12.50"`) for logs and the -banner, and the error type is hand-written: + the guard every mutating command runs before raising an event. "Closed" means + every operation a wallet performs takes `Money` and yields `Money` (or a + `MoneyError`), so amounts never leak into raw integers. +- **`subtract` is where overdraft lives.** It returns `Result`: subtracting more than you hold is `MoneyError::Overdraw`, not a + silent negative balance. This is the *only* place the "never below zero" rule + is checked — the aggregate reuses it rather than re-implementing it. +- **`const` where possible.** `ZERO`, `cents`, `from_units`, `cents_value`, + `is_positive`, `is_zero`, and `add` are `const fn`, so `Money::ZERO` and + `Money::cents(100)` can be used in const contexts. `subtract` and + `require_positive` are not `const` because they return a `Result`. + +> **Note** **Key term — `#[must_use]`.** Annotating a function with `#[must_use]` +> tells the compiler to warn when its return value is ignored. On an immutable +> operation like `add` it is the guardrail that turns "I forgot the result is a +> new value" into a compile-time warning instead of a lost update. + +## Step 3 — Render and report `Money` failures by hand + +Two more pieces complete the value object: a human-readable `Display`, and the +typed error its operations return. Lumen hand-writes both `Display` and +`std::error::Error` for `MoneyError` rather than deriving them — that keeps the +book's one-dependency promise honest all the way down to the error enums. + +Add the error type and the two `Display` impls to `src/money.rs`: ```rust,ignore /// The typed error a `Money` operation can fail with. @@ -139,41 +250,136 @@ impl fmt::Display for MoneyError { } impl std::error::Error for MoneyError {} -``` -> **One-dependency note.** `MoneyError` hand-writes `Display` and -> `std::error::Error` rather than deriving them with `thiserror`. That is on -> purpose: Lumen depends on exactly one Firefly crate plus `axum` and `serde`, -> and the book keeps that promise *all the way down* — even the error enums. Two -> trait impls per error type is a small price for an honest dependency list. - -> **Design note.** `Money` is a frozen, value-equal type — Rust makes the -> immutability a compiler guarantee. The "integer minor units, never a float" -> discipline is what any correct money type needs: floating-point drift is the -> classic correctness bug a money type exists to prevent. - -## The `Wallet` aggregate +impl fmt::Display for Money { + /// Renders the amount as a fixed two-decimal major-unit string + /// (`1250` cents → `"12.50"`), the human-readable form used in logs. + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let sign = if self.cents < 0 { "-" } else { "" }; + let abs = self.cents.abs(); + write!(f, "{sign}{}.{:02}", abs / 100, abs % 100) + } +} +``` -`Money` solves *representation*. The `Wallet` aggregate owns *behavior*: it is -the consistency boundary, the single entry point through which every change to a +What just happened: + +- **`MoneyError` is a closed enum** with two cases — exactly the two ways a + `Money` operation can fail. Because it derives `PartialEq, Eq`, tests can + assert `err == MoneyError::Overdraw` directly. +- **`Display` carries the message text**, and `impl std::error::Error for + MoneyError {}` makes it a first-class error so `?` and trait objects work. The + empty block is enough because `Error` has default methods. +- **`Money`'s own `Display`** turns `1250` into `"12.50"` for logs and the + banner — the major-unit form a human reads, kept separate from the + minor-unit form the wire carries. + +> **Design note.** `MoneyError` hand-writes `Display` and `std::error::Error` +> rather than deriving them with `thiserror`. That is on purpose: Lumen depends +> on exactly one Firefly crate plus `axum` and `serde`, and the book keeps that +> promise all the way down — even the error enums. Two trait impls per error type +> is a small price for an honest dependency list, and `Money` itself is a frozen, +> value-equal type that Rust makes immutable as a compiler guarantee. + +> **Tip** **Checkpoint.** Run `cargo test --lib money` (or `cargo build`). +> `src/money.rs` now compiles on its own: a private-field value object, three +> operations, a two-variant error, and a `Display` that prints `"12.50"`. The +> arithmetic is exact and the type cannot be constructed into an invalid state. + +## Step 4 — Set up the `Wallet` aggregate and its events + +`Money` solved representation. The `Wallet` aggregate owns *behavior*: it is the +consistency boundary, the single entry point through which every change to a wallet must flow, so the invariants cannot be bypassed. -Lumen's `Wallet` is event-sourced (the full event-store machinery arrives in -[Event Sourcing](./11-event-sourcing.md)), but the DDD shape is visible already. -The aggregate embeds the framework's `AggregateRoot` — the uncommitted-event -buffer plus a version — and `#[derive(AggregateRoot)]` generates the -`AGGREGATE_TYPE` constant and the `aggregate()` / `aggregate_mut()` accessors: +Lumen's `Wallet` is event-sourced — every command produces a domain event, and +the wallet's state is the *result* of folding those events. The full event-store +machinery arrives in [Event Sourcing](./11-event-sourcing.md); here you build +just the DDD shape. The aggregate embeds the framework's `AggregateRoot` (an +uncommitted-event buffer plus a version), and two derives do the mechanical +work. + +> **Note** **Key term — `#[derive(AggregateRoot)]`.** This derive finds the +> embedded `firefly` `AggregateRoot` field on your struct and generates an +> `AGGREGATE_TYPE` associated constant plus `aggregate()` / `aggregate_mut()` +> accessors over it. The embedded `AggregateRoot` itself carries the +> uncommitted-event buffer, the aggregate id, and the version — so your struct +> holds only the projected state and the rules. The Spring/Axon analog is an +> `@Aggregate` root that the framework manages. + +> **Note** **Key term — `#[derive(DomainEvent)]`.** This derive stamps an event +> payload struct with a stable `EVENT_TYPE` discriminator (its struct name) and +> generates a `to_domain_event(...)` conversion onto the framework's wire event. +> You declare the payload as a plain serializable struct; the derive supplies the +> type tag so you never spell event names as bare string literals at the call +> sites. + +Create `src/domain.rs` with its imports, the `AGGREGATE_TYPE` constant, and the +three event payloads: ```rust,ignore // samples/lumen/src/domain.rs use firefly::eventsourcing::{AggregateRoot, DomainEvent}; use firefly::prelude::*; +use serde::{Deserialize, Serialize}; use crate::money::{Money, MoneyError}; /// The aggregate-type discriminator stamped onto every event a Wallet raises. +/// `#[derive(AggregateRoot)]` also exposes it as `Wallet::AGGREGATE_TYPE`. pub const AGGREGATE_TYPE: &str = "Wallet"; +/// Payload of the event raised when a wallet is opened. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, DomainEvent)] +pub struct WalletOpened { + pub wallet_id: String, + pub owner: String, + /// The opening balance, in minor units (cents). + pub opening_balance: i64, +} + +/// Payload of the event raised when money is credited to a wallet. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, DomainEvent)] +pub struct MoneyDeposited { + pub wallet_id: String, + /// The deposited amount, in minor units (cents). + pub amount: i64, +} + +/// Payload of the event raised when money is debited from a wallet. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, DomainEvent)] +pub struct MoneyWithdrawn { + pub wallet_id: String, + /// The withdrawn amount, in minor units (cents). + pub amount: i64, +} +``` + +What just happened, line by line: + +- **`use firefly::eventsourcing::{AggregateRoot, DomainEvent};`** imports the two + framework *types* — the embeddable root struct and the wire-event struct. +- **`use firefly::prelude::*;`** brings the framework's derive macros into scope, + including `AggregateRoot`, `DomainEvent`, and `Schema` (used by the read-model + view in Step 8). Everything reaches you through the single `firefly` facade. +- **`AGGREGATE_TYPE`** is the string discriminator stamped onto every event the + wallet raises. It is declared as a public constant *and* re-exposed as + `Wallet::AGGREGATE_TYPE` by the derive, so both spellings name the same value. +- **Each event payload is past tense** (`WalletOpened`, not `OpenWallet`) and + carries `#[derive(DomainEvent)]`. The derive gives each one a `EVENT_TYPE` + const equal to its struct name (`"WalletOpened"`, etc.), which you use at the + raise sites instead of typing the string by hand. + +## Step 5 — Declare the aggregate root struct + +Now the aggregate itself. It embeds the framework's `AggregateRoot` as a field +named `root`, carries the projected state (`owner`, `balance`, `opened`), and +derives `AggregateRoot` to generate the accessors and the `AGGREGATE_TYPE` +constant. + +Add the struct to `src/domain.rs`: + +```rust,ignore /// The event-sourced wallet aggregate. #[derive(Debug, Clone, AggregateRoot)] #[firefly(aggregate_type = "Wallet")] @@ -189,13 +395,35 @@ pub struct Wallet { } ``` -The aggregate's projected state — `owner`, `balance`, `opened` — is the *result* -of applying its events. The behavior methods enforce the rules. +What just happened: + +- **`root: AggregateRoot`** is the embedded framework field. It holds the + uncommitted-event buffer, the aggregate id (`root.id`), and the version + (`root.version`). The derive locates this field by its type. +- **`#[firefly(aggregate_type = "Wallet")]`** tells the derive what string to use + for `Wallet::AGGREGATE_TYPE`. It matches the `AGGREGATE_TYPE` constant you + declared in Step 4 — both name `"Wallet"`. +- **`owner` / `balance` / `opened`** are the *projected state* — the result of + applying the wallet's events. `balance` is a `Money` value object, so the + aggregate reuses all the exact-arithmetic guarantees from Steps 1–3. `opened` + distinguishes a real wallet from an empty event stream ("absent"). +- **`Clone`** lets a handler take a working copy of a rehydrated wallet without + touching the original — useful under the eventual consistency CQRS introduces. + +> **Tip** **Checkpoint.** `cargo build` still fails — `Wallet` has no methods yet +> and `DomainError` is undefined — but the derives should resolve. If the +> compiler complains that it cannot find an `AggregateRoot` field, confirm the +> embedded field is typed exactly `AggregateRoot` (from +> `firefly::eventsourcing`), not your own type. + +## Step 6 — Write the factory method: `open` -### The factory: `open` +`open` is the *sole* way to bring a wallet into existence. It validates inputs, +constructs the aggregate, and `raise`s the opening event. Using a factory rather +than a public constructor guarantees the `WalletOpened` event is never forgotten +— there is no back-channel that produces a wallet without recording its birth. -`open` is the sole way to bring a wallet into existence. It validates inputs, -constructs the aggregate, and `raise`s the opening event: +Add an `impl Wallet` block with `open`: ```rust,ignore impl Wallet { @@ -221,7 +449,11 @@ impl Wallet { }; wallet.raise( WalletOpened::EVENT_TYPE, - &WalletOpened { wallet_id: id, owner, opening_balance: opening_balance.cents_value() }, + &WalletOpened { + wallet_id: id, + owner, + opening_balance: opening_balance.cents_value(), + }, ); wallet.balance = opening_balance; wallet.opened = true; @@ -230,20 +462,41 @@ impl Wallet { } ``` -Two invariants are enforced before any event is raised: the owner must be -non-blank (`OwnerRequired`), and the opening balance must not be negative (a -*zero* opening balance is allowed). Using a factory rather than a public -constructor guarantees the `WalletOpened` event is never forgotten — there is no -back-channel that produces a wallet without recording its birth. - -### The behavior methods: `deposit` and `withdraw` +What just happened, in order: + +- **Validate first, construct second.** Two invariants are enforced *before* any + event is raised: the owner must be non-blank (`OwnerRequired`), and the opening + balance must not be negative (a *zero* opening balance is explicitly allowed — + the check is `< 0`, not `<= 0`). +- **`AggregateRoot::new(&id, AGGREGATE_TYPE)`** constructs the embedded root with + this wallet's id and its aggregate-type tag, at version 0 with an empty event + buffer. +- **`wallet.raise(WalletOpened::EVENT_TYPE, &WalletOpened { ... })`** records the + birth event. `WalletOpened::EVENT_TYPE` is the discriminator the + `#[derive(DomainEvent)]` generated (the string `"WalletOpened"`), so the call + site never spells it by hand. `raise` is a small helper you add in Step 9; it + serializes the payload and pushes it onto `root`. +- **State is updated after the event.** `wallet.balance = opening_balance` and + `wallet.opened = true` set the projected state to match what the event + describes. The event is the fact; the fields are the cached projection of it. + +> **Note** **Key term — factory method.** A *factory method* is a static +> (associated) function that builds a fully valid instance, instead of exposing a +> public constructor. It is the one door into the aggregate, so it can enforce +> the birth invariants and guarantee the `WalletOpened` event is always raised. +> The Spring/DDD analog is a static factory on the aggregate root (or a domain +> service that produces it). + +## Step 7 — Write the behavior methods: `deposit` and `withdraw` The two mutating commands follow one shape: check the wallet exists, validate the amount, apply the `Money` operation, raise the event, update state. The -overdraft rule is enforced exactly once — by `Money::subtract`: +overdraft rule is enforced exactly once — by `Money::subtract`, which you already +built in Step 2. + +Add these methods to the same `impl Wallet` block: ```rust,ignore -impl Wallet { /// Credits `amount` to the wallet, raising a `MoneyDeposited` event. pub fn deposit(&mut self, amount: Money) -> Result<(), DomainError> { self.require_opened()?; @@ -276,24 +529,42 @@ impl Wallet { Err(DomainError::NotFound(self.root.id.clone())) } } -} ``` -Read `withdraw` carefully — it is where the consistency boundary earns its keep. -The order is *validate, then mutate*: `require_opened`, `require_positive`, and -the `subtract` overdraft check all run **before** the event is raised. If the -withdrawal would overdraw, `Money::subtract` returns `MoneyError::Overdraw`, the -`?` converts it to `DomainError::InsufficientFunds` (via the `From` impl below), -and the method returns *without raising anything*. The aggregate's invariant — -"balance never goes below zero" — is physically unreachable from outside, -because the only path to a withdrawal runs this gauntlet first. - -### The typed `DomainError` family +What just happened — read `withdraw` carefully, because it is where the +consistency boundary earns its keep: + +- **The order is *validate, then mutate*.** `require_opened()`, + `require_positive()`, and the `subtract` overdraft check all run **before** the + event is raised. If the withdrawal would overdraw, `Money::subtract` returns + `MoneyError::Overdraw`, the `?` converts it to `DomainError::InsufficientFunds` + (via the `From` impl you add in Step 8), and the method returns *without + raising anything*. +- **The invariant is unreachable from outside.** "Balance never goes below zero" + cannot be violated, because the only path to a withdrawal runs this gauntlet + first. There is no setter on `balance`, no way to skip `subtract`. That is the + difference between a service-level guard (a convention someone can forget) and + an aggregate invariant (a physical constraint). +- **`require_opened` turns "absent" into a typed error.** A command against a + wallet that was never opened returns `DomainError::NotFound(id)`, which the web + boundary later maps to a 404. `deposit` and `withdraw` both gate on it first. +- **`self.root.id.clone()`** reads the id off the embedded root to stamp each + event with the wallet it belongs to. + +> **Note** The whole reason an aggregate exists is to be the *only* +> way to change its state. Because `deposit` and `withdraw` take `&mut self` and +> there are no public setters, every mutation funnels through these methods and +> past their guards. A future developer cannot "just write to the store" and +> skip the rules — the rules are the door. + +## Step 8 — Add the typed `DomainError` family The errors are a closed enum with stable `Display` strings — stable because tests assert on them and they surface verbatim as the RFC 9457 problem `detail` -once mapped at the HTTP boundary (see [CQRS](./09-cqrs.md) and -[First HTTP API](./06-first-http-api.md)): +once mapped at the HTTP boundary (you wire that mapping in [CQRS](./09-cqrs.md) +and [Your First HTTP API](./06-first-http-api.md)). + +Add `DomainError` and its impls to `src/domain.rs`: ```rust,ignore /// The typed domain-error family. @@ -332,26 +603,51 @@ impl From for DomainError { } ``` -The `From for DomainError` impl is what lets `withdraw` write -`self.balance.subtract(amount)?` and have an overdraw surface as -`InsufficientFunds` — the value object reports the arithmetic fact, the -aggregate translates it into domain language. Like `MoneyError`, `DomainError` -hand-writes its `Display` and `Error` impls: no `thiserror`, one dependency. - -> **Note.** Lumen returns a typed `DomainError` rather than throwing: +What just happened: + +- **`From for DomainError` is the bridge.** It is what lets + `withdraw` write `self.balance.subtract(amount)?` and have an arithmetic + overdraw surface as `InsufficientFunds`. The value object reports the + arithmetic fact (`Overdraw`); the aggregate translates it into domain language + (`InsufficientFunds`). The `?` operator calls this `From` impl automatically. +- **The `Display` strings are the contract.** `"insufficient funds"`, + `"amount must be positive"`, `"wallet {id} not found"`, `"owner is required"` + — these exact strings become the RFC 9457 problem `detail` at the web + boundary, and tests assert on them, so they are stable. +- **Hand-written `Display` and `Error`, again.** Like `MoneyError`, `DomainError` + spells out its `Display` and `Error` impls instead of deriving with + `thiserror`: no extra crate, one dependency. + +> **Note** Lumen returns a typed `DomainError` rather than throwing. > `InsufficientFunds` / `NonPositiveAmount` / `OwnerRequired` become 422 problems -> and `NotFound` becomes a 404, decided by a `match` at the web boundary (a -> returned value, checked by the compiler — no exception-to-status table to keep -> in sync). +> and `NotFound` becomes a 404, decided by a `match` at the web boundary — a +> returned value, checked by the compiler, with no exception-to-status table to +> keep in sync. You will write that `match` in [CQRS](./09-cqrs.md). + +> **Tip** **Checkpoint.** `cargo build` now resolves `Wallet::open` / +> `deposit` / `withdraw` and their error type. The `raise` helper is still +> missing (next step), so the build is not green yet — but every domain rule is +> now expressed as code. -## The read-model view +## Step 9 — Add the `raise` helper and the read-model view -The aggregate is the *write* model. What `GET /api/v1/wallets/:id` returns is a -flat, query-optimized `WalletView` — the read model the previous chapter framed -as a repository. The aggregate produces it on demand: +Two pieces finish `src/domain.rs`. First, the private `raise` helper that the +command methods call — it serializes a `#[derive(DomainEvent)]` payload and +pushes it onto the embedded root. Second, the flat read-model view the aggregate +hands out. + +> **Note** **Key term — read model / projection.** A *read model* (or +> *projection*) is a flat, query-optimized view of an aggregate, separate from +> the rich aggregate itself. The aggregate is the *write* model — rule-enforcing, +> event-raising; the read model is what queries return — serializable, no +> behavior. Keeping them separate is the heart of CQRS +> ([CQRS](./09-cqrs.md)). The Spring analog is a JPA read projection / DTO +> distinct from the managed entity. + +Add the `view` method and `raise` helper to the `impl Wallet` block, then the +`WalletView` struct: ```rust,ignore -impl Wallet { /// The current read-model view of this aggregate. pub fn view(&self) -> WalletView { WalletView { @@ -361,6 +657,13 @@ impl Wallet { version: self.root.version, } } + + /// Serialises a `#[derive(DomainEvent)]` payload and raises it onto the + /// embedded root under `event_type`. + fn raise(&mut self, event_type: &str, payload: &P) { + let bytes = serde_json::to_vec(payload).expect("domain event payload serialises"); + self.root.raise(event_type, bytes); + } } /// The read-model projection of a wallet — the wire shape served by @@ -376,78 +679,160 @@ pub struct WalletView { } ``` +What just happened: + +- **`raise` is the one place events are serialized.** It takes the + `EVENT_TYPE` discriminator and a serializable payload, encodes the payload to + bytes with `serde_json::to_vec`, and calls `self.root.raise(event_type, + bytes)` — the framework method on `AggregateRoot` that appends to the + uncommitted-event buffer and bumps the version. Every command method routes + through this helper, so the serialization lives in exactly one spot. +- **`view()` produces the read model on demand.** It copies `id`, `owner`, + `balance` (as bare cents via `cents_value()`), and `version` (off + `root.version`) into a flat `WalletView`. The aggregate never serializes + *itself* — it hands out a view. +- **`WalletView` derives `Schema`.** That makes it appear in the auto-generated + OpenAPI docs as a component schema, so `GET /api/v1/wallets/:id`'s response is + documented with zero extra code (see [OpenAPI](./06a-openapi.md)). +- **`version` lets a client detect staleness.** Under the eventual consistency + CQRS introduces, a client can compare versions to notice it read a stale + projection. + Keeping `Wallet` (rich, rule-enforcing, event-raising) and `WalletView` (flat, serializable, no behavior) as separate types is the same domain/persistence -separation the previous chapter drew around the repository: the aggregate never -serializes itself, and the wire shape never carries an invariant. The `version` -field lets a client detect staleness under the eventual consistency CQRS -introduces. +separation [Persistence](./07-persistence.md) drew around the repository: the +aggregate never serializes itself, and the wire shape never carries an +invariant. -## Proving the invariants +> **Tip** **Checkpoint.** `cargo build` is green. `src/money.rs` and +> `src/domain.rs` both compile, and `Wallet::open(...).view()` round-trips a +> wallet from a factory call to a serializable view — with every rule enforced in +> between. + +## Step 10 — Prove the invariants with unit tests Because the aggregate is a plain struct with plain methods, you can exercise -every rule with no database and no HTTP — the unit tests that ship in -`samples/lumen/src/domain.rs`: +every rule with no database and no HTTP. These are the unit tests that ship in +`samples/lumen/src/domain.rs`. Add a `#[cfg(test)] mod tests` block: ```rust,ignore -#[test] -fn open_validates_owner_and_balance() { - assert_eq!(Wallet::open("w1", " ", Money::cents(100)).unwrap_err(), DomainError::OwnerRequired); - assert_eq!(Wallet::open("w1", "alice", Money::cents(-1)).unwrap_err(), DomainError::NonPositiveAmount); - let w = Wallet::open("w1", "alice", Money::ZERO).unwrap(); - assert!(w.opened); - assert_eq!(w.balance, Money::ZERO); -} +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn open_validates_owner_and_balance() { + assert_eq!( + Wallet::open("w1", " ", Money::cents(100)).unwrap_err(), + DomainError::OwnerRequired + ); + assert_eq!( + Wallet::open("w1", "alice", Money::cents(-1)).unwrap_err(), + DomainError::NonPositiveAmount + ); + let w = Wallet::open("w1", "alice", Money::ZERO).unwrap(); + assert!(w.opened); + assert_eq!(w.balance, Money::ZERO); + } -#[test] -fn withdraw_rejects_overdraft() { - let mut w = Wallet::open("w1", "alice", Money::cents(100)).unwrap(); - assert_eq!(w.withdraw(Money::cents(101)).unwrap_err(), DomainError::InsufficientFunds); - // The failed command raised no event beyond the open. - assert_eq!(w.root.uncommitted().len(), 1); -} + #[test] + fn withdraw_rejects_overdraft() { + let mut w = Wallet::open("w1", "alice", Money::cents(100)).unwrap(); + assert_eq!( + w.withdraw(Money::cents(101)).unwrap_err(), + DomainError::InsufficientFunds + ); + // The failed command raised no event beyond the open. + assert_eq!(w.root.uncommitted().len(), 1); + } + + #[test] + fn deposit_and_withdraw_update_balance_and_raise_events() { + let mut w = Wallet::open("w1", "alice", Money::cents(100)).unwrap(); + w.deposit(Money::cents(50)).unwrap(); + assert_eq!(w.balance, Money::cents(150)); + w.withdraw(Money::cents(30)).unwrap(); + assert_eq!(w.balance, Money::cents(120)); + } -#[test] -fn deposit_and_withdraw_update_balance_and_raise_events() { - let mut w = Wallet::open("w1", "alice", Money::cents(100)).unwrap(); - w.deposit(Money::cents(50)).unwrap(); - assert_eq!(w.balance, Money::cents(150)); - w.withdraw(Money::cents(30)).unwrap(); - assert_eq!(w.balance, Money::cents(120)); + #[test] + fn wallet_view_wire_shape() { + let w = Wallet::open("wlt_1", "alice", Money::cents(250)).unwrap(); + let json = serde_json::to_string(&w.view()).unwrap(); + assert_eq!( + json, + r#"{"id":"wlt_1","owner":"alice","balance":250,"version":1}"# + ); + } } ``` -The `withdraw_rejects_overdraft` test makes the consistency boundary concrete: -after a rejected withdrawal, the aggregate's uncommitted-event buffer still holds -exactly one event — the `WalletOpened` from the factory. The overdraft never -produced a `MoneyWithdrawn`, so nothing partial can ever be persisted. That is -the difference between a service-level guard (a convention) and an aggregate -invariant (a physical constraint): the model exposes no mechanism to violate it. - -## What changed in Lumen +What just happened: + +- **`open_validates_owner_and_balance`** proves the birth invariants: a blank + owner is `OwnerRequired`, a negative opening balance is `NonPositiveAmount`, + and a *zero* opening balance succeeds (the wallet is `opened` with a `ZERO` + balance). +- **`withdraw_rejects_overdraft`** is the load-bearing test. After a rejected + withdrawal, the aggregate's uncommitted-event buffer (`w.root.uncommitted()`) + still holds *exactly one* event — the `WalletOpened` from the factory. The + overdraft never produced a `MoneyWithdrawn`, so nothing partial can ever be + persisted. This is the consistency boundary made concrete. +- **`deposit_and_withdraw_update_balance_and_raise_events`** walks the happy + path: a deposit lifts the balance, a withdrawal lowers it, and the arithmetic + is exact (`100 + 50 - 30 = 120`). +- **`wallet_view_wire_shape`** pins the wire contract. A freshly opened wallet's + `view()` serializes to exactly + `{"id":"wlt_1","owner":"alice","balance":250,"version":1}` — confirming + `Money`'s `#[serde(transparent)]` (the balance is the bare number `250`, not + `{ "cents": 250 }`) and `WalletView`'s field order. + +> **Tip** **Checkpoint.** Run `cargo test --lib`. All the domain tests pass — and +> they ran with no database, no HTTP server, and no framework runtime. That is +> the payoff of a domain core that is just structs and methods: the rules are +> testable in microseconds. + +## Recap — Lumen's domain core Lumen now has a domain core that owns its rules: -- **`src/money.rs`** — a `Money` value object: immutable, integer cents, - `#[serde(transparent)]` so it rides the wire as a bare number, closed under - `add` / `subtract` / `require_positive`, with a hand-written `MoneyError` - (no `thiserror`). +- **`src/money.rs`** — a `Money` value object: immutable, integer cents, private + field, `#[serde(transparent)]` so it rides the wire as a bare number, closed + under `add` / `subtract` / `require_positive`, with a hand-written + `MoneyError` (no `thiserror`) and a `Display` that prints `"12.50"`. - **`src/domain.rs`** — the `Wallet` aggregate carrying `#[derive(AggregateRoot)]` (which generates `AGGREGATE_TYPE` and the - `aggregate()` accessors). `open` / `deposit` / `withdraw` enforce the three - invariants — owner required, amounts positive, no overdraft — *before* raising - an event, so a rejected command leaves the event buffer untouched. -- **The typed `DomainError`** family with stable `Display` strings that surface + `aggregate()` / `aggregate_mut()` accessors over the embedded root). `open` / + `deposit` / `withdraw` enforce the three invariants — owner required, amounts + positive, no overdraft — *before* raising an event, so a rejected command + leaves the event buffer untouched. +- **Three `#[derive(DomainEvent)]` payloads** (`WalletOpened`, + `MoneyDeposited`, `MoneyWithdrawn`), each stamped with a stable `EVENT_TYPE` + discriminator, raised through the single private `raise` helper. +- **The typed `DomainError` family** with stable `Display` strings that surface as RFC 9457 problem details, plus the `From` bridge that turns an arithmetic overdraw into `InsufficientFunds`. - **`WalletView`** — the flat read-model projection the aggregate hands out via - `view()`, kept a separate type from the rule-enforcing aggregate. - -The event payloads (`WalletOpened` / `MoneyDeposited` / `MoneyWithdrawn`) and -the `rehydrate` / `apply` fold that reconstruct a wallet from its stream get -their full treatment in [Event Sourcing](./11-event-sourcing.md). Here, the -shape that matters is the DDD one: a value object you cannot corrupt and an -aggregate you cannot bypass. + `view()`, deriving `Schema` for the docs, kept a separate type from the + rule-enforcing aggregate. + +You also now know: + +- The difference between a **value object** (no identity, immutable, compared by + value — `Money`) and an **aggregate** (an identity and a consistency boundary — + `Wallet`), and why each rule belongs where it does. +- That an aggregate makes its invariants *physically* unreachable by validating + before it mutates and exposing no setters — the difference between a convention + and a constraint. +- That `#[derive(AggregateRoot)]` and `#[derive(DomainEvent)]` supply the event + buffer and the type discriminators, so the only event-sourcing code you write + by hand is the rules. + +The event payloads' full lifecycle — how they are persisted, and the +`rehydrate` / `apply` fold that reconstructs a wallet from its stream — gets its +treatment in [Event Sourcing](./11-event-sourcing.md). Here, the shape that +matters is the DDD one: a value object you cannot corrupt and an aggregate you +cannot bypass. ## Exercises @@ -455,9 +840,9 @@ aggregate you cannot bypass. wallet with `Money::cents(100)` and call `withdraw(Money::cents(200))`. Assert the error is `DomainError::InsufficientFunds` *and* that `w.root.uncommitted().len()` is still `1` — proving the rejected command - raised no event. + raised no event beyond the open. -2. **Add a `transfer_within` rule (domain only).** Write a free function +2. **Add a `transfer` rule (domain only).** Write a free function `fn transfer(from: &mut Wallet, to: &mut Wallet, amount: Money) -> Result<(), DomainError>` that calls `from.withdraw(amount)?` then `to.deposit(amount)?`. Test that a transfer exceeding the source balance @@ -465,7 +850,7 @@ aggregate you cannot bypass. real, persisted version becomes the saga in [Sagas](./12-sagas.md).) 3. **Confirm the wire shape.** Serialize a freshly opened wallet's `view()` with - `serde_json` and assert it equals + `serde_json::to_string` and assert it equals `{"id":"wlt_1","owner":"alice","balance":250,"version":1}` — verifying that `Money`'s `#[serde(transparent)]` and `WalletView`'s field order produce the contract the read model and clients share. @@ -475,5 +860,19 @@ aggregate you cannot bypass. of deriving with `thiserror`, and what it would cost the book's one-dependency promise to add the crate. -Next, separate the write path from the read path with the command/query bus. See -[CQRS](./09-cqrs.md). +5. **Allow a zero deposit — then decide against it.** Change `deposit` to accept + a zero amount and run the suite; note which test breaks and why + `require_positive` rejecting zero is the right rule for a wallet command. + Revert the change. + +## Where to go next + +- Separate the write path from the read path with the command/query bus in + **[CQRS](./09-cqrs.md)** — where `Wallet`'s commands become bus-dispatched + handlers and `DomainError` becomes the RFC 9457 problem mapping. +- Expose these rules over HTTP in + **[Your First HTTP API](./06-first-http-api.md)**, where a returned + `DomainError` renders as a 422 or 404 problem document. +- Persist the events and reconstruct a wallet from its stream in + **[Event Sourcing](./11-event-sourcing.md)** — the full lifecycle of the + payloads you raised here. diff --git a/docs/book/src/09-cqrs.md b/docs/book/src/09-cqrs.md index bb5140b..fbc6bad 100644 --- a/docs/book/src/09-cqrs.md +++ b/docs/book/src/09-cqrs.md @@ -1,68 +1,136 @@ # CQRS -Lumen's `Wallet` aggregate enforces its own rules, and the read model has a -home. But the controller still needs a way to *deliver* an instruction to the -write side and a *question* to the read side — and to do it without the two -paths sharing a code path, so reads can be cached and writes can be validated -independently. - -**CQRS** — Command Query Responsibility Segregation — draws that bright line. -Writes become **commands** (`OpenWallet`, `Deposit`, `Withdraw`); reads become a -**query** (`GetWallet`). Each travels a typed `Bus`, matched to its handler by -`std::any::TypeId`, through a middleware chain that validates commands and caches -queries. This chapter wires Lumen's bus end to end, exactly as `samples/lumen` -does. - -> **By the end of this chapter, Lumen will** have `src/commands.rs`: the -> `OpenWallet` / `Deposit` / `Withdraw` commands and the `GetWallet` query as -> `#[derive(Command)]` / `#[derive(Query)]` structs, and a **handler bean** — -> `WalletHandlers`, a `#[derive(Service)]` whose `#[handlers]` impl carries the -> `#[command_handler]` / `#[query_handler]` methods and `#[autowired]`s the -> `Ledger` + `ReadModel` — that the framework resolves from the container and -> drains onto a bus declared as a `#[bean]`, the validation + query-cache -> middleware auto-installed, and the read-after-write cache invalidation that -> keeps a balance from going stale after a deposit. - -> **Design note.** The `Bus` is Firefly's command/query dispatcher: it matches -> each message to its handler by `TypeId` and runs it through a middleware chain -> that validates commands and caches queries. A handler is an `async fn` a macro -> registers — `bus.send` / `bus.query` dispatch to it. Lumen's handlers live on a -> **DI bean** (`#[derive(Service)]` + `#[handlers]`), so each reaches its -> collaborators through `self.` — Spring's `@Component` -> command/query handler. (A simpler app can write the handler as a free -> `async fn` instead; the [free-fn alternative](#the-free-fn-handler-alternative) -> below covers that form.) The whole path is ordinary Rust: no proxies, no -> reflection, just a typed registry and a method call. - -## Commands, queries, and the `Message` trait - -Every command and query implements `Message`. Hand-writing it is one line, but -the trait's optional methods — `validate`, `cache_ttl` — are overridable -defaults that the matching middleware picks up automatically: +In [Domain-Driven Design](./08-domain-driven-design.md) Lumen's `Wallet` +aggregate learned to enforce its own rules, and the read model found a home. But +a controller still needs a way to *deliver* an instruction to the write side and +ask a *question* of the read side — and to do it without the two paths sharing a +code path, so reads can be cached and writes validated independently. + +This chapter draws that bright line. It wires Lumen's command/query bus end to +end, exactly as the shipped [`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen) +crate does: four message structs, one handler bean, and the controller seam that +dispatches through the bus and keeps the read cache honest after a write. + +By the end of this chapter you will: + +- Explain what **Command/Query Responsibility Segregation** buys you, and how + Firefly keeps a single typed bus while still reporting commands and queries + apart. +- Define Lumen's `OpenWallet` / `Deposit` / `Withdraw` commands and its + `GetWallet` query as `#[derive(Command)]` / `#[derive(Query)]` structs, with + field-level validation and a query cache TTL generated for you. +- Write the `WalletHandlers` **handler bean** — a `#[derive(Service)]` whose + `#[handlers]` impl carries `#[command_handler]` / `#[query_handler]` methods + that reach their collaborators through `#[autowired]` fields. +- Understand how `FireflyApplication` drains those handlers onto a + framework-provided `Bus` and installs the correlation, query-cache, and + validation middleware — with no wiring code in Lumen. +- Dispatch from the controller with `bus.send` / `bus.query`, map a `CqrsError` + to the right RFC 9457 status, and enforce read-after-write consistency by + invalidating the cached query family after every mutation. + +## Concepts you will meet + +Before the first message, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — Command/Query Responsibility Segregation (CQRS).** A +> pattern that routes state-changing **commands** and read-only **queries** +> through separate handlers, so the two halves can evolve, scale, and be +> optimised independently — reads cached, writes validated. The Spring analog is +> a CQRS application split into `@CommandHandler` / `@QueryHandler` components +> (e.g. as Axon Framework names them). + +> **Note** **Key term — message.** A *message* is the typed value you hand the +> bus: a command (it mutates) or a query (it reads). Every Lumen message is a +> plain serializable struct. In Spring/Axon terms a message is the command or +> query DTO you `send` or `query` through a gateway. + +> **Note** **Key term — bus.** The *bus* is Firefly's command/query dispatcher. +> It matches each message to exactly one handler by `std::any::TypeId`, runs it +> through a middleware chain, and returns the handler's result. The Spring/Axon +> analog is the `CommandGateway` / `QueryGateway`, except here it is one +> in-process `Arc` the framework provides. + +> **Note** **Key term — handler bean.** A *handler bean* is an ordinary DI bean +> whose methods serve commands and queries. Its collaborators arrive by +> constructor injection, and the framework registers each method on the bus at +> boot. This is Spring's `@Component` that carries `@CommandHandler` / +> `@QueryHandler` methods. + +> **Note** **Key term — middleware.** A *middleware* wraps every dispatch with +> cross-cutting behaviour — validation, caching, correlation — before and after +> the handler runs. The Spring analog is a `HandlerInterceptor` or an Axon +> `MessageHandlerInterceptor`. Firefly installs a small default chain for you. + +> **Design note.** The whole path is ordinary Rust: no proxies, no reflection, +> just a typed registry keyed by `TypeId` and a method call. Lumen's handlers +> live on a DI bean (`#[derive(Service)]` + `#[handlers]`), so each reaches its +> collaborators through `self.`. A simpler app can write a +> handler as a free `async fn` instead — the [free-fn +> alternative](#step-4--know-the-free-fn-handler-alternative) below covers that +> form. + +## Step 1 — Understand the `Message` trait + +**Action.** Before writing any messages, look at the contract every command and +query satisfies. Every message implements `Message`. You will never hand-write +this impl — the derives generate it — but knowing its shape explains what the +middleware reacts to: ```rust,ignore pub trait Message: Clone + Serialize + Send + Sync + 'static { + fn kind() -> MessageKind { MessageKind::Command } // Command / Query split fn validate(&self) -> Result<(), CqrsError> { Ok(()) } // ValidationMiddleware fn cache_ttl(&self) -> Option { None } // QueryCache } ``` -`Clone` stands in for pass-by-value handler invocation; `Serialize` seeds the -query cache key. Lumen never writes that impl by hand — it derives it. - -## Lumen's commands and query - -The four messages are plain structs carrying `#[derive(Command)]` / -`#[derive(Query)]`, which generate the `Message` impl. The `#[firefly(validate)]` -field attribute makes a field required (the generated `validate()` rejects an -empty `String` or a non-positive number), and `#[firefly(cache_ttl = "...")]` is -reflected on the query's generated `cache_ttl`: +**What just happened.** The trait's *supertraits* state what a message must be, +and its *methods* are overridable defaults the matching middleware picks up +automatically: + +- `Clone` stands in for pass-by-value handler invocation, and `Serialize` seeds + the query-cache key (the cache hashes the message's JSON). +- `kind()` reports whether the message is a command or a query. The default is + `MessageKind::Command`; `#[derive(Query)]` overrides it. +- `validate()` is the pre-dispatch validation hook the `ValidationMiddleware` + calls. The default accepts everything, so a plain message passes untouched. +- `cache_ttl()` is the cache opt-in the `QueryCache` middleware reads. The + default `None` means "not cacheable", so commands fall straight through the + cache. + +> **Note** **Key term — `MessageKind`.** A two-variant enum, +> `MessageKind::Command` / `MessageKind::Query`, that records the write/read +> nature of a message type. The bus stores each handler's kind at registration +> so it can list commands and queries separately — that is the segregation in +> "Command/Query Responsibility Segregation". + +> **Tip** **Checkpoint.** You should be able to say, in one breath, what each of +> the three methods is for: `kind()` splits command from query, `validate()` +> gates dispatch, `cache_ttl()` opts a query into the cache. The rest of the +> chapter is mostly making the derives fill these in for you. + +## Step 2 — Define Lumen's commands and query + +**Action.** Create `src/commands.rs`. The four messages are plain structs +carrying `#[derive(Command)]` / `#[derive(Query)]`, which generate the `Message` +impl. The `#[firefly(validate)]` field attribute makes a field required (the +generated `validate()` rejects an empty `String` or a non-positive number), and +`#[firefly(cache_ttl = "...")]` is reflected on the query's generated +`cache_ttl`: ```rust,ignore // samples/lumen/src/commands.rs +use std::sync::Arc; + use firefly::prelude::*; use serde::{Deserialize, Serialize}; +use crate::domain::{DomainError, Wallet, WalletView}; +use crate::ledger::{Ledger, ReadModel}; +use crate::money::Money; + /// `POST /api/v1/wallets` command — open a new wallet. #[derive(Debug, Clone, Default, Serialize, Deserialize, Command, Builder, Schema)] #[serde(default)] @@ -110,24 +178,49 @@ pub struct GetWallet { } ``` +**What just happened.** Three derives are doing the heavy lifting: + +- `#[derive(Command)]` / `#[derive(Query)]` generate each struct's `Message` + impl. `Command` keeps the default `kind()` of `MessageKind::Command`; `Query` + overrides it to `MessageKind::Query`. That single difference is the whole CQRS + split — `OpenWallet` / `Deposit` / `Withdraw` register as commands and + `GetWallet` registers as a query, with no extra annotation. +- `#[firefly(validate)]` on a field makes it required: the generated `validate()` + rejects an empty `String` or a non-positive number *at compile-time-generated + code*, not by runtime reflection. On `Deposit::amount` it rejects a zero or + negative amount before the handler ever runs, so the aggregate is never even + called with structurally wrong data. +- `#[firefly(cache_ttl = "30s")]` on `GetWallet` is reflected on the generated + `cache_ttl()`, which the `QueryCache` middleware reads off the message to + memoise reads for 30 seconds. + A few choices echo the domain chapter. The commands carry `i64` cents, not a -`Money` — the handler constructs the value object, keeping the wire contract a -bare number and the validation simple. `#[firefly(validate)]` on `amount` -rejects a zero or negative amount *before* the handler runs, so the aggregate is -never even called with structurally wrong data. And `#[serde(rename = ...)]` -keeps the JSON camelCase (`openingBalance`, `walletId`) while the Rust fields -stay snake_case. - -> **Note.** `#[firefly(validate)]` makes a field required — the generated -> `validate()` rejects an empty `String` or a non-positive number before the -> handler runs — and the check is generated by the derive macro at compile time, -> not reflected at runtime. `#[firefly(cache_ttl = "30s")]` sets the query's -> cache TTL, which the `QueryCache` middleware picks up off the message. - -## The handler bean — `#[derive(Service)]` + `#[handlers]` - -Lumen's handlers live on a **DI bean**, the Rust analog of a Spring `@Component` -that carries `@CommandHandler` / `@QueryHandler` methods. `WalletHandlers` is a +`Money` value object — the handler constructs `Money`, keeping the wire contract +a bare number and the validation simple. And `#[serde(rename = ...)]` keeps the +JSON camelCase (`openingBalance`, `walletId`) while the Rust fields stay +snake_case. + +> **Note** `OpenWallet` also derives `Builder` (Lombok's `@Builder`) and +> `Schema` (it feeds the OpenAPI docs). `Builder` gives it a fluent constructor — +> `OpenWallet::builder().owner("ada").build()` — with `opening_balance` +> defaulting to zero. Neither derive affects the CQRS behaviour; they are along +> for the ride because `OpenWallet` is also a request body. + +> **Tip** **Checkpoint.** `cargo build` compiles `src/commands.rs`. The validation +> and cache behaviour is testable without a bus, because the derives put the +> methods on the type itself: +> +> ```rust,ignore +> assert!(OpenWallet::default().validate().is_err()); // empty owner rejected +> assert!(Deposit { wallet_id: "wlt_1".into(), amount: 0 }.validate().is_err()); +> assert!(GetWallet::default().cache_ttl().is_some()); // the 30s TTL +> ``` + +## Step 3 — Write the handler bean + +**Action.** Add the handler bean to `src/commands.rs`. Lumen's handlers live on a +**DI bean**, the Rust analog of a Spring `@Component` that carries +`@CommandHandler` / `@QueryHandler` methods. `WalletHandlers` is a `#[derive(Service)]` whose collaborators — the write-side `Ledger` and the read-side `ReadModel` — are `#[autowired]` from the container. The `#[handlers]` impl-level macro (the CQRS sibling of `#[rest_controller]`) marks the methods: @@ -136,13 +229,13 @@ Result<.., CqrsError>`, so a handler reaches its collaborators through `self` no process-global, no composition root: ```rust,ignore -use std::sync::Arc; - -use firefly::prelude::*; +// samples/lumen/src/commands.rs (continued) -use crate::domain::{DomainError, Wallet, WalletView}; -use crate::ledger::{Ledger, ReadModel}; -use crate::money::Money; +/// Maps a `DomainError` onto the bus's `CqrsError` channel. The web layer +/// restores the precise HTTP status from the detail message. +fn to_cqrs(e: DomainError) -> CqrsError { + CqrsError::handler(e.to_string()) +} /// The CQRS **handler bean** — Spring's `@Component` command/query handler. Its /// collaborators are `#[autowired]` from the DI container; `#[handlers]` @@ -202,49 +295,60 @@ impl WalletHandlers { } ``` -Each command handler constructs the `Money` value object from the command's -`i64`, delegates to the autowired `Ledger` application service (which rehydrates -the aggregate, runs the domain command, and persists — see +**What just happened.** Each command handler constructs the `Money` value object +from the command's `i64`, delegates to the autowired `Ledger` application service +(which rehydrates the aggregate, runs the domain command, and persists — see [Event Sourcing](./11-event-sourcing.md)), and maps a `DomainError` onto the -bus's `CqrsError` channel: - -```rust,ignore -fn to_cqrs(e: DomainError) -> CqrsError { - CqrsError::handler(e.to_string()) -} -``` +bus's `CqrsError` channel via `to_cqrs`. The `get_wallet` query is the read-after-write pattern in miniature: it serves from the projected `ReadModel` first, and *only* if the projection has not yet -caught up does it fall back to folding the event stream. That fallback is what -keeps a read immediately after a write from returning a stale balance under the -eventual consistency the projection introduces. - -Behind the macro, each `#[command_handler]` / `#[query_handler]` submits a -`BeanHandlerRegistration` into a compile-time `inventory` registry. At boot -`FireflyApplication` resolves `WalletHandlers` from the container — wiring its -`#[autowired]` `Ledger` + `ReadModel` — and installs a bus closure that captures -the resolved bean, so each dispatch calls `self.open_wallet(..)` and friends. -Lumen writes **no** registration call: the framework drains the bean handlers -for you (the wiring section, below). - -> **Note.** A `#[handlers]` method takes `&self` plus exactly one message -> argument and returns a `Result<.., CqrsError>`. Because the bean is a regular -> container bean, its collaborators arrive by **constructor injection** through +caught up does it fall back to folding the event stream +(`Wallet::rehydrate(..).view()`). That fallback is what keeps a read immediately +after a write from returning a stale balance under the eventual consistency the +projection introduces. + +> **Note** A `#[handlers]` method takes `&self` plus exactly one message argument +> and returns a `Result<.., CqrsError>`. Because the bean is a regular container +> bean, its collaborators arrive by **constructor injection** through > `#[autowired]` fields — the same wiring every other Firefly bean uses, with no > process-global to seed. Adding a handler is adding a method; the framework > finds it. -## The free-`fn` handler alternative - -A handler need not be a bean. The free-`fn` form is the simpler option for a -collaborator-free handler (and the `macro-quickstart` sample uses it): mark a -free `async fn(msg) -> Result` with `#[command_handler]` / -`#[query_handler]`. The macro reads the argument type as the dispatch key, -generates a `register_(bus)` helper, **and** submits a `HandlerRegistration` -into the `inventory` registry the framework drains -(`register_discovered_handlers`) — so the free-fn handler is discovered and -installed exactly like the bean form: +**Why it matters.** Behind the macro, each `#[command_handler]` / +`#[query_handler]` submits a `BeanHandlerRegistration` into a compile-time +`inventory` registry. At boot `FireflyApplication` resolves `WalletHandlers` from +the container — wiring its `#[autowired]` `Ledger` + `ReadModel` — and installs a +bus closure that captures the resolved bean, so each dispatch calls +`self.open_wallet(..)` and friends. Lumen writes **no** registration call: the +framework drains the bean handlers for you (Step 5). + +> **Tip** **Checkpoint.** `cargo build` still compiles. You can exercise the bean +> directly with no HTTP and no bus, constructing it with the same collaborators +> the container would inject: +> +> ```rust,ignore +> let handlers = WalletHandlers { +> ledger: Arc::new(Ledger::new( +> Arc::new(MemoryEventStore::new()), +> Arc::new(InMemoryBroker::new()), +> )), +> read_model: Arc::new(ReadModel::default()), +> }; +> let opened = handlers +> .open_wallet(OpenWallet { owner: "alice".into(), opening_balance: 100 }) +> .await +> .unwrap(); +> assert_eq!(opened.balance, 100); +> ``` + +## Step 4 — Know the free-`fn` handler alternative + +**Action.** Nothing to write for Lumen here — but it is worth knowing the second +form, because a simpler app reaches for it. A handler need not be a bean. The +free-`fn` form is the natural option for a *collaborator-free* handler (the +framework's `macro-quickstart` sample uses it): mark a free `async fn(msg) -> +Result` with `#[command_handler]` / `#[query_handler]`: ```rust,ignore // The simpler form — a free fn with no collaborators to inject. @@ -254,43 +358,26 @@ pub async fn place_order(cmd: PlaceOrder) -> Result { } ``` -Because a free function can't own a `Ledger` or a `ReadModel`, this form fits -handlers that compute purely from the message (or reach a process-global). The -moment a handler needs injected collaborators — as all of Lumen's do — the bean -form above is the natural fit: it gets constructor injection for free and keeps -the handler a plain method on a `@Component`. - -## Wiring the bus - -Lumen writes **no** bus-wiring code. The `Bus` and the `QueryCache` are declared -as `#[bean]`s in `LumenBeans` (the `#[derive(Configuration)]` holder in -`src/web.rs`), the `WalletApi` controller autowires the `Arc`, and the -framework — `FireflyApplication` — does the rest at boot: - -- it drains the discovered **bean** handlers with - `firefly::cqrs::register_discovered_handler_beans(&bus, &container)`: it - resolves `WalletHandlers` from the container — autowiring its `Ledger` + - `ReadModel` — and installs each `#[command_handler]` / `#[query_handler]` - method onto the bus; -- it also drains any free-`fn` handlers with - `firefly::cqrs::register_discovered_handlers(&bus)`, so the two forms coexist - (Lumen has only bean handlers, so this drains none of its own); -- it auto-installs the bus middleware chain: a correlation propagator always, the - `QueryCache` read-cache middleware whenever a `QueryCache` bean is present, and - validation (already installed by the core). +**What just happened.** The macro reads the argument type (`PlaceOrder`) as the +dispatch key, generates a `register_place_order(bus)` helper, **and** submits a +`HandlerRegistration` into the `inventory` registry the framework drains — so the +free-fn handler is discovered and installed exactly like the bean form. -Lumen calls none of these drains. The bean handlers are resolved from the same -container that builds the controller and the saga, so every collaborator — -handler, controller, projection — shares the one `Ledger` and one `ReadModel` the -container holds: +**Why it matters.** Because a free function can't own a `Ledger` or a +`ReadModel`, this form fits handlers that compute purely from the message (or +reach a process-global). The moment a handler needs injected collaborators — as +*all* of Lumen's do — the bean form from Step 3 is the natural fit: it gets +constructor injection for free and keeps the handler a plain method on a +`@Component`. Lumen has only bean handlers; the free-fn path drains none of its +own. -```rust,ignore -// What FireflyApplication does for you, conceptually — no Lumen code calls this. -firefly::cqrs::register_discovered_handlers(&bus); // free-fn handlers -firefly::cqrs::register_discovered_handler_beans(&bus, &container); // WalletHandlers' 4 methods -``` +## Step 5 — Let the framework wire the bus -The bus and query cache are plain `#[bean]` factories: +**Action.** Again, no wiring code to write — that is the point. The `Bus` and the +`QueryCache` are declared as `#[bean]`s in `LumenBeans` (the +`#[derive(Configuration)]` holder in `src/web.rs`), the `WalletApi` controller +autowires the `Arc`, and `FireflyApplication` does the rest at boot. The +query cache is a plain `#[bean]` factory: ```rust,ignore // samples/lumen/src/web.rs — LumenBeans (#[derive(Configuration)]). @@ -303,28 +390,57 @@ impl LumenBeans { } // ... event_store, jwt_service, ledger, security beans ... } -// The read store is *not* a `#[bean]` here — `ReadModel` carries -// `#[derive(Repository)]` on its own struct, so the scan registers it directly. +// The read store is *not* a `#[bean]` here — `ReadModel` is its own bean, +// registered by the scan directly. +``` + +**What just happened.** At boot, `FireflyApplication`: + +- **Drains the discovered bean handlers** with + `firefly::cqrs::register_discovered_handler_beans(&bus, &container)`: it + resolves `WalletHandlers` from the container — autowiring its `Ledger` + + `ReadModel` — and installs each `#[command_handler]` / `#[query_handler]` + method onto the bus. +- **Drains any free-`fn` handlers** with + `firefly::cqrs::register_discovered_handlers(&bus)`, so the two forms coexist + (Lumen has only bean handlers, so this drains none of its own). +- **Auto-installs the bus middleware chain**: validation (installed first by the + core), then a correlation propagator, then the `QueryCache` read-cache + middleware whenever a `QueryCache` bean is present. + +Lumen calls none of these drains. Conceptually, the framework runs: + +```rust,ignore +// What FireflyApplication does for you — no Lumen code calls this. +firefly::cqrs::register_discovered_handlers(&bus); // free-fn handlers +firefly::cqrs::register_discovered_handler_beans(&bus, &container); // WalletHandlers' 4 methods ``` -> **Where does the `Bus` come from?** It is a framework-provided infrastructure -> bean: the core registers an `Arc` into the container before the scan, so the -> `WalletApi` controller can autowire it (`#[autowired] pub bus: Arc`) and the -> framework can drain the discovered handlers onto it. You declare the -> *application* beans (`QueryCache`, the ledger); the bus is wired in for you. +> **Note** **Where does the `Bus` come from?** It is a framework-provided +> infrastructure bean: the core registers an `Arc` into the container before +> the scan, so the `WalletApi` controller can autowire it (`#[autowired] pub bus: +> Arc`) and the framework can drain the discovered handlers onto it. You +> declare the *application* beans (`QueryCache`, the ledger); the bus is wired in +> for you. -Middleware runs first-registered = outermost. Two app-visible entries ship in this -chain — and the framework installs them automatically (a third, authorization, -arrives at the HTTP edge with [Security](./14-security.md)): +**Why it matters.** The bean handlers are resolved from the *same* container that +builds the controller and the saga, so every collaborator — handler, controller, +projection — shares the one `Ledger` and one `ReadModel` the container holds. +There is no second copy of the read model to drift out of sync. + +Three middleware entries ship in the dispatch chain. The framework installs them +automatically (a fourth, authorization, arrives at the HTTP edge with +[Security](./14-security.md)). Middleware runs first-registered = outermost: | Middleware | Behaviour | |-----------------------------|---------------------------------------------------------------| +| `ValidationMiddleware` | calls `Message::validate` before dispatch, short-circuits on error — installed first by the core, so it is outermost | +| `CorrelationMiddleware` | ensures-or-generates the correlation id for the dispatch (next step) | | `QueryCache::middleware()` | memoises results for messages whose `cache_ttl` is `Some` — installed when a `QueryCache` bean exists | -| `ValidationMiddleware` | calls `Message::validate` before dispatch, short-circuits on error |
send / query a message @@ -341,17 +457,17 @@ arrives at the HTTP edge with [Security](./14-security.md)): Q + font-family="SF Mono,JetBrains Mono,Menlo,Consolas,monospace">V V + font-family="SF Mono,JetBrains Mono,Menlo,Consolas,monospace">Q - Q = QueryCache - V = ValidationMiddleware + V = ValidationMiddleware + Q = QueryCache @@ -359,26 +475,33 @@ arrives at the HTTP edge with [Security](./14-security.md)): your handler -
A message is matched to its handler by TypeId, then runs the registered middleware chain (here QueryCache then ValidationMiddleware) before the handler executes.
+
A message is matched to its handler by TypeId, then runs the registered middleware chain (validation outermost, then the query cache) before the handler executes.
-## Command/query segregation +> **Tip** **Checkpoint.** `cargo run` boots Lumen and the startup report's CQRS +> line counts your handlers — three commands and one query. The admin `/cqrs` +> view on the management port (`:8081`) lists them, badged blue for commands and +> green for queries. + +## Step 6 — See how the bus segregates commands and queries -The bus dispatches commands and queries through one registry keyed by `TypeId`, -but it does not treat them as interchangeable: each registered handler carries -the **kind** of the message it serves. That kind is a property of the message -type, exposed as `Message::kind() -> MessageKind`: +**Action.** Look at how the bus keeps the two halves apart, even though they share +one registry. The bus dispatches commands and queries through one registry keyed +by `TypeId`, but it does not treat them as interchangeable: each registered +handler carries the **kind** of the message it serves, exposed as `Message::kind() +-> MessageKind`: ```rust,ignore pub enum MessageKind { Command, Query } ``` -The default is `MessageKind::Command`. `#[derive(Command)]` keeps that default; -`#[derive(Query)]` overrides `kind()` to return `MessageKind::Query`. Nothing in -Lumen's `src/commands.rs` changes — `OpenWallet` / `Deposit` / `Withdraw` are -already commands and `GetWallet` is already a query, so the segregation falls out -of the derives the chapter introduced. The bus records each message's kind at -registration time and lets you ask about the two halves separately: +**What just happened.** The default is `MessageKind::Command`. +`#[derive(Command)]` keeps that default; `#[derive(Query)]` overrides `kind()` to +return `MessageKind::Query`. Nothing in Lumen's `src/commands.rs` changes — +`OpenWallet` / `Deposit` / `Withdraw` are already commands and `GetWallet` is +already a query, so the segregation falls out of the derives Step 2 introduced. +The bus records each message's kind at registration time and lets you ask about +the two halves separately: ```rust,ignore use firefly::cqrs::{Bus, MessageKind}; @@ -411,61 +534,68 @@ filtered to one kind. `handler_count()` is the total registry size; removes a handler, returning whether one was present (useful when a test wants to swap a handler without rebuilding the bus). -This is exactly what the admin `/cqrs` view consumes: because the bus now knows -each handler's kind, the dashboard tags every registration with a badge (commands -blue, queries green) and shows separate command/query counts, rather than one -undifferentiated handler list. +**Why it matters.** This is exactly what the admin `/cqrs` view consumes: because +the bus knows each handler's kind, the dashboard tags every registration with a +badge (commands blue, queries green) and shows separate command/query counts, +rather than one undifferentiated handler list. -> **Note.** Firefly keeps a single `Bus` and recovers the command/query split -> from each message's `kind()` (set by the `Command` / `Query` derive), rather -> than from two distinct buses. `command_handler_names()` / -> `query_handler_names()` are the filtered views the admin `/cqrs` dashboard -> renders; `has_handler::()` / `unregister::()` test membership and remove -> a handler by type. +> **Note** Firefly keeps a single `Bus` and recovers the command/query split from +> each message's `kind()` (set by the `Command` / `Query` derive), rather than +> from two distinct buses. `command_handler_names()` / `query_handler_names()` +> are the filtered views the admin `/cqrs` dashboard renders; `has_handler::()` +> / `unregister::()` test membership and remove a handler by type. -## Correlation propagation +## Step 7 — Follow the correlation id across the dispatch boundary -A command rarely acts alone. `bus.send(Deposit { .. })` runs a handler that may +**Action.** Understand the middleware that keeps one logical request traceable. A +command rarely acts alone. `bus.send(Deposit { .. })` runs a handler that may start the transfer saga ([Sagas](./12-sagas.md)) or `tokio::spawn` a follow-up task — and each of those leaves the original request task. For the logs and traces to read as *one* operation, they must all share a single correlation id. -`firefly::cqrs::CorrelationMiddleware` enforces that at the dispatch boundary. -The framework installs it on every `FireflyApplication` bus as the outermost -middleware, before the query-cache and validation layers, so you never wire it by -hand. If you build a bus yourself, add it like any other middleware: +> **Note** **Key term — correlation id.** A single identifier stamped on +> everything done for one logical request, so its logs and traces can be stitched +> together. Firefly threads it through a task-local; the web layer sets one per +> HTTP request. The Spring analog is the MDC `traceId` propagated by Sleuth / +> Micrometer Tracing. + +`firefly::cqrs::CorrelationMiddleware` enforces that at the dispatch boundary. The +framework installs it on every `FireflyApplication` bus, between the validation +and query-cache layers, so you never wire it by hand. If you build a bus +yourself, add it like any other middleware: ```rust,ignore use firefly::cqrs::{Bus, CorrelationMiddleware}; let bus = Bus::new(); -bus.use_middleware(CorrelationMiddleware::new()); // outermost — runs first +bus.use_middleware(CorrelationMiddleware::new()); // earlier-registered = more outer ``` -On each dispatch the middleware **ensures-or-generates** a correlation id: if the -request is already running under one — the `firefly-web` correlation layer sets a -task-local id per HTTP request — it reuses that id, so the command and the -saga/spawned task it triggers all trace to the same value. If no ambient id is -present (a background job, a test, an internal dispatch), it generates a fresh -one for the span of that dispatch and restores the prior scope on the way out, so -sibling operations never leak ids into one another. +**What just happened.** On each dispatch the middleware **ensures-or-generates** a +correlation id: if the request is already running under one — the `firefly-web` +correlation layer sets a task-local id per HTTP request — it reuses that id, so +the command and the saga/spawned task it triggers all trace to the same value. If +no ambient id is present (a background job, a test, an internal dispatch), it +generates a fresh one for the span of that dispatch and restores the prior scope +on the way out, so sibling operations never leak ids into one another. ```rust,ignore // Inside a handler (or anything it calls), the id is observable: let trace = firefly_kernel::correlation_id(); // Some() under the middleware ``` -Because the framework installs `ValidationMiddleware` outermost on Lumen's bus (installed first by Core), followed by `CorrelationMiddleware`, -the same id that the HTTP layer stamped on `POST /wallets/:id/deposit` flows into -the `Deposit` handler, into the transfer saga it may start, and into the events -the saga publishes — without any handler touching the id explicitly. It sits -outermost in the chain shown earlier — the correlation scope is already open -before `QueryCache` and `ValidationMiddleware` run, so anything they log carries -the id too: +**Why it matters.** On Lumen's bus the framework installs `ValidationMiddleware` +first (so it is outermost), then `CorrelationMiddleware`, then `QueryCache`. The +same id that the HTTP layer stamped on `POST /wallets/:id/deposit` flows into the +`Deposit` handler, into the transfer saga it may start, and into the events the +saga publishes — without any handler touching the id explicitly. Because +correlation sits ahead of `QueryCache` in the chain, the correlation scope is +already open before the cache layer runs, so anything the cache logs carries the +id too:
send / query a message @@ -494,7 +624,7 @@ the id too: - C = Correlation Q = QueryCache V = ValidationMiddleware + V = ValidationMiddleware C = Correlation Q = QueryCache @@ -502,7 +632,7 @@ the id too: your handler -
Registering CorrelationMiddleware first puts it outermost: the correlation scope opens before QueryCache and ValidationMiddleware run, so everything they log carries the id.
+
The framework registers ValidationMiddleware first (outermost), then CorrelationMiddleware, then QueryCache: the correlation scope opens before the cache layer runs, so everything it logs carries the id.
> **Design note.** `CorrelationMiddleware` ensures one logical request keeps one @@ -512,12 +642,13 @@ the id too: > scope on the way out. Firefly threads the id through a task-local that this > middleware scopes per dispatch, so a handler never has to pass it by hand. -## Dispatching from the controller +## Step 8 — Dispatch from the controller -The `#[rest_controller]` (built in [First HTTP API](./06-first-http-api.md)) -holds the `Bus` and dispatches through `send` / `query`. `Bus::query` is a -readability synonym for `send`. A failed dispatch is a `CqrsError`, which the web -layer maps to the right RFC 9457 status: +**Action.** Wire the HTTP surface to the bus. The `#[rest_controller]` (built in +[Your First HTTP API](./06-first-http-api.md)) holds the `Bus` and dispatches +through `send` / `query`. `Bus::query` is a readability synonym for `send`. A +failed dispatch is a `CqrsError`, which the web layer maps to the right RFC 9457 +status: ```rust,ignore // samples/lumen/src/web.rs — WalletApi handlers. @@ -540,11 +671,18 @@ async fn get( } ``` +**What just happened.** `api.bus.send(body)` matches `body`'s type +(`OpenWallet`) to the `open_wallet` command handler and runs it through the +middleware chain; `api.bus.query(GetWallet { id })` does the same for the query. +The controller autowires the `Arc` (`#[autowired] pub bus: Arc`), so +`api.bus` already has a receiver — no hand-built state. + `cqrs_to_web` is the seam where a domain failure becomes an HTTP status. It reads the `CqrsError` and its detail string — which, recall, is the `DomainError`'s stable `Display` text from the previous chapter — and chooses the status: ```rust,ignore +// samples/lumen/src/web.rs fn cqrs_to_web(err: CqrsError) -> WebError { match err { CqrsError::Validation(detail) => WebError::from(FireflyError::validation(detail)), @@ -565,15 +703,32 @@ fn cqrs_to_web(err: CqrsError) -> WebError { } ``` -This is why the domain chapter insisted the `Display` strings be *stable*: they -are the contract `cqrs_to_web` matches on to recover the precise status. - -## Read-after-write: invalidating the cache - -`GetWallet` is cached for 30 seconds. Without care, a deposit would update the -balance while a cached `GetWallet` kept serving the old one for up to 30 seconds. -Lumen closes that gap by invalidating the cached query family after every -mutation: +**Why it matters.** This is why the domain chapter insisted the `Display` strings +be *stable*: they are the contract `cqrs_to_web` matches on to recover the precise +status. A validation `CqrsError` becomes a 422 problem; a "not found" handler +detail becomes a 404; an insufficient-funds or non-positive-amount detail becomes +a 422; anything else falls through to a 500 — all rendered as RFC 9457 +`application/problem+json`. + +> **Tip** **Checkpoint.** With `cargo run` up, open a wallet and read it back: +> +> ```bash +> curl -s -XPOST localhost:8080/api/v1/wallets \ +> -H 'content-type: application/json' \ +> -d '{"owner":"alice","openingBalance":100}' +> # 201 with {"id":"...","owner":"alice","balance":100} +> +> curl -s -XPOST localhost:8080/api/v1/wallets \ +> -H 'content-type: application/json' -d '{"owner":""}' +> # 422 problem+json — the empty owner failed the #[firefly(validate)] check +> ``` + +## Step 9 — Keep reads fresh after a write + +**Action.** Close the read-after-write gap. `GetWallet` is cached for 30 seconds. +Without care, a deposit would update the balance while a cached `GetWallet` kept +serving the old one for up to 30 seconds. Lumen invalidates the cached query +family after every mutation: ```rust,ignore // samples/lumen/src/web.rs — deposit handler. @@ -590,7 +745,7 @@ async fn deposit( } ``` -This is where `WalletApi` finally grows the second field [First HTTP +**What just happened.** This is where `WalletApi` grows the field [Your First HTTP API](./06-first-http-api.md) deferred: alongside `bus`, the controller **autowires** the `Arc` from the container (`#[autowired] pub query_cache: Arc`), so `api.query_cache` has a receiver. The same @@ -600,18 +755,32 @@ handler. `QueryCache::invalidate_type::()` evicts every cached result for exactly that query type. The withdraw handler does the same, and the transfer -saga ([Sagas](./12-sagas.md)) — which touches two wallets — invalidates the -whole `GetWallet` family. The query cache's backend swap (Redis / Postgres) and -event-driven invalidation get their own treatment in [Caching](./17-caching.md); -here, the point is that the *bus* is where read-after-write consistency lives, -not the handler. - -## The reactive bus - -The bus also exposes a reactive surface that wraps the eventual result in a lazy -`Mono` — the same handler lookup, the same middleware chain, run only when the -`Mono` is subscribed, blocked, or awaited. The methods take `&Arc` so the -`Mono` can own the bus: +saga ([Sagas](./12-sagas.md)) — which touches two wallets — invalidates the whole +`GetWallet` family. + +**Why it matters.** Read-after-write consistency lives at the *bus boundary*, not +inside the handler. The handler computes the new state; the controller, having +just mutated, evicts the cache so the next `GetWallet` recomputes. The query +cache's backend swap (Redis / Postgres) and event-driven invalidation get their +own treatment in [Caching](./17-caching.md); here, the point is that a mutation +and its cache eviction sit side by side on the write path. + +> **Tip** **Checkpoint.** Deposit into the wallet you opened, then read it back — +> the new balance comes through immediately, even though `GetWallet` is cached for +> 30 seconds, because the deposit handler evicted the cached entry: +> +> ```bash +> curl -s -XPOST localhost:8080/api/v1/wallets//deposit \ +> -H 'content-type: application/json' -d '{"amount":50}' +> curl -s localhost:8080/api/v1/wallets/ # balance reflects the deposit +> ``` + +## Step 10 — Dispatch reactively (optional) + +**Action.** When you want a lazy, composable result, use the bus's reactive +surface. The bus wraps the eventual result in a lazy `Mono` — the same handler +lookup, the same middleware chain, run only when the `Mono` is subscribed, +blocked, or awaited. These methods take `&Arc` so the `Mono` can own the bus: | Method | Returns | |---------------------------------|---------------| @@ -627,8 +796,8 @@ background job or a reactive pipeline assembled before the request context is in play). The plain `send_mono` / `query_mono` inherit whatever context is ambient at subscribe time. -A reactive `GetWallet`, composing on the `Mono` from -[The Reactive Model](./05-reactive-model.md): +A reactive `GetWallet`, composing on the `Mono` from [The Reactive +Model](./05-reactive-model.md): ```rust,ignore use std::sync::Arc; @@ -641,16 +810,21 @@ let balance = bus .await?; // Some() or None ``` -Because `firefly-reactive` fixes its error channel to `FireflyError`, a failed -dispatch is mapped from `CqrsError` into a status-faithful `FireflyError` -(validation → 422, missing handler → 500), with the original `CqrsError` -preserved as `source()`. So a reactive command flows straight into the RFC 9457 -problem stack while staying inspectable. +**What just happened.** `query_mono` describes the dispatch without running it; +`.map(..)` composes a transformation onto the still-lazy `Mono`; `.block().await` +finally runs the chain and yields `Result, FireflyError>` — `Some` on +a hit, `None` if the `Mono` completed empty. -## Proving the handler bean +**Why it matters.** Because `firefly-reactive` fixes its error channel to +`FireflyError`, a failed dispatch is mapped from `CqrsError` into a status-faithful +`FireflyError` (validation → 422, missing handler → 500), with the original +`CqrsError` preserved as `source()`. So a reactive command flows straight into the +RFC 9457 problem stack while staying inspectable. -Lumen's `src/commands.rs` exercises the handler bean directly with no HTTP — the -test that ships in the crate. The bean operates on its `#[autowired]` +## Step 11 — Prove the wiring with tests + +**Action.** Lumen's `src/commands.rs` exercises the handler bean directly with no +HTTP — the test that ships in the crate. The bean operates on its `#[autowired]` collaborators, so the test constructs it with the same `Ledger` + `ReadModel` the container would inject and calls its methods (the full bus wiring is covered end-to-end by the HTTP tests, which boot the whole `FireflyApplication`): @@ -686,7 +860,11 @@ async fn handler_bean_operates_on_its_autowired_collaborators() { } ``` -And the validation derive is testable on its own — no bus needed, because +**What just happened.** Because the handler is a plain method on a plain struct, +the test needs no bus and no DI container — just the collaborators. It opens, +deposits, and reads back, asserting the balance moves as expected. + +The validation derive is testable on its own too — no bus needed, because `#[derive(Command)]` generates `validate()` directly on the type: ```rust,ignore @@ -706,38 +884,47 @@ fn get_wallet_carries_cache_ttl() { } ``` -## What changed in Lumen +> **Tip** **Checkpoint.** `cargo test` is green. The handler-bean test and the +> validation/cache tests pass without a running server, and the HTTP integration +> tests boot the full `FireflyApplication` to cover the bus end to end. + +## Recap — what changed in Lumen Lumen's read and write paths are now separate, typed, and bus-dispatched: - **`src/commands.rs`** — `OpenWallet` / `Deposit` / `Withdraw` carry - `#[derive(Command)]` with `#[firefly(validate)]` on required fields; - `GetWallet` carries `#[derive(Query)]` with `#[firefly(cache_ttl = "30s")]`. - The derives generate the `Message` impl, the `validate()` checks, and the - query's `cache_ttl`. + `#[derive(Command)]` with `#[firefly(validate)]` on required fields; `GetWallet` + carries `#[derive(Query)]` with `#[firefly(cache_ttl = "30s")]`. The derives + generate the `Message` impl, the `validate()` checks, the query's `cache_ttl`, + and each message's `kind()` (the command/query split). - **The `WalletHandlers` bean** (`#[derive(Service)]` + `#[handlers]`) carries the `#[command_handler]` / `#[query_handler]` methods and `#[autowired]`s the `Ledger` + `ReadModel` — a Spring `@Component` command/query handler. Command handlers build the `Money` value object and delegate to `self.ledger`; the query serves `self.read_model` and falls back to folding the stream for read-after-write freshness. A simpler app can write a collaborator-free handler - as a free `async fn` instead (the same `#[command_handler]` macro applies to - free functions). + as a free `async fn` instead (the same `#[command_handler]` macro applies). - **Constructor injection, no process-global.** The handler bean reaches its collaborators through `#[autowired]` fields the container fills, so there is no - `OnceLock` to seed and no `bind` step — the `ledger` `#[bean]` is now a pure - factory. + `OnceLock` to seed and no `bind` step — the `ledger` `#[bean]` is a pure factory. - **The bus** is a framework-provided bean the `WalletApi` autowires; the framework resolves the handler bean from the container and drains its methods onto the bus (`register_discovered_handler_beans`, alongside the free-`fn` - `register_discovered_handlers`) and auto-installs the correlation, - `QueryCache`, and validation middleware. The controller dispatches via - `bus.send` / `bus.query`, with `cqrs_to_web` mapping a `CqrsError` (carrying the - domain `Display` string) to the right RFC 9457 status — 422 for business rules, - 404 for not-found. + `register_discovered_handlers`) and auto-installs the validation, correlation, + and `QueryCache` middleware. The controller dispatches via `bus.send` / + `bus.query`, with `cqrs_to_web` mapping a `CqrsError` (carrying the domain + `Display` string) to the right RFC 9457 status — 422 for business rules, 404 for + not-found. +- **Command/query segregation** falls out of the derives: one `Bus`, + `command_handler_names()` / `query_handler_names()` filtering by `kind()`, and + the admin `/cqrs` dashboard rendering the two halves apart. - **Read-after-write** is enforced at the bus boundary: `query_cache.invalidate_type::()` runs after every mutation. +You also now know that the bus exposes a reactive surface (`send_mono` / +`query_mono`, returning a lazy `Mono`) whose error channel is `FireflyError`, +so a reactive dispatch flows straight into the RFC 9457 problem stack. + ## Exercises 1. **Watch validation short-circuit.** In a test, build a `Bus`, add @@ -749,9 +936,9 @@ Lumen's read and write paths are now separate, typed, and bus-dispatched: 2. **Prove the cache, then bust it.** Against the framework-assembled router (`build_router().await`), `query(GetWallet { id })` twice and confirm the - second is served from cache (instrument the `ReadModel::find` or trace a - counter). Deposit into the wallet, then `query` again — assert the new balance - comes back, proving `invalidate_type::()` did its job. + second is served from cache (instrument `ReadModel::find` or trace a counter). + Deposit into the wallet, then `query` again — assert the new balance comes back, + proving `invalidate_type::()` did its job. 3. **Add a `CloseWallet` command.** Define `CloseWallet { #[firefly(validate)] wallet_id: String }` with `#[derive(Command)]`, then add a `#[command_handler] @@ -766,7 +953,22 @@ Lumen's read and write paths are now separate, typed, and bus-dispatched: return just the balance as JSON. Note where the `FireflyError` channel takes over from `CqrsError`. +5. **Inspect the split.** In a test, register all four of Lumen's handlers on a + `Bus`, then assert `bus.command_handler_names()` has three entries and + `bus.query_handler_names()` has one. Confirm `bus.handler_count()` is `4` and + that `bus.has_handler::()` is `true`. This is exactly what the admin + `/cqrs` dashboard renders. + +## Where to go next + The bus dispatches *within* the service. To propagate what happened *between* collaborators — the read-model projection, external subscribers — fan out domain events. Continue to -[Event-Driven Architecture & Messaging](./10-eda-messaging.md). +**[Event-Driven Architecture & Messaging](./10-eda-messaging.md)**. + +- The handlers delegate to the `Ledger`, which rehydrates the aggregate and + persists its events — that machinery is **[Event Sourcing](./11-event-sourcing.md)**. +- A command that touches two wallets runs as a compensating saga in **[Sagas, + Workflows & TCC](./12-sagas.md)**. +- The query cache's backend swap and event-driven invalidation get their own + treatment in **[Caching](./17-caching.md)**. diff --git a/docs/book/src/10-eda-messaging.md b/docs/book/src/10-eda-messaging.md index c7f128d..d221b71 100644 --- a/docs/book/src/10-eda-messaging.md +++ b/docs/book/src/10-eda-messaging.md @@ -1,13 +1,3 @@ - - # Event-Driven Architecture & Messaging By the end of the [CQRS chapter](./09-cqrs.md), Lumen could open a wallet, @@ -15,97 +5,186 @@ deposit, withdraw, and read a balance — but the command side and the query sid were quietly cheating. The `Wallet` aggregate raised crisp domain events (`WalletOpened`, `MoneyDeposited`, `MoneyWithdrawn`), the `Ledger` persisted them, and then nothing carried them anywhere. The read model the `GetWallet` -query serves had to be repaired on the fly by re-folding the event stream. - -By the end of *this* chapter, Lumen closes the loop. Every event the ledger -persists is also **published** to a `Broker`, and a read-model **projection** — -a `#[derive(Service)]` bean whose `#[event_listener]` method consumes those -events — keeps the query side current without the write side knowing it exists. -That is event-driven architecture: a fact is published once, and any number of -independent reactions subscribe to it. The audit trail, the welcome -notification, the balance read model — each becomes a subscriber you can add -months later without touching a single command handler. +query serves had to be repaired on the fly by re-folding the event stream every +time it was read. + +By the end of *this* chapter, Lumen closes that loop. Every event the ledger +persists is also **published** to a broker, and a read-model **projection** — a +bean whose method consumes those published events — keeps the query side current +without the write side ever knowing it exists. That is event-driven +architecture: a fact is published once, and any number of independent reactions +subscribe to it. The audit trail, the welcome notification, the balance read +model — each becomes a subscriber you can add months later without touching a +single command handler. + +We will build the loop the way Lumen's own source builds it: a one-function +bridge that turns a persisted domain event into a wire envelope, a publish call +at the end of the ledger's commit, and a projection bean that the framework +discovers and wires for you. Then we will tour the messaging machinery around it +— glob topics, consumer groups, retry/dead-letter, filters, the reactive +surface, in-process events, and the production transports — so you know which +tool reaches for which job. + +By the end of this chapter you will: + +- Distinguish a **domain event** (event-sourcing's durable fact) from a + **messaging event** (the wire envelope), and bridge one to the other with a + single mapping function. +- Publish every committed event from the `Ledger` to a `Broker`, in the order + that guarantees a subscriber never sees an uncommitted fact. +- Write the read-model **projection** as a `#[derive(Service)]` bean whose + `#[event_listener]` method the framework discovers and subscribes for you — + and understand why rebuilding from the stream makes it idempotent. +- Use the broker's reach: glob topic patterns, consumer groups, retry with + dead-lettering, per-envelope filters, and the reactive `Flux` surface. +- Tell the broker's three event roles apart — `#[event_listener]`, + `#[application_event_listener]` / `#[transactional_event_listener]`, and + `externalize_after_commit` — and swap the in-memory broker for Kafka, RabbitMQ, + Postgres, or Redis without changing a handler. + +## Concepts you will meet + +Before the first line of code, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — event-driven architecture (EDA).** A style in which +> components communicate by *publishing facts* rather than calling each other. A +> producer announces that something happened; any number of *subscribers* react, +> and the producer neither knows nor cares who they are. The Spring analog is +> Spring Cloud Stream / Spring for Apache Kafka — a publish/subscribe layer over +> a message broker. + +> **Note** **Key term — broker.** A *broker* is the transport that carries a +> published event to its subscribers. `firefly-eda` defines a transport-agnostic +> `Broker` *port*; the in-process `InMemoryBroker` is the default, and Kafka, +> RabbitMQ, Postgres, and Redis implement the same port. This is the role +> Spring's `MessageChannel` / `KafkaTemplate` + listener container plays. + +> **Note** **Key term — projection.** A *projection* consumes a stream of events +> and maintains a derived, query-optimized view (the *read model*). It is the +> read side of Command/Query Responsibility Segregation, kept current by +> reacting to the write side's events. Spring developers build these with +> `@KafkaListener` / `@EventListener` methods writing into a query store. + +> **Note** **Key term — idempotent.** An operation is *idempotent* when applying +> it more than once has the same effect as applying it once. Under at-least-once +> delivery a broker may hand the same event to a subscriber twice; an idempotent +> projection absorbs the redelivery without corrupting its view. `firefly-eda` is the framework's **event-driven architecture port**. It defines the `Event` envelope every Firefly event flows through, the -`Publisher`/`Subscriber`/`Broker` ports, an in-process `InMemoryBroker`, and the -messaging machinery — glob topics, consumer groups, retry/DLQ, event filters, -and a reactive `Flux` subscription surface. The production transports (Kafka, -RabbitMQ, Postgres outbox, Redis Streams) implement the same ports and slot in -at wiring time, so Lumen's projection never changes when the broker does. +`Publisher` / `Subscriber` / `Broker` ports, an in-process `InMemoryBroker`, and +the messaging machinery — glob topics, consumer groups, retry/DLQ, event +filters, and a reactive `Flux` subscription surface. The production transports +(Kafka, RabbitMQ, Postgres outbox, Redis Streams) implement the same ports and +slot in at wiring time, so Lumen's projection never changes when the broker +does. > **Design note.** The `Broker` is Firefly's transport-agnostic messaging port: -> you publish an `Event` to it and subscribe handlers with -> `#[event_listener(topic = …)]`. `wrap_listener` adds retry and dead-lettering; -> subscriptions accept glob topic patterns. Production transports (Kafka, -> RabbitMQ, Postgres outbox, Redis Streams) implement the same port and slot in -> at wiring time, so a handler never changes when the broker does. +> you publish an `Event` to it and subscribe handlers to it. `wrap_listener` adds +> retry and dead-lettering; subscriptions accept glob topic patterns. Because +> every production transport implements the same port, a handler never changes +> when the broker does — the wiring chooses the adapter, the code stays put. -## Two kinds of "event" in one wallet +## Step 1 — Tell the two kinds of "event" apart Before wiring anything, it is worth being precise about the word *event*, because Lumen ends up using it for two different things and confusing them leads -to the wrong port. +to reaching for the wrong port. + +> **Note** **Key term — domain event vs messaging event.** A *domain event* (the +> event-sourcing kind) is the durable, versioned fact an aggregate raises; it +> lives in the event *store* and is the [next chapter](./11-event-sourcing.md)'s +> subject. A *messaging event* is the wire envelope that carries a fact *to +> subscribers*; it lives on the *broker*. In Spring terms: a domain event is a +> JPA-persisted record in your event table; a messaging event is the payload you +> hand to `KafkaTemplate.send(...)`. A **domain event** in the event-sourcing sense — `firefly::eventsourcing`'s `DomainEvent` — is the durable, versioned fact the `Wallet` aggregate raises and -the [next chapter](./11-event-sourcing.md) makes the source of truth. It lives -in the event *store*. +the next chapter makes the source of truth. It lives in the event *store*. A **messaging event** — `firefly::eda`'s `Event` — is the wire envelope that carries a fact *to subscribers*. It lives on the *broker*. Lumen bridges the two -with one function (`to_envelope`, below): the ledger persists a `DomainEvent`, -then maps it onto an `Event` and publishes it. This chapter is about the second -kind — getting the fact onto the wire and reacting to it. The first kind is the -next chapter's subject. +with one function (`to_envelope`, built in Step 3): the ledger persists a +`DomainEvent`, then maps it onto an `Event` and publishes it. This chapter is +about the second kind — getting the fact onto the wire and reacting to it. The +first kind is the next chapter's subject. -## The `Event` envelope +> **Tip** **Checkpoint.** You can state, in one sentence each, what a domain +> event is (a durable fact in the store) and what a messaging event is (a wire +> envelope on the broker), and which chapter owns each. Keep that distinction in +> hand — every code path below sits firmly on one side of it. -`Event` is Firefly's canonical wire envelope, with a stable JSON shape (fixed -field names and omission rules) so producers and consumers agree on the bytes -regardless of broker or service — any system that honours the contract -interoperates. Construct one with `Event::new`, which also stamps -`correlation_id` from the kernel's task-local correlation scope: +## Step 2 — Read the `Event` envelope + +`Event` is Firefly's canonical wire envelope. It has a *stable JSON shape* — +fixed field names and omission rules — so producers and consumers agree on the +bytes regardless of broker or service. Any system that honours the contract +interoperates: the same envelope is wire-compatible across the Java, .NET, Go, +and Python ports of Firefly. + +Construct one with `Event::new`, which also stamps `correlation_id` from the +kernel's task-local correlation scope (so a published event carries the same +correlation id as the request that produced it): ```rust use firefly_eda::Event; let ev = Event::new( - "orders.created", // topic - "OrderCreated", // event type - "orders-svc", // source + "orders.created", // topic — where it is published + "OrderCreated", // event type — the logical name + "orders-svc", // source — the producing service Some(br#"{"id":"o1"}"#.to_vec()), // payload (base64 on the wire) ) -.with_header("x-tenant", "acme") -.with_key(b"customer-42".to_vec()); // partition / routing key +.with_header("x-tenant", "acme") // arbitrary routing/metadata header +.with_key(b"customer-42".to_vec()); // partition / routing key ``` -The `key` carries the intended partition/routing key per the `Event` contract; -it is omitted from the wire when absent, so older events stay byte-for-byte -identical. Note that the current adapters do not key off it yet: the Kafka -adapter derives its record key from `correlation_id` (falling back to the event -id), and the RabbitMQ adapter routes on the topic. Treating `key` as the -partition/routing key is design intent rather than a guarantee of today's -adapters. +What just happened, field by field. `Event::new` takes four positional +arguments — `topic`, `event_type`, `source`, and an optional `payload` — and +fills in the rest: a fresh `id`, the current `time` (UTC), and the ambient +`correlation_id`. The two builder methods are additive: `with_header` inserts a +string→string header into a sorted map (so the encoding is deterministic), and +`with_key` sets the optional partition/routing key. -### Lumen's domain-event-to-envelope bridge - -Lumen never builds an `Event` by hand in a handler. The ledger owns one mapping -function that turns a persisted `DomainEvent` into the canonical envelope, -carrying the JSON-encoded domain event as the payload and the wallet id as the -intended partition key — the `Event` contract's design for keeping per-wallet -events ordered, once the adapters key off it: +The `key` carries the intended partition/routing key per the `Event` contract; +it is *omitted* from the wire when absent, so events produced before the field +existed stay byte-for-byte identical. One honest caveat: the current adapters do +not yet route off `key`. The Kafka adapter derives its record key from +`correlation_id` (falling back to the event id), and the RabbitMQ adapter routes +on the topic. Treating `key` as the partition/routing key is the contract's +*design intent* rather than a guarantee of today's adapters. + +> **Tip** **Checkpoint.** You can build an `Event` and read back its fields: +> `Event::new("t", "T", "s", None).with_header("k", "v").headers.get("k")` +> returns `Some("v")`, and `.with_key(b"abc".to_vec()).key` is +> `Some(b"abc".to_vec())`. An absent key never appears on the wire. + +## Step 3 — Bridge a domain event to the envelope + +Lumen never builds an `Event` by hand inside a handler. The ledger owns one +mapping function that turns a persisted `DomainEvent` into the canonical +envelope — carrying the JSON-encoded domain event as the payload and the wallet +id as the intended partition key. Put it in `samples/lumen/src/ledger.rs` +alongside the two shared constants the publisher and the projection both key off: ```rust use firefly::eda::Event; use firefly::eventsourcing::DomainEvent; -/// The EDA topic every wallet domain event is published to. +use crate::domain::AGGREGATE_TYPE; // the const "Wallet" + +/// The EDA topic every wallet domain event is published to. The projection +/// and any external subscriber key off it. pub const EVENTS_TOPIC: &str = "wallets.events"; + /// The logical EDA source stamped on published events. pub const EVENT_SOURCE: &str = "lumen"; -/// Maps a persisted `DomainEvent` onto the canonical EDA `Event` envelope. +/// Maps a persisted `DomainEvent` onto the canonical EDA `Event` envelope, +/// carrying the JSON-encoded domain event as the payload and the wallet id as +/// the partition key (so per-wallet events stay ordered on a real broker). pub fn to_envelope(event: &DomainEvent) -> Event { let payload = serde_json::to_vec(event).expect("domain event serialises"); Event::new( @@ -115,35 +194,42 @@ pub fn to_envelope(event: &DomainEvent) -> Event { Some(payload), ) .with_key(event.aggregate_id.clone().into_bytes()) - .with_header("aggregateType", "Wallet") + .with_header("aggregateType", AGGREGATE_TYPE) .with_header("aggregateId", event.aggregate_id.clone()) .with_header("version", event.version.to_string()) } ``` -Three design choices repay attention. The **topic** (`wallets.events`) is a -shared constant — the publisher and the projection key off the same value, so -the channel name can never drift. The **key** is the wallet id — the intended -partition key so that, once a broker keys off it, every event for one wallet -lands on the same partition and stays in order. (Today's Kafka adapter keys -records on `correlation_id` and the RabbitMQ adapter routes on the topic, so -this is the `Event` contract's design intent rather than a current guarantee.) -The **headers** (`aggregateId`, `version`) carry just enough -routing metadata for a subscriber to find and re-fold the affected aggregate -without decoding the payload — which is exactly what Lumen's projection does -below. +What just happened, with three design choices worth pausing on: + +- The **topic** (`wallets.events`) is a shared constant. The publisher and the + projection key off the *same* `EVENTS_TOPIC` value, so the channel name can + never drift between them — a renamed constant moves both sides at once. +- The **key** is the wallet id. It is the intended partition key so that, once a + broker routes off it, every event for one wallet lands on the same partition + and stays in order. (Today's Kafka adapter keys records on `correlation_id` + and RabbitMQ routes on the topic, so this is the contract's design intent, not + a current guarantee — exactly as Step 2 explained.) +- The **headers** (`aggregateType`, `aggregateId`, `version`) carry just enough + routing metadata for a subscriber to find and re-fold the affected aggregate + *without decoding the payload*. That is precisely what Lumen's projection does + in Step 5 — it reads `aggregateId` from a header and never touches the body. -## Publishing from the ledger +> **Tip** **Checkpoint.** A unit test on `to_envelope` (Lumen ships one) asserts +> `env.topic == EVENTS_TOPIC`, `env.event_type == "WalletOpened"`, +> `env.key == Some(b"wlt_x".to_vec())`, and the `aggregateId` / `version` headers +> are set. If those hold, the bridge is faithful. + +## Step 4 — Publish from the ledger (save before you publish) The `Ledger` is the single write path every command and the transfer saga call. After it appends an aggregate's uncommitted events to the store with optimistic -concurrency, it publishes each one — `to_envelope` then `broker.publish` — so -the projection downstream can react: +concurrency, it publishes each one — `to_envelope` then `broker.publish` — so the +projection downstream can react. Here is the `commit` method on `Ledger`: ```rust -use std::sync::Arc; use firefly::eda::Broker; -use firefly::eventsourcing::{EventSourcingError, EventStore}; +use firefly::eventsourcing::EventSourcingError; use crate::domain::{DomainError, Wallet}; @@ -173,24 +259,37 @@ async fn commit(&self, wallet: &mut Wallet, expected: i64) -> Result<(), DomainE } ``` +What just happened. `take_uncommitted()` drains the events the domain command +produced; if there are none, there is nothing to do. Then `store.append(...)` +persists them at the `expected` version — the optimistic-concurrency check. Only +*after* that succeeds does the loop turn each event into an envelope and publish +it. The `broker` here is an `Arc`: the ledger codes against the +*port*, never a concrete transport. + Notice the ordering: **append before publish.** A subscriber must never see a -fact that did not persist. If the append fails (including the -optimistic-concurrency race) the loop is never reached, so no event is broadcast. -The store backing this `Ledger` is the in-memory `MemoryEventStore`; the +fact that did not persist. If the append fails — including the +optimistic-concurrency race — the loop is never reached, so no event is +broadcast. The store backing this ledger is the in-memory event store; the [next chapter](./11-event-sourcing.md) is where that store earns the name *event-sourced*. -> **Note.** Append before publish: a subscriber must never see a fact that did +> **Note** Append before publish: a subscriber must never see a fact that did > not persist. The gap between append and publish — where a crash could persist a -> fact but drop the broadcast — is what the transactional outbox in the +> fact but drop the broadcast — is exactly what the transactional outbox in the > [next chapter](./11-event-sourcing.md) eliminates. -## The in-process broker +> **Tip** **Checkpoint.** `cargo test -p lumen` still passes: the ledger's +> open/deposit/withdraw round-trip persists three events and publishes three +> envelopes, in that order, with no subscriber yet attached. + +## Step 5 — Watch the in-process broker fan out -`InMemoryBroker` is the default — fan-out delivery, glob topic matching, and -per-`(topic, group)` round-robin, with no external dependency. It is the broker -Lumen's `WebStack` exposes as `web.broker`, and it is everything the teaching -build (and the test suite) needs. Subscribe a handler, publish an event: +`InMemoryBroker` is the default transport — fan-out delivery, glob topic +matching, and per-`(topic, group)` round-robin, with no external dependency. It +is the broker the framework's web stack exposes (and registers into the DI +container as the `Arc` port), and it is everything the teaching build +and the test suite need. Before wiring Lumen's projection, see the broker in +isolation — subscribe a handler, publish an event: ```rust use firefly_eda::{handler, Event, InMemoryBroker}; @@ -203,34 +302,68 @@ async fn main() { .subscribe( "wallets.events", handler(|ev: Event| async move { - println!("observed {} for {}", ev.event_type, - ev.headers.get("aggregateId").map(String::as_str).unwrap_or("?")); + println!( + "observed {} for {}", + ev.event_type, + ev.headers.get("aggregateId").map(String::as_str).unwrap_or("?") + ); Ok(()) }), ) .unwrap(); - let ev = Event::new("wallets.events", "WalletOpened", "lumen", - Some(br#"{"wallet_id":"wlt_1"}"#.to_vec())); + let ev = Event::new( + "wallets.events", + "WalletOpened", + "lumen", + Some(br#"{"wallet_id":"wlt_1"}"#.to_vec()), + ); broker.publish(ev).await.unwrap(); broker.close().unwrap(); } ``` -`InMemoryBroker::publish` awaits each subscribed handler sequentially on the -publisher's task; the first handler error short-circuits and is returned to the -publisher. After `close()`, publish and subscribe fail with `EdaError::Closed`. +What just happened. `handler(closure)` wraps an async closure as a reference- +counted delivery callback (the type the broker stores per subscription). +`subscribe(topic, handler)` registers it for the topic; the inherent method on +the concrete `InMemoryBroker` is synchronous, so it returns a `Result` you +`.unwrap()` rather than `.await`. `publish(ev).await` then runs every matching +handler sequentially on the publisher's task. `close()` releases the broker. + +> **Note** `InMemoryBroker::publish` awaits each subscribed handler sequentially +> on the publisher's task; the first handler error short-circuits and is returned +> to the publisher (wrapped in `EdaError::Handler`). After `close()`, both +> publish and subscribe fail with `EdaError::Closed`. (When you reach for the +> `dyn Broker` *port* instead of the concrete type — as Lumen's ledger does — +> the trait methods are `async`, so you `.await` `subscribe` too. The concrete +> inherent methods are sync; the port methods are async. Same broker, two +> surfaces.) -## The read-model projection — a `#[event_listener]` bean +> **Tip** **Checkpoint.** Run that `main`. It prints `observed WalletOpened for +> wlt_1`. If you swap the subscribe topic to a glob like `wallets.*`, it still +> matches — that is Step 6. + +## Step 6 — Close the loop with a projection bean Here is where Lumen closes the CQRS loop. The **projection** is a DI bean — the -Rust analog of a Spring `@Component @EventListener`. `WalletProjection` is a -`#[derive(Service)]` whose collaborators are `#[autowired]` from the container: -the `Ledger` (for the event store it replays) and the `ReadModel` it feeds — the -*same* `ReadModel` the `GetWallet` query reads. The `#[handlers]` impl marks its -method with `#[event_listener(topic = "wallets.events")]`, so for each delivered -event it reaches its collaborators through `self`, reloads the affected wallet's -stream, folds it into a `WalletView`, and upserts it: +Rust analog of a Spring `@Component` with an `@EventListener` method. +`WalletProjection` is a `#[derive(Service)]` whose collaborators are +`#[autowired]` from the container: the `Ledger` (for the event store it replays) +and the `ReadModel` it feeds — the *same* `ReadModel` the `GetWallet` query +reads. A `#[handlers]` impl marks its method with +`#[event_listener(topic = "wallets.events")]`, so for each delivered event the +framework calls it; it reaches its collaborators through `self`, reloads the +affected wallet's stream, folds it into a `WalletView`, and upserts it. + +> **Note** **Key term — `#[derive(Service)]` / `#[handlers]` / `#[event_listener]`.** +> `#[derive(Service)]` marks a struct as a singleton DI bean whose `#[autowired]` +> fields the container fills (Spring's `@Service`/`@Component`). `#[handlers]` on +> the impl tells the framework to scan its methods for handler attributes. +> `#[event_listener(topic = …)]` subscribes one method to a broker topic — the +> `@KafkaListener` analog. You write the reaction; the framework does the +> subscribing. + +Add this to `samples/lumen/src/ledger.rs`: ```rust use std::sync::Arc; @@ -239,7 +372,7 @@ use firefly::eda::Event; use firefly::prelude::*; use crate::domain::Wallet; -use crate::ledger::{Ledger, ReadModel}; +// `Ledger` and `ReadModel` are defined earlier in this same module. /// The read-model **projection bean** — Spring's `@Component @EventListener`. It /// `#[autowired]`s the `Ledger` (for the event store it replays) and the @@ -277,7 +410,16 @@ impl WalletProjection { } ``` -Two properties make this a *good* projection rather than just a working one. +What just happened, line by line. The struct has two `#[autowired]` fields, so +the container constructs `WalletProjection` by handing it the existing `Ledger` +and `ReadModel` singletons — no `new`, no wiring code. The `project` method reads +`aggregateId` from a header (the routing metadata Step 3 stamped); if it is +missing, the event is not for us and we return `Ok(())`. Otherwise we `load` the +wallet's full event stream, `rehydrate` the aggregate, take its `.view()`, and +`upsert` it into the read model. The method returns `FireflyResult<()>` — the +framework's `Result<(), FireflyError>`. + +Two properties make this a *good* projection rather than merely a working one. It is **idempotent.** Rather than mutating the read-model row from the single delivered event (`balance += amount`), it reloads the wallet's full stream and @@ -290,29 +432,33 @@ needs to find the stream. It is **decoupled.** `WalletProjection` imports no command, calls no handler, and has no idea a deposit was processed. It reacts purely to the published fact. You can add a `FraudDetector` or a `WelcomeNotifier` subscriber next to it -without touching a line of the command path. +without touching a line of the command path — which is exactly Exercise 1. -> **Note.** `#[event_listener(topic = "wallets.events")]` on a `#[handlers]` bean +> **Note** `#[event_listener(topic = "wallets.events")]` on a `#[handlers]` bean > method submits a `BeanListenerRegistration` into the `inventory` registry the -> framework drains (`subscribe_discovered_listener_beans(broker, container)`): at -> boot `FireflyApplication` resolves `WalletProjection` from the container — -> autowiring its `Ledger` + `ReadModel` — and subscribes its `project` method to -> the topic. The subscription is wired for you; you write only the reaction. +> framework drains. At boot, `FireflyApplication` resolves `WalletProjection` +> from the container — autowiring its `Ledger` + `ReadModel` — and subscribes its +> `project` method to the topic via +> `subscribe_discovered_listener_beans(broker, container)`. The subscription is +> wired for you; you write only the reaction. -### How the projection reaches its collaborators +> **Tip** **Checkpoint.** `cargo test -p lumen` passes the full HTTP loop: a +> `POST /api/v1/wallets/:id/deposit` flows command → ledger → store → broker → +> projection → read model, and the next `GET /api/v1/wallets/:id` is served from +> the projected view — no manual repair. + +## Step 7 — Understand how the projection is wired (no composition root) Because the projection is a regular container bean, its collaborators arrive by **constructor injection** through `#[autowired]` fields — no process-global to seed, no `bind` step. The container hands `WalletProjection` the same `Ledger` (hence the same event store) and the same `ReadModel` it hands the CQRS handlers, so the events the handlers publish are exactly the events the projection consumes -and projects into the read the `GetWallet` query serves. (When a listener needs -*no* injected collaborators, the simpler free-`fn` form — a bare -`#[event_listener(topic = "…")] async fn(ev: Event) -> FireflyResult<()>` — is -the alternative, discovered the same way via `subscribe_discovered_listeners`.) +and projects into the read the `GetWallet` query serves. -That is why the `ledger` `#[bean]` factory is now a **pure factory** — it builds -the `Ledger` and returns it, with no projection-seeding side effect: +That is why the `ledger` `#[bean]` factory in `samples/lumen/src/web.rs` is now a +**pure factory** — it builds the `Ledger` and returns it, with no +projection-seeding side effect: ```rust,ignore // samples/lumen/src/web.rs — the `ledger` #[bean] factory. @@ -323,20 +469,35 @@ fn ledger(&self, store: Arc, broker: Arc) -> Ledge } ``` -The subscription itself is wired by `FireflyApplication` — +What just happened. The factory's parameters are themselves autowired: the +container provides the `MemoryEventStore` bean and the `Arc` port +(which the web stack registers, defaulting to `InMemoryBroker`). The factory +upcasts the concrete store to the `dyn EventStore` port and constructs the +`Ledger`. There is no subscribe call here, and no composition root anywhere. + +The subscription itself is wired by `FireflyApplication` at boot: `subscribe_discovered_listener_beans(broker, container)` resolves the projection bean and drains its `#[event_listener]` method onto the broker (alongside the -free-`fn` `subscribe_discovered_listeners`) — so neither the `ledger` factory nor -any composition root calls a subscribe helper by hand. With that, every -`POST /api/v1/wallets/:id/deposit` flows command → ledger → store → broker → -projection → read model, and the next `GET /api/v1/wallets/:id` is served from the -projected view. The HTTP test suite proves the loop converges: it deposits, -withdraws, then reads back the balance and `version` the projection folded — no -manual repair needed. - -## Glob topics and consumer groups - -A subscription topic is a glob pattern (`*`, `?`, `[..]`, `{a,b}`); a published +free-`fn` `subscribe_discovered_listeners` for listeners that need no injected +collaborators). So neither the `ledger` factory nor any composition root calls a +subscribe helper by hand. + +> **Design note.** The whole loop is *declared*, not *assembled*. You declare the +> store bean, the read-model bean, the ledger factory, and the projection bean; +> the framework discovers each, autowires its dependencies, and subscribes the +> listener — the Spring `@Configuration` + `@Bean` + component-scan story, with +> the listener-endpoint registry replaced by the inventory drain. When a listener +> needs *no* injected collaborators, the simpler free-`fn` form — a bare +> `#[event_listener(topic = "…")] async fn(ev: Event) -> FireflyResult<()>` — is +> the alternative, discovered the same way. + +> **Tip** **Checkpoint.** Read Lumen's startup report. The +> `:: cqrs handlers: … | event listeners: … | scheduled tasks: …` line now counts +> at least one event listener — the projection the framework just subscribed. + +## Step 8 — Reach further: glob topics and consumer groups + +A subscription topic is a glob *pattern* (`*`, `?`, `[..]`, `{a,b}`); a published event is delivered to every subscription whose pattern matches its topic. Lumen subscribes to the exact `wallets.events`, but a multi-event service could fan a single listener across a family: @@ -346,9 +507,12 @@ broker.subscribe("wallets.*", handler(|ev| async move { Ok(()) })).unwrap(); // matches wallets.events, wallets.audit, ... ``` -Consumer groups give competing-consumer delivery: within a group each matching -event goes to exactly **one** member (round-robin); distinct groups each get -their own copy: +> **Note** **Key term — consumer group.** A *consumer group* is a set of +> subscribers that *compete* for a topic's events: each matching event goes to +> exactly **one** member of the group (round-robin), while distinct groups each +> get their own copy. This is Kafka's consumer-group model and Spring's +> `group` / `@KafkaListener(groupId=…)` — the way you scale a workload +> horizontally without double-processing. ```rust,ignore broker.subscribe_group("wallets.events", "projections", handler1).unwrap(); @@ -357,16 +521,29 @@ broker.subscribe_group("wallets.events", "projections", handler2).unwrap(); ``` This is how you would scale Lumen's projection horizontally: run several -projector instances in one group and the broker shares the partitions among -them, each instance owning a slice of the wallet space. +projector instances in one group and the broker shares the events among them +(round-robin per `(topic, group)`), each instance owning a slice of the wallet +space. An ungrouped subscription — the kind `#[event_listener]` makes by default +— always receives its own copy. + +> **Tip** **Checkpoint.** In an `InMemoryBroker` test, subscribe two handlers to +> the same group and publish two events; each handler runs once. Subscribe two +> *ungrouped* handlers and publish one event; both run. That is fan-out versus +> competing-consumer, in four lines. -## Retry and dead-letter +## Step 9 — Make failures survivable: retry and dead-letter `wrap_listener(handler, publisher, policy)` is the adapter-agnostic retry/DLQ wrapper. A failing delivery is retried up to `retries` times with linear backoff (`retry_delay * attempt`); on exhaustion the event is republished to the -dead-letter topic (when set), carrying the original payload/key/headers plus -`x-original-topic` and `x-exception` diagnostic headers: +dead-letter topic (when set), carrying the original payload, key, and headers +plus `x-original-topic` and `x-exception` diagnostic headers: + +> **Note** **Key term — dead-letter topic (DLT/DLQ).** When a message keeps +> failing, you do not want it blocking the stream forever. A *dead-letter topic* +> is where exhausted messages are parked for later inspection or replay. This is +> Spring Kafka's `DefaultErrorHandler` dead-letter routing and +> `@RetryableTopic`. ```rust use std::sync::Arc; @@ -387,18 +564,33 @@ broker.subscribe("wallets.events", wrapped).unwrap(); # }); ``` +What just happened. `ListenerPolicy::with_retries(3)` sets three retries after +the first attempt (four attempts total); `.retry_delay(...)` adds linear +backoff; `.dead_letter_topic(...)` names the topic to park exhausted events on. +`wrap_listener` returns a new `Handler` you subscribe in place of the inner one. +A policy with no retries, no topic, and no store is a pass-through — it returns +the original handler unchanged, so wrapping is zero-overhead when unconfigured. + Lumen's projection takes a gentler path — it *swallows* a transient store miss and returns `Ok(())` rather than failing the delivery, so a single poison message never stalls the stream. That is the right call for a rebuild-from-stream -projection (the next redelivery, or the next event for the wallet, converges -anyway). A side-effecting listener — one that sends an email or calls an external -API — is where `wrap_listener` and a dead-letter topic earn their keep. +projection: the next redelivery, or the next event for the wallet, converges +anyway. A *side-effecting* listener — one that sends an email or calls an +external API — is where `wrap_listener` and a dead-letter topic earn their keep, +because there the work cannot simply be re-derived. For an inspectable record of failures (rather than a routing topic), wire an `EdaDeadLetterStore` via `ListenerPolicy::dead_letter_store`: an exhausted event -is captured into the store (queryable with `list` / `get` / `remove`). +is captured into the store (queryable with `list` / `get` / `remove`). You can +set both — capture *and* route — on one policy. + +> **Tip** **Checkpoint.** Wrap an always-failing handler with +> `ListenerPolicy::with_retries(2).dead_letter_topic("orders.DLT")` over a broker +> that records publishes; after one delivery, exactly one event lands on +> `orders.DLT` with an `x-original-topic` header, and the wrapped handler returns +> `Ok(())` rather than erroring. -## Event filters +## Step 10 — Gate delivery with event filters `EventFilter` is a per-envelope delivery gate layered over topic matching. Where the broker decides *which* subscriptions a topic reaches, a filter decides @@ -417,15 +609,33 @@ broker.subscribe("wallets.events", gated).unwrap(); # }); ``` -An event must pass *every* filter to be delivered; a non-matching event is -dropped before the handler body runs. - -## The reactive subscription surface - -`InMemoryBroker::subscribe_reactive(topic)` is the reactive twin of -`subscribe` — a `Flux` that emits every event delivered to the topic, -composing with Firefly's full reactive-streams operator set. `publish_mono(event)` -is the cold reactive publish (nothing happens until the `Mono` is subscribed): +What just happened. `HeaderEventFilter::new(name, pattern)` compiles an anchored +regex against the named header (a missing header is treated as the empty string). +`with_filters(handler, [filters])` wraps the handler so it runs only for events +that pass *every* filter; a non-matching event is dropped before the handler body +runs — the wrapped handler simply returns `Ok(())`. An empty filter list returns +the handler unchanged (zero overhead). `PredicateEventFilter::new(closure)` is +the escape hatch when a regex over a header is not enough — it gates on any +property of the envelope. + +> **Tip** **Checkpoint.** Build `HeaderEventFilter::new("aggregateType", +> r"^Wallet$")`, wrap a counting handler with `with_filters`, then deliver one +> envelope whose `aggregateType` is `"Account"` and one that is `"Wallet"`. Only +> the second increments the counter. A header filter is cheaper than an `if` +> inside the handler because the drop happens before your code runs — Exercise 3. + +## Step 11 — Consume reactively as a `Flux` + +`InMemoryBroker::subscribe_reactive(topic)` is the reactive twin of `subscribe` — +a `Flux` that emits every event delivered to the topic, composing with +Firefly's full reactive-streams operator set. `publish_mono(event)` is the cold +reactive publish: nothing happens until the returned `Mono` is subscribed. + +> **Note** **Key term — `Flux` / `Mono`.** `Flux` is a reactive stream of +> *many* `T`; `Mono` is a reactive stream of *at most one* `T`. They are +> Firefly's port of Project Reactor's `Flux` / `Mono` (Spring WebFlux). Both are +> *cold* and *lazy*: building one does no work; the work runs when you subscribe +> (here, `.block().await`). ```rust use std::sync::Arc; @@ -447,13 +657,26 @@ assert_eq!(events[0].topic, "wallets.events"); # }); ``` -Deliveries are buffered through a bounded channel; when the downstream consumer -falls behind, the newest events are dropped (`onBackpressureDrop`) rather than -blocking or failing the publisher — extending "a slow consumer never fails -publishers" to the reactive surface. This is the same `Flux` Lumen's optional -streaming endpoint composes over (see [Production & Deployment](./20-production.md)). +What just happened. `subscribe_reactive("wallets.*")` returns a `Flux` +backed by a bounded channel. `publish_mono(...)` builds a cold `Mono<()>`; +`.block().await` drives it, running the publish. Closing the broker drops the +sender, which terminates the `Flux`. Then `flux.take(1).collect_list()` composes +two operators into a `Mono>`; `.block().await` yields a +`Result>, _>`, so the two `.unwrap()`s unwrap the `Result` and +then the `Option`. -## In-process events and after-commit externalization +> **Note** Deliveries are buffered through a bounded channel; when the downstream +> consumer falls behind, the newest events are dropped (`onBackpressureDrop`) +> rather than blocking or failing the publisher — extending the broker's "a slow +> consumer never fails publishers" invariant to the reactive surface. This is the +> same `Flux` Lumen's optional streaming endpoint composes over (see +> [Production & Deployment](./20-production.md)). + +> **Tip** **Checkpoint.** The assertion above holds: one published event arrives +> on the `Flux`, and its `topic` is `wallets.events` even though you subscribed to +> the glob `wallets.*`. + +## Step 12 — In-process events and after-commit externalization The broker carries events *between* services. Inside one service you often want the same decoupling without a network hop: one component raises a fact, others @@ -461,6 +684,11 @@ react, and none of them knows the others exist. That is Spring's `ApplicationEventPublisher` / `@EventListener`, and Firefly ships it as a thread-safe, async, in-process bus alongside the broker. +> **Note** **Key term — in-process event bus.** A *process-local* publish/subscribe +> bus: you `publish_event(value)` and any `#[application_event_listener]` for that +> type reacts — no broker, no network, no serialization. Spring's +> `ApplicationEventPublisher.publishEvent(...)` + `@EventListener`. + Publish with `publish_event`, and listen with `#[application_event_listener]` on a free async function that takes the event by shared reference. Listeners are discovered across the crate graph (the same `inventory` scan that finds your @@ -502,8 +730,8 @@ phase; a rolled-back transaction fires the `after_rollback` listeners and never the `after_commit` ones, so a failed write can never leak a "success" side-effect. With no transaction active the listener falls back to running immediately (treating the work as already committed), so the same handler is -useful in a unit test or a datasource-less path. If you want transactional event -semantics without a SQL datasource at all, register the +useful in a unit test or a datasource-less path. If you want transactional +event semantics without a SQL datasource at all, register the `LocalTransactionManager` (the Rust equivalent of Spring's `ResourcelessTransactionManager`). @@ -528,15 +756,23 @@ publish_event(WalletOpened { id: wallet_id }).await; forwards through `publish_to_broker` (which serializes the payload and publishes via the `register_broker`-registered `Broker`). A committed transaction reaches Kafka, RabbitMQ, or whichever transport you wired; a rolled-back one publishes -nothing. +nothing. Forwarding after commit is best-effort — a missing broker or a publish +failure does not unwind the already-committed transaction; reach for a real +outbox (next chapter) when you need at-least-once. + +Three distinct roles, easy to keep straight: + +- `#[event_listener("topic")]` *consumes* from a broker topic — the + `@KafkaListener` analog (Lumen's projection in Step 6). +- `#[application_event_listener]` / `#[transactional_event_listener]` handle + *in-process* events. +- `externalize_after_commit` is the *bridge* from the second to a broker producer. -Three distinct roles, easy to keep straight: `#[event_listener("topic")]` -*consumes* from a broker topic (the `@KafkaListener` analog above); -`#[application_event_listener]` / `#[transactional_event_listener]` handle -*in-process* events; and `externalize_after_commit` is the bridge from the -second to a broker producer. +> **Tip** **Checkpoint.** You can name, for each of those three, whether it +> crosses a process boundary (only the first and the bridge do) and whether it is +> transaction-aware (the transactional listener and the bridge are). -## Production transports +## Step 13 — Swap in a production transport Each transport crate implements the same `Broker` port; swap the constructor and keep every handler. Code against `firefly_eda::Broker` and select the adapter at @@ -545,14 +781,15 @@ wiring time — for a `FireflyApplication` service that is a `firefly.*` config with a Kafka one and the projection, the ledger, and every command keep compiling unchanged. -| Crate | Backend | Constructor | -|------------------------|-----------------|----------------------------------------------| -| `firefly-eda-kafka` | Apache Kafka | `new_kafka_broker(KafkaConfig)?` | -| `firefly-eda-rabbitmq` | RabbitMQ | `RabbitMqBroker::new(RabbitMqBrokerConfig)` | -| `firefly-eda-postgres` | Postgres outbox | `PostgresBroker::new(PostgresConfig::new(dsn))` | +| Crate | Backend | Constructor | +|------------------------|-----------------|---------------------------------------------------| +| `firefly-eda-kafka` | Apache Kafka | `new_kafka_broker(KafkaConfig)?` | +| `firefly-eda-rabbitmq` | RabbitMQ | `RabbitMqBroker::new(RabbitMqBrokerConfig)` | +| `firefly-eda-postgres` | Postgres outbox | `PostgresBroker::new(PostgresConfig::new(dsn))` | | `firefly-eda-redis` | Redis Streams | `RedisStreamsBroker::connect(RedisConfig::new(url))?` | -Kafka, for example — note the handler body is identical to Lumen's: +Kafka, for example — note the handler body is identical to Lumen's, and because +you hold a `Box` here the trait methods are `async`: ```rust,no_run use firefly_eda::{handler, Event}; @@ -579,6 +816,13 @@ broker.publish(ev).await?; # } ``` +What just happened. `new_kafka_broker(KafkaConfig { … })?` returns a +`Box` (hence the `?`). On the *port*, `subscribe` and `publish` are +`async` trait methods, so you `.await` both — the only difference from the +concrete `InMemoryBroker` in Step 5, whose inherent methods are sync. The closure +inside `handler(...)` is byte-for-byte what you would write for the in-memory +broker. + Redis Streams uses a connect-then-start lifecycle: ```rust,no_run @@ -600,19 +844,32 @@ broker.start().await?; # } ``` -> **Note** — The Postgres broker is a **transactional outbox**: events are -> written in the same transaction as your state change and drained to consumers -> via `LISTEN`/`NOTIFY`, giving at-least-once delivery without a separate broker. -> That closes the append-then-publish gap discussed above; the +What just happened. `RedisStreamsBroker::connect(config)?` dials Redis and +returns the broker; `RedisConfig::new(url).with_streams([...]).with_group(...)` +is the builder. You `subscribe` before `start()` — `start()` begins consuming +from the declared streams. (RabbitMQ has the same connect/start shape; +`RabbitMqBroker::new(config)` returns the broker and `start()` declares its +topology.) + +> **Note** The Postgres broker is a **transactional outbox**: events are written +> in the same transaction as your state change and drained to consumers via +> `LISTEN`/`NOTIFY`, giving at-least-once delivery without a separate broker. That +> closes the append-then-publish gap from Step 4; the > [next chapter](./11-event-sourcing.md) covers the outbox primitive directly. +> **Tip** **Checkpoint.** Add the `eda-kafka` feature and provide the `dyn Broker` +> port as a `#[bean]` that constructs `new_kafka_broker(...)`. You do not need a +> running Kafka — `cargo build` confirms the projection, the ledger, and the +> command handlers compile unchanged against the port. That is Exercise 4. + ## Broker health -`EventPublisherHealthIndicator` adapts any broker implementing the -`BrokerHealth` ping probe to a `firefly_observability::Indicator`, surfacing -broker liveness on `/actuator/health` under the `eventPublisher` id — so when -Lumen graduates to a real broker, its readiness shows up alongside the rest of -the service's health (see [Observability](./15-observability.md)). +`EventPublisherHealthIndicator` adapts any broker implementing the `BrokerHealth` +ping probe to a `firefly_observability::Indicator`, surfacing broker liveness on +`/actuator/health` under the `eventPublisher` id — so when Lumen graduates to a +real broker, its readiness shows up alongside the rest of the service's health +(see [Observability](./15-observability.md)). The in-memory broker reports `UP` +until it is closed. ## Recap — what changed in Lumen @@ -628,10 +885,22 @@ and projects it back automatically. | `WalletProjection` (`#[derive(Service)]` + `#[handlers]`) | The projection **bean**: `#[autowired]`s the `Ledger` + `ReadModel`, its `#[event_listener]` method rebuilds the read model from the stream | | `#[event_listener(topic = "wallets.events")]` | Marks the bean method; submits a `BeanListenerRegistration` the framework drains (`subscribe_discovered_listener_beans`) — resolving the bean and subscribing the method | | Constructor injection | The projection reaches its collaborators through `#[autowired]` fields — no `OnceLock`, no `bind`; the `ledger` `#[bean]` is a pure factory | -| framework `Broker` (`InMemoryBroker`) | The default transport — swap the adapter for Kafka/RabbitMQ/Redis, keep the listener | - -Three principles carry forward: **save before you publish** so a subscriber -never sees an uncommitted fact; **make projections idempotent** so at-least-once +| framework `Broker` (`InMemoryBroker`) | The default transport — swap the adapter for Kafka/RabbitMQ/Redis/Postgres, keep the listener | + +You also now know: + +- The difference between a **domain event** (durable, in the store) and a + **messaging event** (the wire envelope, on the broker) — and which chapter owns + each. +- The broker's reach: glob topics, fan-out vs. competing-consumer **groups**, + `wrap_listener` retry/dead-letter, per-envelope **filters**, and the reactive + `Flux` surface. +- The three event roles — `#[event_listener]` (broker consume), + `#[application_event_listener]` / `#[transactional_event_listener]` (in-process), + and `externalize_after_commit` (the bridge). + +Three principles carry forward: **save before you publish** so a subscriber never +sees an uncommitted fact; **make projections idempotent** so at-least-once redelivery is harmless (Lumen re-folds the stream rather than applying a delta); and **depend on the `Broker` port, not the adapter** so the in-memory broker becomes Kafka with a one-line change. @@ -655,17 +924,37 @@ recomputed. 2. **Prove idempotency.** In a test, build a `Ledger` over a `MemoryEventStore` and an `InMemoryBroker`, subscribe the projection, open a wallet, and deposit twice. Then publish the *same* `MoneyDeposited` envelope a second time with - `broker.publish(to_envelope(&event))` and assert the read-model `WalletView` - balance is unchanged — the rebuild-from-stream fold absorbs the redelivery. + `broker.publish(to_envelope(&event)).await` and assert the read-model + `WalletView` balance is unchanged — the rebuild-from-stream fold absorbs the + redelivery. -3. **Gate by aggregate type.** Wrap the projection's handler with - `with_filters` and a `HeaderEventFilter::new("aggregateType", r"^Wallet$")`, - then publish an envelope whose `aggregateType` header is `"Account"` and - confirm the projection does not run for it. Explain why a header filter is a - cheaper guard than checking inside the handler body. +3. **Gate by aggregate type.** Wrap the projection's handler with `with_filters` + and a `HeaderEventFilter::new("aggregateType", r"^Wallet$")`, then publish an + envelope whose `aggregateType` header is `"Account"` and confirm the + projection does not run for it. Explain why a header filter is a cheaper guard + than checking inside the handler body. (Hint: the drop happens before the + handler is invoked.) 4. **Swap in a real broker (sketch).** Add the `eda-kafka` feature to the crate and provide the `dyn Broker` port as a `#[bean]` that constructs `new_kafka_broker(...)` instead of relying on the default in-memory broker. You do not need a running Kafka — the point is to confirm the projection, the ledger, and the command handlers compile unchanged against the `Broker` port. + +5. **Route a failure.** Wrap an always-failing handler with + `wrap_listener(inner, broker.clone(), ListenerPolicy::with_retries(2) + .dead_letter_topic("wallets.events.DLT"))`, subscribe it, and subscribe a + second handler to `wallets.events.DLT`. Publish one event and assert the + dead-letter handler observes it carrying an `x-original-topic` header of + `wallets.events`. + +## Where to go next + +- Make these events durable and replayable in **[Event + Sourcing](./11-event-sourcing.md)** — where the in-memory store becomes the + source of truth and the transactional outbox closes the append-then-publish gap. +- Surface broker liveness and request metrics in + **[Observability](./15-observability.md)** — the `eventPublisher` health + indicator joins the rest of the actuator surface. +- Wire a real Kafka or RabbitMQ transport and the reactive streaming endpoint in + **[Production & Deployment](./20-production.md)**. diff --git a/docs/book/src/11-event-sourcing.md b/docs/book/src/11-event-sourcing.md index 89f77aa..4ca27bf 100644 --- a/docs/book/src/11-event-sourcing.md +++ b/docs/book/src/11-event-sourcing.md @@ -1,61 +1,86 @@ - - # Event Sourcing The [last chapter](./10-eda-messaging.md) left one question politely unasked. -Lumen's `Ledger` persists wallet events and the projection rebuilds the read -model by re-folding the stream — but *what stream?* So far the wallet's -canonical state has been implied. By the end of this chapter it is explicit and -load-bearing: the `Wallet` aggregate holds **no stored balance at all**. Its -balance is a pure function of an append-only stream of `WalletOpened`, -`MoneyDeposited`, and `MoneyWithdrawn` events, recomputed every time the -aggregate is loaded. +Lumen's `Ledger` persists wallet events and a projection rebuilds the read model +by re-folding the stream — but *what stream?* So far the wallet's canonical state +has been implied. By the end of this chapter it is explicit and load-bearing: the +`Wallet` aggregate holds **no stored balance at all**. Its balance is a pure +function of an append-only stream of `WalletOpened`, `MoneyDeposited`, and +`MoneyWithdrawn` events, recomputed every time the aggregate is loaded. That is **event sourcing**: instead of storing current state and discarding each change, you store the *sequence of changes* and derive state by replaying them. A financial ledger is the ideal domain for it — accountants have known for centuries that a ledger's authority comes from its entries, not from the running total at the foot of the column. The total is a *derived fact*; the entries are -the *source of truth*. By the end of this chapter, an auditor asking "what was -wallet `wlt_…`'s balance after the third movement?" gets an answer Lumen can -*prove* from the stream, not merely report from a column. - -`firefly-eventsourcing` provides the framework's **event-sourced aggregate** -primitives: an `AggregateRoot` that tracks uncommitted events, an `EventStore` -with optimistic concurrency, snapshots, projections, a global cross-aggregate -stream, a transactional outbox, and multi-tenancy. Where the -[EDA chapter](./10-eda-messaging.md)'s `Event` envelope was the *transport* for a -fact, the `DomainEvent` here is the *record* of it — the durable truth from which -state is rebuilt. - -> **Design note.** `firefly-eventsourcing` provides the event-sourced aggregate -> primitives used here: `AggregateRoot::raise` to record an event, an `EventStore` -> with optimistic concurrency, and projections that build read models. A command -> `raise`s an event; the same `apply` fold runs on both the write path and replay — -> that symmetry is the correctness guarantee of event sourcing. The `DomainEvent` -> serializes to a stable, versioned, language-neutral JSON contract, so any service -> that honors it interoperates regardless of the language it is written in. - -## State storage vs event storage - -The clearest way to feel the shift is to compare what Lumen's storage *holds* in -each model. - -In the **state-storage model**, the store keeps only the wallet's current state: +the *source of truth*. By the end, an auditor asking "what was wallet `wlt_…`'s +balance after the third movement?" gets an answer Lumen can *prove* from the +stream, not merely report from a column. + +This chapter is a guided build. We introduce each piece from first principles, +write it block by block against the real `firefly-eventsourcing` API, and stop at +checkpoints so you can confirm what you have before moving on. Nothing here is +hand-waved: every type, method, and derive matches the crate that ships in +[`samples/lumen`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen). + +By the end of this chapter you will: + +- Explain the difference between **state storage** and **event storage**, and why + the wallet's balance becomes a computation rather than a column. +- Define domain events with `#[derive(DomainEvent)]` and an event-sourced + aggregate with `#[derive(AggregateRoot)]`, and know exactly what each derive + generates. +- Implement the canonical command shape — validate, `raise`, then `apply` — and + understand why the **same fold runs on both the write path and replay**. +- Persist and reload events through the `EventStore` port with **optimistic + concurrency**, and handle a concurrency conflict correctly. +- Recognise the production-grade seams the crate provides — snapshots, + projections, the global stream, the transactional outbox, upcasters, and + multi-tenancy — and know when each one earns its keep. + +## Concepts you will meet + +Each idea below is reintroduced in context where it is first used; this is the +short version so the vocabulary is not new when you reach it. + +> **Note** **Key term — event sourcing.** A persistence style where you store the +> ordered *sequence of changes* (events) to an entity rather than its current +> state, and recompute state by replaying that sequence. The Java/Spring analog +> is the `firefly-event-sourcing-spring-boot-starter` (or Axon Framework's +> event-sourced aggregates). + +> **Note** **Key term — domain event.** An immutable record that *something +> happened* in the domain, named in the past tense (`MoneyDeposited`). In +> event sourcing the events are the system of record. This is distinct from the +> EDA `Event` envelope of the [last chapter](./10-eda-messaging.md), which is the +> *transport* for a fact; the `DomainEvent` here is the durable *record* of it. + +> **Note** **Key term — aggregate.** A cluster of domain objects treated as a +> single consistency boundary, with one **aggregate root** as its entry point. +> Every command goes through the root, which enforces the aggregate's invariants. +> Lumen's aggregate is the `Wallet`; its root is the embedded framework +> `AggregateRoot`. This is the Domain-Driven Design "aggregate" Spring developers +> know from `@Entity` roots — but here it is reconstructed from events, not loaded +> from a row. + +> **Note** **Key term — optimistic concurrency.** A way to detect concurrent +> writes without locking: each write declares the version it expected to find, +> and the store rejects it if another writer got there first. The Spring/JPA +> analog is `@Version` optimistic locking. + +## Step 1 — Feel the shift: state storage vs event storage + +Before writing a line, look at what Lumen's storage *holds* in each model. The +contrast is the whole motivation for this chapter. + +In the **state-storage model** — the default everywhere else — the store keeps +only the wallet's current state: | id | owner | balance | version | |----|-------|---------|---------| | wlt_a1 | alice | 120 | 3 | -Every deposit and withdrawal overwrites `balance`. The history is gone — you know +Every deposit and withdrawal overwrites `balance`. The history is gone: you know the wallet holds 120 cents now; you cannot know how it got there. In the **event-storage model**, the store keeps the stream: @@ -67,17 +92,31 @@ In the **event-storage model**, the store keeps the stream: | wlt_a1 | 3 | MoneyWithdrawn | `{"wallet_id":"wlt_a1","amount":30}` | The current balance is still 120 cents — but now you can read every decision that -led to it, replay to any version, and audit the lot. The trade-off is real: reads -cost a replay (mitigated by **snapshots**) and events are immutable (schema -evolution handled by **upcasters**). Both have first-class support below. +led to it, replay to any version, and audit the lot. + +What just happened: the same final balance now has a *derivation*. The trade-off +is real and worth naming up front — reads cost a replay (mitigated by +**snapshots**, Step 8) and events are immutable (schema change handled by +**upcasters**, Step 11). Both have first-class support, and you will meet them in +turn. -> **Note** — Event sourcing is not the same as the -> [previous chapter](./10-eda-messaging.md)'s EDA. There, the aggregate stored -> its state and *published* events as a side effect. Here the events *are* the -> state: there is no `balance` column to keep in sync — the balance is computed -> by folding the stream every time the aggregate loads. +> **Note** Event sourcing is *not* the same as the +> [previous chapter](./10-eda-messaging.md)'s EDA. There, the aggregate stored its +> state and *published* events as a side effect. Here the events *are* the state: +> there is no `balance` column to keep in sync — the balance is computed by +> folding the stream every time the aggregate loads. -## The mental model +> **Tip** **Checkpoint.** You can state, in one sentence, what each table loses or +> keeps: state storage keeps the answer and discards the working; event storage +> keeps the working and recomputes the answer. The rest of the chapter makes that +> recomputation concrete. + +## Step 2 — The mental model: raise, append, fold + +Everything below is three moves repeated. A command **raises** an event onto the +aggregate; the store **appends** the raised events durably under optimistic +concurrency; a later load **folds** the stream back into current state. Hold this +cycle in mind — every API in the chapter is one of these three moves.
The event-sourcing cycle. raise stages an event on the aggregate, EventStore::append persists the uncommitted events under optimistic concurrency, and a later load rehydrates the aggregate by folding its stream back into the current state.
-## The Wallet's domain events +The framework piece that powers all three moves is `firefly-eventsourcing`, +re-exported through the facade as `firefly::eventsourcing`. + +> **Note** **Key term — `firefly-eventsourcing`.** The framework's event-sourcing +> crate. It provides the `AggregateRoot` (uncommitted-event buffer + version), the +> `EventStore` port (append/load with optimistic concurrency), snapshots, +> projections, a global cross-aggregate stream, a transactional outbox, upcasters, +> and multi-tenancy. You depend on none of it directly — it arrives through the +> single `firefly` facade, and the two derives (`DomainEvent`, `AggregateRoot`) +> come in through `firefly::prelude`. -In Lumen the three events are plain payload structs carrying -`#[derive(DomainEvent)]`. The derive stamps each with a stable `EVENT_TYPE` -discriminator (its struct name) and a `to_domain_event(...)` conversion onto the -framework wire event — so the event type is never spelled as a bare string -literal at the call sites: +## Step 3 — Define the Wallet's domain events + +The action: declare the three events the wallet can produce. In Lumen each is a +plain payload struct carrying `#[derive(DomainEvent)]`. They live in +`src/domain.rs`. ```rust -use firefly::eventsourcing::DomainEvent; +use firefly::eventsourcing::{AggregateRoot, DomainEvent}; +use firefly::prelude::*; use serde::{Deserialize, Serialize}; /// Payload of the event raised when a wallet is opened. @@ -146,33 +195,61 @@ pub struct MoneyWithdrawn { } ``` -`#[derive(DomainEvent)]` generates, for each struct, a `pub const EVENT_TYPE: -&'static str` equal to the struct name (`"WalletOpened"`, …), an `event_type()` -accessor, and a `to_domain_event(aggregate_id, aggregate_type, version)` that -JSON-encodes the payload into a framework `DomainEvent`. That generated -`EVENT_TYPE` is the only thing the aggregate and its `apply` fold reference, so a -rename of the struct flows through automatically. - -> **Note.** `#[derive(DomainEvent)]` generates, per struct, a `pub const -> EVENT_TYPE` equal to the struct name (the routing discriminator), an -> `event_type()` accessor, and a `to_domain_event(...)` that JSON-encodes the -> payload into a framework `DomainEvent` — so the event type is never a bare string -> literal at the call sites, and a struct rename flows through automatically. The -> generated JSON is a stable, versioned, language-neutral contract that any service -> honoring it can consume. - -## The Wallet aggregate — raise, then apply - -The `Wallet` carries `#[derive(AggregateRoot)]`, which finds the embedded -framework `AggregateRoot` field (`root`) and generates `Wallet::AGGREGATE_TYPE` -plus `aggregate()` / `aggregate_mut()` accessors. The projected state (`owner`, -`balance`, `opened`) is *not* stored — it is folded from the stream: +What just happened, block by block: + +- The two **types** `AggregateRoot` and `DomainEvent` come from + `firefly::eventsourcing`. The two **derive macros** of the same name come from + `firefly::prelude::*` — the glob that re-exports every framework macro, so a + service depends on one crate yet still writes `#[derive(DomainEvent)]`. +- `Serialize`/`Deserialize` make each payload JSON-encodable; the derive needs + `Serialize` because it JSON-encodes the payload into the stored event. +- Each event is named in the **past tense** and carries only the data the fact + needs. `opening_balance` and `amount` are in minor units (cents) — Lumen never + stores floating-point money. + +Now the important part — what `#[derive(DomainEvent)]` generates. For each struct +it produces: + +- a `pub const EVENT_TYPE: &'static str` equal to the struct name + (`"WalletOpened"`, `"MoneyDeposited"`, `"MoneyWithdrawn"`) — the routing + discriminator; +- an `event_type()` accessor returning that const; +- a `to_domain_event(aggregate_id, aggregate_type, version)` method that + JSON-encodes the payload into a framework `DomainEvent`. + +That generated `EVENT_TYPE` const is the *only* thing the aggregate and its fold +reference, so the event type is never a bare string literal at the call sites — +and a rename of the struct flows through automatically. + +> **Note** **Key term — `DomainEvent` (the wire type).** Beside the derive there +> is a concrete `firefly::eventsourcing::DomainEvent` struct: the wire shape of +> every persisted event, with an `aggregate_id`, `aggregate_type`, 1-based +> `version`, `event_type`, `time`, a base64 `payload`, optional `metadata`, and an +> optional `tenant_id`. Its JSON is a stable, versioned, language-neutral contract +> — byte-compatible with the Java, .NET, Go, and Python ports, so any service that +> honours it interoperates regardless of language. + +> **Tip** **Checkpoint.** `cargo build` compiles the three structs. In a quick +> test you can assert `WalletOpened::EVENT_TYPE == "WalletOpened"` and round-trip +> a payload through `serde_json::to_vec` / `from_slice`. The events exist; nothing +> raises them yet. + +## Step 4 — Define the Wallet aggregate + +The action: declare the aggregate that produces those events. The `Wallet` +carries `#[derive(AggregateRoot)]`, which finds the embedded framework +`AggregateRoot` field and wires up the type discriminator and accessors. Crucially, +the projected state (`owner`, `balance`, `opened`) is **not stored** — it is folded +from the stream. ```rust use firefly::eventsourcing::{AggregateRoot, DomainEvent}; use crate::money::Money; +/// The aggregate-type discriminator stamped onto every event a wallet raises. +pub const AGGREGATE_TYPE: &str = "Wallet"; + #[derive(Debug, Clone, AggregateRoot)] #[firefly(aggregate_type = "Wallet")] pub struct Wallet { @@ -186,10 +263,36 @@ pub struct Wallet { } ``` -Every command follows the canonical event-sourcing shape: validate the -invariant, `raise` the matching event onto the embedded root, then apply it to -in-memory state. The write path and the replay path run the *same* `apply` code — -that symmetry is the correctness guarantee of event sourcing. +What just happened: + +- The embedded `root: AggregateRoot` field is the framework's bookkeeping — it + holds the aggregate id, the current version, and the buffer of *uncommitted* + events that commands raise but the store has not yet persisted. Rust composes + this field rather than subclassing a base class. +- `#[derive(AggregateRoot)]` locates that `root` field (the default field name; + override with `#[firefly(field = "...")]`) and generates a + `Wallet::AGGREGATE_TYPE` const plus `aggregate()` / `aggregate_mut()` accessors + onto the embedded root. `#[firefly(aggregate_type = "Wallet")]` sets the + discriminator explicitly (it would default to the struct name anyway). +- `owner`, `balance`, and `opened` are **projected fields**: they exist only in + memory and are reconstructed by folding the stream. `Money` is Lumen's + cents-based value object from `src/money.rs`. + +> **Note** **Key term — uncommitted events.** Events a command has `raise`d onto +> the aggregate root but the store has not yet persisted. They live in the root's +> buffer until you `take_uncommitted()` them and hand them to `EventStore::append`. +> Think of them as the aggregate's pending write. + +> **Tip** **Checkpoint.** `cargo build` succeeds and `Wallet::AGGREGATE_TYPE` +> evaluates to `"Wallet"`. The aggregate is declared but has no behaviour yet — +> Step 5 adds the commands. + +## Step 5 — Write a command: validate, raise, apply + +The action: give the wallet behaviour. Every command follows the canonical +event-sourcing shape — validate the invariant, `raise` the matching event onto the +embedded root, then apply it to in-memory state. Here is `deposit`, plus the small +private helper that serialises a payload and raises it. ```rust,ignore /// Credits `amount` to the wallet, raising a `MoneyDeposited` event. @@ -215,18 +318,65 @@ fn raise(&mut self, event_type: &str, payload: &P) { } ``` -`AggregateRoot::raise` buffers the event (so the ledger can persist it) and bumps -the version. `withdraw` is the same shape, with one extra guard: it computes the -remaining balance *first* and lets `Money::subtract` reject an overdraw — so a -failed withdrawal raises **no** event at all, leaving the stream clean. That -overdraft guard is the trigger the transfer saga relies on in -[Sagas, Workflows & TCC](./12-sagas.md). +What just happened, in order: + +1. `require_opened()?` enforces the invariant: you cannot deposit into a wallet + that was never opened. A failed check returns `DomainError::NotFound` and + raises **no** event. +2. `amount.require_positive()?` rejects a non-positive deposit before any event is + recorded. +3. `self.raise(MoneyDeposited::EVENT_TYPE, …)` records the fact. Note the event + type is the generated const, never a string literal. The private `raise` + helper JSON-encodes the payload and calls `self.root.raise(event_type, bytes)`. +4. `self.balance = self.balance.add(amount)` updates the in-memory projection. + +The framework's `AggregateRoot::raise` does two things: it pushes the event onto +the uncommitted buffer (so the ledger can persist it later) and bumps the version +by one. That version bump is what later powers optimistic concurrency. + +`withdraw` is the same shape with one extra guard worth seeing, because the +transfer saga in [Sagas, Workflows & TCC](./12-sagas.md) depends on it: + +```rust,ignore +/// Debits `amount` from the wallet, raising a `MoneyWithdrawn` event. +pub fn withdraw(&mut self, amount: Money) -> Result<(), DomainError> { + self.require_opened()?; + let amount = amount.require_positive()?; + let remaining = self.balance.subtract(amount)?; // Overdraw → InsufficientFunds + self.raise( + MoneyWithdrawn::EVENT_TYPE, + &MoneyWithdrawn { + wallet_id: self.root.id.clone(), + amount: amount.cents_value(), + }, + ); + self.balance = remaining; + Ok(()) +} +``` + +Why it matters: `Money::subtract` is computed *first* and rejects an overdraw with +`MoneyError::Overdraw` (mapped to `DomainError::InsufficientFunds`) **before** +`raise` is ever reached. A failed withdrawal therefore raises no event at all, +leaving the stream clean. That overdraft guard is the failure trigger the transfer +saga relies on. + +> **Tip** **Checkpoint.** With `open`, `deposit`, and `withdraw` written, a unit +> test can open a wallet, deposit 50, withdraw 30, then call +> `wallet.take_uncommitted()` and assert it holds exactly three events in order. +> Lumen's own `domain.rs` tests do precisely this. + +## Step 6 — Rehydrate: fold the stream back into state -### Rehydration — folding the stream +The action: rebuild a wallet from its events. **Rehydration** is the load path — +it replays the full ordered stream through the same `apply` the commands use. An +empty stream yields an *unopened* wallet, which is how the ledger distinguishes +"absent" from "exists". -Rehydration is the load path: rebuild a wallet by folding its full ordered stream -through the same `apply` the commands use. An empty stream yields an unopened -wallet — which is how the ledger distinguishes "absent" from "exists": +> **Note** **Key term — rehydration.** Reconstructing an aggregate's current state +> by folding its event stream from the beginning. The Spring/Axon analog is an +> event-sourced repository's `load`, which replays the aggregate's events into a +> fresh instance. ```rust,ignore /// Rebuilds a wallet by folding `events` (its full ordered stream). @@ -271,35 +421,47 @@ fn apply(&mut self, event: &DomainEvent) { } ``` -Note that `apply` folds `MoneyWithdrawn` with a raw subtraction -(`Money::cents(self.balance.cents_value() - p.amount)`) rather than the -overdraft-guarded `Money::subtract` the `withdraw` command uses. That asymmetry is -deliberate: replay never re-validates. The guard already ran at *write* time and a -failed withdrawal raised no event, so every event in the stream is a fact that -already passed its invariant — replay simply applies it. - -The folding logic in `apply` is matched on the *same* `EVENT_TYPE` constant the -commands raise under, so the two halves can never disagree about an event's name. -Lumen's unit tests prove the replay law directly: open + deposit + withdraw on a -*writer* wallet, take its uncommitted stream, then `Wallet::rehydrate` a fresh -wallet from that stream and assert the rebuilt balance, owner, and version match — -state recomputed from events, never stored. - -> **Design note.** The discipline: a command `raise`s the event, `apply` mutates -> the projected fields, and load replays the same `apply` to rebuild state. Lumen -> registers no handler table — it `match`es on the generated `EVENT_TYPE` const, the -> Rust-idiomatic way to keep the write fold and the replay fold from ever -> disagreeing about an event's name. - -## Raising and appending events - -The framework `AggregateRoot` accumulates `DomainEvent`s as you `raise` them; you -`take_uncommitted` and `append` them to the store. `append` enforces optimistic -concurrency: pass the version you loaded, and a concurrent writer's append fails -with `EventSourcingError::Concurrency`: +What just happened: + +- `rehydrate` starts from a blank wallet (`opened: false`, zero balance) and folds + each event through `apply`, keeping `root.version` in step with the stream head. + After the fold, `root.version` equals the version of the last event — which is + exactly the token the next command will append against. +- `apply` matches on `event.event_type` against the generated `EVENT_TYPE` + constants — the same constants the commands raise under — so the write fold and + the replay fold can never disagree about an event's name. + +One subtlety worth pausing on. `apply` folds `MoneyWithdrawn` with a *raw* +subtraction (`self.balance.cents_value() - p.amount`) rather than the +overdraft-guarded `Money::subtract` the `withdraw` *command* uses. That asymmetry +is deliberate: **replay never re-validates**. The guard already ran at write time, +and a failed withdrawal raised no event, so every event in the stream is a fact +that already passed its invariant. Replay simply applies it. + +> **Design note.** This is the correctness guarantee of event sourcing made +> concrete. A command `raise`s an event and `apply` mutates the projected fields; +> a load replays the *same* `apply` to rebuild state. Lumen registers no handler +> table — it `match`es on the generated `EVENT_TYPE` const, the Rust-idiomatic way +> to keep the write fold and the replay fold from ever disagreeing about an +> event's name. + +> **Tip** **Checkpoint.** This is the law to prove: open + deposit + withdraw on a +> *writer* wallet, take its uncommitted stream, then `Wallet::rehydrate` a fresh +> wallet from that stream and assert the rebuilt balance, owner, and version all +> match — state recomputed from events, never stored. Lumen's `rehydrate_folds_the_full_stream` +> test does exactly this. + +## Step 7 — Persist and reload through the `EventStore` + +The action: make the events durable. The framework `AggregateRoot` accumulates +`DomainEvent`s as you `raise` them; you `take_uncommitted` them and `append` them +to an `EventStore`. The store enforces optimistic concurrency — you pass the +version you loaded, and a concurrent writer's append fails. + +Here is the move in isolation, against the in-process store: ```rust -use firefly_eventsourcing::{AggregateRoot, EventStore, MemoryEventStore}; +use firefly::eventsourcing::{AggregateRoot, EventStore, MemoryEventStore}; #[tokio::main] async fn main() { @@ -319,7 +481,14 @@ async fn main() { } ``` -The `EventStore` port: +What just happened: two `raise` calls buffer two events and bump the root to +version 2. `take_uncommitted()` drains the buffer (a Rust-idiomatic fusion of +"return the events" + "clear them"). `append(&id, 0, events)` persists them, where +`0` is the **expected version** — the head we expected to find before writing. +Because the aggregate is brand new, that head is `0`; the append succeeds. Reading +the stream back returns both events in order. + +The `EventStore` port — the contract every store implements: ```rust,ignore #[async_trait] @@ -334,19 +503,39 @@ pub trait EventStore: Send + Sync { } ``` -The default is `MemoryEventStore` — the in-process store Lumen runs on, ideal for -development and tests. `SqlEventStore` backs it with a SQL store over the -`firefly-transactional` `Database` port for production; swapping it is a one-line -change to the `event_store` `#[bean]` in `LumenBeans`, exactly like swapping the -broker in the last chapter. +> **Note** **Key term — `EventStore` port / `MemoryEventStore` adapter.** The +> `EventStore` trait is the persistence boundary — a *port* in the hexagonal +> sense. `MemoryEventStore` is the in-process *adapter* Lumen runs on by default, +> ideal for development and tests. `SqlEventStore::new(db)` is the production +> adapter over the `firefly-transactional` `Database` port. Swapping them is a +> one-line change to the `event_store` `#[bean]` in `LumenBeans` — exactly like +> swapping the broker in the [last chapter](./10-eda-messaging.md). -## The Ledger ties it together +That bean is the only place the choice lives: -Lumen's `Ledger` is the application service that owns the store (and the broker -from the [last chapter](./10-eda-messaging.md)). Every command rehydrates, runs -the domain method, and commits with optimistic concurrency. Here is `deposit` and -the load path — the version the wallet rehydrated to *is* the `expected_version` -the append must match: +```rust,ignore +#[bean] +impl LumenBeans { + /// The in-memory event store (`@Bean`). + #[bean] + fn event_store(&self) -> MemoryEventStore { + MemoryEventStore::new() + } + // ... +} +``` + +> **Tip** **Checkpoint.** Run the example above (or the equivalent `#[tokio::test]`). +> `store.load("u1")` returns a `Vec` of length 2. If you instead call +> `store.append(&user.id, 5, events)` for a fresh aggregate, you get +> `Err(EventSourcingError::Concurrency)` — proof the expected-version check is live. + +## Step 8 — Wire it into the Ledger and handle concurrency + +The action: tie persistence to the domain in one application service. Lumen's +`Ledger` (introduced in the [last chapter](./10-eda-messaging.md)) owns the store +and the broker. Every command rehydrates, runs the domain method, and commits with +optimistic concurrency. Here is `deposit` and the load path: ```rust,ignore /// Credits `amount` to `wallet_id`, persisting + publishing `MoneyDeposited`. @@ -376,34 +565,42 @@ pub async fn load_events(&self, wallet_id: &str) -> Result, Dom } ``` -The `commit` method (shown in full in the [last chapter](./10-eda-messaging.md)) -appends at `expected` then publishes each event. The two chapters meet here: this -one supplies the durable, replayable store; that one carries each appended event -onto the wire so the projection can react. - -### Optimistic concurrency in practice - -Two concurrent requests — say a deposit from the app and a fee withdrawal from a -job — can both load wallet `wlt_a1` at version 3, each apply a change, and each -try to append at `expected_version = 3`. The first append wins and the stream -advances to 4; the second now mismatches and the store returns -`EventSourcingError::Concurrency`. Lumen maps that to a `DomainError::NotFound` -detail ("concurrent modification") so the caller retries from a fresh load. You -never manage version numbers by hand — the version the wallet rehydrated to is -the token, and the store enforces it. - -> **Note.** `append(id, expected_version, events)` enforces optimistic -> concurrency: the version the wallet rehydrated to is the token, and a stale -> append fails with `EventSourcingError::Concurrency`. Catch it and retry the -> load-mutate-save cycle (or surface a 409) — never swallow it. - -## Typed aggregates and the repository - -Lumen folds the stream by hand in `Wallet::apply` because it teaches the -mechanic clearly. For larger aggregates the framework offers a thinner path: -implement `EventSourcedAggregate` — a typed `apply_event` plus optional snapshot -serialization — and let `EventSourcedRepository` tie `load` (snapshot + replay) -and `save` (append + snapshot policy) together: +What just happened: `deposit` loads the wallet (rehydrating from its stream), +captures `wallet.root.version` as `expected`, runs the domain command, then +commits at `expected`. The version the wallet rehydrated to **is** the token the +append must match. `commit` (shown in full in the +[last chapter](./10-eda-messaging.md)) appends at `expected`, then publishes each +appended event to the broker so the projection can react. The two chapters meet +here: this one supplies the durable, replayable store; that one carries each +appended event onto the wire. + +Now the concurrency case, because in a real system two writers race. Suppose a +deposit from the app and a fee withdrawal from a job both load wallet `wlt_a1` at +version 3, each apply a change, and each try to append at `expected_version = 3`. +The first append wins and the stream advances to 4; the second now mismatches, and +the store returns `EventSourcingError::Concurrency`. Lumen maps that to a +`DomainError::NotFound` carrying a "concurrent modification" detail so the caller +retries from a fresh load. You never manage version numbers by hand — the version +the wallet rehydrated to is the token, and the store enforces it. + +> **Note** `append(id, expected_version, events)` enforces optimistic concurrency: +> the rehydrated version is the token, and a stale append fails with +> `EventSourcingError::Concurrency`. Catch it and retry the load-mutate-save cycle +> (or surface a 409) — never swallow it, or you risk losing a write. + +> **Tip** **Checkpoint.** Append the open event for a wallet at +> `expected_version = 0`. Then, *without reloading*, raise a second event and +> append it *also* at `expected_version = 0`. The second append returns +> `EventSourcingError::Concurrency`. A fresh load (which advances `expected` to 1) +> would have succeeded — that is the whole mechanism in four lines. + +## Step 9 — The thinner path: typed aggregates and the repository + +Lumen folds the stream by hand in `Wallet::apply` because it teaches the mechanic +clearly. For larger aggregates the framework offers a thinner path: implement +`EventSourcedAggregate` — a typed `apply_event` plus optional snapshot +serialisation — and let `EventSourcedRepository` tie `load` (snapshot + replay) and +`save` (append + snapshot policy) together. ```rust,ignore use firefly_eventsourcing::{ @@ -442,39 +639,78 @@ assert!(reloaded.is_some()); # } ``` +What just happened: `EventSourcedAggregate` is the trait contract — it exposes the +embedded root via `root()` / `root_mut()` and the read-side fold via `apply_event`. +The repository then orchestrates the glue every event-sourced service otherwise +hand-writes: `save` computes the expected version from the uncommitted batch and +appends with optimistic concurrency; `load` returns `Ok(Some(_))` when the +aggregate has events and `Ok(None)` when it was never persisted. An event with no +handler should return `EventSourcingError::Projection` so reconstruction fails +loudly rather than silently corrupting state. + `EventSourcedRepository::with_snapshots(store, snapshots, interval)` enables -periodic state captures so rehydration does not replay the entire history. +periodic state captures so rehydration does not replay the entire history — which +is the next step. -## Snapshots — when streams get long +> **Tip** **Checkpoint.** You can articulate when each path is right: hand-fold +> (`Wallet::apply`) when the aggregate is small and you want the mechanic in view; +> `EventSourcedRepository` when you want load/save/snapshot orchestration handled +> for you. Both end at the same `EventStore`. + +## Step 10 — Snapshots: bounding replay cost Event sourcing trades write simplicity for read cost: a wallet with 10,000 -movements replays 10,000 events every load. **Snapshots** cut that down. A -snapshot is a serialized checkpoint of the aggregate's state at a version; on -load, the repository deserializes the latest snapshot and replays only the events -after it. A snapshot at version 9,000 turns a 10,000-event replay into 1,000. +movements replays 10,000 events every load. **Snapshots** cut that down. + +> **Note** **Key term — snapshot.** A serialised checkpoint of an aggregate's +> state at a particular version. On load, the repository deserialises the latest +> snapshot and replays only the events *after* it — turning a 10,000-event replay +> into 1,000 if the snapshot sits at version 9,000. The Axon analog is its snapshot +> trigger. Lumen's wallets are short-lived enough that the in-memory store's full replay is -fine, so the sample does not wire snapshots — but the seam is there: -`with_snapshots(store, MemorySnapshotStore::new(), 100)` would checkpoint every -time a wallet's stream crosses a 100-event boundary. Snapshots are an -optimization, never a correctness requirement: remove them and the system is -slower but still correct. - -> **Note.** `with_snapshots(store, MemorySnapshotStore::new(), interval)` -> checkpoints aggregate state every time a stream crosses an interval boundary; on -> load, the repository deserializes the latest snapshot and replays only the events -> after it. The interval-crossing trigger handles a batch that straddles the -> threshold (version 95 → 105 still snapshots). Snapshots are an optimization, never -> a correctness requirement. - -## Projections — building read models - -A `Projection` is a read-side handler. Register projections on a -`ProjectionRunner` and replay an aggregate's events through them. This is the -event-*store* sibling of the [last chapter](./10-eda-messaging.md)'s event-*bus* -listener: Lumen's `project_wallet_event` reacts to events as they are published, -whereas a `ProjectionRunner` can replay history from the beginning to rebuild a -read model from scratch: +fine, so the sample does not wire snapshots — but the seam is one constructor call: + +```rust,ignore +use firefly_eventsourcing::{EventSourcedRepository, MemorySnapshotStore}; + +// Checkpoint each time a wallet's stream crosses a 100-event boundary. +let repo = EventSourcedRepository::::with_snapshots( + store, + Arc::new(MemorySnapshotStore::new()), + 100, +); +``` + +What just happened: `with_snapshots(store, snapshots, interval)` checkpoints +aggregate state every time a stream *crosses* an interval boundary. The trigger is +a crossing, not exact divisibility, so a batch that straddles the threshold +(version 95 → 105) still snapshots. On load, the repository restores the latest +snapshot and replays only the events after it. + +> **Design note.** Snapshots are an optimisation, never a correctness requirement. +> Remove them and the system is slower but still correct — the events remain the +> source of truth, and the snapshot is just a cached fold of the prefix. + +## Step 11 — Projections, the global stream, and the outbox + +These three seams are how event sourcing feeds the rest of a system. You will not +wire all of them into the teaching baseline, but knowing the shape of each is part +of understanding the model. + +### Projections — building read models from history + +> **Note** **Key term — projection.** A read-side handler that consumes events to +> build a query-optimised read model. It must be **idempotent**, because events +> may be replayed during recovery. The Spring analog is a query-side +> `@EventListener` that updates a read table. + +A `Projection` is registered on a `ProjectionRunner`, which can replay an +aggregate's events through it. This is the event-*store* sibling of the +[last chapter](./10-eda-messaging.md)'s event-*bus* listener: Lumen's live +`WalletProjection` reacts to events as they are published, whereas a +`ProjectionRunner` can replay history from the beginning to rebuild a read model +from scratch. ```rust,ignore use std::sync::Arc; @@ -494,12 +730,12 @@ lost or its schema changes, you stop the projector, clear the read model, and replay every stream — the history is right there in the store. A state-storage model cannot do this; it discarded the history at write time. -## The global stream +### The global stream — read models across aggregates -`EventStore::stream_all` exposes the global, cross-aggregate, ordered event -stream with a resumable cursor — the engine for read models that span many -aggregates (think: "all movements across all wallets, in order"). The runner -consumes it in batches, at-least-once and in-order: +`EventStore::stream_all` exposes the global, cross-aggregate, ordered event stream +with a resumable cursor — the engine for read models that span many aggregates +(think "all movements across all wallets, in order"). The runner consumes it in +batches, at-least-once and in-order: ```rust,ignore // Drive one batch; returns the next cursor + any per-event error. @@ -511,15 +747,24 @@ let (next_cursor, err) = runner let cursor = runner.replay_all(&store, None, 100, None).await?; ``` -## The transactional outbox +What just happened: `drive_once` applies one page and returns the cursor to resume +from, advancing it only past *successfully* applied events — so a failed event is +retried on the next call rather than skipped. `replay_all` drains the entire global +stream from a start cursor, paging `batch_size` at a time. + +### The transactional outbox — closing the append-then-publish gap The [last chapter](./10-eda-messaging.md) noted a gap in `Ledger::commit`: it appends, then publishes, and a crash *between* the two persists the fact but drops -the broadcast. `TransactionalOutbox` closes that gap. Instead of publishing -directly, a writer `enqueue`s the `DomainEvent`; a background relay polls and -forwards each pending record to an `OutboxSink`, retrying up to `max_attempts`. -The default `EdaSink` bridges each `DomainEvent` to a `firefly_eda::Event` and -publishes it — the same `to_envelope`-shaped bridge, but durable: +the broadcast. `TransactionalOutbox` closes that gap. + +> **Note** **Key term — transactional outbox.** A pattern where a writer +> *enqueues* an event durably (ideally in the same store transaction as the +> append) instead of publishing it directly, and a background relay forwards each +> pending record to a broker, retrying on failure. Recording the event durably +> *before* dispatching it is what guarantees at-least-once delivery across +> crashes. This is the same outbox pattern Spring teams implement around their +> message broker. ```rust,ignore use std::sync::Arc; @@ -539,28 +784,30 @@ let dead = outbox.dead_letters().await; // exhausted records, for inspection outbox.stop().await; ``` -Exhausted records become dead letters — excluded from the publish loop and -surfaced for inspection or manual retry. This is the upgrade path for a -production Lumen: enqueue into the outbox inside the same store transaction as the -append, and let the relay guarantee at-least-once delivery to the broker even -across crashes — which is exactly why the projection was built to be -**idempotent** in the last chapter. - -> **Design note.** `TransactionalOutbox` closes the append-then-publish gap: -> instead of publishing directly, a writer `enqueue`s the `DomainEvent` in the same -> store transaction as the append, and a background relay forwards each pending -> record to an `OutboxSink`, retrying up to `max_attempts` and dead-lettering on -> exhaustion. Recording the event durably *before* dispatching it is what gives -> at-least-once delivery across crashes — and why the projection was built to be -> idempotent. - -## Schema evolution — upcasters - -`EventUpcaster` migrates events on the **read** paths only, so consumers always -observe current-schema events while the stored history stays untouched. Suppose -Lumen later needs a `reference` field on every deposit for reconciliation: new -events carry it, old `MoneyDeposited` events do not, and an upcaster fills the gap -on load: +What just happened: a writer `enqueue`s a `DomainEvent`; the relay (started with +`start()`) polls and forwards each pending record to an `OutboxSink`, retrying up +to `max_attempts`. The default `EdaSink` bridges each `DomainEvent` to a +`firefly_eda::Event` and publishes it — durable this time. Records that exhaust +`max_attempts` become **dead letters**: excluded from the publish loop and surfaced +via `dead_letters()` for inspection or manual retry. This is the production upgrade +path — and exactly why the projection was built to be **idempotent** in the last +chapter: at-least-once delivery means an event can arrive twice. + +## Step 12 — Schema evolution and multi-tenancy + +Two more seams round out the model. Both operate on the read path, so the stored +history stays immutable. + +### Upcasters — migrating old events on read + +> **Note** **Key term — upcaster.** A transform applied to a stored event when it +> is *read*, migrating it from an old schema to the current one. Consumers always +> observe current-schema events; the stored history is never rewritten. This is +> the event-sourcing answer to schema migration. + +Suppose Lumen later needs a `reference` field on every deposit for reconciliation: +new events carry it, old `MoneyDeposited` events do not, and an upcaster fills the +gap on load: ```rust,ignore use std::sync::Arc; @@ -570,17 +817,27 @@ let store = MemoryEventStore::with_upcasters(vec![Arc::new(MyUpcaster)]); // every event returned by load / load_after passes through applicable upcasters ``` -Old data becomes readable without a migration; new data is written in the current -schema. The events themselves stay immutable — you never rewrite history. +An `EventUpcaster` implements `applies_to(&event) -> bool` and +`upcast(event) -> DomainEvent`. Old data becomes readable without a migration; new +data is written in the current schema; the events themselves stay immutable. You +never rewrite history. + +### Multi-tenancy — one store, many tenants -## Multi-tenancy +An optional `DomainEvent::tenant_id` (stamped from `AggregateRoot::with_tenant`, +persisted and filterable, omitted from JSON when `None`) is threaded through +`append` / `load` / `stream_all`. One store serves many tenants with per-tenant +isolation on the global stream — the route a multi-bank Lumen deployment would take +to keep each tenant's wallet streams separate. Because the field is omitted from +JSON when `None`, a single-tenant Lumen serialises byte-for-byte identically to the +cross-language wire format. -An optional `DomainEvent::tenant_id` (stamped from -`AggregateRoot::with_tenant`, persisted and filterable, omitted from JSON when -`None`) is threaded through `append` / `load` / `stream_all`, so one store serves -many tenants with per-tenant isolation on the global stream — the route a -multi-bank Lumen deployment would take to keep each tenant's wallet streams -separate. +> **Tip** **Checkpoint.** You can name, for each seam, what it costs you if you +> *don't* use it: no snapshots → slower loads; no outbox → a crash can drop a +> publish; no upcaster → old events become unreadable after a schema change; no +> tenant id → you need one store per tenant. None of them changes the source of +> truth — they are all read-path or delivery concerns layered over the same +> immutable stream. ## Recap — what changed in Lumen @@ -589,25 +846,31 @@ immutable stream, and the stream is the system of record. | Piece | Role | |-------|------| -| `#[derive(DomainEvent)]` | Generates `EVENT_TYPE` + `to_domain_event(...)` for each payload struct | -| `#[derive(AggregateRoot)]` | Generates `AGGREGATE_TYPE` + `aggregate()`/`aggregate_mut()` over the embedded `root` | -| `Wallet::raise` / `apply` | Command applies the event; the same fold runs on write and on replay | -| `Wallet::rehydrate` | Rebuilds a wallet by folding its full stream — empty stream = unopened | +| `#[derive(DomainEvent)]` | Generates `EVENT_TYPE` + `event_type()` + `to_domain_event(...)` for each payload struct | +| `#[derive(AggregateRoot)]` | Generates `AGGREGATE_TYPE` + `aggregate()` / `aggregate_mut()` over the embedded `root` | +| `Wallet` command (`deposit` / `withdraw`) | Validate the invariant, `raise` the event, apply to state | +| `Wallet::apply` / `rehydrate` | The same fold runs on write and on replay — an empty stream is "unopened" | | `EventStore` / `MemoryEventStore` | The append-only log; `SqlEventStore` for production | | `append(id, expected_version, …)` | Optimistic concurrency — the rehydrated version is the token | +| `EventSourcedRepository` | Ties load (snapshot + replay) and save (append + snapshot policy) together | | `ProjectionRunner` | Rebuilds read models from history (the store-side sibling of the EDA listener) | | `TransactionalOutbox` | Closes the append-then-publish gap with at-least-once relay | +| `EventUpcaster` / `tenant_id` | Schema evolution on read; per-tenant isolation across one store | -Three ideas carry forward. **The events are the truth** — there is no balance -column to drift. **Write and replay share one fold** — `apply` runs the same way -whether a command just raised the event or a load is rebuilding from history, -which is the correctness guarantee. **Depend on the `EventStore` port** — the -in-memory store becomes SQL with a one-line swap, just as the broker became Kafka. +Three ideas carry forward: + +- **The events are the truth.** There is no balance column to drift; the balance + is folded from the stream on every load. +- **Write and replay share one fold.** `apply` runs the same way whether a command + just raised the event or a load is rebuilding from history — and replay never + re-validates, because every stored event already passed its invariant. That + symmetry is the correctness guarantee. +- **Depend on the `EventStore` port.** The in-memory store becomes SQL with a + one-line bean swap, just as the broker became Kafka — the domain never changes. When a business process spans multiple aggregates and needs compensation — moving -money from one wallet to another, atomically — folding a single stream is no -longer enough. Continue to [Sagas, Workflows & TCC](./12-sagas.md), where the -transfer saga drives two wallets and rolls the debit back when the credit fails. +money from one wallet to another, atomically — folding a single stream is no longer +enough. That is the next chapter. ## Exercises @@ -619,13 +882,13 @@ transfer saga drives two wallets and rolls the debit back when the credit fails. 2. **Prove the overdraft guard raises no event.** Open a wallet with 100 cents, attempt to `withdraw` 101, and assert it errors with - `DomainError::InsufficientFunds`. Then call `root.uncommitted()` and assert the - buffer still holds exactly one event (the `WalletOpened`) — the failed command - left the stream clean. + `DomainError::InsufficientFunds`. Then call `wallet.root.uncommitted()` and + assert the buffer still holds exactly one event (the `WalletOpened`) — the + failed command left the stream clean. 3. **Force an optimistic-concurrency conflict.** Append the open event for a - wallet at `expected_version = 0`. Then, without reloading, raise a second - event and append it *also* at `expected_version = 0`. Assert the second append + wallet at `expected_version = 0`. Then, without reloading, raise a second event + and append it *also* at `expected_version = 0`. Assert the second append returns `EventSourcingError::Concurrency`, and explain why a fresh load (which advances `expected` to 1) would have succeeded. @@ -634,3 +897,19 @@ transfer saga drives two wallets and rolls the debit back when the credit fails. `replay` one wallet's stream through it, and assert the count. Then clear the map and replay again — confirming the read model is rebuildable from the store alone, with no live event traffic. + +5. **Swap the store (on paper).** Read the `event_store` `#[bean]` in + `LumenBeans`, then write the one-line change that would return a + `SqlEventStore::new(db)` instead of a `MemoryEventStore::new()`. Note that no + command, no `apply`, and no `rehydrate` would change — only the bean. That is + the payoff of depending on the `EventStore` port. + +## Where to go next + +- Coordinate a process across **two** wallets — debit one, credit the other, and + compensate when the credit fails — in + **[Sagas, Workflows & TCC](./12-sagas.md)**. The transfer saga is built directly + on the overdraft guard and the optimistic-concurrency token from this chapter. +- Revisit how each appended event reaches the projection on the wire in + **[Event-Driven Architecture & Messaging](./10-eda-messaging.md)** — the + transport half of the story this chapter completed. diff --git a/docs/book/src/12-sagas.md b/docs/book/src/12-sagas.md index 908dfe5..ef8d490 100644 --- a/docs/book/src/12-sagas.md +++ b/docs/book/src/12-sagas.md @@ -1,32 +1,86 @@ # Sagas, Workflows & TCC -By the end of this chapter, Lumen can **move money between two wallets** — and -do it *safely*. A transfer is not a single command: it debits one wallet, then +By the end of this chapter Lumen can **move money between two wallets** — and do +it *safely*. A transfer is not a single command: it debits one wallet, then credits another, and those are two independent writes to two independent event streams. If the credit leg fails after the debit already committed, the source owner is out of pocket with nothing on the other side. There is no -`BEGIN … COMMIT` that spans two aggregates, so Lumen reaches for the pattern -that does the job: a **saga** that compensates the debit when the credit fails. - -We build the `POST /api/v1/transfers` endpoint on top of the `Ledger` the CQRS -handlers already use, so a transfer raises *real* `MoneyWithdrawn` / -`MoneyDeposited` events on both streams — and a refund raises a real -`MoneyDeposited` on the source stream. Everything stays observable on the -event-sourced ledger you built in chapter 11. - -`firefly-orchestration` ships the three classic **distributed-transaction -engines** every Firefly platform agrees on. Each composes async steps, runs as a -plain future on the caller's task, applies a per-step retry policy, threads a -typed context blackboard, and respects cooperative cancellation. And — this is -the change since the last edition — you no longer hand-build them as values. -Lumen declares each engine with an attribute macro on an `impl` block, exactly -as it declares CQRS handlers and controllers. - -| Engine | Topology | Compensation | Declared with | -|------------|----------------------------|------------------------------------|------------------------------| -| `Saga` | Dependency-ordered steps | Reverse-order, configurable policy | `#[saga]` + `#[saga_step]` | +`BEGIN … COMMIT` that spans two aggregates, so Lumen reaches for the patterns +that do this job across a distributed boundary: a **saga** that compensates the +debit when the credit fails, a **workflow** that runs pre-flight checks in +parallel, and a **TCC** coordinator that reserves on both sides before +committing either. + +You build all three on top of the event-sourced `Ledger` you grew in +[Event Sourcing](./11-event-sourcing.md), so a transfer raises *real* +`MoneyWithdrawn` / `MoneyDeposited` events on both streams — and a refund raises +a real `MoneyDeposited` on the source stream. Nothing here is a toy; every leg +drives the same application service the CQRS handlers use, and every outcome is +observable on the ledger. + +By the end of this chapter you will: + +- Explain *why* a money transfer across two aggregates needs a saga, not a + database transaction, and what a *compensation* is. +- Declare a `Saga` with `#[firefly::saga]` and `#[saga_step]` — including + `depends_on` ordering, a named `compensate` method, and per-step retry — then + run it and read its `Outcome`. +- Declare a `Workflow` with `#[firefly::workflow]` that runs independent checks + in a parallel layer and joins their typed verdicts in a decision node. +- Declare a TCC coordinator with `#[firefly::tcc]` and `#[participant]` to + reserve-then-confirm across two resources. +- Mount all three on Lumen's web surface, rendering a clean rollback as an + RFC 9457 `422` problem instead of a `500`. +- Choose the right engine for a given process, and recognise the + eventual-consistency trade-off each one makes. + +## Concepts you will meet + +Before the first line of code, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — saga.** A *saga* is a sequence of local transactions +> where each step has a *compensating* action that semantically undoes it. If a +> later step fails, the engine runs the completed steps' compensations in reverse +> order. It is how you get "all-or-nothing" across services that cannot share one +> database transaction. The Java analog is the `@Saga` / `@SagaStep` pattern; +> pyfly spells it with saga decorators. + +> **Note** **Key term — compensation.** A *compensation* is not a database +> rollback — it is a *semantic undo*. "Re-credit the source" is a brand-new +> `deposit` that restores the balance and leaves an auditable refund event +> behind; it does not erase history, it appends a correcting fact. + +> **Note** **Key term — workflow (DAG).** A *workflow* is a directed acyclic +> graph of steps. Steps with no dependency between them run concurrently in the +> same *topological layer*; a step that declares `depends_on` waits for its +> predecessors. Use it when a process has independent branches that should run in +> parallel and then join. + +> **Note** **Key term — TCC (Try-Confirm-Cancel).** *TCC* is a two-phase +> protocol: **Try** every participant (reserve resources), then **Confirm** all +> on success; on any Try failure, **Cancel** the participants already tried. +> Where a saga applies each leg immediately and undoes it later, TCC reserves +> first and only commits once every reservation succeeded. + +> **Note** **Key term — eventual consistency.** Operating across independent +> aggregates without a distributed lock means there is a window where one leg has +> committed and another has not. These engines guarantee consistency *in the end* +> — all legs committed, or all compensated — not at every instant. + +`firefly-orchestration` ships the three classic distributed-transaction engines +every Firefly platform agrees on. Each composes async steps, runs as a plain +future on the caller's task, applies a per-step retry policy, threads a typed +context blackboard, and respects cooperative cancellation. And — this is the key +property — you do not hand-build them as values. Lumen declares each engine with +an attribute macro on an `impl` block, exactly as it declares CQRS handlers and +controllers. + +| Engine | Topology | Compensation | Declared with | +|------------|----------------------------|------------------------------------|------------------------------------| +| `Saga` | Dependency-ordered steps | Reverse-order, configurable policy | `#[saga]` + `#[saga_step]` | | `Workflow` | DAG with parallel layers | Reverse-order, configurable policy | `#[workflow]` + `#[workflow_step]` | -| `Tcc` | Try-all then Confirm-all | Cancel-tried on a Try failure | `#[tcc]` + `#[participant]` | +| `Tcc` | Try-all then Confirm-all | Cancel-tried on a Try failure | `#[tcc]` + `#[participant]` | > **Design note.** Firefly's orchestration model is *declarative*. You write an > ordinary `impl` block of `async fn(&self, …) -> Result` methods and @@ -34,11 +88,11 @@ as it declares CQRS handlers and controllers. > node, `#[participant]` for a TCC actor. The macro lowers those methods onto the > same `firefly-orchestration` engines — `depends_on` orders them, `compensate` > names the undo, a step's `Ok(T)` is published for later steps, and an `Err(E)` -> triggers compensation in reverse order. If you've used Java's `@Saga` or +> triggers compensation in reverse order. If you have used Java's `@Saga` or > pyfly's saga decorators, this is the Rust spelling: the control flow lives in > methods you can read top to bottom, and the wiring is generated for you. -## The problem with distributed writes +## Step 1 — Understand the problem with distributed writes Make the failure modes concrete before writing a line of code. A Lumen transfer has two legs: @@ -54,9 +108,8 @@ balances inconsistent. The principled answer is **eventual consistency with explicit compensation**. Each leg commits to its own stream independently, and you design a recovery path -— a *compensating transaction* — for every step that could succeed before a -later one fails. A compensation is not a database rollback; it is a *semantic -undo*: "re-credit the source" is a brand-new `deposit` that restores the +— a compensating transaction — for every step that could succeed before a later +one fails. "Re-credit the source" is a brand-new `deposit` that restores the balance, and it leaves an auditable refund event behind.
@@ -84,93 +137,37 @@ balance, and it leaves an auditable refund event behind.
The transfer saga: debit then credit. A failed credit runs the debit's compensation in reverse, refunding the source.
-## Saga — dependency-ordered steps with compensation +What just happened: you named the two writes, saw why neither retry nor +skip-the-failure is safe, and settled on the saga shape — debit, then credit, +with a refund waiting if the credit ever fails. The rest of the chapter turns +that shape into code. -A `Saga` runs steps in dependency order; on any step failure it compensates the -completed steps in reverse order. You declare it with `#[firefly::saga]` on an -`impl` block, marking each leg with `#[saga_step]`: +> **Tip** **Checkpoint.** You can state, in one sentence each, why a money +> transfer cannot be a single database transaction and what a compensation does +> that a rollback does not. If both are clear, you are ready to declare the saga. -```rust -#[firefly::saga(name = "money-transfer", policy = "stop_on_error")] -impl TransferSaga { - #[saga_step(id = "reserve", compensate = "refund")] - async fn reserve(&self, #[input] req: TransferReq) -> Result { /* … */ } - - async fn refund(&self, #[from_step("reserve")] r: Reserved) -> Result<(), MyErr> { /* … */ } - - #[saga_step(id = "credit", depends_on = ["reserve"], retry = 3, backoff_ms = 100)] - async fn credit(&self, #[from_step("reserve")] r: Reserved) -> Result<(), MyErr> { /* … */ } -} -``` - -The macro generates two methods on the type: - -- `TransferSaga::saga(self: Arc) -> Saga` — builds the engine from your - steps, their `depends_on` order, compensations, and retry policies. -- `TransferSaga::run(self: Arc, input) -> Result` — - serialises `input` into the step context and runs the whole DAG, compensating - on failure. +## Step 2 — Declare the wire types -Each step is an `async fn(&self, …) -> Result`. Its **parameters are -injected** from the saga context with markers the macro reads and strips: - -- `#[input]` (the whole input) or `#[input("field")]` (one field of it); -- `#[from_step("id")]` — the `Ok` value an earlier step published; -- `#[variable("key")]` — a saga-scoped context variable; -- `#[ctx]` — the `StepContext` blackboard itself. - -A step's `Ok(T)` is serialised and made available to later steps via -`#[from_step]`; an `Err(E)` (where `E: std::error::Error + Send + Sync`) -triggers compensation in reverse order. `#[saga_step]` accepts `id` (required), -`depends_on = ["…"]`, `compensate = "method"`, and the per-step recovery knobs -`retry`, `backoff_ms`, `timeout_ms`, and `jitter`. `#[saga(...)]` accepts a -`name`, a `crate` facade override, and a compensation `policy`: - -- **`best_effort`** (the engine default) — log and continue compensating the - remaining steps even if one compensation fails. -- **`stop_on_error`** — abort rollback at the first compensation failure and - surface a `SagaError::Compensation` wrapping the original. -- plus `retry_with_backoff`, `circuit_breaker`, `best_effort_parallel`, and - `grouped_parallel` for larger fan-outs. - -> **Spring parity.** This is the same shape as Java's `@Saga` / `@SagaStep` and -> pyfly's saga decorators — a step method, a compensation named by string, a -> `depends_on` order — but lowered onto Rust's type system, so a step that -> returns the wrong type or names a compensation that does not exist is a -> *compile error*, not a runtime surprise. - -> **Note** — the macro lowers onto a lower-level programmatic builder. If you -> ever need to construct a saga dynamically (steps known only at run time), the -> same engine exposes `Saga::new(name).step(Step::with_context(id, action) -> .with_context_compensation(undo))`. The declarative `#[saga]` form is the way -> you'll write them in practice; the builder is the seam it expands onto. - -## Lumen's transfer saga - -Lumen's transfer is a two-step saga: `debit` the source, then `credit` the -destination. The debit's compensation refunds the source; the credit is the last -leg, so it needs no compensation — a failure there rolls back only the debit. The -whole thing lives in `src/transfer.rs`, and every leg drives the same `Ledger` -the CQRS handlers use. - -First, the wire types — the request body and the result `POST /api/v1/transfers` -returns: +Lumen's transfer lives in `src/transfer.rs`. Start with the types that cross the +HTTP boundary: the request body and the result `POST /api/v1/transfers` returns. ```rust use serde::{Deserialize, Serialize}; /// `POST /api/v1/transfers` command — move `amount` (cents) from `from` to `to`. -#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, firefly::Schema)] #[serde(default)] pub struct TransferRequest { + /// The source wallet id (debited). pub from: String, + /// The destination wallet id (credited). pub to: String, /// The amount to move, in minor units (cents); must be `> 0`. pub amount: i64, } /// The result of a completed (or compensated) transfer. -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, firefly::Schema)] pub struct TransferResult { /// `"completed"` when both legs succeeded — the lowercase `SagaStatus`. pub status: String, @@ -184,10 +181,19 @@ pub struct TransferResult { } ``` -The result echoes the `SagaStatus` as a lowercase string plus the two step -lists, so the API tells the caller *exactly* what the engine did — which steps -ran and which were rolled back. A transfer also has a typed error that -distinguishes a malformed request from a clean, compensated business failure: +What just happened: `TransferRequest` carries the two wallet ids and an amount in +minor units (cents). `TransferResult` echoes the saga's status as a lowercase +string plus the two step lists, so the API tells the caller *exactly* what the +engine did — which steps ran and which were rolled back. + +> **Note** **Key term — `firefly::Schema`.** The `Schema` derive teaches the +> auto-generated OpenAPI docs (served on the management port) what this DTO looks +> like. It is the Rust analog of springdoc's model reflection, computed at +> compile time. You met the management-port docs in +> [Quickstart](./02-quickstart.md); every DTO that crosses the wire derives it. + +A transfer also needs a typed error that distinguishes a malformed request from a +clean, compensated business failure: ```rust /// The typed error a transfer surfaces to its caller. @@ -212,18 +218,23 @@ impl std::fmt::Display for TransferError { impl std::error::Error for TransferError {} ``` -> **One dependency, even here.** `TransferError` hand-writes `Display` + -> `std::error::Error` instead of pulling in `thiserror` — the same discipline -> the rest of Lumen keeps, so the whole framework and every typed error still -> arrive through the single `firefly` facade. +What just happened: `Invalid` is a bad request (a `422` that never touched the +ledger); `Compensated` is a business failure that ran, rolled back cleanly, and +carries the failing leg's cause. Keeping them as distinct variants lets the +endpoint map each to the right HTTP status. -### Declaring the saga +> **Note** Lumen hand-writes `Display` + `std::error::Error` instead of pulling +> in `thiserror`. That is the same one-dependency discipline the rest of the book +> keeps: the whole framework and every typed error still arrive through the +> single `firefly` facade, with no extra crate to align. -The saga is an `impl` block on a tiny struct that holds the `Ledger`. Each leg -is an annotated method that calls the ledger directly and returns a typed -`Result<(), DomainError>`. There is no closure capture, no `Mutex` to smuggle -the cause out of an erased error channel, no builder call — the macro reads the -attributes and generates all of it: +## Step 3 — Declare the saga + +The saga is an `impl` block on a tiny struct that holds the `Ledger`. Each leg is +an annotated method that calls the ledger directly and returns a typed +`Result<(), DomainError>`. There is no closure capture, no `Mutex` to smuggle the +cause out of an erased error channel, no builder call — the macro reads the +attributes and generates all of it. ```rust use std::sync::Arc; @@ -268,20 +279,77 @@ impl TransferSaga { } ``` -**How it reads.** `debit` is the first step (`id = "debit"`), and it names its -undo with `compensate = "refund_debit"`. `refund_debit` carries *no* -`#[saga_step]` marker — it is a plain method referenced by name, and the macro -includes it in the generated saga only because `debit` points at it. `credit` -declares `depends_on = ["debit"]`, so the engine runs it strictly after the -debit; it has no compensation because it is the last leg, so the only thing to -undo on its failure is the debit, which the engine handles automatically. +How it reads, block by block: + +- `debit` is the first step (`id = "debit"`), and it names its undo with + `compensate = "refund_debit"`. +- `refund_debit` carries *no* `#[saga_step]` marker — it is a plain method + referenced by name, and the macro includes it in the generated saga only + because `debit` points at it. +- `credit` declares `depends_on = ["debit"]`, so the engine runs it strictly + after the debit. It has no compensation because it is the last leg: the only + thing to undo on its failure is the debit, which the engine handles + automatically. + +Each leg takes `#[input] req: TransferRequest`. That marker is the heart of the +model. + +> **Note** **Key term — parameter injection.** Each step's parameters are +> *injected* from the saga context by markers the macro reads and strips: +> `#[input]` is the whole input (or `#[input("field")]` for one field); +> `#[from_step("id")]` is the `Ok` value an earlier step published; +> `#[variable("key")]` is a saga-scoped context variable; and `#[ctx]` is the +> `StepContext` blackboard itself. Because every step here needs the whole +> request, every parameter is `#[input] req: TransferRequest`. + +A step's `Ok(T)` is serialised and made available to later steps via +`#[from_step]`; an `Err(E)` (where `E: std::error::Error + Send + Sync`) triggers +compensation in reverse order. Because the methods return `DomainError` directly, +the typed failure cause is preserved all the way through the engine — no shared +`Mutex` needed. + +The `#[saga_step]` attribute accepts `id` (required), `depends_on = ["…"]`, +`compensate = "method"`, and the per-step recovery knobs `retry`, `backoff_ms`, +`timeout_ms`, and `jitter`. The `#[saga(...)]` attribute accepts a `name`, a +`crate` facade override, and a compensation `policy`: + +- **`best_effort`** (the engine default) — log and continue compensating the + remaining steps even if one compensation fails. +- **`stop_on_error`** — abort rollback at the first compensation failure and + surface a `SagaError::Compensation` wrapping the original. +- plus `retry_with_backoff`, `circuit_breaker`, `best_effort_parallel`, and + `grouped_parallel` for larger fan-outs. + +> **Note** This is the same shape as Java's `@Saga` / `@SagaStep` and pyfly's +> saga decorators — a step method, a compensation named by string, a `depends_on` +> order — but lowered onto Rust's type system. A step that returns the wrong type +> or names a compensation that does not exist is a *compile error*, not a runtime +> surprise. + +> **Design note.** Nothing here is reflection or runtime scanning. `#[saga]` +> expands at compile time into the exact `Saga::new(...).step(...)` calls you +> would otherwise write by hand, threaded through the `firefly` facade's `__rt` +> contract so a one-dependency service compiles it without ever naming +> `firefly-orchestration`. If you ever need to construct a saga dynamically +> (steps known only at run time), the same engine exposes the programmatic seam +> the macro lowers onto: +> `Saga::new(name).step(Step::with_context(id, action).with_context_compensation(undo))`. + +> **Tip** **Checkpoint.** You have a `TransferSaga` struct, a `#[firefly::saga]` +> `impl` with two `#[saga_step]` legs and one named compensation. `cargo build` +> should compile it — and if you rename `refund_debit` without updating the +> `compensate = "…"` string, the build should fail with a message pointing at the +> offending line. Try it, then change it back. -Each leg takes `#[input] req: TransferRequest`, so the macro deserialises the -saga's input (the request) into that parameter on every step and compensation. -Because the methods return `DomainError` directly, the typed failure cause is -preserved all the way through the engine — no shared `Mutex` needed. +## Step 4 — Run the saga -### Running the saga +The macro generates two methods on the type: + +- `TransferSaga::saga(self: Arc) -> Saga` — builds the engine from your + steps, their `depends_on` order, compensations, and retry policies. +- `TransferSaga::run(self: Arc, input) -> Result` — + serialises `input` into a fresh step context and runs the whole DAG, + compensating on failure. `run_transfer` validates the request, constructs the saga value behind an `Arc`, and calls the generated `run`. On success it reads the `Outcome`; on failure it @@ -327,35 +395,49 @@ pub async fn run_transfer( } ``` -**What the generated `run` does for you.** `saga.run(req.clone())` is the macro's -`TransferSaga::run(self: Arc, input)`: it serialises `req` into a fresh -`StepContext`, builds the saga (`debit` → `credit`, with the debit's -compensation attached), and runs the DAG. On the happy path it returns an -`Outcome` whose `status` is `Completed`, `steps_executed` lists the legs that -ran, and `steps_rolled` is empty. On failure it returns a `SagaFailure`: its -`outcome()` is fully populated (status `Compensated`, with `steps_rolled` naming -the compensations that ran), and its `error()` is a `SagaError`. We match -`SagaError::Step { source, .. }` to recover the leg's `DomainError` message — -that is how `POST /api/v1/transfers` answers `insufficient funds` instead of an -opaque "step \"credit\" failed". - -> **Design note.** Nothing here is reflection or runtime scanning. `#[saga]` -> expands at compile time into the exact `Saga::new(...).step(...)` calls you -> would otherwise write by hand, threaded through the `firefly` facade's `__rt` -> contract so a one-dependency service compiles it without ever naming -> `firefly-orchestration`. A step that returns the wrong type, or a `compensate` -> that points at a method that doesn't exist, fails the build with a message -> pointing at the offending line. - -### The endpoint - -The controller method in `src/web.rs` is thin: drive the saga, then translate -the typed outcome into the HTTP contract. A clean rollback is a *business* -failure, so it surfaces as a `422` problem carrying the cause — not a `500`: +What the generated `run` does for you, line by line: + +- `saga.run(req.clone())` serialises `req` into a fresh `StepContext`, builds the + saga (`debit` → `credit`, with the debit's compensation attached), and runs the + DAG. +- On the happy path it returns an `Outcome` whose `status` is `Completed`, + `steps_executed` lists the legs that ran, and `steps_rolled` is empty. Note the + field is `outcome.steps_rolled` on the engine side; `run_transfer` copies it + into the wire field `steps_rolled_back`. +- On failure it returns a `SagaFailure`: its `outcome()` is fully populated + (status `Compensated`, with `steps_rolled` naming the compensations that ran), + and its `error()` is a `SagaError`. We match `SagaError::Step { source, .. }` + to recover the leg's `DomainError` message — that is how + `POST /api/v1/transfers` answers `insufficient funds` instead of an opaque + `step "credit" failed`. + +> **Note** **Key term — `Outcome` / `SagaFailure`.** `Outcome` is the saga's +> terminal record: `status` (a `SagaStatus` that displays lowercase — +> `completed` / `compensated` / `failed`), `steps_executed`, and `steps_rolled`. +> `SagaFailure` is the failure pair — `outcome()` gives the same record, and +> `error()` gives the typed `SagaError` that ended the run. There is no separate +> "did it roll back?" flag to consult; the outcome tells you everything. + +> **Tip** **Checkpoint.** `run_transfer` compiles and its three branches are +> clear: an `Invalid` validation failure, an `Ok(Outcome)` happy path, and a +> `SagaError::Step` failure unwrapped into `TransferError::Compensated`. You are +> ready to mount it. + +## Step 5 — Mount the saga endpoint + +The controller method in `src/web.rs` is thin: drive the saga, then translate the +typed outcome into the HTTP contract. A clean rollback is a *business* failure, +so it surfaces as a `422` problem carrying the cause — not a `500`: ```rust /// `POST /api/v1/transfers` — run a money transfer as a saga. -#[post("/transfers")] +#[post( + "/transfers", + summary = "Transfer funds (saga)", + description = "Moves funds between two wallets as a compensating saga (debit then credit).", + tags = ["Transfers"], + status = 200 +)] async fn transfer( State(api): State, Json(body): Json, @@ -374,16 +456,27 @@ async fn transfer( } ``` -Note the `invalidate_type::()` at the end: a transfer changed two -balances, so the cached `GetWallet` views must be dropped to keep a read after -the write honest. That cache and its invalidation are the subject of -[Caching](./17-caching.md); the transfer is just one more mutation that has to -play by its rules. +What just happened: + +- `run_transfer` returns the typed `TransferError`; the `map_err` translates both + variants into a validation problem. `FireflyError::validation(...)` renders as + an RFC 9457 `422 application/problem+json` document carrying the detail string, + so the caller sees `insufficient funds`, not a stack trace. +- `invalidate_type::()` drops the cached `GetWallet` views, because a + transfer changed two balances and a read after the write must be honest. That + cache and its invalidation are the subject of [Caching](./17-caching.md); the + transfer is just one more mutation that plays by its rules. -### What the saga does on each path +> **Note** This handler lives inside Lumen's `#[rest_controller(path = "...")]` +> `impl WalletApi`, mounted automatically at boot — you never edit `main` to add +> a route. `WebResult` is `Result`, and any `WebError` renders as +> an RFC 9457 problem. You first met both in +> [Your First HTTP API](./06-first-http-api.md). + +## Step 6 — Read the three saga paths The tests in `src/transfer.rs` exercise all three paths, and they are the best -documentation of the behavior. The happy path moves funds and rolls back +documentation of the behaviour. The **happy path** moves funds and rolls back nothing: ```rust @@ -439,26 +532,40 @@ assert_eq!(Wallet::rehydrate(&src.id, &src_events).view().balance, 1_000); assert_eq!(src_events.len(), 3); // open + withdraw + refund-deposit ``` -> **Sagas are eventually consistent.** A saga does not give you serializability. -> Between the moment the source is debited and the moment the credit commits (or -> the refund runs), another request could read the source and see a balance -> lower than it will ultimately be. That is the trade-off for operating across -> independent aggregates without a distributed lock: consistency *in the end* — -> all legs committed, or all compensated — not at every instant. +What just happened: the third assertion is the whole point of compensation as a +*semantic undo*. The balance is restored to `1_000`, but the stream is **not** +two events long as if nothing happened — it is *three* events long: the open, the +withdraw, and the refund deposit. The history of what actually occurred is +preserved and auditable. -## Workflow — a DAG with parallel layers and conditions +> **Note** A saga does not give you serializability. Between the moment the +> source is debited and the moment the credit commits (or the refund runs), +> another request could read the source and see a balance lower than it will +> ultimately be. That is the trade-off for operating across independent +> aggregates without a distributed lock: consistency *in the end* — all legs +> committed, or all compensated — not at every instant. -When a process has *independent* steps that should run in parallel, reach for a -`Workflow`: a DAG of nodes with `depends_on` declarations. Independent nodes run -concurrently within a topological layer, and a node that declares dependencies -runs only after they complete. You declare it with `#[firefly::workflow]` and -mark each node with `#[workflow_step]` — the same parameter injection as a saga. +> **Tip** **Checkpoint.** Run `cargo test -p lumen transfer`. The happy-path, +> overdraft, and credit-failure tests pass, and the credit-failure test confirms +> three events on the source stream. That three-event trail is your proof the +> compensation appended rather than erased. -`#[workflow_step]` accepts `id` (required), `depends_on = ["…"]`, -`compensate = "method"`, `when = "expr"` (a skip condition — the node is skipped -when the predicate is false), and `fire_and_forget` (schedule the node without -blocking the layer). The macro generates `Workflow::workflow(self: Arc)` -and `run(self, input) -> Result<(), WorkflowError>`. +## Step 7 — Add a parallel compliance workflow + +A large transfer should be gated behind compliance checks *before* the money +moves. Those checks are independent of each other — a balance check and a +per-transfer ceiling have nothing to do with one another — so they should run in +parallel. That is a `Workflow`: a DAG of nodes with `depends_on` declarations, +where independent nodes run concurrently within a topological layer and a node +that declares dependencies runs only after they complete. + +You declare it with `#[firefly::workflow]` and mark each node with +`#[workflow_step]` — the same parameter injection as a saga. `#[workflow_step]` +accepts `id` (required), `depends_on = ["…"]`, `compensate = "method"`, +`when = "expr"` (a skip condition — the node is skipped when the predicate is +false), and `fire_and_forget` (schedule the node without blocking the layer). The +macro generates `Workflow::workflow(self: Arc)` and +`run(self, input) -> Result<(), WorkflowError>`.
@@ -486,14 +593,8 @@ and `run(self, input) -> Result<(), WorkflowError>`.
The compliance workflow:
balance-check and limit-check have no dependency on each other, so they run in the same layer; approve waits for both and consumes their verdicts.
-### Lumen's compliance workflow - -A large transfer should be gated behind compliance checks *before* the money -moves. Lumen's `src/compliance.rs` runs two independent checks in parallel and -then an approval gate that consumes both. `balance-check` and `limit-check` have -no dependency on each other, so the engine runs them in the same topological -layer; `approve` declares `depends_on` on both and reads their boolean verdicts -through `#[from_step(...)]`: +Lumen's `src/compliance.rs` runs two independent checks in parallel and then an +approval gate that consumes both. First the error type and the policy input: ```rust use std::sync::Arc; @@ -507,6 +608,33 @@ use crate::transfer::TransferRequest; /// The per-transfer ceiling, in minor units (cents). pub const MAX_TRANSFER_CENTS: i64 = 1_000_000; // 10,000.00 +/// Why a transfer failed compliance. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ComplianceError { + /// The source wallet does not exist, so its balance cannot be checked. + NotFound(String), + /// A check failed — the transfer is not allowed (the string says why). + Rejected(String), +} + +impl std::fmt::Display for ComplianceError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + ComplianceError::NotFound(id) => write!(f, "source wallet {id} not found"), + ComplianceError::Rejected(why) => write!(f, "transfer rejected: {why}"), + } + } +} + +impl std::error::Error for ComplianceError {} +``` + +Now the workflow itself. `balance-check` and `limit-check` have no dependency on +each other, so the engine runs them in the same topological layer; `approve` +declares `depends_on` on both and reads their boolean verdicts through +`#[from_step(...)]`: + +```rust /// The compliance workflow: each node drives the `Ledger` or a policy input. struct ComplianceCheck { ledger: Ledger, @@ -560,12 +688,21 @@ impl ComplianceCheck { } ``` -This is the payoff of the injection model. `balance_check` returns `Ok(true)` or -`Ok(false)`; the engine serialises that `bool` under the node id `balance-check`. -`approve` declares `#[from_step("balance-check")] funds_ok: bool`, and the macro -deserialises the stored value back into that parameter — typed both ends, with -no manual context plumbing. `balance-check` reads the *real* source aggregate -from the `Ledger`; only the per-transfer ceiling is a new policy input. +What just happened — and why it matters: this is the payoff of the injection +model. `balance_check` returns `Ok(true)` or `Ok(false)`, and the engine +serialises that `bool` under the node id `balance-check`. `approve` declares +`#[from_step("balance-check")] funds_ok: bool`, and the macro deserialises the +stored value back into that parameter — typed at both ends, with no manual +context plumbing. `balance-check` reads the *real* source aggregate from the +`Ledger`; only the per-transfer ceiling is a new policy input. + +> **Tip** **Checkpoint.** Notice the difference in topology from the saga: the +> saga's `credit` declares `depends_on = ["debit"]` so the two run *in series*; +> the workflow's `balance-check` and `limit-check` declare *no* dependency on each +> other, so they run *in the same layer*. Only `approve` waits. That single +> `depends_on` difference is the difference between a chain and a DAG. + +## Step 8 — Run the workflow and recover the cause `run_compliance` builds the workflow behind an `Arc` and calls the generated `run`. `Ok(())` means approved; an `Err` is recovered into a typed @@ -609,6 +746,12 @@ fn compliance_cause(failure: WorkflowError) -> ComplianceError { } ``` +What just happened: `WorkflowError::Node` boxes the failing node's error as a +`source`. `compliance_cause` first tries `downcast_ref::()` to +recover the exact typed variant; if that succeeds it returns the original error +verbatim. The fall-through string-matching is a belt-and-suspenders path for when +the boxed type cannot be downcast. + The endpoint in `src/web.rs` is a read-only pre-check that never moves funds — `200 OK` with the decision when approved, `404` when the source wallet is unknown, and `422` carrying the reason when a compliance check rejects: @@ -616,7 +759,13 @@ unknown, and `422` carrying the reason when a compliance check rejects: ```rust /// `POST /api/v1/transfers/compliance` — gate a transfer through the parallel /// compliance workflow (balance + limit checks → approve). -#[post("/transfers/compliance")] +#[post( + "/transfers/compliance", + summary = "Compliance-gated transfer (workflow)", + description = "Runs the parallel compliance workflow (balance + limit checks) before approving a transfer.", + tags = ["Transfers"], + status = 200 +)] async fn transfer_compliance( State(api): State, Json(body): Json, @@ -636,20 +785,29 @@ async fn transfer_compliance( } ``` -> **Where Lumen would grow this.** The experience-tier starter, -> `firefly-starter-experience`, builds on exactly this workflow engine with -> *signal-driven* steps that park until an external caller delivers a named -> signal, then resume from where they left off. We return to that tier in -> [HTTP Clients](./13-http-clients.md). +What just happened: a missing source maps to `FireflyError::not_found` (a `404` +problem, consistent with `GET /wallets/:id`), and a rejected check maps to +`FireflyError::validation` (a `422` problem). Because the check never moves +funds, there is no cache to invalidate. + +> **Note** The experience-tier starter, `firefly-starter-experience`, builds on +> exactly this workflow engine with *signal-driven* steps that park until an +> external caller delivers a named signal, then resume from where they left off. +> We return to that tier in [HTTP Clients](./13-http-clients.md). -## TCC — reserve all, then confirm all or cancel all +> **Tip** **Checkpoint.** Run `cargo test -p lumen compliance`. A funded, +> in-limit transfer is approved; an overdrawn one is `Rejected`; an over-ceiling +> one is `Rejected` with a "ceiling" message; an unknown source is `NotFound`. -`Tcc` runs a two-phase protocol: **Try** every participant (reserve resources), -then **Confirm** all on success; on any Try failure, **Cancel** the participants -already tried, in reverse order. Where a saga applies each leg immediately and -*undoes* a committed leg on failure, TCC reserves first and only commits once -every reservation succeeded — so a failed reservation is cancelled, never -compensated after the fact. +## Step 9 — Reframe the transfer as TCC + +The same transfer can be modelled a second way — reserve-then-capture — and Lumen +ships both so you can compare them. `Tcc` runs a two-phase protocol: **Try** every +participant (reserve resources), then **Confirm** all on success; on any Try +failure, **Cancel** the participants already tried, in reverse order. Where a +saga applies each leg immediately and undoes a committed leg on failure, TCC +reserves first and only commits once every reservation succeeded — so a failed +reservation is cancelled, never compensated after the fact. You declare it with `#[firefly::tcc]` and mark each *try* method with `#[participant(name, confirm, cancel)]`. The confirm and cancel methods are plain @@ -695,8 +853,6 @@ and `run(self, input) -> Result<(), TccError>`.
The two-phase transfer: Try holds on the source and verifies the destination; Confirm captures on the destination; a failed Try cancels by releasing the source hold.
-### Lumen's two-phase transfer - Lumen's `src/tcc_transfer.rs` models the transfer as a reserve-then-capture. The source's try *holds* the funds by debiting now; its confirm is a no-op (the debit already captured), and its cancel releases the hold with a refund. The @@ -707,12 +863,23 @@ confirm captures by crediting: use std::sync::Arc; use firefly::orchestration::TccError; +use serde::{Deserialize, Serialize}; use crate::domain::DomainError; use crate::ledger::Ledger; use crate::money::Money; use crate::transfer::{TransferError, TransferRequest}; +/// The wire result of a confirmed two-phase transfer. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, firefly::Schema)] +pub struct TccTransferResult { + /// `"confirmed"` when both participants captured. + pub status: String, + pub from: String, + pub to: String, + pub amount: i64, +} + /// The two-phase transfer coordinator: each participant drives the `Ledger`. struct TwoPhaseTransfer { ledger: Ledger, @@ -758,11 +925,19 @@ impl TwoPhaseTransfer { } ``` -The `source` participant names all three phases — `confirm = "capture_source"`, -`cancel = "release_source"` — while `dest` omits `cancel` because its try holds -nothing. The `capture_source` confirm takes only `&self`: a participant method -with no injected parameters is valid, and a no-op confirm is exactly the right -shape when the try already captured. +How it reads: the `source` participant names all three phases — +`confirm = "capture_source"`, `cancel = "release_source"` — while `dest` omits +`cancel` because its try holds nothing. The `capture_source` confirm takes only +`&self`: a participant method with no injected parameters is valid, and a no-op +confirm is exactly the right shape when the try already captured. + +> **Tip** **Checkpoint.** Compare the source and destination participants. The +> source's try *commits a side-effect* (the withdraw), so it needs a real cancel +> that refunds. The destination's try only *reads* (verifies existence), so it +> holds nothing and needs no cancel. The asymmetry is intentional and is exactly +> why TCC lets you skip a cancel when there is nothing to release. + +## Step 10 — Run the TCC and mount it `run_tcc_transfer` builds the coordinator behind an `Arc` and runs it. On success both sides captured (`status: "confirmed"`); on any reservation failure the tried @@ -811,12 +986,40 @@ fn tcc_cause(err: TccError) -> String { } ``` -`TccError::Try { source, .. }` carries the reservation that failed (e.g. an -overdrawn source or a missing destination); `TccError::Confirm(errors)` collects -the confirm-phase failures if a capture somehow fails after every reservation -succeeded. The endpoint mirrors the saga's: `200 OK` with the confirmed result, +What just happened: `TccError::Try { source, .. }` carries the reservation that +failed (e.g. an overdrawn source or a missing destination); `tcc_cause` renders +its message. `TccError::Confirm(errors)` collects the confirm-phase failures if a +capture somehow fails after every reservation succeeded — its messages are joined +with `; `. The endpoint mirrors the saga's: `200 OK` with the confirmed result, or `422` when a reservation failed and the source hold was released. +```rust +/// `POST /api/v1/transfers/2pc` — run a two-phase (Try/Confirm/Cancel) transfer +/// via the TCC coordinator. +#[post( + "/transfers/2pc", + summary = "Two-phase transfer (TCC)", + description = "Runs a Try/Confirm/Cancel two-phase transfer via the TCC coordinator.", + tags = ["Transfers"], + status = 200 +)] +async fn transfer_2pc( + State(api): State, + Json(body): Json, +) -> WebResult> { + let result = run_tcc_transfer(&api.ledger, &body) + .await + .map_err(|e| match e { + TransferError::Invalid(detail) => WebError::from(FireflyError::validation(detail)), + TransferError::Compensated(detail) => { + WebError::from(FireflyError::validation(detail)) + } + })?; + api.query_cache.invalidate_type::(); + Ok(Json(result)) +} +``` + The tests pin the two-phase semantics. A transfer to a missing destination *holds then releases* the source, leaving it untouched: @@ -833,36 +1036,40 @@ assert!(matches!(err, TransferError::Compensated(_))); assert_eq!(balance(&ledger, &src.id).await, 1_000); ``` -> **TCC vs. Saga.** Lumen ships *both* framings of the same transfer so you can -> compare them. The saga applies each leg locally and refunds the debit if the -> credit fails — simplest when an undo is itself a clean local action. The TCC -> reserves on both sides, then commits or releases together — better when a -> participant can cheaply *hold* a reservation and you want all-or-nothing -> semantics with no window where one side has committed and the other has not. - -## Cancellation - -All three engines respect a `CancellationToken` for cooperative cancellation. -The engines depend only on `futures`, so any executor (Tokio included) drives -them. When you need it, the lower-level builder API exposes -`run_cancellable(&token)`; cancel the token from a timeout or a shutdown signal -to drain the run. - -## Choosing an engine - -| Need | Engine | -|-----------------------------------------------|------------| -| Dependency-ordered process, undo on failure | `Saga` | -| Parallel branches that join | `Workflow` | -| Reserve-then-commit across resources | `Tcc` | +> **Note** Lumen ships *both* framings of the same transfer so you can compare +> them. The saga applies each leg locally and refunds the debit if the credit +> fails — simplest when an undo is itself a clean local action. The TCC reserves +> on both sides, then commits or releases together — better when a participant can +> cheaply *hold* a reservation and you want all-or-nothing semantics with no +> window where one side has committed and the other has not. + +> **Tip** **Checkpoint.** Run `cargo test -p lumen tcc_transfer`. The success +> test moves funds and reports `confirmed`; the missing-destination test holds +> then releases the source so its balance is back to `1_000`; the short-source +> test aborts before holding anything. + +## Step 11 — Cancellation + +All three engines respect a `CancellationToken` for cooperative cancellation. The +engines depend only on `futures`, so any executor (Tokio included) drives them. +The declarative `run` always honours a token threaded through the context; when +you need to drive it explicitly, the lower-level builder API exposes +`run_cancellable(&token)`. Cancel the token from a timeout or a shutdown signal to +drain the run. + +```rust,ignore +// Sketch — the builder seam `#[saga]` lowers onto, for a run you cancel yourself. +let token = firefly::orchestration::CancellationToken::new(); +let outcome = saga.run_cancellable(&token).await?; +// elsewhere: token.cancel(); // drains the run cooperatively +``` -Lumen's money-transfer is a `Saga` (debit → credit, with a debit refund). Its -pre-flight compliance gate is a `Workflow` (balance + limit checks in parallel, -then approve). And the same transfer reframed as reserve-then-capture is a `Tcc`. -All three are declared the same way — an annotated `impl` block — and all three -are mounted on the web surface. +What just happened: cancellation is *cooperative* — the engine checks the token +before executing the next step, so an in-flight step finishes but no further step +starts. A cancelled run surfaces as `SagaError::Cancelled` (and the equivalents +on `WorkflowError` / `TccError`), not as a step failure. -## What changed in Lumen +## Recap — what changed in Lumen - Lumen now declares its orchestrations with **macros**, not hand-built values. The transfer is a `#[firefly::saga(name = "money-transfer")]` `impl` whose @@ -881,43 +1088,66 @@ are mounted on the web surface. - All three are mounted on the web surface in `src/web.rs`: `POST /api/v1/transfers` (saga), `POST /api/v1/transfers/compliance` (workflow), and `POST /api/v1/transfers/2pc` (TCC) — each rendering a clean - rollback as a `422` business problem and invalidating the `GetWallet` cache + rollback as a `422` RFC 9457 problem and invalidating the `GetWallet` cache when it moves funds. - Because each leg returns its typed `DomainError` / `ComplianceError`, the failing cause is preserved through the engine and recovered from `SagaError::Step`, `WorkflowError::Node`, and `TccError::Try` — no `Mutex` smuggling, no opaque boxed strings. -- The behaviors — happy path, overdraft short-circuit, credit-failure refund +- The behaviours — happy path, overdraft short-circuit, credit-failure refund (three events on the source stream), parallel rejection, and a released TCC hold — are all pinned by tests, so the prose can never drift from the code. -## Exercises +You also now know how to **choose an engine**: -1. **Add a `notify` step to the saga.** Append a third `#[saga_step(id = - "notify", depends_on = ["credit"])]` method to `TransferSaga` that "sends a - receipt" (return `Ok(())` for now). Assert that on the happy path - `steps_executed == ["debit", "credit", "notify"]`, and that when the credit - fails the notify step never runs and only the debit is rolled back. +| Need | Engine | +|-----------------------------------------------|------------| +| Dependency-ordered process, undo on failure | `Saga` | +| Parallel branches that join | `Workflow` | +| Reserve-then-commit across resources | `Tcc` | -2. **Make the debit retry.** Give the `debit` step - `#[saga_step(id = "debit", compensate = "refund_debit", retry = 2, - backoff_ms = 50)]`. Drive a flaky `Ledger` that fails the first withdraw and - succeeds the second, and assert the transfer still completes — proving the - per-step retry recovers a transient failure before compensation is ever - considered. +Lumen's money-transfer is a `Saga` (debit → credit, with a debit refund). Its +pre-flight compliance gate is a `Workflow` (balance + limit checks in parallel, +then approve). And the same transfer reframed as reserve-then-capture is a `Tcc`. +All three are declared the same way — an annotated `impl` block — and all three +are mounted on the web surface. +## Exercises + +1. **Add a `notify` step to the saga.** Append a third + `#[saga_step(id = "notify", depends_on = ["credit"])]` method to + `TransferSaga` that "sends a receipt" (return `Ok(())` for now). Assert that on + the happy path `steps_executed == ["debit", "credit", "notify"]`, and that + when the credit fails the notify step never runs and only the debit is rolled + back. +2. **Make the debit retry.** Give the `debit` step + `#[saga_step(id = "debit", compensate = "refund_debit", retry = 2, backoff_ms = 50)]`. + Drive a flaky `Ledger` that fails the first withdraw and succeeds the second, + and assert the transfer still completes — proving the per-step retry recovers a + transient failure before compensation is ever considered. 3. **Add a KYC node to the workflow.** Add a third independent `#[workflow_step(id = "kyc-check")]` to `ComplianceCheck` that returns a `bool`, and make `approve` `depends_on` all three, reading the new verdict via `#[from_step("kyc-check")]`. Assert that `kyc-check` runs in the same parallel layer as the existing checks and that a failed KYC rejects the transfer. - 4. **Confirm the TCC is all-or-nothing.** Write a test that runs - `run_tcc_transfer` with an overdrawn source and asserts neither balance moved - — the source try aborts before holding anything, so there is nothing to - cancel. Contrast it with the missing-destination test, where the source *is* - held and then released. - -To call the external services these engines coordinate — a Payments processor, -an FX provider — you need an HTTP client. Continue to -[HTTP Clients](./13-http-clients.md). + `run_tcc_transfer` with an overdrawn source and asserts neither balance moved — + the source try aborts before holding anything, so there is nothing to cancel. + Contrast it with the missing-destination test, where the source *is* held and + then released. +5. **Switch the saga's compensation policy.** Change the saga attribute to + `#[firefly::saga(name = "money-transfer", policy = "stop_on_error")]` and read + the docs for `SagaError::Compensation`. Reason about (or test) what the + `run_transfer` error branch would surface if a *compensation* itself failed — + and why `best_effort` is the engine's default. + +## Where to go next + +- To call the external services these engines coordinate — a Payments processor, + an FX provider — you need an HTTP client. Continue to + **[HTTP Clients](./13-http-clients.md)**. +- The transfer endpoints invalidate the `GetWallet` cache on every move; learn how + that read-side cache and its invalidation work in **[Caching](./17-caching.md)**. +- Revisit the event-sourced `Ledger` every leg drives in + **[Event Sourcing](./11-event-sourcing.md)** to see where the + `MoneyWithdrawn` / `MoneyDeposited` events these sagas raise come from. diff --git a/docs/book/src/13-http-clients.md b/docs/book/src/13-http-clients.md index 463b425..922d81b 100644 --- a/docs/book/src/13-http-clients.md +++ b/docs/book/src/13-http-clients.md @@ -1,47 +1,110 @@ # HTTP Clients -By the end of this chapter, you will know how Lumen reaches *out* — how a wallet -service settles a transfer through an external **Payments** processor, and how an -**experience tier** sits in front of Lumen and composes it with its neighbors -into one journey-shaped API. Lumen itself stays deliberately self-contained -(every leg of its transfer drives the in-process `Ledger`), so this chapter is -the "next adapter you would add": a typed outbound client wired into the transfer -flow, plus the BFF that fans out across services. - -When a transfer needs to settle against a real payment rail, the credit leg stops -being a local `Ledger::deposit` and becomes a network call. That call can time -out, fail halfway, or land on an overloaded service — failure modes a local -method call never had. `firefly-client` gives you a typed client for that call -instead of a hand-rolled `reqwest` session threaded with retry and timeout logic. - -The crate ships two HTTP clients that share the same automatics — default -`Accept`/`Content-Type`, correlation-id and W3C trace-context propagation, and -RFC 7807 problem decode into a typed `FireflyError`: +Until now every leg of a Lumen transfer has been a *local* method call: the +credit step is `ledger.deposit(&req.to, ...)`, in-process, infallible except for +the domain rules it enforces. That is deliberate — Lumen is self-contained, and +keeping it that way let the earlier chapters teach domain modelling, CQRS, event +sourcing, and sagas without a network in the way. This chapter is the moment the +network arrives. It is the *next adapter you would add*: when a transfer has to +settle against a real payment rail, the credit leg stops being a local +`Ledger::deposit` and becomes a call to an external **Payments** service — a call +that can time out, fail halfway, or land on an overloaded host, failure modes a +local method never had. + +`firefly-client` gives you a *typed* client for that call instead of a +hand-rolled `reqwest` session threaded with retry and timeout logic. You will +meet three client styles — eager, reactive, and declarative — that all share one +set of automatics, then see how an **experience tier** sits in front of Lumen and +composes it with its neighbours into one journey-shaped API. Everything is +reachable through the one `firefly` facade you have depended on since +[Quickstart](./02-quickstart.md). + +By the end of this chapter you will: + +- Build an eager `RestClient` with `RestBuilder`, call it, and decode an upstream + problem document into a typed error. +- Build the reactive `WebClient` and choose between its `body_to_mono` / + `body_to_flux` / `exchange` terminals — and know why it has *no* baked-in retry. +- Write a declarative `#[http_client]` trait and let the macro generate the + request-issuing implementation, the mirror image of a `#[rest_controller]`. +- Wrap an outbound call in a `CircuitBreaker` so a sick upstream cannot drag + Lumen down with it. +- Understand the experience-tier (BFF) pattern and the strict + `channel → experience → domain → core` dependency direction. + +## Concepts you will meet + +Before the first client, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — HTTP client.** A *client* here is an object your service +> uses to make *outbound* HTTP calls to another service. It is the inverse of a +> controller, which *receives* inbound calls. Firefly ships a typed client so +> the request shape, the response decode, and the error handling are all checked +> by the compiler instead of left to a raw `reqwest` call. + +> **Note** **Key term — RFC 9457 problem document.** A standard JSON error body +> (media type `application/problem+json`) carrying `type`, `title`, `status`, +> and `detail` fields. RFC 9457 is the current standard (it obsoletes RFC 7807). +> Firefly *produces* these from failing handlers and *consumes* them on the +> client side, decoding an upstream problem into a typed `FireflyError` so an +> external failure carries the upstream's status and detail straight through +> Lumen's own error stack. + +> **Note** **Key term — correlation id / trace context.** A *correlation id* is a +> per-request identifier that travels with a request so its log lines and the log +> lines of every service it calls can be stitched together. The W3C *trace +> context* (`traceparent` / `tracestate` headers) does the same for distributed +> tracing. Every Firefly client forwards both automatically, so a request that +> fans out to three upstreams stays one coherent trace. + +> **Note** **Key term — reactive publisher (`Mono` / `Flux`).** A `Mono` is a +> deferred async value that resolves to at most one `T`; a `Flux` is a +> deferred async *stream* of `T`. You met them in +> [The Reactive Model](./05-reactive-model.md). The reactive client returns them +> so an outbound call drops straight into a reactive pipeline. The Spring analog +> is Project Reactor's `Mono` / `Flux`. + +> **Note** **Key term — Backend-for-Frontend (BFF).** A thin server-side +> application that aggregates several domain services into one *journey-shaped* +> API tailored to a particular frontend, instead of making the frontend call +> each service and merge the results itself. Covered in depth in +> [The Experience Tier](./20a-experience-tier.md); introduced here. + +The crate ships its clients behind one front door, `firefly::client`, and the +declarative pieces are also re-exported through `firefly::prelude`. The two HTTP +clients share the same automatics — default `Accept` / `Content-Type`, +correlation-id and W3C trace-context propagation, and RFC 9457 problem decode +into a typed `FireflyError`: - the **eager `RestClient`** (built with `RestBuilder`) — an `async fn` that awaits a `Result`, with a built-in retry budget; -- the **reactive `WebClient`** — whose terminal operators hand back `Mono` / - `Flux`, so an outbound call drops straight into a reactive pipeline. +- the **reactive `WebClient`** (built with `WebClientBuilder`) — whose terminal + operators hand back `Mono` / `Flux`, so an outbound call composes end-to-end + with a reactive pipeline. On top of the `WebClient` sits the **declarative `#[http_client]`** trait — the -Spring `@HttpExchange` analog — covered [below](#the-declarative-client-http_client). +Spring 6 `@HttpExchange` analog — which you write as a trait and let the macro +implement. The crate also ships builders and scaffolds for GraphQL, SOAP, gRPC, +and WebSocket clients, selected by feature so heavy dependencies stay out of +services that do not use them. -The crate also ships scaffolds for SOAP, gRPC, GraphQL, and WebSocket clients. -Everything is reachable through the one `firefly` facade as `firefly::client`. +> **Design note.** Both HTTP clients are *values built with a fluent builder* — +> there is no annotated interface to generate from for the eager and reactive +> surfaces, and no reflection. The resilience decorators (covered near the end) +> wrap the call from the outside rather than being baked in. That keeps each +> client small and keeps retry/circuit-breaking policy a property of the call +> site, not a hidden default. -> **Two clients, one contract.** `RestClient` is the eager `async fn`-and-await -> client with a built-in retry budget; `WebClient` is the reactive one whose -> `body_to_mono` / `body_to_flux` / `exchange` terminals return publishers you -> compose. Both are values built with a fluent builder — no annotated interface -> to generate from, no reflection — and the resilience decorators (covered below) -> wrap the call. +## Step 1 — Build the eager `RestClient` -## The eager `RestClient` +The eager client is the one to reach for when you just want to `await` a result. +You build it with `RestBuilder`, configuring the base URL, default headers, a +per-request timeout, and an attempt budget, then call `request` with a method, a +path, and an optional body. -Build a client with `RestBuilder`, then call `request` with a method, path, and -optional body. The retry budget, timeout, and default headers are configured on -the builder. Here Lumen builds the Payments client it would call from the credit -leg of a transfer: +Here Lumen builds the Payments client it would call from the credit leg of a +transfer: ```rust,no_run use std::time::Duration; @@ -70,7 +133,7 @@ async fn main() { match payments.request::<_, Payment>(Method::POST, "/payments", Some(&req)).await { Ok(payment) => println!("settled {} ({})", payment.id, payment.status), Err(err) => { - // Upstream RFC 7807 problems are decoded into a typed FireflyError. + // Upstream RFC 9457 problems are decoded into a typed FireflyError. if let Some(fe) = err.as_firefly() { eprintln!("payments upstream {}: {}", fe.status, fe.detail); } @@ -79,28 +142,71 @@ async fn main() { } ``` -A non-2xx `application/problem+json` response is decoded into a `FireflyError`, -so an upstream failure carries the upstream's status and detail straight through -Lumen's own error stack. `err.as_firefly()` is the typed accessor that recovers -the upstream's typed problem, and `ClientError` also offers -predicate helpers like `is_not_found()`, `is_unprocessable_entity()`, and -`is_retryable()` so a caller can branch on the failure class without matching -raw status codes. +What just happened, block by block: + +- `RestBuilder::new("https://payments.internal")` primes a builder at a base URL + (trailing slashes are trimmed so `base + path` concatenation stays clean). +- `.with_header("X-Tenant", "lumen")` sets a default header sent on *every* + request this client makes. +- `.with_timeout(Duration::from_secs(5))` caps each attempt at five seconds. +- `.with_retries(3)` sets the *total attempt budget* to three. Note this is the + number of attempts, not extra retries: `1` means one attempt with no retry, and + the client retries only on network errors and `429` / `5xx` statuses, with + exponential backoff (100 ms doubling per attempt, capped at 2 s). +- `.build()` finalises the `RestClient`. +- `payments.request::<_, Payment>(Method::POST, "/payments", Some(&req))` sends + the request: the turbofish names the body type (inferred here) and the response + type `Payment`. It JSON-encodes the body, sets `Content-Type` / + `Accept: application/json`, forwards the correlation id and trace context, and + decodes a 2xx body into `Payment`. + +Why it matters: a non-2xx `application/problem+json` response is decoded into a +`FireflyError`, so an upstream failure carries the upstream's status and detail +straight through Lumen's own error stack. `err.as_firefly()` is the typed +accessor that recovers the upstream's decoded problem. + +> **Tip** **Checkpoint.** You can call `request::<_, T>(method, path, body)` and +> get back a `Result`. On the error path, `err.as_firefly()` +> returns `Some(&FireflyError)` whenever the failure was an upstream HTTP error +> (not a transport / encode / decode failure), and `fe.status` / `fe.detail` +> echo the upstream's problem. + +### Branching on the failure class + +You rarely want to match raw status codes. `ClientError` offers predicate helpers +so a caller can branch on the *class* of failure: -> **Where this plugs into the saga.** In [Sagas](./12-sagas.md) the credit leg -> was `ledger.deposit(&to, amount)`. In a split deployment that becomes -> `payments.request::<_, Payment>(Method::POST, "/settle", …)`. The saga shape -> does not change — it is still a `Step` with the debit's compensation — only -> the *body* of the credit step now does I/O across the network, which is -> exactly why the compensation (refund the debit) matters more than ever. +```rust,ignore +match payments.request::<_, Payment>(Method::POST, "/payments", Some(&req)).await { + Ok(payment) => { /* settled */ } + Err(err) if err.is_unprocessable_entity() => { /* 422 — map onto Lumen's own 422 */ } + Err(err) if err.is_retryable() => { /* 429 / 5xx / transport — worth a retry */ } + Err(err) => { /* everything else */ } +} +``` + +The predicates mirror the way the framework renders problems elsewhere: +`is_validation()` (400), `is_unauthorized()` (401/403), `is_not_found()` (404), +`is_conflict()` (409), `is_unprocessable_entity()` (422), +`is_rate_limited()` (429), `is_server_error()` (5xx), and `is_retryable()` (the +same rule the client applies internally — transport failures, `429`, and any +`5xx`). -### Idempotency on the wire +> **Note** **Where this plugs into the saga.** In [Sagas](./12-sagas.md) the +> credit leg was `ledger.deposit(&req.to, amount)`. In a split deployment that +> becomes `payments.request::<_, Payment>(Method::POST, "/payments", …)`. The +> saga *shape* does not change — it is still a `#[saga_step]` with the debit's +> `compensate = "refund_debit"` — only the *body* of the credit step now does I/O +> across the network, which is exactly why the compensation (refund the debit) +> matters more than ever. + +## Step 2 — Keep the settlement call idempotent A transfer's settlement call must be idempotent: if `POST /payments` times out -and the saga's retry fires it again, Payments must not create two payments. Carry -a stable `Idempotency-Key` — typically the transfer id — so the upstream +and the saga's retry fires it again, Payments must not create *two* payments. +Carry a stable `Idempotency-Key` — typically the transfer id — so the upstream deduplicates a re-delivered request. Set it as a default header on a per-call -builder, or thread it through the request: +builder: ```rust,ignore let payments = RestBuilder::new("https://payments.internal") @@ -109,15 +215,28 @@ let payments = RestBuilder::new("https://payments.internal") .build(); ``` +What just happened: because the key is a default header, *every* attempt this +client makes — including the retries the budget triggers — carries the same key. The deduplication itself is the upstream's job; the client's job is to forward the key *consistently* across retries. -## The reactive `WebClient` +Why it matters: this is the outbound mirror of the inbound idempotency you got +for free in [Quickstart](./02-quickstart.md) — there Lumen *records* an +`Idempotency-Key` and replays the stored response; here Lumen *sends* one so the +service it calls can do the same. + +> **Tip** **Checkpoint.** A retried `POST` carrying a stable `Idempotency-Key` +> reaches the upstream with the *same* key every time. If you set the key per +> attempt instead of per business operation, deduplication breaks — make it a +> default header keyed on the business id (the transfer id), not on the attempt. + +## Step 3 — Build the reactive `WebClient` The reactive client returns `Mono` / `Flux`, so an outbound call drops straight -into a reactive pipeline and composes end-to-end with the -[`NdJson` / `Sse` responders](./05-reactive-model.md) Lumen's streaming endpoint -uses. The fluent chain reads top to bottom — build, address, send, decode: +into a reactive pipeline and composes end-to-end with the `NdJson` / `Sse` +responders ([The Reactive Model](./05-reactive-model.md)) Lumen's streaming +endpoint uses. You build it with `WebClientBuilder`; the fluent request chain +reads top to bottom — build, address, send, decode: ```rust,no_run use firefly::client::WebClientBuilder; @@ -164,15 +283,39 @@ async fn main() { } ``` -The terminal operators: +What just happened, reading one chain at a time: + +- `client.method(Method::POST)` (or the `.get()` / `.post()` / `.put()` / + `.delete()` / `.patch()` shorthands) starts a request; `.uri(...)` sets the + path; `.body(&...)` JSON-encodes a body; `.retrieve()` finalises the request + into a *response spec*. No I/O has happened yet — the request is sent lazily + when the returned publisher is subscribed. +- `.body_to_mono::()` says "decode the whole body as one `Payment`" and + yields a `Mono`. `.block().await` subscribes and waits, returning + `Result, FireflyError>` — the `Result` carries any terminal + error, the `Option` models an empty (`204`) body. +- `.body_to_flux::()` says "decode this streamed body + element-by-element" and yields a `Flux`; `.collect_list()` gathers + it into a `Mono>`. +- `.exchange()` hands back the raw response (status + headers + body) *without* + raising on a non-2xx, as a `Mono`. + +> **Note** **Key term — terminal operator.** A *terminal operator* is the method +> that ends the fluent chain and decides the shape of the result. On a +> `WebClient`'s response spec the three terminals are: +> +> | Operator | Returns | Behavior | +> |-------------------------|---------------------------|------------------------------------------------| +> | `body_to_mono::()` | `Mono` | the whole body decoded as one `T` | +> | `body_to_flux::()` | `Flux` | a streamed NDJSON/SSE body, element-by-element | +> | `exchange()` | `Mono` | the raw status + headers + body, no raise | -| Operator | Returns | Behavior | -|------------------------------|---------------------------|------------------------------------------------| -| `body_to_mono::()` | `Mono` | the whole body decoded as one `T` | -| `body_to_flux::()` | `Flux` | a streamed NDJSON/SSE body, element-by-element | -| `exchange()` | `Mono` | the raw status + headers + body, no raise | +> **Tip** **Checkpoint.** A `WebClient` chain that ends in `.body_to_mono::()` +> gives you a `Mono` you can `.block().await` (yielding +> `Result, FireflyError>`) or compose further. Nothing fires until you +> subscribe — if you build the chain and never block/await it, no request is sent. -### Streaming semantics +## Step 4 — Stream a response with `body_to_flux` `body_to_flux` consumes the byte stream chunk-by-chunk and decodes one element per frame, lazily and with backpressure — a slow downstream throttles the @@ -186,43 +329,76 @@ response `Content-Type`: A malformed element terminates the stream with a decode `FireflyError` — the first error is terminal, the reactive-streams contract Firefly's `Flux` honors. -This is the *consumer* side of the -same wire format Lumen's own `GET /api/v1/wallets/:id/events` endpoint produces: -one service streams the wallet's event log, another reads it back element by -element. -### Inspecting the raw response +Why it matters: this is the *consumer* side of the same wire format Lumen's own +`GET /api/v1/wallets/:id/events` endpoint (the feature-gated `streaming` +endpoint) *produces*. One service streams the wallet's event log; another reads +it back element by element — the exact symmetry the reactive model buys you. -`exchange()` hands back a `WebClientResponse` **without** raising on a non-2xx, -so you can inspect it and decide: +> **Tip** **Checkpoint.** Point a `body_to_flux::()` at an +> `application/x-ndjson` endpoint and `.take(5)` it; only five elements are +> pulled and the upstream stops producing. Point it at a `text/event-stream` +> endpoint and the SSE `data:` frames decode the same way — the content type, not +> the call, picks the decoder. + +## Step 5 — Inspect the raw response with `exchange` + +`exchange()` hands back a `WebClientResponse` *without* raising on a non-2xx, so +you can inspect the status and decide what to do — the right terminal when a +non-2xx is *expected* and should not short-circuit the pipeline: ```rust,ignore let resp = client.get().uri("/health").retrieve().exchange().block().await?.unwrap(); if resp.is_success() { let body: serde_json::Value = resp.body_json()?; } else if let Some(problem) = resp.problem() { - // a decoded RFC 7807 FireflyError, if the body was a problem document + // a decoded RFC 9457 FireflyError, if the body was a problem document } ``` -### No baked-in retry +What just happened: `.exchange().block().await` returns +`Result, FireflyError>`; the `?` unwraps the `Result` +(only a transport-level failure errors here) and `.unwrap()` the `Option`. +`resp.is_success()` tests the 2xx range, `resp.body_json::()` decodes the +buffered body, and `resp.problem()` decodes a non-2xx `application/problem+json` +body into a `FireflyError` (returning `None` for a 2xx). The difference from +`body_to_mono` is the *raise* behaviour: `body_to_mono` turns a non-2xx into the +`Mono`'s terminal `Err`, while `exchange` hands you the raw response to branch on. -> **Retries are composed, not baked in.** Unlike `RestBuilder::with_retries`, -> the `WebClient` has **no** retry budget. Compose retries on the returned -> publisher with `Mono::retry` / `Mono::retry_backoff`, so retry policy stays a -> property of the call site, not the client: -> -> ```rust,ignore -> use firefly::reactive::{Backoff, Mono}; -> use std::time::Duration; -> -> let payment = Mono::retry_backoff( -> || client.get().uri("/payments/p1").retrieve().body_to_mono::(), -> Backoff::new(3, Duration::from_millis(100)), -> ); -> ``` +## Step 6 — Compose retries (the `WebClient` bakes in none) -## The declarative client (`#[http_client]`) +Unlike `RestBuilder::with_retries`, the `WebClient` has **no** retry budget. That +is intentional: retry policy stays a property of the *call site*, not the client. +Compose retries on the returned publisher with `Mono::retry` / +`Mono::retry_backoff`: + +```rust,ignore +use firefly::reactive::{Backoff, Mono}; +use std::time::Duration; + +let payment = Mono::retry_backoff( + || client.get().uri("/payments/p1").retrieve().body_to_mono::(), + Backoff::new(3, Duration::from_millis(100)), +); +``` + +What just happened: `Mono::retry_backoff` takes a *factory closure* (it must +rebuild the request on each attempt, since a subscribed `Mono` is consumed) and a +`Backoff::new(max_retries, base)` schedule. Each failure re-runs the factory after +an exponentially growing delay. `Mono::retry(factory, n)` is the fixed-count +sibling with no backoff. + +Why it matters: the same `WebClient` can be cautious on one endpoint and +aggressive on another, because the policy lives on the call rather than the +client. This mirrors how the reactive model composes `retry` onto a publisher +rather than configuring it once globally. + +> **Tip** **Checkpoint.** A `WebClient` call wrapped in `Mono::retry_backoff` +> retries on its own schedule; the bare `WebClient` never retries. If you find +> yourself wishing `WebClientBuilder` had a `.with_retries`, that is the signal to +> reach for `Mono::retry_backoff` instead. + +## Step 7 — Write a declarative `#[http_client]` trait Writing the call chain by hand is fine for one-off requests, but a *service you call repeatedly* deserves a typed interface. `#[http_client]` is the analog of @@ -230,7 +406,13 @@ Spring 6's `@HttpExchange` (the modern OpenFeign replacement): you write a **trait** of methods carrying the same verb attributes a `#[rest_controller]` uses, and the macro generates a `Impl` that issues the requests over a `WebClient`. It is the mirror image of a controller — same vocabulary, request -issued instead of received. +*issued* instead of *received*. + +> **Note** **Key term — declarative client.** A *declarative client* is an +> interface you *describe* (verbs, paths, arguments) and let the framework +> *implement*, instead of writing the request-issuing code yourself. The macro +> reads the trait and generates the body. The Spring analog is `@HttpExchange` on +> a Java interface (formerly Spring Cloud OpenFeign's `@FeignClient`). ```rust,ignore use firefly::prelude::*; // #[http_client], ClientError, Mono, Flux @@ -266,6 +448,13 @@ pub trait OrdersClient { } ``` +What just happened: the macro emitted the trait (minus the verb / per-arg marker +attributes) plus a concrete `OrdersClientImpl` struct that wraps a `WebClient` and +implements the trait by translating each method's verb, path template, and bound +arguments into a fluent `WebClient` request. The trait-level +`path = "/api/v1/orders"` is joined onto every method path; `name = "orders"` +names the DI bean; `bean` opts into registration (Step 8). + Construct it from a base URL, or inject a tuned `WebClient`: ```rust,ignore @@ -274,49 +463,94 @@ let order = api.get_order("42".into()).await?; // or: OrdersClientImpl::with_client(my_web_client) // shared pool / timeouts ``` -**Path syntax is the framework's `:id`** (the same as `#[rest_controller]`), not +`OrdersClientImpl::new(base_url)` builds a fresh `WebClient` rooted at the URL (and +applies the trait's `accept` / `content_type` defaults, if any). +`OrdersClientImpl::with_client(web_client)` is the DI seam — pass an +already-configured `WebClient` (timeouts, default headers, a shared connection +pool), the analog of Spring's `HttpServiceProxyFactory`. + +### How arguments bind + +Path syntax is the framework's `:id` (the same as `#[rest_controller]`), not Spring's `{id}` — so a controller and its mirror-image client read identically, -and writing `{id}` is a compile error pointing you at `:id`. **Argument binding** -needs no attributes in the common case: an unannotated arg whose name matches a -`:var` segment is the path variable, the lone unannotated non-scalar arg on a -`POST`/`PUT`/`PATCH` is the JSON body, and everything else is a query param. -Override with `#[path]` / `#[query("k")]` / `#[header("X")]` / `#[body]`. Every -`:var` must bind to exactly one argument or the macro refuses to compile, so a -rename surfaces loudly instead of silently dropping the value. - -**Return shapes:** an `async fn` returning `Result` is the -ergonomic default; `Result` works for any `E: From`; a -*non-async* `fn` returning `Mono` / `Flux` hands back the deferred reactive -value directly (a `Flux` defaults to `Accept: application/x-ndjson`); and -`WebClientResponse` is the raw `.exchange()` escape hatch. - -> **Error fidelity.** On an awaited `Result` method every failure -> arrives as `ClientError::Problem` carrying a `FireflyError` with the original -> status and code — so `is_not_found()` / `is_server_error()` / `is_retryable()` -> still classify correctly — rather than the structured `Transport` / `Decode` / -> `Encode` variants, which survive only on the `Mono` / `Flux` return forms -> (where the `FireflyError` terminal *is* the reactive error channel). Match on -> the reactive form when you need byte-exact variants. +and writing `{id}` is a compile error pointing you at `:id`. Argument binding +needs no attributes in the common case: + +- an unannotated argument whose name matches a `:var` segment is the **path + variable** (percent-encoded); +- the lone unannotated non-scalar argument on a `POST` / `PUT` / `PATCH` is the + **JSON body**; +- everything else is a **query param** (`Option` omits itself when `None`; + `Vec` / `&[_]` repeats the key). + +Override any of these with `#[path]` / `#[query("k")]` / `#[header("X")]` / +`#[body]`. Every `:var` must bind to exactly one argument or the macro refuses to +compile, so a rename surfaces loudly instead of silently dropping the value. + +### Return shapes + +An `async fn` returning `Result` is the ergonomic default; +`Result` works for any `E: From`; a *non-async* `fn` returning +`Mono` / `Flux` hands back the deferred reactive value directly (a `Flux` +defaults to `Accept: application/x-ndjson`); and `WebClientResponse` is the raw +`.exchange()` escape hatch. + +> **Note** **Error fidelity.** On an awaited `Result` method every +> failure arrives as `ClientError::Problem` carrying a `FireflyError` with the +> original status and code — so `is_not_found()` / `is_server_error()` / +> `is_retryable()` still classify correctly — rather than the structured +> `Transport` / `Decode` / `Encode` variants. Those structured variants survive +> only on the `Mono` / `Flux` return forms (where the `FireflyError` terminal *is* +> the reactive error channel). Match on the reactive form when you need byte-exact +> variants. + +> **Tip** **Checkpoint.** A trait under `#[http_client]` produces a +> `Impl` you can construct with `::new(url)`. Calling `get_order("42".into())` +> issues `GET /api/v1/orders/42` and decodes the body into `Order`. If you typo a +> `:var` so it binds no argument, the build fails — that is the macro doing its +> job. + +## Step 8 — Autowire the client as a bean With `#[http_client(... bean)]` the generated `OrdersClientImpl` is registered as -a `@Service` and bound to `dyn OrdersClient`, so a collaborator just -`#[autowired] orders: Arc` — the Feign-client autowire payoff, -resolving a shared `WebClient` bean (a named one with `client = "…"`). - -## Composing with resilience +a `@Service`-style bean and bound to `dyn OrdersClient`, so a collaborator just +declares `#[autowired] orders: Arc` and the container resolves +it — the Feign-client autowire payoff you met in +[Dependency Wiring](./04-dependency-wiring.md). Registration pulls a shared +`WebClient` bean from the container (a named one when you write `client = "…"`), +so every declarative client over the same upstream can share one tuned connection +pool. + +What just happened: `bean` ties the declarative client into the same DI graph +that wires Lumen's controllers and handlers. The trait must be object-safe for the +`dyn` bind (the macro checks this up front and adds the `Send + Sync` +supertraits), so a non-object-safe shape fails with a clear message instead of a +downstream `dyn Trait` error. + +> **Tip** **Checkpoint.** A `#[http_client(... bean)]` trait makes +> `Arc` an injectable dependency. Add `#[autowired] orders: +> Arc` to any bean and the container hands you the generated +> impl — no manual construction at the call site. + +## Step 9 — Wrap the call in a circuit breaker Both clients are deliberately small. For circuit breaking, rate limiting, or -bulkheads, wrap calls in `firefly-resilience` decorators (covered in -[Caching](./17-caching.md) and applied the same way to outbound calls). The -circuit breaker is what keeps a sick Payments service from dragging Lumen down -with it: +bulkheads, wrap calls in `firefly-resilience` decorators (the same ones +[Caching](./17-caching.md) applies to inbound work, applied the same way to +outbound calls). The circuit breaker is what keeps a sick Payments service from +dragging Lumen down with it. + +> **Note** **Key term — circuit breaker.** A *circuit breaker* watches a +> dependency's recent failures. After enough failures it *opens* and rejects +> further calls immediately for a cooldown, instead of letting every caller wait +> on a doomed timeout — then it half-opens to probe recovery. The Spring/Java +> analog is Resilience4j's `CircuitBreaker`. ```rust,ignore use firefly::resilience::{CircuitBreaker, CircuitConfig}; // CircuitBreaker::execute returns the operation's value (Result), so the -// guarded call still yields the Payment. (Chain::execute is for guarded ops -// whose value you discard — it returns Result<(), _>.) +// guarded call still yields the Payment. let breaker = CircuitBreaker::new(CircuitConfig::default()); let payment = breaker.execute(|| async { @@ -324,24 +558,34 @@ let payment = breaker.execute(|| async { }).await?; ``` -When repeated calls fail, the breaker opens and rejects subsequent calls -immediately with `ResilienceError::CircuitOpen` instead of waiting on a timeout — -so one slow upstream cannot exhaust Lumen's task pool. Resilience belongs at the -*client* layer, configured once, not scattered through every handler. +What just happened: `CircuitBreaker::new(CircuitConfig::default())` builds a +breaker; `breaker.execute(|| async { ... })` runs the closure under supervision, +recording each outcome and propagating the operation's `Result` (so the +guarded call still yields the `Payment`). + +Why it matters: when repeated calls fail, the breaker opens and rejects +subsequent calls immediately with `ResilienceError::CircuitOpen` instead of +waiting on a timeout — so one slow upstream cannot exhaust Lumen's task pool. +Resilience belongs at the *client* layer, configured once, not scattered through +every handler. -## The experience tier: a Lumen BFF +> **Tip** **Checkpoint.** Drive the upstream to fail enough times and the next +> `breaker.execute(...)` returns `Err(ResilienceError::CircuitOpen)` *immediately* +> — no timeout wait. `err.is_circuit_open()` confirms it. + +## Step 10 — Meet the experience tier (a Lumen BFF) A mobile or web frontend rarely wants a single domain service's raw shape — it -wants a *journey*: "show me this wallet's balance **and** its pending payments, -in one call." Calling Lumen for the balance and Payments for the pending list and -merging in the client means two round trips, two failure domains, and the -frontend leaking knowledge of both services' internals. The -**Backend-for-Frontend (BFF)** pattern moves that composition server-side. +wants a *journey*: "show me this wallet's balance **and** its pending payments, in +one call." Calling Lumen for the balance and Payments for the pending list and +merging in the client means two round trips, two failure domains, and the frontend +leaking knowledge of both services' internals. The Backend-for-Frontend (BFF) +pattern moves that composition server-side. -Firefly ships a dedicated starter for this tier: -`firefly-starter-experience`. It builds on `WebStack` (so it inherits CORS, -security headers, request metrics, correlation, and the actuator surface) and -adds the BFF building blocks: +Firefly ships a dedicated starter for this tier, `firefly-starter-experience`. It +builds on the same `WebStack` Lumen uses (so it inherits CORS, security headers, +request metrics, correlation, and the actuator surface) and adds the BFF building +blocks: - `DomainClients` — a registry of named `RestClient`s for the downstream domain services; @@ -355,8 +599,7 @@ A Lumen experience service registers its downstream clients up front, then composes them: ```rust,ignore -use firefly_starter_experience::{ExperienceStack, DomainClients}; -use firefly::starter_web::CoreConfig; +use firefly::starter_experience::{ExperienceStack, CoreConfig}; let bff = ExperienceStack::new(CoreConfig { app_name: "lumen-mobile-bff".into(), @@ -369,79 +612,111 @@ let wallets = bff.clients.register("wallets", "https://lumen.internal"); let payments = bff.clients.register("payments", "https://payments.internal"); ``` -The composition root then fans out across the registered clients — both calls -go out concurrently, so the composite latency is bounded by the slower upstream -rather than their sum — and degrades gracefully if one upstream is circuit-open -(show the balance, leave the pending list empty) instead of failing the whole -response. - -> **The tier boundary is strict.** The dependency direction is -> `channel → experience → domain → core`. An experience service **never** owns a -> database, **never** calls a core service directly, and **never** calls a -> sibling experience service — it composes *domain* SDKs only. Lumen is a -> domain/core-style service built on the `firefly` facade; the BFF is a separate -> crate that depends on `firefly-starter-experience` and on Lumen's published -> client. That separation is why the experience starter is *not* bundled into -> the one-dependency facade — a domain service does not need it. - -> **The BFF pattern.** A BFF is a thin application that aggregates several -> domain services into one journey-shaped API, server-side. `Mono::zip_with` -> (and the concurrent fan-out above) launches the upstream calls together, so the -> composite latency is bounded by the slower upstream rather than their sum, and -> a circuit-open upstream degrades gracefully instead of failing the whole -> response. The team-ownership model follows from the tier boundary: domain teams -> own stable, fine-grained contracts; the frontend team owns the BFF that adapts -> them. +What just happened: `ExperienceStack::new(CoreConfig { app_name, .. })` wires the +web tier plus the BFF building blocks; `bff.clients` is the `DomainClients` +registry, and `register(name, base_url)` returns an `Arc` already +wired with correlation + trace propagation. The composition root then fans out +across the registered clients — both calls go out concurrently, so the composite +latency is bounded by the slower upstream rather than their sum — and degrades +gracefully if one upstream is circuit-open (show the balance, leave the pending +list empty) instead of failing the whole response. + +> **Note** The experience tier has a chapter of its own — +> [The Experience Tier](./20a-experience-tier.md) — which covers +> `SignalService`, `WorkflowState`, the concurrent fan-out (`Mono::zip_with`), +> and partial-degradation handlers in depth. This section is the introduction; +> the full treatment lives there. + +> **Design note.** The tier boundary is strict: the dependency direction is +> `channel → experience → domain → core`. An experience service *never* owns a +> database, *never* calls a core service directly, and *never* calls a sibling +> experience service — it composes *domain* SDKs only. Lumen is a domain/core-style +> service built on the `firefly` facade; the BFF is a separate crate that depends +> on `firefly-starter-experience` and on Lumen's published client. That separation +> is why the experience starter is *not* bundled into the one-dependency facade — +> a domain service does not need it. ## Other protocols -The crate ships builders/scaffolds for the protocols a back-office platform -needs — SOAP (CXF-style envelope), gRPC, GraphQL, and WebSocket — selected by -feature so heavy dependencies stay out of services that do not use them. The -REST, GraphQL, and SOAP surfaces are fully wired; the streaming protocols (gRPC -and WebSocket) are feature-gated. - -Outbound calls inherit the caller's correlation id automatically, so a request -that fans out to three upstreams stitches together in your traces. - -## What changed in Lumen - -- We sketched the **Payments client** Lumen would build to settle a transfer's - credit leg over the network — `RestBuilder::new(...).with_retries(...).build()` - — and showed how an upstream RFC 7807 problem decodes into a typed - `FireflyError` via `err.as_firefly()`. -- We saw how the saga's credit step changes from a local `Ledger::deposit` to a - resilient outbound call, why an `Idempotency-Key` is mandatory across retries, - and why the debit's compensation matters more once I/O can fail. -- We introduced the reactive `WebClient` (`body_to_mono` / `body_to_flux` / - `exchange`), its streaming decode (the consumer side of Lumen's own NDJSON/SSE - endpoint), and the rule that retries are *composed* on the returned `Mono`, not - baked into the client. -- We met the **experience tier** — `firefly-starter-experience` with - `DomainClients`, `SignalService`, and `WorkflowState` — and saw how a Lumen - BFF composes the wallet and payments services into one journey-shaped API, - with the strict `experience → domain → core` dependency direction. +Beyond REST, the crate ships builders and scaffolds for the protocols a +back-office platform needs, selected by feature so heavy dependencies stay out of +services that do not use them: + +- `GraphQlBuilder` / `GraphQlClient` — POST a `{ query, variables?, + operationName? }`, raise `ClientError::GraphQl` on a non-empty `errors` array, + decode `data` into a typed `T`. Always available (no extra deps). +- `SoapBuilder` / `SoapClient` — wrap a body in a SOAP 1.1 envelope, POST + `text/xml` with an optional `SOAPAction` header, return the raw response XML. + Always available. +- `GrpcBuilder` — build a `tonic` channel for a caller-supplied generated stub. + Behind the `grpc` feature (`grpc-tls` for TLS). +- `WsBuilder` / `WsClient` — connect and stream over `tokio-tungstenite`. Behind + the `websocket` feature. + +The REST, GraphQL, and SOAP surfaces are fully wired; the streaming protocols +(gRPC and WebSocket) are feature-gated. As with the HTTP clients, every outbound +call inherits the caller's correlation id automatically, so a request that fans +out to three upstreams stitches together in your traces. + +## Recap — what changed in Lumen + +| Before | After this chapter | +|--------|--------------------| +| every transfer leg is a local `ledger.deposit(...)` | the credit leg can become a resilient outbound `payments.request(...)` over the network | +| no outbound failure modes | upstream RFC 9457 problems decode into a typed `FireflyError`, classified by `is_*` predicates | +| no network idempotency | a stable `Idempotency-Key` forwarded consistently across retries | +| — | the reactive `WebClient` (`body_to_mono` / `body_to_flux` / `exchange`), with retries *composed* via `Mono::retry_backoff`, not baked in | +| — | declarative `#[http_client]` traits autowired as `Arc` beans, plus a `CircuitBreaker`-guarded call | + +You also now know: + +- That the eager `RestClient`, the reactive `WebClient`, and the declarative + `#[http_client]` share one set of automatics — default headers, correlation / + trace propagation, and RFC 9457 problem decode. +- Why the `WebClient` has no retry budget: retry policy is a property of the call + site, expressed with `Mono::retry` / `Mono::retry_backoff`. +- That a declarative client mirrors a `#[rest_controller]` (same `:id` path + syntax, same verb attributes), and `bean` ties it into the DI graph. +- The experience-tier (BFF) pattern and its strict `channel → experience → + domain → core` boundary — the full treatment of which lives in + [The Experience Tier](./20a-experience-tier.md). ## Exercises -1. **Decode an upstream problem.** Stand up a stub that answers - `POST /payments` with a `422 application/problem+json` body. Call it through a - `RestClient` and assert that `err.as_firefly()` returns `Some`, that - `fe.status == 422`, and that `err.is_unprocessable_entity()` is `true` — so - the saga can map the upstream rejection onto Lumen's own `422` instead of a - `500`. +1. **Decode an upstream problem.** Stand up a stub that answers `POST /payments` + with a `422 application/problem+json` body. Call it through a `RestClient` and + assert that `err.as_firefly()` returns `Some`, that `fe.status == 422`, and + that `err.is_unprocessable_entity()` is `true` — so the saga can map the + upstream rejection onto Lumen's own `422` instead of a `500`. + +2. **Compose a retry.** Wrap a `WebClient` call to a flaky stub in + `Mono::retry_backoff(|| …, Backoff::new(3, Duration::from_millis(50)))`. Make + the stub fail twice then succeed, and assert the call ultimately resolves — + then confirm the bare `WebClient` (no wrapper) gives up after one attempt. -2. **Wrap the credit leg in a circuit breaker.** Take the transfer saga's credit +3. **Wrap the credit leg in a circuit breaker.** Take the transfer saga's credit step and replace `ledger.deposit(...)` with a `CircuitBreaker`-guarded `payments.request(...)`. Drive the stub to fail enough times to trip the breaker, and assert the next call returns `ResilienceError::CircuitOpen` *immediately* (no timeout) — and that the saga still compensates the debit. -3. **Compose a BFF summary.** Build a tiny `ExperienceStack`, register a - `wallets` and a `payments` client against two local stubs, and write a handler - that fetches the balance and the pending list concurrently. Make the payments - stub return an error and assert the handler still returns the balance with an - empty pending list — proving partial degradation rather than a `500`. - -The next chapter secures the inbound side — JWT bearer auth and path-based RBAC -on Lumen's mutating routes. Continue to [Security](./14-security.md). +4. **Write a declarative client.** Define a `#[http_client(path = "/api/v1/orders")]` + trait with a `get_order(&self, id: String) -> Result` + method, construct it with `::new("http://localhost:PORT")` against a local + stub, and assert it issues `GET /api/v1/orders/42`. Then change the path to use + Spring's `{id}` and confirm the build fails with the `:id` hint. + +5. **Compose a BFF summary.** Build a tiny `ExperienceStack`, register a `wallets` + and a `payments` client against two local stubs, and write a handler that + fetches the balance and the pending list concurrently. Make the payments stub + return an error and assert the handler still returns the balance with an empty + pending list — proving partial degradation rather than a `500`. + +## Where to go next + +- Secure the *inbound* side — JWT bearer auth and path-based RBAC on Lumen's + mutating routes — in **[Security](./14-security.md)**. +- Go deeper on composing domain services into a journey-shaped API in + **[The Experience Tier](./20a-experience-tier.md)**. +- Revisit the resilience decorators (circuit breaker, rate limiter, bulkhead, + timeout) applied to inbound work in **[Caching](./17-caching.md)**. diff --git a/docs/book/src/14-security.md b/docs/book/src/14-security.md index 7b19dbf..7ae51a3 100644 --- a/docs/book/src/14-security.md +++ b/docs/book/src/14-security.md @@ -1,76 +1,113 @@ # Security -In Chapter 13 you saw how Lumen *would* call an external payments or FX -provider. Lumen itself, though, is still wide open: any caller can open a -wallet, deposit, withdraw, or move money between wallets. Before Part V can ship -Lumen to production, you have to close that door. +In [HTTP Clients](./13-http-clients.md) you saw how Lumen *would* call an +external payments or FX provider. Lumen itself, though, is still wide open: any +caller can open a wallet, deposit, withdraw, or move money between wallets. +Before Part V can ship Lumen to production, you have to close that door — and +you will do it without adding a dependency, hand-rolling crypto, or rewriting a +single handler. By the end of this chapter Lumen will **authenticate** every request with a signed JWT, **authorize** the mutating routes with a path-based RBAC filter chain, and leave the public reads and the management surface open. The whole -thing is built on `firefly-security`, reached through the one `firefly` facade — -no new dependency, no hand-rolled crypto, and (true to Lumen's promise) not even -a `thiserror` derive. - -> **What's in the box.** `firefly-security` is a complete resource-server -> stack: a `BearerLayer` + `Verifier` for token validation, a URL-based -> authorization `FilterChain`, method guards, JWKS validation, an OAuth2 client + -> authorization server, a role hierarchy, CSRF, and bcrypt + Argon2id password encoders. Auth -> failures render as RFC 9457 `application/problem+json` on a 401/403 — a stable, -> standards-based wire contract that off-the-shelf clients and gateways -> understand. - -## The mental model - -
- - - incoming request - - - - - - - BearerLayer - - reads Authorization: Bearer <tok> - calls the Verifier (JwtService) - stores Authentication on request - allow_anonymous → pass empty ctx - - - - - - - - FilterChain (RBAC) - - permit_method("GET", "/api/v1/...") - permit("/actuator/") - require("/api/v1/wallets", CUSTOMER) - 401 / 403 problem+json on a miss - - - - - - - WalletApi handlers - -
The security request pipeline. Every request passes the BearerLayer (authentication) and the RBAC FilterChain (authorization) before reaching the WalletApi handlers.
-
- -`firefly-security` carries far more than Lumen uses — JWKS, OAuth2 (client + -authorization server), `RoleHierarchy`, method-level `guards`, -`CsrfLayer`, `BcryptPasswordEncoder`. We tour those at the end; first, the four -pieces Lumen actually wires. - -## Minting and verifying tokens — JwtService +thing is built on the framework's security tier, reached through the one +`firefly` facade you have depended on since [Quickstart](./02-quickstart.md). + +By the end of this chapter you will: + +- Mint and verify signed HS256 tokens with `JwtService`, and understand why + every issued token carries a bounded `exp`. +- Adapt that service into a `Verifier` and turn a request's bearer token into an + `Authentication` (principal, roles, claims). +- Compose a `BearerLayer` and a path-ordered RBAC `FilterChain`, and understand + fail-closed (deny-by-default) ordering. +- Wire both as `#[bean]`s and watch `FireflyApplication` auto-discover and layer + them — no `.with_security(...)` call anywhere. +- Push authorization down to a service method with `#[firefly::pre_authorize]` / + `#[firefly::post_authorize]` over an ambient security context. +- Move the same posture into configuration so production swaps the demo key for + a real IdP with no code edit. + +## Concepts you will meet + +Before the first line of code, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — authentication vs. authorization.** *Authentication* +> answers "who is this caller?" — it validates a credential and resolves a +> principal. *Authorization* answers "may they do this?" — it checks that the +> resolved principal is allowed to perform the operation. They are two distinct +> stages, and Firefly keeps them in two distinct components. + +> **Note** **Key term — JWT (JSON Web Token).** A *JWT* is a compact, +> URL-safe, signed token carrying a JSON payload of *claims* (`sub`, `roles`, +> `exp`, …). Because the signature proves the payload was not tampered with, a +> stateless service can trust a token's claims without a server-side session or +> a round-trip to a database. The Spring analog is a Spring Security resource +> server validating a `Bearer` token. + +> **Note** **Key term — RBAC (Role-Based Access Control).** *RBAC* grants +> access by the *roles* a caller holds (here `CUSTOMER`) rather than by their +> identity. A rule says "this route requires role X"; a caller passes if their +> token carries X. This is the model Spring Security's URL authorization rules +> use. + +> **Note** **Key term — resource server.** A *resource server* is a service +> that protects its own endpoints by validating an access token minted +> elsewhere (an identity provider). It never logs anyone in; it only *verifies* +> the credential it is handed. Lumen is a resource server — in the demo it also +> mints its own tokens for testing, but the verification path is identical to a +> production IdP. + +The framework's security tier carries far more than Lumen uses — JWKS, OAuth2 +(client + authorization server), role hierarchy, method guards, CSRF, and +password encoders. We tour those at the end; first, the four pieces Lumen +actually wires, in the order a request meets them. + +## The request pipeline at a glance + +Every request travels through two security stages before it reaches a handler: + +```text + incoming request + │ + ▼ + ┌───────────────────────────────────────┐ + │ BearerLayer │ (authentication) + │ • reads Authorization: Bearer │ + │ • calls the Verifier (JwtService) │ + │ • stores Authentication on the request│ + │ • allow_anonymous → pass empty ctx │ + └───────────────────────────────────────┘ + │ + ▼ + ┌───────────────────────────────────────┐ + │ FilterChain (RBAC) │ (authorization) + │ permit_method("GET", "/api/v1/...") │ + │ permit("/actuator/") │ + │ require("/api/v1/wallets", CUSTOMER) │ + │ 401 / 403 problem+json on a miss │ + └───────────────────────────────────────┘ + │ + ▼ + WalletApi handlers +``` + +The `BearerLayer` authenticates (who?), the `FilterChain` authorizes (may +they?), and only a request that clears both reaches Lumen's handlers. Auth +failures render as RFC 9457 `application/problem+json` on a 401 or 403 — a +stable, standards-based wire contract that off-the-shelf clients and gateways +already understand. + +> **Note** **Key term — RFC 9457 problem+json.** RFC 9457 (which obsoletes RFC +> 7807) standardizes machine-readable error bodies under the +> `application/problem+json` media type — a JSON object with `type`, `title`, +> `status`, and `detail` members. Firefly renders every security rejection this +> way, so a 401 or 403 is a structured document, not a blank body. You met this +> renderer in [Your First HTTP API](./06-first-http-api.md); here it carries the +> auth failures too. + +## Step 1 — Mint and verify tokens with `JwtService` Lumen is a stateless API. Sessions would need sticky routing or a shared store on every replica; a signed JWT lets each request carry its own credential, so @@ -79,9 +116,18 @@ the service scales horizontally with no shared state. The framework's using a symmetric HS256 key — which is exactly what makes Lumen runnable and testable with no external IdP. -Here is the whole of Lumen's token surface, from `src/security.rs`: +> **Note** **Key term — symmetric (HS256) signing.** HS256 signs and verifies +> with the *same* shared secret. That is the simplest scheme to run — one key, +> no key server — and the right fit for a self-contained sample. (A production +> deployment usually moves to *asymmetric* RS256, where an IdP signs with a +> private key and your service verifies with the matching public key fetched +> from a JWKS endpoint. Step 7 shows that swap.) + +Create `src/security.rs` and start with the signing key, the role constant, and +the shared service: ```rust,ignore +// src/security.rs use firefly::security::{ BearerConfig, BearerLayer, FilterChain, JwtService, SecurityError, Verifier, VerifierFn, }; @@ -109,20 +155,42 @@ pub fn mint_token(subject: &str, roles: &[&str]) -> String { } ``` -`JwtService::new(secret)` builds an HS256 service. `encode` signs a JSON -payload and — this is the load-bearing detail — **injects an `exp` claim** when -the payload has none, defaulting to one hour out. Every token Lumen issues has a -bounded lifetime; a token with no `exp` is rejected at `decode` time. The -`mint_token` helper is what the HTTP tests call to obtain a credential the -verifier will accept. - -## The Verifier and the Authentication - -A `Verifier` is the resource-server port: validate a token, return an -`Authentication` (the principal, username, roles, and the raw claims). -`VerifierFn` adapts a plain async closure into one. Lumen's verifier delegates -straight to `JwtService::to_authentication`, which maps `sub` → principal and -`roles` → roles, then re-wraps any failure as a `SecurityError::Verification`: +What just happened, block by block: + +- The `use` line pulls the entire token surface from `firefly::security` — the + facade re-exports the framework's security crate, so there is no new + dependency to add to `Cargo.toml`. +- `JwtService::new(secret)` builds an HS256 service over the secret. Construction + takes anything `AsRef<[u8]>`, so the inline byte-string key works directly. +- `encode` signs a JSON payload and — this is the load-bearing detail — + **injects an `exp` claim** when the payload has none, defaulting to one hour + out (`DEFAULT_EXPIRATION_SECONDS = 3600`). Every token Lumen issues therefore + has a bounded lifetime. A token minted *without* `exp` (one that would never + expire) is rejected at decode time, because `decode` lists `exp` as a required + claim. +- `mint_token` is the helper the HTTP tests call to obtain a credential the + verifier will accept. The `.expect(...)` is safe here: signing a fixed, + well-formed claim shape cannot fail. + +> **Tip** **Checkpoint.** A quick mental dry run: `mint_token("u-alice", +> &["CUSTOMER"])` returns a three-segment `header.payload.signature` string. +> Decoding its middle segment (base64) would show `sub`, `roles`, and an +> auto-stamped `exp` roughly one hour in the future. + +## Step 2 — Turn the service into a `Verifier` + +`JwtService` can already verify — it implements the `Verifier` trait directly. +But Lumen wraps it in a small adapter so the *error shape* is exactly what the +`BearerLayer` wants to render. + +> **Note** **Key term — `Verifier` (the authentication port).** A `Verifier` is +> the resource-server *port*: given a raw token, validate it and return an +> `Authentication` (the principal, username, roles, and raw claims), or a +> `SecurityError` on failure. It is a trait, so any token validator — the demo +> HS256 service, a JWKS verifier, your own closure — satisfies the same +> contract. `VerifierFn` adapts a plain async closure into one. + +Add the verifier builder: ```rust,ignore /// Builds the resource-server Verifier: validates the token's HS256 @@ -139,22 +207,38 @@ pub fn build_verifier() -> impl Verifier { } ``` -`Authentication` carries the fields a handler or filter inspects: - -| Field | Type | From the claim | -|-------------|-----------------------------------|-------------------------------| -| `principal` | `String` | `sub` | -| `username` | `String` | `sub` (or a friendlier claim) | -| `roles` | `Vec` | `roles` | -| `claims` | `HashMap` | every decoded claim | - -Its helpers — `has_role(r)`, `has_any_role(&[..])`, `has_authority(a)` (matches a -role *or* a fine-grained permission/scope), and `Authentication::anonymous()` — -cover the common checks. Lumen's unit test asserts the round-trip directly: +What just happened: `VerifierFn(closure)` wraps a plain `async` closure as a +`Verifier`. The closure delegates to `JwtService::to_authentication`, which +decodes the token and maps its claims onto an `Authentication` — `sub` becomes +the principal, the `roles` array becomes roles, and every decoded claim is kept. +Any failure (bad signature, expired, missing `exp`) is re-wrapped as +`SecurityError::Verification(..)`; the `BearerLayer` turns that into the canonical +401 problem. + +> **Note** **Key term — `Authentication`.** `Authentication` is the resolved +> caller the rest of the stack inspects. It is the Rust analog of Spring +> Security's `Authentication` object. Its fields: +> +> | Field | Type | From the claim | +> |-------------|---------------------------------------|-------------------------------| +> | `principal` | `String` | `sub` | +> | `username` | `String` | `preferred_username` / `name`, else `sub` | +> | `roles` | `Vec` | `roles` | +> | `authorities` | `Vec` | `permissions` (and OAuth2 scopes) | +> | `claims` | `HashMap` | every decoded claim | +> +> Its helpers cover the common checks: `has_role(r)`, +> `has_any_role(&[..])`, `has_authority(a)` (matches a role *or* a fine-grained +> permission/scope), `has_any_authority(&[..])`, and the +> `Authentication::anonymous()` constructor. + +A unit test asserts the round-trip directly — mint a token, verify it, confirm +the principal and role survived: ```rust,ignore #[tokio::test] async fn mint_then_verify_roundtrips_claims() { + use firefly::security::Authentication; let token = mint_token("u-alice", &[CUSTOMER_ROLE]); let auth: Authentication = build_verifier().verify(&token).await.unwrap(); assert_eq!(auth.principal, "u-alice"); @@ -163,23 +247,42 @@ async fn mint_then_verify_roundtrips_claims() { ``` A tampered token (`"not.a.jwt"`) or one signed with the wrong key is rejected -with `SecurityError::Verification` — the two negative tests in `security.rs` -prove it. +with `SecurityError::Verification` — two negative tests in `security.rs` prove +it: -> **Production swap.** Move to a real identity provider by replacing -> `build_verifier` with `firefly::security::JwksVerifier`, pointed at your IdP's -> JWKS URI (RS256, `kid` cache, `iss`/`aud` validation, `exp` required). The -> `Verifier` port is identical, so `security_layers` — and every handler — is -> untouched. That is the "swap the adapter, keep the code" promise applied to -> identity. +```rust,ignore +#[tokio::test] +async fn tampered_token_is_rejected() { + let err = build_verifier().verify("not.a.jwt").await.unwrap_err(); + assert!(matches!(err, SecurityError::Verification(_))); +} +``` + +> **Tip** **Checkpoint.** Run `cargo test mint_then_verify` (or the whole +> `security` module). The round-trip test passes, and the two rejection tests +> confirm a bad credential never resolves to an `Authentication`. Authentication +> now works end to end, before any HTTP wiring. -## URL authorization — the FilterChain +## Step 3 — Compose the `BearerLayer` and the RBAC `FilterChain` `JwtService` answers *who is this caller?*; the `FilterChain` answers *may they -do this?* It matches request paths against rules in declaration order — first -match wins — and renders a 401 (no/invalid credential) or 403 (authenticated but -under-privileged) as an RFC 9457 problem. Lumen composes the bearer layer and -the chain in one place: +do this?* The chain matches request paths against rules in declaration order — +**first match wins** — and renders a 401 (no/invalid credential) or 403 +(authenticated but under-privileged). Lumen composes the bearer layer and the +chain in one function. + +> **Note** **Key term — `BearerLayer`.** The `BearerLayer` is the tower +> middleware that performs authentication on the wire: it reads the +> `Authorization: Bearer ` header, calls the `Verifier`, and stores the +> resulting `Authentication` on the request before the chain runs. It is the +> Rust analog of Spring Security's bearer-token authentication filter. + +> **Note** **Key term — `FilterChain`.** The `FilterChain` is the path-based +> authorization matcher — the Rust analog of Spring Security's URL authorization +> rules (`authorizeHttpRequests`). You build it with `permit` / `require` / +> `permit_method` calls; each adds one ordered rule. + +Add the composition function: ```rust,ignore /// Builds the BearerLayer + FilterChain that protect the service. @@ -206,93 +309,143 @@ pub fn security_layers() -> (BearerLayer, FilterChain) { } ``` -Two design choices are worth dwelling on: +Two design choices are worth dwelling on, because they decide who gets a 401 vs. +who slips through: - **`allow_anonymous(true)` on the bearer layer.** With it set, a request with no `Authorization` header is *not* rejected at the bearer layer — it reaches the chain carrying an anonymous `Authentication`. That keeps a single decision-maker: the `FilterChain` decides every route. A public `GET` passes; - a `require` route with no valid token becomes a 401. Without - `allow_anonymous`, the bearer layer would reject anonymous traffic before the - chain could permit the public reads. + a `require` route with no valid token becomes a 401. Without `allow_anonymous`, + the bearer layer would reject anonymous traffic *before* the chain could permit + the public reads — so the public wallet read would break. - **Order matters.** `permit_method("GET", "/api/v1/wallets")` and `permit("/actuator/")` come *first*, so the public reads and the management surface are decided before the broad `require("/api/v1/wallets", ...)` could - catch them. `any_request_permit()` re-opens the unmatched tail. - -> **Warning.** Once any rule is declared, a `FilterChain` is **fail-closed**: a -> request matching no rule is rejected with 403 (deny-by-default). -> Re-open the unmatched tail explicitly with `any_request_permit()` / -> `any_request_authenticated()` / `any_request_deny()`. A chain with *no* rules is -> a no-op and passes everything (never a surprise blanket lockout). - -## Layering it onto the router - -Lumen does not layer security by hand. The `FilterChain` and the `BearerLayer` -are each declared as a `#[bean]` in `LumenBeans` (the `#[derive(Configuration)]` -holder in `src/web.rs`), and `FireflyApplication` auto-discovers and applies them -— the Rust analog of Spring's `SecurityFilterChain` bean. There is no -`.with_security(...)` call and no manual `.layer(bearer)`: + catch them. First match wins, so a more specific permit must precede a broader + require. `any_request_permit()` then re-opens the unmatched tail (see the + warning below). + +> **Warning** Once any rule is declared, a `FilterChain` is **fail-closed**: a +> request matching no rule is rejected with 403 (deny-by-default, matching Spring +> Security 6). Re-open the unmatched tail explicitly with `any_request_permit()` +> / `any_request_authenticated()` / `any_request_deny()`. A chain with *no* rules +> at all is a no-op and passes everything — so an empty chain is never a surprise +> blanket lockout, but the moment you add your first rule, everything you did +> not name is denied unless a catch-all re-opens it. + +> **Tip** **Checkpoint.** Trace each route through the rule list by hand: `GET +> /api/v1/wallets/w-1` hits the first `permit_method` and passes; `GET +> /actuator/health` hits `permit("/actuator/")` and passes; `POST +> /api/v1/wallets` falls past both permits to `require("/api/v1/wallets", +> [CUSTOMER])`; an unmatched path like `GET /favicon.ico` reaches +> `any_request_permit()` and passes. If you mentally reordered the requires above +> the permits, the public read would now demand a token — that is the "first +> match wins" trap in action. + +## Step 4 — Wire the layers as beans + +Lumen does **not** layer security by hand. The `FilterChain` and the +`BearerLayer` are each declared as a `#[bean]` in `LumenBeans` — the +`#[derive(Configuration)]` holder in `src/web.rs` you have been growing since the +DI chapter — and `FireflyApplication` auto-discovers and applies them. This is +the Rust analog of Spring's `SecurityFilterChain` bean: declaring the bean *is* +the wiring. + +> **Note** **Key term — security as discovered beans.** In Spring Boot you +> register a `SecurityFilterChain` `@Bean` and the framework applies it; you +> never call a `with_security(...)` method. Firefly works the same way: a +> `FilterChain` bean and a `BearerLayer` bean are auto-discovered at boot and +> layered onto the router. There is no `.with_security(...)` call and no manual +> `.layer(bearer)` in app code. + +Add the two bean methods to the existing `#[bean] impl LumenBeans` block: ```rust,ignore -// samples/lumen/src/web.rs — LumenBeans (#[derive(Configuration)]). +// samples/lumen/src/web.rs — inside #[bean] impl LumenBeans { ... } +use firefly::security::{BearerLayer, FilterChain}; + +/// The HTTP security filter chain (path-based RBAC) — the Spring +/// `SecurityFilterChain` bean. `FireflyApplication` auto-discovers + applies it. #[bean] -impl LumenBeans { - /// The HTTP security filter chain (path-based RBAC) — the Spring - /// `SecurityFilterChain` bean. `FireflyApplication` auto-discovers + applies it. - #[bean] - fn security_filter_chain(&self) -> FilterChain { - crate::security::security_layers().1 - } +fn security_filter_chain(&self) -> FilterChain { + crate::security::security_layers().1 +} - /// The bearer-token authentication layer — auto-discovered + layered onto - /// the API by `FireflyApplication`. - #[bean] - fn bearer_layer(&self) -> BearerLayer { - crate::security::security_layers().0 - } +/// The bearer-token authentication layer — auto-discovered + layered onto +/// the API by `FireflyApplication`. +#[bean] +fn bearer_layer(&self) -> BearerLayer { + crate::security::security_layers().0 } ``` -At boot the framework resolves the `FilterChain` bean and sets it on the web -stack, then resolves the `BearerLayer` bean and layers it around the whole router -so the chain sees a populated `Authentication`. The chain runs *inside* the -inherited correlation / security-headers / CORS edge, so even a 401 response -carries those headers and a correlation id; the bearer layer goes on the outside -(axum runs the last-added layer first): authenticate, then authorize. Declaring -the two beans is the *entire* wiring — no `with_security`, no `apply_middleware` -call in app code. - -## Method security +What just happened, and what the framework does with it at boot: + +- Each `#[bean]` method declares one component for the container to construct. + `security_layers()` returns the `(BearerLayer, FilterChain)` tuple; one bean + hands back `.0`, the other `.1`. +- At startup `run()` resolves the `FilterChain` bean and sets it on the web + stack, then resolves the `BearerLayer` bean and layers it around the whole + router so the chain always sees a populated `Authentication`. +- The chain runs *inside* the inherited correlation / security-headers / CORS + edge, so even a 401 response carries those headers and a correlation id. The + bearer layer goes on the *outside* — axum runs the last-added layer first, so + the order is **authenticate, then authorize**. + +Declaring the two beans is the *entire* wiring: no `with_security`, no +`apply_middleware` call, no edit to `main`. This is the "no `main` churn" +property from [Quickstart](./02-quickstart.md) at work — security is just more +beans for the framework to discover. + +> **Tip** **Checkpoint.** Run `cargo run` and read the startup report's +> `:: beans ::` line — `security_filter_chain` and `bearer_layer` now appear in +> the discovered-bean inventory. Then `curl -i -X POST localhost:8080/api/v1/wallets +> -H 'content-type: application/json' -d '{"owner":"mallory","openingBalance":10}'`: +> you get a `401` with `content-type: application/problem+json`, because the +> mutation now requires a `CUSTOMER` token. The public read still works: +> `curl localhost:8080/api/v1/wallets/anything` is no longer rejected for lack of +> a token (it 404s on an unknown id, which is a different, business-level +> outcome). + +## Step 5 — Push authorization down to a method The `FilterChain` guards *routes*. But authorization is often a property of a -*service method* — a domain operation that several handlers, a scheduled job, and -a CQRS handler all call. Pushing the check down to the method means it holds no -matter how the operation is reached, and the route table stays about routes. +*service method* — a domain operation that several handlers, a scheduled job, +and a CQRS handler all call. Pushing the check down to the method means it holds +no matter how the operation is reached, and the route table stays about routes. Firefly does this with two attribute macros and an **ambient security context**. The macros declare the rule; the context carries the caller's `Authentication` through the call stack so the method never has to thread an argument or touch the `Request`. +> **Note** **Key term — ambient security context.** The *ambient context* is a +> task-local slot holding the current `Authentication` — the Rust analog of +> Spring's `SecurityContextHolder` and its thread-local. The `BearerLayer` +> installs it for the duration of each request, so any method reached downstream +> can read the caller without it riding every function signature. Because the +> slot is task-local it nests cleanly and never leaks across spawned tasks. + ### `#[firefly::pre_authorize(...)]` `#[firefly::pre_authorize(...)]` guards a function *before* its body runs. It attaches to any function returning `Result` where `E: -From`, and reads the ambient `Authentication` -(below) to decide. The rules: +From`, and reads the ambient `Authentication` to +decide. The rules: | Rule | Passes when | |-------------------------------|-----------------------------------------------------| -| *(empty)* / `authenticated` | a caller is in scope (the default) | +| *(empty)* / `authenticated` | a real (non-anonymous) caller is in scope (default) | | `role = "ADMIN"` | the caller has role `ADMIN` | | `any_role = ["A", "B"]` | the caller has *any* of the listed roles | | `authority = "wallet:write"` | the caller holds that authority (role or scope) | | `any_authority = ["a", "b"]` | the caller holds *any* of the listed authorities | -On denial the macro returns early with `Err(..)`: an **`Unauthenticated`** -`SecurityError` when no caller is in scope, a **`Forbidden`** one when a caller is -present but the authorities don't match. +On denial the macro returns early with `Err(..)`: an `Unauthenticated` +`SecurityError` when no caller is in scope, a `Forbidden` one when a caller is +present but the authorities don't match. The `?` inside the generated code +propagates that error through your `From` impl. ```rust,ignore use firefly_security::SecurityError; @@ -326,12 +479,13 @@ wallet only if you own it." `#[firefly::post_authorize(...)]` attaches to an `async fn` returning `Result` and evaluates a boolean expression once the body has produced its `Ok(T)`. The expression sees two bindings: -- `result` — a `&T`, the value the function is about to return (the *return - object*). +- `result` — a `&T`, the value the function is about to return (Spring's + *return object*). - `auth` — a `&Authentication`, the ambient caller. If the expression is `false` the value is **discarded** and the call resolves to -a `Forbidden` error instead: +a `Forbidden` error instead; if no context is active at all it resolves to +`Unauthenticated`: ```rust,ignore /// A caller may fetch a wallet only if they own it. @@ -341,14 +495,14 @@ pub async fn get_wallet(id: WalletId) -> Result { } ``` -### The ambient context — `firefly_security` +### The ambient context functions -Both macros read an ambient `Authentication` rather than an argument. That scope -is managed by a small set of functions in `firefly_security` — the security -context that travels with the task: +Both macros read the ambient `Authentication` rather than an argument. That scope +is managed by a small set of functions in `firefly_security`, reached through the +facade as `firefly::security`: ```rust,ignore -use firefly_security::{ +use firefly::security::{ with_authentication_scope, current_authentication, check_access, AccessRule, Authentication, SecurityError, }; @@ -358,7 +512,7 @@ let wallet = with_authentication_scope(auth, async { withdraw(id, amount).await // #[pre_authorize] inside sees `auth` }).await?; -// Read the current caller anywhere downstream (None if unauthenticated). +// Read the current caller anywhere downstream (None if no scope is active). let who: Option = current_authentication(); // Imperative check when a macro doesn't fit — returns the Authentication on @@ -366,9 +520,8 @@ let who: Option = current_authentication(); let auth: Authentication = check_access(&AccessRule::Role("CUSTOMER"))?; ``` -`AccessRule` is the runtime form of the macro rules: -`AccessRule::Authenticated`, `Role(&str)`, `AnyRole(&[&str])`, `Authority(&str)`, -and `AnyAuthority(&[&str])`. +`AccessRule` is the runtime form of the macro rules: `AccessRule::Authenticated`, +`Role(&str)`, `AnyRole(&[&str])`, `Authority(&str)`, and `AnyAuthority(&[&str])`. The payoff is that **`BearerLayer` installs the scope for you**. On every request — both the verified path *and* the anonymous (`allow_anonymous`) path — the @@ -378,7 +531,90 @@ never sees the `Request`. URL rules and method rules then compose: the `FilterChain` is your coarse perimeter, the method macros are your defense in depth. -## Wiring security from configuration +> **Tip** **Checkpoint.** The key invariant to hold in your head: a +> `#[pre_authorize]` method called *outside* any scope (e.g. directly from a +> plain `#[test]` without `with_authentication_scope_sync`) returns +> `Unauthenticated` — the macro fails closed when there is no caller, exactly +> like the route chain. + +## Step 6 — Prove it end to end over HTTP + +The HTTP suite (`tests/http.rs`, in the sample at `src/http_test.rs`) drives the +fully-wired router with `tower::ServiceExt::oneshot` and asserts the security +behavior directly — no socket bound. The router comes from `build_router`, which +boots the same app `main()` does: + +```rust,ignore +// The testable in-process public router — every bean (including the +// FilterChain + BearerLayer) is auto-discovered, exactly as in `main`. +#[cfg(test)] +pub(crate) async fn build_router() -> axum::Router { + firefly::FireflyApplication::new(APP_NAME) + .version(VERSION) + .bootstrap() + .await + .expect("lumen bootstrap") + .api_router +} +``` + +> **Note** **Testing seam.** `bootstrap()` is the sibling of `run()` from +> [Quickstart](./02-quickstart.md): it assembles the same app — security beans +> included — but returns a `Bootstrapped` value *without* serving, so a test can +> drive the wired public router (`Bootstrapped::api_router`) in-process. You met +> this in [Your First HTTP API](./06-first-http-api.md); here it lets the suite +> exercise the real `BearerLayer` + `FilterChain`. + +A request helper builds the `Authorization` header from `mint_token`, so an +authenticated request is just `post(path, body, true)` and an unauthenticated one +is `post(path, body, false)`: + +```rust,ignore +fn bearer() -> String { + format!("Bearer {}", mint_token("u-alice", &[CUSTOMER_ROLE])) +} + +fn post(path: &str, body: serde_json::Value, auth: bool) -> Request { + let mut b = Request::post(path).header("content-type", "application/json"); + if auth { + b = b.header("authorization", bearer()); + } + b.body(Body::from(serde_json::to_vec(&body).unwrap())).unwrap() +} +``` + +A mutation with **no** token is a 401 problem: + +```rust,ignore +#[tokio::test] +async fn missing_token_is_401_problem_on_mutations() { + let app = build_router().await; + let res = send( + &app, + post( + "/api/v1/wallets", + serde_json::json!({ "owner": "mallory", "openingBalance": 10 }), + false, // no Authorization header + ), + ) + .await; + assert_eq!(res.status(), StatusCode::UNAUTHORIZED); + assert!(content_type(&res).contains("application/problem+json")); +} +``` + +Every other test in the suite — open, get, deposit/withdraw, transfer — runs as +the minted `CUSTOMER` (it passes `true`), so authentication is exercised on the +happy path too. A 401 proves the perimeter is closed; the green happy-path tests +prove a valid token still gets through. + +> **Tip** **Checkpoint.** Run `cargo test` for Lumen. The `missing_token_is_401` +> test is red-then-green proof the front door is shut, and the wallet round-trip +> tests confirm a `CUSTOMER` token still opens it. If the 401 test fails with a +> `201 Created`, the `FilterChain` bean is not being discovered — confirm both +> `#[bean]` methods compile inside the `LumenBeans` block. + +## Step 7 — Move the posture into configuration Lumen inlines its signing key and builds the bearer layer by hand because that makes the sample runnable as-is. A deployed service instead reads its security @@ -387,26 +623,29 @@ container, no framework callback. The properties live under `firefly.security.*` and bind through `serde`: ```rust,ignore -use firefly_security::{ +use firefly::security::{ SecurityProperties, JwtProperties, BearerProperties, verifier_from_config, bearer_layer_from_config, }; ``` -```toml -# firefly.security.* — JWKS resource-server example -[firefly.security.jwt] -jwk_set_uri = "https://idp.example.com/.well-known/jwks.json" -issuer_uri = "https://idp.example.com/" -audience = "lumen" -algorithm = "RS256" - -[firefly.security.bearer] -header_name = "Authorization" -allow_anonymous = true +A JWKS resource-server posture in `firefly.yaml`: + +```yaml +firefly: + security: + jwt: + jwk-set-uri: "https://idp.example.com/.well-known/jwks.json" + issuer-uri: "https://idp.example.com/" + audience: "lumen" + algorithm: "RS256" + bearer: + header-name: "Authorization" + allow-anonymous: true ``` -The structs mirror that shape: +The structs mirror that shape; each derives `Default` + `Deserialize` with +`#[serde(default)]`, so a missing field falls back to its zero value: ```rust,ignore pub struct SecurityProperties { @@ -414,8 +653,6 @@ pub struct SecurityProperties { pub bearer: BearerProperties, } -// Both structs derive `Default, Deserialize` with `#[serde(default)]`, so a -// missing field falls back to its zero value (empty `String`, `0`). pub struct JwtProperties { pub jwk_set_uri: String, pub issuer_uri: String, @@ -435,7 +672,7 @@ Two builder functions turn bound properties into ready components: ```rust,ignore use std::sync::Arc; -use firefly_security::{Verifier, BearerLayer, SecurityError}; +use firefly::security::{Verifier, BearerLayer, SecurityError}; // Pick a verifier by what configuration provides — JWKS first, then HMAC, // then nothing. @@ -446,83 +683,59 @@ let verifier: Option> = verifier_from_config(&props.jwt)?; let bearer: Option = bearer_layer_from_config(&props)?; ``` -`verifier_from_config(&JwtProperties)` resolves the verifier by precedence: a -non-empty `jwk_set_uri` builds a JWKS (RS256) resource-server verifier; otherwise -a non-empty `secret` builds an HMAC (HS256/384/512) verifier; otherwise it -returns `None`. `bearer_layer_from_config(&SecurityProperties)` builds the -verifier the same way and, if there is one, wraps it in a `BearerLayer` with the -configured header name and anonymous policy already applied — the same layer -`security_layers` builds by hand, sourced from config instead. Switching Lumen -from the demo HMAC key to a production IdP becomes a configuration change, with -no edit to `security.rs`. - -## Proving it end to end - -The HTTP suite (`tests/http.rs`) drives the fully-wired router with -`tower::oneshot` and asserts the security behavior directly. A mutation with no -token is a 401 problem; a malformed body on an authenticated request is a 422 -problem: - -```rust,ignore -#[tokio::test] -async fn missing_token_is_401_problem_on_mutations() { - let res = build_router() - .await - .oneshot(post( - "/api/v1/wallets", - serde_json::json!({ "owner": "mallory", "openingBalance": 10 }), - false, // no Authorization header - )) - .await - .unwrap(); - assert_eq!(res.status(), StatusCode::UNAUTHORIZED); - assert!(content_type(&res).contains("application/problem+json")); -} -``` - -The test helper builds the `Authorization` header from `mint_token`, so an -authenticated request is just `post(path, body, true)`: - -```rust,ignore -fn bearer() -> String { - format!("Bearer {}", mint_token("u-alice", &[CUSTOMER_ROLE])) -} -``` - -Every other test in the suite — open, get, deposit/withdraw, transfer — runs as -the minted `CUSTOMER`, so authentication is exercised on the happy path too. - -## The rest of firefly-security (Lumen's growth room) +What just happened: `verifier_from_config(&JwtProperties)` resolves the verifier +by precedence — a non-empty `jwk_set_uri` builds a JWKS (RS256) resource-server +verifier; otherwise a non-empty `secret` builds an HMAC (HS256/384/512) verifier; +otherwise it returns `None`. `bearer_layer_from_config(&SecurityProperties)` +builds the verifier the same way and, if there is one, wraps it in a +`BearerLayer` with the configured header name and anonymous policy already +applied — the same layer `security_layers` builds by hand, sourced from config +instead. Switching Lumen from the demo HMAC key to a production IdP becomes a +configuration change, with no edit to `security.rs`. + +> **Note** **Key term — JWKS (JSON Web Key Set).** A *JWKS* is the public-key +> set an identity provider publishes at a well-known URL. A resource server +> fetches it to verify RS256 tokens, keying on each token's `kid` (key id) and +> caching the result. `JwksVerifier` is the framework's drop-in `Verifier` for +> this: same `Verifier` port, so `security_layers` — and every handler — is +> untouched when you swap the demo HMAC verifier for it. That is the "swap the +> adapter, keep the code" promise applied to identity. + +## The rest of the security tier — Lumen's growth room Lumen uses the symmetric-key fast path. The same crate carries the production surface you reach for as a real wallet service matures: -- **JWKS verification.** `JwksVerifier` is a drop-in `Verifier` for RS256 tokens - from an external IdP (Keycloak, Auth0, Cognito): `kid` cache, `iss`/`aud` - checks, `exp` required, and the same `sub`/`roles`/`permissions` claim mapping. -- **Method guards.** For per-handler checks, the `guards` module composes typed - predicates: `has_role("CUSTOMER").or(has_authority("wallet:approve"))`, then - `guard.authorize(Some(&auth))?` — 401 with no principal, 403 if the predicate - is false. For a declarative spelling, see [Method security](#method-security) - above. +- **JWKS verification.** `JwksVerifier::new("https://idp.example.com/.well-known/jwks.json")` + is a drop-in `Verifier` for RS256 tokens from an external IdP (Keycloak, + Auth0, Cognito): `kid` cache, `iss`/`aud` checks via `.issuer(..)` / + `.audience(..)`, `exp` required, and the same `sub`/`roles`/`permissions` claim + mapping. +- **Method guards.** For imperative per-handler checks, the `guards` module + composes typed predicates: `guards::has_role("CUSTOMER").or(guards::has_authority("wallet:approve"))`, + then `guard.authorize(Some(&auth))?` — `Unauthenticated` with no principal, + `Forbidden` if the predicate is false. For a declarative spelling, prefer the + [method-security macros](#step-5--push-authorization-down-to-a-method) above. - **Role hierarchy.** `RoleHierarchy::from_string("ADMIN > CUSTOMER")` parses the - spec; attach it with `chain.with_role_hierarchy(..)` so granting `ADMIN` implies - `CUSTOMER`. + spec; attach it with `chain.with_role_hierarchy(..)` so granting `ADMIN` + implies `CUSTOMER` everywhere the chain checks a role. - **Pattern rules.** Alongside the prefix rules Lumen uses, the chain offers an fnmatch-style glob DSL — `permit_pattern("/public/**")`, `require_pattern("/api/admin/**", &["ADMIN"])`, - `require_authority("/api/reports/**", &["reports:read"])`, + `require_authority("/api/reports/**", &["reports:read"])`, and `authenticated("/api/**")`. - **Sessions.** For browser flows where logout must mean logout, the `firefly-session` crate adds a `SessionLayer` over a `SessionStore` (`MemorySessionStore` for dev, a Redis-backed store for scale). A handler pulls the request's `Session` with the `SessionExt` extractor and calls - `session.rotate_id()` after login (session-fixation defense), - `session.set_attribute("user_id", id)`, and `session.invalidate()` on logout. -- **OAuth2.** The `oauth2` module covers both sides: `ClientRegistration` + - `OAuth2LoginHandler` for the authorization-code login flow (state + nonce + - PKCE S256, OIDC id-token validation), and an `AuthorizationServer` that issues - tokens for `client_credentials` / `refresh_token`. + `session.rotate_id().await` after login (session-fixation defense), + `session.set_attribute("user_id", &id).await`, and `session.invalidate().await` + on logout. +- **OAuth2.** The `oauth2` module covers both sides: `ClientRegistration` (with + `google` / `github` / `keycloak` presets) + `OAuth2LoginHandler` for the + authorization-code login flow (state + nonce + PKCE S256, OIDC id-token + validation), and an `AuthorizationServer` that issues tokens for + `client_credentials` / `refresh_token`. - **CSRF & passwords.** `CsrfLayer` implements the double-submit-cookie pattern for cookie-session flows; `BcryptPasswordEncoder` (default work factor 12) hashes credentials, and `Argon2PasswordEncoder` (Argon2id, OWASP defaults via @@ -531,6 +744,8 @@ surface you reach for as a real wallet service matures: bcrypt hashes and the self-describing `$argon2id$` PHC strings both interchange with the `firefly-idp-internal-db` adapter and every other port. +Both encoders share one trait, so they are interchangeable: + ```rust use firefly_security::{Argon2PasswordEncoder, BcryptPasswordEncoder, PasswordEncoder}; @@ -546,58 +761,69 @@ assert!(argon_hash.starts_with("$argon2id$")); assert!(argon.verify("s3cret", &argon_hash).unwrap()); ``` -## What changed in Lumen - -This chapter closed Lumen's open front door without adding a dependency or a -line of business logic to the handlers: - -- A single HS256 **`JwtService`** (`src/security.rs`) both mints the demo tokens - and verifies incoming ones; `encode` auto-stamps a one-hour `exp`, and a token - without `exp` is rejected at decode time. -- **`build_verifier`** turns the service into a `Verifier` via `VerifierFn`, - mapping `sub` → principal and `roles` → roles onto an `Authentication`, with a - bad token surfacing as `SecurityError::Verification` → a 401 problem. -- **`security_layers`** composes a `BearerLayer` (with `allow_anonymous(true)`) - and a path-ordered RBAC `FilterChain`: public `GET /api/v1/wallets/:id`, - public `/actuator/*`, and `CUSTOMER`-only `POST /api/v1/wallets`, - `/deposit`, `/withdraw`, and `/transfers`. -- **The `FilterChain` + `BearerLayer` `#[bean]`s** are auto-discovered and - layered by `FireflyApplication`: it sets the chain inside the - correlation/headers edge and layers the bearer auth on the outside — no - `with_security` / `apply_middleware` call in Lumen's code. -- **Method security** pushes authorization onto domain operations: - `#[firefly::pre_authorize(role = "CUSTOMER")]` guards before the body runs, - `#[firefly::post_authorize(result.owner == auth.principal)]` vets the return - object, and the `BearerLayer` installs the ambient context - (`with_authentication_scope` / `current_authentication` / `check_access`) so a - service method enforces the rule without ever seeing the `Request`. -- **Config-driven wiring** lets a deployed service read its posture from - `firefly.security.*`: `verifier_from_config` picks JWKS (RS256), HMAC, or none - by precedence, and `bearer_layer_from_config` hands back the ready-to-mount - `BearerLayer` — moving from the demo key to a production IdP with no code edit. -- The HTTP suite proves the contract: an unauthenticated mutation is a 401 - problem, the happy paths run as a minted `CUSTOMER`, and the unit tests - round-trip and reject tokens. +## Recap — what changed in Lumen + +This chapter closed Lumen's open front door without adding a dependency or a line +of business logic to the handlers: + +| Before | After this chapter | +|--------|--------------------| +| any caller could open/deposit/withdraw/transfer | mutating routes require a `CUSTOMER` JWT; reads and `/actuator/*` stay public | +| no token machinery | one HS256 `JwtService` mints and verifies, auto-stamping a one-hour `exp` | +| no authorization | a path-ordered, fail-closed RBAC `FilterChain` plus method-level `#[pre_authorize]` / `#[post_authorize]` | +| — | the `FilterChain` + `BearerLayer` `#[bean]`s, auto-discovered and layered by `FireflyApplication` — no `with_security` call | + +You also now know: + +- That `JwtService::encode` auto-stamps a one-hour `exp` and `decode` rejects any + token without one, so every credential is bounded. +- That `build_verifier` turns the service into a `Verifier` via `VerifierFn`, + mapping `sub` → principal and `roles` → roles, with a bad token surfacing as + `SecurityError::Verification` → a 401 problem. +- That `security_layers` composes a `BearerLayer` (with `allow_anonymous(true)`) + and a first-match-wins `FilterChain`, where rule order decides who is permitted. +- That declaring the chain and layer as `#[bean]`s is the *entire* wiring — the + framework sets the chain inside the correlation/headers edge and layers bearer + auth on the outside (authenticate, then authorize). +- That method security pushes authorization onto domain operations through an + ambient context the `BearerLayer` installs, so a service method enforces the + rule without ever seeing the `Request`. +- That `verifier_from_config` / `bearer_layer_from_config` move the whole posture + into `firefly.security.*`, so the demo key becomes a production IdP with no + code edit. ## Exercises -1. **Add an ADMIN-only route.** Give Lumen a hypothetical `GET - /api/v1/wallets` collection list and protect it with `require_pattern` so - only an `ADMIN` may list every wallet, while `CUSTOMER` keeps access to the - single-wallet read. Mint an `ADMIN` token in a test and assert a `CUSTOMER` - token gets a 403. +1. **Add an ADMIN-only route.** Give Lumen a hypothetical `GET /api/v1/wallets` + collection list and protect it with `require_pattern("/api/v1/wallets", + &["ADMIN"])` so only an `ADMIN` may list every wallet, while `CUSTOMER` keeps + access to the single-wallet read. Mint an `ADMIN` token in a test and assert a + `CUSTOMER` token gets a 403. 2. **Role hierarchy.** Introduce a `SUPER` role that implies `CUSTOMER`. Build a - `RoleHierarchy` from `"SUPER > CUSTOMER"`, attach it with + `RoleHierarchy::from_string("SUPER > CUSTOMER")`, attach it with `chain.with_role_hierarchy(..)`, mint a `SUPER`-only token, and assert it passes the `require("/api/v1/wallets", &["CUSTOMER"])` rule. 3. **Swap in JWKS.** Sketch a `build_verifier_jwks()` that returns a `JwksVerifier::new("https://idp.example.com/.well-known/jwks.json")` and - confirm (by reading the `Verifier` trait) that `security_layers` needs no - other change. Why does the rest of Lumen not care which verifier it got? + confirm (by reading the `Verifier` trait) that `security_layers` needs no other + change. Why does the rest of Lumen not care which verifier it got? 4. **Expiry.** Lower the token lifetime with `JwtService::new(KEY).expiration_seconds(1)`, mint a token, wait two seconds, and assert the verifier now returns `SecurityError::Verification`. +5. **Method security in isolation.** Decorate a plain function with + `#[firefly::pre_authorize(role = "CUSTOMER")]`, call it from a `#[test]` + *without* a scope and assert `Unauthenticated`, then wrap the call in + `firefly::security::with_authentication_scope_sync(auth, || ...)` with a + `CUSTOMER` auth and assert it passes. + +## Where to go next A secure service is only trustworthy if you can *see* what it is doing. The next chapter gives Lumen eyes and ears — structured logs, health, metrics, and the -admin dashboard. Continue to [Observability](./15-observability.md). +admin dashboard. + +- Make Lumen observable in **[Observability](./15-observability.md)** — the + management surface beside the security perimeter you just built. +- Revisit how the framework discovers and wires beans like the `FilterChain` in + **[Dependency Wiring](./04-dependency-wiring.md)**. +- Drive the wired router in tests with `bootstrap()` in **[Testing](./18-testing.md)**. diff --git a/docs/book/src/15-observability.md b/docs/book/src/15-observability.md index 6cc989d..1c8c076 100644 --- a/docs/book/src/15-observability.md +++ b/docs/book/src/15-observability.md @@ -1,69 +1,117 @@ # Observability -In Chapter 14 you locked Lumen's mutating routes behind a JWT and a role filter. -The service is now safe — but it is still a black box. When a deposit is slow in -production you need to know *where* the time went; when the broker degrades you -want a dashboard that turns red before your pager does; when an auditor asks why -a transfer was rejected you need a structured log line with the context to -reconstruct the decision. - -By the end of this chapter Lumen will expose an **actuator admin surface** on a -separate port — health, info, metrics, loggers, scheduled tasks — feed it an -**info contributor** describing the build, emit **structured logs** with a -correlation id on every request, and surface all of it in the **self-hosted admin -dashboard** with its fifteen views, including the **Beans** view. Lumen is -**observable by default**, and `FireflyApplication` does the wiring: it installs -the logging layer, the health composite, the metric registry, the request-metrics +In [Security](./14-security.md) you locked Lumen's mutating routes behind a JWT +and a role filter. The service is now safe — but it is still a black box. When a +deposit is slow in production you need to know *where* the time went; when the +broker degrades you want a dashboard that turns red before your pager does; when +an auditor asks why a transfer was rejected you need a structured log line with +enough context to reconstruct the decision. + +The good news — and the theme of this whole chapter — is that almost none of +this is code you write. `FireflyApplication::run()` already installed the logging +layer, the health composite, the metric registry, the request-metrics middleware, W3C trace-context origination, and a self-hosted `/admin` dashboard -bound to the live components — all with no observability code in `main.rs`. This -chapter explains what each piece reports. +bound to the live components. Lumen has been observable since +[Your First HTTP API](./06-first-http-api.md); this chapter teaches you to *read* +what is already there, and to add the handful of optional pieces — an info +contributor, a health probe, a domain metric, a custom dashboard view — that only +your application can supply. + +By the end of this chapter you will: + +- Reach Lumen's **management surface** — `/actuator/*` on the management port — + and explain why it lives on a separate listener from the public API. +- Register an **info contributor** so `/actuator/info` reports which + infrastructure this instance is running. +- Add a **health indicator** to the composite and watch it roll up into + `/actuator/health`. +- Read the **request metrics** the framework already records, and record your own + counter and gauge on the same registry. +- Understand **structured, correlation-enriched logging** and **W3C + trace-context** — how one request stitches together across logs, events, and + outbound calls with no manual threading. +- Open the **self-hosted admin dashboard**, read its fifteen views including the + populated **Beans** view, and add a custom view of your own. + +## Concepts you will meet + +Each idea below is reintroduced in context where it is first used; this is the +short version so the vocabulary is in place before the first command. + +> **Note** **Key term — management surface / actuator.** The *management surface* +> is a set of operational HTTP endpoints — health checks, build info, metrics, +> configuration introspection, runtime log-level control — that exist for +> operators and tooling, not for end users. Firefly serves them under +> `/actuator/*` on a **separate port** from your business API. This mirrors +> Spring Boot Actuator. + +> **Note** **Key term — info contributor.** An *info contributor* is a small +> callback that adds a JSON section to `/actuator/info`. You register it on the +> application builder; the framework calls it when an operator hits the endpoint. +> The Spring analog is an `InfoContributor` bean. + +> **Note** **Key term — health indicator and composite.** A *health indicator* is +> an async probe that reports `UP` / `DEGRADED` / `DOWN` (with an optional message +> and details). A *composite* aggregates many indicators into one rollup and +> serves it at `/actuator/health`. This is Spring Boot's `HealthIndicator` plus +> its health aggregator. + +> **Note** **Key term — correlation id.** A *correlation id* is one identifier +> attached to everything a single request touches — every log line, every event +> it publishes, every outbound call it makes — so you can reconstruct the whole +> story from one value. Firefly sets it in a task-local scope on the way in; the +> Spring analog is an MDC entry threaded through a request. + +## The two ports, and what each serves + +Before the first endpoint, fix the mental model. Lumen runs **two listeners**: + +- the **public API** on `0.0.0.0:8080` — your business routes and nothing else; +- the **management surface** on `0.0.0.0:8081` — `/actuator/*` plus the + self-hosted `/admin` dashboard plus the auto-generated API docs. -> **An `/actuator` management surface.** Firefly exposes a JSON management -> surface under `/actuator/*` — health, info, metrics, env, loggers, scheduled -> tasks — with structured, correlation-enriched logs and an embedded admin -> dashboard. The logger endpoint reports `configuredLevel` / `effectiveLevel`, -> the conventional shape standard metrics and management tooling expects, so -> off-the-shelf scrapers and dashboards work unchanged. +`FireflyApplication` assembles and serves both routers; Lumen writes no actuator +or admin wiring at all. The split is the point: an operational endpoint like +`/actuator/env` (which can echo configuration) or `/admin` (a live dashboard) +never leaks onto the public network, because the public listener simply does not +mount those paths. -## The management surface, self-hosted +> **Note** **Key term — bind address override.** `FIREFLY_SERVER_ADDR` and +> `FIREFLY_MANAGEMENT_ADDR` are the two environment variables that move the public +> and management listeners independently (defaulting to `0.0.0.0:8080` / +> `0.0.0.0:8081`). You met them in [Quickstart](./02-quickstart.md); they are how +> you put the management port on a private interface in production. -Lumen serves the public API on `0.0.0.0:8080` and the **management surface** -(actuator + the admin dashboard) on `0.0.0.0:8081` — a separate listener, so -`/actuator/*` and `/admin/*` never leak onto the public network. -`FireflyApplication` assembles and serves both routers; Lumen writes no actuator -or admin wiring at all. (Override the bind addresses with `FIREFLY_SERVER_ADDR` / -`FIREFLY_MANAGEMENT_ADDR`.) +> **Tip** **Checkpoint.** With Lumen running (`cargo run --bin lumen`), the +> management surface answers on `8081` and the public API on `8080`. Confirm the +> split: `curl localhost:8081/actuator/health` returns JSON, while +> `curl localhost:8080/actuator/health` returns a 404 problem document — the +> actuator is not on the public port. -The one piece of observability *application* code Lumen could add is an -`/actuator/info` contributor, registered fluently on the application builder: +## Step 1 — Reach the actuator -```rust,ignore -use firefly::starter_core::InfoContributor; +Even with no observability code of your own, the actuator is live. Start Lumen, +then from a second terminal walk three endpoints. -let contributor: InfoContributor = Box::new(|| { - let mut info = serde_json::Map::new(); - info.insert( - "sample".into(), - serde_json::json!({ "name": "lumen", "store": "in-memory", "eventBus": "in-memory" }), - ); - info -}); +```bash +curl localhost:8081/actuator/health +# {"status":"UP", ...} -firefly::FireflyApplication::new("lumen") - .info_contributor(contributor) // adds a section to /actuator/info - .run() - .await -``` +curl localhost:8081/actuator/info +# {"app":{"name":"lumen","version":"26.6.28"}, ...} -The framework builds the management router (`/actuator/*` + the self-hosted -`/admin` dashboard) from the live components, threads your info contributors into -`/actuator/info`, and serves it on the management port — no `actuator_router(..)` -call, no second-listener bookkeeping in app code. +curl localhost:8081/actuator/metrics +# {"names":[ "http_server_requests_seconds", ... ]} +``` -## The actuator endpoints +What just happened: `/actuator/health` aggregated every registered health +indicator into one `status`; `/actuator/info` echoed the app name you passed to +`FireflyApplication::new("lumen")` plus the framework version; `/actuator/metrics` +listed the meters the framework has been recording since boot — including the +per-route request timer you will read in Step 4. -`actuator_router` exposes the management endpoints below. Lumen's reach the admin -port at `http://127.0.0.1:8081/actuator/*`: +The full management surface is below. Lumen's reaches the management port at +`http://localhost:8081/actuator/*`: | Endpoint | Returns | |--------------------------------|----------------------------------------------------| @@ -77,166 +125,313 @@ port at `http://127.0.0.1:8081/actuator/*`: | `/actuator/version` | the running version | | `/actuator/beans` | every DI bean (type, scope, stereotype, primary) | | `/actuator/mappings` | every `#[rest_controller]` route (method/path) | -| `/actuator/conditions` | the `@Profile`/`@ConditionalOn…` guards per bean | +| `/actuator/conditions` | the conditional guards per bean | | `/actuator/loggers[/:name]` | runtime log-level control | | `/actuator/threaddump` | a thread/task dump | | `/actuator/httpexchanges` | recent HTTP exchanges (when wired) | | `/actuator/caches[/:name]` | cache listing + eviction (when wired) | -| `/actuator/refresh` | reload config (the `ReloadableConfig` hook) | +| `/actuator/refresh` | reload config (the `Refresher` hook) | + +> **Note** The `beans` / `mappings` / `conditions` reports mirror Spring Boot +> Actuator's dependency-injection introspection — they are auto-registered by the +> framework alongside the rest, so you can introspect the wired object graph over +> HTTP without any app code. You saw the same inventory printed at boot in +> [Quickstart](./02-quickstart.md); these endpoints serve it live. + +> **Tip** **Checkpoint.** All three `curl`s above return JSON. If `curl` connects +> but every path 404s, you are hitting `8080` (public) instead of `8081` +> (management). The public port has no `/actuator/*`. + +## Step 2 — Describe this instance with an info contributor + +`/actuator/info` already reports the `app` block — name and version — but it +cannot know what *infrastructure* this particular instance is running. That is +application knowledge, so it is the one piece of observability code Lumen could +add. You supply it as an **info contributor** registered fluently on the +application builder. + +> **Note** **Key term — `InfoContributor`.** The type is +> `Box serde_json::Map + Send + Sync>` — a boxed +> closure that returns a JSON object. Each contributor's map becomes one section +> of `/actuator/info`. The closure runs on every request to the endpoint, so it +> can report live values. + +```rust,ignore +use firefly::starter_core::InfoContributor; + +let contributor: InfoContributor = Box::new(|| { + let mut info = serde_json::Map::new(); + info.insert( + "sample".into(), + serde_json::json!({ "name": "lumen", "store": "in-memory", "eventBus": "in-memory" }), + ); + info +}); + +firefly::FireflyApplication::new("lumen") + .info_contributor(contributor) // adds a "sample" section to /actuator/info + .run() + .await +``` -### The info contributor +What just happened, block by block: -An `InfoContributor` is a boxed closure returning a `serde_json::Map`; each one -adds a section to `/actuator/info`. Lumen's reports its store and event-bus kind, -so an operator hitting `/actuator/info` sees that this instance is running the -in-memory infrastructure: +- `InfoContributor` is re-exported through the facade at + `firefly::starter_core::InfoContributor`, so Lumen still depends on only the one + `firefly` crate. +- The closure builds a `serde_json::Map` with a single key, `sample`, whose value + describes the store and event-bus kind this instance is running. +- `.info_contributor(contributor)` registers it on the builder. The framework + threads every registered contributor into the `/actuator/info` handler when it + builds the management router — no `actuator_router(..)` call and no + second-listener bookkeeping in your code. + +After this, `/actuator/info` reports both blocks: ```jsonc // GET /actuator/info { - "app": { "name": "lumen", "version": "26.6.24" }, + "app": { "name": "lumen", "version": "26.6.28" }, "sample": { "name": "lumen", "store": "in-memory", "eventBus": "in-memory" } } ``` -The `app` block is filled from the `CoreConfig.app_name` / `app_version` Lumen -set in Chapter 3; the `sample` block is the contributor above. - -### Health - -A health `Indicator` is an async probe returning a `HealthResult` (status + -message + details); a `Composite` aggregates them into the canonical rollup — -`DOWN` if any indicator is `DOWN`, else `DEGRADED` if any is `DEGRADED`, else -`UP`. The framework's `Core` already carries a `HealthComposite`; you bridge an -observability indicator onto it with `core.add_observability_indicator(..)`. A -real Lumen deployment would add a broker-liveness indicator and a store probe — -the cleanest place is to declare the indicator as a `#[bean]` (the framework -discovers it), or to reach the composite through a `FireflyApplication::on_ready` -hook: +The `app` block is filled from the `app_name` / `app_version` Lumen set in +[Configuration](./03-configuration.md); the `sample` block is the contributor +above. An operator hitting `/actuator/info` now sees at a glance that this +instance is on the in-memory infrastructure, not Postgres + Kafka. + +> **Tip** **Checkpoint.** After adding the contributor and re-running, +> `curl localhost:8081/actuator/info` shows a top-level `sample` object reporting +> `"store":"in-memory"`. You can register more than one contributor; their maps +> are merged into the same JSON document. + +## Step 3 — Add a health indicator + +The composite that backs `/actuator/health` starts with the framework's own +indicators. A real Lumen deployment would add its own — a broker-liveness probe, +a store reachability check — so an orchestrator can tell a degraded instance from +a healthy one. + +> **Note** **Key term — `IndicatorFn`.** `IndicatorFn::new(name, closure)` adapts +> a plain async closure into a health `Indicator`. The closure returns a +> `HealthResult` — `HealthResult::up()`, `HealthResult::degraded(msg)`, or +> `HealthResult::down(msg)`, each optionally enriched with `.with_detail(..)`. The +> composite rolls the results up: `DOWN` if any indicator is `DOWN`, else +> `DEGRADED` if any is `DEGRADED`, else `UP`. + +The framework's `Core` already carries the `HealthComposite`. You bridge an +indicator onto it with `Core::add_observability_indicator(..)`. There are two +clean places to do it: declare the indicator as a `#[bean]` (the framework +discovers it during the component scan), or reach the composite from a +`FireflyApplication::on_ready` hook after the container is scanned. The hook form +looks like this: ```rust,ignore -use firefly_observability::{HealthResult, IndicatorFn}; +use firefly::observability::{HealthResult, IndicatorFn}; -// Inside an on_ready hook, the Core's composite is reachable from the web stack: +// `core` is the wired Core (it owns the HealthComposite). Registering an +// indicator on it makes the probe appear under /actuator/health. core.add_observability_indicator(IndicatorFn::new("event-bus", || async { - HealthResult::up() // a real probe would ping the broker + HealthResult::up() // a real probe would ping the broker and return down() on failure })); ``` -`/actuator/health/liveness` and `/actuator/health/readiness` are the sub-paths -your orchestrator's probes hit — separate so an in-flight migration that fails -readiness need not trigger a liveness restart. +What just happened: `IndicatorFn::new("event-bus", ..)` wraps the async closure +as an `Indicator` named `event-bus`; `add_observability_indicator` registers it +on the composite. The next `/actuator/health` call runs every indicator +concurrently and folds the results into the overall `status`, listing your probe +by name in the per-component breakdown. + +> **Note** Health exposes two sub-paths your orchestrator's probes hit: +> `/actuator/health/liveness` and `/actuator/health/readiness`. They are separate +> so an in-flight migration that fails *readiness* (don't send me traffic yet) +> need not trip *liveness* (kill and restart me). Returning `degraded` keeps a +> probe `UP` while still flagging trouble on the rollup. + +> **Tip** **Checkpoint.** After wiring an indicator, `curl +> localhost:8081/actuator/health` shows your probe's name alongside `status`. Make +> the closure return `HealthResult::down("broker unreachable")` once and watch the +> overall `status` flip to `DOWN` — that is the precedence rule in action. + +## Step 4 — Read the request metrics you already have + +You did not have to ask for per-route latency: request metrics are +auto-instrumented **on by default**, both at the `Core` layer (so even a bare +`Core` emits them) and through the web stack, which fills in a default +`RequestMetricsConfig` if you left one unset. + +> **Note** **Key term — request metrics.** For every request the middleware +> records the labeled timer `http_server_requests_seconds` plus a companion +> `…_max` gauge, tagged `method` / templated `uri` (the matched route, so +> `/api/v1/wallets/:id` not the concrete id) / `status` / `outcome` / +> `exception`. A clean request carries `exception="None"`. This is the +> Micrometer/Spring Boot convention, so off-the-shelf scrapers read it unchanged. + +Because the meter has been recording since the moment Lumen booted in +[Your First HTTP API](./06-first-http-api.md), this chapter only *exposes* it. Hit +a route a few times, then read the meter: + +```bash +curl localhost:8080/api/v1/wallets/$ID # generate some traffic +curl localhost:8081/actuator/metrics/http_server_requests_seconds +``` -## Request metrics — already on +The dot-and-underscore meter name maps straight to Prometheus, so pointing a +Prometheus `scrape_config` at `/actuator/prometheus` lights up Grafana with no +extra code: -Request metrics are auto-instrumented **on by default** — at the `Core` layer -(so even a bare `Core` emits them) and through `WebStack::new`, which fills in a -`RequestMetricsConfig` if you left one unset. For every request the middleware -records the labeled `http_server_requests_seconds` timer plus a companion `_max` -gauge, tagged `method` / templated `uri` (the axum matched path, so -`/api/v1/wallets/:id` not the concrete id) / `status` / `outcome` / `exception`, -and bridges them onto the actuator `MetricRegistry`. A clean request carries -`exception="None"`. So the moment Lumen booted in Chapter 6 it was already -emitting per-route latency; this chapter just exposes it at `/actuator/metrics`. +```bash +curl localhost:8081/actuator/prometheus | grep http_server_requests_seconds +``` -> **Note.** To turn the auto-instrumentation off, set -> `CoreConfig { disable_request_metrics: true, .. }`; to tune the rolling-max -> window or path exclusions, supply a `request_metrics: Some(RequestMetricsConfig { .. })`. +> **Note** To turn the auto-instrumentation off, set +> `CoreConfig { disable_request_metrics: true, .. }`. To tune the rolling-max +> window or path exclusions instead of disabling, supply +> `request_metrics: Some(RequestMetricsConfig { .. })`. Both are configured the +> same way you configured everything else in [Configuration](./03-configuration.md). -The dot-case meter names map straight to Prometheus -(`http_server_requests_seconds`), so pointing a Prometheus `scrape_config` at -`/actuator/prometheus` lights up Grafana with no extra code. +### Recording your own meters -Beyond the request timer, you record your own meters on the same registry. Pull -it off the `Core` with `metric_registry()` (the registry is also a resolvable -bean), then increment a counter or set a gauge — both surface at -`/actuator/metrics` and `/actuator/prometheus` immediately: +Beyond the request timer, you record domain meters on the **same** registry, so +they surface at `/actuator/metrics` and `/actuator/prometheus` immediately. Pull +the registry off the `Core` with `metric_registry()` (it is also a resolvable DI +bean you can `#[autowired]`), then create a counter or a gauge: ```rust,ignore let metrics = core.metric_registry(); // A domain counter, bumped each time the transfer saga completes. -metrics.counter("lumen_transfers_total").inc(); // or .add(1) for an explicit count +let transfers = metrics.counter("lumen_transfers_total"); +transfers.inc(); // or transfers.add(3) for an explicit count // A gauge sampling a live value (e.g. wallets currently held in the read model). -metrics.gauge("lumen_wallets_active").set(wallet_count as f64); +let active = metrics.gauge("lumen_wallets_active"); +active.set(wallet_count as f64); ``` -## Structured logging and correlation +What just happened: `counter(name)` and `gauge(name)` return an +`Arc` / `Arc` registered under that name. `Counter::inc()` adds +one (`add(n)` adds an explicit count); `Gauge::set(v)` records a sampled value. +Both meters now appear in the listing and the Prometheus scrape without any +exporter wiring. + +> **Tip** **Checkpoint.** After incrementing `lumen_transfers_total` and reading +> `/actuator/metrics`, the meter listing includes `lumen_transfers_total`; the +> Prometheus scrape shows its current count. The registry is shared, so your +> domain meters and the framework's request timer live side by side. + +## Step 5 — Structured logging and correlation `FireflyApplication` installs a `tracing` layer that formats every event as one structured line and enriches it with the request's correlation id (set by the -correlation middleware, on by default). It calls `init_logging` for you at boot -(best-effort, so a test harness that already owns the global subscriber does not -panic) and — with the `admin` feature on — tees the records into the dashboard's -live log buffer: +correlation middleware, on by default). It calls `init_logging` for you at boot — +best-effort, so a test harness that already owns the global subscriber does not +panic — and, with the `admin` feature on, tees the records into the dashboard's +live log buffer. + +> **Note** **Key term — `init_logging`.** `init_logging(LogConfig)` installs the +> structured `tracing` subscriber as the global default. Its sibling +> `init_logging_with_layers([..])` does the same but stacks extra `tracing` layers +> over the correlation layer — the hook the admin dashboard uses to tee every log +> record into its in-memory buffer while the console JSON stream keeps flowing. +> You never call either yourself; the framework does it at boot. ```rust,ignore // What FireflyApplication does at boot — Lumen writes none of this: -let _ = web.init_logging(); // (or init_logging_with_layers([log_buffer]) for the admin tail) +let _ = web.init_logging(); +// (or web.init_logging_with_layers(vec![log_buffer]) when the admin tail is on) ``` -After that, plain `tracing` macros produce enriched lines; fields recorded on an -enclosing span merge into each event. The field names (`time`, `level`, `msg`, -`service`, `correlationId`) follow a stable, documented schema, so one log -pipeline parses every Firefly service consistently. +After that, plain `tracing` macros produce enriched lines, and fields recorded on +an enclosing span merge into each event: + +```rust,ignore +tracing::info!(wallet_id = %id, amount = %money, "deposit accepted"); +``` + +The field names (`time`, `level`, `msg`, `service`, `correlationId`) follow a +stable, documented schema, so one log pipeline parses every Firefly service +consistently. Because the correlation id lives in a task-local scope, it flows automatically into every log line, every event Lumen's ledger publishes (`Event::new` stamps it), and every outbound client call (the W3C `traceparent` is propagated). A request that opens a wallet, publishes `WalletOpened`, and projects it into the -read model stitches together under one id with no manual threading. - -> **Correlation flows automatically.** `init_logging` installs a structured -> `tracing` subscriber; the task-local correlation id is attached to every log -> line in place of manual thread-local plumbing, so a request stitches together -> across logs, events, and outbound calls with no field threading. +read model stitches together under one id with no manual threading — the +task-local correlation id stands in for the thread-local MDC plumbing you would +write by hand in other stacks. ### Configuring logging Logging is configured the way you configure everything else — from the one main config file. Bind the `firefly.logging.*` section into a `LogConfig` with -`firefly_observability::log_config_from_properties(props, base)`: +`firefly::observability::log_config_from_properties(props, base)`: ```yaml firefly: logging: - format: json # json | text (logfmt) | console - level: # one level map (like logging.level.) - root: info # root level - firefly_web: warn # per-logger levels - app::ledger: trace + format: json # json (default) | text (logfmt) | console + level: info # root level + level.firefly_web: warn # per-logger levels (Spring's logging.level.) + level.app::ledger: trace + service: lumen # the `service` field stamped on every line file: name: lumen.log # enable the rolling file appender max-size: 10MB max-history: 7 ``` -Per-logger levels, the output format, and the rolling file appender all come -from config; an external logging file can be folded in with -`apply_external_config`. And every level can be changed **without a restart** -through `POST /actuator/loggers/{name}` — the actuator's runtime logger control. - -## Tracing / OpenTelemetry - -`firefly-observability` exposes the building blocks that compose with the -`tracing` ecosystem and propagates W3C trace-context (`traceparent` / -`tracestate`) on the HTTP edges and outbound client calls. The default middleware -chain `FireflyApplication` applies includes the `TraceContextLayer`, which -**originates** trace context: it validates an inbound `traceparent` / `tracestate` -and, when one is absent, *mints a W3C root span* (a 32-hex trace-id and a 16-hex -span-id), inserts it into the request, and enriches every log line with -`trace_id` / `span_id`. So a request that arrives with no trace header still -leaves Lumen as the head of a well-formed distributed trace. The OpenTelemetry SDK -wiring — exporters, sampling, resource attributes — is left to your application, -where you add your preferred OTel `tracing` layer alongside the correlation -layer. Lumen ships without an exporter (it is teaching code with no external -collector), but the trace-context origination + propagation is already on the -edges. - -When you do want spans flowing to a collector, build an OTLP tracer and add -`tracing-opentelemetry`'s layer to the subscriber Firefly installed — the -correlation layer keeps working alongside it: +What these keys do: `format` picks the output renderer; the bare `level` is the +root level, and `level.` overrides one logger (matching Spring's +`logging.level.`); `service` is stamped on every line; the `file` block +switches on the rolling file appender and tunes its rotation. Per-logger levels, +the output format, and the rolling file appender therefore all come from config. +An external logging file can additionally be folded in with +`apply_external_config`. + +And every level can be changed **without a restart** through +`POST /actuator/loggers/{name}` — the actuator's runtime logger control. The +endpoint reports each logger's `configuredLevel` / `effectiveLevel`, the +conventional shape management tooling expects: + +```bash +# Raise app::ledger to TRACE on a running instance, no redeploy. +curl -X POST localhost:8081/actuator/loggers/app::ledger \ + -H 'content-type: application/json' \ + -d '{"configuredLevel":"TRACE"}' +``` + +> **Tip** **Checkpoint.** `curl localhost:8081/actuator/loggers` lists every +> logger with its `configuredLevel` / `effectiveLevel`. POST a new level to one +> logger, GET it back, and confirm the level changed on the live process. + +## Step 6 — Trace context and OpenTelemetry + +The default middleware chain `FireflyApplication` applies includes the +`TraceContextLayer`, which **originates** distributed trace context on every +request. + +> **Note** **Key term — W3C trace context.** `traceparent` / `tracestate` are the +> standard HTTP headers that carry a distributed trace across service boundaries: +> a 32-hex trace-id and a 16-hex span-id identify where the request sits in a +> larger call tree. *Originating* means: when an inbound request carries no +> `traceparent`, the layer mints a fresh root span so the request still leaves +> Lumen as the head of a well-formed trace. + +So the layer validates an inbound `traceparent` / `tracestate` when present and +mints a W3C root span when absent, inserts it into the request, and enriches every +log line with `trace_id` / `span_id`. A request that arrives with no trace header +still becomes the head of a distributed trace, and the `traceparent` Lumen +propagates on outbound calls becomes the parent/child edge to the next service. + +The OpenTelemetry SDK wiring — exporters, sampling, resource attributes — is left +to your application, where you add your preferred OTel `tracing` layer alongside +the correlation layer. Lumen ships without an exporter (it is teaching code with +no external collector), but the trace-context origination and propagation are +already on the edges. When you do want spans flowing to a collector, build an OTLP +tracer and add `tracing-opentelemetry`'s layer to the subscriber Firefly +installed — the correlation layer keeps working alongside it: ```rust,ignore use opentelemetry_otlp::WithExportConfig; @@ -245,7 +440,11 @@ use tracing_subscriber::prelude::*; // Build an OTLP pipeline pointing at your collector. let tracer = opentelemetry_otlp::new_pipeline() .tracing() - .with_exporter(opentelemetry_otlp::new_exporter().tonic().with_endpoint("http://otel-collector:4317")) + .with_exporter( + opentelemetry_otlp::new_exporter() + .tonic() + .with_endpoint("http://otel-collector:4317"), + ) .install_batch(opentelemetry_sdk::runtime::Tokio)?; // Register the OTel layer alongside Firefly's structured-log + correlation layers. @@ -258,19 +457,26 @@ The `traceparent` headers Firefly already propagates become the parent/child edges between spans, so a request that fans out to an outbound call appears as a single distributed trace in your backend. -## Global exception advice +## Step 7 — Global exception advice (optional) Lumen's errors already render as RFC 9457 `application/problem+json` at the -handler boundary. For a *cross-cutting* rewrite — mapping a whole class of errors -to a custom status or body without touching each handler — the framework offers a -transparent global advice layer, the Rust analog of Spring's `@ControllerAdvice`. -Register an `ExceptionHandlerRegistry` bean and `FireflyApplication` installs an -`ExceptionAdviceLayer` as the outermost layer, post-processing every -`application/problem+json` response through your registered transforms: +handler boundary — you saw that from the very first endpoint in +[Your First HTTP API](./06-first-http-api.md). For a *cross-cutting* rewrite — +mapping a whole class of errors to a custom status or body without touching each +handler — the framework offers a transparent global advice layer. + +> **Note** **Key term — global exception advice.** A registry of transforms that +> post-process every `application/problem+json` response after the handler +> produces it — the Rust analog of Spring's `@ControllerAdvice`. You register the +> registry as a `#[bean]`; the framework installs an `ExceptionAdviceLayer` as the +> outermost layer only when the registry is non-empty, so a service that declares +> no such bean keeps the plain RFC 9457 path. + +Register an `ExceptionHandlerRegistry` bean and key transforms by problem type: ```rust,ignore -use firefly_web::{ExceptionHandlerRegistry}; -use firefly_kernel::{ProblemDetail, TYPE_NOT_FOUND}; +use firefly::web::ExceptionHandlerRegistry; +use firefly::kernel::{ProblemDetail, TYPE_NOT_FOUND}; // A #[bean] returning a registry: every "not found" becomes a friendlier 410. #[bean] @@ -283,55 +489,45 @@ fn exception_advice(&self) -> ExceptionHandlerRegistry { } ``` -The framework only installs the layer when the registry is non-empty, so a -service that declares no such bean keeps the plain RFC 9457 path. Controller-local -overrides still win over the global rules. +What just happened: `on_type(TYPE_NOT_FOUND, transform)` registers a closure that +receives the produced `ProblemDetail` and returns a rewritten one — here flipping +the status from 404 to 410 (`Gone`). The framework runs the matching transform on +every outgoing problem document. Controller-local overrides still win over the +global rules, so a handler can opt out of the cross-cutting rewrite. + +> **Tip** **Checkpoint.** With the bean registered, request a missing wallet and +> confirm the response status is now `410` while the body stays a valid +> `application/problem+json` document. Remove the bean and the status returns to +> the default `404` — proof the layer only installs when the registry is non-empty. -## The admin dashboard +## Step 8 — The self-hosted admin dashboard The actuator surface is JSON for machines. `firefly-admin` mounts a single-page admin dashboard — vendored, no `npm` build — that ties health, metrics, loggers, -beans, mappings, caches, CQRS handlers, sagas, traces, and a live log tail into -one pane of glass with Server-Sent-Event streams. With the facade's `admin` -feature enabled, **`FireflyApplication` self-hosts it on the management port** and -binds it to the live components: the health composite, the metric registry, the -CQRS bus, the scheduler, the DI container (Beans), an environment snapshot built -from the active profiles and the `FIREFLY_*` process environment, a trace buffer -fed by the HTTP-exchanges recorder, and a log buffer fed by the tee'd logging -layer. The `env` / `config` / `mappings` panels show **real data**, not stubs. -Lumen writes none of this wiring — it ships the dashboard on `/admin/` simply by -being a `FireflyApplication`. - -> **Advanced: standalone mount.** The dashboard router is also reachable directly -> when you want to host it outside `FireflyApplication` (a custom server, a test). -> `mount(AdminConfig, AdminDeps)` returns the router; `AdminDeps::new` takes the -> required collaborators and the rest are optional fields filled with struct-update -> syntax: -> -> ```rust,ignore -> use std::sync::Arc; -> use firefly::admin::{mount, AdminConfig, AdminDeps, LogBuffer, TraceBuffer}; -> -> let deps = AdminDeps { -> scheduler: Some(scheduler), // → Scheduled Tasks view -> bus: Some(bus), // → CQRS view -> container: Some(container), // → Beans view -> ..AdminDeps::new( -> "lumen", VERSION, -> health_composite, // Arc -> metric_registry, // Arc -> Arc::new(TraceBuffer::new()), -> LogBuffer::new(), -> ) -> }; -> let dashboard = mount(AdminConfig::default(), deps); -> ``` -> -> `FireflyApplication` performs exactly this mount for you, sourcing every -> collaborator from the live web stack and the scanned container. +beans, mappings, caches, CQRS handlers, traces, and a live log tail into one pane +of glass with Server-Sent-Event streams. + +> **Note** **Key term — self-hosted dashboard.** The dashboard is a vanilla-JS +> single-page app served by the framework itself on the management port — there is +> no separate frontend service to deploy and no build step. With the facade's +> `admin` feature enabled, `FireflyApplication` mounts it on `/admin/` and binds it +> to the live components. + +With the `admin` feature on, **`FireflyApplication` self-hosts it on the +management port** and binds it to the real collaborators: the health composite, the +metric registry, the CQRS bus, the scheduler, the DI container (which backs the +Beans view), an environment snapshot built from the active profiles and the +`FIREFLY_*` process environment, a trace buffer fed by the HTTP-exchanges +recorder, and a log buffer fed by the tee'd logging layer. The `env` / `config` / +`mappings` panels show **real data**, not stubs. Lumen writes none of this wiring +— it ships the dashboard on `/admin/` simply by being a `FireflyApplication`. + +```bash +cargo run --bin lumen --features admin +# then open http://localhost:8081/admin/ in a browser +``` -The dashboard auto-discovers what those collaborators expose and renders it in -**fifteen built-in views**, grouped in the sidebar: +The dashboard renders fifteen built-in views, grouped in the sidebar: | Section | Views | |----------------|----------------------------------------------------------------| @@ -344,47 +540,54 @@ The dashboard auto-discovers what those collaborators expose and renders it in Each view is backed by a `/admin/api/*` JSON endpoint; the SSE streams (`/admin/api/sse/{health,metrics,traces,logfile,beans,runtime,server}`) push updates without your code polling. Admin and actuator paths are excluded from -trace capture so they never pollute the trace panel. +trace capture, so they never pollute the HTTP Traces panel. ### The Beans view -The newest view is **Beans** — the dashboard's window onto the dependency-injection -container. `FireflyApplication` always passes the scanned container, so the +The **Beans** view is the dashboard's window onto the dependency-injection +container. Because `FireflyApplication` always passes the scanned container, the dashboard serves: -| Endpoint | Returns | -|---------------------------|----------------------------------------------------------| -| `GET /admin/api/beans` | every registered bean with its stereotype and scope | -| `GET /admin/api/beans/graph` | the dependency graph between beans | -| `GET /admin/api/beans/:name` | one bean's detail (type, scope, dependencies) | -| `GET /admin/api/sse/beans` | a live snapshot at each refresh interval | +| Endpoint | Returns | +|------------------------------|------------------------------------------------------| +| `GET /admin/api/beans` | every registered bean with its stereotype and scope | +| `GET /admin/api/beans/graph` | the dependency graph between beans | +| `GET /admin/api/beans/:name` | one bean's detail (type, scope, dependencies) | +| `GET /admin/api/sse/beans` | a live snapshot at each refresh interval | The Overview view also rolls up a `beans` block (`{ total, stereotypes }`) and a `wiring` block (live CQRS-handler and scheduled-task counts) drawn from the same container, so the landing page shows how much the service is wired without opening -the full Beans view. Lumen's Beans view is **populated**, not sparse: the -framework component-scans the `LumenBeans` configuration, so the event store, read -model, query cache, JWT service, the `FilterChain` / `BearerLayer`, the ledger -application service, and the `WalletApi` controller all appear as beans with their -stereotypes and the autowired dependencies between them. (Were you to host the -dashboard standalone without a container, the endpoints degrade gracefully to an -empty `{ total: 0 }` block.) - -> **Beans and server mode.** The Beans view is the dashboard's window onto the -> DI container. `firefly-admin` also runs in *server mode* (`AdminServerConfig`): -> instances self-register through an `AdminClient`, and the server aggregates a -> fleet of services into the Instances view. The dashboard is a vanilla-JS SPA -> driven entirely by the `/admin/api` JSON + SSE endpoints — no frontend build -> step. - -### Custom views +the full Beans view. + +Lumen's Beans view is **populated**, not sparse: the framework component-scans the +configuration that declares Lumen's beans, so the event store, read model, query +cache, JWT service, the `FilterChain` / `BearerLayer`, the ledger application +service, and the `WalletApi` controller all appear as beans with their stereotypes +and the autowired dependencies between them. (Were you to host the dashboard +standalone without a container, these endpoints degrade gracefully to an empty +`{ "total": 0 }` block.) + +> **Note** `firefly-admin` also runs in *server mode*: instances self-register +> through an admin client, and a central server aggregates a fleet of services +> into the Instances view. The dashboard is the same vanilla-JS SPA driven +> entirely by the `/admin/api` JSON + SSE endpoints — there is no frontend build +> step in either mode. + +> **Tip** **Checkpoint.** With `--features admin`, open `http://localhost:8081/admin/`, +> select **Beans**, and find the `WalletApi` controller. Its autowired `bus` / +> `ledger` / `query_cache` dependencies should show as edges in the bean graph — +> proof the view is reading the real container, not a stub. + +### A custom view To add your own sidebar view, implement the `AdminView` trait and push it onto -`AdminDeps::views`; the dashboard lists it under `/admin/api/views[/:id]`. A -Lumen "Treasury" view surfaces the total custody balance across all wallets, -queried from the read model: +`AdminDeps::views`; the dashboard lists it under `/admin/api/views[/:id]`. A Lumen +"Treasury" view surfaces the total custody balance across all wallets, queried +from the read model: ```rust,ignore +use std::sync::Arc; use firefly::admin::AdminView; struct TreasuryView { @@ -403,53 +606,109 @@ impl AdminView for TreasuryView { serde_json::json!({ "custodyTotal": total, "wallets": self.read_model.len() }) } } +``` + +The four trait methods are: `view_id` (the registry key and `/views/{id}` path +segment), `display_name` and `icon` (what the sidebar renders), and the async +`data()` that produces the view's JSON payload. Register the view before mounting +by pushing it onto `AdminDeps::views`: -// Register it before mounting: -let mut deps = AdminDeps::new(/* … */); +```rust,ignore +let mut deps = AdminDeps::new(/* required collaborators … */); deps.views.push(Arc::new(TreasuryView { read_model: Arc::clone(&read_model) })); ``` -## What changed in Lumen - -This chapter explained Lumen's always-on observability surface — all of it wired -by `FireflyApplication`, with no observability code in `main.rs`: - -- An optional **`InfoContributor`** (registered with `.info_contributor(..)` on - the application builder) describes the in-memory store and event bus on - `/actuator/info`; the framework serves the full `/actuator/*` surface — - `health`, `info`, `metrics`, `loggers`, `scheduledtasks`, the DI-introspection reports `beans` / `mappings` / `conditions`, and the rest — on the - management port. -- **`init_logging`** (called by the framework, best-effort so the test harness can - own the subscriber) switches on structured, correlation-enriched logging; the - correlation id flows into every log line, published event, and outbound call - automatically. The **`TraceContextLayer`** originates a W3C `traceparent` when - one is absent. -- The **request-metrics** middleware records `http_server_requests_seconds` per - templated route, exposed at `/actuator/metrics` and `/actuator/prometheus`. -- The **self-hosted admin dashboard** ties it all together in fifteen views on the - management port — including the **Beans** view, which is *populated* because the - framework component-scans `LumenBeans` — with real env/config/mappings, live SSE - streams, and a live log tail. +> **Note** When you let `FireflyApplication` self-host the dashboard you never +> build `AdminDeps` yourself — the framework sources every collaborator from the +> live web stack and the scanned container. You only construct `AdminDeps` +> directly in the advanced case below, where you host the dashboard outside a +> `FireflyApplication`. + +> **Design note.** The dashboard router is reachable directly when you want to host +> it outside `FireflyApplication` — a custom server, or a test. `mount(AdminConfig, +> AdminDeps)` returns the router; `AdminDeps::new` takes the required collaborators +> and the rest are optional fields filled with struct-update syntax: +> +> ```rust,ignore +> use std::sync::Arc; +> use firefly::admin::{mount, AdminConfig, AdminDeps, LogBuffer, TraceBuffer}; +> +> let deps = AdminDeps { +> scheduler: Some(scheduler), // → Scheduled Tasks view +> bus: Some(bus), // → CQRS view +> container: Some(container), // → Beans view +> ..AdminDeps::new( +> "lumen", +> firefly::VERSION, +> health_composite, // Arc +> metric_registry, // Arc +> Arc::new(TraceBuffer::new()), +> LogBuffer::new(), +> ) +> }; +> let dashboard = mount(AdminConfig::default(), deps); +> ``` +> +> `FireflyApplication` performs exactly this mount for you, which is why Lumen +> ships the dashboard with no admin code of its own. + +## Recap — what changed in Lumen + +This chapter taught you to read and extend Lumen's always-on observability +surface — all of it wired by `FireflyApplication`, with no observability code in +`main.rs`: + +| Concern | Who wired it | Where you read / extend it | +|---------|--------------|----------------------------| +| Management surface on `:8081` | the framework | `curl /actuator/*`; override with `FIREFLY_MANAGEMENT_ADDR` | +| `/actuator/info` instance metadata | framework `app` block + your contributor | `.info_contributor(..)` on the builder | +| Health rollup | framework composite + your indicators | `add_observability_indicator(IndicatorFn::new(..))` | +| Request metrics (`http_server_requests_seconds`) | the framework, on by default | `/actuator/metrics`, `/actuator/prometheus` | +| Your domain meters | you, on the shared registry | `core.metric_registry().counter(..)` / `.gauge(..)` | +| Structured, correlation-enriched logs | `init_logging` at boot | plain `tracing` macros; tune via `firefly.logging.*` | +| W3C trace context | the `TraceContextLayer` | originated/propagated on the edges automatically | +| Self-hosted admin dashboard | `FireflyApplication` + the `admin` feature | `/admin/` — fifteen views including populated **Beans** | + +You also now know that the correlation id flows automatically into every log +line, published event, and outbound call; that the `TraceContextLayer` originates +a W3C `traceparent` when one is absent; and that global exception advice is an +opt-in `#[bean]` the framework installs only when present. ## Exercises -1. **Reach the actuator.** Run `cargo run --bin lumen`, then - `curl http://127.0.0.1:8081/actuator/info` and confirm the `sample` block - reports the in-memory store. Hit `/actuator/health` and `/actuator/metrics`. +1. **Reach the actuator.** Run `cargo run --bin lumen`, then `curl + localhost:8081/actuator/info` and confirm the `sample` block reports the + in-memory store. Hit `/actuator/health` and `/actuator/metrics`, then confirm + `curl localhost:8080/actuator/health` returns a 404 problem — the actuator is + not on the public port. 2. **Add a health indicator.** Wire an `IndicatorFn::new("read-model", ..)` onto the composite with `add_observability_indicator` (from an `on_ready` hook, or declare it as a `#[bean]`) that returns `UP` when the read model holds at least - one wallet view, and watch it appear under `/actuator/health`. -3. **A Lumen metric.** Record a counter — e.g. `lumen_transfers_total` — on the - `metric_registry()` each time the transfer saga completes, and verify it - appears at `/actuator/metrics`. (Recall the housekeeping heartbeat in - Chapter 16 keeps an `AtomicU64` you could surface the same way.) -4. **Explore the Beans view.** Run `cargo run --bin lumen --features admin`, open - `http://127.0.0.1:8081/admin/`, and find the Beans view — note that it is - *populated* (the framework scanned `LumenBeans`). Locate the `WalletApi` - controller and confirm its autowired `bus` / `ledger` / `query_cache` - dependencies show in the bean graph. - -With Lumen observable, the next chapter adds background work and the path to -outbound notifications. Continue to -[Scheduling & Notifications](./16-scheduling-notifications.md). + one wallet view and `DEGRADED` otherwise, then watch it appear under + `/actuator/health`. +3. **A Lumen metric.** Record a counter — e.g. `lumen_transfers_total` — on + `core.metric_registry()` each time the transfer saga completes, and verify it + appears at `/actuator/metrics` and in the `/actuator/prometheus` scrape. +4. **Change a log level live.** `curl localhost:8081/actuator/loggers` to list the + loggers, then `POST` a new `configuredLevel` to one of them and GET it back to + confirm the change took effect on the running process — no restart. +5. **Explore the Beans view.** Run `cargo run --bin lumen --features admin`, open + `http://localhost:8081/admin/`, and find the Beans view — note that it is + *populated*. Locate the `WalletApi` controller and confirm its autowired `bus` + / `ledger` / `query_cache` dependencies show in the bean graph. + +## Where to go next + +A service you can see is a service you can operate. The next chapter gives Lumen +work to do on its own — and a way to reach customers. + +- Add background jobs and outbound notifications in + **[Scheduling & Notifications](./16-scheduling-notifications.md)** — and watch + the new `#[scheduled]` tasks appear under `/actuator/scheduledtasks` and the + Scheduled Tasks dashboard view. +- Revisit how the framework discovers and wires the beans the **Beans** view + shows in **[Dependency Wiring](./04-dependency-wiring.md)**. +- Drive the wired router — and assert on health and metrics — in tests with + `bootstrap()` in **[Testing](./18-testing.md)**. +- Move the management port onto a private interface and turn on real + infrastructure in **[Production & Deployment](./20-production.md)**. diff --git a/docs/book/src/16-scheduling-notifications.md b/docs/book/src/16-scheduling-notifications.md index d5f7800..6dced03 100644 --- a/docs/book/src/16-scheduling-notifications.md +++ b/docs/book/src/16-scheduling-notifications.md @@ -2,35 +2,81 @@ A real wallet service does more than answer HTTP. It sweeps abandoned wallets overnight, recomputes interest, retries stuck transfers, and — the part a -customer notices — emails a daily statement. Two framework concerns cover that -back-office work: running code on a timer (`firefly-scheduling`) and delivering -messages through swappable providers (`firefly-notifications`), with outbound -webhooks (`firefly-callbacks`) and inbound webhooks (`firefly-webhooks`) rounding -out the integration story. - -By the end of this chapter Lumen will run a **scheduled housekeeping heartbeat**, -declared with `#[scheduled]` and started by the framework, and you will know -exactly where a daily-statement notification, an outbound balance-changed -webhook, and an inbound payment-provider callback would hang off it. Lumen keeps -the heartbeat deliberately tiny — it records that it ran — so the macro is shown -wired end to end without dragging in a provider SDK. - -> **The back-office concerns.** `#[scheduled]` runs code on a timer (cron, -> fixed rate, fixed delay). `firefly-notifications` delivers messages through a -> `Channel` + `Dispatcher` abstraction; `firefly-callbacks` pushes signed -> outbound webhooks; `firefly-webhooks` validates and ingests inbound ones. Each -> swaps a provider for a one-line registration, never a code change. - -## The scheduled heartbeat - -Lumen's `src/housekeeping.rs` is the whole feature. A zero-argument `async fn` -carries `#[scheduled(...)]`; the macro generates a `schedule_(scheduler)` -registration helper **and** submits a `ScheduledRegistration` into a compile-time -`inventory` registry, so the framework finds and registers the task for you. Here -is the file, end to end: +customer notices — emails a daily statement. None of that is triggered by a +request: it runs on a clock, or in response to something happening elsewhere. +This chapter gives Lumen its first piece of *background* work and maps the +integration surface that hangs off it. + +Four framework concerns cover this back-office story, and Firefly ships each one +behind the same `firefly` facade you have depended on since +[Quickstart](./02-quickstart.md): + +- **Scheduling** (`firefly-scheduling`) — running code on a timer. +- **Notifications** (`firefly-notifications`) — delivering messages through + swappable providers (email, SMS, push). +- **Outbound webhooks** (`firefly-callbacks`) — pushing signed events to other + systems that want to react to Lumen. +- **Inbound webhooks** (`firefly-webhooks`) — receiving and validating callbacks + from Lumen's external payment provider. + +We will build the scheduling piece end to end — a real, registered, running task +— and then map exactly where the notification, the outbound webhook, and the +inbound callback attach to it. Lumen keeps the scheduled task deliberately tiny — +it records that it ran — so you see the wiring without dragging a provider SDK +into the teaching baseline. + +By the end of this chapter you will: + +- Declare a scheduled task with `#[scheduled]` and understand how the framework + *discovers* and starts it without a line of wiring in `main`. +- Tell the four trigger kinds apart — cron, zoned cron, fixed-rate, fixed-delay — + and pick the right one for a given job. +- Read the cron grammar Firefly accepts, including the time-zone and macro forms. +- Dispatch a notification through the channel-agnostic `Dispatcher` and see how a + real provider slots in behind the same trait. +- Sketch a signed outbound webhook with `firefly-callbacks` and a validated + inbound webhook with `firefly-webhooks`, and know where each would hang off + Lumen's schedule. + +## Concepts you will meet + +Before the first line of code, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — scheduled task.** A *scheduled task* is a piece of code +> the framework runs on a timer rather than in response to a request. You write +> the work; a *trigger* decides when it fires. The Spring analog is a method +> annotated `@Scheduled`. + +> **Note** **Key term — trigger.** A *trigger* is the rule that answers "when +> does this task run next?" — every minute, at 2 a.m. daily, 30 seconds after the +> last run finished. Firefly ships four trigger kinds (cron, zoned cron, +> fixed-rate, fixed-delay); Spring expresses the same choices through +> `@Scheduled(cron=…)`, `fixedRate`, and `fixedDelay`. + +> **Note** **Key term — link-time discovery.** Firefly finds your scheduled +> tasks at *link time* using the `inventory` crate: each `#[scheduled]` submits a +> registration into a compile-time registry, and the framework drains that +> registry at startup. The Spring analog is component scanning — except it +> happens at compile/link time with no runtime reflection, so "what is scheduled" +> is a fixed, inspectable set. + +> **Note** **Key term — channel / dispatcher.** A *channel* is a transport that +> delivers a message (email, SMS, push); a *dispatcher* routes a message to the +> channel registered for its kind. You code against the channel *port* (a trait) +> and register a concrete provider at wiring time. Spring's analog is a +> `NotificationService` fronting pluggable senders. + +## Step 1 — Declare a scheduled task + +Lumen's background work lives in `src/housekeeping.rs`. The whole feature is one +zero-argument `async fn` carrying a `#[scheduled(...)]` attribute. Create the +file with this content: ```rust,ignore +// src/housekeeping.rs use std::sync::atomic::{AtomicU64, Ordering}; + use firefly::prelude::*; /// The number of times the heartbeat has run — observable from a test (and, in @@ -44,10 +90,71 @@ pub async fn ledger_heartbeat() -> Result<(), std::io::Error> { HEARTBEAT_TICKS.fetch_add(1, Ordering::Relaxed); Ok(()) } +``` + +Then add the module to your crate root so it is compiled and scanned. In +`src/main.rs` the line already exists in Lumen's module list (you set it up in +[Quickstart](./02-quickstart.md)); if you are following along incrementally, add +it now: + +```rust,ignore +// src/main.rs +mod housekeeping; +``` + +What just happened, piece by piece: + +- **`#[scheduled(fixed_rate = "60s", initial_delay = "5s")]`** is the whole + declaration. `fixed_rate = "60s"` says "fire every 60 seconds"; `initial_delay + = "5s"` says "wait 5 seconds after startup before the first fire." Durations are + written as humanized strings (`"60s"`, `"5m"`, `"500ms"`). +- **`ledger_heartbeat` is the work.** It is a plain `async fn` taking no + arguments and returning a `Result`. Here it just bumps an atomic counter; a real + deployment would sweep abandoned wallets or kick off a statement run. +- **`firefly::prelude::*`** brings in everything the framework surface needs — + including the `#[scheduled]` macro itself and the `Scheduler` type you will meet + in Step 3. The one facade import covers it. + +> **Note** **Key term — `inventory` registry.** `inventory` is the Rust crate +> Firefly uses for link-time discovery. The `#[scheduled]` macro does two things: +> it generates a `schedule_ledger_heartbeat(&scheduler)` helper, and it submits a +> `ScheduledRegistration` into the `inventory` registry. You never call the +> helper — the framework iterates the registry at boot. This is the same +> discovery mechanism that finds your controllers and CQRS handlers. + +> **Tip** **Checkpoint.** `cargo build` compiles cleanly. You wrote a timer-driven +> function and registered nothing by hand — the attribute did the registration. + +## Step 2 — Let the framework own the scheduler + +You did not write a `tokio::spawn`, a `Scheduler::new()`, or a `start()` call — +and you will not. `FireflyApplication::run()` (the single line in Lumen's `main`) +owns the scheduler. During the boot pipeline you read about in +[Quickstart, Step 6](./02-quickstart.md#step-6--understand-what-run-does), the +framework: + +1. Constructs a `Scheduler`. +2. Drains the `inventory` registry — calling + `firefly::scheduling::register_discovered_scheduled(&scheduler)` to register + every `#[scheduled]` task (and a sibling call for tasks declared as bean + methods). +3. Starts the scheduler on a background tokio task, so it runs for the life of + the process. + +That means `main` never changes when you add a scheduled task — the new task is +*discovered*, not threaded through an entry point. This is the same property that +held for controllers and CQRS handlers in earlier chapters. + +For testing, Lumen keeps a small helper that builds a *fresh* scheduler and runs +the same discovery against it, so a test can introspect the schedule without +booting the whole application or waiting for a tick: +```rust,ignore +// src/housekeeping.rs (continued) /// Registers the heartbeat on a fresh scheduler and returns it — used by the -/// tests to assert it registered. `main()` does NOT call this: `FireflyApplication` -/// drains the same `inventory` registry and starts the scheduler. +/// tests to assert it registered. `main()` does NOT call this: +/// `FireflyApplication` drains the same `inventory` registry and starts the +/// scheduler. pub fn build_scheduler() -> std::sync::Arc { let scheduler = std::sync::Arc::new(Scheduler::new()); // `#[scheduled]` tasks are DISCOVERED and registered through the @@ -62,57 +169,82 @@ pub fn heartbeat_ticks() -> u64 { } ``` -Three things are happening: - -- **The macro generates the wiring and registers the task.** `#[scheduled(fixed_rate - = "60s", initial_delay = "5s")]` on `ledger_heartbeat` emits a - `schedule_ledger_heartbeat(&scheduler)` helper *and* submits a - `ScheduledRegistration` to the `inventory` registry the framework drains. You - write the work; the macro writes the trigger + registration. `Scheduler` comes - from `firefly::prelude::*`, so the one facade import covers it. -- **The framework owns the scheduler.** `FireflyApplication` calls - `firefly::scheduling::register_discovered_scheduled(&scheduler)` to drain the - registry, then starts the scheduler on a background task — Lumen writes no - `build_scheduler()` call and no `tokio::spawn`. (`build_scheduler` survives only - so the test module can introspect the schedule.) -- **The heartbeat is observable.** It bumps an `AtomicU64`, which a test reads — - and which, in a real deployment, you would surface as a counter on - `/actuator/metrics` (Chapter 15). - -The framework starts the scheduler on a background task, because `Scheduler::start` -runs until the scheduler is stopped. `Scheduler::start` runs each task on its own -tokio task with panic recovery, and `stop()` shuts down gracefully (in-flight runs -finish first). The scheduled tasks also surface on `/actuator/scheduledtasks`. - -### What the tests assert - -`housekeeping.rs`'s test module proves both halves — the task registered, and it -ticks when called: +What just happened: `build_scheduler` exists *only* for the tests. It calls the +exact same `register_discovered_scheduled` the framework calls, so the test +exercises real discovery. `Scheduler::new()` returns an empty scheduler whose +distributed-lock provider is a no-op (single-instance behaviour); the +registration call populates it from the `inventory` registry. -```rust,ignore -#[test] -fn scheduled_task_registers() { - let scheduler = build_scheduler(); - let names: Vec = scheduler.tasks().into_iter().map(|t| t.name).collect(); - assert!(names.contains(&"ledger_heartbeat".to_string())); -} +> **Note** **Key term — distributed lock.** When you run more than one copy of a +> service, you usually want a scheduled job to run on *exactly one* of them. A +> *distributed lock* (Spring/ShedLock's model) lets a task acquire a named lock +> before each tick and skip the tick if another instance holds it. The default +> `Scheduler::new()` uses a no-op lock (every tick runs), which is correct for a +> single instance; Redis- and Postgres-backed locks ship for the clustered case. + +The scheduler runs each task on its own tokio task with panic recovery — a +panicking task is logged and the schedule continues — and `stop()` shuts down +gracefully, letting in-flight runs finish first. Because `run()` traps +SIGINT/SIGTERM, that graceful shutdown is wired into Lumen's lifecycle for free. + +> **Tip** **Checkpoint.** Run Lumen with `cargo run` and watch the startup +> report's `scheduled tasks:` count tick up to include `ledger_heartbeat`. Five +> seconds after boot the heartbeat begins firing once a minute — silently, since +> it only bumps a counter. + +## Step 3 — Observe the schedule from a test -#[tokio::test] -async fn heartbeat_runs() { - let before = heartbeat_ticks(); - ledger_heartbeat().await.unwrap(); - assert_eq!(heartbeat_ticks(), before + 1); +The task is registered and ticking, but how do you *prove* it without waiting 60 +seconds? Two seams make the schedule observable. First, the scheduler exposes a +snapshot of every registered task; second, the heartbeat's atomic counter records +each run. Lumen's test module asserts both: + +```rust,ignore +// src/housekeeping.rs (test module) +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn scheduled_task_registers() { + let scheduler = build_scheduler(); + let names: Vec = scheduler.tasks().into_iter().map(|t| t.name).collect(); + assert!(names.contains(&"ledger_heartbeat".to_string())); + } + + #[tokio::test] + async fn heartbeat_runs() { + let before = heartbeat_ticks(); + ledger_heartbeat().await.unwrap(); + assert_eq!(heartbeat_ticks(), before + 1); + } } ``` -`scheduler.tasks()` returns a `Vec` (each has a `name`), so a test -can introspect the schedule without waiting for a tick — and the dashboard's -Scheduled Tasks view (Chapter 15) reads the same list. +What just happened: + +- **`scheduler.tasks()`** returns a `Vec` — an immutable snapshot + taken at registration time, each entry carrying a `name`, the trigger + descriptor, and any lock metadata. The first test introspects the schedule with + no waiting: it registered, so its name is present. +- **`ledger_heartbeat().await`** calls the body directly. Because the work is a + plain `async fn`, a test can invoke it without the scheduler at all and assert + the side effect — the counter advanced by exactly one. -## The trigger menu +The same `tasks()` snapshot powers the actuator: scheduled tasks surface on +`GET /actuator/scheduledtasks` (on the management port) and in the admin +dashboard's Scheduled Tasks view, both introduced in +[Observability](./15-observability.md). -`#[scheduled]` covers the everyday case; the underlying `Scheduler` exposes all -four trigger kinds directly, which is what a real Lumen statement run would use: +> **Tip** **Checkpoint.** `cargo test heartbeat` passes. You proved the two +> halves — the task registered, and its body runs and is observable — without a +> single `sleep` in the test. + +## Step 4 — Choose the right trigger + +`#[scheduled]` covers the everyday case, but the underlying `Scheduler` exposes +all four trigger kinds directly, which is what a real Lumen statement run would +use. Each kind answers "when next?" differently: ```rust,ignore use std::{sync::Arc, time::Duration}; @@ -121,49 +253,79 @@ use firefly::prelude::Scheduler; let s = Arc::new(Scheduler::new()); // Cron: a 5-field expression (or the 6-field form with leading seconds). +// Returns Result — the expression is parsed and can be rejected. s.cron("daily-statements", "0 2 * * *", || async { Ok(()) }).unwrap(); -// FixedRate: fire every period from a fixed anchor (slips on a slow run). +// FixedRate: fire every period from a fixed anchor (the schedule slips if a run +// is slow, because the grid is anchored, not chained to the last finish). s.fixed_rate("metrics-emit", Duration::from_secs(30), || async { Ok(()) }); -// FixedDelay: fire `delay` after the previous run finished. +// FixedDelay: fire `delay` after the previous run *finished* (no overlap). s.fixed_delay("sweep-abandoned", Duration::from_secs(300), || async { Ok(()) }); ``` -| Trigger | Behavior | -|---------------------|-------------------------------------------------------------| -| `CronTrigger` | fires when the **local** wall clock matches the expression | -| `ZonedCronTrigger` | fires per the expression in an IANA time zone | -| `FixedRateTrigger` | fires every period from a fixed start anchor (slips) | -| `FixedDelayTrigger` | fires the delay after the previous run finished | +What just happened: you registered three tasks on a scheduler by hand. Each +closure is a factory — the scheduler calls it once per firing to get a fresh +future — and each returns `Result<(), TaskError>`, so a failing run is logged at +`warn` and the schedule continues. Note `cron` returns a `Result` because the +expression is parsed; the rate/delay registrations take a typed `Duration` and +cannot fail. -The cron grammar is the canonical 5-field `minute hour day-of-month month -day-of-week`, plus an optional 6-field form with leading seconds, the `?` -placeholder, and the `@daily` / `@hourly` / `@weekly` macros. For a time zone, -build a `ZonedCronTrigger`: +The four kinds, and when to reach for each: + +| Trigger | Behaviour | +|---------------------|-------------------------------------------------------------| +| `CronTrigger` | Fires when the **local** wall clock matches the expression | +| `ZonedCronTrigger` | Fires per the expression in an IANA time zone | +| `FixedRateTrigger` | Fires every period from a fixed start anchor (slips on slow runs) | +| `FixedDelayTrigger` | Fires the delay after the previous run finished | + +> **Design note.** The fixed-rate vs fixed-delay distinction is the one that bites +> people. *Fixed-rate* hangs a grid off a fixed anchor: every 30 s on the dot, so +> if one run takes 35 s, the next fires immediately (it slipped). *Fixed-delay* +> chains: it waits the delay *after* each run finishes, so a slow run pushes the +> next one out and two runs never overlap. Use fixed-rate for steady sampling +> (emit a metric every 30 s); use fixed-delay for serial work that must not pile +> up (sweep, then wait, then sweep again). + +For a time zone — Lumen would run statements at 9 a.m. in the customer's region — +build a `ZonedCronTrigger` instead of relying on the host's local clock: ```rust,ignore use firefly::scheduling::{parse_cron, ZonedCronTrigger}; -// Lumen would run statements at 9am in the customer's region. -// 1-5 is Monday through Friday (the dow domain is 0=Sunday .. 6=Saturday). +// 1-5 is Monday through Friday (the day-of-week domain is 0 = Sunday .. 6 = Saturday). let expr = parse_cron("0 9 * * 1-5").unwrap(); let trigger = ZonedCronTrigger::in_zone(expr, "America/New_York").unwrap(); ``` -> **Cron grammar.** Firefly's cron parser accepts the canonical 5-field -> expression, an optional 6-field leading-seconds form, the `?` placeholder, and -> the `@daily` / `@hourly` / `@weekly` macros. `#[scheduled]` generates the -> registration helper (`schedule_`) so you write only the work function. - -## Notifications — where the daily statement hangs - -The heartbeat is the hook for outbound messaging. `firefly-notifications` gives -you a channel-agnostic `Notification` envelope, a `Channel` transport trait, and -a `Dispatcher` that routes a message to the channel registered for its `Kind`. -The default `MemoryChannel` records every message (ideal for tests); a real -provider adapter slots in behind the same trait. A Lumen statement run, in -sketch: +What just happened: `parse_cron` turns the 5-field string into a typed +`CronExpr`, and `ZonedCronTrigger::in_zone` evaluates that expression in the named +IANA zone. Both calls return a `Result` — a malformed expression or an unknown +zone name is a hard error you handle at registration, never a silent +mis-schedule. To register a zoned cron task in one call, the scheduler also +offers `s.cron_in_zone(name, expr, zone, run)`. + +> **Note** **Cron grammar.** Firefly's parser accepts the canonical 5-field +> expression `minute hour day-of-month month day-of-week`, an optional 6-field +> form with a leading **seconds** field, the Quartz `?` placeholder (treated as +> `*`), and the `@hourly` / `@daily` / `@weekly` / `@monthly` / `@yearly` macros. +> Day-of-week runs `0` (Sunday) through `6` (Saturday). When both day-of-month and +> day-of-week are restricted, the rule fires when **either** matches (Vixie cron +> behaviour). The `#[scheduled]` attribute accepts the same `cron = "…"` (with an +> optional `zone = "…"`) in place of `fixed_rate` / `fixed_delay`. + +> **Tip** **Checkpoint.** You can name the four triggers and explain why a +> statement run uses cron (a wall-clock time), a metrics emitter uses fixed-rate +> (steady sampling), and a sweep uses fixed-delay (no overlap). + +## Step 5 — Dispatch a notification + +The heartbeat is the hook for outbound messaging. On a real statement tick, Lumen +would build one message per wallet and hand it to a dispatcher. The +`firefly-notifications` crate gives you a channel-agnostic `Notification` +envelope, a `Channel` transport trait, and a `Dispatcher` that routes a message +to the channel registered for its `Kind`: ```rust,ignore use std::sync::Arc; @@ -184,10 +346,33 @@ dispatcher .unwrap(); ``` -The `Dispatcher` routes by `Kind` (`Kind::EMAIL` / `Kind::SMS` / `Kind::PUSH`, or -a custom `Kind::new("...")`); a message for an unregistered kind is a -`NotificationError::NoChannel`. For production, register a real channel in place -of `MemoryChannel` — same trait, real delivery: +What just happened, block by block: + +- **`Dispatcher::new()`** creates an empty router. **`register`** adds a channel, + keyed on the `Kind` it serves. Here `MemoryChannel::new(Kind::EMAIL)` is a + built-in channel that simply records every message it receives — ideal for + tests, and exactly what Lumen uses so the teaching baseline pulls in no provider + SDK. +- **`dispatch`** builds a `Notification` envelope and routes it. The `channel` + field (`Kind::EMAIL`) selects the registered channel; `to`, `subject`, and + `body` are the message. `..Notification::default()` fills the remaining fields + (id, template, variables, timestamp) with their zero values. +- **The `Kind` newtype** carries the canonical channels `Kind::EMAIL`, + `Kind::SMS`, and `Kind::PUSH`, plus `Kind::new("...")` for a custom transport. + Dispatching to a kind with no registered channel returns + `NotificationError::NoChannel` — a typed error, not a silent drop. + +> **Note** **Key term — port and adapter.** A *port* is the trait you code +> against (`Channel`); an *adapter* is a concrete implementation behind it +> (`MemoryChannel`, or a real SMTP sender). Because every channel is an +> `Arc`, the heavy provider SDKs stay out of any service that does +> not select that channel: you write your statement logic against the port and +> register the concrete adapter at wiring time. This is hexagonal architecture, +> the same shape you met in [Domain-Driven Design](./08-domain-driven-design.md). + +For production, register a real channel in place of `MemoryChannel` — the same +`Channel` trait, real delivery. Each provider lives in its own crate so its SDK +only compiles into services that opt in: | Crate | Channel | Backing | |----------------------------------|---------|----------------------------------| @@ -197,49 +382,68 @@ of `MemoryChannel` — same trait, real delivery: | `firefly-notifications-sendgrid` | email | SendGrid | | `firefly-notifications-resend` | email | Resend | -Because every channel is an `Arc`, the heavy provider SDKs stay out -of a service that does not select that channel: code against the `Channel` port, -register the concrete adapter at wiring time. So Lumen's daily statement is "on -the heartbeat tick, build a `Notification` per wallet and `dispatch` it" — the -provider is a wiring decision, not a rewrite. +So Lumen's daily statement is, in one sentence: "on the heartbeat tick, build a +`Notification` per wallet and `dispatch` it." Swapping the in-memory channel for +a real SMTP one is a one-line registration change, never a rewrite of the +statement logic. + +> **Tip** **Checkpoint.** You can trace a message from `dispatch` through the +> `Kind`-keyed routing to a channel, and you can name where a real email provider +> would slot in (the `register` call) without touching the statement code. -## Outbound webhooks — telling other systems a balance changed +## Step 6 — Push an outbound webhook When another system needs to *react* to a Lumen event — a fraud monitor that -wants every large deposit, say — you push it an outbound webhook with +wants every large deposit, say — Lumen pushes it an outbound webhook with `firefly-callbacks`. Services register `Target`s; the `HmacDispatcher` signs each -payload with HMAC-SHA256, retries with exponential backoff, and records every -`Attempt` to a pluggable `Store` for audit: +payload, retries with exponential backoff, and records every `Attempt` to a +pluggable `Store` for audit: ```rust,ignore use std::sync::Arc; use firefly_callbacks::{CallbackEvent, DispatcherConfig, HmacDispatcher, MemoryStore}; let store = Arc::new(MemoryStore::new()); -let dispatcher = HmacDispatcher::new(store, DispatcherConfig::default()); // 3 attempts, 200ms, doubling +// Defaults: 3 attempts, 200 ms initial delay, doubling between retries. +let dispatcher = HmacDispatcher::new(store, DispatcherConfig::default()); // On a large deposit, Lumen would publish a CallbackEvent; the dispatcher signs // and POSTs it to every registered Target with a stable HMAC-SHA256 signature. ``` -Each delivery carries `X-Firefly-Event`, `X-Firefly-Event-Id`, -`X-Firefly-Timestamp`, and an `X-Firefly-Signature: sha256=` keyed on -the target's secret, so any receiver that knows the shared secret verifies -Lumen's deliveries with a standard HMAC check. +What just happened: `HmacDispatcher::new` takes a `Store` (here the in-memory one, +which keeps every delivery attempt for inspection) and a `DispatcherConfig`. Any +field left at its zero value is filled with the default, so +`DispatcherConfig::default()` means 3 attempts with a 200 ms first delay, +doubling. On a triggering event, Lumen publishes a `CallbackEvent`; the dispatcher +POSTs the payload to each registered `Target` and records an `Attempt` row per +try, regardless of outcome. -## Inbound webhooks — receiving a provider callback +> **Note** **Key term — HMAC signature.** HMAC (hash-based message authentication +> code) lets a receiver verify a payload came from you and was not tampered with, +> using a shared secret. Firefly signs each delivery with HMAC-SHA256 keyed on the +> target's secret, so any receiver holding the same secret can verify the request +> with a standard library call — no Firefly-specific code required. -The mirror image is `firefly-webhooks`: when Lumen's external payment provider -calls *back* (a charge settled, a payout failed), the inbound pipeline validates -the signature, deduplicates, and dispatches to a processor. The signature -validators ship for the common providers: +Each delivery carries these headers, byte-identical to Firefly's Java, .NET, Go, +and Python ports, so a receiver written against any of them verifies Lumen's +deliveries unchanged: -| Validator | Header | Algorithm | -|-------------------|-------------------------|-----------------------------------------------| -| `HmacValidator` | `X-Signature` (default) | HMAC-SHA256 hex (optional `sha256=` prefix) | -| `StripeValidator` | `Stripe-Signature` | `t=,v1=`, 5-minute tolerance | -| `GitHubValidator` | `X-Hub-Signature-256` | HMAC-SHA256 hex | -| `TwilioValidator` | `X-Twilio-Signature` | HMAC-SHA1 base64 of URL + sorted form fields | +- `X-Firefly-Event` — the event type. +- `X-Firefly-Event-Id` — the event id. +- `X-Firefly-Timestamp` — Unix seconds when the request was sent. +- `X-Firefly-Signature` — `sha256=` keyed on the target's secret. + +> **Tip** **Checkpoint.** You can describe what an outbound webhook is (Lumen +> POSTing a signed event to a registered target), and name the header a receiver +> checks (`X-Firefly-Signature`). + +## Step 7 — Receive an inbound webhook + +The mirror image is `firefly-webhooks`: when Lumen's external payment provider +calls *back* — a charge settled, a payout failed — the inbound pipeline validates +the signature, deduplicates, and dispatches the event to a processor. Set up a +pipeline with a provider validator and mount its router: ```rust,ignore use std::sync::Arc; @@ -250,35 +454,74 @@ pipeline.register_validator(StripeValidator::new(b"whsec_test")); let app: axum::Router = web::router(pipeline); // mount under /api/webhooks/... ``` +What just happened: `Pipeline::new` takes a *dead-letter queue* — here the +in-memory `MemoryDlq`, where events that fail processing land for later +inspection. `register_validator` attaches a per-provider signature check; the +`StripeValidator` is keyed on the webhook secret (`whsec_test`). `web::router` +turns the pipeline into an `axum::Router` you mount alongside Lumen's other +routes. + +> **Note** **Key term — dead-letter queue (DLQ).** A *dead-letter queue* is where +> a message goes when it cannot be processed — a validated webhook whose processor +> errored, for instance. Parking it in a DLQ instead of dropping it means you can +> inspect, fix, and replay it later. This is the same EDA pattern you met in +> [EDA & Messaging](./10-eda-messaging.md). + +Validators ship for the common providers; each knows the header and algorithm its +provider uses, so registering one is the whole integration: + +| Validator | Header | Algorithm | +|-------------------|-------------------------|-----------------------------------------------| +| `HmacValidator` | `X-Signature` (default) | HMAC-SHA256 hex (optional `sha256=` prefix) | +| `StripeValidator` | `Stripe-Signature` | `t=,v1=`, 5-minute tolerance | +| `GitHubValidator` | `X-Hub-Signature-256` | HMAC-SHA256 hex | +| `TwilioValidator` | `X-Twilio-Signature` | HMAC-SHA1 base64 of URL + sorted form fields | + You test that receiver with the `firefly-testkit` signers — `sign_stripe`, `sign_github`, `sign_twilio`, `sign_hmac` — which produce header values byte-identical to what the validators expect, so a signed test request validates -exactly as a real provider's would (Chapter 18). +exactly as a real provider's would. You will use these in +[Testing](./18-testing.md). + +> **Tip** **Checkpoint.** You can name both halves of the integration story: +> outbound (`firefly-callbacks`, you sign and push) and inbound +> (`firefly-webhooks`, you validate and ingest), and point to the validator that +> matches a given provider. -## What changed in Lumen +## Recap — what changed in Lumen This chapter gave Lumen its first background work and mapped its integration -surface: +surface. -- **`src/housekeeping.rs`** declares the `ledger_heartbeat` with - `#[scheduled(fixed_rate = "60s", initial_delay = "5s")]`; the macro submits a +| Before | After this chapter | +|--------|--------------------| +| no background work | `src/housekeeping.rs` declares `ledger_heartbeat` with `#[scheduled(fixed_rate = "60s", initial_delay = "5s")]` | +| nothing on a timer | the framework discovers and starts the task; it ticks once a minute, observable via an `AtomicU64` and `scheduler.tasks()` | +| `main` threads nothing | `main` is still the single `FireflyApplication::run()` line — the task is discovered, not wired | +| — | the heartbeat's counter is the seam where a daily-statement notification, an outbound balance-changed webhook, and an inbound provider callback would attach | + +You also now know: + +- That `#[scheduled]` generates a `schedule_` helper *and* submits a `ScheduledRegistration` to the `inventory` registry the framework drains with - `register_discovered_scheduled(&scheduler)` — no manual `schedule_` call. -- **`FireflyApplication`** drains that registry and starts the scheduler on a - background tokio task; Lumen's `main()` is the single `FireflyApplication::run()` - line and spawns nothing. The tasks surface on `/actuator/scheduledtasks` and in - the dashboard's Scheduled Tasks view. -- The heartbeat's `AtomicU64` is the seam where a real **daily-statement - notification** (`firefly-notifications`), an **outbound balance-changed - webhook** (`firefly-callbacks`), and an **inbound provider callback** - (`firefly-webhooks`) would attach — each a registration, not a rewrite. + `register_discovered_scheduled(&scheduler)` — so you never hand-maintain a list + of registration calls. +- The four trigger kinds and when each applies, plus the cron grammar Firefly + accepts (5-field, 6-field with seconds, `?`, the `@daily`-style macros, IANA + zones via `ZonedCronTrigger`). +- That notifications, outbound webhooks, and inbound webhooks each swap a provider + for a one-line registration, never a code change, because every transport is a + trait object (`Arc`, a registered `Target`, a registered + `Validator`). + +Lumen now does background work and knows exactly where its messaging hangs. ## Exercises 1. **Cron the statement run.** Replace the `fixed_rate` heartbeat with a - `#[scheduled(cron = "0 2 * * *")]` task (or register `s.cron("statements", - "0 2 * * *", ..)` directly) and assert it appears in `scheduler.tasks()` by - name. + `#[scheduled(cron = "0 2 * * *")]` task (or register + `s.cron("statements", "0 2 * * *", ..)` directly on a `Scheduler`) and assert + it appears in `scheduler.tasks()` by name — no waiting for a tick. 2. **Dispatch on the tick.** Inside `ledger_heartbeat`, build a `Dispatcher` with a `MemoryChannel::new(Kind::EMAIL)`, dispatch a one-line statement, and assert (via `MemoryChannel::messages`) that the message was recorded. @@ -286,10 +529,18 @@ surface: `web::router(..)`, and use `firefly_testkit::sign_stripe` to drive a signed request through it with a `TestClient` — assert it is accepted, then tamper with the body and assert it is rejected. -4. **Audit the outbound.** Wire an `HmacDispatcher` over a `MemoryStore`, - dispatch a `CallbackEvent` to a `Target`, and read the recorded `Attempt` - rows from the store to confirm the retry policy fired. - -Lumen now does background work and knows where its messaging hangs. The next -chapter deepens the read-side cache the CQRS layer introduced. Continue to -[Caching](./17-caching.md). +4. **Audit the outbound.** Wire an `HmacDispatcher` over a `MemoryStore`, dispatch + a `CallbackEvent` to a `Target`, and read the recorded `Attempt` rows from the + store to confirm the retry policy fired. +5. **Pick fixed-delay over fixed-rate.** Register a `fixed_delay` task whose body + sleeps longer than the delay, run the scheduler briefly, and observe that runs + never overlap — then explain why a fixed-rate task with the same period would + have slipped instead. + +## Where to go next + +- Deepen the read-side cache the CQRS layer introduced in **[Caching](./17-caching.md)**. +- Revisit the actuator's `/actuator/scheduledtasks` feed and the admin + dashboard's Scheduled Tasks view in **[Observability](./15-observability.md)**. +- Test the schedulers, dispatchers, and webhook validators with the in-memory + channels and the `firefly-testkit` signers in **[Testing](./18-testing.md)**. diff --git a/docs/book/src/17-caching.md b/docs/book/src/17-caching.md index 0433bd5..90e61a6 100644 --- a/docs/book/src/17-caching.md +++ b/docs/book/src/17-caching.md @@ -1,30 +1,72 @@ # Caching -By the end of this chapter, you will understand exactly how Lumen's -`GET /api/v1/wallets/:id` serves a wallet view from a 30-second cache — and why -every deposit, withdrawal, and transfer *invalidates* that cache so a read after -a write never lies. The cache itself was switched on back in -[CQRS](./09-cqrs.md) with one annotation and one middleware; this chapter opens -it up: the unified cache port behind it, the backends you can swap in, and the -`firefly-resilience` decorators that protect the slow call a cache miss falls -through to. - -`firefly-cache` exposes a single port — `Adapter` — and ships in-process, -no-op, and fallback implementations plus a typed memoization wrapper. Code -against the port and select the backend (in-memory, Redis, Postgres) at wiring -time. All of it reaches Lumen through the one `firefly` facade, as -`firefly::cache` and `firefly::resilience`. - -> **The cache and resilience model.** `Adapter` is Firefly's single cache port; -> `Typed::get_or_set` is the read-through memoization primitive; the -> query-side `#[firefly(cache_ttl = "30s")]` is the declarative cache TTL Lumen -> uses. The resilience decorators — circuit breaker, rate limiter, bulkhead, -> timeout — compose into a `Chain` around any async call. - -## Lumen's read-side cache - -Lumen's `GetWallet` query carries its caching policy as a declaration sitting -right next to the type — no cache code in the handler: +Lumen's `GET /api/v1/wallets/:id` already serves a wallet view from a 30-second +cache — you switched it on back in [CQRS](./09-cqrs.md) with one annotation and +one bean, and have used it ever since without thinking about it. This chapter +opens that machinery up. We will trace a read from the query bus down to the +byte-level cache port underneath it, prove *why* every deposit, withdrawal, and +transfer must *invalidate* that cache so a read after a write never lies, and +then wrap the slow call a cache miss falls through to in the resilience +decorators that keep it from taking the whole service down. + +Two crates carry this story, and both reach Lumen through the one `firefly` +facade: `firefly-cache` (as `firefly::cache`) exposes a single cache port plus +a handful of backends and a typed wrapper, and `firefly-resilience` (as +`firefly::resilience`) ships the circuit breaker, rate limiter, bulkhead, and +timeout decorators. The CQRS read cache you already have — `firefly::cqrs::QueryCache` +— sits on top of the cache port. + +By the end of this chapter you will: + +- Explain how `#[firefly(cache_ttl = "30s")]` on a query turns into a real, + honored 30-second cache, and which bean honors it. +- Keep a read-after-write *honest* by invalidating a query family at every write + boundary, and prove the loop closes with Lumen's own HTTP test. +- Read and code against the `Adapter` cache port — the single trait every backend + (in-memory, Redis, Postgres) implements — and swap the backend at one wiring + point. +- Memoize an arbitrary value outside the query bus with `Typed::get_or_set`. +- Wrap a slow loader (or any outbound call) in a resilience `Chain` so a timeout, + an open circuit, or a full bulkhead fails fast instead of hanging. + +## Concepts you will meet + +Each of these is reintroduced in context where it is first used; this is the +short version so the words are not new when you hit them. + +> **Note** **Key term — cache.** A *cache* is a fast, usually in-memory store +> that holds the result of an expensive computation so the next request can skip +> the work. The hard part is never the storing — it is knowing when a stored value +> has gone stale. In Spring this is the `@Cacheable` / `@CacheEvict` family backed +> by a `CacheManager`. + +> **Note** **Key term — cache port.** A *port* is an abstract interface that +> consumers depend on instead of a concrete backend, so the backend can be swapped +> without touching the consumers. Firefly's cache port is the `Adapter` trait; +> the Spring analog is the `Cache` / `CacheManager` SPI behind `@Cacheable`. + +> **Note** **Key term — TTL.** *Time to live* is how long a cached entry stays +> valid before it expires and is treated as absent. A 30-second TTL means a read +> within 30 seconds of the last fill is served from cache; after that it re-runs +> the work. TTL alone is a stale-data ceiling, not a correctness guarantee — that +> is what invalidation is for. + +> **Note** **Key term — read-through / cache-aside.** A *read-through* (or +> *cache-aside*) read checks the cache first; on a miss it runs the real work +> (the *loader*), stores the result, and returns it. The next read inside the TTL +> skips the loader. `Typed::get_or_set` is Firefly's read-through primitive. + +> **Note** **Key term — resilience decorator.** A *resilience decorator* wraps an +> async call to bound its failure: a *circuit breaker* stops calling a sick +> dependency, a *rate limiter* caps the outbound rate, a *bulkhead* caps +> concurrency, and a *timeout* bounds duration. This mirrors Resilience4j in the +> Spring world. + +## Step 1 — See the cache you already have + +You did not have to write any cache code to get a cached read — you *declared* +it. Lumen's `GetWallet` query carries its caching policy as an attribute sitting +right next to the type, in `src/commands.rs`: ```rust /// `GET /api/v1/wallets/:id` query. `#[firefly(cache_ttl = "30s")]` is reflected @@ -33,13 +75,24 @@ right next to the type — no cache code in the handler: #[derive(Debug, Clone, Default, Serialize, Deserialize, Query)] #[firefly(cache_ttl = "30s")] pub struct GetWallet { + /// The wallet id to fetch. pub id: String, } ``` -The `#[derive(Query)]` macro reads that attribute and emits a `cache_ttl()` on -the generated `Message` impl — a fact the test pins so it can never silently -disappear: +What just happened: the `#[derive(Query)]` macro reads the `#[firefly(cache_ttl = +"30s")]` attribute and emits a `cache_ttl()` method on the generated `Message` +implementation. The attribute is *declarative* — it states the policy where the +type is defined, and the framework wires the behavior. Nothing in the query +handler mentions caching at all. + +> **Note** **Key term — declarative caching.** *Declarative* caching means the +> policy lives as an annotation on the type or method, not as imperative code in +> the body. Spring's `@Cacheable(ttl = ...)` is the analog; here it is +> `#[firefly(cache_ttl = "30s")]`. + +Because the TTL is now a fact on the generated code, Lumen pins it with a unit +test so it can never silently disappear: ```rust #[test] @@ -48,45 +101,105 @@ fn get_wallet_carries_cache_ttl() { } ``` -The TTL is inert until something *honors* it. That something is the `QueryCache` -middleware. In Lumen the `QueryCache` is declared as a `#[bean]` in `LumenBeans` -(the `#[derive(Configuration)]` holder in `src/web.rs`), and `FireflyApplication` -auto-installs its read-cache middleware on the bus whenever such a bean is present: +What just happened: the test constructs a default `GetWallet`, calls the +generated `cache_ttl()`, and asserts it returns `Some(_)`. If someone deletes the +attribute, this test fails — the caching contract is guarded, not assumed. + +> **Tip** **Checkpoint.** Open `samples/lumen/src/commands.rs` and find the +> `#[firefly(cache_ttl = "30s")]` line above `GetWallet`, plus the +> `get_wallet_carries_cache_ttl` test. The attribute and the assertion are the +> two ends of the same declaration. + +## Step 2 — Find the bean that honors the TTL + +A `cache_ttl()` on a message is inert until something *reads* it on the dispatch +path. That something is the `QueryCache` bean and the bus middleware it installs. + +> **Note** **Key term — bus middleware.** *Middleware* wraps every message that +> flows through the CQRS bus, running before and after the handler. The read-cache +> middleware checks the cache before the handler runs and fills it after, so a +> cached query never reaches the handler at all. This is Spring's `@Cacheable` +> interception, realized as a bus interceptor. + +In Lumen the `QueryCache` is declared as a single `#[bean]` inside `LumenBeans` +(the `#[derive(Configuration)]` holder in `src/web.rs`): ```rust use firefly::cqrs::QueryCache; -// samples/lumen/src/web.rs — the query_cache #[bean]. +// samples/lumen/src/web.rs — inside `#[bean] impl LumenBeans { ... }`. + +/// The read-side query cache honouring `GetWallet`'s 30s TTL (`@Bean`). #[bean] fn query_cache(&self) -> QueryCache { QueryCache::new() } ``` -`QueryCache::new()` builds the cache; the framework calls `query_cache.middleware()` -for you and registers it on the bus (validation is installed by the core). When a -`GetWallet` flows through the bus, the middleware checks the cache: a hit returns -the memoized `WalletView` without ever reaching the handler; a miss runs the -handler, stores the result under the query's key for 30 seconds, and returns it. -The same bean is autowired into the controller so the write side can reach back -into it. +What just happened: `QueryCache::new()` builds an empty, in-memory query cache +keyed by message type plus a hash of the message value. Declaring it as a `#[bean]` +is all the wiring you do — when `FireflyApplication::run()` component-scans the +container and finds a `QueryCache` bean, it calls `query_cache.middleware()` for +you and registers that middleware on the bus. (Validation middleware is installed +by the core; you do not register either by hand.) + +So when a `GetWallet` flows through the bus, the read-cache middleware: + +- **on a hit** returns the memoized `WalletView` *without ever reaching the + handler*; +- **on a miss** runs the handler, stores the result under the query's key for the + declared 30 seconds, and returns it. + +> **Design note.** This is the same auto-configuration pattern you saw in +> [Quickstart](./02-quickstart.md): "auto-configures the CQRS bus … the read-cache +> middleware whenever a `QueryCache` bean is present." You add a *bean*, not a +> registration call. Spring's analog is auto-configuring `@EnableCaching` behavior +> once a `CacheManager` bean exists. + +The same bean is also `#[autowired]` into the controller, so the write side can +reach the exact cache the middleware reads: + +```rust +// samples/lumen/src/web.rs — the WalletApi controller. +#[derive(Clone, Controller)] +pub struct WalletApi { + #[autowired] + pub bus: Arc, + #[autowired] + pub ledger: Arc, + /// The query cache, invalidated after a mutation so a read-after-write + /// never serves a stale balance within the 30s `GetWallet` TTL (autowired). + #[autowired] + pub query_cache: Arc, +} +``` + +What just happened: the framework installs *one* `QueryCache` as bus middleware +and hands the *same* `Arc` to the controller. `QueryCache` is +`Arc`-backed and cheap to clone, so both handles share the same entries — the +middleware fills the cache, and the controller can drop entries from it. -> **One bean, two readers.** The bean the framework installs as bus middleware is -> the same `Arc` the `WalletApi` controller autowires (`#[autowired] -> pub query_cache: Arc`). The middleware reads and fills the cache; the -> controller — holding the same handle — invalidates it, so the mutating handlers -> can drop stale entries the moment they change a balance. +> **Tip** **Checkpoint.** In `src/web.rs`, the `query_cache` `#[bean]` and the +> `#[autowired] pub query_cache: Arc` field refer to the same shared +> cache. One reads and fills it (middleware); the other invalidates it +> (controller). -### Read-after-write invalidation +## Step 3 — Keep read-after-write honest -A 30-second TTL is great for a read-heavy view and a disaster for correctness if +A 30-second TTL is a gift for a read-heavy view and a disaster for correctness if you never invalidate. Deposit `$2.50`, then read the balance within 30 seconds, -and a naive cache would happily serve the *old* number. Lumen avoids that by -invalidating the whole `GetWallet` family after every mutation. From the -controller in `src/web.rs`: +and a cache that only knew about the TTL would happily serve the *old* number. + +> **Note** **Key term — invalidation.** *Invalidation* (or *eviction*) is the +> deliberate removal of a cached entry that is now wrong, forcing the next read to +> re-run the work. Read-after-write correctness comes from invalidating at the +> write boundary — the moment a balance changes — not from waiting for a TTL. + +Lumen avoids the staleness by invalidating the whole `GetWallet` family after +every mutation. Here is the deposit handler in `src/web.rs`: ```rust -#[post("/wallets/:id/deposit")] +#[post("/wallets/:id/deposit", summary = "Deposit funds", status = 200)] async fn deposit( State(api): State, Path(id): Path, @@ -99,70 +212,147 @@ async fn deposit( } ``` -`invalidate_type::()` drops every cached `GetWallet` entry, so the -next read re-runs the handler and reflects the write. The withdraw handler does -the same, and so does the transfer endpoint from [Sagas](./12-sagas.md) — a -transfer changes *two* balances, so it must invalidate the family too: +What just happened, line by line: + +- `api.bus.send(cmd)` dispatches the `Deposit` command through the bus and awaits + the resulting `WalletView`. `map_err(cqrs_to_web)?` turns a CQRS error into an + RFC 9457 `application/problem+json` web error. +- `api.query_cache.invalidate_type::()` drops *every* cached + `GetWallet` entry. Internally `invalidate_type::()` deletes every cache key + prefixed with `Q`'s type name plus the `:` separator, so the whole `GetWallet` + family is cleared — the next read re-runs the handler and reflects the write. + +Why it matters: the TTL bounds how stale a value *can* get; the explicit +invalidation guarantees a read *after a write you made* is never stale at all. + +The withdraw handler does exactly the same, and so does the transfer endpoint +from [Sagas](./12-sagas.md) — a transfer changes *two* balances, so it must +invalidate the family too: ```rust // In the transfer handler — a transfer touches both wallets' views. api.query_cache.invalidate_type::(); ``` -The end-to-end HTTP test proves the loop closes: deposit then withdraw, then read -back, and the cached view reflects both writes rather than a stale balance. +What just happened: because the cache key includes the message *value*, a transfer +between wallet A and wallet B would otherwise have to evict two specific keys. +Invalidating the whole `GetWallet` type is simpler and always correct — it can +never miss a key — at the cost of dropping cache entries for unrelated wallets, +which simply re-fill on their next read. + +> **Design note.** Lumen pairs *read-through caching on the message* +> (`#[firefly(cache_ttl)]`) with *explicit eviction at the write boundary* +> (`invalidate_type`). The reader memoizes; the writer drops the family the moment +> it changes a balance. The backing store is the same swappable `Adapter` port +> every other cache consumer uses (Step 4), so this policy is independent of where +> the bytes actually live. + +The end-to-end HTTP test proves the loop closes. Open a wallet with a balance of +`100`, deposit `+250`, withdraw `-50`, then read it back through the cached `GET`: ```rust // after a deposit(+250) and a withdraw(-50) on an opening balance of 100: -let view: WalletView = body_json(res).await; +let view: WalletView = get_wallet(&app, &opened.id).await; assert_eq!(view.balance, 300); // read-after-write is honest assert_eq!(view.version, 3); ``` -> **Read-through plus explicit evict.** The cache declaration lives on the -> message (`#[firefly(cache_ttl)]`) and the eviction is an explicit -> `invalidate_type` at the write boundary — read-after-write stays honest because -> the writer drops the family the moment it changes a balance. The backing store -> is the same swappable `Adapter` port every other consumer uses. +What just happened: each mutating call invalidated the `GetWallet` family, so the +final `GET` re-ran the query against the read model rather than replaying a stale +cached view. The balance reflects both writes (`100 + 250 - 50 = 300`) and the +version is `3` (one event per mutation on top of the open). + +> **Tip** **Checkpoint.** Run the wallet HTTP tests: +> `cargo test -p lumen deposit_and_withdraw_update_the_balance`. A green test means +> the read-after-write loop closes — the cache is honored *and* invalidated. -## The cache port +## Step 4 — Trace the read down to the cache port -Everything above runs on `MemoryAdapter` by default, but the `QueryCache` — like -every other consumer (the session store) — depends on the -abstract `Adapter` port, never on a concrete client. That is what lets you move -Lumen's cache to Redis without touching a handler. The port: +Everything in Steps 1–3 runs on an in-process cache by default, but the +`QueryCache` — like every other cache consumer — ultimately depends on the +abstract `Adapter` port, never on a concrete client. That single seam is what +lets you move Lumen's cache to Redis without touching a single handler. + +Here is the port (from `firefly-cache`, reachable as `firefly::cache::Adapter`): ```rust,ignore +use std::time::Duration; +use async_trait::async_trait; + #[async_trait] pub trait Adapter: Send + Sync { + /// Returns the cached bytes for `key`, or `CacheError::NotFound` when absent. async fn get(&self, key: &str) -> Result, CacheError>; + + /// Stores `value` under `key` for `ttl` (None or zero = no expiry). async fn set(&self, key: &str, value: &[u8], ttl: Option) -> Result<(), CacheError>; + + /// Removes the entry. A missing key is a no-op. async fn delete(&self, key: &str) -> Result<(), CacheError>; + + /// Removes every entry. async fn clear(&self) -> Result<(), CacheError>; + + /// Human-readable adapter identifier (`memory`|`redis`|`noop`|...). + fn name(&self) -> String; + + /// Returns Ok when the backend is reachable. + async fn health_check(&self) -> Result<(), CacheError>; + + // The methods below ship default impls so older adapters keep compiling; + // backends with a cheaper native path (Redis SET NX, SCAN MATCH) override them. + + /// Writes only when `key` is absent; true when the write happened. async fn set_if_absent(&self, key: &str, value: &[u8], ttl: Option) -> Result; + + /// Whether a live entry exists for `key`. async fn exists(&self, key: &str) -> Result; + + /// Removes every entry whose key starts with `prefix`; returns the count. async fn delete_prefix(&self, prefix: &str) -> Result; - async fn health_check(&self) -> Result<(), CacheError>; + + /// A point-in-time counter snapshot, or None when the adapter has none. + async fn stats(&self) -> Option; } ``` -A miss is `CacheError::NotFound`; `ttl: None` (or zero) means no expiry. +What just happened: values cross the port as raw `Vec` — the port itself +knows nothing about your types. A cache miss is signalled by the +`CacheError::NotFound` variant (not an `Option`), and a `ttl` of `None` (or zero) +means "no expiry." The four trait methods with default implementations +(`set_if_absent`, `exists`, `delete_prefix`, `stats`) let an adapter ship without +them and let a richer backend override with a native, cheaper path. + +> **Note** **Key term — adapter.** An *adapter* is a concrete implementation of a +> port. Firefly ships several, and you choose one at wiring time: + +| Implementation | Backing | Use | +|-----------------------|------------------------|-------------------------------------------| +| `MemoryAdapter` | `HashMap` + `RwLock` | in-process, TTL-aware — **the default** | +| `NoOpAdapter` | none | tests / a deliberately disabled cache | +| `FallbackAdapter` | composite (two ports) | primary-then-secondary, writes to both | +| `RedisAdapter` | Redis (RESP) | distributed cache (`firefly-cache-redis`) | -| Implementation | Backing | Use | -|-----------------------------------------|----------------------------|----------------------------------------------| -| `MemoryAdapter` | `HashMap` + `RwLock` | in-process, TTL-aware, the default | -| `NoOpAdapter` | none | tests / disabled-cache configs | -| `FallbackAdapter { primary, secondary }` | composite | primary-then-secondary, writes to both | -| `RedisAdapter` (`firefly-cache-redis`) | Redis (RESP) | distributed cache | +A `NoOpAdapter` reports every `get` as `NotFound` and silently succeeds every +write — it is the cache that does nothing, perfect for a test that wants the +handler to run every time. `MemoryAdapter` is the live, TTL-aware in-process map +Lumen uses out of the box. -## Typed memoization +> **Tip** **Checkpoint.** You can name all four adapters and say which is the +> default (`MemoryAdapter`) and which disables caching (`NoOpAdapter`). All of +> them are `firefly::cache::*` and implement the one `Adapter` trait. -Lumen's `QueryCache` keys and serializes for you, but when you cache something -*outside* the query bus — say, a wallet's risk score fetched from an external -service — the `Typed` wrapper is the primitive. It serializes values as -`serde_json` bytes (wire-compatible with the other ports) and gives you -`get_or_set`: consult the cache, call the loader on a miss, persist, and return -the value. A caching error never masks a successful loader result: +## Step 5 — Memoize a value outside the query bus + +`QueryCache` keys and serializes query results for you. But when you want to cache +something that does *not* flow through the query bus — say, a wallet's risk score +fetched from an external service — the `Typed` wrapper is the primitive. + +> **Note** **Key term — `Typed`.** `Typed` wraps an `Adapter` with +> JSON-encoded read/write helpers for a concrete type `T`. It serializes values as +> `serde_json` bytes (wire-compatible with the other ports) and gives you +> `get_or_set`: consult the cache, call the loader on a miss, persist the result, +> and return it. A caching error never masks a successful loader result. ```rust use std::sync::Arc; @@ -170,7 +360,9 @@ use std::time::Duration; use firefly::cache::{MemoryAdapter, Typed}; #[derive(serde::Serialize, serde::Deserialize)] -struct WalletView { id: String } +struct WalletView { + id: String, +} #[tokio::main(flavor = "current_thread")] async fn main() -> Result<(), firefly::cache::CacheError> { @@ -188,50 +380,106 @@ async fn main() -> Result<(), firefly::cache::CacheError> { } ``` -## Choosing and composing backends +What just happened, block by block: + +- `Arc::new(MemoryAdapter::new())` builds the byte-level cache and wraps it in an + `Arc` so `Typed` can share it. `Typed::new(cache)` layers JSON encoding on top + for the `WalletView` type. +- `get_or_set(key, ttl, loader)` is the read-through call. On the first run the key + is absent, so the loader closure runs, its `WalletView` is JSON-encoded and + stored under the key for 60 seconds, and the value is returned. A second call + within 60 seconds skips the loader and decodes the stored bytes. +- The loader returns `Result` — any error inside it + surfaces; but a failure to *write* the loaded value back to the cache does + **not** mask a successful load (the value is still returned). + +`Typed` also offers `put` (always write and return — the always-store path), +`delete` (remove one key), and `delete_prefix` (evict a key family), but +`get_or_set` is the workhorse. + +> **Tip** **Checkpoint.** You can describe the three outcomes of `get_or_set`: a +> hit (decode and return, no loader), a miss (run loader, store, return), and a +> store-failure-after-load (return the value anyway, drop the write error). + +## Step 6 — Swap and compose backends at one wiring point -`Core::new` (and therefore `WebStack`, which Lumen builds on) uses -`MemoryAdapter` by default; pass an `Arc` in -`CoreConfig.cache` to swap it. For high availability, compose Redis with an -in-process fallback so a Redis blip degrades to local caching instead of -failing: +The default cache is `MemoryAdapter`. Where does that default live? `Core::new` +(and therefore `WebStack`, which Lumen builds on) reads `CoreConfig.cache: Option>` +and substitutes a `MemoryAdapter` when it is `None`. To use a different backend, +you pass a different `Arc` there — one constructor, nothing else. + +> **Note** **Key term — `FallbackAdapter`.** A `FallbackAdapter` is itself an +> `Adapter` that wraps a *primary* and a *secondary*: it tries the primary first +> and, on a transport failure (anything other than a plain miss), demotes the +> request to the secondary and writes to both. Consumers never see the failover — +> they just see an `Adapter`. + +For high availability, compose Redis with an in-process fallback so a Redis blip +degrades to local caching instead of failing the request: ```rust,ignore use std::sync::Arc; -use firefly::cache::{FallbackAdapter, MemoryAdapter}; +use firefly::cache::{FallbackAdapter, MemoryAdapter, RedisAdapter}; -// Tries Redis first; on a transport error or miss, falls through to memory -// and writes to both. -let cache = FallbackAdapter::new(redis_adapter, Arc::new(MemoryAdapter::new())); +// Connect the distributed primary (RESP over the network)... +let redis = Arc::new(RedisAdapter::connect("redis://127.0.0.1:6379/0").await?); + +// ...and fall through to a local in-process cache on a transport error or miss, +// writing to both so the local layer warms up. +let cache: Arc = + Arc::new(FallbackAdapter::new(redis, Arc::new(MemoryAdapter::new()))); ``` +What just happened: `RedisAdapter::connect(url)` dials Redis and returns a ready +adapter; `FallbackAdapter::new(primary, secondary)` composes it with a +`MemoryAdapter`. The composite is *also* an `Adapter`, so you hand it to +`CoreConfig.cache` exactly as you would any single backend. + Because everything downstream depends on the port, swapping the backend changes -one constructor — Lumen's handlers, the `QueryCache`, and the session store are +*one* constructor — Lumen's handlers, the `QueryCache`, and the session store are untouched. A single-process Lumen keeps the default in-memory cache; a multi-node -deployment swaps in `RedisAdapter` so a `GetWallet` cached on one node is seen on -the next, and so an `invalidate_type` on any node clears the shared entry. - -> **The in-memory → real-infra swap.** This mirrors the event store and broker -> swap you have already seen: develop and test against the in-memory adapter, -> wire the distributed backend in production via `CoreConfig`. The teaching -> baseline stays a no-infra `cargo run`; the production path is one line of -> wiring. - -## Resilience decorators - -`firefly-resilience` guards the calls a cache miss falls through to — and any -outbound call, like the Payments settlement from -[HTTP Clients](./13-http-clients.md). Four primitives, composable into a -`Chain`: - -| Decorator | Guards against | Error on trip | -|------------------|----------------------------------------------|--------------------------------| -| `CircuitBreaker` | cascading failure of a slow / failing dep | `ResilienceError::CircuitOpen` | -| `RateLimiter` | outbound rate overrun (token bucket) | `ResilienceError::RateLimited` | +deployment swaps in `RedisAdapter` so a `GetWallet` cached on one node is visible +on the next, and so an `invalidate_type` on any node clears the shared entry. + +> **Design note.** This mirrors the event-store and broker swap you have already +> seen: develop and test against the in-memory adapter, wire the distributed +> backend in production via `CoreConfig`. The teaching baseline stays a no-infra +> `cargo run`; the production path is one line of wiring. [Production & +> Deployment](./20-production.md) does exactly this swap for real. + +> **Tip** **Checkpoint.** You can name the single seam — `CoreConfig.cache` — that +> changes the cache backend for the whole service, and explain why no handler, +> `QueryCache`, or controller has to change when you swap it. + +## Step 7 — Protect the loader with resilience decorators + +A cache miss falls through to a slow call: the read model, the event store, or an +external service. If that call hangs or starts failing, an unguarded loader can +drag the whole service down with it. `firefly-resilience` guards exactly that — and +any outbound call, like the Payments settlement from [HTTP +Clients](./13-http-clients.md). + +> **Note** **Key term — circuit breaker.** A *circuit breaker* watches a guarded +> call. While it is *closed*, calls flow through and failures are counted; after +> enough failures it *opens* and short-circuits subsequent calls with an immediate +> error, sparing the sick dependency; after a cooldown it lets one trial call +> through (*half-open*) to decide whether to close again. This is Resilience4j's +> `CircuitBreaker`. + +There are four decorators, each shielding one failure mode: + +| Decorator | Guards against | Error on trip | +|------------------|----------------------------------------------|---------------------------------| +| `CircuitBreaker` | cascading failure of a slow / failing dep | `ResilienceError::CircuitOpen` | +| `RateLimiter` | outbound rate overrun (token bucket) | `ResilienceError::RateLimited` | | `Bulkhead` | resource exhaustion from runaway concurrency | `ResilienceError::BulkheadFull` | -| `Timeout` | stuck calls | `ResilienceError::Timeout` | +| `Timeout` | stuck calls | `ResilienceError::Timeout` | -A `Chain` composes them into a single guarded call: +> **Note** **Key term — `Chain`.** A `Chain` composes decorators into a single +> guarded call. Decorators run left-to-right with the leftmost outermost, so +> `Chain::new().with(timeout).with(breaker).with(bulkhead)` evaluates as +> `timeout(breaker(bulkhead(call)))` — a deadline bounds the whole call while the +> breaker and bulkhead protect the inner operation. ```rust,no_run use std::{sync::Arc, time::Duration}; @@ -242,7 +490,7 @@ let breaker = Arc::new(CircuitBreaker::new(CircuitConfig::default())); let guarded = Chain::new() .with(Timeout::new(Duration::from_secs(2))) // per-call deadline (outermost) - .with(breaker) // open the circuit on repeated failures + .with_shared(breaker.clone()) // open the circuit on repeated failures .with(Bulkhead::new(20)); // cap concurrent in-flight calls guarded.execute(|| async { @@ -253,38 +501,63 @@ guarded.execute(|| async { # } ``` -Each decorator also stands alone: +What just happened, line by line: + +- `CircuitBreaker::new(CircuitConfig::default())` builds a breaker with the default + policy (trip after 5 failures, stay open 30 seconds). It is wrapped in `Arc` so + you can both hand it to the chain *and* keep a handle to inspect its state. +- `Chain::new()` starts an empty chain. `.with(decorator)` appends a decorator the + chain *owns*; `.with_shared(arc_decorator)` appends one you keep a handle to — + that is why the breaker uses `.with_shared(breaker.clone())` while the freshly + built `Timeout` and `Bulkhead` use `.with(...)`. +- `guarded.execute(|| async { ... })` runs your closure through all three + decorators, leftmost outermost. The closure returns `Result<(), ResilienceError>`; + if any decorator trips, `execute` returns that decorator's error and your + operation may never run. + +> **Warning** `Chain::with(...)` takes ownership and requires its argument to +> implement the decorator trait directly — a bare `Arc` does *not*. +> When you want to keep a handle to a breaker (to read its state, or to share it +> across chains), use `.with_shared(breaker.clone())`, which takes the `Arc`. Using +> `.with(breaker)` on an `Arc` will not compile. + +Each decorator also stands alone. Unlike `Chain::execute` (whose value you +discard), `CircuitBreaker::execute` *returns the operation's value*, so a guarded +read still hands you the `WalletView`: ```rust,ignore use std::time::Duration; use firefly::resilience::{Bulkhead, CircuitBreaker, CircuitConfig, RateLimiter, Timeout}; let cb = CircuitBreaker::new(CircuitConfig::default()); -let _ = cb.execute(|| async { settle().await }).await; +let _ = cb.execute(|| async { settle().await }).await; // returns settle()'s value -let rl = RateLimiter::new(100.0, 200); // 100 rps, burst 200 +let rl = RateLimiter::new(100.0, 200); // 100 rps, burst 200 let _ = rl.execute(|| async { call().await }).await; let bh = Bulkhead::new(20); -let _ = bh.try_execute(|| async { call().await }).await; // non-blocking; BulkheadFull if full +let _ = bh.try_execute(|| async { call().await }).await; // non-blocking; BulkheadFull if full let to = Timeout::new(Duration::from_secs(2)); let _ = to.execute(|| async { slow_call().await }).await; ``` -> **Decorator ordering.** `Chain` composes the four decorators in call-wrap -> order — timeout outermost, breaker, bulkhead innermost — so a deadline bounds -> the whole guarded call while the breaker and bulkhead protect the inner -> operation. `CircuitBreaker::execute` returns the operation's value (so a guarded -> read still hands you the `WalletView`), while `Chain::execute` is for guarded -> operations whose value you discard. +What just happened: each primitive has its own `execute` that wraps a closure +returning `Result` and propagates the operation's value on +success. `Bulkhead` additionally offers `try_execute`, the non-blocking variant +that returns `BulkheadFull` immediately rather than waiting for a free slot. -## A complete read path +> **Tip** **Checkpoint.** You can explain the difference between `Chain::execute` +> (value discarded, returns `Result<(), _>`) and `CircuitBreaker::execute` (returns +> the operation's `T`), and you know to reach for `.with_shared(arc.clone())` when +> the chain needs a breaker you still hold. -The pieces fit together as a cache-aside read protected by a circuit breaker — -exactly the shape a multi-node Lumen would use to serve a wallet view from Redis, -repairing from the read model (or the event stream) on a miss while a breaker -protects that repair: +## Step 8 — Assemble a resilient cache-aside read + +The two halves of this chapter compose into a single shape: a cache-aside read +whose loader is protected by a circuit breaker. This is exactly what a multi-node +Lumen would use to serve a wallet view from Redis, repairing from the read model +(or the event stream) on a miss while the breaker protects that repair: ```rust,ignore use std::sync::Arc; @@ -298,55 +571,93 @@ let breaker = CircuitBreaker::new(CircuitConfig::default()); let view = typed .get_or_set("wallet:wlt_alice", Some(Duration::from_secs(30)), || async { // the loader is what the circuit protects: the read model / event store. - breaker.execute(|| async { load_wallet_view("wlt_alice").await }).await + breaker + .execute(|| async { load_wallet_view("wlt_alice").await }) + .await .map_err(|e| firefly::cache::CacheError::Backend(e.to_string())) }) .await?; ``` -You now have a fast, resilient read path — the same one Lumen's declarative -`#[firefly(cache_ttl = "30s")]` gives you for free on the query bus. - -## What changed in Lumen - -- We opened up the read-side cache Lumen has used since CQRS: - `#[firefly(cache_ttl = "30s")]` on `GetWallet` is honored by the `QueryCache` - read-cache middleware that `FireflyApplication` auto-installs on the bus - whenever a `QueryCache` `#[bean]` is present. -- We saw that the `QueryCache` is a single `#[bean]` — installed as bus middleware - by the framework and **autowired** into the controller — so every mutating - handler — deposit, withdraw, **and** transfer — can call - `invalidate_type::()` and keep read-after-write honest within the - 30-second TTL. -- We traced the cache down to its `Adapter` port, the swappable backends - (`MemoryAdapter` default → `RedisAdapter` for a multi-node deployment via - `CoreConfig.cache`), the `Typed::get_or_set` memoization primitive, and the - `FallbackAdapter` for Redis-with-local-fallback. -- We covered the `firefly-resilience` decorators (`CircuitBreaker`, `RateLimiter`, - `Bulkhead`, `Timeout`) and the `Chain` that composes them — the guard that - protects both a cache loader and the outbound Payments call from - [HTTP Clients](./13-http-clients.md). +What just happened: `get_or_set` is the outer cache-aside read. On a hit it returns +the decoded `WalletView` and the loader never runs. On a miss the loader runs the +real repair — but wrapped in `breaker.execute(...)`, so a streak of failures opens +the circuit and the *next* miss fails fast with `CircuitOpen` instead of hammering +a sick read model. The `map_err` adapts the `ResilienceError` into a +`CacheError::Backend` so it fits `get_or_set`'s error type. + +You now have a fast, resilient read path — built by hand from the two primitives, +and the same shape Lumen's declarative `#[firefly(cache_ttl = "30s")]` gives you +for free on the query bus. + +> **Tip** **Checkpoint.** You can trace the layering: `get_or_set` (cache-aside) +> wraps `breaker.execute` (failure protection) wraps the real loader (read model / +> event store). The cache absorbs the happy path; the breaker absorbs the failure +> path. + +## Recap — what you now understand about Lumen's cache + +- The read-side cache Lumen has used since [CQRS](./09-cqrs.md) is *declarative*: + `#[firefly(cache_ttl = "30s")]` on `GetWallet` is honored by the read-cache bus + middleware that `FireflyApplication` auto-installs whenever a `QueryCache` + `#[bean]` is present. +- The `QueryCache` is a single bean — installed as bus middleware by the framework + and `#[autowired]` into the controller — so every mutating handler (deposit, + withdraw, **and** transfer) calls `invalidate_type::()` to keep + read-after-write honest within the 30-second TTL. +- Underneath the `QueryCache` is the swappable `Adapter` port: `MemoryAdapter` by + default, `NoOpAdapter` to disable caching, `FallbackAdapter` for + Redis-with-local-fallback, and `RedisAdapter` for a multi-node deployment — + selected at one wiring point, `CoreConfig.cache`. +- `Typed::get_or_set` is the read-through memoization primitive for values + outside the query bus; a write failure after a successful load never masks the + value. +- `firefly-resilience` ships `CircuitBreaker`, `RateLimiter`, `Bulkhead`, and + `Timeout`, composable through a `Chain`, to guard both a cache loader and any + outbound call (like the Payments settlement from [HTTP + Clients](./13-http-clients.md)). ## Exercises -1. **Prove the TTL.** Write a test that opens a wallet, reads it (priming the - cache), then deposits *directly through the ledger* (bypassing the controller, - so no invalidation runs) and reads again within 30 seconds. Assert you still - see the *old* balance — demonstrating the TTL is real — then call - `query_cache.invalidate_type::()` and assert the read now reflects +1. **Prove the TTL is real.** Write a test that opens a wallet, reads it (priming + the cache), then deposits *directly through the ledger* (bypassing the + controller, so no `invalidate_type` runs) and reads again within 30 seconds. + Assert you still see the *old* balance — demonstrating the TTL is genuinely + serving a memoized value — then call + `query_cache.invalidate_type::()` and assert the next read reflects the deposit. -2. **Swap in a fallback adapter.** Build a `FallbackAdapter` whose primary always - errors on `get`/`set` and whose secondary is a `MemoryAdapter`. Wire it into - `CoreConfig.cache`, run the deposit/withdraw/read flow, and assert correctness - is unaffected — the cache degrades to the in-process layer instead of failing. - -3. **Guard a loader with a `Chain`.** Wrap a deliberately slow loader in a - `Chain::new().with(Timeout::new(...))` and assert that a loader exceeding the - deadline surfaces `ResilienceError::Timeout` rather than hanging. Then add a - `CircuitBreaker` to the chain, trip it with repeated failures, and assert the - next call fails fast with `ResilienceError::CircuitOpen`. - -The remaining chapters fold Lumen back together through the declarative-macro -lens, then cover shipping it. Continue to -[Declarative Services with Macros](./21-declarative-macros.md). +2. **Disable the cache with `NoOpAdapter`.** The `QueryCache` itself is in-memory, + but the byte-level cache the rest of the service uses is `CoreConfig.cache`. + Build a `CoreConfig` with `cache: Some(Arc::new(NoOpAdapter::default()))`, boot + the service, and confirm the byte-level cache always reports a miss while the + wallet flow still passes — useful when you want to measure cold-path latency. + +3. **Swap in a fallback adapter.** Build a `FallbackAdapter` whose primary always + errors on `get`/`set` (a hand-rolled `Adapter` that returns + `CacheError::Backend(...)`) and whose secondary is a `MemoryAdapter`. Wire it + into `CoreConfig.cache`, run the deposit/withdraw/read flow, and assert + correctness is unaffected — the cache degrades to the in-process layer instead + of failing. + +4. **Guard a loader with a `Chain`.** Wrap a deliberately slow loader in a + `Chain::new().with(Timeout::new(Duration::from_millis(50)))` and assert a loader + exceeding the deadline surfaces `ResilienceError::Timeout` (check + `err.is_timeout()`) rather than hanging. Then add `.with_shared(breaker.clone())` + for a `CircuitBreaker`, trip it with repeated failures, and assert the next call + fails fast with `ResilienceError::CircuitOpen` (check `err.is_circuit_open()`). + +5. **Memoize outside the bus.** Use `Typed::get_or_set` to cache a computed + value (e.g. a wallet's risk score) under a 10-second TTL. Call it twice with a + loader that increments a counter, and assert the counter advanced only once — + proving the second call hit the cache rather than re-running the loader. + +## Where to go next + +- See how *every* declaration in this chapter — `#[firefly(cache_ttl)]`, + `#[bean]`, `#[autowired]`, `#[rest_controller]` — is produced by Firefly's macro + layer in **[Declarative Services with Macros](./21-declarative-macros.md)**. +- Drive the cached read-after-write loop end-to-end, in-process and with no socket + bound, in **[Testing](./18-testing.md)**. +- Perform the in-memory → Redis cache swap for a real deployment in **[Production + & Deployment](./20-production.md)**. diff --git a/docs/book/src/18-testing.md b/docs/book/src/18-testing.md index d1d0c6d..852503a 100644 --- a/docs/book/src/18-testing.md +++ b/docs/book/src/18-testing.md @@ -2,64 +2,142 @@ Every chapter so far has shown Lumen's listings *and* the tests that keep them honest — that is the whole point of the book: the prose is verified against a -crate that compiles and passes its suite. This chapter steps back and looks at -the test strategy as a whole, the way you would design it for your own service. - -By the end of this chapter you will know how Lumen tests at three levels — pure -unit tests with no I/O, in-process HTTP tests that drive the *real* router -through `tower::oneshot`, and the `firefly-testkit` helpers (`TestClient`, -`Slice`, `assert_event_published`) that make all of it terse — and how the same -crate scales up to real-infrastructure integration tests when you need a live -Postgres or Kafka. Lumen's gate is **42 unit tests + 12 HTTP tests + 1 doctest**, -all hermetic; the streaming feature adds 3 more. - -> **Design note.** Firefly's testing surface is built as three deliberate tiers -> that map onto how a service actually layers: plain `#[tokio::test]` unit tests -> with no I/O, in-process slice and HTTP tests that drive the real router without -> binding a socket, and env-gated integration tests against live infrastructure. -> `TestClient` is Firefly's in-process HTTP client, `Slice` is a focused -> dependency-injection container for a single test, and `assert_event_published` -> is the event-emission assertion — one terse helper per tier. The split will -> feel familiar if you've used a batteries-included framework, but each piece is -> a native Firefly API designed around the in-memory-first stack the rest of the -> book builds on. +crate that compiles and passes its suite. This chapter steps back from any one +feature and looks at the test strategy as a whole — the way you would design it +for your own service. You will not learn a single new business rule here; you +will learn how to prove the ones you already wrote, at three levels, without +booting a server or starting a database. + +The good news is that Firefly's in-memory-first stack makes almost every test a +plain function call. Lumen's default infrastructure — its event store, event +broker, and read model — is pure Rust running in-process, so a test never binds +a socket, never opens a connection, and never waits on a container. The result +is a suite that is fast, deterministic, and green on a bare laptop. + +By the end of this chapter you will: + +- Understand Firefly's three testing tiers — pure unit tests, in-process HTTP + tests that drive the *real* router, and env-gated integration tests against + live infrastructure — and when to reach for each. +- Drive a fully-wired application router in-process with `bootstrap()` and + `tower::oneshot`, with no socket bound and no mocks. +- Use the `firefly-testkit` helpers — `TestClient`, `Slice`, + `assert_event_published`, and the webhook signers — to write the same tests + far more tersely. +- Build a focused dependency-injection slice for a single unit, install a fake + collaborator (the `@MockBean` analog), and drive one controller over mocks + (the `@WebMvcTest` analog). +- Write an integration test that uses real Postgres or Kafka when present and + **skips cleanly** when it is not, so `cargo test` stays green everywhere. + +## Concepts you will meet + +Before the first test, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — testing tier.** A *tier* is one layer of the test +> pyramid: pure unit tests at the bottom (fastest, most numerous), in-process +> HTTP/slice tests in the middle, and integration tests against live +> infrastructure at the top (slowest, fewest). Firefly gives you one terse helper +> per tier. The split mirrors the JUnit + Spring Boot test stack: plain `@Test`, +> `@SpringBootTest` / `@WebMvcTest`, and `@Testcontainers`. + +> **Note** **Key term — in-process HTTP test.** An *in-process* test drives the +> real HTTP router by handing it a `Request` and `await`ing the `Response` +> directly — no port is opened and no server task is spawned. It is the speed of +> a unit test with the coverage of an end-to-end test. The Spring analog is +> `MockMvc` (and Spring's `WebTestClient` in `MOCK` mode). + +> **Note** **Key term — test seam.** A *seam* is a place the framework exposes +> specifically so tests can reach inside. Firefly's seam is `bootstrap()`: it +> assembles the same fully-wired application `run()` would serve, but hands it +> back as a value *without* binding a socket. Spring's `@SpringBootTest` boots +> the same context the production `main` does; `bootstrap()` is its Rust analog. + +> **Note** **Key term — mock / fake.** A *fake* is a stand-in collaborator you +> install in place of the real one — an in-memory repository instead of a +> database, a canned service instead of a network call. Installing one is the +> `@MockBean` move from Spring: override a bean under its port so the unit under +> test wires the fake instead of the real implementation. ## The in-process testing model -Lumen's default stack is entirely in-memory — `MemoryEventStore`, -`InMemoryBroker`, a `Mutex` read model — so almost every test runs as a -plain `#[tokio::test]` with **no socket and no external service**. The HTTP tests -do not bind a port; they hand a `Request` to the router and `await` the -`Response`. That is fast, deterministic, and CI-friendly. - -The model is worth stating up front, because Lumen's tests are built around it. -Each test boots **one** app context with `build_router()` and drives every request -against it — Spring Boot's `@SpringBootTest` model. `build_router()` bootstraps a -`FireflyApplication` -(`FireflyApplication::new(APP_NAME).version(VERSION).bootstrap().await`) and hands -back its public router — the same component-scanned container, auto-mounted -controller, inventory-drained handler/projection beans, and auto-installed -middleware the real `main()` boots. The CQRS handlers (`WalletHandlers`) and the -read-model projection (`WalletProjection`) are now **autowired DI beans**, not -free functions over a process-global, so each test's container is self-consistent: -the `Ledger`, `ReadModel`, and `QueryCache` singletons one container resolves are -the *same* instances every handler and the projection share. A wallet a command -opens is therefore the wallet a later query reads, because both run against the -one container the test booted. The `axum::Router` is cheap to `clone` (it is -`Arc`-backed), so each request `clone`s the shared app rather than rebuilding it. - -## Unit tests with no infrastructure - -The domain and the value object are pure, so their tests need nothing. Lumen's -`money.rs` and `domain.rs` assert invariants directly — exact-cents arithmetic, -positive amounts, sufficient funds, owner required. The CQRS layer is just as -direct: the handlers are a `#[derive(Service)]` bean (`WalletHandlers`), so a unit -test constructs it with its collaborators in hand and calls a method straight, -asserting the result. This is the heart of `commands.rs`'s test module: +Lumen's default stack is entirely in-memory — a `MemoryEventStore`, an +`InMemoryBroker`, and a `Mutex` read model — so almost every test runs +as a plain `#[tokio::test]` with **no socket and no external service**. Even the +HTTP tests do not bind a port: they hand a `Request` to the router and `await` +the `Response`. That single fact is what makes the suite fast and CI-friendly, +and it is worth stating up front because every tier below is built around it. + +The model has one organizing rule: each test boots **one** application context +and drives every request against it. That is exactly Spring Boot's +`@SpringBootTest` model — one wired context per test method — and in Lumen the +helper that gives it to you is `build_router()`: + +```rust,ignore +// src/web.rs — the test seam, compiled only under #[cfg(test)]. +#[cfg(test)] +pub(crate) async fn build_router() -> axum::Router { + firefly::FireflyApplication::new(APP_NAME) + .version(VERSION) + .bootstrap() + .await + .expect("lumen bootstrap") + .api_router +} +``` + +What just happened: `bootstrap()` runs the same boot pipeline as `run()` — +component-scan the DI container, auto-mount every `#[rest_controller]`, +auto-discover security and middleware, drain the inventory-registered CQRS +handlers / EDA listeners / `#[scheduled]` tasks — and returns a `Bootstrapped` +value instead of serving it. Its `.api_router` field is the public +`axum::Router`, fully wired, with no listener bound. `build_router()` is just +`main()` minus the `.run()` serve step. + +> **Note** **Key term — bootstrap seam.** `bootstrap()` is the sibling of +> `run()` you met in [Quickstart](./02-quickstart.md): `run()` assembles the app +> *and serves it*; `bootstrap()` assembles the identical app and returns the +> `Bootstrapped` handle so a test can drive `Bootstrapped::api_router` +> in-process. Same beans, same wiring, no socket. + +Because the CQRS handlers (`WalletHandlers`) and the read-model projection +(`WalletProjection`) are **autowired DI beans** — not free functions over a +process-global — each test's container is self-consistent. The `Ledger`, +`ReadModel`, and `QueryCache` singletons that one container resolves are the +*same* instances every handler and the projection share. So a wallet a command +opens is the wallet a later query reads, because both run against the one +container the test booted. And since an `axum::Router` is cheap to `clone` (it is +`Arc`-backed), each request clones the shared app rather than rebuilding it. + +> **Tip** **Checkpoint.** You can already run the whole suite. From the workspace +> root, `cargo test -p firefly-sample-lumen` builds Lumen and runs its tests; you +> should see `42 unit + 12 HTTP + 1 doctest` pass. The rest of this chapter +> explains what those tests *are*. + +## Tier 1 — Unit tests with no infrastructure + +The bottom tier needs nothing: no router, no container, no I/O. Lumen's value +object and aggregate are pure Rust, so their tests construct a value and assert +an invariant directly. `money.rs` and `domain.rs` check exact-cents arithmetic, +positive amounts, sufficient funds, and the "owner required" rule with plain +`assert!`s. + +The CQRS layer is just as direct. The handlers live on a `#[derive(Service)]` +bean (`WalletHandlers`) whose collaborators — the write-side `Ledger` and the +read-side `ReadModel` — are `#[autowired]` from the container at boot. But +nothing stops you from constructing the bean yourself with those collaborators in +hand and calling a method straight. This is the heart of `commands.rs`'s test +module: ```rust,ignore +use firefly::eda::InMemoryBroker; +use firefly::eventsourcing::MemoryEventStore; + #[tokio::test] async fn handler_bean_operates_on_its_autowired_collaborators() { + // Build the handler bean with the same Ledger + ReadModel the container + // would inject — no bus, no process-global, no boot. let handlers = WalletHandlers { ledger: Arc::new(Ledger::new( Arc::new(MemoryEventStore::new()), @@ -82,46 +160,78 @@ async fn handler_bean_operates_on_its_autowired_collaborators() { } ``` -This unit test builds the handler bean with the same `Ledger` + `ReadModel` the -container would inject and drives its methods directly — no bus, no process-global. -The full application boot installs the *same* bean on the bus by draining the -inventory registry (`register_discovered_handlers`), so the test exercises the real -handlers without standing up the container. +What just happened, and why it matters: you built the handler bean by hand with +an in-memory `Ledger` and a fresh `ReadModel`, then called `open_wallet` and +`deposit` directly and asserted the returned balances. No bus dispatch, no DI +container, no HTTP. The full application boot installs the *same* bean on the bus +by draining the inventory registry (`register_discovered_handlers`), so this test +exercises the real handler logic without standing any of that up. When you want +to know "does the handler do the right arithmetic?", this is the cheapest place +to find out. -Validation is tested without ever touching HTTP — call `.validate()` on the -command directly, because `#[derive(Command)]` generated it from the -`#[firefly(validate)]` fields: +Validation is tested the same way — without ever touching HTTP. `OpenWallet` +carries `#[derive(Command)]`, which generated a `.validate()` from its +`#[firefly(validate)]` fields, so you call it on the command directly: ```rust,ignore #[test] fn open_wallet_validates_owner() { - assert!(OpenWallet::default().validate().is_err()); // empty owner + assert!(OpenWallet::default().validate().is_err()); // empty owner fails assert!(OpenWallet { owner: "alice".into(), opening_balance: 0 }.validate().is_ok()); } ``` -Security (`security.rs`), the saga (`transfer.rs`), and the scheduled task -(`housekeeping.rs`) each carry their own `#[cfg(test)] mod tests` in the same -spirit — mint-then-verify a token, run the saga happy path and the compensation -path, register the heartbeat and assert it ticks. - -## In-process HTTP tests with `tower::oneshot` - -The end-to-end suite lives in `src/http_test.rs` (a `#[cfg(test)] mod http_test` -declared in `main.rs`, so it runs as part of the binary's own test target) and -drives the **fully-wired** -`build_router()` — which bootstraps a `FireflyApplication` and returns its public -router: the auto-mounted `#[rest_controller]` routes, the CQRS handler bean, the -event-sourced ledger, the read-model projection bean, the transfer saga, *and* the -auto-discovered JWT/RBAC enforcement from Chapter 14. No mocks: every layer is the -production layer, just over in-memory infrastructure. - -The pattern is one `Router` per test + `tower::ServiceExt::oneshot`. A test boots -the app once (`let app = build_router().await`) and drives every request against -it; the helpers take the booted `&axum::Router` and `clone` it per request -(`app.clone().oneshot(req)`), so they all share the one container: +What just happened: the empty default fails validation (no owner), and a +well-formed command passes — all before any handler runs. The web layer never +sees an invalid command because the bus rejects it first; this test pins that +rejection at the cheapest possible level. + +> **Note** Security (`security.rs`), the transfer saga (`transfer.rs`), the +> compliance workflow (`compliance.rs`), the two-phase transfer +> (`tcc_transfer.rs`), and the scheduled task (`housekeeping.rs`) each carry +> their own `#[cfg(test)] mod tests` in the same spirit: mint-then-verify a +> token, run the saga happy path *and* its compensation path, run the workflow's +> approve/reject branches, and register the heartbeat and assert it ticks. These +> are the chapters [Security](./14-security.md), [Sagas, Workflows & +> TCC](./12-sagas.md), and [Scheduling & +> Notifications](./16-scheduling-notifications.md) proving themselves. + +> **Tip** **Checkpoint.** Together these account for Lumen's **42 unit tests**: +> `money` and `domain` invariants, `commands` validation plus the handler bean, +> `security` mint/verify/reject, `transfer`/`tcc_transfer` happy + compensation, +> `compliance` approve/reject, and `housekeeping` registration + tick. Run +> `cargo test -p firefly-sample-lumen --lib` to see just these. + +## Tier 2 — In-process HTTP tests with `tower::oneshot` + +The middle tier proves the whole stack composes. Lumen's end-to-end suite lives +in `src/http_test.rs` — a `#[cfg(test)] mod http_test` declared in `main.rs`, so +it runs as part of the binary's own test target — and drives the **fully-wired** +`build_router()`: the auto-mounted `#[rest_controller]` routes, the CQRS handler +bean, the event-sourced ledger, the read-model projection bean, the transfer +saga, *and* the auto-discovered JWT/RBAC enforcement from +[Security](./14-security.md). No mocks: every layer is the production layer, just +over in-memory infrastructure. + +> **Note** **Key term — `tower::oneshot`.** `oneshot` (from +> `tower::ServiceExt`) sends exactly one request through a `Service` — here an +> `axum::Router` — and resolves to its `Response`, then drops the service. It is +> how you call a router as a plain async function. The router's body type comes +> from `http_body_util::BodyExt`, which you use to collect the response bytes. + +### Step 1 — Write the request/response helpers + +The pattern is one `Router` per test plus `oneshot` per request. A test boots the +app once with `let app = build_router().await` and drives every request against +it; a small `send` helper clones the shared `&Router` per request so they all +share the one container. Here are the helpers `http_test.rs` defines once at the +top of the file: ```rust,ignore +use axum::body::Body; +use axum::http::{Request, StatusCode}; +use axum::response::Response; +use axum::Router; use http_body_util::BodyExt; use tower::ServiceExt; @@ -130,57 +240,134 @@ async fn send(app: &Router, req: Request) -> Response { app.clone().oneshot(req).await.unwrap() } +/// Builds a POST with a JSON body, optionally carrying a bearer token. fn post(path: &str, body: serde_json::Value, auth: bool) -> Request { let mut b = Request::post(path).header("content-type", "application/json"); if auth { - b = b.header("authorization", bearer()); // Bearer + b = b.header("authorization", bearer()); // "Bearer " } b.body(Body::from(serde_json::to_vec(&body).unwrap())).unwrap() } +/// Buffers the response body and decodes it as JSON into `T`. async fn body_json(res: Response) -> T { let bytes = res.into_body().collect().await.unwrap().to_bytes(); serde_json::from_slice(&bytes).unwrap() } ``` -A test then boots the app, opens a wallet, and asserts the projected read came -back through CQRS — all against the one app context: +What just happened: `send` is the whole mechanism — `app.clone().oneshot(req)` +runs the request through the real router in-process. `post` assembles a JSON +request and, when `auth` is true, attaches an `Authorization: Bearer …` header +minted by `bearer()` (which calls Lumen's `mint_token("u-alice", +&[CUSTOMER_ROLE])` from the security module). `body_json` drains the response body +with `BodyExt::collect` and deserializes it. Three helpers, and every test below +reads like a script. + +### Step 2 — Drive a round-trip through CQRS + +With the helpers in place, a test boots the app, opens a wallet through the +public API, and asserts the projected read comes back through CQRS — all against +the one app context: ```rust,ignore #[tokio::test] async fn open_then_get_round_trips_through_cqrs() { let app = build_router().await; // one app context per test let opened = open_wallet(&app, "alice", 1_000).await; // POST /api/v1/wallets, asserts 201 + assert_eq!(opened.owner, "alice"); assert_eq!(opened.balance, 1_000); // GET dispatches the #[query_handler] on the handler bean; it reads the // projection (or repairs from the event stream) — both resolved from the - // same container as the command that opened the wallet. + // SAME container as the command that opened the wallet. let fetched = get_wallet(&app, &opened.id).await; + assert_eq!(fetched.id, opened.id); assert_eq!(fetched.balance, 1_000); } ``` -The same file proves the saga (`transfer_saga_happy_path_moves_funds_between_wallets`), -the compensation path (`transfer_saga_overdraft_compensates_and_is_422`), and the -problem-rendering for the three failure modes — a missing token is a 401, an -empty owner is a 422, an unknown id is a 404 — each asserting the -`application/problem+json` content type. That single suite is the proof that the -whole stack composes. +What just happened, and why it matters: the `POST` ran a command through the bus, +which appended events to the in-memory ledger; the `GET` ran a query that read +the projection those events fed. Both resolved the *same* `Ledger` and +`ReadModel` from the one container the test booted, so the read sees the write. +This single test proves the command side, the query side, the projection, and +their shared wiring all fit together — something no unit test can show, because +the seam being tested *is* the wiring. + +### Step 3 — Prove the failure modes render as problems + +The same file proves the saga happy path +(`transfer_saga_happy_path_moves_funds_between_wallets`), the compensation path +(`transfer_saga_overdraft_compensates_and_is_422`), and the problem-rendering for +the failure modes. A missing token is a 401, an empty owner is a 422, and an +unknown id is a 404 — each asserting the `application/problem+json` content type: + +```rust,ignore +#[tokio::test] +async fn missing_token_is_401_problem_on_mutations() { + let app = build_router().await; + let res = send( + &app, + post( + "/api/v1/wallets", + serde_json::json!({ "owner": "mallory", "openingBalance": 10 }), + false, // no Authorization header + ), + ) + .await; + assert_eq!(res.status(), StatusCode::UNAUTHORIZED); + assert!(content_type(&res).contains("application/problem+json")); +} +``` + +What just happened: the unauthenticated `POST` was rejected by the +auto-discovered security layer with a 401, *and* the body came back as an RFC +9457 `application/problem+json` document — not a blank 401. The same shape holds +for the 422 (validation) and 404 (unknown wallet) tests. That single suite is the +proof that the whole stack — routing, security, CQRS, event sourcing, sagas, and +problem rendering — composes correctly. + +> **Note** **Key term — RFC 9457 problem response.** RFC 9457 (which obsoletes +> the older RFC 7807) defines `application/problem+json`: a structured error body +> with a `type`, `title`, `status`, and `detail`. Firefly renders every handler +> error and every unmatched route as one automatically, which is why the tests +> can assert on the content type. You met this in [Your First HTTP +> API](./06-first-http-api.md). + +> **Tip** **Checkpoint.** These twelve scenarios — open → get → deposit/withdraw +> → transfer (happy + compensated) → compliance workflow → two-phase transfer → +> 401/422/404 problems — are Lumen's **12 HTTP tests**. Run `cargo test +> -p firefly-sample-lumen --test '*' 2>/dev/null || cargo test +> -p firefly-sample-lumen` and watch the `http_test` module pass. + +## Tier 2, the terse way — the `firefly-testkit` + +Lumen's own HTTP tests use the raw `tower::oneshot` form on purpose, to show the +mechanism with no magic. In *your* service you would reach for `firefly-testkit`, +which packages exactly that boilerplate into reusable helpers. It is a separate +crate with feature-gated tiers, so you only pull in what you use: + +```toml +# Cargo.toml — add as a dev-dependency, switching on the helpers you need. +[dev-dependencies] +firefly-testkit = { version = "26.6.28", features = ["web", "container"] } +``` -## The testkit: `TestClient`, `Slice`, event assertions +> **Note** The default surface (the webhook signers, `SpyBroker`, and the JSON +> helpers) carries no heavy dependencies. The `web` feature adds the in-process +> `TestClient`; `container` adds the DI `Slice`; and `testcontainers` adds the +> integration-test fixtures. A service that only signs webhooks gets a lean +> build. -`firefly-testkit` packages the boilerplate above into reusable helpers. Lumen's -own tests use the raw `tower::oneshot` form to show the mechanism with no magic, -but in your service the testkit makes the same tests far shorter. Three pieces -matter most. +Three pieces matter most. ### TestClient — an in-process HTTP client (feature `web`) `TestClient::new(router)` wraps any axum `Router` and gives you `get` / `post` / -`put` / `patch` / `delete` plus a fluent assertion API on the `TestResponse`. The -`open_then_get` test above, rewritten with `TestClient`: +`put` / `patch` / `delete` (async) plus a fluent assertion API on the +`TestResponse` it returns. The `open_then_get` test above, rewritten with +`TestClient`: ```rust,ignore use firefly_testkit::TestClient; @@ -191,8 +378,8 @@ async fn open_then_get_with_testclient() { let created = client .post("/api/v1/wallets", &serde_json::json!({ "owner": "alice", "openingBalance": 1000 })) - .await - .assert_status(201); + .await; + created.assert_status(201); let id = created.json_path("$.id").unwrap(); client @@ -203,27 +390,46 @@ async fn open_then_get_with_testclient() { } ``` -`assert_status`, `assert_success`, `assert_header`, `assert_body_contains`, -`assert_json_eq`, `assert_json_path` / `assert_json_path_exists` / -`assert_json_path_absent`, and `json::()` / `json_path("$.field")` cover the -common assertions; the path grammar is the single-result JSONPath subset (a -leading `$`, dotted or bracketed member access, array indexing). Each assertion -returns `&Self` so they chain. (Blocking variants — `post_blocking`, -`get_blocking`, … — exist for non-async test contexts.) - -### Slice — a focused DI container for a test - -`Slice` builds a minimal `firefly-container` for a slice test: register only the -collaborators the unit under test needs, then resolve them. It gives you a -focused dependency-injection container — the wiring for the unit under test -without standing up the full application context: +What just happened: `TestClient` did the request-building and body-buffering for +you. `post(path, &body)` serializes the JSON and sets `content-type`; +`assert_status` checks the code; `json_path("$.id")` selects a single field; and +`assert_json_path("$.balance", 1000)` asserts one value deep in the body without +spelling out the whole document. Each assertion returns `&Self`, so they chain. + +The assertion surface is: `assert_status`, `assert_success`, `assert_header` / +`assert_header_present`, `assert_body_contains`, `assert_json_eq`, +`assert_json_path` / `assert_json_path_exists` / `assert_json_path_absent`, plus +the extractors `json::()`, `json_path("$.field")`, `text()`, `header(name)`, +and `body_bytes()`. The path grammar is a single-result JSONPath subset: a +leading `$`, dotted (`$.user.name`) or bracketed (`$['user']['name']`) member +access, and array indexing (`$[0]`, `$.items[2].id`) — no wildcards, filters, or +recursive descent. + +> **Note** Every verb also has a blocking variant — `get_blocking`, +> `post_blocking`, … — that drives the request on an internal current-thread +> runtime, so a plain `#[test]` (no `#[tokio::test]`) reads exactly like a +> synchronous HTTP client. Use the blocking form outside a Tokio runtime and the +> async form inside one. + +### Slice — a focused DI container for one test (feature `container`) + +The HTTP tests boot the *whole* application. Sometimes you want the opposite: the +wiring for a single unit and nothing else — no router, no datasource. `Slice` +builds a minimal `firefly-container` for exactly that. You register only the +collaborators the unit under test needs, then resolve them. + +> **Note** **Key term — slice test.** A *slice* loads a focused subset of the +> object graph instead of the whole application context. It is faster than a full +> boot and isolates the unit under test. Spring's slice annotations +> (`@WebMvcTest`, `@DataJpaTest`) are the direct analog; `Slice` is the explicit +> builder Rust needs in their place, since there is no package scanning. ```rust,ignore use firefly_testkit::Slice; use firefly_container::{Container, ContainerError, Scope}; let slice = Slice::new() - .instance(ReadModel::default()) // a ready instance (a "mock_bean") + .instance(ReadModel::default()) // a ready instance (the mock/override path) .register::(Scope::Singleton, |c: &Container| { Ok(MyService::new()) // a factory; resolve deps from `c` }) @@ -232,25 +438,40 @@ let slice = Slice::new() let read_model: std::sync::Arc = slice.get(); ``` -`register` / `register_named` take a factory `|c: &Container| -> Result` (it may resolve its own dependencies from `c`); `instance` / -`instance_named` install a ready value (the override/mock path); `bind` coerces a -concrete type to a trait object; and `eager` forces construction at build time. -`build()` returns a `BuiltSlice` you resolve from with `get::()` / -`get_named::(name)`. +What just happened: `instance(value)` installs a ready singleton; `register::(scope, factory)` registers a bean built by a factory that can resolve its own +dependencies from the container `c`; and `build()` returns a `BuiltSlice` you +resolve from with `get::()` (or `get_named::(name)`). There is also +`eager::()`, which forces a bean's construction at `build()` time so a missing +collaborator fails *there* (the fail-fast gate that mirrors Spring's slice +startup) rather than lazily on first use. + +The `instance` + `bind` pair **is** the `@MockBean`. Install a fake under a port +and the bean under test wires it instead of the real collaborator: + +```rust,ignore +let slice = Slice::new() + .instance(FakeRepo::default()) // the fake (a "mock_bean") + .bind::(|a| a) // expose it as the `dyn Repo` port + .register::(Scope::Singleton, |c| { + Ok(Service { repo: c.resolve::()? }) // wires the fake + }) + .eager::() // fail fast if `Repo` is missing + .build(); +``` -The `instance` + `bind` pair **is** the `@MockBean`: install a fake under a port -and the bean under test wires it instead of the real collaborator. Because the -fake is held by the container, `get::()` after `build()` hands back the -*same* instance, so you configure and assert against it via interior mutability. +Because the fake is held by the container, `get::()` after `build()` +hands back the *same* instance the service wired in. So you configure and assert +against it through interior mutability — the mock-verification move from Spring, +without a mocking framework. -### `@WebMvcTest` — a controller over mocked services with `web_client` +### `@WebMvcTest` — one controller over mocked services with `web_client` -A `Slice` registers the controller bean plus its **mocked** collaborators; -`built.web_client::(C::routes)` then resolves that controller and wraps its -`#[rest_controller]`-generated router in a [`TestClient`](#testclient--an-in-process-http-client-feature-web) — -Spring's `@WebMvcTest(Controller.class)` + `@MockBean(Service.class)`, with no -full-application boot and no datasource: +Combine the two: register a controller bean plus its **mocked** collaborators, +then call `built.web_client::(C::routes)` to resolve that controller and +wrap its `#[rest_controller]`-generated router in a `TestClient`. This is Spring's +`@WebMvcTest(Controller.class)` + `@MockBean(Service.class)` — one controller's +web layer exercised over fakes, with no full-application boot and no datasource: ```rust,ignore use firefly_testkit::Slice; @@ -270,22 +491,37 @@ let client = Slice::new() client.get_blocking("/api/v1/wallets/unknown").assert_status(404); ``` -`web_client` (feature `web`) takes the controller's generated `fn routes(state: -C) -> Router`; it clones the resolved bean into the router state, so the whole -web layer of one controller is exercised over fakes. For a **`@DataJpaTest`**, -the same `Slice` registers a repository over an in-memory SQLite `Db` (build it -with `firefly::data_sqlx::repository_for`, as `lumen-ledger`'s `-models` tests -do) — a focused persistence slice with no web stack. - -### Asserting emitted events - -`SpyBroker` records what a handler published; `assert_event_published(&spy, -"Type")` asserts an event of that type was recorded and returns it (the -`_with` variant also checks the payload, parsed as a JSON object, contains the -given key/value pairs (a subset match); `assert_no_events_published` asserts -none). `must_encode` / `must_decode` are -panic-on-failure JSON helpers. A Lumen-flavored example — proving an open emits a -`WalletOpened`: +What just happened: `web_client` (feature `web`) takes the controller's generated +`fn routes(state: C) -> Router`, clones the resolved bean into the router's state, +and wraps the result in a `TestClient`. The whole web layer of one controller is +now driven over fakes. (`FakeWalletService` / `WalletController` here are +illustrative shapes for *your* service — Lumen's own controller autowires the +real bus, so its web coverage comes from the Tier 2 HTTP tests above.) + +> **Note** For a **`@DataJpaTest`** — a persistence slice with no web stack — the +> same `Slice` registers a repository over an in-memory SQLite database. Build the +> repository with `firefly::data_sqlx::repository_for::(db)`, exactly as +> `lumen-ledger`'s `-models` tests do: they point a `Db` at an in-memory SQLite +> URL (`sqlite:file:…?mode=memory&cache=shared`) and exercise the real derived +> queries with no Postgres in sight. You met those repositories in [Persistence & +> Reactive Repositories](./07-persistence.md). + +### Asserting emitted events with `SpyBroker` + +The third everyday helper proves a handler *published* the right event. +`SpyBroker` records what a handler published, and the assertion helpers read it +back: + +- `assert_event_published(&spy, "Type")` asserts an event of that type was + recorded and returns it. +- `assert_event_published_with(&spy, "Type", &json)` also checks the payload + (parsed as a JSON object) contains the given key/value pairs — a *subset* match, + so extra fields are ignored. +- `assert_no_events_published(&spy)` asserts none were recorded. +- `must_encode` / `must_decode` are panic-on-failure JSON helpers for building + and reading payloads. + +A Lumen-flavored example — proving an open emits a `WalletOpened`: ```rust,ignore use firefly_testkit::{assert_event_published, must_encode, SpyBroker}; @@ -293,78 +529,129 @@ use firefly_testkit::{assert_event_published, must_encode, SpyBroker}; #[test] fn open_emits_wallet_opened() { let spy = SpyBroker::new(); - // The ledger would publish through the broker; here we record the envelope - // the projection would consume. - spy.record("wallets.events", "WalletOpened", - &must_encode(&serde_json::json!({ "id": "wlt_1", "owner": "alice" }))); + // The ledger publishes through the broker; here we record the envelope the + // projection would consume. + spy.record( + "wallets.events", + "WalletOpened", + &must_encode(&serde_json::json!({ "id": "wlt_1", "owner": "alice" })), + ); let event = assert_event_published(&spy, "WalletOpened"); assert_eq!(event.topic, "wallets.events"); } ``` +What just happened: `spy.record(topic, type, payload)` stores an event envelope, +and `assert_event_published` finds the first one of the named type (or fails the +test, listing what *was* published). The returned `RecordedEvent` carries +`topic`, `event_type`, and the raw `payload` bytes, so you can assert further. +Wire a `SpyBroker` into a `Ledger` in a real test and you can prove a deposit +emits a `MoneyDeposited` with the right amount. + ### Webhook signers -When Lumen grows an inbound webhook (Chapter 16), the testkit's HMAC signers — -`sign_hmac`, `sign_stripe`, `sign_github`, `sign_twilio` — produce header values -byte-identical to what each `firefly-webhooks` validator expects, so a signed -test request validates exactly as a real provider's would: +When Lumen grows an inbound webhook (the [Scheduling & +Notifications](./16-scheduling-notifications.md) chapter), the testkit's HMAC +signers — `sign_hmac`, `sign_stripe`, `sign_github`, `sign_twilio` — produce +header values byte-identical to what each `firefly-webhooks` validator expects, so +a signed test request validates exactly as a real provider's would: ```rust,ignore use firefly_testkit::sign_stripe; let sig = sign_stripe(b"whsec_test", br#"{"type":"charge.succeeded"}"#, 1_700_000_000); -// Attach `sig` as the `Stripe-Signature` header on a TestClient POST. +// Attach `sig` as the `Stripe-Signature` header on a TestClient POST and the +// validator accepts it exactly as it would a real Stripe delivery. ``` +What just happened: `sign_stripe(secret, body, unix_ts)` builds the +`t=,v1=` value Stripe sends in `Stripe-Signature`, signing +`.` with HMAC-SHA256. Because the signer matches the validator's wire +shape exactly, a test that signs its own payload proves your receiver accepts a +genuine delivery. + ## Testing reactive pipelines -The streaming endpoint (Chapter 20) builds a `Flux`. A reactive pipeline is -tested by driving it to a terminal — `block()`, `collect_list()`, `count()` — and -asserting the resolved value. This is `firefly-reactive`'s way to verify a stream -end to end; it will feel familiar if you've worked with a reactive-streams -library, but it is plain async Rust assertions over a resolved `Flux`: +The streaming endpoint (introduced in [Production & +Deployment](./20-production.md)) builds a `Flux`. You met `Mono` and `Flux` in +[The Reactive Model](./05-reactive-model.md); here is how you *test* one. + +> **Note** **Key term — terminal operation.** A reactive pipeline is lazy: the +> operators (`filter`, `map`, …) describe work but run nothing until a *terminal* +> consumes the stream. `collect_list()`, `count()`, and `block()` are terminals — +> they drive the pipeline to completion and resolve a value. Spring Reactor's +> `block()` / `collectList()` are the direct analog. + +You test a pipeline by driving it to a terminal and asserting the resolved value: ```rust use firefly_reactive::Flux; #[tokio::test] async fn pipeline_filters_and_maps() { - let out = Flux::range(1, 5) - .filter(|x| x % 2 == 1) - .map(|x| x * 10) - .collect_list() - .block() + let out = Flux::range(1, 5) // emits 1, 2, 3, 4, 5 (start, count) + .filter(|x| x % 2 == 1) // keep the odds: 1, 3, 5 + .map(|x| x * 10) // scale: 10, 30, 50 + .collect_list() // Flux -> Mono> + .block() // Result>, FireflyError> .await - .unwrap() - .unwrap(); + .unwrap() // unwrap the Result + .unwrap(); // unwrap the Option (the stream was non-empty) assert_eq!(out, vec![10, 30, 50]); } ``` -Lumen's streaming tests (`src/streaming_test.rs`, gated behind the `streaming` -feature) take the HTTP route instead: open a wallet, deposit, then `GET -/events` and assert two NDJSON lines (`WalletOpened` + `MoneyDeposited`) by -default, `text/event-stream` with `?format=sse`, and a 404 for an unknown wallet. - -## Real-infrastructure integration tests - -Lumen runs hermetically, but the production adapters need real services. The -workspace ships a `docker-compose.yml` with Postgres, Redis, RabbitMQ, a -Kafka-compatible Redpanda, Keycloak, S3/Blob emulators, and an SMTP capture. The -convention throughout the adapter crates is: a test reads a connection URL from -the environment and **skips when it is unset**, so the default `cargo test` stays -green on a bare machine while CI flips the full suite on: +What just happened, and why the double `unwrap`: `Flux::range(1, 5)` emits five +values starting at `1`. `filter` and `map` transform them lazily. `collect_list()` +turns the `Flux` into a `Mono>` — a single value holding the whole +list — and `block().await` drives it to completion. `block()` returns +`Result>, FireflyError>`: the `Result` surfaces a pipeline error, +and the `Option` is `None` only for an empty stream, so a successful non-empty run +needs both `unwrap`s. This is plain async Rust assertions over a resolved stream — +no special test runtime. + +> **Note** Lumen's streaming tests (`src/streaming_test.rs`, gated behind the +> `streaming` feature) take the HTTP route instead of testing the `Flux` directly: +> they open a wallet, deposit, then `GET /events` and assert two NDJSON lines +> (`WalletOpened` + `MoneyDeposited`) by default, `text/event-stream` with +> `?format=sse`, and a 404 for an unknown wallet. Those are the `+3 streaming +> tests` you turn on with `--features streaming`. + +## Tier 3 — Real-infrastructure integration tests + +Lumen runs hermetically, but the production adapters you reach for in [Production +& Deployment](./20-production.md) need real services. The workspace ships a +`docker-compose.yml` with Postgres, Redis, RabbitMQ, a Kafka-compatible Redpanda, +Keycloak, S3/Blob emulators, and an SMTP capture. + +The convention throughout the adapter crates keeps the default `cargo test` green +on a bare machine: a test reads a connection URL from the environment and +**skips when it is unset**. CI flips the full suite on by exporting the variable. + +> **Note** **Key term — env-gated test.** An *env-gated* test only runs when a +> named environment variable is present (a `DATABASE_URL`, a `REDIS_URL`). +> Marking it `#[ignore]` keeps it out of the default run; reading the variable and +> returning early means even `--ignored` skips cleanly where the service is +> absent. This is the Rust analog of Spring's `@Testcontainers` / +> `@EnabledIf`-guarded tests. ```rust,ignore #[tokio::test] #[ignore = "requires postgres (DATABASE_URL)"] async fn postgres_event_store_round_trips() { - let Ok(url) = std::env::var("DATABASE_URL") else { return }; // skip on a bare machine - // ... drive the Postgres-backed EventStore against the live database + // Skip on a bare machine: no DATABASE_URL -> return before touching the DB. + let Ok(url) = std::env::var("DATABASE_URL") else { return }; + // ... drive the Postgres-backed EventStore against the live database at `url`. } ``` +What just happened: the `#[ignore]` keeps this test out of `cargo test`'s default +run entirely. When you opt in with `--ignored`, the `let … else { return }` guard +still skips cleanly if `DATABASE_URL` is unset, so the only way it actually +touches Postgres is when you point it at a live one. To run the env-gated suite, +start the backing services and export the URLs: + ```bash docker compose up -d # start the backing services DATABASE_URL=postgres://firefly:firefly@localhost:5442/firefly \ @@ -373,9 +660,21 @@ REDIS_URL=redis://localhost:6379/0 \ docker compose down ``` +> **Note** The compose file maps Postgres to host port **5442** (not the default +> 5432) to avoid colliding with a local Postgres you may already run — which is +> why the `DATABASE_URL` above says `localhost:5442`. + +The testkit can shorten this tier too. With the `testcontainers` feature, +`firefly_testkit::containers` maps a started service's `(host, port)` to the +canonical `firefly.*` config keys (`config_for(&container)`) and offers a +`docker_available()` skip guard — the Rust analog of Spring's +`@ServiceConnection`. It is decoupled from any specific container library: feed it +the connection details any tool already hands you. + ## Running Lumen's suite -From the workspace root (with `export PATH="/opt/homebrew/bin:$PATH"`): +From the workspace root (with `export PATH="/opt/homebrew/bin:$PATH"` on macOS so +the toolchain resolves): ```bash cargo build -p firefly-sample-lumen @@ -385,26 +684,37 @@ cargo clippy -p firefly-sample-lumen --all-targets -- -D warnings cargo fmt -p firefly-sample-lumen -- --check ``` -If a snippet in any chapter drifts from the file, this gate fails — which is how -the book stays honest. - -## What changed in Lumen - -Nothing in `src/` — this chapter is the retrospective on the test code that grew -alongside every feature: - -- **Unit tests** per module: `money` and `domain` invariants, `commands` - validation + the handler bean over its autowired collaborators, `security` - mint/verify/reject, `transfer` happy + compensation, `housekeeping` registration - + tick. -- The **`src/http_test.rs`** end-to-end suite boots one `build_router()` app - context per test and drives every request against it with `tower::oneshot`, - covering open → get → deposit/withdraw → transfer (happy + compensated) → - projection convergence → 401/422/404 problems. -- **`src/streaming_test.rs`** (feature-gated) exercises the NDJSON / SSE endpoint. -- The **`firefly-testkit`** helpers — `TestClient`, `Slice`, - `assert_event_published`, the HMAC signers — are the terse path to the same - coverage in your own service. +> **Tip** **Checkpoint.** A clean run prints `test result: ok` for the unit and +> HTTP tiers and the doctest, with zero clippy warnings and a clean `fmt --check`. +> If a snippet in any chapter drifts from the file, this gate fails — which is +> precisely how the book stays honest. + +## Recap — how Lumen proves itself + +Nothing changed in `src/` this chapter; it is the retrospective on the test code +that grew alongside every feature. You now know: + +- **The three tiers, and one helper per tier.** Pure `#[tokio::test]` unit tests + with no I/O; in-process HTTP/slice tests that drive the real router without + binding a socket; and env-gated integration tests against live infrastructure. +- **`bootstrap()` is the test seam.** It assembles the same fully-wired app + `run()` would serve and returns `Bootstrapped::api_router` — no socket — so + `build_router()` gives each test one self-consistent container where a write is + visible to a later read. +- **Tier 1 — unit tests.** Construct a value object, aggregate, or handler bean + with its collaborators in hand and assert directly; call `.validate()` on a + command without HTTP. Lumen's **42 unit tests** live here. +- **Tier 2 — in-process HTTP.** `tower::oneshot` drives `build_router()` end to + end over in-memory infrastructure; Lumen's **12 HTTP tests** cover open → get → + deposit/withdraw → transfer (happy + compensated) → workflow → 2PC → + 401/422/404 RFC 9457 problems. `firefly-testkit`'s `TestClient`, `Slice` + (`@MockBean` / `@WebMvcTest` / `@DataJpaTest`), and `SpyBroker` make the same + coverage terse in your own service. +- **Reactive pipelines** are tested by driving a `Flux` to a terminal + (`collect_list().block()`) — the chapter's single **doctest**. +- **Tier 3 — integration.** `#[ignore]`d, env-gated tests read a connection URL, + skip cleanly when it is unset, and run against `docker compose` services (or the + testkit's `containers` fixtures) when it is set. ## Exercises @@ -418,15 +728,28 @@ alongside every feature: mutation.) 2. **A `Slice` test for the read model.** Use `Slice` to register a `ReadModel::default()` instance, project a `WalletOpened` into it by hand, and - assert `find` returns the view — all without the bus or the router. + assert `find` returns the view — all without the bus or the router. Add + `.eager::()` and confirm `build()` succeeds, then resolve it with + `slice.get::()`. 3. **Event assertion on the ledger.** Wire a `SpyBroker` into a `Ledger` in a test, commit a deposit, and use `assert_event_published_with(&spy, - "MoneyDeposited", &serde_json::json!({ "amount": 50 }))` to prove the - payload's `amount` field equals 50. -4. **A skipping integration test.** Write an `#[ignore]`d test that reads - `DATABASE_URL`, returns early when unset, and otherwise opens a wallet against - a Postgres-backed event store. Confirm it skips with a plain `cargo test` and - runs with the variable set. - -With the stack proven at every level, the remaining chapters cover the CLI and -shipping Lumen to production. Continue to [The CLI](./19-cli.md). + "MoneyDeposited", &serde_json::json!({ "amount": 50 }))` to prove the payload's + `amount` field equals 50. Then add `assert_no_events_published` to a no-op path + and watch it pass. +4. **A `@WebMvcTest`-style slice.** Sketch a fake service behind a port, register + it with `.instance(...)` + `.bind::(|a| a)`, register a + controller over it, and call `web_client::(C::routes)` to drive one route + over the fake with `get_blocking`. Assert a 404 for an unknown id. +5. **A skipping integration test.** Write an `#[ignore]`d test that reads + `DATABASE_URL`, returns early when unset, and otherwise opens a wallet against a + Postgres-backed event store. Confirm it skips with a plain `cargo test`, skips + with `--ignored` when the variable is unset, and runs with the variable set. + +## Where to go next + +- Scaffold, inspect, and operate Lumen with the developer tooling in **[The + CLI](./19-cli.md)** — including the `firefly` commands that run these same + checks. +- Swap the in-memory defaults for real Postgres and Kafka, then ship Lumen, in + **[Production & Deployment](./20-production.md)** — where the Tier 3 integration + tests finally have live infrastructure to run against. diff --git a/docs/book/src/19-cli.md b/docs/book/src/19-cli.md index 033005b..ac58e60 100644 --- a/docs/book/src/19-cli.md +++ b/docs/book/src/19-cli.md @@ -1,28 +1,95 @@ # The CLI -So far you have built Lumen by hand — a file at a time, `cargo build` after -each chapter. By the end of this chapter you will know the other way to start a -service like Lumen: the `firefly` developer CLI scaffolds a project, generates -the same artifacts the earlier chapters wrote by hand, runs the binary with -profiles and overrides, manages migrations, exports an OpenAPI document, and -introspects a running Lumen over its actuator surface — all from one binary -built for a compiled Cargo workspace. - -> **One binary, the whole lifecycle.** `firefly` scaffolds a project, generates -> code artifacts, runs the binary with profiles and overrides, stamps -> build-info, manages migrations, exports OpenAPI, and introspects a running -> service — the everyday developer loop in a single command-line tool -> (`new` / `run` / `generate` / `db` / `doctor` / `sbom` / `license`). - -## Installing +So far you have built **Lumen** — the digital-wallet and ledger service from +[Quickstart](./02-quickstart.md) onward — by hand: a file at a time, a +`cargo build` after each chapter. That was deliberate, so every line is +something you typed and understand. This chapter teaches the *other* way to do +the same work: the `firefly` developer CLI. It is a single compiled binary that +scaffolds a project, generates the same artifacts the earlier chapters wrote by +hand, runs the binary with profiles and config overrides, stamps build metadata, +manages migrations, exports an OpenAPI document, and introspects a *running* +Lumen over its actuator surface — the everyday developer loop in one tool. + +Nothing in this chapter changes `samples/lumen` itself; it is purely +operational. But by the end you will be able to drive the whole lifecycle from +the command line, and — just as importantly — you will know exactly which +framework crate each command talks to, because the CLI never invents an API. It +calls the same `firefly-migrations`, `firefly-openapi`, and actuator endpoints +you have already met. + +By the end of this chapter you will: + +- Install the `firefly` binary and read its command catalogue. +- Scaffold a new service two ways — picking an *archetype* and turning on + *features* — and preview the exact file plan with `--dry-run`. +- Generate individual code artifacts (a CQRS command, a query, an aggregate, a + saga, a migration) into an existing project, and read what the generators + actually emit. +- Run a Firefly app through the CLI, mapping profile and override flags to the + `FIREFLY_*` environment variables the framework reads at startup. +- Introspect a *running* Lumen — its health, routes, beans, and metrics — over + the actuator port, and understand why a compiled binary requires `--url`. + +## Concepts you will meet + +Before the first command, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — archetype.** An *archetype* is a project template that +> decides the starting shape of your crate: which modules exist, which Firefly +> features are switched on, and what the example code looks like. The CLI ships +> six (`core`, `web-api`, `web`, `hexagonal`, `library`, `cli`). The Spring +> analog is a Spring Initializr "project type" plus its preselected +> dependencies. + +> **Note** **Key term — feature.** A *feature* is an opt-in subsystem the +> scaffold wires in — `web`, `data`, `cqrs`, `eda`, `cache`, `security`, and so +> on. Each maps to one or more `firefly-*` crates added to the generated +> `Cargo.toml`. In Spring terms, choosing a feature is like ticking a starter on +> the Initializr. + +> **Note** **Key term — actuator surface.** The *actuator surface* is the set of +> operational HTTP endpoints — `/actuator/health`, `/actuator/info`, +> `/actuator/metrics`, `/actuator/mappings`, `/actuator/beans`, +> `/actuator/conditions`, `/actuator/env` — that a running Firefly app serves on +> its **management** port (`8081` by default), separate from the public API on +> `8080`. This mirrors Spring Boot Actuator. The CLI's introspection commands +> are thin clients over these endpoints. + +> **Note** **Key term — Command/Query Responsibility Segregation (CQRS).** A +> pattern that routes state-changing **commands** and read-only **queries** +> through separate handlers on a shared *bus*. You built Lumen's command and +> query handlers in [CQRS](./09-cqrs.md); the CLI can scaffold the same pieces +> for a new project with `firefly generate command` / `firefly generate query`. + +## Step 1 — Install the binary + +The CLI lives in the framework's `crates/cli`. Install it from a checkout, then +ask it to describe itself. ```bash cargo install --path crates/cli # installs the `firefly` binary firefly --help # prints the banner + every command -firefly --version # 26.6.24 +firefly --version # 26.6.28 ``` -## Command overview +What just happened: `cargo install` compiled the `firefly` binary and put it on +your `PATH`. `--version` prints the framework calendar version — the same +`26.6.28` Lumen depends on, because the CLI is versioned with the rest of the +workspace. + +> **Tip** **Checkpoint.** `firefly --version` prints `26.6.28` and +> `firefly --help` lists subcommands including `new`, `generate`, `run`, `db`, +> `openapi`, `doctor`, and `health`. If `firefly` is "command not found", make +> sure `~/.cargo/bin` is on your `PATH`. + +If you would rather not install the binary, you can drive the CLI through Cargo +from a framework checkout — see [Step 9](#step-9--run-the-cli-through-cargo). + +## Step 2 — Read the command catalogue + +The whole developer loop fits in one table. Skim it now; the rest of the chapter +walks the commands you will use most. | Command | Purpose | |------------------------------------------------------|-----------------------------------------------| @@ -34,6 +101,7 @@ firefly --version # 26.6.24 | `firefly doctor` | toolchain checks (rustc, cargo, git, …) | | `firefly db ` | migration management | | `firefly openapi --format json\|yaml [-o file]` | export an OpenAPI 3.1 document | +| `firefly openapi-client --spec ` | generate a typed Rust client from a spec | | `firefly actuator --url ` | query a running app's `/actuator/*` | | `firefly routes\|env\|health\|metrics --url ` | remote introspection of a running app | | `firefly beans\|conditions --url ` | DI / auto-config report of a running app | @@ -41,76 +109,182 @@ firefly --version # 26.6.24 | `firefly sbom [--json]` | software bill of materials from `Cargo.lock` | | `firefly license` | framework + dependency license report | -## Scaffolding a project +What just happened: that is the full surface. Notice the shape of the loop — +*scaffold* (`new`), *grow* (`generate`), *run* (`run`), *package* (`build`), +*operate* (`db`, `openapi`), *introspect* (`actuator`, `routes`, `health`, …), +and *audit* (`doctor`, `sbom`, `license`). Every command maps to a framework +crate or an actuator endpoint you have already met. -`firefly new` generates a workspace-less Cargo crate with a `src/` tree, a -`firefly.yaml`, a `.gitignore`, a `README.md`, a `Dockerfile`, and `tests/` — -roughly the shape Lumen started from in chapter 2: +## Step 3 — Scaffold a project + +`firefly new` generates a workspace-less Cargo crate: a `src/` tree shaped by the +archetype, a `firefly.yaml`, a `.gitignore`, a `README.md`, a `Dockerfile`, and +a `tests/` directory. It is the same starting shape Lumen had after +[Quickstart](./02-quickstart.md). ```bash -firefly new lumen --archetype web-api --features web,data,cqrs --git -firefly new my-lib --archetype library --dep-path ../../crates # local dev deps -firefly new --list # archetypes + features -firefly new svc --dry-run # plan without writing +firefly new lumen2 --archetype web-api --features web,data,cqrs --git +firefly new my-lib --archetype library --dep-path ../../ # local dev deps +firefly new --list # archetypes + features +firefly new svc --dry-run # plan without writing ``` -Archetypes: `core`, `web-api`, `web`, `hexagonal`, `library`, `cli`. The -generated `firefly-*` dependencies are git / path / version configurable -(`--dep-path` / `--dep-version`, defaulting to the canonical GitHub repo). -`--git` initializes a repository with an initial commit; `--force` overwrites an -existing target directory. - -> **Archetypes scaffold a working service.** `firefly new --archetype web-api` -> stamps the entry point, a controller, and the dependency set so the first -> `cargo run` boots. `--features` selects the opt-in adapters the -> [facade chapter](./21-declarative-macros.md) lists -> (`web`, `data`, `cqrs`, `eda`, `cache`, `security`, …). - -## Generating artifacts - -`firefly generate` writes a code artifact into the current project, detecting the -package, archetype, and feature flags from `Cargo.toml` + `firefly.yaml`. These -are the same pieces you wrote by hand for Lumen — a command + handler, an -aggregate, a saga: +What just happened, command by command: + +- The first line scaffolds a `web-api` project named `lumen2` with the `web`, + `data`, and `cqrs` features switched on, and (because of `--git`) initializes a + Git repository with an initial commit. +- `--dep-path ../../` points the generated `firefly-*` dependencies at a local + workspace checkout instead of the canonical GitHub repo. Each crate resolves + into its own `crates/` automatically. +- `--list` prints the archetype and feature catalogues, then exits without + creating anything. +- `--dry-run` prints the exact file plan — every path that *would* be written — + without touching the filesystem. + +The six archetypes are `core`, `web-api`, `web`, `hexagonal`, `library`, and +`cli`. The `web-api` archetype stamps an entry point, a controller, and the +layered `models/services/repositories` tree wired against the real web starter, +so the very first `cargo run` boots. The generated `firefly-*` dependency source +is configurable: `--dep-path ` for a local checkout, `--dep-version ` +for a published crates.io release, otherwise the canonical Git repo. `--force` +overwrites an existing target directory. + +> **Note** A feature you do not select is simply absent from the generated +> `Cargo.toml` — picking `web,data,cqrs` adds `firefly-web`, `firefly-data` + +> `firefly-migrations`, and `firefly-cqrs`, and nothing else. The full feature +> list (`web`, `data`, `mongodb`, `eda`, `cache`, `client`, `security`, +> `scheduling`, `observability`, `cqrs`, `shell`, `transactional`) is printed by +> `firefly new --list`, and the underlying crates are the ones the +> [macros chapter](./21-declarative-macros.md) catalogues. + +> **Tip** **Checkpoint.** Run `firefly new lumen2 --archetype web-api --dry-run`. +> You should see a plan listing `Cargo.toml`, `firefly.yaml`, `.gitignore`, +> `README.md`, `Dockerfile`, `src/main.rs`, `src/lib.rs`, `src/controllers.rs`, +> the `models/services/repositories` tree, and `tests/api.rs` — with nothing +> written to disk. Drop `--dry-run` and the same files appear under `lumen2/`. + +## Step 4 — Generate individual artifacts + +Once a project exists, `firefly generate` (alias `g`) writes one artifact at a +time into it, detecting the package, archetype, and feature flags from +`Cargo.toml` + `firefly.yaml`. These are exactly the pieces you wrote by hand for +Lumen — a command and its handler, a query, an aggregate, a saga, a migration. ```bash -firefly generate command OpenWallet # command + handler in src/cqrs/ -firefly generate query GetWallet # query + handler -firefly generate aggregate Wallet # a struct embedding firefly_eventsourcing::AggregateRoot +firefly generate command OpenWallet # src/cqrs/open_wallet_command{,_handler}.rs +firefly generate query GetWallet # src/cqrs/get_wallet_query{,_handler}.rs +firefly generate aggregate Wallet # src/domain/wallet.rs (embeds AggregateRoot) firefly generate saga MoneyTransfer --dry-run -firefly generate migration AddWallets # V###__add_wallets.sql in migrations/ -firefly g handler Deposit # `g` is the alias +firefly generate migration AddWallets # migrations/V###__add_wallets.sql +firefly g handler Deposit # `g` is the alias +``` + +The artifact kinds are `handler`, `route`, `entity`, `repository`, `dto`, +`aggregate`, `command`, `query`, `saga`, and `migration`. Names are accepted in +any case and converted as needed (`OpenWallet`, `open-wallet`, and `open_wallet` +all produce the same files). `--force` overwrites an existing file; `--dry-run` +plans without writing. + +What just happened, with the two CQRS generators as the worked example. A +`generate command OpenWallet` writes **two** files into `src/cqrs/`: + +```rust,ignore +// src/cqrs/open_wallet_command.rs +use firefly_cqrs::{CqrsError, Message}; +use serde::{Deserialize, Serialize}; + +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct OpenWallet { + /// Target aggregate identifier. + pub id: String, +} + +impl Message for OpenWallet { + fn validate(&self) -> Result<(), CqrsError> { + if self.id.trim().is_empty() { + return Err(CqrsError::validation("id is required")); + } + Ok(()) + } +} ``` -Artifact kinds: `handler`, `route`, `entity`, `repository`, `dto`, `aggregate`, -`command`, `query`, `saga`, `migration`. Names are accepted in any case and -converted as needed; `--force` overwrites, `--dry-run` plans without writing. +```rust,ignore +// src/cqrs/open_wallet_command_handler.rs +use firefly_cqrs::{Bus, CqrsError}; -## Running the app +use super::open_wallet_command::OpenWallet; -`firefly run` is a thin wrapper over `cargo run` that maps profile and -configuration flags to the `FIREFLY_*` environment variables the framework reads -at startup, then exec's Cargo: +/// Register the `OpenWallet` command handler on `bus`. Call once at startup. +pub fn register_open_wallet_handler(bus: &Bus) { + bus.register(|command: OpenWallet| async move { + // Implement the OpenWallet command behaviour here. + Ok::<_, CqrsError>(command.id) + }); +} +``` + +The command is a plain message struct implementing `firefly_cqrs::Message` +(its `validate` runs in the bus's validation middleware before the handler). +The handler is a `register__handler(bus: &Bus)` *registrar function* that +calls the closure-based `bus.register(...)` — the same registration shape you +used in [CQRS](./09-cqrs.md). `generate query GetWallet` mirrors this with a +`GetWallet` query struct and a `register_get_wallet_handler(bus: &Bus)`. + +> **Note** The generators target the real `firefly-*` APIs, not placeholder +> bodies. `generate aggregate Wallet` writes `src/domain/wallet.rs` with a struct +> that embeds `firefly_eventsourcing::AggregateRoot` (the uncommitted-events +> buffer), exposing `raise(...)` and `take_events(...)`. `generate saga +> MoneyTransfer` writes `src/sagas/money_transfer_saga.rs` with a +> `build_money_transfer_saga()` function over the `firefly_orchestration::Saga` +> builder — `Saga::new("money-transfer")`, `Step::new(...)`, +> `.with_compensation(...)`. These are the same constructs you met in +> [Event Sourcing](./11-event-sourcing.md) and [Sagas](./12-sagas.md). + +> **Tip** **Checkpoint.** Inside a scaffolded project, run +> `firefly generate command OpenWallet --dry-run`. You should see a plan naming +> `src/cqrs/open_wallet_command.rs` and `src/cqrs/open_wallet_command_handler.rs` +> as `create` actions, with nothing written. + +## Step 5 — Run the app + +`firefly run` is a thin wrapper over `cargo run`. It maps profile and config +override flags to the `FIREFLY_*` environment variables the framework reads at +startup, then exec's Cargo from the detected project root. + +> **Note** **Key term — config-override flag.** A `-D key=value` flag overrides +> one configuration value. The CLI maps it to an environment variable by +> stripping a leading `firefly.`, upper-casing, and replacing `.`/`-` with `_`, +> then prepending `FIREFLY_`. So `-D logging.level-root=DEBUG` becomes +> `FIREFLY_LOGGING_LEVEL_ROOT=DEBUG`. This is the same env convention +> [Configuration](./03-configuration.md) describes. ```bash firefly run # cargo run firefly run -p dev -p test # FIREFLY_PROFILES_ACTIVE=dev,test -firefly run -D server.port=9090 # FIREFLY_SERVER_PORT=9090 +firefly run -D logging.level-root=DEBUG # FIREFLY_LOGGING_LEVEL_ROOT=DEBUG firefly run --env FIREFLY_SERVER_ADDR=0.0.0.0:8080 # a raw env var for the process firefly run --debug # FIREFLY_LOGGING_LEVEL_ROOT=DEBUG firefly run --release --bin lumen # cargo run --release --bin lumen firefly run --dry-run # print the resolved env + cargo command ``` -> **Flags become `FIREFLY_*` env vars.** `firefly run -p dev -D server.port=9090` -> maps to `FIREFLY_PROFILES_ACTIVE=dev` and `FIREFLY_SERVER_PORT=9090`, then -> exec's `cargo run`. The resolution order is CLI flag → env → config file. A -> Firefly service is a single compiled binary, so there is no live-reload or -> worker-process selection — you rebuild and rerun. - -For Lumen specifically, recall that `FireflyApplication` honors -`FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR`, so the equivalent of the -two-port bind is: +What just happened: the flags resolve into an environment that is applied before +`cargo run`. `-p`/`--profile` is repeatable or comma-separated and flattens into +a single `FIREFLY_PROFILES_ACTIVE`; `-D key=value` maps to `FIREFLY_`; +`--env KEY=VALUE` passes a raw variable straight through; `--debug` is shorthand +for `FIREFLY_LOGGING_LEVEL_ROOT=DEBUG`; `--release` and `--bin ` pass +through to Cargo. A Firefly service is a single compiled binary, so there is no +live-reload or worker-process selection — you rebuild and rerun. `--dry-run` +prints the resolved environment and the exact `cargo run` command without +executing, which is the safest way to learn the mapping. + +> **Warning** A `-D` override only takes effect if the framework actually reads +> that key. Lumen binds its two ports from `FIREFLY_SERVER_ADDR` / +> `FIREFLY_MANAGEMENT_ADDR` (a full `host:port`), not from a `server.port` key — +> so to move Lumen's ports, set the address env vars directly. The equivalent of +> the two-port bind is: ```bash firefly run --bin lumen \ @@ -118,10 +292,18 @@ firefly run --bin lumen \ --env FIREFLY_MANAGEMENT_ADDR=127.0.0.1:8081 ``` -## Building for release +This is the same seam [Quickstart](./02-quickstart.md) used with raw +`FIREFLY_*` variables — `firefly run --env` just sets them for you. -Plain compilation is `cargo build`; the `build` group adds the two artifacts a -release pipeline needs: +> **Tip** **Checkpoint.** Run `firefly run -p dev -D logging.level-root=DEBUG +> --dry-run` from inside a project. The output prints `Would run: cargo run` and +> an environment block listing `FIREFLY_PROFILES_ACTIVE=dev` and +> `FIREFLY_LOGGING_LEVEL_ROOT=DEBUG`. Nothing is launched. + +## Step 6 — Build for release + +Plain compilation is `cargo build`. The `build` group adds the two artifacts a +release pipeline needs on top of the compiled binary. ```bash firefly build info # write build-info.json (git SHA + UTC time) @@ -130,16 +312,27 @@ firefly build image -t lumen:1.0.0 # OCI image via Cloud Native Buildpacks firefly build image --builder docker # or a plain Dockerfile build ``` -`build info` writes the `build-info.json` that `/actuator/info` surfaces — the -admin-port `info` endpoint chapter 15 wired for Lumen reads it when present, so -the git SHA and build time show up next to the `InfoContributor` block. +What just happened: `build info` writes a `build-info.json` of the shape +`{"git": {"sha": …}, "build": {"time": …}}` (an empty SHA when git is +unavailable). That file is the data source the `/actuator/info` build +contributor reads when present, so the git SHA and build time appear next to the +`InfoContributor` block you wired in [Observability](./15-observability.md). +`build image` builds an OCI image — by default via Cloud Native Buildpacks (the +`pack` tool), or with `--builder docker` against the scaffolded `Dockerfile`. -## Database migrations +> **Tip** **Checkpoint.** Run `firefly build info -o /tmp/build-info.json` and +> open the file. It is valid JSON with a top-level `git` and `build` object, and +> `build.time` is an RFC 3339 UTC timestamp ending in `Z`. -Lumen runs over an in-memory event store, so it ships no SQL migrations — but -the moment you swap in the Postgres event store from chapter 20, `firefly db` -manages the schema. It wraps the -[`firefly-migrations`](./07-persistence.md) forward-only runner: +## Step 7 — Manage database migrations + +Lumen runs over an in-process event store, so it ships **no** SQL migrations — +its `samples/lumen` tree has no `migrations/` directory at all. But the moment +you swap in the Postgres event store from +[Production & Deployment](./20-production.md), `firefly db` manages the schema. +It drives the framework's own forward-only migration runner, the same +[`firefly-migrations`](./07-persistence.md) library the generated projects ship +with. ```bash firefly db init # migrations/ + starter V001__init.sql @@ -148,37 +341,68 @@ firefly db upgrade --url sqlite://app.db # apply pending migrations firefly db status --url sqlite://app.db # show applied + pending ``` -The database URL resolves from `--url`, then `$DATABASE_URL`, then +What just happened: `db init` creates the `migrations/` directory with a starter +`V001__init.sql`; `db migrate -m ` writes a new empty +`V###__.sql` with the version auto-incremented from the highest existing +migration; `db upgrade` applies every pending migration (idempotently — a +re-run applies zero); `db status` reports applied and pending migrations. The +database URL resolves from `--url`, then `$DATABASE_URL`, then `firefly.datasource.url` in `firefly.yaml`, defaulting to `sqlite://firefly.db`. -> **Forward-only migrations.** `firefly db` drives Firefly's own forward-only -> migration runner. Because the runner is forward-only, `firefly db downgrade` -> is unsupported — write a corrective migration instead. +> **Note** The migration runner is **forward-only** (an append-only history, +> Flyway-style). Because of that there is no `firefly db downgrade` — running it +> fails loudly rather than silently no-op'ing. To undo a change, write a new +> corrective migration with `firefly db migrate` instead. + +> **Warning** The CLI's migration backend is **SQLite via `rusqlite`**. A +> `postgres://` or `mysql://` URL returns a clear "not wired into the CLI" error. +> For another driver in production, adapt the `firefly_migrations::Database` port +> and call `firefly_migrations::run` directly from your build, rather than +> through the convenience CLI. -> **Note** — The CLI migration backend is **SQLite via `rusqlite`**; a -> `postgres://` / `mysql://` URL returns a clear "not wired into the CLI" error. -> For another driver, adapt the `firefly_migrations::Database` port and call -> `run` directly from your build, rather than through the CLI. +> **Tip** **Checkpoint.** In a scratch directory run `firefly db init`, then +> `firefly db status --url ":memory:"`. You should see one *pending* migration +> (`V001__init.sql`) and zero applied, because each `:memory:` connection starts +> empty. -## Exporting OpenAPI +## Step 8 — Export OpenAPI and generate clients + +The CLI can emit an OpenAPI document for the current project and, going the other +direction, generate a typed Rust client from any spec. ```bash firefly openapi # OpenAPI 3.1 JSON to stdout firefly openapi --format yaml -o openapi.yaml +firefly openapi-client --spec openapi.json -o client.rs --client-name WalletClient ``` -The document metadata (`info.title` / `info.version` / `description`) is read -from `firefly.yaml` then `Cargo.toml`. A compiled binary cannot boot an -arbitrary app to enumerate live routes, so the CLI emits a metadata-stamped -**skeleton**; to emit Lumen's real routes, build them with -`firefly_openapi::Builder` (which reads the `#[rest_controller]` route table) and -serve them with `Builder::router()`. - -## Introspecting a running app - -These commands query a *running* Lumen over HTTP — a compiled binary has no -offline DI context to boot, so `--url` is required. Point them at Lumen's admin -port (the actuator surface from chapter 15): +What just happened: `firefly openapi` reads the document metadata +(`info.title` / `info.version` / `info.description`) from `firefly.yaml`, then +`Cargo.toml`, and emits an OpenAPI 3.1 document. Because a compiled binary cannot +boot an arbitrary app to enumerate live routes, the exported document is a +metadata-stamped **skeleton** — correct `info` block and the standard +`ProblemDetail` component (Firefly renders errors as +`application/problem+json` per RFC 9457), but empty `paths`. To emit Lumen's +*real* routes, build them with `firefly_openapi::Builder` (which reads the +`#[rest_controller]` route table) and serve them with `Builder::router()` — the +live spec your app already publishes at `/v3/api-docs` on the management port. + +`firefly openapi-client` is the inverse: given an OpenAPI 3.x document, it emits +a self-contained typed client over `firefly_client::RestClient` — a model +struct/enum per `components.schemas` entry and one `async fn` per operation, with +typed path parameters and JSON bodies. `--client-name` names the generated +struct (default `ApiClient`). + +> **Tip** **Checkpoint.** Run `firefly openapi --format yaml | head`. The first +> line is `openapi: 3.1.0`, followed by an `info:` block carrying your project's +> title and version. + +## Step 9 — Introspect a running app + +These commands query a *running* Lumen over HTTP. A compiled binary has no +offline DI context to boot — there is nothing to introspect without a live +process — so `--url` is required, pointed at Lumen's **management** port (the +actuator surface from [Observability](./15-observability.md)). ```bash firefly health --url http://localhost:8081 # -> /actuator/health @@ -191,13 +415,31 @@ firefly beans --url http://localhost:8081 # the DI container's bean table firefly conditions --url http://localhost:8081 # the auto-configuration report ``` -> **Introspecting a running service.** `firefly beans` renders the -> [DI container's bean table](./04a-dependency-injection.md), `firefly conditions` -> the conditional-bean evaluation report, and `firefly routes` the route table — -> all read over HTTP from a running service's actuator port -> (`/actuator/beans`, `/actuator/conditions`, and `/actuator/mappings`). +What just happened: each command GETs a mapped actuator endpoint and pretty-prints +the JSON. `routes` maps to `/actuator/mappings` (every `#[rest_controller]` +route), `health`/`env`/`metrics`/`info` map to their like-named endpoints, and +`beans`/`conditions` render the DI bean table and the conditional-bean +evaluation report — Spring Boot Actuator's DI introspection. `firefly actuator +` is the general form; `firefly health|env|routes|metrics|beans| +conditions` are convenience shortcuts. `--json` emits the raw body for piping. -## Diagnosing, completing, and auditing +> **Note** **Key term — bean.** A *bean* is an object the framework constructs +> and manages for you. `/actuator/beans` lists every one (type, scope, +> stereotype), and `/actuator/conditions` reports the `@Profile` / +> `@ConditionalOn…` guards each conditional bean declared. These are read over +> HTTP from a running service, the same way you would query Spring's `/beans` and +> `/conditions`. See [Dependency Injection](./04a-dependency-injection.md) for +> the bean container itself. + +> **Tip** **Checkpoint.** In one terminal run `cargo run --bin lumen`; in +> another, run `firefly health --url http://localhost:8081`. You should see a +> JSON body with `"status":"UP"`. If `firefly routes --url …` returns an error +> about a missing in-process context, you omitted `--url` — these commands +> always require it. + +## Step 10 — Diagnose, complete, and audit + +The remaining commands report on your environment and dependencies. ```bash firefly info # framework version + which optional adapters are built @@ -208,14 +450,25 @@ firefly sbom --json # machine-readable, for a compliance pipeline firefly license # the framework + dependency license report ``` -`firefly doctor` is the first thing to run on a fresh machine: it reports your -`rustc` / `cargo` versions and whether `git`, `clippy`, `rustfmt`, and `docker` -are on the `PATH`, ending with "All required checks passed!" or a list of what to fix. +What just happened: `firefly doctor` is the first thing to run on a fresh +machine. It reports your `rustc` and `cargo` versions (the two *required* tools) +and whether `git`, `clippy`, `rustfmt`, and `docker` are on the `PATH` (the +*optional* ones), plus the detected project's package, archetype, and whether a +`firefly.yaml` and `migrations/` are present — ending with "All required checks +passed!" or a list of what to fix. `firefly completion ` prints a +shell-completion script generated from the live CLI definition, so it always +matches the available subcommands and flags. `firefly sbom` and `firefly license` +read `Cargo.lock` to produce a Software Bill of Materials and a dependency +license report for a compliance pipeline. -## Running through cargo +> **Tip** **Checkpoint.** Run `firefly doctor`. Inside the framework workspace it +> reports `rustc` and `cargo` as passing required checks and prints a `Project` +> block. The final line is "All required checks passed!". -If you have not installed the binary, drive the CLI through cargo from a -framework checkout: +## Step 11 — Run the CLI through Cargo + +If you have not installed the binary, drive the CLI through Cargo from a +framework checkout — handy in CI, or while iterating on the CLI itself. ```bash make cli ARGS="doctor" @@ -223,35 +476,63 @@ make cli ARGS="new orders --archetype web-api" cargo run -p firefly-cli --bin firefly -- info ``` -## What changed in Lumen - -Nothing in `samples/lumen` itself — this chapter is operational. But you saw the -CLI path to every artifact Lumen grew by hand: `firefly new --archetype web-api` -scaffolds the chapter-2 skeleton, `firefly generate command/query/aggregate/saga` -writes the CQRS, DDD, and orchestration pieces, `firefly run --bin lumen` -launches it with `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` overrides, and -`firefly health/routes/beans --url http://localhost:8081` introspects the actuator surface from -chapter 15. The CLI never -invents APIs — every command maps to a framework crate (`firefly-migrations`, -`firefly-openapi`, the actuator endpoints) you have already met. +What just happened: each form runs the very same `firefly` binary, just without +installing it first. The `--` separates Cargo's own arguments from the ones +passed through to `firefly`. + +## Recap — the CLI maps to crates you already know + +You did not change `samples/lumen` in this chapter; it is operational. But you +saw the CLI path to every artifact Lumen grew by hand: + +- `firefly new --archetype web-api` scaffolds the + [Quickstart](./02-quickstart.md) skeleton — entry point, controller, layered + tree, `Cargo.toml`, `firefly.yaml`, `Dockerfile`, `tests/`. +- `firefly generate command/query/aggregate/saga/migration` writes the CQRS, + DDD, orchestration, and schema pieces — as registrar functions and real + `firefly-*` constructs, not placeholders. +- `firefly run --bin lumen` launches it, mapping `-p`/`-D`/`--env` flags to the + `FIREFLY_*` environment, and `--env FIREFLY_SERVER_ADDR/MANAGEMENT_ADDR` moves + the two ports. +- `firefly build info` stamps the `build-info.json` the `/actuator/info` build + contributor surfaces; `firefly db` drives the forward-only + `firefly-migrations` runner once you adopt a SQL store. +- `firefly health/routes/beans/conditions --url http://localhost:8081` + introspects the actuator surface over HTTP, which is why `--url` is mandatory: + a compiled binary has no offline context to boot. + +The throughline: the CLI never invents an API. Every command calls a framework +crate (`firefly-migrations`, `firefly-openapi`, `firefly-client`) or an actuator +endpoint you have already met, so the command line is just a faster door to the +same building. ## Exercises 1. **Scaffold a Lumen twin.** Run `firefly new lumen2 --archetype web-api - --features web,cqrs --dry-run`, then without `--dry-run`. Compare the + --features web,cqrs --dry-run`, then again without `--dry-run`. Compare the generated `src/` tree to Lumen's, and `cargo build` it. -2. **Generate the CQRS pieces.** In the scaffolded project, run `firefly - generate command OpenWallet` and `firefly generate query GetWallet`. Inspect - the generated handlers and note how they match the `#[command_handler]` / - `#[query_handler]` shape from chapter 9. -3. **Run with a profile and an override.** Start the app with `firefly run -p - dev -D server.port=9090 --dry-run` and read the resolved `FIREFLY_*` - environment it would export. Then drop `--dry-run` and confirm the port. +2. **Generate the CQRS pieces.** In the scaffolded project, run `firefly generate + command OpenWallet` and `firefly generate query GetWallet`. Open the four + generated files and confirm the handlers are `register__handler(bus: + &Bus)` registrar functions calling `bus.register(...)` — the registration + shape from [CQRS](./09-cqrs.md), not a macro. +3. **Learn the env mapping.** Start the app with `firefly run -p dev -D + logging.level-root=DEBUG --dry-run` and read the resolved `FIREFLY_*` + environment it would export. Then move the ports for real with `firefly run + --bin lumen --env FIREFLY_SERVER_ADDR=127.0.0.1:9090 --env + FIREFLY_MANAGEMENT_ADDR=127.0.0.1:9091` and `curl localhost:9091/actuator/health`. 4. **Introspect the real Lumen.** `cargo run --bin lumen`, then in another shell run `firefly health --url http://localhost:8081`, `firefly routes --url - http://localhost:8081`, and `firefly beans --url http://localhost:8081`. - Match the route table against the endpoint table in `web.rs`. + http://localhost:8081`, and `firefly beans --url http://localhost:8081`. Match + the route table against the endpoint constants in `src/web.rs`. +5. **Audit the toolchain.** Run `firefly doctor` on your machine and note which + optional tools (`git`, `clippy`, `rustfmt`, `docker`) are present, then run + `firefly sbom --json | head` to see the resolved-dependency manifest the CLI + reads from `Cargo.lock`. + +## Where to go next With a project scaffolded, generated, run, and introspected, the next chapter -takes Lumen all the way to production. Continue to -[Production & Deployment](./20-production.md). +takes Lumen all the way to production — swapping the in-process event store for +Postgres and Kafka, where `firefly db` and `firefly build` finally earn their +keep. Continue to **[Production & Deployment](./20-production.md)**. diff --git a/docs/book/src/20-production.md b/docs/book/src/20-production.md index 2bb3cf8..4c0c53c 100644 --- a/docs/book/src/20-production.md +++ b/docs/book/src/20-production.md @@ -1,110 +1,220 @@ # Production & Deployment -Lumen has grown from a bare scaffold into a secure, observable, event-sourced -CQRS service with a saga and a scheduled task. This last chapter of the build arc -looks at how the **one-line `main`** boots and runs reliably in production, and -turns on the optional **reactive streaming endpoint**. It also covers everything -between "it works on my machine" and a service in production: graceful shutdown, -the management split, configuration, packaging, and the swap from in-memory -infrastructure to real Postgres + Kafka. - -By the end of this chapter you will understand Lumen's boot pipeline end to end: -two servers (public API + management) with graceful SIGINT/SIGTERM shutdown, and -the streaming endpoint (`GET /api/v1/wallets/:id/events` → NDJSON / SSE) wired as -a `RouteContributor` bean behind the `streaming` feature. - -> **The one-line `main`.** Lumen's whole entry point is -> `firefly::FireflyApplication::new("lumen").run().await`. `run()` is the process -> supervisor: it builds and serves the public + management ports, traps -> SIGINT/SIGTERM, gives each server its own drain signal, and exits cleanly once -> in-flight work drains. A streaming endpoint returns a `Flux` as NDJSON or SSE. - -## The boot pipeline and graceful shutdown +Lumen has grown across the book from a bare scaffold into a secure, observable, +event-sourced CQRS service with a saga, a workflow, a two-phase transfer, and a +scheduled housekeeping task. Everything you added arrived the same way — by +*declaring a bean* the framework discovers — and the entry point never changed. +This final chapter of the build arc closes the loop: we look at exactly how that +**one-line `main`** boots and shuts down reliably, turn on the optional +**reactive streaming endpoint**, and walk the path from "it works on my machine" +to a container running in production — graceful shutdown, the public/management +port split, environment-driven configuration, packaging, and the swap from the +in-memory event store and broker to durable Postgres and Kafka. + +Nothing here rewrites Lumen. The streaming endpoint is one more bean; the +Postgres swap is one bean factory edited in place; the rest is operational +posture. That is the payoff of the port-and-adapter design you have been building +all along. + +By the end of this chapter you will: + +- Trace `run()` end to end — the eight-stage boot pipeline, the two servers, and + the graceful SIGINT/SIGTERM drain that maps a clean shutdown to `Ok(())`. +- Add the optional reactive streaming endpoint (`GET /api/v1/wallets/:id/events` + → NDJSON or SSE) as a feature-gated `RouteContributor` bean, and understand why + the 404 is resolved before the streaming body starts. +- Serve the actuator on a separate, firewalled management port and point your + orchestrator's liveness/readiness probes at the right sub-paths. +- Turn on production-hardening middleware through `CoreConfig` and read the + effective filter chain outermost-to-innermost. +- Swap the in-memory event store and broker for Postgres and Kafka by editing one + `#[bean]` factory, with nothing downstream changing. +- Package Lumen as a container and check it against a deployment checklist. + +## Concepts you will meet + +Before the first step, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — graceful shutdown.** *Graceful shutdown* means that when +> the process is asked to stop, it stops accepting new requests, lets in-flight +> requests finish (within a budget), and only then exits. The Spring Boot analog +> is the `server.shutdown=graceful` setting plus the embedded server's drain; +> Firefly does this by default with no configuration. + +> **Note** **Key term — management surface.** The *management surface* is the set +> of operational HTTP endpoints — health, info, metrics, environment, log-level +> control — that exist for operators and orchestrators, not for end users. +> Firefly serves them on a separate listener from your business API. This mirrors +> Spring Boot Actuator on a dedicated `management.server.port`. + +> **Note** **Key term — RouteContributor.** A *`RouteContributor`* is a bean that +> contributes a sub-router (`axum::Router`) to the public API. The framework +> discovers every `RouteContributor` bean and merges its routes into the assembled +> app, so you can add routes without touching `main` or any `#[rest_controller]`. +> The Spring analog is contributing a `RouterFunction` bean that +> the context picks up automatically. + +> **Note** **Key term — reactive stream / `Flux`.** A *`Flux`* is a reactive +> sequence of zero-or-more `T` values produced over time with backpressure — the +> Rust analog of Project Reactor's `Flux`. Returned from a handler as +> `application/x-ndjson` or `text/event-stream`, it streams element-by-element to +> the client rather than buffering a whole response. + +> **Note** **Key term — port and adapter.** A *port* is an abstract capability +> the domain depends on (here `EventStore`, `Broker`); an *adapter* is a concrete +> implementation of that port (in-memory today, Postgres/Kafka in production). The +> domain talks only to the port, so swapping the adapter changes nothing +> downstream. This is the hexagonal-architecture pattern Spring expresses with +> interfaces and `@Bean` factories. + +## Step 1 — Read the one-line `main` one more time + +Open `src/main.rs`. After every chapter of the build arc, the entry point is +still a single call: + +```rust,ignore +// src/main.rs +#[tokio::main] +async fn main() -> Result<(), firefly::BoxError> { + firefly::FireflyApplication::new("lumen").run().await +} +``` + +What just happened: that one `run()` call component-scans Lumen's beans — the +CQRS bus, the event-sourced ledger, the read-model projection, the query cache, +and the security chain, all over in-memory infrastructure — auto-mounts every +`#[rest_controller]`, auto-discovers security and the read-cache bus middleware, +drains the inventory-registered CQRS handlers / EDA listener / `#[scheduled]` +task, self-hosts the admin dashboard, prints the banner and the line-by-line +startup report, and serves both ports with graceful shutdown. Everything you have +seen chapter by chapter is *declared as a bean*; `main` just hands the crate to +the framework. + +> **Note** Lumen's binary boots with an in-memory event store and broker, so +> `cargo run --bin lumen` needs nothing external — no database, no message +> broker. The tests drive the same wiring in-process through `build_router()`, +> which calls `FireflyApplication::bootstrap()` rather than serving on a socket. + +> **Tip** **Checkpoint.** `cargo run --bin lumen` prints the Firefly banner, the +> two management URLs, and the startup report, then stays running on `:8080` +> (public) and `:8081` (management). `Ctrl-C` exits cleanly. If that works, the +> rest of this chapter is about what is happening underneath and how to take it to +> production. + +## Step 2 — Understand the boot pipeline There is no hand-written lifecycle wiring in Lumen — `run()` does it all. Under -the hood, `FireflyApplication::bootstrap()` assembles the app and -`Bootstrapped::serve()` runs it on the lifecycle `Application`, which traps -SIGINT/SIGTERM, gives each server task its own drain signal, and grants a drain -budget before exiting. The pipeline, in order: +the hood `run()` is exactly two calls: + +```rust,ignore +// firefly::FireflyApplication::run (simplified) +pub async fn run(self) -> Result<(), BoxError> { + self.bootstrap().await?.serve().await +} +``` + +`bootstrap()` assembles the fully wired application and returns a `Bootstrapped` +value (the router, the DI container, the scheduler, and the two bind addresses) +*without* serving. `serve()` then runs it on the lifecycle `Application`, which +traps SIGINT/SIGTERM, gives each server task its own drain signal, and grants a +drain budget before exiting. The pipeline, in order: 1. **Build the web stack** and tee logging into the admin capture buffer. -2. **Component-scan the container** — auto-register the framework's infra beans, - then discover Lumen's `#[derive(Configuration)]`/`#[bean]`, - `#[derive(Controller)]`, and `#[autowired]` beans. -3. **Auto-configure the CQRS bus** — correlation always; the read-cache +2. **Component-scan the container** — auto-register the framework's infrastructure + beans, then discover Lumen's `#[derive(Configuration)]` / `#[bean]` factories, + `#[derive(Controller)]` controllers, and `#[autowired]` fields. +3. **Auto-configure the CQRS bus** — correlation propagation always; the read-cache middleware because Lumen declares a `QueryCache` bean. 4. **Auto-discover security** — the `FilterChain` + `BearerLayer` beans - (Chapter 14), layered onto the API with no `.security(...)` call. + ([Security](./14-security.md)), layered onto the API with no `.security(...)` + call. 5. **Auto-mount controllers** — every `#[rest_controller]` and every - `RouteContributor` bean (including the streaming endpoint below), then apply - the middleware chain + W3C trace origination. -6. **Drain the discovered handlers** — the CQRS handlers, the EDA listener, and - the `#[scheduled]` housekeeping task, from the inventory registries. + `RouteContributor` bean (including the streaming endpoint added in Step 3), + then apply the middleware chain and originate W3C trace context. +6. **Drain the discovered handlers** — the CQRS command/query handlers, the EDA + projection listener, and the `#[scheduled]` housekeeping task, from the + inventory registries. 7. **Self-host the admin dashboard** on the management port and auto-serve the - OpenAPI docs. + OpenAPI docs (Swagger UI, ReDoc, the OpenAPI 3.1 spec) — all on the management + port, never the public one. 8. **Print the startup report**, then **serve both ports** with graceful drain. +> **Note** **Key term — `bootstrap()` vs `serve()`.** `bootstrap()` is the test +> seam: it returns the wired `Bootstrapped` app — including +> `Bootstrapped::api_router`, the fully assembled public router — without binding +> a socket, so tests drive the real app in-process. `serve()` is the production +> path that actually listens. `run()` is just `bootstrap().await?.serve().await`. + The two servers and the drain are the part that matters for production: - **Two servers, two drains.** The public API serves on `:8080` and the - management surface (`/actuator/*` + the self-hosted `/admin` dashboard) on - `:8081`. Each runs on its own task with its own `shutdown` handle, so a signal - drains both listeners independently. + management surface (`/actuator/*` plus the self-hosted `/admin` dashboard plus + the API docs) on `:8081`. Each runs on its own task with its own `shutdown` + handle, so a signal drains both listeners independently — `axum::serve(...) + .with_graceful_shutdown(shutdown.wait())` per server. - **`run()` blocks until a signal.** It returns when SIGINT/SIGTERM is received - and the drain completes. A clean shutdown surfaces as a *cancelled* error, - which `run()` itself maps to `Ok(())`; any other error propagates out of `main`. + and the drain completes. A clean shutdown surfaces internally as a *cancelled* + error, which `serve()` maps to `Ok(())`; any other error propagates out of + `main` and the process exits non-zero. - **Env-overridable binds.** `FIREFLY_SERVER_ADDR` and `FIREFLY_MANAGEMENT_ADDR` override the defaults (`0.0.0.0:8080` / `0.0.0.0:8081`), so a container reads - its ports from the environment. + its ports from the environment with no code change. -## The full boot sequence - -Read top to bottom, `main()` is one line — the framework does the six-step boot -for you: +The "cancelled is clean" mapping is worth seeing exactly, because it is why +`Ctrl-C` is not an error: ```rust,ignore -// src/main.rs -#[tokio::main] -async fn main() -> Result<(), firefly::BoxError> { - firefly::FireflyApplication::new("lumen").run().await +// Bootstrapped::serve (the tail of it) +match application.run().await { + Ok(()) => Ok(()), + // A handle/signal-triggered stop is a clean shutdown, not a failure. + Err(err) if err.is_cancelled() => Ok(()), + Err(err) => Err(Box::new(err)), } ``` -That single `run()` call component-scans Lumen's beans — the CQRS bus, the -event-sourced ledger, the read model, the projection (seeded inside the `ledger` -`#[bean]`), and the security chain over in-memory infrastructure — auto-mounts -the controllers, drains the discovered handlers/listener/`#[scheduled]` task, -self-hosts the admin dashboard, prints the banner and the line-by-line startup -report, and serves both ports. Everything you have seen chapter by chapter is -*declared as a bean*; `main` just hands the crate to the framework. +What just happened: the lifecycle `Application` runs both server tasks; a +SIGINT/SIGTERM cancels them, which surfaces as a *cancelled* error; `serve()` +catches exactly that case and returns `Ok(())`, so `main` exits zero. Any genuine +failure (a port already bound, a panic in a server task) propagates and the +process exits non-zero — which is what you want an orchestrator to restart on. -> **Teaching code runs with no dependencies.** Lumen's binary boots with an -> in-memory event store and broker, so `cargo run --bin lumen` needs nothing -> external. The tests drive `build_router()` (which calls -> `FireflyApplication::bootstrap()`) in-process rather than this binary. +> **Tip** **Checkpoint.** Run `cargo run --bin lumen`, then press `Ctrl-C`. The +> process exits with no stack trace and a zero status (`echo $?` prints `0`). +> That is the cancelled-to-`Ok(())` mapping in action. -## The reactive streaming endpoint (feature `streaming`) +## Step 3 — Add the reactive streaming endpoint (feature `streaming`) Lumen's last endpoint streams a wallet's event history. It is feature-gated so -the teaching baseline stays lean — `Cargo.toml` declares it, and it needs nothing -beyond the facade (`firefly::reactive::Flux` + `firefly::web::{NdJson, Sse}`): +the teaching baseline stays lean — it needs nothing beyond the `firefly` facade +(`firefly::reactive::Flux` plus `firefly::web::{NdJson, Sse}`). `Cargo.toml` +already declares the flag, off by default: ```toml +# Cargo.toml [features] +# The reactive streaming endpoint is feature-gated so the teaching baseline +# stays lean; this chapter turns it on. It needs nothing beyond the facade. default = [] streaming = [] ``` +### 3a — Declare the route as a bean + The endpoint is wired by **declaring a bean**, not by editing an entry point. A `#[derive(Service)]` that `provides = "dyn firefly::web::RouteContributor"` contributes the sub-router; `FireflyApplication` resolves it as the -`dyn RouteContributor` port and merges its routes, so a feature-gated endpoint -appears purely because its crate compiled it in: +`dyn RouteContributor` port (Step 2, stage 5) and merges its routes — so a +feature-gated endpoint appears in the API purely because its crate compiled it +in. Add this to `src/web.rs`: ```rust,ignore +// src/web.rs /// (feature `streaming`) A `RouteContributor` bean adding the reactive -/// `GET /api/v1/wallets/:id/events` endpoint. The framework discovers it and -/// merges its routes — no composition-root step. +/// `GET /api/v1/wallets/:id/events` endpoint. The framework discovers it +/// (resolved as the `dyn RouteContributor` port) and merges its routes — a +/// feature-gated endpoint wired by declaring a bean, not by a composition root. #[cfg(feature = "streaming")] #[derive(Service)] #[firefly(provides = "dyn firefly::web::RouteContributor")] @@ -121,18 +231,54 @@ impl firefly::web::RouteContributor for StreamingRoutes { } ``` -The handler itself lives on the sub-router `streaming_router` builds. It loads -the wallet's persisted events, maps them to the view shape, wraps them in a -`Flux`, and returns NDJSON by default or SSE when `?format=sse` is passed: +What just happened, block by block: + +- `#[derive(Service)]` makes `StreamingRoutes` a DI bean. The + `#[firefly(provides = "dyn firefly::web::RouteContributor")]` attribute registers + it under the `RouteContributor` *port*, so the framework finds it when it + collects route contributors — you never name `StreamingRoutes` anywhere else. +- `#[autowired] api: Arc` injects the same controller bean the + `#[rest_controller]` uses, so the stream reads the very wallets the rest of the + API writes. +- `impl RouteContributor` returns the sub-router that `streaming_router` builds. + `RouteContributor::routes(&self) -> axum::Router` is the one method the trait + requires. + +> **Note** Everything in this section is behind `#[cfg(feature = "streaming")]`, +> so with the feature off the file compiles to nothing extra and the endpoint +> does not exist. Turning it on is a build flag, not a code change to `main`. + +### 3b — Build the sub-router and the handler + +The sub-router maps the one route onto the handler over the controller state, and +the handler loads the wallet's persisted events, maps them to the view shape, +wraps them in a `Flux`, and returns NDJSON by default or SSE when `?format=sse` +is passed: ```rust,ignore +// src/web.rs +/// Builds the streaming sub-router over the controller state. +#[cfg(feature = "streaming")] +fn streaming_router(api: WalletApi) -> axum::Router { + axum::Router::new() + .route( + "/api/v1/wallets/:id/events", + axum::routing::get(stream_events), + ) + .with_state(api) +} + +/// The reactive streaming handler: builds a `Flux` over the +/// wallet's persisted stream and returns it as NDJSON (one JSON document per +/// line) or, with `?format=sse`, as Server-Sent Events. +#[cfg(feature = "streaming")] async fn stream_events( State(api): State, Path(id): Path, axum::extract::Query(params): axum::extract::Query, ) -> Response { use crate::domain::WalletEvent; - use axum::response::Response; + use axum::response::IntoResponse; use firefly::reactive::Flux; use firefly::web::{NdJson, Sse}; @@ -152,39 +298,103 @@ async fn stream_events( } ``` -`NdJson(flux)` renders one JSON document per line (`application/x-ndjson`); -`Sse(flux)` renders Server-Sent Events (`text/event-stream`). The 404 for an -unknown wallet is resolved *before* the response head is committed, because once -a streaming body starts you can no longer change the status. `tests/streaming.rs` -(run with `--features streaming`) proves all three behaviors: +What just happened, block by block: + +- `streaming_router` returns a plain `axum::Router` with `GET + /api/v1/wallets/:id/events` mapped to `stream_events` and the `WalletApi` state + attached. This is the sub-router `StreamingRoutes::routes` hands back. +- `stream_events` first calls `api.ledger.load_events(&id)`. If the wallet is + absent the ledger returns `Err(NotFound)`, and the handler renders that as an + RFC 9457 `application/problem+json` 404 *and returns* — before any streaming + body has started. +- On success it maps the domain events to the `WalletEvent` view shape, wraps the + `Vec` in `Flux::just(...)`, and chooses the encoding: `Sse(flux)` for + `?format=sse`, otherwise `NdJson(flux)`. + +> **Note** **Key term — `NdJson` and `Sse`.** `NdJson(flux)` (`pub struct +> NdJson(pub Flux)`) renders the `Flux` as one JSON document per line with +> content type `application/x-ndjson`; `Sse(flux)` renders Server-Sent Events with +> content type `text/event-stream`. Both wrap a `Flux` and implement +> `IntoResponse`, so a handler returns them directly. + +> **Warning** Order matters here. The 404 for an unknown wallet must be resolved +> *before* the response head is committed, because once a streaming body starts +> the status line is already on the wire and can no longer change. That is why +> `load_events` is awaited and checked first, and only then is a `Flux` built. + +> **Note** `Flux::just(items)` materializes a known `Vec` — fine for a finite +> event history that is already loaded. A production stream over a live, unbounded +> source (e.g. a broker subscription) would use `Flux::from_stream(...)` instead, +> so the body is produced lazily with backpressure rather than buffered up front. + +### 3c — Prove the three behaviors with a test + +`src/streaming_test.rs` (compiled only under `#[cfg(all(test, feature = +"streaming"))]`) boots one app context, opens a wallet, makes a deposit — so the +stream has two events — and asserts the NDJSON default, the SSE switch, and the +404. The default case: ```rust,ignore +// src/streaming_test.rs #[tokio::test] async fn events_stream_as_ndjson_by_default() { let app = build_router().await; - let id = open_with_deposit(&app).await; // two events: opened + deposited + let id = open_with_deposit(&app).await; // two events: WalletOpened + MoneyDeposited let res = app .clone() - .oneshot(Request::get(format!("/api/v1/wallets/{id}/events")).body(Body::empty()).unwrap()) + .oneshot( + Request::get(format!("/api/v1/wallets/{id}/events")) + .body(Body::empty()) + .unwrap(), + ) .await .unwrap(); assert_eq!(res.status(), StatusCode::OK); - // content-type contains "ndjson"; the body is two JSON lines: - // one "WalletOpened", one "MoneyDeposited". + + let ct = res + .headers() + .get("content-type") + .and_then(|v| v.to_str().ok()) + .unwrap_or_default() + .to_owned(); + assert!(ct.contains("ndjson"), "default stream should be NDJSON, got {ct:?}"); + + let body = res.into_body().collect().await.unwrap().to_bytes(); + let text = String::from_utf8(body.to_vec()).unwrap(); + let lines: Vec<&str> = text.lines().filter(|l| !l.trim().is_empty()).collect(); + assert_eq!(lines.len(), 2, "expected 2 NDJSON lines, got: {text:?}"); + assert!(text.contains("WalletOpened")); + assert!(text.contains("MoneyDeposited")); } ``` -> **Streaming responses.** Returning a `Flux` as `application/x-ndjson` or -> `text/event-stream` streams element-by-element with backpressure. `Flux::just` -> materializes a known `Vec`; a production stream would use `Flux::from_stream` -> over a live subscription so the body is produced lazily rather than buffered. - -## The management split in production +What just happened: `build_router().await` returns the fully wired public router +in-process (it calls `FireflyApplication::bootstrap()` under the hood, as in +[Testing](./18-testing.md)). The test drives it with `tower::ServiceExt::oneshot` +— no socket bound — opens a wallet with a deposit, then `GET`s the events stream +and asserts the response is `200`, is `application/x-ndjson`, and carries exactly +two JSON lines (the `WalletOpened` and the `MoneyDeposited`). Sibling tests assert +that `?format=sse` flips the content type to `text/event-stream` and that an +unknown wallet id is a `404`. + +> **Tip** **Checkpoint.** Build and test with the feature on: +> +> ```bash +> cargo test -p firefly-sample-lumen --features streaming +> ``` +> +> All three streaming tests pass. Then run the binary the same way — +> `cargo run --bin lumen --features streaming` — open a wallet, deposit, and +> `curl http://127.0.0.1:8080/api/v1/wallets//events` to see the two NDJSON +> lines on the public port. + +## Step 4 — Split the management surface for production Always serve the actuator on a **different listener** from the public API so `/actuator/*` is reachable by your orchestrator but never on the public network — -exactly the `:8080` / `:8081` split Lumen uses. Firewall the admin port. The -health sub-paths feed your orchestrator's probes: +exactly the `:8080` / `:8081` split Lumen uses by default. Firewall the management +port to your cluster's internal network. The health sub-paths feed your +orchestrator's probes: | Probe | Endpoint | |-----------|-------------------------------------| @@ -192,13 +402,46 @@ health sub-paths feed your orchestrator's probes: | readiness | `/actuator/health/readiness` | | overall | `/actuator/health` | -## Production hardening middleware +What just happened: the management router (Step 2, stage 7) mounts the full +actuator tree plus the admin dashboard and the API docs. The liveness probe +reports only indicators tagged for liveness (is the process alive?); readiness +reports only readiness indicators (can it serve traffic — dependencies up?). Point +your orchestrator's liveness probe at `/actuator/health/liveness` and its +readiness probe at `/actuator/health/readiness`, both on `:8081`. + +> **Note** Metrics for scraping live on the same management port: +> `/actuator/prometheus` serves labeled Prometheus exposition, and +> `/actuator/metrics` serves the JSON view. Point your scraper at `:8081`, +> never `:8080`. + +> **Tip** **Checkpoint.** With Lumen running, from a second terminal: +> `curl localhost:8081/actuator/health/readiness` returns a JSON body with a +> `"status"`, and the same path on `:8080` returns nothing — the public port has +> no `/actuator/*`. + +## Step 5 — Turn on production-hardening middleware + +The framework already turns on the web-tier batteries when `FireflyApplication` +builds the web stack: the RFC 9457 problem renderer, correlation-id propagation, +W3C trace-context origination, request metrics, and idempotency replay. The +remaining production middleware is opt-in through `CoreConfig`, tuned via +`FireflyApplication::configure(|cfg| { … })`, and each knob weaves its layer in at +the correct filter order: + +```rust,ignore +// Opt-in production middleware, tuned at the entry point. +firefly::FireflyApplication::new("lumen") + .configure(|cfg| { + cfg.cors = Some(firefly::web::CorsConfig::default()); + cfg.security_headers = Some(firefly::web::SecurityHeadersConfig::default()); + cfg.csrf = Some(firefly::web::CsrfLayer::new()); // browser flows only + cfg.request_log = Some(firefly::web::RequestLogLayer::default()); + }) + .run() + .await +``` -The framework already turns on CORS, OWASP security headers, request metrics, -and the access log (the web-tier batteries) when `FireflyApplication` builds the -web stack. The remaining production middleware is opt-in through `CoreConfig` — -tuned via `FireflyApplication::configure(|cfg| { … })` — weaving in at the -correct filter order: +The knobs and what each adds: | Knob | Adds | |--------------------|-------------------------------------------------------| @@ -208,18 +451,56 @@ correct filter order: | `request_log` | one structured access-log event per request | | `request_metrics` | `http_server_requests_seconds` + `_max` (actuator) | | `http_exchanges` | recent-exchange recorder + `/actuator/httpexchanges` | -| `loggers` | `/actuator/loggers` runtime log-level control | +| `loggers` | `/actuator/loggers` runtime log-level control | + +What just happened: `configure(|cfg| { … })` hands you the `CoreConfig` before the +web stack is built, so the layers you switch on are woven in at boot. Every +optional knob defaults to OFF, except request metrics, which are on by default +(Spring-Boot-style auto-instrumentation) and tuned — or disabled — through +`request_metrics` / `disable_request_metrics`. + +The effective chain, outermost (nearest the network) to innermost (nearest your +handler), is: + +```text +CorsLayer (cors) — preflight + simple-request edge +ProblemLayer (always) — panic → RFC 9457 500 +SecurityHeadersLayer (security_headers) — decorate every response +TraceContextLayer (always) — validate/originate W3C traceparent +CorrelationLayer (always) — X-Correlation-Id (+ request ctx) +MetricsLayer (request_metrics) — http_server_requests_* +HttpExchangesLayer (http_exchanges) — record into the recorder +RequestLogLayer (request_log) — one access-log event +CsrfLayer (csrf) — double-submit cookie +IdempotencyLayer (always) — replay on Idempotency-Key + │ + ▼ + your router +``` + +Idempotency stays innermost so a replayed request still passes every outer +concern (correlation, metrics, the access log). The W3C `TraceContextLayer` sits +just outside correlation so it can originate a root span and a `traceparent` that +the inner correlation layer then echoes on the response. -The effective chain (outermost → innermost) is CORS → Problem → SecurityHeaders → -Correlation → Metrics → HttpExchanges → RequestLog → CSRF → Idempotency → your -router. Idempotency stays innermost so a replayed request still passes every -outer concern. +> **Design note.** This is the same actuator-and-middleware posture Spring Boot +> ships, but switched on declaratively at one call site rather than through a +> scatter of properties and `@Configuration` classes. A bare `FireflyApplication` +> already gives you the always-on Problem → TraceContext → Correlation → +> Idempotency core; `configure(...)` adds the rest. -## Configuration in production +> **Tip** **Checkpoint.** Run Lumen with `security_headers` on and +> `curl -i localhost:8080/api/v1/wallets/anything`. The response carries +> `X-Content-Type-Options: nosniff` and `X-Frame-Options: DENY` even on the 404 +> problem body — proof the layer decorates every response, including recovered +> errors. + +## Step 6 — Configure for production from the environment Bind configuration from layered sources with environment overrides on top, so a -container reads its settings from the environment (Chapter 3). For Lumen, the two -bind addresses are already env-driven: +container reads its settings from the environment ([Configuration](./03-configuration.md)). +For Lumen the two bind addresses are already env-driven, so the production +container needs no config file just to move its ports: ```bash FIREFLY_PROFILE=prod \ @@ -228,43 +509,96 @@ FIREFLY_MANAGEMENT_ADDR=0.0.0.0:8081 \ ./lumen ``` +What just happened: `FIREFLY_SERVER_ADDR` / `FIREFLY_MANAGEMENT_ADDR` are read at +construction time (Step 2) and override the `0.0.0.0:8080` / `0.0.0.0:8081` +defaults, while `FIREFLY_PROFILE=prod` selects the production property layer. `FIREFLY_*` variables beat the YAML files, secrets are masked in `/actuator/env`, -and `${...}` placeholders resolve env-then-config-then-default. The JWT signing -key (Chapter 14) is the obvious thing to inject this way rather than inline. +and `${...}` placeholders resolve env-then-config-then-default. + +> **Warning** The JWT signing key from [Security](./14-security.md) is the obvious +> thing to inject this way — from the environment or a secret store — rather than +> the inline `DEMO_SIGNING_KEY` constant Lumen ships for teaching. Never bake a +> real signing key into the binary or commit it to source. -## From in-memory to real infrastructure +## Step 7 — Swap in-memory infrastructure for Postgres and Kafka -This is the payoff of the whole architecture: Lumen swaps its in-memory defaults -for real backends by **changing a `#[bean]` factory in `LumenBeans`**, and -nothing downstream changes. The `event_store` bean returns a `MemoryEventStore` -today; production returns a durable store instead, and (where Lumen overrides -the broker) a `#[bean]` returns a Kafka adapter behind the `Broker` port: +This is the payoff of the whole architecture. Lumen swaps its in-memory defaults +for real backends by **changing a `#[bean]` factory in `LumenBeans`**, and nothing +downstream changes. Recall the in-memory factory in `src/web.rs`: ```rust,ignore +// src/web.rs — today #[bean] impl LumenBeans { - // today: + /// The in-memory event store (`@Bean`). #[bean] - fn event_store(&self) -> MemoryEventStore { MemoryEventStore::new() } + fn event_store(&self) -> MemoryEventStore { + MemoryEventStore::new() + } - // production: a Postgres-backed event store behind the EventStore port. - // #[bean] - // fn event_store(&self) -> PostgresEventStore { PostgresEventStore::connect(...) } + // … the ledger factory autowires the EventStore + the framework Broker port + #[bean] + fn ledger(&self, store: Arc, broker: Arc) -> Ledger { + let store: Arc = store; + Ledger::new(store, broker) + } } ``` -The `ledger` factory depends on the `EventStore` *port*, and the `Ledger`, the -projection, the CQRS handlers, the saga, and every test are written against the -`EventStore` and `Broker` ports — so the wallet domain never learns it moved -from a `HashMap` to Postgres + Kafka. That is "swap the adapter, keep the code," -applied to the storage and messaging tiers, with the swap localized to one bean -factory. +To go to Postgres, return the framework's SQL-backed event store +(`firefly::eventsourcing::SqlEventStore`) behind the same `EventStore` port. It +takes a `Database` port, so the swap is contained to the factory: -## Container packaging +```rust,ignore +// src/web.rs — production: a Postgres-backed event store behind the EventStore port. +use firefly::eventsourcing::{EventStore, SqlEventStore}; + +#[bean] +impl LumenBeans { + /// An async `#[bean]` factory: connect the pool, build the SQL event store, + /// create its table once, and hand back a `dyn EventStore` for the `ledger` + /// factory to autowire. Any error here aborts startup (fail-fast). + #[bean] + async fn event_store(&self, db: Arc) -> SqlEventStore { + let store = SqlEventStore::new(db); + store.initialize().expect("create event-store table"); + store + } +} +``` + +What just happened: the `ledger` factory depends on the `EventStore` *port*, and +the `Ledger`, the read-model projection, the CQRS handlers, the saga, the TCC +transfer, and every test are written against the `EventStore` and `Broker` ports — +so the wallet domain never learns it moved from a `HashMap` to Postgres. The same +shape applies to messaging: where Lumen overrides the broker, a `#[bean]` returns +a Kafka adapter behind the framework's `Broker` port, and the EDA projection +listener consumes from Kafka instead of the in-process bus without changing a line +of the projection. + +> **Note** The `event_store` bean here is an `async fn`. The framework awaits async +> bean factories during the component-scan (Step 2, stage 2), so the pool is dialed +> and the store is live before anything resolves it — and a connection failure +> aborts startup rather than surfacing on the first request. That is the fail-fast +> property you want in production. + +> **Design note.** "Swap the adapter, keep the code" applied to the storage and +> messaging tiers, with the swap localized to one bean factory. The domain, the +> handlers, the projection, the saga, and the tests are written against ports — +> exactly the hexagonal design this book has built toward. -A typical multi-stage build for Lumen: +> **Tip** **Checkpoint.** You do not need to actually stand up Postgres to learn +> the shape: read the `ledger` factory and confirm it names `Arc`, +> not `MemoryEventStore`. Anything that autowires the *port* is swap-ready by +> construction. + +## Step 8 — Package Lumen as a container + +A typical multi-stage build compiles the release binary in a Rust image, then +copies just the binary into a slim runtime image: ```dockerfile +# Dockerfile FROM rust:1.88 AS build WORKDIR /app COPY . . @@ -276,15 +610,25 @@ EXPOSE 8080 8081 ENTRYPOINT ["/usr/local/bin/lumen"] ``` -Because `run()` traps SIGTERM and drains, the container stops cleanly when the -orchestrator sends a termination signal — no `--init` shim or signal-forwarding -wrapper required. +What just happened: the `build` stage compiles the `firefly-sample-lumen` package +(its `[[bin]]` is named `lumen`, so the artifact lands at +`target/release/lumen`); the runtime stage copies only that binary onto a minimal +Debian image and exposes both ports. Because `run()` traps SIGTERM and drains +(Step 2), the container stops cleanly when the orchestrator sends a termination +signal — no `--init` shim or signal-forwarding wrapper required. + +> **Tip** **Checkpoint.** `docker build -t lumen .` produces an image, and +> `docker run -p 8080:8080 -p 8081:8081 lumen` boots it. `docker stop` on that +> container exits gracefully (no SIGKILL fallback within the drain budget), +> because the binary handles SIGTERM itself. ## A deployment checklist - [ ] Actuator (`:8081`) on a **separate, firewalled** port from the API (`:8080`). -- [ ] Liveness/readiness probes pointed at `/actuator/health/{liveness,readiness}`. -- [ ] `security_headers`, `cors`, and (for browser flows) `csrf` on. +- [ ] Liveness/readiness probes pointed at + `/actuator/health/{liveness,readiness}`. +- [ ] `security_headers`, `cors`, and (for browser flows) `csrf` switched on + through `configure(...)`. - [ ] `request_log` + `request_metrics` on; logs shipped as JSON, metrics scraped from `/actuator/prometheus`. - [ ] Correlation propagation verified end-to-end across services. @@ -295,41 +639,61 @@ wrapper required. - [ ] The verification gate green: `cargo test -p firefly-sample-lumen` and `--features streaming`, plus `clippy -D warnings` and `fmt --check`. -## What changed in Lumen - -This chapter looked at the boot pipeline and added the optional stream — the -last code the arc adds: - -- **`src/main.rs`** is one line: `FireflyApplication::new("lumen").run().await`. - `run()` component-scans the beans, auto-mounts the controllers, drains the - discovered handlers/listener/`#[scheduled]` task, self-hosts the admin - dashboard, prints the startup report, and serves the public + management ports - with graceful SIGINT/SIGTERM drain. A clean shutdown is a *cancelled* error - that `run()` maps to `Ok(())`. -- **`Cargo.toml`** declares the `streaming` feature; **`src/web.rs`** adds the - feature-gated `StreamingRoutes` `RouteContributor` bean (and its - `stream_events` handler) that returns a `Flux` as NDJSON (default) - or SSE (`?format=sse`), with the 404 resolved before the response head — wired - purely by declaring the bean. -- **`tests/streaming.rs`** proves the NDJSON default, the SSE switch, and the - 404, all behind the `streaming` feature. -- The chapter showed the **in-memory → Postgres + Kafka** swap as a one-bean - change in `LumenBeans`, proving the port-and-adapter design end to end. +## Recap — what changed in Lumen + +| Before this chapter | After this chapter | +|---------------------|--------------------| +| a wired service you ran but had not taken to production | a deployable container with the management split, hardening middleware, and a port-swap path | +| no streaming endpoint | the feature-gated `StreamingRoutes` `RouteContributor` bean serving `GET /api/v1/wallets/:id/events` as NDJSON (default) or SSE (`?format=sse`) | +| in-memory event store / broker only | the one-`#[bean]` swap to `SqlEventStore` + a Kafka `Broker` adapter, with nothing downstream changed | + +You also now know: + +- That `run()` is `bootstrap().await?.serve().await`: an eight-stage boot, two + servers each with its own drain, and a *cancelled* error mapped to `Ok(())` so a + signalled shutdown exits zero. +- That a feature-gated endpoint is wired purely by declaring a `RouteContributor` + bean — `main` never changes — and that a streaming handler must resolve its 404 + before the response head is committed. +- That production hardening (CORS, OWASP headers, CSRF, access log) is opt-in + through `CoreConfig` at one `configure(...)` call site, weaving into the correct + filter order. +- That the storage and messaging swap is one bean factory, because the domain, + handlers, projection, saga, and tests all depend on the `EventStore` and + `Broker` ports. + +That completes the guided build arc. Lumen began as an empty directory in +[Quickstart](./02-quickstart.md); it is now a secure, observable, event-sourced +CQRS service that streams its history and deploys as a single container — and the +one-line `main` never changed. ## Exercises -1. **Run and drain.** `cargo run --bin lumen`, open a wallet, then Ctrl-C and - watch the graceful drain. Confirm the process exits zero. +1. **Run and drain.** `cargo run --bin lumen`, open a wallet, then `Ctrl-C` and + watch the graceful drain. Confirm the process exits zero with `echo $?`, and + that no stack trace prints — the cancelled-to-`Ok(())` mapping from Step 2. 2. **Override the ports.** Start Lumen with `FIREFLY_SERVER_ADDR=0.0.0.0:9000 FIREFLY_MANAGEMENT_ADDR=0.0.0.0:9001 cargo run --bin lumen` and confirm the API - and management surfaces move. + moved to `:9000` and the actuator to `:9001`, independently. 3. **Stream the history.** Build with `--features streaming`, open a wallet and deposit, then `curl http://127.0.0.1:8080/api/v1/wallets//events` (NDJSON) - and `...?format=sse` (SSE). Compare the two content types. -4. **Sketch the Postgres swap.** Write the `event_store` `#[bean]` in `LumenBeans` - that would return a Postgres-backed `EventStore`, and explain in one sentence - why the `ledger` factory, the projection, and the tests need no change. - -That completes the guided tour of Lumen. The remaining chapters revisit the -declarative macros as a capstone and provide reference material: the -[Module Index](./91-appendix-modules.md) and the [Glossary](./92-glossary.md). + and `…?format=sse` (SSE). Compare the two `content-type` headers, and confirm + `GET /api/v1/wallets/wlt_missing/events` returns a `404` problem document. +4. **Harden the chain.** Add `cfg.security_headers = Some(...)` and + `cfg.request_log = Some(...)` through `configure(...)`, re-run, and inspect a + response with `curl -i`. Find the OWASP headers, then place each layer in the + outermost-to-innermost chain from Step 5. +5. **Sketch the Postgres swap.** Write the `event_store` `#[bean]` in `LumenBeans` + that returns a `SqlEventStore` over a `Database` port, and explain in one + sentence why the `ledger` factory, the read-model projection, and the tests + need no change. + +## Where to go next + +- Revisit the declarative macros that made all of this possible — the `#[bean]`, + `#[rest_controller]`, `#[command_handler]`, `#[saga]`, and `#[scheduled]` + attributes as a capstone — in + **[Declarative Services with Macros](./21-declarative-macros.md)**. +- Look up any building block by crate in the + **[Module Index](./91-appendix-modules.md)**, or any term in the + **[Glossary](./92-glossary.md)**. diff --git a/docs/book/src/20a-experience-tier.md b/docs/book/src/20a-experience-tier.md index 24ae70d..a5da5de 100644 --- a/docs/book/src/20a-experience-tier.md +++ b/docs/book/src/20a-experience-tier.md @@ -1,55 +1,125 @@ # The Experience Tier (BFF) -Lumen, as the book has built it, is a self-contained service: it owns the -wallet domain *and* the HTTP API that fronts it. Real Firefly systems split that -responsibility across **three service tiers**, and Lumen sits at the bottom two. -By the end of this chapter you will know where a frontend-facing API for Lumen -belongs — the **experience tier** — and you will build a small Backend-for-Frontend -that composes Lumen as a downstream domain SDK, drives a multi-step "fund a -wallet and confirm" journey with a signal-gated workflow, and survives a client -disconnect by persisting journey state. - -This is the one Firefly tier Lumen itself never enters, so the chapter introduces -it from first principles and then wires it against the service you already have. - -> **What the experience tier is.** The experience tier is Firefly's -> Backend-for-Frontend layer, built on the `firefly-starter-experience` crate: it -> composes downstream domain SDKs, drives multi-request journeys with signal-gated -> workflows, and persists journey state across requests. A signal gate parks a -> workflow step until a named signal arrives; a domain-SDK registry resolves each -> downstream client by logical name. - -## The three service tiers - -Firefly services are layered into three **service** tiers (distinct from the +Lumen, as the book has built it, is a self-contained service: it owns the wallet +domain *and* the HTTP API that fronts it. Real Firefly estates split that +responsibility across **three service tiers**, and Lumen sits in the lower two. +This chapter introduces the one tier Lumen itself never enters — the +**experience tier**, Firefly's Backend-for-Frontend layer — and then wires a +small BFF against the service you already have. The BFF composes Lumen as a +downstream domain SDK, drives a multi-step "fund a wallet and confirm" journey +with a signal-gated workflow, and survives a client disconnect by persisting the +journey's state. + +Because this tier is new ground, the chapter teaches it from first principles: +what the three tiers are and why the dependency direction is one-way, what an +`ExperienceStack` gives you, how to register a domain SDK, how a signal gate +parks a workflow until an external event arrives, and how to persist and query a +journey that spans several HTTP requests. Every API here is drawn from the real +`firefly-starter-experience` crate — the same surface its own boot test exercises +end to end. + +By the end of this chapter you will: + +- Explain the `channel → experience → domain → core` tier model and why the + experience tier may compose *domain* SDKs only — never a database, a core + service, or a sibling BFF. +- Build an `ExperienceStack` (the BFF starter) and understand how it inherits the + full web batteries while adding five experience-tier building blocks. +- Register Lumen as a named domain SDK through `DomainClients` and resolve it by + logical name from any handler or workflow step. +- Model a multi-request journey as a `Workflow` whose gate parks on a named + signal, and resume it by delivering that signal from a later request. +- Persist a journey's state with `WorkflowState` (Redis-capable) and answer + "where is my journey?" with `WorkflowQueryService`, so a client can reconnect + after a disconnect. +- Assemble the three-endpoint atomic-REST controller that ties it all together. + +## Concepts you will meet + +These are the ideas the chapter leans on. Each is reintroduced in context the +first time it is used; this is the short version. + +> **Note** **Key term — Backend-for-Frontend (BFF).** A BFF is an HTTP service +> that exists to serve *one* frontend or channel: it aggregates several +> downstream services into endpoints shaped exactly for that UI's screens and +> flows. It owns no database of its own. The Spring analog is a Spring Cloud +> Gateway / aggregation service sitting in front of your domain microservices — +> here it is a first-class, batteries-included tier. + +> **Note** **Key term — domain SDK.** A *domain SDK* is just an HTTP client +> pointed at a downstream domain service's public API, dressed up with the +> framework's correlation propagation, JSON codec, error decoding, and +> retry/backoff. A BFF calls its dependencies through these SDKs exactly as any +> external client would — it never reaches into their internals. + +> **Note** **Key term — signal gate.** A *signal gate* is a workflow step that +> parks (suspends) until a named *signal* is delivered from outside the workflow +> — typically by a later HTTP request. It models "wait for the customer to +> confirm" inside an otherwise sequential journey. The Java/Firefly analog is a +> `@WaitForSignal` step; there is no direct Spring Boot equivalent. + +## Step 1 — Understand the three service tiers + +Firefly estates are layered into three **service** tiers (distinct from the crate-graph tiers in the architecture docs). The dependency direction is strict -and one-way: `channel → experience → domain → core`. +and one-way: `channel → experience → domain → core`. A tier may only call the +tier directly below it. | Service tier | Owns | Talks to | Rust starter | |--------------|------|----------|--------------| -| **core** | the database (R2DBC/sqlx, migrations, CRUD) | nothing below | `firefly-starter-core` / `firefly-starter-data` | +| **core** | the database (sqlx, migrations, CRUD) | nothing below | `firefly-starter-core` / `firefly-starter-data` | | **domain** | sagas, CQRS, event sourcing, third-party adapters | **core** SDKs | `firefly-starter-domain` | | **experience (BFF)** | signal-driven workflows, stateless aggregation, atomic REST | **domain** SDKs *only* | `firefly-starter-experience` | -Lumen, with its event-sourced ledger, CQRS bus, and transfer saga, is a -**domain** service. An **experience** service is a Backend-for-Frontend: it -aggregates several domain SDKs into APIs shaped for *one* frontend or channel. It -**never** owns a database, **never** calls a core service directly, and **never** -calls a sibling experience service — it composes domain SDKs (over +Lumen — with its event-sourced ledger, its CQRS bus, and its transfer saga — is a +**domain** service. An **experience** service is the BFF that fronts it: it +aggregates one or more domain SDKs into endpoints shaped for a single frontend or +channel. It **never** owns a database, **never** calls a core service directly, +and **never** calls a sibling experience service. It composes domain SDKs (over `firefly-client`) and nothing else. -> **The tier boundary is a type.** The strict `channel → experience → domain → -> core` direction is enforced by the starters themselves: an experience stack can -> register domain SDKs and nothing else. A BFF never owns a database and never -> calls a core service or a sibling BFF directly — the dependency direction is -> one-way by construction. - -## `ExperienceStack` — batteries for a BFF +What just happened: you placed Lumen on the map. The book has been building a +domain service all along; this chapter builds the tier *above* it. Knowing the +direction matters because it is not a convention you remember — it is enforced by +the starters themselves. + +> **Design note.** The tier boundary is a *type*, not a code-review rule. An +> experience stack exposes a registry that holds domain SDKs and nothing else +> (you will meet it in Step 3), and it has no data-access surface to register a +> database against. A BFF that tried to own a table or dial a core service simply +> has no API to do so — the dependency direction is one-way by construction. + +> **Tip** **Checkpoint.** You can state, without looking, what each tier owns and +> who it may call. The experience tier composes domain SDKs only; Lumen is the +> domain service the BFF in this chapter will call. + +## Step 2 — Build the BFF stack with `ExperienceStack` + +A BFF lives in its own crate. Unlike a plain domain service — which depends only +on the one `firefly` facade — a BFF depends on the experience-tier starter +directly, because the facade carries the core and web starters but not the +experience one. + +> **Note** **Key term — experience starter.** `firefly-starter-experience` is the +> crate that turns a web service into a BFF. It builds on the web starter (so you +> get every HTTP battery) and adds the journey machinery. The Spring analog is a +> Spring Boot *starter* that bundles a coherent slice of capability behind one +> dependency. + +```toml +# Cargo.toml of a BFF crate (e.g. lumen-bff). Note: the experience starter is a +# direct dependency — the `firefly` facade carries the core + web starters but +# not this one. +[dependencies] +firefly-starter-experience = { version = "26.6.28" } +axum = "0.7" +serde = { version = "1", features = ["derive"] } +serde_json = "1" +tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "signal"] } +uuid = { version = "1", features = ["v4"] } +``` -`ExperienceStack::new(cfg)` builds a full `WebStack` (so it inherits every web -battery from the [production chapter](./20-production.md) — CORS, security -headers, request metrics, access log, correlation, idempotency, the actuator -surface) and adds the four experience-tier building blocks: +With the dependency in place, one call builds the whole stack: ```rust,ignore use firefly_starter_experience::{CoreConfig, ExperienceStack}; @@ -61,7 +131,17 @@ let bff = ExperienceStack::new(CoreConfig { }); ``` -The stack adds five experience-tier public fields on top of the embedded `web: WebStack` (`Bff` is an alias for `ExperienceStack`): +What just happened: `ExperienceStack::new(cfg)` builds a full `WebStack` +underneath — so the BFF inherits every web battery from the +[production chapter](./20-production.md): CORS, security headers, request +metrics, the access log, correlation-id propagation, idempotency, and the +actuator surface — and then layers the five experience-tier building blocks on +top. The `CoreConfig` is the same typed configuration value the rest of the book +has used; the experience starter stamps `starter_name` to `"starter-experience"` +when you leave it at its default, so the banner and `/actuator/info` report the +tier. + +The five experience-tier fields sit on top of the embedded `web: WebStack`: | Field | Type | Role | |-------|------|------| @@ -71,67 +151,116 @@ The stack adds five experience-tier public fields on top of the embedded `web: W | `query` | `Arc` | the journey-status query surface | | `children` | `Arc` | child-workflow composition for nested journeys | -`ExperienceStack` `Deref`s to its `WebStack` (which derefs to `Core`), so every -web + core method — `apply_middleware`, `actuator_router`, `new_application`, -`with_security`, `cache`, `bus`, `scheduler` — is reachable directly on the BFF -value. These are the lower-level building blocks `FireflyApplication` drives for -you on a plain domain service like Lumen; a BFF reaches for them directly because -its router is assembled by hand from workflow controllers rather than -auto-mounted. There is also a bootstrap pair, `register_experience_stack(cfg)` -(== `ExperienceStack::new`) and `enable_experience_stack(cfg)` (stamps the tier -defaults onto a `CoreConfig`), so you can reach the tier through whichever -spelling reads more naturally at the call site. - -> **`FireflyApplication` is the primary bootstrap.** For an ordinary domain or -> experience service, the turnkey path is `FireflyApplication::new(name).run().await` -> — it component-scans beans, auto-mounts controllers, drains the inventory-registered -> handlers/listeners/scheduled tasks, applies security + middleware, self-hosts the -> admin dashboard, and serves both ports. The `ExperienceStack` methods below -> (`apply_middleware`, `with_security`, `new_application`) are the explicit -> building blocks you compose by hand when a BFF's signal-gated journey controllers -> need wiring the auto-mount path does not cover; they remain fully supported. - -> **Two ways to build the stack.** `ExperienceStack::new(cfg)` and the equivalent -> `register_experience_stack(cfg)` both build the BFF; `enable_experience_stack(cfg)` -> stamps the tier defaults onto a `CoreConfig`. Because `ExperienceStack` derefs to -> `WebStack` (which derefs to `Core`), the BFF *is* a web service plus the journey -> machinery. - -## Composing domain SDKs — `DomainClients` - -A BFF reaches its downstream domain services through named `RestClient`s, one per -dependency. `DomainClients` is the registry; register Lumen under a logical name -and resolve it by that name from any handler or workflow step: +`ExperienceStack` `Deref`s to its `WebStack` (which in turn derefs to `Core`), so +every web + core method and field — `apply_middleware`, `actuator_router`, +`new_application`, `with_security`, and the `cache`, `bus`, `scheduler` fields — +is reachable directly on the `bff` value. `Bff` is a type alias for +`ExperienceStack`, so you can spell the type whichever way reads better at the +call site. + +> **Note** There are two more spellings for the same wiring, kept for services +> migrating from other Firefly ports. `register_experience_stack(cfg)` is an +> alias for `ExperienceStack::new(cfg)`, and `enable_experience_stack(cfg)` takes +> a `CoreConfig`, stamps the tier's defaults onto it (inheriting the web +> batteries), and hands it back — you then pass that to `ExperienceStack::new`. +> Reach for whichever reads most naturally; they wire the identical stack. + +> **Design note.** On a plain domain service like Lumen, +> `FireflyApplication::new(name).run().await` is the turnkey path — it +> component-scans beans, auto-mounts controllers, drains the inventory-registered +> handlers and listeners, applies security and middleware, self-hosts the admin +> dashboard, and serves both ports. A BFF reaches for the lower-level building +> blocks (`apply_middleware`, `with_security`, `new_application`) directly because +> its router is assembled by hand from signal-gated journey controllers rather +> than auto-mounted. Those methods are the same ones `FireflyApplication` drives +> for you under the hood, and they remain fully supported. + +> **Tip** **Checkpoint.** `ExperienceStack::new(...)` returns a value on which +> `bff.app_name` reports your app name and `bff.starter_name` is +> `"starter-experience"`. `bff.clients.is_empty()`, `bff.signals.list_active()`, +> and `bff.query.active()` are all empty — the building blocks are wired and +> waiting. + +## Step 3 — Register Lumen as a domain SDK + +A BFF reaches each downstream domain service through a named REST client — one +per dependency. `DomainClients` is the registry: you register Lumen under a +logical name, then resolve it by that name from any handler or workflow step, +without threading a builder through every call site. + +> **Note** **Key term — `RestClient`.** The `RestClient` is `firefly-client`'s +> HTTP client. It carries correlation-id propagation, a JSON codec, RFC 9457 +> `application/problem+json` error decoding (a non-2xx response becomes a typed +> error), and retry/backoff — the same client the +> [HTTP-clients chapter](./13-http-clients.md) covered. `register` hands you back +> an `Arc` for immediate use. ```rust,ignore // Experience -> Domain only. Register Lumen as a downstream domain SDK. bff.clients.register("wallets", "https://lumen.internal"); -// later, in a handler or workflow step: +// Later, in a handler or workflow step, resolve it by its logical name: let wallets = bff.clients.get("wallets").expect("wallets SDK"); -// wallets is an Arc with correlation-id propagation, JSON codec, -// RFC 7807 error decoding, and retry/backoff all inherited from firefly-client. +// `wallets` is an Arc with correlation-id propagation, a JSON codec, +// RFC 9457 problem decoding, and retry/backoff — all inherited from firefly-client. ``` -`register(name, base_url)` builds a default client; `register_client(name, -client)` takes a pre-tuned `RestClient` (custom timeout, headers, retry policy). -Re-registering a name replaces it (last wins), and `names()` lists every -registered SDK. +What just happened: `register(name, base_url)` builds a default `RestClient` for +that base URL, stores it under the logical name, and returns the `Arc` +it built. `get(name)` resolves it later, returning `None` when nothing is +registered under that name. Because every registered client points at a domain +service, this registry is exactly where the "experience → domain only" rule lives. + +Once you hold a client, you call the downstream API through `request`: + +```rust,ignore +use http::Method; +use serde_json::json; + +// Call Lumen's public API as any external client would. A non-2xx response +// decodes into a typed ClientError (RFC 9457 problem document). +let _: serde_json::Value = wallets + .request(Method::POST, "/wallets/w-1/reserve", Some(&json!({ "amount": 5000 }))) + .await?; +``` + +The registry has a small, predictable surface: + +- `register(name, base_url)` builds and stores a default client (last write wins + if the name already exists). +- `register_client(name, client)` stores a pre-tuned `RestClient` — use it when a + domain SDK needs a custom timeout, default headers, or retry policy. +- `get(name)` resolves a client, or `None`. +- `names()` lists every registered SDK (sorted), and `len()` / `is_empty()` + report the registry size. -> **Resolve clients by name.** `DomainClients` lets a step resolve the right -> downstream client by logical name instead of threading a `RestBuilder` through -> every call site. The clients carry the same correlation propagation and RFC 7807 -> decoding the [HTTP-clients chapter](./13-http-clients.md) covered. +> **Design note.** Resolving a client by logical name — `"wallets"` rather than a +> hard-coded URL — is what keeps the BFF's journey code decoupled from where Lumen +> actually runs. Point the name at `https://lumen.internal` in production and at +> `http://localhost:8080` in a test, and not one line of the workflow changes. -## Signal-driven journeys +> **Tip** **Checkpoint.** After `bff.clients.register("wallets", ...)`, +> `bff.clients.get("wallets")` returns `Some(_)`, `bff.clients.names()` is +> `["wallets"]`, and `bff.clients.len()` is `1`. Resolving an unregistered name +> returns `None`, never a panic. + +## Step 4 — Model the journey as a signal-gated workflow A BFF journey is rarely one request. "Fund a wallet, wait for the customer to -confirm the amount, then commit the transfer" is three interactions over time. A -**workflow** with a **signal gate** models exactly that: steps that call domain -SDKs, and a gate that parks until an atomic endpoint delivers a named signal. +confirm the amount, then commit the transfer" is three interactions spread over +time. A **workflow** with a **signal gate** models exactly that: steps that call +domain SDKs, and a gate that parks until an external caller delivers a named +signal. + +> **Note** **Key term — workflow and node.** A `Workflow` is a directed graph of +> `Node`s, each an async step, executed in dependency order. It is the same +> DAG-with-compensation engine from the [sagas chapter](./12-sagas.md) +> (`firefly-orchestration`) — here driven by signals rather than run to +> completion in one call. `Node::new(name, action)` defines a step; +> `.depends_on([...])` declares which steps must finish first. -The journey is a `Workflow` of `Node`s; `Node::wait_for_signal` parks on the -stack's `SignalService` until the signal arrives: +The journey is a `Workflow` of `Node`s; `Node::wait_for_signal` builds the gate +node, parking on the stack's `SignalService` until the named signal arrives: ```rust,ignore use std::sync::Arc; @@ -152,30 +281,75 @@ let workflow = Workflow::new("fund-and-confirm") .node(Node::new("commit", || async { Ok(()) }).depends_on(["await-confirm"])); ``` -`Workflow::run().await` executes the nodes in dependency order; when it reaches -`await-confirm` it parks. An atomic endpoint later calls -`bff.signals.deliver(&journey_id, "confirmed", payload)` and the parked node -resumes. Delivery is **buffered** — if the signal arrives before the gate parks, -there is no lost wakeup. `signals.list_active()` lists the journeys currently -parked on a gate. +What just happened, node by node: + +- `Node::new("reserve", || async { Ok(()) })` is the first step. In a real BFF its + body resolves `bff.clients.get("wallets")` and calls Lumen's reserve endpoint; + here the body is a stub that returns `Ok(())` so the shape is clear. A node + action returns `Result<(), BoxError>`. +- `Node::wait_for_signal("await-confirm", &signals, journey_id.clone(), + "confirmed")` builds the **gate** node. It takes the node name, a reference to + the stack's `Arc`, the journey's correlation id, and the signal + name to wait for. `.depends_on(["reserve"])` makes it run after `reserve`. +- `Node::new("commit", ...).depends_on(["await-confirm"])` is the final step, + which runs only once the gate releases. + +`workflow.run().await` executes the nodes in dependency order and returns +`Result<(), WorkflowError>`. When the run reaches `await-confirm` it **parks** — +the future suspends inside the gate node and does not progress. You typically +spawn the run on a task so the HTTP handler that started it can return +immediately: + +```rust,ignore +// Run the journey on a task; it parks on the `await-confirm` gate. +tokio::spawn(async move { + let _ = workflow.run().await; +}); +``` -> **Signal gates.** `Node::wait_for_signal(...)` parks a workflow node until -> `signals.deliver(...)` delivers the named signal — the external event that -> satisfies the gate. Delivery is buffered, so a signal that arrives before the -> gate parks is not lost. The workflow is the same DAG-with-compensation engine -> from the [sagas chapter](./12-sagas.md) (`firefly-orchestration`), here driven -> by signals rather than run to completion in one call. +Later, an atomic endpoint delivers the signal and the parked node resumes: -## Persisting journey state — `WorkflowState` +```rust,ignore +// From a later request (POST /journeys/{id}/data): +bff.signals.deliver(&journey_id, "confirmed", serde_json::json!({ "ok": true })); +``` -Because a journey spans several requests, its state must outlive any one of them. -`WorkflowState` round-trips a workflow run's `StepContext` snapshot through the -stack's cache `Adapter`, keyed by correlation id. The in-memory adapter is the -default; point it at `firefly-cache-redis`'s `RedisAdapter` for cross-restart -durability — the convention the experience tier is built around. +> **Note** **Key term — signal delivery (buffered).** `signals.deliver(id, +> signal, payload)` wakes the parked gate. Delivery is **buffered**: if the +> signal arrives *before* the gate has parked, the payload is held and the next +> `wait_for_signal` for that pair resolves immediately — so there is no lost +> wakeup in a race. `deliver` returns `true` when a live waiter consumed the +> signal and `false` when it was buffered (do not treat `false` as an error). +> `signals.list_active()` lists every journey currently parked on, or holding a +> buffered signal for, a gate. + +> **Tip** **Checkpoint.** Spawn `workflow.run()`, then poll +> `bff.signals.list_active()` — once the run reaches the gate it contains +> `journey_id`. Call `bff.signals.deliver(&journey_id, "confirmed", payload)` and +> the workflow completes; `list_active()` no longer lists the id. + +## Step 5 — Persist the journey with `WorkflowState` + +A journey spans several requests, so its state must outlive any single one. If +the customer closes the tab between "reserve" and "confirm," a purely in-memory +waiter would be lost. `WorkflowState` solves this by round-tripping a workflow +run's `StepContext` snapshot through the stack's cache `Adapter`, keyed by +correlation id. + +> **Note** **Key term — `StepContext`.** A `StepContext` is the per-run bag of +> facts a workflow carries: its correlation id, the inputs, each step's result, +> and free-form variables. It serializes to and from a JSON snapshot, which is +> what `WorkflowState` stores. It lives in `firefly-orchestration`, so you import +> it from there. + +> **Note** **Key term — cache `Adapter`.** The cache `Adapter` is Firefly's +> pluggable key/value backend. The in-memory adapter is the default; point it at +> `firefly-cache-redis`'s `RedisAdapter` for cross-restart durability — the +> convention the experience tier is built around. `ExperienceStack` wires +> `state` over the same adapter the `Core` holds, so swapping in Redis is a config +> change, not a code change. ```rust,ignore -use firefly_starter_experience::CoreConfig; use firefly_orchestration::StepContext; // Save when a journey parks: @@ -186,137 +360,224 @@ bff.state.save(&ctx).await?; // Rehydrate from a later request to advance it: if let Some(ctx) = bff.state.load("j-1").await? { - // ... advance the journey ... + // ... advance the journey using ctx ... + let _ = ctx; } // Discard when the journey completes: bff.state.delete("j-1").await?; ``` -A miss on an unknown journey is `Ok(None)`, not an error — so a status check on a -journey that never existed is a clean 404, not a 500. +What just happened: -> **Durable journey state.** `WorkflowState` persists a workflow run's -> `StepContext` through the **cache** `Adapter`, keyed by correlation id — point -> it at Redis for cross-restart durability, the Firefly convention of "hold -> workflow state in Redis." A parked journey saves its `StepContext`; a later -> request loads it and resumes, surviving the client disconnect an in-memory -> waiter would not. +- `StepContext::new()` makes an empty context; `set_correlation_id` keys it (this + is the journey id `WorkflowState` stores under), and `set_variable("phase", …)` + records where the journey is. You read it back with `ctx.variable("phase")`. +- `bff.state.save(&ctx).await?` persists the snapshot under the context's + correlation id, returning `Result<(), CacheError>`. +- `bff.state.load("j-1").await?` returns `Result, + CacheError>`. A **miss on an unknown journey is `Ok(None)`, not an error** — so a + status check on a journey that never existed renders as a clean 404, never a + 500. +- `bff.state.delete("j-1").await?` evicts the state when the journey finishes, so + completed runs do not linger. -## Querying journey status — `WorkflowQueryService` +> **Design note.** This is the seam that makes a BFF resilient to client +> disconnects. A parked journey saves its `StepContext` before suspending; a later +> request — possibly from a fresh browser session, possibly after the BFF +> restarted (with the Redis adapter) — loads it back and resumes. The in-memory +> waiter alone could not survive that gap; the persisted state can. -The frontend polls "where is my journey?" while it waits. `WorkflowQueryService` -holds the live `StepContext` per run and answers named queries against it — the -main recovery mechanism when a client reconnects: +> **Tip** **Checkpoint.** `save` a `StepContext` carrying a `phase` variable, +> then `load` it from a fresh handle: the variable survives the round-trip. +> `delete` it, and `load` returns `Ok(None)`. + +## Step 6 — Answer status polls with `WorkflowQueryService` + +While the customer waits at the confirm screen, the frontend polls "where is my +journey?" `WorkflowQueryService` holds the live `StepContext` per run and answers +*named* queries against it — the main recovery mechanism when a client reconnects. ```rust,ignore -bff.query.register(&journey_id, ctx.clone()); // on start -bff.query.register_query(&journey_id, "phase", |ctx| { // a named query +let journey_id = "j-1".to_string(); + +// On start: register the run's live context. +bff.query.register(&journey_id, ctx.clone()); + +// Register a named query that projects a value out of the context. +bff.query.register_query(&journey_id, "phase", |ctx| { ctx.variable("phase").unwrap_or(serde_json::json!("UNKNOWN")) }); -let phase = bff.query.query(&journey_id, "phase")?; // GET /journeys/{id} -bff.query.unregister(&journey_id); // on completion -``` -## The atomic-endpoint contract +// On GET /journeys/{id}: run the named query. +let phase = bff.query.query(&journey_id, "phase")?; + +// On completion: drop the run. +bff.query.unregister(&journey_id); +``` -Put together, an experience controller exposes one request per journey phase — -the **atomic REST** shape: +What just happened: `register(id, ctx)` enrolls a run by correlation id with its +live `StepContext`. `register_query(id, name, |ctx| value)` attaches a named +projection — a closure that derives a JSON value from the context (here, the +`phase` variable). `query(id, name)` runs that projection and returns +`Result` — an unknown run or an unknown query name is a +typed error, which the controller maps to a 404. `unregister(id)` removes the run +when the journey ends; `active()` lists every registered run. + +> **Design note.** Two surfaces answer "where is my journey?" and they +> complement each other. `WorkflowState` (Step 5) is the *durable* record — it +> survives a restart and backs the 404-or-phase decision. `WorkflowQueryService` +> is the *live* projection over the in-process run — richer, cheaper to query, and +> the natural place to derive a "next step" DTO while the process is up. A +> production status endpoint reads the live query when the run is in memory and +> falls back to the persisted state otherwise. + +> **Tip** **Checkpoint.** `register` a run, `register_query` a `"phase"` +> projection, and `query(id, "phase")` returns the phase value. Querying an +> unregistered id or an unknown query name returns an `Err`, not a panic. + +## Step 7 — Assemble the atomic-endpoint controller + +Put the pieces together and an experience controller exposes one HTTP request per +journey phase — the **atomic REST** shape. + +> **Note** **Key term — atomic endpoint.** An *atomic endpoint* performs exactly +> one phase of a journey and returns. State lives in the cache (Redis-capable) +> between calls, so the client drives the journey one request at a time and can +> resume after a disconnect — instead of holding one long-lived connection open +> across the whole flow. | Method & path | Does | |---------------|------| -| `POST /journeys` | start the workflow (calls the "wallets" SDK to reserve), persist `WorkflowState`, park on the gate, return the journey id | -| `POST /journeys/:id/data` | deliver the `confirmed` signal — the parked workflow resumes and commits the transfer via the "wallets" SDK | -| `GET /journeys/:id` | report the persisted phase (or 404 if unknown) | - -Each phase is one HTTP request; state lives in the cache (Redis-capable), so a -client can resume a journey after a disconnect. Because the controller runs -through `bff.apply_middleware(routes)`, every response inherits the web batteries -— the start response carries `X-Frame-Options: DENY` and a correlation id just as -Lumen's own responses do, and the same `ExperienceStack::with_security(chain)` -filter chain from the [security chapter](./14-security.md) guards the mutating -routes. - -This is exactly the contract the crate's own boot test proves end to end (its +| `POST /journeys` | start the workflow (calls the `"wallets"` SDK to reserve), persist `WorkflowState`, park on the gate, return the journey id | +| `POST /journeys/:id/data` | deliver the `confirmed` signal — the parked workflow resumes and commits the transfer via the `"wallets"` SDK | +| `GET /journeys/:id` | report the persisted phase (or 404 if the journey is unknown) | + +You build these as ordinary `axum` routes, then run the router through the BFF's +inherited middleware so every response carries the web batteries: + +```rust,ignore +use axum::routing::{get, post}; +use axum::Router; + +// `routes` is an axum Router with the three handlers and the BFF as state. +let routes = Router::new() + .route("/journeys", post(start_journey)) + .route("/journeys/:id/data", post(deliver_journey_signal)) + .route("/journeys/:id", get(journey_status)) + .with_state(app_state); + +// Inherit the web batteries: CORS, security headers, correlation, metrics, … +let api = bff.apply_middleware(routes); +``` + +What just happened: each phase is one HTTP request, and the workflow state lives +in the cache between them, so a client can resume the journey after a disconnect. +Because the router runs through `bff.apply_middleware(routes)`, every response +inherits the web batteries — the start response carries `X-Frame-Options: DENY` +and an `X-Correlation-Id` just as Lumen's own responses do — and the same +`ExperienceStack::with_security(chain)` filter chain from the +[security chapter](./14-security.md) guards the mutating routes. + +This is exactly the contract the crate's own boot test proves end to end. (Its checkout journey uses an `AWAITING_PAYMENT` phase rather than Lumen's -`AWAITING_CONFIRM`, but the shape is identical): two mock domain SDKs composed -through a signal-gated workflow, driven with `tower::oneshot` — start parks on -the gate, the status endpoint reports the persisted phase, delivering the signal -advances the workflow off-task, and the final status flips to `COMPLETED`. - -## Where Lumen fits - -Drawn as the full estate, Lumen is one of the domain services a BFF composes: - -
- - - web / mobile app - channel tier - - - - - lumen-bff - (experience) - this chapter: - DomainClients + signals + state - - - - Experience → Domain only - - lumen - (domain) - the service the book built - (ledger, CQRS, saga) - - - - - accounts - (core) - owns the database - -
Lumen as one service in the estate. Calls flow strictly downward; the BFF on the experience tier composes Lumen on the domain tier, which owns its logic, while the core service owns the database.
-
- -The BFF never reaches into Lumen's event store or bus — it speaks to Lumen's -public HTTP API through the registered `"wallets"` SDK, exactly as any external -client would, and adds only the journey orchestration a frontend needs. - -## What changed in Lumen - -Lumen itself is unchanged — it is a *domain* service, and this chapter is about -the tier *above* it. What you built is the mental model and the wiring for a -frontend-facing BFF: an `ExperienceStack` inheriting Lumen's web batteries, -`DomainClients` registering Lumen as the `"wallets"` SDK (Experience → Domain -only), a signal-gated `Workflow` modeling a multi-request "fund and confirm" -journey, `WorkflowState` persisting that journey through a Redis-capable cache -adapter, and `WorkflowQueryService` answering status polls — every API drawn from -the real `firefly-starter-experience` surface. +`AWAITING_CONFIRM`, but the shape is identical.) Two mock domain SDKs are composed +through a signal-gated workflow and driven with `tower::oneshot`: starting the +journey reserves through the first SDK and parks on the gate; the status endpoint +reports the persisted phase; delivering the signal advances the workflow off-task +and ships through the second SDK; and the final status flips to `COMPLETED`. + +> **Tip** **Checkpoint.** Driving the router with three calls — `POST /journeys`, +> then `GET /journeys/:id` (reports `AWAITING_CONFIRM`), then `POST +> /journeys/:id/data`, then `GET /journeys/:id` again (reports `COMPLETED`) — +> walks the whole journey. The first response carries the inherited +> `X-Frame-Options: DENY` header. + +## Step 8 — See where Lumen fits + +Drawn as the full estate, Lumen is one of the domain services a BFF composes. The +channel tier (a web or mobile app) calls the experience tier (`lumen-bff`), which +calls the domain tier (`lumen`), which calls the core tier (`accounts`, which +owns the database). Every arrow points strictly downward, and the experience tier +only ever reaches the domain tier: + +```text + web / mobile app (channel tier) + │ + ▼ Experience → Domain only + lumen-bff (experience: DomainClients + signals + state) + │ + ▼ + lumen (domain: ledger, CQRS, transfer saga) + │ + ▼ + accounts (core: owns the database) +``` + +What just happened: you placed your BFF in the estate. The BFF never reaches into +Lumen's event store or its CQRS bus — it speaks to Lumen's public HTTP API +through the registered `"wallets"` SDK, exactly as any external client would, and +adds only the journey orchestration a frontend needs. Lumen, the domain service, +owns its own logic; the core service below it owns the database. + +## Recap — what this chapter built + +You did not change Lumen — it is a *domain* service, and this chapter is about the +tier *above* it. What you built is the mental model and the wiring for a +frontend-facing BFF: + +- The `channel → experience → domain → core` tier model, and why the experience + tier composes *domain* SDKs only — never a database, a core service, or a + sibling BFF. +- An `ExperienceStack` (`Bff`) that inherits Lumen's web batteries and adds five + building blocks: `clients`, `signals`, `state`, `query`, and `children`. +- `DomainClients` registering Lumen as the `"wallets"` SDK and resolving it by + logical name, over a `RestClient` with correlation propagation and RFC 9457 + problem decoding. +- A signal-gated `Workflow` whose `Node::wait_for_signal` gate parks the + "fund and confirm" journey until `signals.deliver(...)` arrives — with buffered + delivery so there is no lost wakeup. +- `WorkflowState` persisting that journey through a Redis-capable cache adapter + (a miss is `Ok(None)`, not an error), so a client can resume after a disconnect. +- `WorkflowQueryService` answering "where is my journey?" status polls from the + live `StepContext`. +- The three-endpoint atomic-REST controller, run through + `bff.apply_middleware(...)` so every response inherits the web batteries. + +Every API here is drawn from the real `firefly-starter-experience` surface — the +same one its boot test exercises end to end. ## Exercises 1. **Register Lumen as a domain SDK.** Build an `ExperienceStack`, call `bff.clients.register("wallets", "http://localhost:8080")`, and confirm - `bff.clients.get("wallets")` returns a client and `bff.clients.names()` - lists it. -2. **Park and resume.** Build a two-node workflow whose second node is a - `Node::wait_for_signal` gate. Run it on a task, assert - `bff.signals.list_active()` contains the journey id, then `deliver` the signal - and confirm the workflow completes. -3. **Persist a journey.** Save a `StepContext` with a `phase` variable via + `bff.clients.get("wallets")` returns `Some(_)` and `bff.clients.names()` lists + `"wallets"`. Then re-register the same name with a different URL and confirm + `bff.clients.len()` stays `1` (last write wins). +2. **Park and resume.** Build a two-node `Workflow` whose second node is a + `Node::wait_for_signal` gate. Spawn `workflow.run()` on a task, poll until + `bff.signals.list_active()` contains the journey id, then + `bff.signals.deliver(&id, "confirmed", json!({}))` and confirm the workflow + completes and `list_active()` no longer lists the id. +3. **Race the gate.** Repeat exercise 2 but call `deliver` *before* spawning the + run. Confirm the workflow still completes — buffered delivery means the signal + is not lost when it beats the gate. +4. **Persist a journey.** `save` a `StepContext` carrying a `phase` variable via `bff.state.save`, `load` it back from a fresh handle, and confirm the variable - survives. Then `delete` it and confirm `load` returns `Ok(None)`. -4. **Atomic endpoints.** Wire the three-route controller above with - `bff.apply_middleware(routes)` and drive it with `tower::oneshot`: start, - poll status (`AWAITING_CONFIRM`), confirm, poll again (`COMPLETED`). Assert the - start response carries the inherited `X-Frame-Options` header. - -With Lumen in production (the previous chapter) and its place in the tiered -estate clear, the capstone re-reads the whole service through the -declarative-macro lens. Continue to -[Declarative Services with Macros](./21-declarative-macros.md). + survives. Then `delete` it and confirm `bff.state.load(...)` returns `Ok(None)`. +5. **Atomic endpoints.** Wire the three-route controller from Step 7 with + `bff.apply_middleware(routes)` and drive it with `tower::oneshot`: start, poll + status (`AWAITING_CONFIRM`), deliver the signal, poll again (`COMPLETED`). + Assert the start response carries the inherited `X-Frame-Options: DENY` header + and an `X-Correlation-Id`. + +## Where to go next + +- Revisit how a domain service like Lumen is taken to production — real Postgres, + Kafka, and the management surface — in + **[Production & Deployment](./20-production.md)**. +- See how the workflow engine the BFF's journey rides on also powers Lumen's own + compensating transfers in **[Sagas, Workflows & TCC](./12-sagas.md)**. +- With Lumen's place in the tiered estate now clear, the capstone re-reads the + whole service through the declarative-macro lens. Continue to + **[Declarative Services with Macros](./21-declarative-macros.md)**. diff --git a/docs/book/src/21-declarative-macros.md b/docs/book/src/21-declarative-macros.md index 184490b..8faf35e 100644 --- a/docs/book/src/21-declarative-macros.md +++ b/docs/book/src/21-declarative-macros.md @@ -1,51 +1,135 @@ # Declarative Services with Macros Lumen is finished. Over twenty chapters it grew from an empty scaffold into a -secure, observable, event-sourced CQRS service with a transfer saga and a -streaming endpoint — and it depends on exactly **one** Firefly crate. This -capstone re-reads the whole service through a single lens: the **declarative -macros**. By the end of this chapter you will be able to point at every +secure, observable, event-sourced CQRS service with a transfer saga, a +compliance workflow, a two-phase transfer, a scheduled heartbeat, and an +optional streaming endpoint — and it depends on exactly **one** Firefly crate. +This capstone re-reads the whole service through a single lens: the +**declarative macros**. By the end you will be able to point at every `#[derive(...)]` and `#[...]` in `samples/lumen` and say precisely what wiring it collapsed into a declaration next to the code. That is the thesis the running -crate proves: *one facade + macros = the framework, with the boilerplate gone.* - -> **Compile-time, not reflective.** Firefly's declarative layer generates wiring -> at compile time with `proc-macro`s — there is no startup scanning cost and no -> reflective surprises. A declaration sits next to the code it describes, and the -> macro emits the `impl`s, routers, and helper functions you would otherwise -> hand-write. If you have used a batteries-included framework before, the shape -> will feel familiar; the difference is that the glue is generated and checked by -> the compiler rather than discovered at startup. - -## One dependency, one prelude - -Every chapter began the same way. Lumen's `Cargo.toml` lists one Firefly crate: +crate proves: *one facade plus macros equals the framework, with the boilerplate +gone.* + +This chapter does not introduce a new feature. It is a guided walk-through of the +declarative layer you have been using all along, slowed down so every macro is +explained from first principles before it is read in context. Where a macro is +exercised in `samples/lumen` we read the verbatim Lumen source; where it is a +first-class part of the framework that Lumen happens not to use, we read a +focused standalone example and say so. + +By the end of this chapter you will: + +- Explain what a *declarative macro* is in Firefly, and why the generated wiring + is checked by the compiler rather than discovered by runtime reflection. +- Trace each macro Lumen uses — `#[derive(Command/Query)]`, `#[handlers]`, + `#[derive(DomainEvent/AggregateRoot)]`, `#[event_listener]`, + `#[rest_controller]`, `#[scheduled]`, `#[firefly::saga/workflow/tcc]` — to the + exact `impl`, router, or registration it emits. +- Name the supporting declarative set Lumen does not use — `#[derive(Builder)]`, + `#[derive(Mapper)]`, `#[derive(Entity)]` / `#[derive(SqlxRepository)]` / + `#[firefly::repository]` / `#[firefly::transactional]`, the method-security and + resilience decorators, `#[cacheable]`, and the rest — and read a correct + example of each. +- Describe the hidden `__rt` contract path that lets a one-crate service compile + whatever a macro expands to. +- Explain the `inventory` drain: how a *declared* bean, listener, task, or + controller becomes *wired* at boot with no hand-written registration call. +- Verify the entire crate builds, tests, and lints clean from the workspace root. + +## Concepts you will meet + +Before the catalogue, here are the four ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — declarative macro.** A *declarative macro* is an +> attribute (`#[...]`) or derive (`#[derive(...)]`) that a `proc-macro` expands +> at **compile time** into the `impl`s, routers, and helper functions you would +> otherwise hand-write. The declaration sits next to the code it describes; the +> compiler checks the generated code like any other source. The Spring analog is +> an annotation (`@RestController`, `@Component`) — except Spring discovers and +> processes annotations by reflection at startup, while Firefly resolves them at +> compile time. + +> **Note** **Key term — facade and prelude.** The *facade* is the single +> `firefly` crate that re-exports the whole framework and every macro; the +> *prelude* is `firefly::prelude`, a module of the high-frequency types you glob +> in with `use firefly::prelude::*;`. Depending on one facade and importing one +> prelude is the entire "one dependency, one import" story. The Spring analog is +> a single Spring Boot starter plus the auto-imported framework types. + +> **Note** **Key term — bean.** A *bean* is an object the framework constructs, +> manages, and hands to whoever declares it needs it (with `#[autowired]`). You +> declare beans; the framework discovers them at startup and wires them together. +> This is exactly Spring's notion of a bean managed by the application context. + +> **Note** **Key term — inventory registry.** `inventory` is a Rust crate that +> lets a macro register a value into a process-global table **at link time** — +> before `main` runs. Each declarative macro that produces a handler, listener, +> task, or controller submits a *registration* into one of these tables; +> `FireflyApplication` **drains** the tables at boot and installs each entry. The +> effect mirrors Spring's classpath component scan, but the inventory is built by +> the linker, not by walking the classpath at runtime. + +## Step 1 — One dependency, one prelude + +Every chapter began the same way, so start there. Open Lumen's `Cargo.toml`. It +lists exactly one Firefly crate; everything declarative arrives through it. ```toml +# samples/lumen/Cargo.toml [dependencies] -firefly = { workspace = true } # the whole framework + every macro -axum = { workspace = true } # you author the controller handlers -serde = { workspace = true } # your messages + event payloads -serde_json = { workspace = true } -tokio = { workspace = true } -uuid = { workspace = true } -chrono = { workspace = true } -async-trait = { workspace = true } +# The one-dependency story: the `firefly` facade re-exports the whole framework +# AND every `#[derive(...)]` / `#[...]` macro. Generated code resolves runtime +# types through the facade, so Lumen never lists the underlying `firefly-*` +# crates. The `admin` feature pulls in the self-hosted admin dashboard. +firefly = { version = "26.6.28", features = ["admin"] } + +# The two ecosystem crates a Firefly service still writes against directly: +# axum (you author the controller handlers) and serde (your messages and event +# payloads are Serialize/Deserialize). `serde_json` encodes the event payloads. +axum = "0.7" +serde = { version = "1", features = ["derive"] } +serde_json = "1" + +# Async runtime for `#[tokio::main]`, and the id/clock crates the domain uses. +tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "signal"] } +uuid = { version = "1", features = ["v4"] } +chrono = "0.4" +async-trait = "0.1" + +[features] +# The reactive streaming endpoint is feature-gated so the teaching baseline +# stays lean; the production chapter turns it on. It needs nothing beyond the +# `firefly` facade. +default = [] +streaming = [] ``` -and every module opens with one glob: +And every module opens with one glob import: ```rust,ignore use firefly::prelude::*; ``` -That glob brings the whole framework into scope — `Bus`, `Container`, -`Scheduler`, `Saga`/`Step`, `Application`/`ShutdownHandle`, `Core`/`CoreConfig`, -`WebResult`/`WebError`, `FireflyError`/`FireflyResult`, `Mono`/`Flux` — **and** -every macro. There are also per-crate aliases (`firefly::cqrs::Bus`, -`firefly::eventsourcing::EventStore`, `firefly::security::JwtService`, …) for the -types you name explicitly. `axum` and `serde` are the only two ecosystem crates -Lumen writes against directly. +What just happened: that glob brings the whole high-frequency surface into scope +— `Bus`, `Container`, `Scheduler`, `Saga` / `Step`, `Application` / +`ShutdownHandle`, `Core` / `CoreConfig`, `WebResult` / `WebError`, `FireflyError` +/ `FireflyResult`, `Mono` / `Flux` — **and** every macro. For the types you name +explicitly there are per-crate aliases (`firefly::cqrs::Bus`, +`firefly::eventsourcing::EventStore`, `firefly::security::JwtService`, …), which +is why several Lumen modules also write `use firefly::cqrs::QueryCache;` or +`use firefly::eda::{Broker, Event};` next to the prelude glob. `axum` and `serde` +are the only two ecosystem crates Lumen writes against directly. + +> **Note** A `proc-macro` crate cannot itself re-export runtime types, so +> macro-generated code references every runtime type through the facade's hidden +> `__rt` contract path — for example `::firefly::__rt::firefly_cqrs::Bus`. That +> is why Lumen, depending only on `firefly`, compiles whatever a macro expands to +> without ever listing the underlying `firefly-*` crates. You never write `__rt` +> yourself; if you rename or shim the facade, pass `#[firefly(crate = +> "my_firefly")]` to any macro to override the leading segment. We return to this +> contract in [Step 11](#step-11--how-the-wiring-actually-lands-the-__rt-contract-and-the-inventory-drain). ### Staying lean @@ -59,80 +143,71 @@ adapters are opt-in cargo features (the swap path every chapter pointed at): | `data-mongodb` | document repository adapter (MongoDB) | | `eda-kafka` / `eda-rabbitmq` / `eda-redis` / `eda-postgres` | event-broker transports | | `cache-redis` / `cache-postgres` | cache backends | -| `admin` | the admin dashboard | +| `admin` | the self-hosted admin dashboard | | `full` | all of the above | -## The macros Lumen uses +> **Tip** **Checkpoint.** Open `samples/lumen/Cargo.toml` and confirm there is +> exactly one `firefly = { … }` line under `[dependencies]`, and that every +> source file under `samples/lumen/src/` opens with `use firefly::prelude::*;`. +> That one dependency and one import are the whole premise this chapter unpacks. -Lumen exercises the full declarative set. Here is the catalogue, each mapped to -the exact Lumen file it lands in: +## Step 2 — The macro catalogue, mapped to Lumen files + +Lumen exercises the core declarative set. Before reading any one macro in depth, +here is the map: each macro, the exact Lumen file it lands in, and what it +generates. | Macro | Lumen file | Generates | |-------|-----------|-----------| | `#[derive(Command)]` / `#[derive(Query)]` | `commands.rs` | the `Message` impl (`#[firefly(validate)]`, `#[firefly(cache_ttl = "…")]`) | -| `#[handlers]` (on the handler-bean `impl`) | `commands.rs`, `ledger.rs` | registers each `#[command_handler]` / `#[query_handler]` / `#[event_listener]` method of a DI bean on the bus / broker | +| `#[derive(Schema)]` | `commands.rs`, `domain.rs`, `web.rs`, … | an OpenAPI schema for the type, so it appears in `/v3/api-docs` | +| `#[handlers]` (on a handler-bean `impl`) | `commands.rs`, `ledger.rs` | registers each `#[command_handler]` / `#[query_handler]` / `#[event_listener]` method of a DI bean on the bus / broker | | `#[command_handler]` / `#[query_handler]` (method markers) | `commands.rs` | mark a CQRS handler method inside a `#[handlers]` impl | -| `#[derive(DomainEvent)]` | `domain.rs` | `EVENT_TYPE` + `to_domain_event` | +| `#[derive(DomainEvent)]` | `domain.rs` | `EVENT_TYPE` discriminator + `to_domain_event` conversion | | `#[derive(AggregateRoot)]` | `domain.rs` | `AGGREGATE_TYPE` + `aggregate()` / `aggregate_mut()` | +| `#[derive(Service)]` / `#[derive(Repository)]` | `commands.rs`, `ledger.rs` | a scanned `@Component` / `@Repository` bean with `#[autowired]` fields | | `#[event_listener(topic = "…")]` (method marker) | `ledger.rs` | mark an EDA listener method inside a `#[handlers]` impl (the projection bean) | -| `#[rest_controller]` + `#[get/post]` | `web.rs` | a `routes(state) -> axum::Router` | -| `#[scheduled(fixed_rate = "…")]` | `housekeeping.rs` | a `schedule_(scheduler)` helper | -| `#[firefly::saga]` + `#[saga_step]` | `transfer.rs` | `TransferSaga::saga()` + a `run`-style step graph with compensation | +| `#[derive(Configuration)]` + `#[bean]` | `web.rs` | a `@Configuration` holder whose `#[bean]` factories declare infra beans | +| `#[derive(Controller)]` + `#[rest_controller]` + `#[get/post]` | `web.rs` | an autowired controller bean and its `WalletApi::routes(state) -> axum::Router` | +| `#[scheduled(fixed_rate = "…")]` | `housekeeping.rs` | a `schedule_(scheduler)` helper plus a drained registration | +| `#[firefly::saga]` + `#[saga_step]` | `transfer.rs` | `TransferSaga::run` / `::saga()` — a step graph with compensation | | `#[firefly::workflow]` + `#[workflow_step]` | `compliance.rs` | a workflow `run` over the DAG of steps | | `#[firefly::tcc]` + `#[participant]` | `tcc_transfer.rs` | a TCC `run` driving each participant's try / confirm / cancel | -Lumen also exercises the declarative orchestration macros above -(`#[firefly::saga]` / `#[firefly::workflow]` / `#[firefly::tcc]`) in its own -source. Several more macros round out the declarative set; Lumen does not -exercise these in its own source — it is event-sourced (so the relational -`#[firefly::repository]` / `#[firefly::transactional]` never appear) and handles -the remaining cross-cutting concerns by other means — but each is a first-class -part of the framework, shown here as a focused standalone example: +The next steps read each of these in its Lumen file, in the order the crate +itself is layered. After that, [Step 10](#step-10--the-rest-of-the-declarative-set-not-used-by-lumen) +catalogues the macros Lumen does *not* exercise — because it is event-sourced and +handles its cross-cutting concerns by other means — each with a correct +standalone example. -| Macro | Purpose | Generates | -|-------|---------|-----------| -| `#[derive(Builder)]` | a fluent constructor with required/defaulted fields | `T::builder()` → fluent setters → `build() -> Result` | -| `#[derive(Mapper)]` | compile-time struct-to-struct conversion | one compile-time `From` per `#[firefly(from = "…")]` | -| `#[derive(Entity)]` | the `@Entity` mapping from annotated struct fields | a `SqlxEntity` impl (`@Table` / `@Id` / `@Version` / `@Column`); scalar fields map automatically, a non-scalar field uses `#[firefly(with(read = …, write = …))]` | -| `#[derive(SqlxRepository)]` | a fully-wired sqlx `@Repository` bean | a `@Repository` bean built from the injected `Db` (via `repository_for`), `ReactiveCrudRepository` **and** `ReactiveSpecificationRepository` (`find_by_spec`, the `JpaSpecificationExecutor` analog) impls by delegation, and the `repository()` accessor `#[firefly::repository]` uses | -| `#[firefly::repository]` | derived-query and custom-query method bodies | method bodies on a `SqlxReactiveRepository` impl from method names or `#[query(…)]` | -| `#[firefly::transactional]` | a declared transaction boundary | a commit-on-`Ok` / rollback-on-`Err` boundary around an `async fn` body | -| `#[firefly::pre_authorize]` / `#[firefly::post_authorize]` | method-level access control | an access check before the body, or a returnObject check after it | -| `#[derive(Validate)]` (+ `Valid`) | JSR-380 bean validation | `impl Validate` running the field `#[validate(email/url/not_empty/length/range/pattern/custom)]` checks; the `Valid` web extractor rejects a constraint failure with 422 | -| `#[cacheable]` / `#[cache_put]` / `#[cache_evict]` | declarative caching | a read-through / write-through / evict body around the process-registered cache adapter; `#[cacheable]` also takes `condition = "…"` (bypass the cache when the param expression is `false`) and `unless = "…"` (don't store when the result expression, `result: &V`, is `true`) | -| `#[async_method]` | fire-and-forget async | rewrites an `async fn(self: Arc, …) -> R` into a non-async `fn … -> TaskHandle` that spawns the body on the registered executor | -| `#[application_event_listener]` / `#[transactional_event_listener]` | in-process events | an `@EventListener` / `@TransactionalEventListener` discovered via `inventory` and fired by `publish_event` (the latter bound to a transaction commit phase) | -| `#[aspect]` (+ `#[before]`/`#[after]`/`#[around]`/…) | aspect-oriented advice | `impl firefly_aop::Aspect` + an `inventory` registration; advice runs around the explicit `advised(…)` weave point | - -The DI stereotype derives (`#[derive(Component/Service/Repository/Configuration/ -AutoConfiguration/Controller)]`, `#[bean]`, `#[autowired]`, `register_all!`) and -`#[derive(ConfigProperties)]` round out the set. `#[derive(AutoConfiguration)]` -is the auto-config holder whose `#[bean]`s back off behind a -`condition_on_missing_bean`, so an application can override any default by -declaring its own bean of the same type; `Container::scan()` auto-registers every -`#[bean]` method, and `Container::scan_packages([..])` restricts discovery to the -named module paths. -Lumen *does* use the container: `web.rs` carries a `#[derive(Configuration)]` -holder (`LumenBeans`) whose `#[bean]` factories declare the event store, read -model, query cache, JWT service, security `FilterChain` / `BearerLayer`, and the -ledger application service, and the `WalletApi` controller is a -`#[derive(Controller)]` bean with `#[autowired]` fields. The framework -component-scans them at boot — see the [DI deep-dive](./04a-dependency-injection.md). - -### CQRS — messages and handlers (`commands.rs`) - -`#[derive(Command)]` / `#[derive(Query)]` generate the `Message` impl. -`#[firefly(validate)]` makes an empty / zero field fail validation before the -handler runs; `#[firefly(cache_ttl = "…")]` feeds the query cache. These are -verbatim Lumen: +> **Tip** **Checkpoint.** Keep this table open in a second pane. As you read each +> step, find the row it corresponds to and confirm the "Generates" column matches +> the explanation. The table is the skeleton; the steps are the muscle. + +## Step 3 — CQRS messages and their handler bean (`commands.rs`) + +> **Note** **Key term — CQRS.** *Command/Query Responsibility Segregation* is a +> pattern that routes state-changing **commands** and read-only **queries** +> through separate handlers on a shared *bus*. A command mutates; a query reads; +> they never share a handler. The Spring analog is a command/query gateway over +> annotated `@CommandHandler` / `@QueryHandler` methods. + +`#[derive(Command)]` and `#[derive(Query)]` generate the `Message` impl that lets +the bus route a struct. `#[firefly(validate)]` on a field makes an empty or zero +value fail validation *before* the handler runs; `#[firefly(cache_ttl = "…")]` on +a query feeds the read cache. Here is the verbatim Lumen declaration, including +the `#[derive(Builder)]` and `#[derive(Schema)]` it also carries: ```rust,ignore -#[derive(Debug, Clone, Default, Serialize, Deserialize, Command)] +// samples/lumen/src/commands.rs +#[derive(Debug, Clone, Default, Serialize, Deserialize, Command, Builder, Schema)] #[serde(default)] pub struct OpenWallet { #[firefly(validate)] + #[builder(into)] // accept &str, String, … pub owner: String, #[serde(rename = "openingBalance")] + #[builder(default)] // unset → 0 pub opening_balance: i64, } @@ -143,14 +218,30 @@ pub struct GetWallet { } ``` -In Lumen the handlers live on a **DI bean** — `WalletHandlers`, a -`#[derive(Service)]` whose collaborators are `#[autowired]` from the container — -and `#[handlers]` registers each method on the bus. This is the Rust analog of -Spring scanning a `@Component`'s `@CommandHandler` / `@QueryHandler` methods: the -handler reaches its collaborators through `self`, so there is **no process-global -and no composition root**. This is verbatim Lumen: +What just happened, derive by derive: + +- `Command` / `Query` emit the `firefly::cqrs::Message` impl — the trait the bus + dispatches on. `#[firefly(validate)]` registers `owner` as a required field, so + `OpenWallet::default().validate()` is an `Err`. `#[firefly(cache_ttl = "30s")]` + is read by the query cache through the generated `Message::cache_ttl`. +- `Schema` emits an OpenAPI schema for the type, so `OpenWallet` shows up in the + spec served at `/v3/api-docs` on the management port — no hand-written schema. +- `Builder` (the Lombok `@Builder` analog) is covered in + [Step 10](#construction--the-fluent-builder-derivebuilder); ignore it for now. + +> **Note** **Key term — handler bean.** A *handler bean* is a DI component whose +> methods are the command/query handlers. Its collaborators are `#[autowired]` +> from the container, so each handler reaches them through `self` — there is no +> process-global and no composition root. The Spring analog is a `@Component` +> whose `@CommandHandler` / `@QueryHandler` methods are scanned and registered. + +In Lumen the handlers live on such a bean — `WalletHandlers`, a +`#[derive(Service)]` whose write-side `Ledger` and read-side `ReadModel` are +`#[autowired]` — and `#[handlers]` registers each method on the bus. This is +verbatim Lumen: ```rust,ignore +// samples/lumen/src/commands.rs #[derive(Service)] struct WalletHandlers { #[autowired] @@ -163,47 +254,72 @@ struct WalletHandlers { impl WalletHandlers { #[command_handler] async fn open_wallet(&self, cmd: OpenWallet) -> Result { - self.ledger.open(&cmd.owner, Money::cents(cmd.opening_balance)).await.map_err(to_cqrs) + if cmd.opening_balance < 0 { + return Err(CqrsError::validation("openingBalance must be >= 0")); + } + self.ledger + .open(&cmd.owner, Money::cents(cmd.opening_balance)) + .await + .map_err(to_cqrs) } #[query_handler] async fn get_wallet(&self, q: GetWallet) -> Result { - if let Some(view) = self.read_model.find(&q.id) { return Ok(view); } - /* … repair from the event stream … */ + if let Some(view) = self.read_model.find(&q.id) { + return Ok(view); + } + let events = self.ledger.load_events(&q.id).await.map_err(to_cqrs)?; + Ok(Wallet::rehydrate(&q.id, &events).view()) } // … deposit / withdraw } ``` -`#[handlers]` is an **impl-level** attribute (like `#[rest_controller]`) applied -to the `impl` block of a registered bean. Each method marked `#[command_handler]` -/ `#[query_handler]` takes `&self` plus one message argument; for every marker the -macro submits a `BeanHandlerRegistration` into a compile-time `inventory` registry -that resolves the bean from the container and installs a closure capturing it. -`FireflyApplication` drains those registrations at boot — so Lumen installs the -four handlers by *declaring* the bean and its methods, with no hand-written -`register(&bus)` call and no `bind` / `OnceLock` publishing wiring state. - -> **The free-`fn` form remains.** `#[command_handler]` / `#[query_handler]` also -> work on a **free `async fn(Msg) -> Result`** (generating a -> `register_(bus)` helper) for a simple, collaborator-free handler — the form +What just happened: `#[handlers]` is an **impl-level** attribute (like +`#[rest_controller]`) applied to the `impl` block of a registered bean. Each +method marked `#[command_handler]` / `#[query_handler]` takes `&self` plus one +message argument and returns `Result`. For every marker the macro +submits a `BeanHandlerRegistration` into a compile-time inventory registry that, +at boot, resolves the bean from the container and installs a closure capturing +it. `FireflyApplication` drains those registrations during +`register_discovered_handlers`. So Lumen installs all four handlers by *declaring* +the bean and its methods — there is no hand-written `register(&bus)` call and no +`OnceLock` publishing wiring state. + +Why it matters: the handler reaches `self.ledger` and `self.read_model` through +the container's injection, so the same handler that an HTTP test drives is the +same handler the live bus dispatches to — one wiring, exercised two ways. + +> **Note** `#[command_handler]` / `#[query_handler]` also work on a **free** +> `async fn(Msg) -> Result`, in which case the macro generates a +> `register_(bus)` helper for a simple, collaborator-free handler — the form > the `macro-quickstart` sample uses. `#[handlers]` is the **bean** form for a -> handler that autowires collaborators, which is Lumen's actual wiring. - -> **What it generates.** `#[derive(Command)]` / `#[derive(Query)]` emit the -> `Message` impl; `#[handlers]` emits a `BeanHandlerRegistration` per marked method -> that resolves the bean and installs it on the bus, drained by the framework from -> inventory. `#[firefly(cache_ttl)]` exposes a TTL the query cache reads. The -> `Bus` is the command/query gateway every dispatch flows through. - -### Event sourcing — domain events and the aggregate (`domain.rs`) - -`#[derive(DomainEvent)]` stamps each payload with a stable `EVENT_TYPE` -discriminator (its struct name) and a `to_domain_event` conversion; -`#[derive(AggregateRoot)]` finds the embedded `AggregateRoot` field and generates -`Wallet::AGGREGATE_TYPE` plus `aggregate()` / `aggregate_mut()`: +> handler that autowires collaborators, which is Lumen's actual wiring. Same +> markers, two shapes; Lumen uses the bean shape. + +> **Tip** **Checkpoint.** Find `get_wallet_carries_cache_ttl` in +> `samples/lumen/src/commands.rs`. It asserts `GetWallet::default().cache_ttl()` +> is `Some(_)` — direct proof that `#[firefly(cache_ttl = "30s")]` reached the +> generated `Message::cache_ttl`. Run `cargo test -p firefly-sample-lumen +> get_wallet_carries_cache_ttl` and watch it pass. + +## Step 4 — Domain events and the aggregate (`domain.rs`) + +> **Note** **Key term — event sourcing.** In *event sourcing* an aggregate's +> state is not stored as a row; it is the fold of an ordered stream of immutable +> **domain events**. To load a wallet you replay its events; to change it you +> append a new event. Each event needs a stable identity so a persisted stream +> stays readable as the schema evolves. The Spring analog is an Axon +> `@EventSourcingHandler` aggregate. + +`#[derive(DomainEvent)]` stamps each payload struct with a stable `EVENT_TYPE` +discriminator (its struct name) and a `to_domain_event` conversion onto the +framework wire event. `#[derive(AggregateRoot)]` finds the embedded +`AggregateRoot` field and generates `Wallet::AGGREGATE_TYPE` plus the `aggregate()` +/ `aggregate_mut()` accessors: ```rust,ignore +// samples/lumen/src/domain.rs #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, DomainEvent)] pub struct WalletOpened { pub wallet_id: String, @@ -221,19 +337,51 @@ pub struct Wallet { } ``` -The only event-sourcing wiring Lumen writes by hand is the `apply` fold; the -discriminators and the wire conversion are generated. +What just happened: the only event-sourcing wiring Lumen writes by hand is the +`apply` fold that projects one event into in-memory state. The discriminators +(`WalletOpened::EVENT_TYPE`, used when the aggregate `raise`s an event) and the +wire conversion are generated. The `#[firefly(aggregate_type = "Wallet")]` +argument pins the aggregate's type string, which the generated +`Wallet::AGGREGATE_TYPE` const exposes and the event store stamps on every +persisted event. + +Why it matters: the discriminator gives each event a stable, versioned JSON +identity, so a stream persisted today stays decodable after the payload struct +grows new fields tomorrow — the property event sourcing depends on. + +> **Tip** **Checkpoint.** In `domain.rs`, `rehydrate_folds_the_full_stream` +> asserts `Wallet::AGGREGATE_TYPE == "Wallet"` and folds an +> open + deposit + withdraw stream back to the right balance and version. That +> single test exercises both derives at once. + +## Step 5 — The projection listener (`ledger.rs`) + +> **Note** **Key term — projection.** A *projection* is a read-model builder: it +> consumes published domain events and writes a query-optimised view. Because it +> rebuilds the view from the event stream (rather than mutating a row from a +> single delivery), it is **idempotent** — an at-least-once redelivery converges +> on the same view. The Spring analog is a `@Component @EventListener` that +> updates a read table. + +First, the read model itself is a bean. Lumen's `ReadModel` is a +`#[derive(Repository)]` data-access component (Spring's `@Repository`) — an +in-memory map of wallet id to `WalletView`, kept dependency-free for the teaching +baseline: -> **What it generates.** `#[derive(DomainEvent)]` emits a stable `EVENT_TYPE` -> discriminator plus a `to_domain_event` conversion; `#[derive(AggregateRoot)]` -> emits `AGGREGATE_TYPE` and the `aggregate()` / `aggregate_mut()` accessors over -> the embedded root. The discriminator pins each event's identity in a stable, -> versioned JSON wire format, so persisted streams stay readable as the schema -> evolves. +```rust,ignore +// samples/lumen/src/ledger.rs +#[derive(Debug, Default, Repository)] +pub struct ReadModel { + rows: Mutex>, +} +``` -### Messaging — the projection listener (`ledger.rs`) +`container.scan()` registers it as a singleton bean, so it can be autowired (as +`Arc`) into the handler and projection beans. A production service +would back this with `firefly`'s reactive repository over Postgres; the in-memory +map keeps the baseline infrastructure-free. -Lumen's read-model projection is an **EDA listener bean** — `WalletProjection`, a +The projection is an **EDA listener bean** — `WalletProjection`, a `#[derive(Service)]` that `#[autowired]`s the `Ledger` (for the event store it replays) and the `ReadModel` it feeds. Inside a `#[handlers]` impl, an `#[event_listener(topic = "…")]` method marks the projection — exactly like the @@ -241,6 +389,7 @@ CQRS bean above, but the marker subscribes the method to an EDA topic rather tha the bus: ```rust,ignore +// samples/lumen/src/ledger.rs #[derive(Service)] struct WalletProjection { #[autowired] @@ -253,188 +402,494 @@ struct WalletProjection { impl WalletProjection { #[event_listener(topic = "wallets.events")] async fn project(&self, ev: Event) -> FireflyResult<()> { + let Some(wallet_id) = ev.headers.get("aggregateId") else { + return Ok(()); + }; // reload the wallet's stream, fold to a WalletView, upsert — idempotent. + if let Ok(events) = self.ledger.store().load(wallet_id).await { + let view = Wallet::rehydrate(wallet_id, &events).view(); + self.read_model.upsert(view); + } Ok(()) } } ``` -`#[handlers]` submits a `BeanListenerRegistration` into inventory that resolves the -bean from the container and subscribes its method to the topic on the very broker -the ledger publishes to. `FireflyApplication` drains it at boot — so the -subscription that closes the CQRS loop is wired entirely through the DI container, -with no `bind` / `OnceLock` and no composition-root `subscribe(&broker)` call. - -> **The free-`fn` form remains.** `#[event_listener(topic = "…")]` also works on a -> **free `async fn(Event) -> FireflyResult<()>`** (generating a -> `subscribe_(broker)` helper) for a simple, collaborator-free listener. +What just happened: `#[handlers]` submits a `BeanListenerRegistration` into +inventory that, at boot, resolves the bean from the container and subscribes its +method to `wallets.events` on the very broker the ledger publishes to. +`FireflyApplication` drains it during `subscribe_discovered_listeners`. The +subscription that closes the CQRS loop — write side appends and publishes, read +side projects — is therefore wired entirely through the DI container, with no +`subscribe(&broker)` call in any composition root. + +> **Note** Like the CQRS markers, `#[event_listener(topic = "…")]` also works on +> a **free** `async fn(Event) -> FireflyResult<()>`, generating a +> `subscribe_(broker)` helper for a simple, collaborator-free listener. > `#[handlers]` is the **bean** form for a projection that autowires collaborators > — Lumen's actual wiring. -> **What it generates.** Inside a `#[handlers]` impl, `#[event_listener(topic = -> "…")]` contributes a `BeanListenerRegistration` the framework drains, resolving -> the bean and subscribing its method to the topic on whatever broker transport is -> wired in. You write only the handler body; the subscription wiring is generated -> and drained for you. +> **Tip** **Checkpoint.** The HTTP test `open_then_get_round_trips_through_cqrs` +> (in `http_test.rs`) opens a wallet over `POST /api/v1/wallets`, then reads it +> over `GET /api/v1/wallets/:id` and sees the projected balance — proof the +> listener bean subscribed and the loop closed. It boots the full +> `FireflyApplication`, so it exercises the inventory drain end to end. -### Web — the controller (`web.rs`) +## Step 6 — The controller (`web.rs`) + +> **Note** **Key term — REST controller.** A *REST controller* is a bean whose +> methods map HTTP verbs and paths to handler functions. In Firefly the handler +> bodies are ordinary `axum` handlers; the macro generates the router and +> mounts it. The Spring analog is `@RestController` with `@GetMapping` / +> `@PostMapping`. `#[rest_controller(path = "…")]` turns an `impl` block into a generated -`WalletApi::routes(state) -> axum::Router`. Each method carries one verb mapping -and uses ordinary axum extractors, returning `WebResult` so a handler error -renders as RFC 9457 `application/problem+json`: +`WalletApi::routes(state) -> axum::Router`. The controller type itself is a +`#[derive(Controller)]` bean whose collaborators are `#[autowired]`: ```rust,ignore -#[rest_controller(path = "/api/v1")] +// samples/lumen/src/web.rs +#[derive(Clone, Controller)] +pub struct WalletApi { + #[autowired] + pub bus: Arc, + #[autowired] + pub ledger: Arc, + #[autowired] + pub query_cache: Arc, +} +``` + +Each method carries one verb mapping, uses ordinary axum extractors, and returns +`WebResult` so a handler error renders as RFC 9457 +`application/problem+json`. The verb attributes also carry the OpenAPI metadata +(`summary`, `description`, `status`, `tags`) the docs generator reads: + +```rust,ignore +// samples/lumen/src/web.rs +#[rest_controller(path = "/api/v1", tag = "Wallets")] impl WalletApi { - #[post("/wallets")] - async fn open(State(api): State, Json(body): Json) - -> WebResult<(axum::http::StatusCode, Json)> { /* dispatch via api.bus */ } - - #[get("/wallets/:id")] - async fn get(State(api): State, Path(id): Path) - -> WebResult> { /* … */ } - // … deposit / withdraw / transfer + #[post( + "/wallets", + summary = "Open a wallet", + description = "Opens a new wallet for an owner with an optional opening balance.", + status = 201 + )] + async fn open( + State(api): State, + Json(body): Json, + ) -> WebResult<(axum::http::StatusCode, Json)> { + let view: WalletView = api.bus.send(body).await.map_err(cqrs_to_web)?; + Ok((axum::http::StatusCode::CREATED, Json(view))) + } + + #[get("/wallets/:id", summary = "Fetch a wallet")] + async fn get( + State(api): State, + Path(id): Path, + ) -> WebResult> { + let view: WalletView = api.bus.query(GetWallet { id }).await.map_err(cqrs_to_web)?; + Ok(Json(view)) + } + // … deposit / withdraw / transfer / compliance / 2pc } // generated: WalletApi::routes(state) -> axum::Router ``` -The macro also submits a `ControllerMount` and a route descriptor into link-time -tables, so `FireflyApplication` **auto-mounts** the controller (resolving its -autowired state from the container) and the OpenAPI generator and the actuator -`/mappings` endpoint can enumerate Lumen's routes without re-parsing the source. -Lumen never hands `WalletApi::routes(state)` to the web stack — declaring the -controller bean is the entire wiring. +What just happened: the macro emits `WalletApi::routes(state)` **and** submits a +`ControllerMount` plus a per-route descriptor into link-time tables. So +`FireflyApplication` **auto-mounts** the controller (resolving its autowired +state from the container through `firefly::web::mount_controllers`), and the +OpenAPI generator and the actuator `/mappings` endpoint can enumerate Lumen's +routes without re-parsing the source. Lumen never hands `WalletApi::routes(state)` +to the web stack — declaring the controller bean is the entire wiring. + +Why it matters: `WebResult` renders any handler error as an RFC 9457 problem +body uniformly, so error shaping is the same across every endpoint with no +per-handler code, and the `:id` path parameter is the ordinary axum `Path` +extractor — Firefly does not invent its own routing. -> **What it generates.** `#[rest_controller]` + `#[get]`/`#[post]` emit a -> `WalletApi::routes(state) -> axum::Router`, a `ControllerMount` the framework -> auto-mounts, and a link-time mapping-table entry per route. `WebResult` -> renders any handler error as an `application/problem+json` body (RFC 9457), so -> error shaping is uniform across every endpoint without per-handler code. +> **Tip** **Checkpoint.** Run Lumen (`cargo run -p firefly-sample-lumen`) and +> open `http://localhost:8081/swagger-ui` on the **management** port. The +> "Open a wallet" summary, the `201` response, and the `Wallets` tag all come +> from the verb attributes above — no separate spec file. -### Scheduling — the heartbeat (`housekeeping.rs`) +## Step 7 — The scheduled heartbeat (`housekeeping.rs`) -`#[scheduled(...)]` generates a `schedule_(scheduler)` helper that registers a -zero-argument `async fn` on a `Scheduler`: +> **Note** **Key term — scheduled task.** A *scheduled task* is a zero-argument +> `async fn` the framework runs on a cadence — a fixed rate, a fixed delay, or a +> cron expression. The Spring analog is `@Scheduled`. + +`#[scheduled(...)]` generates a `schedule_(scheduler)` helper that registers +the function on a `Scheduler`, and also submits a `ScheduledRegistration` the +framework drains: ```rust,ignore +// samples/lumen/src/housekeeping.rs #[scheduled(fixed_rate = "60s", initial_delay = "5s")] -pub async fn ledger_heartbeat() -> Result<(), std::io::Error> { /* … */ } +pub async fn ledger_heartbeat() -> Result<(), std::io::Error> { + HEARTBEAT_TICKS.fetch_add(1, Ordering::Relaxed); + Ok(()) +} // generated: schedule_ledger_heartbeat(&scheduler) ``` -> **What it generates.** `#[scheduled(...)]` emits a `schedule_(scheduler)` -> helper *and* a `ScheduledRegistration` the framework drains -> (`register_discovered_scheduled(&scheduler)`), registering a zero-argument -> `async fn` on a `Scheduler`. Use `fixed_rate = "60s"` for a fixed cadence (with -> an optional `initial_delay`), or `cron = "…"` for a cron expression. +What just happened: `#[scheduled]` emits the `schedule_(scheduler)` helper +*and* the registration. At boot the framework calls +`register_discovered_scheduled(&scheduler)`, which drains the inventory and +installs every `#[scheduled]` task — so Lumen never calls `schedule_` by +hand. Use `fixed_rate = "60s"` for a fixed cadence (with an optional +`initial_delay`), or `cron = "…"` for a cron expression. -### Construction — the fluent builder (`#[derive(Builder)]`) +> **Tip** **Checkpoint.** `scheduled_task_registers` (in `housekeeping.rs`) +> builds a fresh scheduler, calls `register_discovered_scheduled`, and asserts +> `scheduler.tasks()` contains `"ledger_heartbeat"` — proof the registration was +> drained from inventory with no manual `schedule_` call. + +## Step 8 — The orchestration trio (`transfer.rs`, `compliance.rs`, `tcc_transfer.rs`) -Rust's stdlib derives already cover value-object boilerplate — debug formatting, -cloning, structural equality, defaults — with `#[derive(Debug, Clone, PartialEq, -Default)]` over `pub` fields. The one ergonomic gap they leave is a *fluent -builder* — and that is what `#[derive(Builder)]` fills. It generates `T::builder()` returning a -`TBuilder` with one setter per field and a `build() -> Result`. By -default every field is **required**: `build` returns an `Err` naming the first -unset field. `#[builder(default)]` falls back to `Default::default()`, -`#[builder(default = "expr")]` to a custom expression, and `#[builder(into)]` -makes the setter accept `impl Into`: +Three declarative orchestration macros round out Lumen's own source. Each turns +an annotated `impl` block into a runnable coordinator. We meet the key terms +first, then read one declaration each. + +> **Note** **Key term — saga.** A *saga* is a distributed transaction made of +> steps, each with a **compensation** that undoes it if a later step fails. There +> is no shared lock; consistency is restored by running compensations in reverse. +> The Spring/Axon analog is a `@Saga`. + +A transfer is *not* a single atomic command: it debits the source, then credits +the destination, and if the credit fails the debit must be refunded. That is the +saga pattern, declared as annotated methods on `TransferSaga`: ```rust,ignore -#[derive(Debug, Clone, Default, Serialize, Deserialize, Command, Builder)] -#[serde(default)] -pub struct OpenWallet { - #[firefly(validate)] - #[builder(into)] // accept &str, String, … - pub owner: String, - #[serde(rename = "openingBalance")] - #[builder(default)] // unset → 0 - pub opening_balance: i64, +// samples/lumen/src/transfer.rs +#[firefly::saga(name = "money-transfer")] +impl TransferSaga { + #[saga_step(id = "debit", compensate = "refund_debit")] + async fn debit(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { + self.ledger.withdraw(&req.from, Money::cents(req.amount)).await?; + Ok(()) + } + + async fn refund_debit(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { + self.ledger.deposit(&req.from, Money::cents(req.amount)).await?; + Ok(()) + } + + #[saga_step(id = "credit", depends_on = ["debit"])] + async fn credit(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { + self.ledger.deposit(&req.to, Money::cents(req.amount)).await?; + Ok(()) + } } +// generated: TransferSaga::run(req) and TransferSaga::saga() +``` -// A build-style constructor, errors surfaced as a String: +What just happened: `#[firefly::saga]` lowers these methods onto the +`firefly-orchestration` `Saga` engine. `depends_on` orders the steps, +`compensate` names the rollback method, and each parameter is injected from the +saga context — here the request, via `#[input]`. The macro generates +`TransferSaga::run`, which `run_transfer` calls. When the credit leg fails, the +engine runs `refund_debit`, so the source stream shows a real debit *and* its +compensating refund. + +> **Note** **Key term — workflow.** A *workflow* is a directed acyclic graph +> (DAG) of steps: independent steps run in parallel; a step with `depends_on` +> runs only after its prerequisites and reads their results via `#[from_step]`. +> Where a saga is a linear chain with compensation, a workflow is a parallel +> fan-in. The Spring analog is a DAG-based process such as Spring Cloud Data +> Flow's task graph. + +Lumen's compliance check runs `balance-check` and `limit-check` in parallel, then +`approve` after both: + +```rust,ignore +// samples/lumen/src/compliance.rs +#[firefly::workflow(name = "transfer-compliance")] +impl ComplianceCheck { + #[workflow_step(id = "balance-check")] + async fn balance_check(&self, #[input] req: TransferRequest) -> Result { /* … */ } + + #[workflow_step(id = "limit-check")] + async fn limit_check(&self, #[input] req: TransferRequest) -> Result { /* … */ } + + #[workflow_step(id = "approve", depends_on = ["balance-check", "limit-check"])] + async fn approve( + &self, + #[from_step("balance-check")] funds_ok: bool, + #[from_step("limit-check")] within_limit: bool, + ) -> Result<(), ComplianceError> { /* … */ } +} +// generated: ComplianceCheck::run(req) +``` + +> **Note** **Key term — TCC (Try / Confirm / Cancel).** *TCC* is a two-phase +> distributed transaction: every participant first **reserves** (try); only when +> all reservations succeed does the coordinator **confirm** them; otherwise it +> **cancels** the ones already tried. Where a saga undoes a committed leg, TCC +> reserves first and commits last. The Spring/Seata analog is the TCC +> transaction mode. + +Lumen's two-phase transfer holds the source, verifies the destination, then +captures on both sides: + +```rust,ignore +// samples/lumen/src/tcc_transfer.rs +#[firefly::tcc(name = "transfer-2pc")] +impl TwoPhaseTransfer { + #[participant(name = "source", confirm = "capture_source", cancel = "release_source")] + async fn hold_source(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { /* withdraw (hold) */ } + async fn capture_source(&self) -> Result<(), DomainError> { Ok(()) } // the debit was the capture + async fn release_source(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { /* deposit (release) */ } + + #[participant(name = "dest", confirm = "capture_dest")] + async fn hold_dest(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { /* verify exists */ } + async fn capture_dest(&self, #[input] req: TransferRequest) -> Result<(), DomainError> { /* deposit (capture) */ } +} +// generated: TwoPhaseTransfer::run(req) +``` + +What just happened across all three: each macro reads its annotated methods and +generates a `run` method over the orchestration engine, wiring the step/participant +graph, the parameter injection (`#[input]` / `#[from_step]`), and the +compensation or cancel path. You write the bodies; the macro writes the +coordinator. + +> **Tip** **Checkpoint.** The HTTP tests +> `transfer_saga_overdraft_compensates_and_is_422`, +> `compliance_workflow_rejects_overdraft_with_422`, and +> `tcc_transfer_overdraft_releases_the_hold_and_is_422` (in `http_test.rs`) +> exercise the failure path of each macro end to end. Run `cargo test +> -p firefly-sample-lumen overdraft` and watch all three pass. + +## Step 9 — The configuration holder and the streaming contributor (`web.rs`) + +Lumen *does* use the DI container directly. `web.rs` carries a +`#[derive(Configuration)]` holder whose `#[bean]` factory methods **declare** the +infrastructure beans: + +> **Note** **Key term — configuration holder and bean factory.** A *configuration +> holder* is a `#[derive(Configuration)]` type whose `#[bean]` methods are +> *factories*: each returns a constructed value the container registers as a bean +> and can autowire elsewhere. The Spring analog is a `@Configuration` class whose +> `@Bean` methods produce beans. + +```rust,ignore +// samples/lumen/src/web.rs +#[derive(Configuration)] +struct LumenBeans; + +#[bean] +impl LumenBeans { + #[bean] + fn event_store(&self) -> MemoryEventStore { MemoryEventStore::new() } + + #[bean] + fn query_cache(&self) -> QueryCache { QueryCache::new() } + + #[bean] + fn jwt_service(&self) -> JwtService { JwtService::new(crate::security::DEMO_SIGNING_KEY) } + + #[bean] + fn security_filter_chain(&self) -> FilterChain { crate::security::security_layers().1 } + + #[bean] + fn bearer_layer(&self) -> BearerLayer { crate::security::security_layers().0 } + + #[bean] + fn ledger(&self, store: Arc, broker: Arc) -> Ledger { + let store: Arc = store; + Ledger::new(store, broker) + } +} +``` + +What just happened: `container.scan()` discovers and registers every `#[bean]` +method, so `build_app` calls no `register_arc` — the **framework** does the +registration. The `ledger` factory even *autowires its own arguments* (`store` +and the framework-provided `Broker` port), so a bean factory is itself a wiring +point. The `security_filter_chain` and `bearer_layer` beans are auto-discovered +and layered onto the API with no `.security(...)` call — the Spring +`SecurityFilterChain` pattern. The full DI mechanics are in the +[Dependency Injection deep-dive](./04a-dependency-injection.md). + +The optional streaming endpoint shows one more declarative seam — a +`RouteContributor` bean: + +> **Note** **Key term — route contributor.** A *route contributor* is a bean that +> hands the framework an extra `axum::Router` to merge into the public API. It is +> how you add routes that do not fit the `#[rest_controller]` shape (here, a +> feature-gated reactive stream) by *declaring a bean* rather than touching a +> composition root. + +```rust,ignore +// samples/lumen/src/web.rs (feature `streaming`) +#[cfg(feature = "streaming")] +#[derive(Service)] +#[firefly(provides = "dyn firefly::web::RouteContributor")] +struct StreamingRoutes { + #[autowired] + api: Arc, +} + +#[cfg(feature = "streaming")] +impl firefly::web::RouteContributor for StreamingRoutes { + fn routes(&self) -> axum::Router { + streaming_router((*self.api).clone()) + } +} +``` + +What just happened: `#[firefly(provides = "dyn firefly::web::RouteContributor")]` +tells the container to register this `#[derive(Service)]` under the +`RouteContributor` port. The framework discovers it and merges its routes — a +feature-gated `GET /api/v1/wallets/:id/events` endpoint wired by declaring a +bean, not by a composition root. + +> **Tip** **Checkpoint.** Open `samples/lumen/src/web.rs` and confirm +> `build_router()` (the test seam) is just +> `FireflyApplication::new(APP_NAME).version(VERSION).bootstrap().await… +> .api_router` — no hand-written builder. Every bean in this step is +> auto-registered by `container.scan()`; the controller is auto-mounted; security +> and the read-cache middleware are auto-discovered. + +## Step 10 — The rest of the declarative set (not used by Lumen) + +Several more macros are first-class parts of the framework that Lumen does **not** +exercise in its own source — it is event-sourced (so the relational +`#[firefly::repository]` / `#[firefly::transactional]` never appear) and handles +the remaining cross-cutting concerns by other means. Each is shown here as a +focused, correct standalone example so the catalogue is complete. + +| Macro | Purpose | Generates | +|-------|---------|-----------| +| `#[derive(Builder)]` | a fluent constructor with required/defaulted fields | `T::builder()` → fluent setters → `build() -> Result` | +| `#[derive(Mapper)]` | compile-time struct-to-struct conversion | one `From` per `#[firefly(from = "…")]` | +| `#[derive(Entity)]` | the `@Entity` mapping from annotated struct fields | a `SqlxEntity` impl (`@Table` / `@Id` / `@Version` / `@Column`) | +| `#[derive(SqlxRepository)]` | a fully-wired sqlx `@Repository` bean | `ReactiveCrudRepository` **and** `ReactiveSpecificationRepository` impls plus the `repository()` accessor | +| `#[firefly::repository]` | derived-query and custom-query method bodies | method bodies on a `SqlxReactiveRepository` impl from method names or `#[query(…)]` | +| `#[firefly::transactional]` | a declared transaction boundary | a commit-on-`Ok` / rollback-on-`Err` boundary around an `async fn` | +| `#[firefly::pre_authorize]` / `#[firefly::post_authorize]` | method-level access control | an access check before the body, or a returnObject check after it | +| `#[derive(Validate)]` (+ `Valid`) | JSR-380 bean validation | an `impl Validate`; the `Valid` extractor rejects a constraint failure with 422 | +| `#[cacheable]` / `#[cache_put]` / `#[cache_evict]` | declarative caching | a read-through / write-through / evict body around the registered cache adapter | +| `#[retry]` / `#[circuit_breaker]` / `#[rate_limit]` / `#[bulkhead]` / `#[timeout]` | resilience decorators | the body wrapped in the matching `firefly_resilience` primitive | +| `#[async_method]` | fire-and-forget async | an `async fn(self: Arc, …) -> R` rewritten to a non-async `fn … -> TaskHandle` | +| `#[application_event_listener]` / `#[transactional_event_listener]` | in-process events | an `@EventListener` / `@TransactionalEventListener` discovered via inventory | +| `#[aspect]` (+ `#[before]`/`#[after]`/`#[around]`) | aspect-oriented advice | `impl firefly_aop::Aspect` + an inventory registration | + +The remaining DI stereotype derives round out the set: +`#[derive(Component/Service/Repository/Configuration/AutoConfiguration/Controller)]`, +`#[bean]`, `#[autowired]`, `register_all!`, and `#[derive(ConfigProperties)]`. +`#[derive(AutoConfiguration)]` is the auto-config holder whose `#[bean]`s back off +behind a `condition_on_missing_bean`, so an application can override any default by +declaring its own bean of the same type; `Container::scan()` auto-registers every +`#[bean]` method, and `Container::scan_packages([..])` restricts discovery to +named module paths. + +### Construction — the fluent builder (`#[derive(Builder)]`) + +Rust's stdlib derives already cover value-object boilerplate — `Debug`, `Clone`, +`PartialEq`, `Default`. The one ergonomic gap they leave is a *fluent builder*, +and that is what `#[derive(Builder)]` (Lombok's `@Builder`) fills. It generates +`T::builder()` returning a `TBuilder` with one setter per field and a +`build() -> Result`. By default every field is **required**: `build` +returns an `Err` naming the first unset field. `#[builder(default)]` falls back to +`Default::default()`, `#[builder(default = "expr")]` to a custom expression, and +`#[builder(into)]` makes the setter accept `impl Into`. Lumen's +`OpenWallet` (from [Step 3](#step-3--cqrs-messages-and-their-handler-bean-commandsrs)) +carries it: + +```rust,ignore let cmd = OpenWallet::builder() .owner("ada") // impl Into .opening_balance(10_000) .build()?; // Result ``` -A required field left unset surfaces as a `build()` error, not a compile error — -the tradeoff a fluent builder makes against a hand-written typed constructor, -where the compiler enforces arity. Reach for `#[derive(Builder)]` when a struct -has many optional/defaulted fields; keep a plain literal when every field is -required and present. - -> **What it generates.** `#[derive(Builder)]` emits `T::builder()` returning a -> `TBuilder` with one setter per field and a `build() -> Result` that -> errors on the first unset required field. `#[builder(default)]` falls back to -> `Default::default()`, `#[builder(default = "expr")]` to a custom expression, and -> `#[builder(into)]` makes a setter accept `impl Into`. Returning a -> `Result` keeps missing-field handling on the normal `?` path rather than a panic. +Returning a `Result` keeps missing-field handling on the normal `?` path rather +than a panic. Reach for `#[derive(Builder)]` when a struct has many +optional/defaulted fields; keep a plain literal when every field is required and +present. ### Conversion — the compile-time mapper (`#[derive(Mapper)]`) `#[derive(Mapper)]` generates a compile-time, type-checked `From` that -maps a source struct to a target struct field-by-field. One -`#[firefly(from = "Source")]` produces one `From` impl, and the attribute is -**repeatable** to map from several sources. Per-field attributes adjust the -mapping: `#[firefly(rename = "src")]` reads a differently named source field, -`#[firefly(into)]` applies `.into()` to the source value, `#[firefly(with = "fn")]` -runs a conversion function, and `#[firefly(default)]` / +maps a source struct to a target field-by-field. One `#[firefly(from = "Source")]` +produces one `From` impl, and the attribute is **repeatable** to map from several +sources. Per-field attributes adjust the mapping: `#[firefly(rename = "src")]` +reads a differently named source field, `#[firefly(into)]` applies `.into()`, +`#[firefly(with = "fn")]` runs a conversion function, and `#[firefly(default)]` / `#[firefly(default_expr = "expr")]` fill a target field with no source read: ```rust,ignore -// A read-model view assembled from the domain aggregate. #[derive(Debug, Clone, Serialize, Deserialize, Mapper)] #[firefly(from = "Wallet")] pub struct WalletView { #[firefly(rename = "root", with = "aggregate_id")] // read src.root, run aggregate_id(..) pub id: String, pub owner: String, // same name on both ends: a plain move - #[firefly(with = "Money::cents_value")] // src.balance: Money -> i64 via a conversion fn + #[firefly(with = "Money::cents_value")] // src.balance: Money -> i64 via a fn pub balance: i64, - #[firefly(default)] // version is set by the projector, not the fold + #[firefly(default)] // version set by the projector, not the fold pub version: i64, } // generates: impl From for WalletView { fn from(src: Wallet) -> Self { … } } - -let view: WalletView = wallet.into(); ``` Because the generated code is a plain `From` impl, every field is checked by the -compiler and there is no runtime cost — that compile-time guarantee is the whole -point. Contrast this with the **runtime** `firefly_data::Mapper`, which converts -via two serde passes and is checked at runtime: use the runtime mapper as a -dynamic fallback when the source type is not known at compile time (e.g. mapping -arbitrary JSON), and prefer `#[derive(Mapper)]` whenever both ends are concrete -types — which, in Lumen's projection, they are. - -> **What it generates.** `#[derive(Mapper)]` emits one `impl From for -> Target` per `#[firefly(from = "Source")]` (the attribute is repeatable). -> Per-field attributes shape each move: `#[firefly(rename = "src")]` reads a -> differently named source field, `#[firefly(into)]` applies `.into()`, -> `#[firefly(with = "fn")]` runs a conversion function, and `#[firefly(default)]` / -> `#[firefly(default_expr = "expr")]` fill a target field with no source read. The -> runtime `firefly_data::Mapper` is the dynamic, serde-based fallback for when the -> source type isn't known until runtime. - -### Persistence — derived queries and transactions (relational) - -These two macros sit on the relational persistence path. Lumen's read model is -an in-memory projection over an event stream (chapter 7), so it uses neither — but -in a relational service they are the everyday tools, so here is each in brief, -with the [persistence chapter](./07-persistence.md) as the full reference. +compiler with no runtime cost — that compile-time guarantee is the whole point. +Contrast it with the **runtime** `firefly_data::Mapper`, which converts via two +serde passes: use the runtime mapper when the source type is not known until +runtime (mapping arbitrary JSON), and prefer `#[derive(Mapper)]` whenever both +ends are concrete types. + +> **Note** Lumen's real `WalletView` is built by a hand-written `Wallet::view` +> method (in `domain.rs`) rather than `#[derive(Mapper)]`; the listing above is +> the equivalent declarative form, shown to illustrate the macro. + +### Persistence — entities, repositories, and transactions (relational) + +These macros sit on the relational persistence path. Lumen's read model is an +in-memory projection over an event stream, so it uses none of them — but in a +relational service they are the everyday tools. The full reference is the +[Persistence chapter](./07-persistence.md). + +`#[derive(Entity)]` generates the `SqlxEntity` mapping (`@Table` / `@Id` / +`@Version` / `@Column`) from annotated fields. Scalar fields map automatically; a +non-scalar field uses `#[firefly(with(read = "path", write = "path"))]`: + +```rust,ignore +#[derive(Debug, Clone, Entity)] +#[firefly(table = "accounts")] +pub struct Account { + #[firefly(id)] + pub id: String, + pub owner: String, + pub status: String, + #[firefly(version)] + pub version: i64, +} +``` + +`#[derive(SqlxRepository)]` builds a fully-wired `@Repository` bean from the +injected `Db` datasource (via `repository_for`). It implements both +`ReactiveCrudRepository` (the `save` / `find_by_id` / `delete_by_id` / `count` +surface) **and** `ReactiveSpecificationRepository` (`find_by_spec`, the +`JpaSpecificationExecutor` analog) by delegation, and exposes the `repository()` +accessor that `#[firefly::repository]` builds on: + +```rust,ignore +#[derive(SqlxRepository)] +#[firefly(entity = "Account")] +pub struct AccountRepo { + db: Arc, +} +``` `#[firefly::repository]` turns a `find_by_…` / `count_by_…` / `exists_by_…` / -`delete_by_…` method name into a working query body. Applied to an `impl` block, it -parses each method name into a derived query, marshals the typed arguments, and -delegates to the tested runtime engine — so you declare a typed method and get a -working, compiler-checked implementation. The runtime method is chosen from the -**return type** (`Vec`/`Option` → find, `i64` → count, `bool` → exists, -`u64` → delete), and the type exposes its backing `SqlxReactiveRepository` through -an accessor (default `self.repository()`, overridable with `#[repository(repo = -"…")]`): +`delete_by_…` method name into a working query body. The runtime method is chosen +from the **return type** (`Vec` / `Option` → find, `i64` → count, `bool` → +exists, `u64` → delete); the placeholder `unimplemented!()` bodies are discarded: ```rust,ignore #[firefly::repository] @@ -446,13 +901,10 @@ impl AccountRepo { } ``` -The placeholder `unimplemented!()` bodies are discarded and replaced by the -generated delegations. - -**Paged derived queries.** Give a `find_by_…` method a trailing `Pageable` -argument (and a `Result, DataError>` return) and the generated body appends -the page's sort and window to the query, delegating to the runtime -`SqlxReactiveRepository::find_by_derived_paged`: +Give a `find_by_…` method a trailing `Pageable` argument (and a +`Result, DataError>` return) and the generated body appends the page's sort +and window, delegating to `find_by_derived_paged`. Note that `Pageable::of` +returns a `Result`: ```rust,ignore #[firefly::repository] @@ -461,17 +913,16 @@ impl AccountRepo { -> Result, DataError> { unimplemented!() } } -// Build the page (1-based index) with sort + window: +// Build the page (1-based index) with sort + window — `of` returns a Result: let page = Pageable::of(1, 20, RequestSort::of([Order::desc("id")])).unwrap(); let rows = repo.find_by_owner("ada", page).await?; ``` -**Custom queries with `#[query(…)]`.** When a name-derived query isn't enough, -annotate a stub method with `#[query(...)]` and write the statement directly. -Native SQL binds each `:name` placeholder to the argument named `name`; the -**return type** selects the operation exactly as for derived queries — `Vec` / -`Option` is a list, `i64` a count, `bool` an existence check, and `u64` a -modifying statement (INSERT/UPDATE/DELETE, returning affected rows): +When a name-derived query is not enough, annotate a stub with `#[query(...)]` and +write the statement directly. Native SQL binds each `:name` placeholder to the +argument named `name`; the **return type** still selects the operation — +`Vec` / `Option` is a list, `i64` a count, `bool` an existence check, and +`u64` a modifying statement (returning affected rows): ```rust,ignore #[firefly::repository] @@ -486,21 +937,23 @@ impl AccountRepo { `#[query(sql = "…")]` is the explicit spelling of the native form, and `#[query(jpql = "…", entity = "Account")]` writes the statement against entity -names. Each lowers to the matching runtime call — -`query_list` / `query_count` / `query_exists` / `query_execute`. +names. + +> **Note** **Key term — transaction boundary.** A *transaction boundary* is a +> region of code whose database work commits together or rolls back together. +> `#[firefly::transactional]` makes that boundary a declaration on an `async fn`. +> The Spring analog is `@Transactional`. -`#[firefly::transactional]` wraps an `async fn`'s body in a transaction governed by -the registered `TransactionManager` — commit on `Ok`, roll back on `Err` — so the -boundary becomes a declaration. The function must be `async`, must return a -`Result`, and its error type must implement -`From` so begin/commit failures surface through the -normal `?` path. Bare or with options (`propagation`, `isolation`, `read_only`, -`timeout_ms`, `manager`): +`#[firefly::transactional]` wraps an `async fn`'s body in a transaction governed +by the registered `TransactionManager` — commit on `Ok`, roll back on `Err`. The +function must be `async`, must return `Result`, and its error type must +implement `From` so begin/commit failures surface +through `?`. Bare, or with options: ```rust,ignore #[firefly::transactional] async fn open_account(repo: &AccountRepo, acct: Account) -> Result<(), DataError> { - repo.insert(&acct).await?; // committed together on Ok, + repo.insert(&acct).await?; // committed together on Ok, repo.insert_audit(&acct).await?; // rolled back together on Err Ok(()) } @@ -510,14 +963,15 @@ async fn reconcile(repo: &LedgerRepo) -> Result<(), DataError> { /* … */ } ``` By default the boundary runs through the **process-global** registered -`TransactionManager`. `manager = ""` (Spring's `@Transactional("txManager")`) -instead binds it to an **explicit** manager the service owns — the expression -yields a value `m` with `&m: &Arc`. Use it for a -multi-datasource service, or to keep per-instance/per-test isolation (each -instance drives its own manager rather than a shared global one). The -`lumen-ledger` `transfer` use case is wired exactly this way: +`TransactionManager`. `manager = ""` (Spring's +`@Transactional("txManager")`) instead binds it to an **explicit** manager the +service owns — the expression yields a value `m` with +`&m: &Arc`. Use it for a multi-datasource service, or to +keep per-instance/per-test isolation. The `lumen-ledger` sample's `transfer` use +case is wired exactly this way: ```rust,ignore +// samples/lumen-ledger — core/src/services/wallet/v1/wallet_service_impl.rs #[firefly::transactional(manager = "self.tx_manager()")] // self owns the manager async fn transfer_tx(&self, from: Uuid, to: Uuid, amount: i64) -> Result { let mut src = self.load_active(from).await?; // debit + credit commit @@ -528,31 +982,38 @@ async fn transfer_tx(&self, from: Uuid, to: Uuid, amount: i64) -> Result Result<(), DataError> { /* … */ } +``` + +These two relational macros are the counterpart to how Lumen achieves consistency *without* a transaction manager: it appends events to the `EventStore` under optimistic concurrency and projects them, rather than mutating rows inside a -`#[transactional]` boundary (chapter 11). Same goal — atomic, consistent writes — -reached by two different architectures. - -> **What it generates.** `#[firefly::repository]` replaces each stub body with a -> delegation to `SqlxReactiveRepository`: name-derived methods to -> `find_by_derived` / `find_by_derived_paged`, and `#[query(…)]` methods to -> `query_list` / `query_count` / `query_exists` / `query_execute`, picking the call -> from the return type. `#[firefly::transactional]` wraps the body in a -> begin/commit/rollback boundary on the registered `TransactionManager`; -> `propagation` / `isolation` / `read_only` / `timeout_ms` tune that boundary. +`#[transactional]` boundary. Same goal — atomic, consistent writes — reached by +two different architectures. ### Method security — `#[pre_authorize]` / `#[post_authorize]` Two macros enforce access control at the method boundary, reading the caller's identity from the ambient security context rather than from a `Request`. The full -treatment is in the [security chapter](./14-security.md); here is the catalogue -entry. +treatment is in the [Security chapter](./14-security.md). -`#[firefly::pre_authorize(...)]` runs an access check **before** the function body. -Apply it to a `fn` returning `Result` whose error type implements -`From`, so a denial surfaces through the normal -`?` path. The rule forms are: +`#[firefly::pre_authorize(...)]` runs an access check **before** the body. Apply +it to a `fn` returning `Result` whose error implements +`From`, so a denial travels the `?` path: ```rust,ignore #[firefly::pre_authorize] // `authenticated` — any caller in scope @@ -566,20 +1027,16 @@ async fn open_account(&self, req: OpenAccount) -> Result { /* #[firefly::pre_authorize(authority = "wallet:write")] // a single fine-grained authority async fn deposit(&self, id: &str, cents: i64) -> Result<(), AppError> { /* … */ } - -#[firefly::pre_authorize(any_authority = ["wallet:write", "wallet:admin"])] -async fn withdraw(&self, id: &str, cents: i64) -> Result<(), AppError> { /* … */ } ``` When no caller is in scope the body is skipped and the macro returns `Err(SecurityError::Unauthenticated.into())`; when a caller is present but lacks the required role/authority it returns `Err(SecurityError::Forbidden.into())`. -`#[firefly::post_authorize()]` runs **after** an `async fn` returns -`Result` and gates the value on a boolean expression. The expression sees -`result` (a `&T` to the returned value — the returnObject) and `auth` (a -`&Authentication`); if it evaluates to `false` the value is discarded and the call -returns `Forbidden`: +`#[firefly::post_authorize()]` runs **after** an `async fn` returns and +gates the value on a boolean expression that sees `result` (a `&T` to the returned +value) and `auth` (a `&Authentication`); if it is `false` the value is discarded +and the call returns `Forbidden`: ```rust,ignore // Only return the wallet if the caller owns it. @@ -587,30 +1044,60 @@ returns `Forbidden`: async fn get_wallet(&self, id: &str) -> Result { /* … */ } ``` -Both macros read the caller from the ambient context in `firefly_security`: -`with_authentication_scope(auth, fut).await` runs a future with an -`Authentication` in scope, `current_authentication() -> Option` -reads it, and `check_access(&AccessRule) -> Result` -is the runtime check the macros expand to, over -`AccessRule::{Authenticated, Role, AnyRole, Authority, AnyAuthority}`. Because -`BearerLayer` scopes the authentication for the whole downstream call (on both the -anonymous and verified paths), these checks work on a service method that never -sees the `Request` — the macro reads from scope, not from a handler argument. +Because `BearerLayer` scopes the authentication for the whole downstream call, +these checks work on a service method that never sees the `Request` — the macro +reads from scope, not from a handler argument. + +### Validation — `#[derive(Validate)]` and `Valid` + +`#[derive(Validate)]` generates an `impl Validate` that runs each field's +`#[validate(email/url/not_empty/length/range/pattern/custom)]` constraint, and the +`Valid` web extractor rejects a constraint failure with `422`: + +```rust,ignore +#[derive(Debug, Deserialize, Validate)] +struct CreateUser { + #[validate(not_empty, length(min = 2, max = 64))] + name: String, + #[validate(email)] + email: String, +} + +// In a controller, `Valid` returns 422 if any constraint fails: +async fn create(Valid(body): Valid) -> WebResult> { /* … */ } +``` + +### Caching, async, in-process events, and aspects -> **What it generates.** `#[pre_authorize]` emits a `check_access(&AccessRule)?` -> against the ambient context before your body, with an empty attribute defaulting -> to `AccessRule::Authenticated`. `#[post_authorize]` evaluates its boolean over -> `result` and `auth` after the body and converts `false` into a `Forbidden` -> error. Both rely on `SecurityError: From` for the function's error type, so -> denials travel the normal `Result` path. +`#[cacheable]` / `#[cache_put]` / `#[cache_evict]` wrap a method body in a +read-through / write-through / evict path around the process-registered cache +adapter. `#[cacheable]` also takes `condition = ""` (bypass the cache +when the parameter expression is `false`) and `unless = ""` (do not +store when the result expression — bound as `result: &V` — is `true`): + +```rust,ignore +#[cacheable(key = "format!(\"order:{}\", id)", unless = "result.is_empty()")] +async fn load_order(&self, id: &str) -> Result { /* … */ } +``` -### Resilience — `#[retry]` / `#[circuit_breaker]` / `#[rate_limit]` / `#[bulkhead]` / `#[timeout]` +`#[async_method]` rewrites an `async fn(self: Arc, …) -> R` into a +non-async `fn … -> TaskHandle` that spawns the body on the registered +executor — fire-and-forget, with a handle to await later. -Where the [`firefly_resilience`](./91-appendix-modules.md) primitives are the -fluent, build-it-yourself surface (`Retry::new().max_attempts(3).execute(op)`), -five **decorator** macros put the same guards on a method — Resilience4j / -Spring-Retry's `@Retry`, `@CircuitBreaker`, `@RateLimiter`, `@Bulkhead`, -`@TimeLimiter`: +`#[application_event_listener]` / `#[transactional_event_listener]` are the +in-process event listeners (Spring's `@EventListener` / +`@TransactionalEventListener`): each is discovered via inventory and fired by +`publish_event`, the transactional one bound to a commit phase. + +`#[aspect]` (with `#[before]` / `#[after]` / `#[around]` advice) generates an +`impl firefly_aop::Aspect` plus an inventory registration; advice runs around the +explicit `advised(…)` weave point. + +### Resilience decorators + +Where the `firefly_resilience` primitives are the build-it-yourself surface +(`Retry::new().max_attempts(3).execute(op)`), five **decorator** macros put the +same guards on a method — the Resilience4j / Spring-Retry analogs: ```rust,ignore #[firefly::retry(max_attempts = 4, delay = "100ms", backoff = 2.0, max_delay = "2s")] @@ -632,12 +1119,9 @@ async fn slow_report(&self) -> Result { /* … */ } Apply them to an `async fn` returning `Result` whose error implements `std::error::Error + Send + Sync + 'static + From`. The decorator threads the body's own failure through the primitive and recovers -the **original `E`** on the way out — so a caller still pattern-matches the domain -error — while a guard's own short-circuit (a timeout, an open circuit, a -rate-limit or bulkhead rejection) surfaces through `E::from(ResilienceError)`. - -The attributes **stack**, outermost first, exactly like layering `@Retry` over -`@CircuitBreaker`: +the **original `E`** on the way out, while a guard's own short-circuit (a timeout, +an open circuit, a rejection) surfaces through `E::from(ResilienceError)`. The +attributes **stack**, outermost first: ```rust,ignore #[firefly::retry(max_attempts = 3, delay = "50ms")] // outer: re-runs the call @@ -645,124 +1129,200 @@ The attributes **stack**, outermost first, exactly like layering `@Retry` over async fn call_upstream(&self) -> Result { /* … */ } ``` -> **What it generates.** Each macro wraps the body in the matching -> `firefly_resilience` primitive's `execute`. The stateful guards -> (`#[circuit_breaker]`, `#[rate_limit]`, `#[bulkhead]`) live in a per-method -> `static` so their state — the breaker's failure count, the bucket's tokens, the -> in-flight permits — is **shared across every call**, the Resilience4j -> registry-bean semantics; `#[retry]` and `#[timeout]` are stateless and rebuilt -> per call. Durations accept a unit-suffixed string (`"100ms"`, `"2s"`, `"1m"`) -> or a bare integer of milliseconds. - -## How it works: the `__rt` contract - -A `proc-macro` crate cannot re-export runtime types, so macro-generated code -references every runtime type through the facade's hidden **`__rt` contract -path** — e.g. `::firefly::__rt::firefly_cqrs::Bus`. That is why Lumen, depending -only on `firefly` (plus the `axum`/`serde` it writes against anyway), compiles -whatever a macro expands to without listing the underlying `firefly-*` crates. -You never write `__rt` yourself. If you rename or shim the facade, pass -`#[firefly(crate = "my_firefly")]` to any macro to override the leading segment. - -The declarative layer goes further than generating helpers: each handler bean, -listener bean, scheduled task, and controller also submits a registration into a -compile-time `inventory` registry, and `FireflyApplication` **drains** those -registries at boot. So Lumen calls *none* of the wiring by hand — no -`register(&bus)`, no `subscribe(&broker)`, no `WalletApi::routes(state)` handed to -the web stack, and no `bind` / `OnceLock` publishing the handlers' collaborators. -It declares the `WalletHandlers` / `WalletProjection` beans, the task, and the -controller, and the framework resolves each bean from the container and installs -it (`register_discovered_handlers` / `subscribe_discovered_listeners` / -`register_discovered_scheduled` / `mount_controllers`). The free-`fn` -`#[command_handler]` / `#[query_handler]` / `#[event_listener]` form still -generates a `register_` / `subscribe_` helper for a collaborator-free -handler (and `#[scheduled]` a `schedule_`); but Lumen's handlers autowire -collaborators, so it uses the `#[handlers]` bean form, and the running service is +The stateful guards (`#[circuit_breaker]`, `#[rate_limit]`, `#[bulkhead]`) keep +their state in a per-method `static`, shared across every call — the Resilience4j +registry-bean semantics; `#[retry]` and `#[timeout]` are stateless and rebuilt per +call. Durations accept a unit-suffixed string (`"100ms"`, `"2s"`, `"1m"`) or a +bare integer of milliseconds. + +### The outbound HTTP client — `#[http_client]` + +`#[http_client]` is the declarative HTTP-interface client (Spring's +`@HttpExchange`). Applied to a `trait`, it emits the trait verbatim **and** a +`Impl` struct that wraps a `WebClient` and implements the trait by +translating each method's verb attribute and `:id`-style path into a call. The +awaited `async fn -> Result` shape decodes the body, surfaces a +404 as `ClientError::Problem`, and supports a custom error via +`E: From`; non-awaited `Mono` / `Flux` returns surface the raw +`ClientError` unchanged: + +```rust,ignore +#[http_client(path = "/api/v1/orders")] +trait OrderClient { + #[get("/:id")] + async fn get_order(&self, id: String) -> Result; + + #[get("/")] + async fn list(&self, status: String, page: Option) -> Result, ClientError>; + + #[post("/")] + async fn create(&self, body: NewOrder) -> Result; + + #[get("/opt/:id")] + async fn find_opt(&self, id: String) -> Result, ClientError>; +} +// generated: struct OrderClientImpl { … } impl OrderClient for OrderClientImpl { … } +``` + +> **Tip** **Checkpoint.** You will not run any of Step 10's examples against +> Lumen — they are catalogue entries, not Lumen source. The litmus test is the +> next step: confirm the *Lumen* macros all compile and pass. + +## Step 11 — How the wiring actually lands: the `__rt` contract and the inventory drain + +You have now seen every macro Lumen uses. The last piece is *why declaring a bean +is the whole wiring*. Two mechanisms make it work. + +First, the **`__rt` contract path** from [Step 1](#step-1--one-dependency-one-prelude). +A `proc-macro` crate cannot re-export runtime types, so macro-generated code names +every runtime type through `::firefly::__rt::firefly_cqrs::Bus` and friends. That +is the reason a one-crate service compiles whatever a macro expands to without +listing the underlying `firefly-*` crates. + +Second, the **inventory drain**. The declarative layer does more than generate +helpers: each handler bean, listener bean, scheduled task, and controller also +submits a registration into a compile-time inventory registry, and +`FireflyApplication` drains those registries at boot. So Lumen calls *none* of the +wiring by hand: + +- no `register(&bus)` — drained by `register_discovered_handlers`, +- no `subscribe(&broker)` — drained by `subscribe_discovered_listeners`, +- no `schedule_(scheduler)` — drained by `register_discovered_scheduled`, +- no `WalletApi::routes(state)` handed to the web stack — drained by + `mount_controllers`, +- and no `OnceLock` publishing the handlers' collaborators — they autowire from + the container. + +Lumen declares the `WalletHandlers` / `WalletProjection` beans, the heartbeat +task, the `LumenBeans` factories, and the `WalletApi` controller, and the +framework resolves each bean from the container and installs it. The free-`fn` +form of `#[command_handler]` / `#[query_handler]` / `#[event_listener]` / +`#[scheduled]` still generates a `register_` / `subscribe_` / +`schedule_` helper for the collaborator-free case; but because Lumen's +handlers autowire collaborators, it uses the bean form, and the running service is wired entirely by the inventory drain. -## The whole crate, declaratively +> **Tip** **Checkpoint.** Run Lumen and read the startup report. The +> `:: cqrs handlers: … | event listeners: … | scheduled tasks: … | controllers: +> … ::` line is the inventory the framework drained — the count is exactly the +> beans, listeners, tasks, and controllers you declared, with no registration +> call anywhere in the source. + +## Step 12 — The whole crate, declaratively Read top to bottom, the macros tell Lumen's story: ```text money.rs (no macros — a pure value object; the no-thiserror promise) - domain.rs #[derive(DomainEvent)] x3 #[derive(AggregateRoot)] - ledger.rs #[derive(Service)] WalletProjection + domain.rs #[derive(DomainEvent)] x3 #[derive(AggregateRoot)] #[derive(Schema)] + ledger.rs #[derive(Repository)] ReadModel #[derive(Service)] WalletProjection #[handlers] + #[event_listener(topic = "wallets.events")] - commands.rs #[derive(Command)] x3 #[derive(Query)] + commands.rs #[derive(Command)] x3 #[derive(Query)] #[derive(Builder/Schema)] #[derive(Service)] WalletHandlers #[handlers] + #[command_handler] x3 + #[query_handler] transfer.rs #[firefly::saga] + #[saga_step] x2 compliance.rs #[firefly::workflow] + #[workflow_step] x3 tcc_transfer.rs #[firefly::tcc] + #[participant] x2 security.rs (JwtService / BearerLayer / FilterChain — runtime APIs) - web.rs #[rest_controller] + #[get] / #[post] x7 + web.rs #[derive(Configuration)] + #[bean] x6 #[derive(Controller)] + #[rest_controller] + #[get] / #[post] x7 housekeeping.rs #[scheduled(fixed_rate = "60s", initial_delay = "5s")] ``` -Note what is *not* a macro: the security filter chain is built with a runtime -builder (`FilterChain::new().require(...)`), because its shape is data, not a -fixed declaration — and Lumen keeps it explicit so the control flow stays -visible. The saga, workflow, and TCC are now declarative macros -(`#[firefly::saga]`, `#[firefly::workflow]`, `#[firefly::tcc]`); only the filter +What is *not* a macro is just as telling: the security filter chain is built with +a runtime builder (`FilterChain::new().require(...)`), because its shape is data, +not a fixed declaration — and Lumen keeps it explicit so the control flow stays +visible. The saga, workflow, and TCC *are* declarative macros; only the filter chain remains a runtime builder. Declarative where it collapses boilerplate, explicit where the graph is the point: that balance is the whole design. -## Verifying the crate +## Step 13 — Verify the crate Everything above compiles and is tested. From the workspace root: ```bash cargo build -p firefly-sample-lumen -cargo test -p firefly-sample-lumen # 42 unit + 12 HTTP + 1 doctest -cargo test -p firefly-sample-lumen --features streaming # + 3 streaming tests +cargo test -p firefly-sample-lumen # 42 unit + 12 HTTP = 54 tests +cargo test -p firefly-sample-lumen --features streaming # 57 tests (+3 streaming) cargo clippy -p firefly-sample-lumen --all-targets -- -D warnings ``` -The HTTP tests drive the framework-assembled router in-process — `build_router()` +The HTTP tests drive the framework-assembled router in-process: `build_router()` bootstraps a `FireflyApplication` (auto-mounting the controller, draining the -handlers/listener, layering security) and returns its public router — through -`tower::ServiceExt::oneshot` (no socket bound), proving the auto-mounted routes, -the CQRS handlers, validation (422), the not-found path (404), the auth boundary -(401), the transfer saga (happy + compensation), and the projection convergence -all work end to end — every prose listing in this book is a slice of that running -crate. - -## What changed in Lumen - -Nothing — this chapter is the retrospective, not a new feature. Re-read as a -catalogue, Lumen's macros are: three `#[derive(DomainEvent)]` + one -`#[derive(AggregateRoot)]` (`domain.rs`); the `WalletProjection` -`#[derive(Service)]` bean with `#[handlers]` + one `#[event_listener]` -(`ledger.rs`); three `#[derive(Command)]` + one `#[derive(Query)]` + the -`WalletHandlers` `#[derive(Service)]` bean with `#[handlers]` + three -`#[command_handler]` + one `#[query_handler]` (`commands.rs`); the declarative -orchestration set — -`#[firefly::saga]` + `#[saga_step]` (`transfer.rs`), `#[firefly::workflow]` + -`#[workflow_step]` (`compliance.rs`), and `#[firefly::tcc]` + `#[participant]` -(`tcc_transfer.rs`); one `#[rest_controller]` with seven verb methods (`web.rs`); -and one `#[scheduled]` (`housekeeping.rs`). Each replaced a chunk of -hand-written wiring with a declaration next to the code — and all of it arrived -through one dependency and one prelude glob. +handlers/listener, layering security) and returns its public router, exercised +through `tower::ServiceExt::oneshot` with no socket bound. They prove the +auto-mounted routes, the CQRS handlers, validation (422), the not-found path +(404), the auth boundary (401), the transfer saga (happy + compensation), the +compliance workflow, the TCC transfer, and the projection convergence all work end +to end — every prose listing in this book is a slice of that running crate. + +> **Tip** **Checkpoint.** All three commands succeed: build is clean, the default +> test run reports `54 passed`, the streaming run reports `57 passed`, and clippy +> is silent under `-D warnings`. That is the whole declarative crate, verified. + +## Recap + +- A **declarative macro** in Firefly expands at compile time into the `impl`s, + routers, and registrations you would otherwise hand-write — checked by the + compiler, never discovered by runtime reflection. +- The macros Lumen uses, file by file: `#[derive(Command/Query/Schema)]` and + `#[handlers]` (`commands.rs`); `#[derive(DomainEvent/AggregateRoot)]` + (`domain.rs`); `#[derive(Repository/Service)]` + `#[event_listener]` + (`ledger.rs`); `#[derive(Configuration)]` + `#[bean]` and + `#[derive(Controller)]` + `#[rest_controller]` (`web.rs`); `#[scheduled]` + (`housekeeping.rs`); and the orchestration trio `#[firefly::saga]` / + `#[firefly::workflow]` / `#[firefly::tcc]`. +- The supporting set Lumen does not use is still first-class: + `#[derive(Builder/Mapper/Validate)]`, the relational + `#[derive(Entity/SqlxRepository)]` / `#[firefly::repository]` / + `#[firefly::transactional]` (with `propagation` / `isolation` / `read_only` / + `timeout_ms` / `manager`, plus `no_rollback_for` / `rollback_only_for`), the + method-security and resilience decorators, `#[cacheable]`, `#[async_method]`, + the in-process event listeners, `#[aspect]`, and `#[http_client]`. +- Macro-generated code names runtime types through the hidden `__rt` contract + path, which is why a one-crate service compiles whatever a macro expands to. +- The **inventory drain** is what turns a declared bean, listener, task, or + controller into wired behaviour at boot — so Lumen writes no `register`, + `subscribe`, `schedule`, or `routes` call by hand. + +This chapter added no feature; it re-read Lumen as a catalogue. Every macro +replaced a chunk of hand-written wiring with a declaration next to the code, and +all of it arrived through one dependency and one prelude glob — the thesis the +running crate proves. ## Exercises 1. **Trace one macro end to end.** Pick `#[derive(Query)]` on `GetWallet`. Find - where its generated `cache_ttl()` is read (the `QueryCache` middleware in + where its generated `cache_ttl()` is read (the `QueryCache` invalidation in `web.rs`) and the test that asserts it (`get_wallet_carries_cache_ttl` in - `commands.rs`). Change the TTL to `"5s"` and re-run the tests. -2. **Add a verb.** Add a `#[get("/wallets/:id/events")]`-style read method to the - `#[rest_controller]` impl (non-streaming: return the event list as JSON) and - confirm the auto-mounted controller serves it with no other change. -3. **Add a scheduled task.** Write a second `#[scheduled(cron = "0 0 * * *")]` - function in `housekeeping.rs` and assert it appears in `scheduler.tasks()` — - the framework drains the new `ScheduledRegistration` from inventory, so you add - no registration call. -4. **Count the wiring you didn't write.** For each macro in the catalogue, name - the helper or impl it generated (`register_*`, `subscribe_*`, `schedule_*`, - `routes`, `EVENT_TYPE`, `AGGREGATE_TYPE`). That list is the boilerplate the - declarative layer wrote for you. - -That is Lumen, complete and declarative. The appendices that follow are -reference: a [module index](./91-appendix-modules.md) and a -[glossary](./92-glossary.md). + `commands.rs`). Change the TTL to `"5s"` and re-run + `cargo test -p firefly-sample-lumen get_wallet_carries_cache_ttl`. +2. **Add a verb.** Add a `#[get("/wallets/:id/balance")]`-style read method to the + `#[rest_controller]` impl in `web.rs` (return the balance as JSON, dispatching + `GetWallet` through the bus) and confirm the auto-mounted controller serves it + with no other change — no `routes()` edit, no registration call. +3. **Add a scheduled task.** Write a second `#[scheduled(cron = "0 0 * * * *")]` + function in `housekeeping.rs` and assert it appears in `scheduler.tasks()` + alongside `ledger_heartbeat` — the framework drains the new + `ScheduledRegistration` from inventory, so you add no registration call. +4. **Read the inventory in the startup report.** Run Lumen and find the + `:: cqrs handlers … | event listeners … | scheduled tasks … | controllers … ::` + line. Count each against the beans you read in this chapter, then add the + verb from exercise 2 and watch the controller route count follow. +5. **Count the wiring you didn't write.** For each macro in + [Step 2](#step-2--the-macro-catalogue-mapped-to-lumen-files)'s table, name the + helper or impl it generated (`register_*`, `subscribe_*`, `schedule_*`, + `routes`, `EVENT_TYPE`, `AGGREGATE_TYPE`, the `Message` impl). That list is the + boilerplate the declarative layer wrote for you. + +## Where to go next + +- Compose Lumen's single crate into a multi-crate, layered service in + **[Layered Microservices](./22-layered-microservices.md)** — where the + `lumen-ledger` sample (with the `#[firefly::transactional]` use case from Step + 10) splits domain, core, web, and models into separate crates. +- Revisit how the framework scans and wires the beans this chapter declared in + the **[Dependency Injection deep-dive](./04a-dependency-injection.md)**. +- The appendices are reference: a **[Module Index](./91-appendix-modules.md)** of + every `firefly-*` crate and a **[Glossary](./92-glossary.md)** of the terms used + throughout the book. diff --git a/docs/book/src/22-layered-microservices.md b/docs/book/src/22-layered-microservices.md index 4c37341..e27698e 100644 --- a/docs/book/src/22-layered-microservices.md +++ b/docs/book/src/22-layered-microservices.md @@ -1,158 +1,613 @@ # Layered Microservices -The samples so far live in a single crate. Real services — like the ones in the -[firefly-oss](https://github.com/firefly-oss) core-banking platform — are split -into **layered modules**, each a separately-compiled unit with one job, so the -public contract can be published without the persistence code, the business -logic can be tested without the web stack, and an SDK consumer pulls in only the -DTOs. +Every Lumen sample so far has lived in a *single* crate. That is the right shape +while you are learning one subsystem at a time, but it is not how a production +core-banking service is actually built. Real services — like the ones in the +[firefly-oss](https://github.com/firefly-oss) platform — are split into +**layered modules**, each a separately-compiled unit with exactly one job: the +public contract can be published without dragging in the persistence code, the +business logic can be unit-tested without the web stack, and an external SDK +consumer pulls in only the DTOs and nothing else. + +In this chapter you build that shape. `lumen-ledger` (under +[`samples/lumen-ledger/`](https://github.com/fireflyframework/fireflyframework-rust/tree/main/samples/lumen-ledger)) +is a wallet/ledger microservice organised as **five crates** — the Rust analog +of a multi-module Maven project — laid out Java-style with **one public type per +file** under a `/v1` package path. It reuses every framework idea you +already met (DI beans, the sqlx repository, transactions, validation, RFC 9457 +problems, OpenAPI) and shows how they compose *across a crate boundary* through +discovery alone. + +By the end of this chapter you will: + +- Lay out a service as five layered crates with the dependency arrows running + strictly inward, and know which framework stereotype belongs to each layer. +- Declare a Spring Data-style `@Repository` over a real `@Entity` with two + derives — no factory, no hand-written CRUD — built from an **async datasource + bean**. +- Write a `@Service` that programs against the repository's + `ReactiveCrudRepository` trait, runs an **atomic transfer** under + `#[transactional]`, and translates a filter into a runtime `Specification`. +- Wire the whole graph with a single `firefly::link!` line and guard it with + `assert_discovered`, then run and test the service in-process. +- Hand a typed SDK to downstream callers — written by hand against the shared + DTOs, or generated from the live OpenAPI document. + +## Concepts you will meet + +Before the first crate, here are the ideas this chapter leans on. Each is +reintroduced in context where it is first used; this is the short version. + +> **Note** **Key term — layered module.** A *layered module* is a separately +> compiled crate that owns exactly one architectural concern — the public +> contract, the persistence model, the business logic, the web surface, or an +> outbound client. Splitting a service this way is the Rust equivalent of a +> Maven multi-module project: each module compiles, tests, and versions on its +> own, and lower layers never import higher ones. + +> **Note** **Key term — stereotype.** A *stereotype* is the role a bean plays in +> the application — controller, service, repository, component, configuration. +> Firefly marks each with its own derive (`#[derive(Controller)]`, +> `#[derive(Service)]`, …) exactly as Spring marks them with `@RestController`, +> `@Service`, `@Repository`, `@Component`, `@Configuration`. The framework +> classifies every discovered bean by its stereotype in the `/actuator/beans` +> report. + +> **Note** **Key term — link-time discovery.** Firefly discovers beans, +> controllers, and schemas at *link time* using the `inventory` crate: each +> macro registers an entry the linker collects into the final binary. The catch +> is that a Rust linker **dead-strips** any crate the binary never references — +> a `Cargo.toml` dependency alone is not a reference. The `firefly::link!` macro +> supplies that reference so a layer crate's registrations survive into the +> binary. + +## Step 1 — Lay out the five crates + +The first decision is the module boundaries. `lumen-ledger` uses five, one per +concern, named after the firefly-oss convention: + +| Crate | Holds | Stereotype it contributes | +|---|---|---| +| `firefly-sample-lumen-ledger-interfaces` | DTOs (`#[derive(Schema, Validate)]`) + the `WalletStatus` enum — the public contract | — (pure data) | +| `firefly-sample-lumen-ledger-models` | the `Wallet` `@Entity` + the sqlx `WalletRepository` + the datasource `@Configuration` | `@Entity`, `@Repository`, `@Bean` | +| `firefly-sample-lumen-ledger-core` | the `@Service`, the `@Mapper`, a `@Component` | `@Service`, `@Component` | +| `firefly-sample-lumen-ledger-web` | the `@RestController` + the `FireflyApplication` binary | `@RestController` | +| `firefly-sample-lumen-ledger-sdk` | a typed outbound client over the API | — (a client library) | + +Each crate sets a short library name so code reads cleanly across the boundary — +`lumen_ledger_interfaces`, `lumen_ledger_models`, `lumen_ledger_core`, +`lumen_ledger_sdk` — while the package name stays fully qualified for publishing. +For example the `-interfaces` `Cargo.toml`: + +```toml +[package] +name = "firefly-sample-lumen-ledger-interfaces" + +[lib] +name = "lumen_ledger_interfaces" +path = "src/lib.rs" + +[dependencies] +firefly = { workspace = true } +serde = { workspace = true } +``` -The `lumen-ledger` sample (`samples/lumen-ledger/`) is a wallet/ledger -microservice built exactly this way: **five crates**, the Rust analog of a -firefly-oss Maven multi-module project, organised Java-style with **one public -type per file** under a `/v1` package path. +The dependency arrows run strictly **inward**: + +```text +interfaces ← models ← core ← web + ↑ + sdk +``` -## The five crates +A lower layer never depends on a higher one. The `-web` crate knows the +`-core` service; the service knows the `-models` repository; the repository +knows the `-interfaces` contract — and the contract crate knows nobody. The +`-sdk` depends only on `-interfaces`, so a caller links the DTOs without ever +pulling in the persistence or web code. Concretely, `-models` depends on +`-interfaces`, `-core` depends on both, and `-web` depends on all three: + +```toml +# firefly-sample-lumen-ledger-web/Cargo.toml +[dependencies] +firefly = { workspace = true, features = ["admin", "data-sqlx"] } +firefly-sample-lumen-ledger-interfaces = { path = "../interfaces" } +firefly-sample-lumen-ledger-models = { path = "../models" } +firefly-sample-lumen-ledger-core = { path = "../core" } +``` -| Crate (`firefly-sample-lumen-ledger-*`) | Spring/firefly-oss analog | Holds | -|---|---|---| -| `…-interfaces` | `-interfaces` | DTOs (`#[derive(Schema, Validate)]`) + enums — the public contract | -| `…-models` | `-models` | the `Wallet` entity + the sqlx `WalletRepository` | -| `…-core` | `-core` | the `@Service`, the `@Mapper`, a `@Component` | -| `…-web` | `-web` | the `@RestController` + the `FireflyApplication` binary | -| `…-sdk` | `-sdk` | a typed outbound client over the API | +> **Note** **Key term — one type per file.** Each leaf file holds exactly one +> `struct` / `trait` / `enum` +> (`dtos/wallet/v1/wallet_response.rs` → `WalletResponse`), matching Java's +> one-class-per-file convention. The intermediate `mod` files +> (`dtos/wallet/v1.rs`) just re-export their leaves, and each crate's `lib.rs` +> adds flat convenience re-exports (`pub use services::wallet::v1::WalletService;`) +> so a consumer writes `lumen_ledger_core::WalletService`, not the full path. -The dependency arrows run **inward**: `interfaces ← models ← core ← web`, and -`sdk ← interfaces`. A lower layer never depends on a higher one — the web crate -knows the service, the service knows the repository, but the contract crate -knows nobody. Each leaf file holds exactly one `struct`/`trait`/`enum` -(`dtos/wallet/v1/wallet_response.rs` → `WalletResponse`), matching Java's -one-class-per-file convention; the intermediate module files just re-export. +> **Tip** **Checkpoint.** You can picture the tree before writing a line: five +> directories under `samples/lumen-ledger/`, each with its own `Cargo.toml`, and +> a `src//v1/` package path inside. The arrows above tell you which +> `Cargo.toml` may list which — if you ever find a lower crate importing a higher +> one, the layering is wrong. -## One stereotype per layer +## Step 2 — Map the stereotypes to the layers -Each layer contributes the framework stereotypes that belong to it. Every type -is a **DI bean** discovered by `container.scan()` — there is no composition root. +Before any code, fix in your head which framework stereotype each layer +contributes. Every type below is a **DI bean** the framework discovers during +`container.scan()` — there is no composition root assembling them by hand, just +as there was none in [Quickstart](./02-quickstart.md). ```text -@RestController (web) → #[rest_controller] + #[derive(Controller)] +@RestController (web) → #[rest_controller] + #[derive(Controller)] WalletController │ autowires @Service (core) → #[derive(Service)] + #[firefly(provides = "dyn WalletService")] │ autowires -@Mapper (core) → #[derive(Component)] WalletMapper (DTO ↔ entity) -@Component (core) → #[derive(Component)] WalletNumberGenerator -@Repository (models) → #[derive(SqlxRepository)] WalletRepository (built from the Db @Bean) +@Mapper (core) → #[derive(Component)] WalletMapper (DTO ↔ entity) +@Component (core) → #[derive(Component)] WalletNumberGenerator +@Repository (models) → #[derive(SqlxRepository)] WalletRepository (built from the Db @Bean) │ over -@Entity (models) → #[derive(Entity)] Wallet (generates the SqlxEntity mapping) +@Entity (models) → #[derive(Entity)] Wallet (generates the SqlxEntity mapping) │ from @Bean (DataSource)(models) → #[bean] async fn data_source() -> Db ``` +> **Note** **Key term — autowiring across crates.** *Autowiring* asks the +> container for a collaborator by type instead of constructing it yourself +> (Spring's `@Autowired`). Discovery is link-time, not per-crate, so an +> `#[autowired]` field in the `-web` controller is satisfied by a `@Service` +> bean declared in `-core`, which in turn autowires a `@Repository` from +> `-models`. The wiring crosses crate boundaries with no extra ceremony — once +> the crates are linked (Step 6), the graph is one container. + The `@Service` programs against the repository's **`ReactiveCrudRepository`** -trait (`save`, `find_by_id`, `delete_by_id`, `count`, … returning `Mono`/`Flux`) +trait (`save`, `find_by_id`, `delete_by_id`, `count`, … returning `Mono` / `Flux`) plus the `#[firefly::repository]` derived queries — `find_by_owner`, `find_by_status(.., Pageable)` (paged), and `count_by_status` — the same Spring -Data surface, generated from the method names. +Data surface, generated from the method names. You met all of these in +[Persistence](./07-persistence.md); here they simply live one crate down. -## The repository, declared Spring Data-style +## Step 3 — Declare the entity, Spring Data-style -A Spring Data repository is a *declaration*: you write the interface, the -framework supplies the implementation. `lumen-ledger` does the same with two -derives — no factory, no hand-written CRUD. +Start at the bottom — the `-models` crate. A Spring Data repository is a +*declaration*: you write the interface, the framework supplies the +implementation. `lumen-ledger` does the same with two derives, and the first is +on the entity. -The entity declares its `@Table` / `@Id` / `@Version` / `@Column` mapping with -`#[derive(Entity)]` — just annotated fields, the JPA `@Entity` experience: +> **Note** **Key term — entity.** An *entity* is the persisted shape of a domain +> object — one row of one table. `#[derive(Entity)]` generates the +> `@Table` / `@Id` / `@Version` / `@Column` mapping from the struct's fields, the +> JPA `@Entity` experience: scalar columns map automatically, and annotated +> fields opt into the special roles (primary key, version, audit timestamps). + +Create `models/src/entities/wallet/v1/wallet.rs`: ```rust,ignore -#[derive(Entity)] +use chrono::{DateTime, Utc}; +use lumen_ledger_interfaces::WalletStatus; +use uuid::Uuid; + +#[derive(Debug, Clone, PartialEq, Eq, firefly::Entity)] #[firefly(table = "wallets")] pub struct Wallet { - #[firefly(id)] pub id: Uuid, + #[firefly(id)] + pub id: Uuid, pub account_number: String, - pub balance: i64, - // an enum maps via an explicit converter (the @Enumerated(STRING) boundary) + pub owner: String, + pub balance: i64, // minor units (cents) + pub currency: String, // ISO-4217 code + // The typed enum maps via an explicit converter — the @Enumerated(STRING) boundary. #[firefly(with(read = "WalletStatus::from_token", write = "WalletStatus::as_str"))] pub status: WalletStatus, - #[firefly(version)] pub version: i64, // @Version - pub created_at: DateTime, // @CreatedDate (auditor-stamped) - pub updated_at: DateTime, // @LastModifiedDate + #[firefly(version)] + pub version: i64, // @Version — bumped by the store on update + pub created_at: DateTime, // @CreatedDate, stamped on insert + pub updated_at: DateTime, // @LastModifiedDate, stamped on every write } ``` -Scalar columns (`String`, `i64`/`i32`, `bool`, `f64`, `Uuid`, `DateTime`) -map automatically; `#[firefly(column = "name")]` renames one. - -The repository is then **one annotation** — `#[derive(SqlxRepository)]` over a -struct holding the entity's repository: +What just happened: the derive read the struct and produced the table mapping. +Scalar columns (`String`, `i64` / `i32`, `bool`, `f64`, `Uuid`, `DateTime`) +map automatically, with `Uuid` and `DateTime` persisted as text; +`#[firefly(column = "name")]` would rename one. The `WalletStatus` enum is *not* +scalar, so it carries an explicit `with(read = …, write = …)` converter — the +read direction (`from_token`) and the write direction (`as_str`) — which is the +JPA `@Enumerated(STRING)` boundary made explicit. The `#[firefly(id)]`, +`#[firefly(version)]`, and the two timestamp fields opt into the special roles: +the store stamps the version and timestamps for you, so the service never touches +them. + +> **Note** **Key term — optimistic locking.** *Optimistic locking* lets two +> readers load the same row, then makes the *second* writer fail if the first +> already changed it — detected by comparing the `@Version` column. No row is +> ever locked for reading; the conflict is caught at write time. A stale write +> surfaces as Spring's `OptimisticLockingFailureException`, which this service +> turns into a `409`. + +## Step 4 — Declare the repository with one derive + +The repository is **one annotation** — `#[derive(SqlxRepository)]` over a struct +whose only field is the framework's reactive repository — plus an optional block +of *derived queries*. + +> **Note** **Key term — derived query.** A *derived query* is a finder whose SQL +> the framework generates from the method's *name* — `find_by_owner`, +> `count_by_status`, `find_by_status(.., Pageable)` — exactly like Spring Data's +> `findByOwner`. You write the signature and leave the body unimplemented; the +> `#[firefly::repository]` macro replaces it. + +Create `models/src/repositories/wallet/v1/wallet_repository.rs`: ```rust,ignore -#[derive(SqlxRepository)] +use firefly::data::{DataError, Pageable}; +use firefly::data_sqlx::SqlxReactiveRepository; +use uuid::Uuid; + +use crate::entities::wallet::v1::Wallet; + +#[derive(firefly::SqlxRepository)] pub struct WalletRepository { repo: SqlxReactiveRepository, } -#[firefly::repository] // the derived queries, on top +#[firefly::repository] // the derived queries, on top impl WalletRepository { - pub async fn find_by_owner(&self, owner: &str) -> Result, DataError> { unimplemented!() } - // find_by_status(.., Pageable), count_by_status, … + /// `SELECT … WHERE owner = ?` + pub async fn find_by_owner(&self, owner: &str) -> Result, DataError> { + unimplemented!() + } + + /// `SELECT COUNT(*) WHERE status = ?` + pub async fn count_by_status(&self, status: &str) -> Result { + unimplemented!() + } + + /// Paged `SELECT … WHERE status = ?` — ORDER BY / LIMIT / OFFSET come from + /// the trailing `Pageable`. + pub async fn find_by_status( + &self, + status: &str, + page: Pageable, + ) -> Result, DataError> { + unimplemented!() + } +} +``` + +What just happened, and why it matters: that one derive does three things at +once. It **registers `WalletRepository` as a `@Repository` bean** (discovered by +the scan, classified correctly in `/actuator/beans`); it **builds the inner +`SqlxReactiveRepository` from the autowired `Db`** datasource — wiring the entity's +`@Version` optimistic locking and `@CreatedDate` / `@LastModifiedDate` auditing +from the `SqlxEntity` mapping the entity derive emitted; and it **implements +`ReactiveCrudRepository` *and* `ReactiveSpecificationRepository` by delegation**. +That last point is what lets the service in Step 5 call `save`, `find_by_id`, +`delete_by_id`, and `find_by_spec` without you writing any of them. + +The `#[firefly::repository]` block adds the derived queries on top: each method +body is `unimplemented!()` in your source, and the macro replaces it with SQL +generated from the method name. No `#[bean]` factory, no hand-written CRUD — the +struct's only state is the inner repository, and the derive builds it. + +> **Tip** **Checkpoint.** Two derives, zero CRUD bodies, and the only field is +> `repo: SqlxReactiveRepository`. If you find yourself writing a +> `save` or a `SELECT` by hand here, step back — the derive already supplies the +> canonical surface. + +### The key type is generic, like Java + +Spring Data's `CrudRepository` leaves `ID` unbounded. Firefly's sqlx +repository accepts **any `Serialize` key** through the `SqlKey` trait +(blanket-implemented), so the wallet repository keys on `Uuid` directly: + +```rust,ignore +pub struct WalletRepository { + repo: SqlxReactiveRepository, } ``` -That derive registers `WalletRepository` as a **`@Repository` bean** (discovered -by the scan, classified correctly in `/beans`), **builds the inner repository -from the injected `Db`** — wiring `@Version` optimistic locking (a stale write -→ `409`) and `@CreatedDate`/`@LastModifiedDate` auditing from the entity — and -implements `ReactiveCrudRepository` by delegation, so the service programs -against the canonical CRUD surface plus the derived queries. +`Uuid`, `i64`, `String`, an enum, or a composite-key struct all work — the key +binds as its serde-JSON form against the id column. Nothing about the repository +is hard-coded to UUIDs. + +## Step 5 — Open the datasource as an async bean + +The repository is a *synchronous* bean: building it is just wrapping the `Db` +handle. What actually performs I/O at startup is the **datasource** — Spring +Boot's auto-configured `DataSource`. In `lumen-ledger` it is an **async `@Bean`** +on the `-models` `@Configuration`. -### The datasource is the async bean +> **Note** **Key term — async bean.** An *async bean* is a bean whose factory is +> an `async fn` — it must `await` work (open a pool, dial a broker) before the +> bean exists. The framework parks such a factory during the synchronous +> `container.scan()` and `await`s it during `Container::init_async_beans()`, run +> by the bootstrap right after the scan. This is Spring Boot's pattern of a +> `@Bean` that performs I/O at context-refresh time, except the I/O is awaited +> instead of blocking a thread. -What *is* an async bean is the **datasource** — Spring Boot's auto-configured -`DataSource`. The `-models` `@Configuration` opens the pool with `await`: +Create `models/src/config/wallet_persistence_config.rs`: ```rust,ignore +use firefly::data_sqlx::Db; +use firefly::prelude::*; + +#[derive(Configuration, Default)] +pub struct WalletPersistenceConfig; + #[firefly::bean] impl WalletPersistenceConfig { + /// The `Db` datasource bean — an async factory that opens the pool and + /// applies the schema with `await`. #[bean] - async fn data_source(&self) -> Db { // the DataSource @Bean - connect_and_migrate().await // open pool + apply schema + async fn data_source(&self) -> Db { + connect_and_migrate().await // open pool + apply schema } } ``` -The framework parks this factory during the synchronous `container.scan()` and -`await`s it during `Container::init_async_beans()` (run by the bootstrap right -after the scan). The `#[derive(SqlxRepository)]` repository — a *synchronous* -bean — then resolves that ready `Db` and builds itself from it. This is the -Spring Boot pattern of a `@Bean` that performs I/O at context-refresh time, -except the I/O is `await`ed instead of blocking a thread. By default the -datasource opens an in-memory SQLite database (the sample runs and tests with no -external server); set `DATABASE_URL=postgres://…` for real PostgreSQL. +What just happened: the `#[bean] async fn data_source` is parked during the scan, +then `await`ed during `init_async_beans()`. Because the datasource is *ready* +before any synchronous bean resolves, the `#[derive(SqlxRepository)]` repository +(a synchronous bean that autowires `Db`) finds a live pool when the framework +builds it. A construction error here aborts startup — fail-fast, surfaced through +`Container::init_async_beans` as a `BeanCreation` error. + +By default `connect_and_migrate()` opens an **in-memory SQLite database**, so the +sample runs and tests with no external server. Set `DATABASE_URL=postgres://…` +and it targets real PostgreSQL instead — the only environment dependency in the +whole sample, and it is optional. + +> **Design note.** Why does the service own its transaction manager (Step 7) but +> the datasource live here? Because the registry of process-global transaction +> managers is *first-wins*, and this sample's test suite boots one isolated +> in-memory database **per test**. A single global manager would cross-contaminate +> them. The datasource bean is fine to share — every consumer resolves the same +> `Db` — but the transaction boundary is bound to a per-instance manager so each +> test stays hermetic. A single-datasource production service may equally register +> one manager at startup and use a bare `#[firefly::transactional]`. + +> **Tip** **Checkpoint.** The `-models` crate now has three files of substance: +> the entity, the repository, and the config. `cargo test -p +> firefly-sample-lumen-ledger-models` exercises the repository directly against +> an isolated in-memory database — including the derived queries and a real +> `@Version` optimistic-lock conflict (a stale write detected with +> `firefly::data_sqlx::is_optimistic_lock`). + +## Step 6 — Write the service against the repository trait + +Move up to `-core`. The `@Service` is the business layer: it autowires the +repository, the mapper, and the number generator, and programs against the +repository's *trait* surface — never its concrete SQL. + +The service is published as a **port** so the controller depends on an interface, +not a struct: -## Any key type — generic like Java +```rust,ignore +use std::sync::Arc; + +use firefly::prelude::*; +use firefly::data_sqlx::Db; + +#[derive(Service)] +#[firefly(provides = "dyn WalletService")] +pub struct WalletServiceImpl { + #[autowired] repository: Arc, + #[autowired] mapper: Arc, + #[autowired] numbers: Arc, + #[autowired] db: Arc, // for the service's own transaction manager +} +``` -Spring Data's `CrudRepository` leaves `ID` unbounded. Firefly's sqlx -repository accepts any **`Serialize`** key through the `SqlKey` trait -(blanket-implemented), so the wallet repository keys on `Uuid` directly: +> **Note** **Key term — provided port.** `#[firefly(provides = "dyn WalletService")]` +> registers the impl under the *trait object* type, so anyone who autowires +> `Arc` (the controller, a test) receives this bean. The +> trait is the published port; the struct is a hidden adapter — Spring's +> "program to an interface, inject the implementation". + +The simple read paths just delegate to the repository's `ReactiveCrudRepository` +trait and map the result through the `@Mapper`: ```rust,ignore -pub struct WalletRepository { repo: SqlxReactiveRepository } -impl ReactiveCrudRepository for WalletRepository { /* … */ } +async fn get(&self, id: Uuid) -> Result { + let wallet = self + .repository + .find_by_id(id) + .await + .map_err(|e| ServiceError::Backend(e.to_string()))? + .ok_or(ServiceError::NotFound)?; + Ok(self.mapper.to_response(&wallet)) +} + +async fn list_by_owner(&self, owner: &str) -> Result, ServiceError> { + let wallets = self + .repository + .find_by_owner(owner) // the derived query + .await + .map_err(|e| ServiceError::Backend(e.to_string()))?; + Ok(wallets.iter().map(|w| self.mapper.to_response(w)).collect()) +} ``` -`Uuid`, `i64`, `String`, an enum, or a composite-key struct all work — the key -binds as its serde-JSON form against the id column. +What just happened: `find_by_id` comes from the `ReactiveCrudRepository` trait the +derive implemented; `find_by_owner` is the derived query from the +`#[firefly::repository]` block. The service never sees SQL — it sees a repository +that already speaks its domain. -## Linking the crates — `firefly::link!` +> **Note** **Key term — mapper.** A *mapper* translates between layers — here the +> `-models` `Wallet` entity and the `-interfaces` `WalletResponse` DTO. Because +> the two types live in *different* crates, Rust's orphan rule forbids +> `impl From for WalletResponse` in `-core`. So `WalletMapper` is a +> hand-written `#[derive(Component)]` bean with a `to_response(&self, &Wallet) -> +> WalletResponse` method — exactly the shape MapStruct's `@Mapper` generates. -Because discovery is link-time (`inventory`), the linker will **dead-strip** a -layer crate's bean/controller/schema registrations unless the binary references -that crate. A Cargo dependency is not a reference. The `-web` binary force-links -each layer with `firefly::link!`: +### Filtering with a runtime Specification + +The `search` use case shows the framework's `Specification` — the Spring Data +`JpaSpecificationExecutor` analog. The service turns each *present* filter field +into an AND-combined predicate, then runs the composed specification: + +```rust,ignore +use firefly::data::{Op, Predicate, ReactiveSpecificationRepository, Specification}; + +async fn search(&self, filter: WalletFilter) -> Result, ServiceError> { + // At least one criterion is required — a no-filter search would be an + // unscoped list-every-wallet enumeration. + if filter.owner.is_none() + && filter.currency.is_none() + && filter.status.is_none() + && filter.min_balance.is_none() + && filter.max_balance.is_none() + { + return Err(ServiceError::Validation("provide at least one filter criterion".into())); + } + + let mut spec = Specification::all(); + if let Some(owner) = filter.owner { + spec = spec.and(Specification::eq("owner", owner)); + } + if let Some(min) = filter.min_balance { + spec = spec.and(Specification::pred(Predicate::new("balance", Op::Gte, min))); + } + // …currency, status, max_balance the same way… + + let wallets = self + .repository + .find_by_spec(spec) // from ReactiveSpecificationRepository + .collect_list() + .block() + .await + .map_err(|e| ServiceError::Backend(e.to_string()))? + .unwrap_or_default(); + Ok(wallets.iter().map(|w| self.mapper.to_response(w)).collect()) +} +``` + +What just happened: `find_by_spec` comes from `ReactiveSpecificationRepository` +(the *other* trait the `SqlxRepository` derive implemented). It returns a `Flux`, +so `.collect_list().block().await` gathers it. `block()` returns +`Result>, _>`, so `.unwrap_or_default()` turns the "no rows" +`None` into an empty `Vec`. The framework compiles the `Specification` to a +dialect-aware `WHERE`, so the same service code runs unchanged on SQLite or +PostgreSQL. + +### The atomic transfer, under one transaction + +The transfer is the heart of a ledger: debit the source and credit the +destination, both-or-neither. That demands a transaction. + +> **Note** **Key term — transactional boundary.** `#[firefly::transactional]` +> wraps a method so every write inside it commits together or rolls back +> together — Spring's `@Transactional`. The `manager = "self.tx_manager()"` +> argument binds the boundary to a manager the *service* owns (evaluated per +> call) rather than the process-global registry. The attribute lives on an +> inherent method (`transfer_tx`), because an `async-trait` method cannot carry +> it cleanly; the trait method just delegates. ```rust,ignore -// crate root of the -web binary -firefly::link!(lumen_ledger_core, lumen_ledger_models, lumen_ledger_interfaces); +use firefly::data_sqlx::SqlxTransactionManager; +use firefly::transactional::TransactionManager; + +impl WalletServiceImpl { + fn tx_manager(&self) -> Arc { + Arc::new(SqlxTransactionManager::new((*self.db).clone())) + } + + #[firefly::transactional(manager = "self.tx_manager()")] + async fn transfer_tx(&self, from: Uuid, to: Uuid, amount: i64) + -> Result + { + if amount <= 0 { return Err(ServiceError::Validation("transfer amount must be positive".into())); } + if from == to { return Err(ServiceError::Validation("cannot transfer to the same wallet".into())); } + + let mut source = self.load_active(from).await?; // 404 if absent, 422 if not active + let mut dest = self.load_active(to).await?; + if source.currency != dest.currency { + return Err(ServiceError::Validation("currency mismatch".into())); + } + if source.balance < amount { + return Err(ServiceError::Validation("insufficient funds".into())); + } + + // Every precondition is checked BEFORE the source is debited, so a + // rejected transfer moves no money. If the credit fails after the debit, + // the transaction rolls the debit back. + source.balance = source.balance.checked_sub(amount) + .ok_or_else(|| ServiceError::Validation("balance underflow".into()))?; + let saved_source = self.persist(source).await?; + dest.balance = dest.balance.checked_add(amount) + .ok_or_else(|| ServiceError::Validation("balance overflow".into()))?; + self.persist(dest).await?; + Ok(saved_source) // the updated source + } +} +``` + +What just happened, and why it matters: `transfer_tx` runs inside a single +transaction bound to `self.tx_manager()`. Every guard (positive amount, distinct +active wallets, matching currency, sufficient funds) fires *before* the first +write, so a rejected transfer never touches a balance. The arithmetic is +`checked_*`, so a ledger overflow is a domain error, not a silent wrap. And if +the credit ever failed after the debit, the boundary rolls the debit back — the +both-or-neither guarantee a ledger lives on. + +> **Note** Because `#[transactional]` requires the error type to be +> `From`, `ServiceError` implements that +> conversion — a transaction-infrastructure failure (begin / commit / rollback) +> surfaces as `ServiceError::Backend`. The `no_rollback_for` / +> `rollback_only_for` arguments to `#[transactional]` (not shown here) let you +> tune which error variants trigger a rollback; the default rolls back on any +> `Err`. + +The `persist` helper centralises the save-and-map, and maps a stale `@Version` +write to a `Conflict`: + +```rust,ignore +async fn persist(&self, wallet: Wallet) -> Result { + let saved = self + .repository + .save(wallet) + .await + .map_err(|e| { + if is_optimistic_lock(&e) { + ServiceError::Conflict("wallet was modified concurrently; retry".into()) + } else { + ServiceError::Backend(e.to_string()) + } + })? + .ok_or_else(|| ServiceError::Backend("save returned no row".into()))?; + Ok(self.mapper.to_response(&saved)) +} +``` + +`deposit` and `withdraw` are plain read-modify-writes that lean on the same +`load_active` + `persist` pair; their concurrency safety comes from the +repository's `@Version` optimistic locking (a stale write → `409`), not a +transaction. + +> **Tip** **Checkpoint.** The `-core` crate now holds the service (with its +> trait, impl, and `ServiceError`), the mapper, and the number generator — every +> one a DI bean, none constructed by hand. The service compiles against +> `-models` and `-interfaces` but knows nothing of `-web`. + +## Step 7 — Wire the crates with `firefly::link!` + +Now the binary. The `-web` crate holds the `@RestController` (Step 8) and the +one-line `FireflyApplication` boot — but a Cargo dependency on the layer crates +is **not enough**. Because discovery is link-time, the linker will dead-strip a +layer crate's bean / controller / schema registrations unless the binary actually +*references* that crate. The `firefly::link!` macro is that reference. + +Create `web/src/main.rs`: + +```rust,ignore +// LINK-TIME WIRING — DO NOT REMOVE. Force-links each layer crate so its beans, +// controllers, and schemas survive dead-code elimination into the binary. +firefly::link!( + lumen_ledger_core, + lumen_ledger_models, + lumen_ledger_interfaces +); + +mod controllers; #[tokio::main] async fn main() -> Result<(), firefly::BoxError> { @@ -163,60 +618,309 @@ async fn main() -> Result<(), firefly::BoxError> { } ``` -`firefly::assert_discovered(&container, min_beans, min_controllers)` guards a -forgotten crate at startup. This one line is the only wiring a layered service -needs; everything else is discovered. +What just happened: `firefly::link!(a, b, c)` expands to `extern crate a as _;` +for each crate, which is exactly the reference the linker needs to keep that +crate's `inventory` registrations. Without it you get the classic "6 of 16 beans" +symptom — the binary compiles, links, runs, and silently drops half its beans. +The `-web` crate itself is referenced (it *is* the binary, and it declares +`mod controllers`), so it does not appear in the `link!` list; the three library +layers do. + +Note that `main` itself is the same one line you wrote in +[Quickstart](./02-quickstart.md) — `FireflyApplication::new(name).run().await` — +just with `.version(firefly::VERSION)` set so `/actuator/info` reports the +framework release. A layered service needs exactly **one** extra line of wiring +(`link!`); everything else is discovered. + +To turn a forgotten `link!` crate from a silent bug into a loud failure, guard +the boot with `assert_discovered`. You call it right after `bootstrap()` returns +(the test seam from Quickstart), using the returned `Bootstrapped::container`: + +```rust,ignore +let app = firefly::FireflyApplication::new("lumen-ledger") + .bootstrap() + .await + .expect("bootstrap"); + +// At least 8 beans (repository, service, mapper, component, config, …) and at +// least 1 controller were discovered — across all three layer crates. +firefly::assert_discovered(&app.container, 8, 1); +``` + +`assert_discovered(&container, min_beans, min_controllers)` panics at startup if +the discovered bean or controller count falls below the floor you assert — the +single most useful check in a layered service. + +> **Tip** **Checkpoint.** `cargo run -p firefly-sample-lumen-ledger-web` boots on +> `:8080` (public) with the management surface on `:8081`, and the startup report +> lists beans drawn from all four code crates. If the bean count looks too small, +> a crate is missing from `firefly::link!`. + +## Step 8 — The production-grade web surface + +The `@RestController` is the last layer, and it is more than CRUD — it carries +the error and validation discipline a Spring Boot service is expected to have, +every failure rendered as RFC 9457 `application/problem+json`. You met each of +these tools in [Your First HTTP API](./06-first-http-api.md) and +[OpenAPI](./06a-openapi.md); here they compose over the layered service. -## A production-grade web surface +The controller is a `#[derive(Controller)]` bean that autowires the `dyn +WalletService` port from `-core` and is auto-mounted by `#[rest_controller]`: -The `@RestController` is more than CRUD — it carries the error and validation -discipline a Spring Boot service is expected to have, all rendered as RFC 9457 -`application/problem+json`: +```rust,ignore +use std::sync::Arc; +use firefly::prelude::*; +use firefly::web::{PageRequest, Path, Query, Valid, WebError, WebResult}; +use lumen_ledger_core::{ServiceError, WalletService}; + +#[derive(Clone, Controller)] +pub struct WalletController { + #[autowired] + service: Arc, +} + +#[rest_controller(path = "/api/v1", tag = "Wallets")] +impl WalletController { + #[post("/wallets", summary = "Open a wallet", status = 201, + header("Idempotency-Key", description = "optional client-supplied key to make retries safe"))] + async fn open( + State(api): State, + headers: axum::http::HeaderMap, + Valid(body): Valid, // 422 on a blank owner / bad currency + ) -> WebResult<(StatusCode, Json)> { + let view = api.service.create(body).await.map_err(service_to_web)?; + Ok((StatusCode::CREATED, Json(view))) + } -| Concern | How | + #[get("/wallets/:id", summary = "Fetch a wallet")] + async fn get( + State(api): State, + Path(id): Path, // 400 on a non-UUID id + ) -> WebResult> { + let view = api.service.get(id).await.map_err(service_to_web)?; + Ok(Json(view)) + } + + #[get("/wallets/page", summary = "List wallets by status (paged)")] + async fn list_paged( + State(api): State, + Query(query): Query, + PageRequest(pageable): PageRequest, // binds page/size/sort + ) -> WebResult>> { + let page = api.service.list_by_status(query.status, pageable).await.map_err(service_to_web)?; + Ok(Json(page)) + } + // …deposit, withdraw, transfer, search, set_status, delete… +} +``` + +What just happened, concern by concern: + +| Concern | How it is handled | |---|---| -| Bean validation at the edge | `Valid` / `Valid` — a blank owner, a non-ISO currency (`#[validate(pattern = "[A-Z]{3}")]`), or a non-positive amount (`#[validate(range(min = 1))]`) is a **422** before the service runs. The same `Validate` runs on query/path objects via `ValidQuery` / `ValidPath`, and a `multipart/form-data` upload binds through the problem-rendering `Multipart` extractor | -| Malformed path / query | `firefly::web::{Path, Query}` extractors — a non-UUID id or a missing `?owner=` is a **400** problem, not axum's plain-text default | -| Atomic transfer | `POST /api/v1/wallets/:id/transfer` debits the source and credits the destination inside **one transaction** — `#[firefly::transactional(manager = "self.tx_manager()")]` on the service. The debit and credit commit together or not at all; a rejected transfer (insufficient funds, inactive party) moves no money. The `manager = …` binds the boundary to a manager the service owns rather than the process-global registry, so the per-test isolated databases stay correct | +| Bean validation at the edge | `Valid` / `Valid` / `Valid` — a blank owner, a non-ISO currency (`#[validate(pattern = "[A-Z]{3}")]`), or a non-positive amount (`#[validate(range(min = 1))]`) is a **422** before the service runs | +| Malformed path / query | the framework's `firefly::web::{Path, Query}` extractors — a non-UUID id or a missing `?owner=` is a **400** problem, not axum's plain-text default | +| Atomic transfer | `POST /api/v1/wallets/:id/transfer` debits the source and credits the destination inside **one transaction** (Step 6). A rejected transfer moves no money | | Optimistic-lock conflict | a stale `@Version` write → `ServiceError::Conflict` → **409** | | Unknown wallet | `ServiceError::NotFound` → **404** | | Status lifecycle | `PATCH /api/v1/wallets/:id/status` transitions `active → frozen → closed`; a frozen wallet rejects a debit with **422** | | Delete | `DELETE /api/v1/wallets/:id` → **204**, delegating to `delete_by_id` | -| Pagination | `GET /api/v1/wallets/page?status=active&page=1&size=20&sort=balance,desc` returns a Spring-Data `Page` (`content` + `totalElements`). The framework's `PageRequest` argument resolver binds `page`/`size`/`sort` into a `Pageable` (exactly like a Spring `Pageable` parameter), which the `@Service` passes straight to the paged `find_by_status` derived query | -| Filtering | `GET /api/v1/wallets/search?owner=¤cy=&status=&minBalance=&maxBalance=` binds a `WalletFilter` query DTO (each field an OpenAPI query parameter); the `@Service` turns the present criteria into a composable `firefly::data::Specification` that `#[derive(SqlxRepository)]`'s `find_by_spec` compiles to a dialect-aware `WHERE` — the Spring Data `JpaSpecificationExecutor` analog. At least one criterion is required (no unscoped list-all) | +| Pagination | `GET /api/v1/wallets/page?status=active&page=1&size=20&sort=balance,desc` returns a Spring-Data `Page` (`content` + `totalElements`). The `PageRequest` resolver binds `page` / `size` / `sort` into a `Pageable` (exactly like a Spring `Pageable` parameter), which the service passes to the paged `find_by_status` derived query | +| Filtering | `GET /api/v1/wallets/search?owner=¤cy=&status=&minBalance=&maxBalance=` binds a `WalletFilter` query DTO (each field an OpenAPI query parameter); the service turns the present criteria into a `Specification` the repository compiles to a dialect-aware `WHERE`. At least one criterion is required | + +> **Note** **Key term — orphan rule.** Rust's *orphan rule* forbids implementing +> a trait for a type when *both* are foreign to the current crate. `WebError` +> (from `firefly`) and `ServiceError` (from `-core`) are both foreign to `-web`, +> so `impl From for WebError` is illegal here. The controller maps +> them with a small free function instead — the same constraint that made the +> `@Mapper` a bean rather than a `From` impl: -Because `WebError` and `ServiceError` are both foreign to the `-web` crate, -the controller maps them with a small `service_to_web` function rather than an -orphan-rule-blocked `impl From for WebError` — a worth-knowing Rust -layering detail the sample documents inline. +```rust,ignore +fn service_to_web(err: ServiceError) -> WebError { + match err { + ServiceError::NotFound => WebError::from(FireflyError::not_found("wallet not found")), + ServiceError::Validation(d) => WebError::from(FireflyError::validation(d)), + ServiceError::Conflict(d) => WebError::from(FireflyError::conflict(d)), + ServiceError::Backend(d) => WebError::from(FireflyError::internal(d)), + } +} +``` -## The SDK and the generator +> **Tip** **Checkpoint.** Every controller handler returns `WebResult`, and +> every domain failure flows through `service_to_web` into a precise problem +> status. Open `http://localhost:8081/swagger-ui` after `cargo run` to see the +> whole wallet surface — bodies, query params, and the declared `Idempotency-Key` +> header — rendered from the inventory. The OpenAPI docs are on the **management** +> port, beside actuator and admin, never on the public API. -The `-sdk` crate is a typed outbound client over `firefly_client::RestClient`, -reusing the `-interfaces` DTOs so a caller never re-declares the contract. You -can also **generate** an equivalent client from the running service's OpenAPI -document: +## Step 9 — Hand callers a typed SDK + +The fifth crate, `-sdk`, is a typed outbound client over +`firefly_client::RestClient`, reusing the `-interfaces` DTOs so a caller never +re-declares the contract. Because `-sdk` depends only on `-interfaces`, importing +it pulls in the DTOs and nothing else — no persistence, no web stack. + +```rust,ignore +use firefly_client::{ClientError, RestBuilder, RestClient, NO_BODY}; +use http::Method; +use lumen_ledger_interfaces::{AmountRequest, CreateWalletRequest, WalletResponse}; + +pub struct WalletClient { + inner: RestClient, +} + +impl WalletClient { + pub fn new(base_url: impl AsRef) -> Self { + Self { inner: RestBuilder::new(base_url).build() } + } + + /// `POST /api/v1/wallets` — open a wallet. + pub async fn create_wallet( + &self, + request: &CreateWalletRequest, + ) -> Result { + self.inner + .request::<_, WalletResponse>(Method::POST, "/api/v1/wallets", Some(request)) + .await + } + + /// `GET /api/v1/wallets/{id}` — fetch one wallet. + pub async fn get_wallet( + &self, + id: impl std::fmt::Display, + ) -> Result { + let path = format!("/api/v1/wallets/{id}"); + self.inner + .request::<(), WalletResponse>(Method::GET, &path, NO_BODY) + .await + } + // …list_wallets, deposit, withdraw… +} +``` + +What just happened: each method maps to one endpoint and (de)serialises the +shared DTOs, so the caller programs against *the same types* the server enforces +— a contract drift fails to compile. Every method returns `Result`; +a non-2xx RFC 9457 body decodes into a typed `FireflyError` reachable via +`ClientError::as_firefly`. The `with_client` constructor wraps an +already-configured `RestClient` (custom headers, retries, timeouts, a bearer +token), the bring-your-own-client path. The full `RestClient` surface is covered +in [HTTP Clients](./13-http-clients.md). + +### Generating the SDK instead + +You can also **generate** an equivalent client from the running service's OpenAPI +document, so you never hand-write a method again: ```bash firefly openapi-client --spec wallet-openapi.json -o src/generated.rs --client-name WalletClient ``` `firefly openapi-client` walks the spec and emits a self-contained client — a -model `struct`/`enum` per `components.schemas` entry and one `async fn` per -operation, with typed path/query parameters and JSON bodies — the Rust analog of -firefly-oss's OpenAPI-generated WebClient SDK. +model `struct` / `enum` per `components.schemas` entry and one `async fn` per +operation, with typed path / query parameters and JSON bodies. The generated file +is headed `// Code generated by \`firefly openapi-client\`. DO NOT EDIT.`. The +full generator catalogue is in [The CLI](./19-cli.md). -## Running it +> **Tip** **Checkpoint.** `cargo test -p firefly-sample-lumen-ledger-sdk` +> compiles the client and runs its contract checks — every method's typed result +> lines up with a shared `-interfaces` DTO. (The network round-trip itself is +> exercised by the `-web` integration test in Step 10.) + +## Step 10 — Run and test the whole graph + +With all five crates in place, run and test the service: ```bash -cargo run -p firefly-sample-lumen-ledger-web # boots on :8080, admin on :8081 -cargo test -p firefly-sample-lumen-ledger-web # in-process cross-crate round-trip +cargo run -p firefly-sample-lumen-ledger-web # boots on :8080, management on :8081 +cargo test -p firefly-sample-lumen-ledger-web # in-process cross-crate round-trip ``` -The integration test boots the whole graph in-process and drives the full -surface — create / fetch / deposit / withdraw, the paged status query, the -status transition, delete, and every problem path (404, the 422 validation -failures, the 400 malformed-path/missing-query) plus the OpenAPI document — -proving every layer wires together through DI alone. The `-models` test -separately proves the `@Version` optimistic-lock conflict (a stale write is -rejected, detected with `firefly::data_sqlx::is_optimistic_lock`). +The integration test boots the whole graph in-process with `bootstrap()` (no +socket bound), asserts discovery with `assert_discovered(&app.container, 8, 1)`, +and drives the full public surface through the returned `api_router` — create / +fetch / deposit / withdraw, the paged status query, the search specification, the +status transition, delete, the atomic transfer (including every rejection path), +and every problem path (404, the 422 validation failures, the 400 +malformed-path / missing-query). It also checks the **management** router: that +the OpenAPI document is served there (and *absent* from the public API), and that +an unknown management path answers an RFC 9457 problem 404 — the same contract as +the public API. + +The point of the test is the architecture, not just the assertions: it proves +every layer wires together through DI alone. The `@RestController` in `-web` +reaches the `@Service` in `-core`, which reaches the `@Repository` in `-models`, +which reaches the `@Bean` datasource — all discovered, none hand-assembled, across +four crate boundaries. + +> **Tip** **Checkpoint.** Both commands succeed. `cargo run` prints a startup +> report whose `:: beans ::` line is drawn from every code crate, and the test +> suite is green — the layered service behaves as one application. + +## Recap — what you built + +You turned a single-crate sample into a five-crate layered microservice without +adding a composition root: + +| Layer | Crate | What it contributes | +|---|---|---| +| contract | `…-interfaces` | DTOs + the `WalletStatus` enum — `#[derive(Schema, Validate)]`, depends on nobody | +| persistence | `…-models` | the `Wallet` `@Entity`, the two-derive `@Repository`, the async datasource `@Bean` | +| business | `…-core` | the `@Service` port, the `@Mapper`, a `@Component`; the atomic `#[transactional]` transfer | +| web | `…-web` | the `@RestController`, the `firefly::link!` wiring, the one-line `FireflyApplication` | +| client | `…-sdk` | a typed `RestClient` over the shared DTOs (or generated from OpenAPI) | + +You also now know: + +- Why the dependency arrows must run strictly inward, and how each layer + contributes exactly the stereotypes that belong to it. +- That a Spring Data-style repository is two derives — `#[derive(Entity)]` and + `#[derive(SqlxRepository)]` — built from an async datasource bean, giving you + `ReactiveCrudRepository` + `ReactiveSpecificationRepository` + derived queries + for free. +- That `firefly::link!` is the *one* line of wiring a layered service needs, that + it exists because discovery is link-time, and that `assert_discovered` turns a + forgotten crate into a loud startup failure. +- How an atomic transfer composes `#[transactional]`, optimistic locking, and a + precondition-first design so a rejected transfer moves no money. +- How to hand callers a typed SDK that reuses the contract crate — or generate + one from the live OpenAPI document. + +## Exercises + +1. **Provoke the dead-strip.** Comment out one crate in the `firefly::link!` + line (say `lumen_ledger_models`), then `cargo run -p + firefly-sample-lumen-ledger-web`. Watch `assert_discovered` fail at startup + with the "discovered N beans but expected at least 8" panic — that is exactly + the bug `link!` prevents. Restore the line. +2. **Trace a request across four crates.** With the service running, `curl -X + POST localhost:8080/api/v1/wallets -H 'content-type: application/json' -d + '{"owner":"ada","currency":"EUR","openingBalance":1000}'`. Name, in order, + which crate handles each hop: the controller (`-web`), the service (`-core`), + the mapper (`-core`), the repository (`-models`), the datasource (`-models`). +3. **Break the transfer atomicity claim.** Read `transfer_tx` and confirm the + currency / funds / active checks all run *before* the first `persist`. Then + `curl` a transfer with `amount` larger than the source balance and verify the + source balance is unchanged afterward (`GET` it) — a rejected transfer moves no + money. +4. **Add a derived query.** Add `find_by_currency(&self, currency: &str) -> + Result, DataError>` to the `#[firefly::repository]` block (body + `unimplemented!()`), expose it through the service and a controller route, and + confirm it works — without writing any SQL. +5. **Generate the SDK.** Run the service, fetch the spec with `curl + localhost:8081/v3/api-docs > wallet-openapi.json`, then `firefly + openapi-client --spec wallet-openapi.json -o /tmp/generated.rs --client-name + WalletClient`. Compare the generated methods to the hand-written `-sdk` + client. + +## Where to go next + +- Drive a fully wired application in-process — the `bootstrap()` seam, the + `api_router` / `management_router`, and cross-crate round-trip tests like the + one in Step 10 — in **[Testing](./18-testing.md)**. +- Revisit the persistence machinery this chapter layered (entities, derived + queries, specifications, optimistic locking) in + **[Persistence & Reactive Repositories](./07-persistence.md)**. +- Take the layered service to production — real PostgreSQL via `DATABASE_URL`, + containers, and the management surface — in + **[Production & Deployment](./20-production.md)**.