Harden database migration and promotion lanes before expansion - PR_26167_207-database-lane-hardening

Author: ToolboxAid

docs_build/database/backup-restore-lane.md

@@ -33,13 +33,25 @@ Validation expectation:
 - the backup file is written outside tracked source paths
 - reports state only the backup path and success/failure, not the connection string
 
+Backup metadata must be recorded without secrets:
+
+- `backupName`
+- `databaseName`
+- `createdAt`
+- `migrationVersion`
+
+Use `schema_migrations` to derive `migrationVersion`; do not use
+`platform_settings` for backup migration state.
+
 ## Restore Checklist
 
 Restore is destructive. Do not run restore unless every checklist item is complete:
 
 - Confirm the target `.env` is the intended database.
 - Run `node .\scripts\validate-runtime-connections.mjs` and record PASS/FAIL.
+- Validate the current `schema_migrations` state before applying restore.
 - Confirm the backup file path and checksum.
+- Confirm the backup metadata `migrationVersion`.
 - Confirm the application is stopped or in a maintenance window.
 - Confirm owner approval.
 - Type or record the exact operator confirmation phrase: `RESTORE CONFIRMED`.
@@ -68,8 +80,10 @@ pg_restore --dbname $databaseUrl --clean --if-exists --single-transaction $backu
 Restore reports must include:
 
 - target host, port, and database name only
+- backup metadata: `backupName`, `databaseName`, `createdAt`, and `migrationVersion`
 - backup file path
 - backup checksum
+- migration state validation before restore
 - confirmation checklist status
 - validation result after restore
 

docs_build/database/promotion-lane.md

@@ -8,6 +8,10 @@ DEV -> IST -> UAT -> PRD
 
 The target name describes where the operator points the configured connection. It must not change application behavior.
 
+This order is mandatory. Do not skip a target, promote backward, or promote a
+later target until the current target's `schema_migrations` state has been
+validated.
+
 ## Copy-Source Flow
 
 Runtime, validation, and migration apply scripts load `.env` only.
@@ -26,9 +30,10 @@ Promotion is manual:
 3. Run `node .\scripts\validate-runtime-connections.mjs`.
 4. Run `node .\scripts\apply-database-ddl.mjs`.
 5. Run `node .\scripts\apply-database-dml.mjs`.
-6. Run `node .\scripts\validate-runtime-connections.mjs` again.
-7. Review `schema_migrations` before promoting the next target.
-8. Start or restart the runtime after validation passes.
+6. Run `node .\scripts\validate-database-drift.mjs`.
+7. Run `node .\scripts\validate-runtime-connections.mjs` again.
+8. Review `schema_migrations` before promoting the next target.
+9. Start or restart the runtime after validation passes.
 
 Do not pass runtime environment parameters such as `--env`, `--environment`, or `ENVIRONMENT=<target>`.
 
@@ -38,7 +43,8 @@ Do not pass runtime environment parameters such as `--env`, `--environment`, or
 
 The migration apply lane records:
 
-- `fileName`
+- `key`
+- `migrationName`
 - `migrationType`
 - `checksum`
 - `appliedAt`
@@ -54,6 +60,8 @@ Each target is eligible for the next promotion step only after:
 
 - validation passes against the current `.env`
 - DDL/DML apply completes without checksum drift
+- drift validation confirms required tables, columns, indexes, constraints, and platform setting keys
+- `schema_migrations` matches the expected applied DDL/DML state for the target
 - repeat validation passes
 - the operator confirms the intended target connection without exposing secrets
 

docs_build/database/runbook.md

@@ -15,10 +15,22 @@ application behavior.
 `schema_migrations` is the authoritative migration state for DDL and DML apply
 history.
 
+Required `schema_migrations` fields are:
+
+- `key`
+- `migrationName`
+- `migrationType`
+- `checksum`
+- `appliedAt`
+- `appliedBy`
+
 `platform_settings` is runtime settings data only. It may hold settings such as
 platform banners and future maintenance toggles, but it must not track, gate, or
 control migration apply state.
 
+The optional current database version report is derived from
+`schema_migrations`; it is not stored in `platform_settings`.
+
 ## Validate
 
 Run connection validation before and after apply work:
@@ -45,10 +57,10 @@ Apply DDL from `docs_build/database/ddl/`:
 node .\scripts\apply-database-ddl.mjs
 ```
 
-The apply lane records each file in `schema_migrations` with file name, type,
-checksum, applied time, and applied-by metadata. If an applied file checksum
-changes, the apply lane must fail visibly. Create a new migration file instead
-of editing an already-applied file.
+The apply lane records each file in `schema_migrations` with `key`,
+`migrationName`, `migrationType`, `checksum`, `appliedAt`, and `appliedBy`
+metadata. If an applied file checksum changes, the apply lane must fail visibly.
+Create a new migration file instead of editing an already-applied file.
 
 ## Apply DML
 
@@ -76,9 +88,10 @@ node --use-system-ca .\scripts\apply-database-seed.mjs
 ```
 
 The DEV seed target keeps User 1, User 2, and User 3 as creator-only identities.
-DavidQ remains owner, admin, and creator. The seed script must refuse unsafe
-non-DEV targets unless a later approved operator lane explicitly changes that
-rule.
+DavidQ remains owner, admin, and creator. The seed script must refuse IST, UAT,
+and PRD targets and must confirm the owner role assignment before seed
+execution. A later approved operator lane is required before any non-DEV seed
+execution is allowed.
 
 ## Backup
 
@@ -90,6 +103,8 @@ Minimum backup checklist:
 
 - Confirm `.env` points at the intended database.
 - Confirm `pg_dump` is installed.
+- Record backup metadata: `backupName`, `databaseName`, `createdAt`, and
+  `migrationVersion`.
 - Write backup files outside tracked source paths.
 - Record backup path and checksum only.
 - Do not write secrets, full URLs, service role keys, or dump contents to
@@ -104,7 +119,9 @@ must require explicit operator confirmation.
 Minimum restore checklist:
 
 - Confirm `.env` points at the intended target database.
+- Validate the current `schema_migrations` state before applying restore.
 - Confirm the backup file path and checksum.
+- Confirm backup metadata, including `migrationVersion`.
 - Stop the application or enter a maintenance window.
 - Confirm owner approval.
 - Record the confirmation phrase `RESTORE CONFIRMED`.
@@ -120,22 +137,33 @@ Promotion is manual and must move in this order:
 DEV -> IST -> UAT -> PRD
 ```
 
-For each target:
+Exact operator workflow for each target:
 
 1. Copy the selected `.env.<target>` file to `.env`.
 2. Confirm `.env` contains the intended database host, port, database name, and
    SSL mode without exposing secrets.
 3. Run `node .\scripts\validate-runtime-connections.mjs`.
-4. Run `node .\scripts\apply-database-ddl.mjs`.
-5. Run `node .\scripts\apply-database-dml.mjs`.
-6. Run `node .\scripts\validate-database-drift.mjs`.
-7. Run `node .\scripts\validate-runtime-connections.mjs` again.
-8. Review `schema_migrations` for the expected DDL/DML files.
-9. Start or restart the runtime.
-10. Promote the next target only after the current target passes.
+4. Create a backup and record backup metadata.
+5. Run `node .\scripts\apply-database-ddl.mjs`.
+6. Run `node .\scripts\apply-database-dml.mjs`.
+7. Run `node .\scripts\validate-database-drift.mjs`.
+8. Run `node .\scripts\validate-runtime-connections.mjs` again.
+9. Review `schema_migrations` for the expected DDL/DML files.
+10. Start or restart the runtime.
+11. Promote the next target only after the current target passes.
 
 Do not use `platform_settings` as a migration gate or promotion gate.
 
+## Owner Operations Boundary
+
+Owner Operations remains status-first and execute-later until the database lanes
+are stable. It may show validation, apply, backup, restore, and migration
+history status, but it must not execute database operations from the browser.
+
+The status surface currently routes through
+`src/dev-runtime/server/local-api-router.mjs`. That filename is retained as a
+legacy runtime filename for now; do not infer local database behavior from it.
+
 ## Rollback Guidance
 
 Prefer forward-fix migrations for application schema corrections. Do not edit an

docs_build/database/seed/account/supabase-dev-identity-bootstrap.md

@@ -5,7 +5,7 @@ Use this only for the DEV Supabase project. Identity seeding is server-side only
 ## Bootstrap Order
 
 1. Run `docs_build/database/ddl/account/supabase-identity-tables.sql` in the Supabase SQL editor or through an approved operator SQL path.
-2. Run `npm run validate:supabase-dev`.
+2. Run `npm run validate:runtime-connections`.
 3. Confirm these checks pass through REST/API:
    - `Service role authentication`
    - `users table`

docs_build/dev/reports/PR_26167_207-database-lane-hardening.md

@@ -0,0 +1,58 @@
+# PR_26167_207-database-lane-hardening
+
+Status: PASS
+
+## Branch Validation
+
+PASS - current branch is `main`.
+
+## Review Resolution Checklist
+
+| Finding | Status | Resolution |
+| --- | --- | --- |
+| Keep `schema_migrations` as the only DDL/DML apply authority | PASS | Migration apply and docs keep `schema_migrations` authoritative. `platform_settings` is documented as runtime settings only. |
+| PR201 promotion lane order and migration validation | PASS | `promotion-lane.md` now states mandatory `DEV -> IST -> UAT -> PRD` order and requires `schema_migrations` plus drift validation before promotion. |
+| PR202 backup metadata and restore migration validation | PASS | `backup-restore-lane.md` now requires `backupName`, `databaseName`, `createdAt`, and `migrationVersion`, and restore validates migration state before applying. |
+| PR203 drift validation expansion | PASS | `validate-database-drift.mjs` now validates tables, columns, indexes, constraints, and required `platform_settings` keys with exact missing-object diagnostics. |
+| PR204 DEV-only seed and owner role requirement | PASS | `apply-database-seed.mjs` refuses non-DEV database names and confirms DavidQ owner assignment before seed execution. |
+| PR205 Owner Operations warning | PASS | `runbook.md` documents `src/dev-runtime/server/local-api-router.mjs` as a legacy filename and keeps Owner Operations status-first, execute-later. |
+| PR206 exact operator workflow | PASS | `runbook.md` now documents validate, backup, apply DDL, apply DML, validate, and promote. |
+| Standard DB script names | PASS | Active package/database lane uses `validate-runtime-connections.mjs`, `apply-database-ddl.mjs`, `apply-database-dml.mjs`, and `apply-database-seed.mjs`; stale database doc command was updated. |
+| Remove active Supabase-specific DB tooling naming | PASS | Removed `validate:supabase-dev` from active `package.json` scripts. Compatibility wrapper files remain only as deprecated entrypoints. |
+| Add `schema_migrations` fields | PASS | Runner now creates/ensures `key`, `migrationName`, `migrationType`, `checksum`, `appliedAt`, and `appliedBy`; existing DB evidence includes those fields. Legacy `fileName` remains only for existing-table compatibility. |
+
+## Migration State Evidence
+
+- DDL/DML apply output reports: `Migration tracking fields: key, migrationName, migrationType, checksum, appliedAt, appliedBy`.
+- Current database columns observed: `key,fileName,migrationType,checksum,appliedAt,appliedBy,migrationName`.
+- `fileName` is retained in the current database for compatibility with already-applied records; active tracking uses `migrationName`.
+
+## Validation Lane Report
+
+Targeted DB validation only:
+
+- `node --check scripts\database-migration-runner.mjs` - PASS
+- `node --check scripts\validate-database-drift.mjs` - PASS
+- `node --check scripts\apply-database-seed.mjs` - PASS
+- `node --check scripts\apply-database-ddl.mjs` - PASS
+- `node --check scripts\apply-database-dml.mjs` - PASS
+- `node --use-system-ca .\scripts\validate-runtime-connections.mjs` - PASS before apply and PASS after apply
+- `node .\scripts\apply-database-ddl.mjs` - PASS, processed=15, applied=0, skipped=15
+- `node .\scripts\apply-database-dml.mjs` - PASS, processed=15, applied=0, skipped=15
+- `node .\scripts\validate-database-drift.mjs --diagnostic-self-test` - PASS
+- `node .\scripts\validate-database-drift.mjs` - PASS, tables=34, constraints=146, indexes=115, platform setting keys=3
+- `node --use-system-ca .\scripts\apply-database-seed.mjs --unsafe-target-self-test` - PASS
+- `node --use-system-ca .\scripts\apply-database-seed.mjs --dry-run` - PASS; User 1-3 creator PASS, DavidQ owner/admin/creator PASS
+- `node -e "JSON.parse(require('fs').readFileSync('package.json','utf8')); console.log('PASS - package.json parsed')"` - PASS
+- `rg -n "validate:supabase-dev|apply:supabase|validate-supabase-dev|apply-supabase-dev-ddl" package.json docs_build\database scripts\database-migration-runner.mjs scripts\apply-database-ddl.mjs scripts\apply-database-dml.mjs scripts\apply-database-seed.mjs scripts\validate-database-drift.mjs` - PASS, no active database lane matches
+- `git diff --check -- scripts\database-migration-runner.mjs scripts\validate-database-drift.mjs scripts\apply-database-seed.mjs package.json docs_build\database\promotion-lane.md docs_build\database\backup-restore-lane.md docs_build\database\runbook.md docs_build\database\seed\account\supabase-dev-identity-bootstrap.md` - PASS
+- `git diff --check` - PASS
+
+Skipped lanes:
+
+- Playwright - SKIP, no Owner Operations UI/browser behavior changed; existing Owner Operations coverage remains from PR205.
+- Full samples smoke - SKIP, samples are not in scope.
+
+## Manual Validation Notes
+
+The hardened DB lane was manually reviewed against the review findings. `schema_migrations` remains the DDL/DML apply state, `platform_settings` remains runtime configuration only, seed is DEV-only and owner-gated, and Owner Operations remains a read/status surface until database lanes are stable.

docs_build/dev/reports/codex_changed_files.txt

@@ -1,5 +1,11 @@
-docs_build/database/README.md
+docs_build/database/backup-restore-lane.md
+docs_build/database/promotion-lane.md
 docs_build/database/runbook.md
-docs_build/dev/reports/PR_26167_206-database-runbook.md
+docs_build/database/seed/account/supabase-dev-identity-bootstrap.md
+package.json
+scripts/apply-database-seed.mjs
+scripts/database-migration-runner.mjs
+scripts/validate-database-drift.mjs
+docs_build/dev/reports/PR_26167_207-database-lane-hardening.md
 docs_build/dev/reports/codex_review.diff
 docs_build/dev/reports/codex_changed_files.txt