feat: auto-manage external DNS records (Selectel) on proxy-host changes

CLAUDE.md

@@ -0,0 +1,101 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Overview
+
+Nginx Proxy Manager — a Docker-packaged admin UI + API over Nginx for reverse-proxy hosts, redirections, streams, 404 hosts, and Let's Encrypt / custom SSL. Three deployable pieces live in one repo: a Node/Express backend (`backend/`), a React SPA (`frontend/`), and a Cypress test suite (`test/`). At runtime everything ships inside a single container running under s6-overlay, where the backend generates Nginx config files from Liquid templates and reloads Nginx.
+
+## Development
+
+The dev environment is a full Docker Compose stack — there is no bare-metal `npm start`. It spins up the fullstack container plus MariaDB, Postgres, PowerDNS, step-ca, Squid proxy, Authentik, and Swagger.
+
+```bash
+./scripts/start-dev          # build + bring up the whole dev stack
+./scripts/start-dev -f       # same, then follow backend logs
+./scripts/stop-dev
+./scripts/destroy-dev        # tear down + remove volumes
+docker logs -f npm2dev.core  # follow the backend/core container
+```
+
+Dev URLs: Admin UI `http://127.0.0.1:3081`, Nginx `http://127.0.0.1:3080`, Swagger `http://127.0.0.1:3001`. The backend listens on port `3000` inside the container.
+
+### Lint / format (Biome, both backend and frontend)
+
+```bash
+# in backend/ or frontend/
+yarn lint        # biome lint
+yarn prettier    # biome format --write
+```
+
+Biome config (`biome.json`) uses **tab indentation** — match it.
+
+### Frontend
+
+```bash
+cd frontend
+yarn dev         # vite dev server (port 5173); auto-compiles locales on start/change
+yarn build       # tsc typecheck + vite build
+yarn test        # vitest
+yarn test -- src/path/to/File.test.tsx   # single test file
+```
+
+Locales are FormatJS-based: extract with `yarn locale-extract`, compile with `yarn locale-compile`. Vite recompiles locales automatically when files under `src/locale/src` change. Run `node check-locales.cjs` to validate translation completeness.
+
+### Backend
+
+```bash
+cd backend
+yarn validate-schema                # validate the OpenAPI/JSON schema
+node scripts/regenerate-config      # regenerate nginx config from templates
+```
+
+The backend is ESM (`"type": "module"`) — use `import`, not `require`. Migrations run automatically on startup via `migrateUp()`.
+
+### End-to-end tests (Cypress)
+
+E2E tests run against the CI docker stack, not the dev stack. Specs are `test/cypress/e2e/*.cy.js`; requests are proxied through Squid (`HTTP_PROXY=127.0.0.1:8128`).
+
+```bash
+cd test
+yarn cypress:dev            # run against a running dev stack
+yarn cypress:headless       # headless run against CI stack
+yarn swagger-lint           # lint the OpenAPI schema (vacuum)
+
+./scripts/ci/fulltest-cypress   # full CI-style run (sqlite/mysql/postgres matrices exist in docker/)
+```
+
+## Architecture
+
+### Backend layering (`backend/`)
+
+Request flow is **route → internal → model**, a strict three-layer split:
+
+- **`routes/`** — Express routers, one per resource. `routes/main.js` mounts them all under `/api`-equivalent paths (`/nginx/proxy-hosts`, `/users`, `/tokens`, etc.). Routes handle HTTP concerns, JWT auth (`lib/express/jwt.js`), permission checks, and schema validation only.
+- **`internal/`** — business logic. This is where the real work lives (e.g. `internal/certificate.js` is ~37KB of Let's Encrypt/certbot orchestration). Internal modules call each other and the models; they never touch `req`/`res`.
+- **`models/`** — Objection.js models over a Knex query builder. `db.js`/`knexfile.js` wire the connection; the same schema targets **SQLite, MySQL/MariaDB, and Postgres** (see `lib/config.js` for `DB_*` env selection).
+
+`app.js` builds the Express app (middleware, security headers, error handler); `index.js` is the entrypoint that runs migrations → setup → schema compile → IP-ranges fetch → starts timers (cert renewal, IP ranges) → listens on 3000.
+
+### Nginx config generation
+
+This is the core mechanism. `internal/nginx.js` renders per-host config from **Liquid templates in `backend/templates/`** (`proxy_host.conf`, `redirection_host.conf`, `stream.conf`, `dead_host.conf`, plus `_`-prefixed partials like `_location.conf`, `_ssl.conf`, `_access.conf`). The `configure()` flow is: test nginx → delete old config → generate new config → test again → on success mark host `nginx_online` in DB, on failure remove the config and record `nginx_err` in the model's `meta`. When editing proxy behavior, the change usually belongs in a template, not in JS.
+
+### Database schema
+
+Knex migrations in `backend/migrations/` (timestamp-prefixed). Never edit an existing migration — add a new one. Migrations must work across all three supported databases.
+
+### Frontend (`frontend/src/`)
+
+React 19 + Vite + TypeScript SPA using **Tabler** (`@tabler/core`) for UI, **TanStack Query** for server state, **Formik** for forms, **react-intl** for i18n, and **react-router-dom** v7. Structure: `api/backend/` (one file per API call, e.g. `createProxyHost.ts`, wrapping `base.ts`), `pages/`, `modules/`, `components/`, `modals/`, `hooks/`, `context/`, `locale/`. `Router.tsx` defines routes; `App.tsx` wires providers. API responses/requests are camelCase↔snake_case converted via `humps`.
+
+### Docker / runtime (`docker/`)
+
+`docker/Dockerfile` is a **buildx multi-arch** build that assumes the frontend is already built (`./scripts/frontend-build`). Runtime uses s6-overlay (`docker/rootfs/`). `docker/docker-compose.dev.yml` defines the dev stack; `docker-compose.ci.*.yml` variants cover the sqlite/mysql/postgres CI matrices.
+
+## Conventions
+
+- Version lives in `.version` (currently 2.15.1); `backend/package.json` and `frontend/package.json` carry an unrelated internal `2.0.0`.
+- Errors: throw the typed errors from `backend/lib/error.js` (e.g. `ItemNotFoundError`); only errors marked `public` leak their message to clients, and stack traces are exposed only in debug mode.
+- CI-only routes (`routes/ci.js`) are mounted at `/ci` when `isCI()` is true — do not rely on them in production paths.
+- `armv7` is unsupported in 2.14+.

backend/internal/dns-provider.js

@@ -0,0 +1,201 @@
+import _ from "lodash";
+import errs from "../lib/error.js";
+import dnsProviderModel from "../models/dns_provider.js";
+import internalAuditLog from "./audit-log.js";
+import { getDriver } from "./dns/index.js";
+
+const omissions = () => {
+	return ["credentials"];
+};
+
+const internalDnsProvider = {
+	/**
+	 * @param   {Access}  access
+	 * @param   {Object}  data
+	 * @returns {Promise}
+	 */
+	create: (access, data) => {
+		return access
+			.can("dns_providers:create", data)
+			.then(() => {
+				return dnsProviderModel
+					.query()
+					.insertAndFetch({
+						...data,
+						owner_user_id: access.token.getUserId(1),
+						meta: data.meta || {},
+					})
+					.then((row) => _.omit(row, omissions()));
+			})
+			.then((row) => {
+				return internalAuditLog
+					.add(access, {
+						action: "created",
+						object_type: "dns-provider",
+						object_id: row.id,
+						meta: _.omit(row, omissions()),
+					})
+					.then(() => row);
+			});
+	},
+
+	/**
+	 * @param  {Access}  access
+	 * @param  {Object}  data
+	 * @param  {Integer} data.id
+	 * @return {Promise}
+	 */
+	update: (access, data) => {
+		return access
+			.can("dns_providers:update", data.id)
+			.then(() => internalDnsProvider.get(access, { id: data.id }))
+			.then((row) => {
+				if (row.id !== data.id) {
+					// Sanity check that something crazy hasn't happened
+					throw new errs.InternalValidationError(
+						`DNS Provider could not be updated, IDs do not match: ${row.id} !== ${data.id}`,
+					);
+				}
+				return dnsProviderModel
+					.query()
+					.where("id", data.id)
+					.patchAndFetchById(data.id, _.omit(data, ["id"]))
+					.then((row) => _.omit(row, omissions()));
+			})
+			.then((row) => {
+				return internalAuditLog
+					.add(access, {
+						action: "updated",
+						object_type: "dns-provider",
+						object_id: row.id,
+						meta: _.omit(data, omissions()),
+					})
+					.then(() => row);
+			});
+	},
+
+	/**
+	 * @param  {Access}   access
+	 * @param  {Object}   data
+	 * @param  {Integer}  data.id
+	 * @param  {Array}    [data.expand]
+	 * @return {Promise}
+	 */
+	get: (access, data) => {
+		const thisData = data || {};
+		return access.can("dns_providers:get", thisData.id).then(() => {
+			const query = dnsProviderModel.query().where("is_deleted", 0).andWhere("id", thisData.id).first();
+
+			if (typeof thisData.expand !== "undefined" && thisData.expand !== null) {
+				query.withGraphFetched(`[${thisData.expand.join(", ")}]`);
+			}
+
+			return query.then((row) => {
+				if (!row?.id) {
+					throw new errs.ItemNotFoundError(thisData.id);
+				}
+				return _.omit(row, omissions());
+			});
+		});
+	},
+
+	/**
+	 * @param   {Access}  access
+	 * @param   {Object}  data
+	 * @param   {Integer} data.id
+	 * @returns {Promise}
+	 */
+	delete: (access, data) => {
+		return access
+			.can("dns_providers:delete", data.id)
+			.then(() => internalDnsProvider.get(access, { id: data.id }))
+			.then((row) => {
+				if (!row?.id) {
+					throw new errs.ItemNotFoundError(data.id);
+				}
+				return dnsProviderModel
+					.query()
+					.where("id", row.id)
+					.patch({ is_deleted: 1 })
+					.then(() =>
+						internalAuditLog.add(access, {
+							action: "deleted",
+							object_type: "dns-provider",
+							object_id: row.id,
+							meta: _.omit(row, omissions()),
+						}),
+					);
+			})
+			.then(() => true);
+	},
+
+	/**
+	 * All Providers
+	 *
+	 * @param   {Access}  access
+	 * @param   {Array}   [expand]
+	 * @param   {String}  [searchQuery]
+	 * @returns {Promise}
+	 */
+	getAll: (access, expand, searchQuery) => {
+		return access.can("dns_providers:list").then(() => {
+			const query = dnsProviderModel.query().where("is_deleted", 0).orderBy("name", "ASC");
+
+			if (typeof searchQuery === "string" && searchQuery.length > 0) {
+				query.where((qb) => {
+					qb.where("name", "like", `%${searchQuery}%`);
+				});
+			}
+
+			if (typeof expand !== "undefined" && expand !== null) {
+				query.withGraphFetched(`[${expand.join(", ")}]`);
+			}
+
+			return query.then((rows) => rows.map((row) => _.omit(row, omissions())));
+		});
+	},
+
+	/**
+	 * Count is used in reports
+	 *
+	 * @param   {Integer} userId
+	 * @returns {Promise}
+	 */
+	getCount: (_userId) => {
+		return dnsProviderModel
+			.query()
+			.count("id as count")
+			.where("is_deleted", 0)
+			.first()
+			.then((row) => Number.parseInt(row.count, 10));
+	},
+
+	/**
+	 * Tests the connection for a DNS provider using its stored credentials
+	 *
+	 * @param   {Access}  access
+	 * @param   {Object}  data
+	 * @param   {Integer} data.id
+	 * @returns {Promise}
+	 */
+	test: (access, data) => {
+		return access
+			.can("dns_providers:get", data.id)
+			.then(() => dnsProviderModel.query().where("is_deleted", 0).andWhere("id", data.id).first())
+			.then(async (row) => {
+				if (!row?.id) {
+					throw new errs.ItemNotFoundError(data.id);
+				}
+				const result = await getDriver(row.type).testConnection(row.credentials);
+				// Persist the last connection-check result so the list view can show a status.
+				const meta = _.assign({}, row.meta, {
+					last_check_ok: result.ok,
+					last_check_error: result.ok ? null : result.error || null,
+				});
+				await dnsProviderModel.query().where("id", row.id).patch({ meta });
+				return result;
+			});
+	},
+};
+
+export default internalDnsProvider;

backend/internal/dns-record.js

@@ -0,0 +1,89 @@
+import _ from "lodash";
+import { dnsRecord as logger } from "../logger.js";
+import dnsProviderModel from "../models/dns_provider.js";
+import { getDriver } from "./dns/index.js";
+
+/**
+ * @param {string[]} desired
+ * @param {Array<{domain:string,zone_id:string,rrset_id:string}>} existingRecords
+ */
+export const diffDomains = (desired, existingRecords) => {
+	const existingByDomain = _.keyBy(existingRecords || [], "domain");
+	const desiredSet = new Set(desired || []);
+	const toCreate = (desired || []).filter((d) => !existingByDomain[d]);
+	const toDelete = (existingRecords || []).filter((r) => !desiredSet.has(r.domain));
+	return { toCreate, toDelete };
+};
+
+const loadProvider = async (providerId) =>
+	dnsProviderModel.query().where("id", providerId).where("is_deleted", 0).first();
+
+/**
+ * Synchronises DNS A-records for a proxy host with its configured provider.
+ * Never throws — failures are returned in dns_err.
+ *
+ * @param {{dns_provider_id:number, domain_names:string[], meta:object}} host
+ * @returns {Promise<{dns_synced:boolean, dns_err:string|null, dns_records:Array}>}
+ */
+export const sync = async (host) => {
+	if (!host.dns_provider_id) {
+		return { dns_synced: false, dns_err: null, dns_records: [] };
+	}
+	const existing = host.meta?.dns_records || [];
+	try {
+		const provider = await loadProvider(host.dns_provider_id);
+		if (!provider) {
+			throw new Error(`DNS provider ${host.dns_provider_id} not found`);
+		}
+		const driver = getDriver(provider.type);
+		const { toCreate, toDelete } = diffDomains(host.domain_names || [], existing);
+
+		const kept = existing.filter((r) => !toDelete.includes(r));
+
+		for (const rec of toDelete) {
+			await driver.deleteRecord(provider.credentials, { zone_id: rec.zone_id, rrset_id: rec.rrset_id });
+		}
+		const created = [];
+		for (const domain of toCreate) {
+			const { zone_id, rrset_id } = await driver.createRecord(
+				provider.credentials,
+				domain,
+				provider.default_ip,
+				provider.ttl,
+			);
+			created.push({ domain, zone_id, rrset_id });
+		}
+		return { dns_synced: true, dns_err: null, dns_records: [...kept, ...created] };
+	} catch (err) {
+		logger.error(`sync failed for host domains ${JSON.stringify(host.domain_names)}: ${err.message}`);
+		return { dns_synced: false, dns_err: err.message, dns_records: existing };
+	}
+};
+
+/**
+ * Removes all DNS records previously created for a host. Never throws.
+ * @param {{dns_provider_id:number, meta:object}} host
+ */
+export const cleanup = async (host) => {
+	if (!host.dns_provider_id) {
+		return;
+	}
+	const existing = host.meta?.dns_records || [];
+	if (!existing.length) {
+		return;
+	}
+	try {
+		const provider = await loadProvider(host.dns_provider_id);
+		if (!provider) {
+			return;
+		}
+		const driver = getDriver(provider.type);
+		for (const rec of existing) {
+			await driver.deleteRecord(provider.credentials, { zone_id: rec.zone_id, rrset_id: rec.rrset_id });
+		}
+	} catch (err) {
+		logger.error(`cleanup failed: ${err.message}`);
+	}
+};
+
+export default { diffDomains, sync, cleanup };