bundle: record and read deployment state via DMS with server-generated IDs

[shreyas-goenka]

What

Wires the direct deployment engine into the Deployment Metadata Service (DMS) so that a bundle with experimental.record_deployment_history enabled:

1. Records each deploy/destroy as a DMS version (with heartbeat + completion).
2. Records each applied resource operation (create/update/delete) under that version.
3. Reads resource state back from DMS on the next deploy.

The key design point: server-generated deployment IDs

The DMS API assigns the deployment ID server-side. Earlier prototypes used the local state lineage (a client-minted UUID) as the deployment ID; that no longer holds. The flow is now:

• First deploy → CreateDeployment with an empty ID → the server mints one and returns it in deployment.Name → we parse it out and persist it in the direct-engine state header (Header.DeploymentID).
• Later deploys → pass the stored ID back → GetDeployment to compute the next version number.

Because the ID lives in the state file (which is synced to the workspace), it survives deleting the local .databricks cache — the redeploy recovers the same deployment and just increments the version. No new CreateDeployment.

// libs/dms/recorder.go — first deploy lets the server assign the ID
dep, _ := r.svc.CreateDeployment(ctx, bundledeployments.CreateDeploymentRequest{
    Deployment: bundledeployments.Deployment{TargetName: r.targetName},
})
r.deploymentID, _ = deploymentIDFromName(dep.Name) // "deployments/{id}" -> {id}

Wiring

• libs/dms — Recorder: create deployment (server-assigned ID) + version, heartbeat the lease, complete it, delete the deployment on destroy.
• bundle/direct/oprecorder.go — operationRecorder: reports each applied operation. The wire resource_key drops the CLI-internal resources. prefix (resources.jobs.foo → jobs.foo); the read path re-adds it.
• bundle/direct/dstate — Open takes a DMS client and overlays DMS resource state when DMS holds a successful version; DeploymentID persisted in the state header.
• bundle/phases — create the version after plan approval, complete it under the lock (deferred before lock.Release), record operations during apply.
• libs/testserver — a stateful fake DMS (deployments / versions / operations / resources) with server-generated IDs.

Testing

• Unit tests: recorder (first deploy / redeploy / destroy / nil no-op), op-recorder prefix stripping, deployment-ID state round-trip.
• Acceptance test acceptance/bundle/dms/record covers the full arc: deploy → server assigns ID → redeploy after rm -rf .databricks (ID recovered, version increments, no new CreateDeployment) → destroy (delete op + deployment deletion).

Gated entirely behind experimental.record_deployment_history + the direct engine; terraform and non-opted bundles are untouched.

This pull request and its description were written by Isaac.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 5590258

Run: 30054051567

	Env	❌​FAIL	🟨​KNOWN	💚​RECOVERED	🙈​SKIP	✅​pass	🙈​skip	Time
❌​	aws linux	3	1	3	4	319	1060	5:06
❌​	aws windows	3	1	3	4	321	1058	6:44
💚​	azure linux			4	4	322	1059	4:29
💚​	azure windows			4	4	324	1057	4:30
💚​	gcp linux			1	5	321	1061	4:16
💚​	gcp windows			1	5	323	1059	4:29

11 interesting tests: 4 SKIP, 3 FAIL, 3 RECOVERED, 1 KNOWN

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
🟨​	TestAccept	🟨​K	🟨​K	💚​R	💚​R	💚​R	💚​R
❌​	TestAccept/bundle/deploy/mlops-stacks	❌​F	❌​F	✅​p	✅​p	✅​p	✅​p
❌​	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct	❌​F	❌​F	✅​p	✅​p	✅​p	✅​p
❌​	TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform	❌​F	❌​F	✅​p	✅​p	✅​p	✅​p
🙈​	TestAccept/bundle/invariant/no_drift	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
🙈​	TestAccept/ssh/connection	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S
💚​	TestFetchRepositoryInfoAPI_FromRepo	💚​R	💚​R	💚​R	💚​R	🙈​S	🙈​S
💚​	TestFetchRepositoryInfoAPI_FromRepo/root	💚​R	💚​R	💚​R	💚​R
💚​	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚​R	💚​R	💚​R	💚​R

Top 2 slowest tests (at least 2 minutes):

duration	env	testname
3:27	azure windows	TestAccept
3:25	gcp windows	TestAccept