Honor [__settings__].default_profile in api, bundle, and auth token (#5214)

janniklasrose · web-flow · commit 6327c33ee7ab · 2026-05-19T11:14:39.000Z
## Changes

`[__settings__].default_profile` is a CLI-level fallback for the active
profile that the SDK does not know about. Until now it was consulted
only
on the `MustWorkspaceClient` / `MustAccountClient` / `MustAnyClient`
paths. Several command surfaces silently ignored it:

- `databricks api {get,post,...}` built its own `*config.Config` and
only
  honored `--profile`.
- Bundle commands (`databricks bundle`, `databricks pipelines`) read the
  profile only from `--profile` / `DATABRICKS_CONFIG_PROFILE`.
- `databricks auth token` (no positional, no flags, no env) jumped
  straight to the interactive picker.

This PR promotes the resolution logic to
`databrickscfg.ResolveDefaultProfile`
and applies it from each affected command:

- `cmd/api/api.go`: `--profile` flag → `DATABRICKS_CONFIG_PROFILE` env →
  `default_profile`, then hand off to the SDK.
- `cmd/root/bundle.go` (`configureProfile`): falls back to
`default_profile` only when the bundle declares no `workspace.host` and
  no `workspace.profile`. The host-empty guard preserves SDK host-based
  resolution when the bundle pins its own host.
- `cmd/auth/token.go` (`resolveNoArgsToken`): inserts a Step 2.5 between
  the env-var check and the interactive picker. If the named profile
  doesn't exist, falls through to the picker (advisory, not fatal).

## Why

Users reasonably expect "I set my default to `tmp` — every CLI
invocation
should default to `tmp`." The previous behavior was inconsistent across
command surfaces and surprised users.

The bundle host-empty guard is the only subtle bit worth flagging:
applying `default_profile` before the SDK sees a bundle's pinned
`workspace.host` could silently route the user to a profile pointing at
a
different host. The guard lets the existing multi-profile-match path
(`resolveProfileAmbiguity`) handle host conflicts as before.

## Tests

- `libs/databrickscfg/ops_test.go`: matrix tests for
`ResolveDefaultProfile`
  (file present/absent, settings present/absent, parse error, reserved
  `__settings__` self-reference, default home-file lookup).
- `cmd/root/bundle_test.go`: four
`TestBundleConfigureWithDefaultProfile_*`
  unit tests covering the positive case, `--profile` override,
  `DATABRICKS_CONFIG_PROFILE` override, and the bundle-host-wins guard.
- Acceptance tests:
  - `acceptance/cmd/api/default-profile/`
  - `acceptance/auth/bundle_default_profile/` (with a bundle-with-host
    sub-case)
  - `acceptance/cmd/auth/token/default-profile/` (with a missing-default
    fall-through case)

  Each new acceptance test covers the positive case (`default_profile`
  honored) and the negative case (`--profile` overrides).

_This PR was written by Claude Code._
diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md
@@ -7,6 +7,7 @@
 ### CLI
 
 * Added `databricks aitools` command group for installing Databricks skills into your coding agents (Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity). Skills are fetched from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) and either symlinked into each agent's skills directory or copied into the current project. Use `databricks aitools install` to set up, `update` to pull newer versions, `list` to see what's available, and `uninstall` to remove them.
+* `[__settings__].default_profile` is now consulted as a fallback by `databricks api`, `databricks auth token`, and bundle commands when neither `--profile` nor `DATABRICKS_CONFIG_PROFILE` is set. `databricks auth token` continues to give precedence to `DATABRICKS_HOST` over `default_profile`. For bundle commands, `default_profile` only applies when the bundle does not pin its own `workspace.host`.
 
 ### Bundles
 * Make sure warnings asking for approval are understood by agents ([#5239](https://github.com/databricks/cli/pull/5239))
diff --git a/acceptance/auth/bundle_default_profile/databricks.yml b/acceptance/auth/bundle_default_profile/databricks.yml
@@ -0,0 +1,5 @@
+bundle:
+  name: test-default-profile
+
+# No workspace.host on purpose: this is the surface where
+# [__settings__].default_profile should be applied.
diff --git a/acceptance/auth/bundle_default_profile/out.test.toml b/acceptance/auth/bundle_default_profile/out.test.toml
diff --git a/acceptance/auth/bundle_default_profile/output.txt b/acceptance/auth/bundle_default_profile/output.txt
@@ -0,0 +1,45 @@
+
+=== Bundle without workspace.host: default_profile is honored
+
+>>> [CLI] bundle validate -o json
+{
+  "host": null,
+  "profile": "default-target"
+}
+
+=== --profile overrides default_profile (negative case)
+
+>>> errcode [CLI] bundle validate -p other -o json
+Warn: [hostmetadata] failed to fetch host metadata for https://other.test, will skip for 1m0s
+Error: Get "https://other.test/api/2.0/preview/scim/v2/Me": (redacted)
+
+
+Exit code: 1
+{
+  "host": null,
+  "profile": "other"
+}
+
+=== Bundle with workspace.host: default_profile is NOT applied
+
+>>> errcode [CLI] bundle validate -o json
+Error: failed during request visitor: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=[DATABRICKS_URL], workspace_id=[NUMID], databricks_cli_path=[CLI]. Env: DATABRICKS_CLI_PATH
+
+
+Exit code: 1
+{
+  "host": "[DATABRICKS_URL]",
+  "profile": null
+}
+
+=== Bundle with workspace.profile: pinned profile wins over default_profile
+
+>>> errcode [CLI] bundle validate -o json
+Error: failed during request visitor: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: profile=other, databricks_cli_path=[CLI]. Env: DATABRICKS_CLI_PATH
+
+
+Exit code: 1
+{
+  "host": null,
+  "profile": "other"
+}
diff --git a/acceptance/auth/bundle_default_profile/script b/acceptance/auth/bundle_default_profile/script
@@ -0,0 +1,58 @@
+sethome "./home"
+
+# Save the test server host so we can pin a bundle-with-host variant below.
+host_value="$DATABRICKS_HOST"
+
+cat > "./home/.databrickscfg" <<EOF
+[default-target]
+host  = $DATABRICKS_HOST
+token = $DATABRICKS_TOKEN
+
+[other]
+host  = https://other.test
+token = other-token
+
+[__settings__]
+default_profile = default-target
+EOF
+
+unset DATABRICKS_HOST
+unset DATABRICKS_TOKEN
+unset DATABRICKS_CONFIG_PROFILE
+
+title "Bundle without workspace.host: default_profile is honored\n"
+trace $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}'
+
+title "--profile overrides default_profile (negative case)\n"
+trace errcode $CLI bundle validate -p other -o json | jq '{host: .workspace.host, profile: .workspace.profile}'
+
+# Switch to a bundle that pins workspace.host. The default_profile guard in
+# configureProfile must NOT apply default_profile here, because that would
+# silently route the user to a profile pointing at a different host than the
+# bundle declares.
+mkdir -p ./bundle-with-host
+cat > ./bundle-with-host/databricks.yml <<EOF
+bundle:
+  name: bundle-with-host
+
+workspace:
+  host: $host_value
+EOF
+
+title "Bundle with workspace.host: default_profile is NOT applied\n"
+(cd ./bundle-with-host && trace errcode $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}')
+
+# Switch to a bundle that pins workspace.profile but no host. The pinned
+# profile must win over default_profile — configureProfile's guard skips
+# default_profile when workspace.profile is already set.
+mkdir -p ./bundle-with-profile
+cat > ./bundle-with-profile/databricks.yml <<EOF
+bundle:
+  name: bundle-with-profile
+
+workspace:
+  profile: other
+EOF
+
+title "Bundle with workspace.profile: pinned profile wins over default_profile\n"
+(cd ./bundle-with-profile && trace errcode $CLI bundle validate -o json | jq '{host: .workspace.host, profile: .workspace.profile}')
diff --git a/acceptance/auth/bundle_default_profile/test.toml b/acceptance/auth/bundle_default_profile/test.toml
@@ -0,0 +1,12 @@
+Ignore = [
+    "home",
+    ".databricks",
+    "bundle-with-host",
+    "bundle-with-profile",
+]
+
+# Negative case: -p other tries to reach a non-existing host. Redact the
+# OS-/network-dependent suffix so the test is stable across runners.
+[[Repls]]
+Old = 'Get "https://other.test/api/2.0/preview/scim/v2/Me": .*'
+New = 'Get "https://other.test/api/2.0/preview/scim/v2/Me": (redacted)'
diff --git a/acceptance/cmd/api/default-profile/out.test.toml b/acceptance/cmd/api/default-profile/out.test.toml
diff --git a/acceptance/cmd/api/default-profile/output.txt b/acceptance/cmd/api/default-profile/output.txt
@@ -0,0 +1,28 @@
+
+=== default_profile is used when no --profile flag and no DATABRICKS_CONFIG_PROFILE
+
+>>> [CLI] api get /api/2.0/clusters/list
+{}
+
+>>> print_requests.py --get //api/2.0/clusters/list
+{
+  "headers": {
+    "Authorization": [
+      "Bearer [DATABRICKS_TOKEN]"
+    ],
+    "User-Agent": [
+      "cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
+    ],
+    "X-Databricks-Org-Id": [
+      "[NUMID]"
+    ]
+  },
+  "method": "GET",
+  "path": "/api/2.0/clusters/list"
+}
+
+=== --profile overrides default_profile (negative case)
+Warn: [hostmetadata] failed to fetch host metadata for https://other.test, will skip for 1m0s
+Error: Get "https://other.test/api/2.0/clusters/list": (redacted)
+
+Exit code: 1
diff --git a/acceptance/cmd/api/default-profile/script b/acceptance/cmd/api/default-profile/script
@@ -0,0 +1,28 @@
+sethome "./home"
+
+# Two profiles plus an explicit default_profile pointing at the test server.
+# The 'other' profile points at an RFC 2606 reserved host so we can assert
+# that --profile overrides default_profile without making a real request.
+cat > "./home/.databrickscfg" <<EOF
+[default-target]
+host  = $DATABRICKS_HOST
+token = $DATABRICKS_TOKEN
+
+[other]
+host  = https://other.test
+token = other-token
+
+[__settings__]
+default_profile = default-target
+EOF
+
+unset DATABRICKS_HOST
+unset DATABRICKS_TOKEN
+unset DATABRICKS_CONFIG_PROFILE
+
+title "default_profile is used when no --profile flag and no DATABRICKS_CONFIG_PROFILE\n"
+MSYS_NO_PATHCONV=1 trace $CLI api get /api/2.0/clusters/list
+trace print_requests.py --get //api/2.0/clusters/list | contains.py "X-Databricks-Org-Id"
+
+title "--profile overrides default_profile (negative case)\n"
+MSYS_NO_PATHCONV=1 errcode $CLI api get /api/2.0/clusters/list -p other 2>&1 | contains.py "other.test"
diff --git a/acceptance/cmd/api/default-profile/test.toml b/acceptance/cmd/api/default-profile/test.toml
@@ -0,0 +1,10 @@
+Ignore = [
+    "home",
+]
+
+# Redact the OS- and network-dependent suffix on the failed lookup so the
+# negative case (--profile overrides default_profile) is stable across
+# runners. We still assert the requested host appears in output.
+[[Repls]]
+Old = 'Get "https://other.test/api/2.0/clusters/list": .*'
+New = 'Get "https://other.test/api/2.0/clusters/list": (redacted)'
diff --git a/acceptance/cmd/auth/token/default-profile/out.test.toml b/acceptance/cmd/auth/token/default-profile/out.test.toml
diff --git a/acceptance/cmd/auth/token/default-profile/output.txt b/acceptance/cmd/auth/token/default-profile/output.txt
@@ -0,0 +1,15 @@
+
+=== default_profile is honored when no args, --profile, or env var
+
+>>> errcode [CLI] auth token
+Warn: [hostmetadata] failed to fetch host metadata for https://myworkspace.test, will skip for 1m0s
+Error: cache: databricks OAuth is not configured for this host. Try logging in again with `databricks auth login --profile myprofile` before retrying. If this fails, please report this issue to the Databricks CLI maintainers at https://github.com/databricks/cli/issues/new
+
+Exit code: 1
+
+=== default_profile pointing at a missing profile falls through to picker
+
+>>> errcode [CLI] auth token
+Error: no profile specified. Use --profile <name> to specify which profile to use
+
+Exit code: 1
diff --git a/acceptance/cmd/auth/token/default-profile/script b/acceptance/cmd/auth/token/default-profile/script
@@ -0,0 +1,34 @@
+sethome "./home"
+
+unset DATABRICKS_HOST
+unset DATABRICKS_TOKEN
+unset DATABRICKS_CONFIG_PROFILE
+
+# default_profile points at "myprofile". Without it, `auth token` would fall
+# through to the non-interactive error "no profile specified".
+cat > "./home/.databrickscfg" <<'ENDCFG'
+[myprofile]
+host = https://myworkspace.test
+auth_type = databricks-cli
+
+[other]
+host = https://other.test
+auth_type = databricks-cli
+
+[__settings__]
+default_profile = myprofile
+ENDCFG
+
+title "default_profile is honored when no args, --profile, or env var\n"
+trace errcode $CLI auth token
+
+title "default_profile pointing at a missing profile falls through to picker\n"
+cat > "./home/.databrickscfg" <<'ENDCFG'
+[myprofile]
+host = https://myworkspace.test
+auth_type = databricks-cli
+
+[__settings__]
+default_profile = does-not-exist
+ENDCFG
+trace errcode $CLI auth token
diff --git a/cmd/api/api.go b/cmd/api/api.go
@@ -11,6 +11,8 @@ import (
 	"github.com/databricks/cli/cmd/root"
 	"github.com/databricks/cli/libs/auth"
 	"github.com/databricks/cli/libs/cmdio"
+	"github.com/databricks/cli/libs/databrickscfg"
+	"github.com/databricks/cli/libs/env"
 	"github.com/databricks/cli/libs/flags"
 	"github.com/databricks/databricks-sdk-go/client"
 	"github.com/databricks/databricks-sdk-go/config"
@@ -77,11 +79,20 @@ func makeCommand(method string) *cobra.Command {
 
 			cfg := &config.Config{}
 
-			// command-line flag can specify the profile in use
-			profileFlag := cmd.Flag("profile")
-			if profileFlag != nil {
+			// Resolve the profile mirroring MustWorkspaceClient precedence:
+			// 1. --profile flag, 2. DATABRICKS_CONFIG_PROFILE env var (the SDK
+			// also reads it, but setting cfg.Profile here keeps any error
+			// messages we render referring to the same name), 3.
+			// [__settings__].default_profile in the config file.
+			if profileFlag := cmd.Flag("profile"); profileFlag != nil {
 				cfg.Profile = profileFlag.Value.String()
 			}
+			if cfg.Profile == "" {
+				cfg.Profile = env.Get(cmd.Context(), "DATABRICKS_CONFIG_PROFILE")
+			}
+			if cfg.Profile == "" {
+				cfg.Profile = databrickscfg.ResolveDefaultProfile(cmd.Context())
+			}
 
 			api, err := client.New(cfg)
 			if err != nil {
diff --git a/cmd/auth/token.go b/cmd/auth/token.go
@@ -18,6 +18,7 @@ import (
 	"github.com/databricks/cli/libs/databrickscfg/profile"
 	"github.com/databricks/cli/libs/env"
 	"github.com/databricks/cli/libs/flags"
+	"github.com/databricks/cli/libs/log"
 	"github.com/databricks/databricks-sdk-go/config"
 	"github.com/databricks/databricks-sdk-go/credentials/u2m"
 	"github.com/databricks/databricks-sdk-go/credentials/u2m/cache"
@@ -327,6 +328,18 @@ func resolveNoArgsToken(ctx context.Context, profiler profile.Profiler, authArgs
 		return envProfile, p, nil
 	}
 
+	// Step 2.5: Try [__settings__].default_profile from the config file.
+	// default_profile is advisory: if it points at a profile that no longer
+	// exists, fall through to the interactive picker rather than erroring.
+	if defaultProfile := databrickscfg.ResolveDefaultProfile(ctx); defaultProfile != "" {
+		p, err := loadProfileByName(ctx, defaultProfile, profiler)
+		if err != nil {
+			log.Warnf(ctx, "default_profile %q not loadable: %v", defaultProfile, err)
+		} else if p != nil {
+			return defaultProfile, p, nil
+		}
+	}
+
 	// Step 3: No env vars resolved. Load all profiles for interactive selection
 	// or non-interactive error.
 	allProfiles, err := profiler.LoadProfiles(ctx, profile.MatchAllProfiles)
diff --git a/cmd/root/auth.go b/cmd/root/auth.go
@@ -12,7 +12,6 @@ import (
 	"github.com/databricks/cli/libs/databrickscfg"
 	"github.com/databricks/cli/libs/databrickscfg/profile"
 	envlib "github.com/databricks/cli/libs/env"
-	"github.com/databricks/cli/libs/log"
 	"github.com/databricks/cli/libs/logdiag"
 	"github.com/databricks/databricks-sdk-go"
 	"github.com/databricks/databricks-sdk-go/config"
@@ -299,14 +298,8 @@ func resolveDefaultProfile(ctx context.Context, cfg *config.Config) {
 	if cfg.Profile != "" || envlib.Get(ctx, "DATABRICKS_CONFIG_PROFILE") != "" {
 		return
 	}
-	configFilePath := envlib.Get(ctx, "DATABRICKS_CONFIG_FILE")
-	resolvedProfile, err := databrickscfg.GetConfiguredDefaultProfile(ctx, configFilePath)
-	if err != nil {
-		log.Warnf(ctx, "Failed to load default profile: %v", err)
-		return
-	}
-	if resolvedProfile != "" {
-		cfg.Profile = resolvedProfile
+	if resolved := databrickscfg.ResolveDefaultProfile(ctx); resolved != "" {
+		cfg.Profile = resolved
 	}
 }
 
diff --git a/cmd/root/bundle.go b/cmd/root/bundle.go
@@ -70,6 +70,16 @@ func getProfile(cmd *cobra.Command) (value string) {
 // configureProfile applies the profile flag to the bundle.
 func configureProfile(cmd *cobra.Command, b *bundle.Bundle) {
 	profile := getProfile(cmd)
+
+	// Fall back to [__settings__].default_profile only when the bundle does
+	// not pin its own host. If the bundle declares workspace.host, applying
+	// default_profile here could route the user to a profile that points at
+	// a different host than the bundle expects — let the SDK resolve auth
+	// from the host instead.
+	if profile == "" && b.Config.Workspace.Host == "" && b.Config.Workspace.Profile == "" {
+		profile = databrickscfg.ResolveDefaultProfile(cmd.Context())
+	}
+
 	if profile == "" {
 		return
 	}
diff --git a/cmd/root/bundle_test.go b/cmd/root/bundle_test.go
diff --git a/libs/databrickscfg/ops.go b/libs/databrickscfg/ops.go
diff --git a/libs/databrickscfg/ops_test.go b/libs/databrickscfg/ops_test.go
