Skip to content

feat: add LocalDNS forward health knobs - #9174

Open
Saewon Kwak (saewoni) wants to merge 1 commit into
mainfrom
sakwa/localdns-forward-knobs
Open

feat: add LocalDNS forward health knobs#9174
Saewon Kwak (saewoni) wants to merge 1 commit into
mainfrom
sakwa/localdns-forward-knobs

Conversation

@saewoni

@saewoni Saewon Kwak (saewoni) commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Change summary

Adds CoreDNS forward plugin health/failfast knobs to LocalDNS node configuration and Corefile rendering.

  • Extends the AKS node config proto with LocalDnsHealthCheck and failfastAllUnhealthyUpstreams on LocalDnsOverrides.
  • Adds the matching AgentBaker datamodel fields for LocalDNS override serialization.
  • Renders CoreDNS forward health_check only when healthCheck.duration is set and non-empty, matching CoreDNS syntax: health_check DURATION [no_rec] [domain FQDN].
  • Renders failfast_all_unhealthy_upstreams when explicitly enabled.
  • Wires both LocalDNS Corefile renderers: aks-node-controller/parser/templates/localdns.toml.gtpl and the legacy pkg/agent/baker.go template.
  • Adds parser matrix coverage for nil/empty healthCheck, empty/missing duration, duration-only, no_rec true/false, domain set/unset/empty, VnetDNS and KubeDNS rendering paths, and failfast true/false.
  • Adds legacy pkg/agent renderer coverage for health_check, failfast_all_unhealthy_upstreams, and empty domain omission.

This intentionally avoids inventing a default health_check duration in AgentBaker. If healthCheck is nil or duration is unset/empty, the templates omit health_check and leave CoreDNS default behavior intact.

Testing

  • cd aks-node-controller && go test ./parser ./pkg/nodeconfigutils
  • go test ./pkg/agent ./pkg/agent/datamodel

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedAug 12, 2026, 6:40 PM

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   12 suites   45s ⏱️
389 tests 389 ✅ 0 💤 0 ❌
392 runs  392 ✅ 0 💤 0 ❌

Results for commit 02661e6.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends LocalDNS configuration to support CoreDNS forward plugin upstream health behavior by adding new knobs to the config model/proto and wiring them into the generated LocalDNS Corefile template.

Changes:

  • Add failfast_all_unhealthy_upstreams and health_check configuration fields to LocalDNS overrides (datamodel + aks-node-controller proto).
  • Render the new forward plugin directives in the LocalDNS Corefile template for both VnetDNS and KubeDNS override blocks.
  • Add a parser unit test covering health_check + failfast_all_unhealthy_upstreams output and regenerate protobuf Go bindings.

Reviewed changes

Copilot reviewed 4 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/agent/datamodel/types.go Adds datamodel structs/fields for LocalDNS forward health knobs.
aks-node-controller/proto/aksnodeconfig/v1/localdns_config.proto Extends the LocalDNS overrides proto schema with failfast + health check message.
aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go Regenerated bindings for the updated LocalDNS proto.
aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go Regenerated bindings (import ordering change).
aks-node-controller/pkg/gen/aksnodeconfig/v1/config.pb.go Regenerated bindings (header formatting).
aks-node-controller/parser/templates/localdns.toml.gtpl Wires new forward plugin directives into Corefile rendering.
aks-node-controller/parser/helper_test.go Adds unit test validating Corefile output for health_check + failfast.
Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (1)

aks-node-controller/parser/templates/localdns.toml.gtpl:121

  • Same issue as the earlier forward block: checking .Interval / .Domain pointer fields can render empty strings, producing invalid health_check arguments. Check getter return values (strings) so empty values are treated as unset.
        {{- if $override.HealthCheck}}
        health_check {{if $override.HealthCheck.Interval}}{{$override.HealthCheck.GetInterval}}{{else}}0.5s{{end}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +48 to +50
{{- if $override.HealthCheck}}
health_check {{if $override.HealthCheck.Interval}}{{$override.HealthCheck.GetInterval}}{{else}}0.5s{{end}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
{{- end}}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in the latest update. Both VnetDNS and KubeDNS template blocks now guard with GetDuration/GetDomain, so empty duration omits health_check and empty domain omits the domain clause.

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from 2e8f898 to d18bb92 Compare August 11, 2026 20:33
Copilot AI review requested due to automatic review settings August 11, 2026 20:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 7 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (2)

aks-node-controller/parser/templates/localdns.toml.gtpl:50

  • The health_check rendering logic checks pointer presence (Interval/Domain) rather than non-empty values. If the config explicitly sets interval/domain to an empty string (non-nil pointer), this will emit health_check (blank interval) and/or domain (blank domain), which can produce an invalid CoreDNS Corefile and break DNS.
        {{- if $override.HealthCheck}}
        health_check {{if $override.HealthCheck.Interval}}{{$override.HealthCheck.GetInterval}}{{else}}0.5s{{end}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

aks-node-controller/parser/templates/localdns.toml.gtpl:121

  • Same issue as above for the cluster listener block: checking Interval/Domain pointer presence can emit empty tokens when the pointer is set to "". Prefer checking the resolved string values (GetInterval/GetDomain) so empty strings fall back to defaults/omission.
        {{- if $override.HealthCheck}}
        health_check {{if $override.HealthCheck.Interval}}{{$override.HealthCheck.GetInterval}}{{else}}0.5s{{end}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

Copilot AI review requested due to automatic review settings August 11, 2026 20:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 7 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (3)

aks-node-controller/parser/templates/localdns.toml.gtpl:121

  • This condition only checks that Duration is non-nil; if Duration is set to an empty string, the template will emit an invalid health_check directive (missing duration) and CoreDNS will fail to parse the forward block. Consider also checking that the duration value is non-empty, and similarly avoid emitting an empty domain value.
        {{- if and $override.HealthCheck $override.HealthCheck.Duration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

pkg/agent/datamodel/types.go:2624

  • The new LocalDNSOverrides knobs (FailfastAllUnhealthyUpstreams / HealthCheck) are added to the datamodel but aren’t consumed anywhere in the AgentBaker localdns Corefile generation path (GenerateLocalDNSCoreFile renders localDNSCoreFileTemplateString in pkg/agent/baker.go, which currently doesn’t reference these fields). As a result, setting these fields via the AgentBaker API won’t change the generated Corefile, so the feature is only partially wired.
	// Determines the handling of requests when all upstream servers are unhealthy.
	FailfastAllUnhealthyUpstreams *bool `json:"failfastAllUnhealthyUpstreams,omitempty"`
	// Configures CoreDNS forward plugin health checking behavior for upstream servers.
	HealthCheck *LocalDNSHealthCheck `json:"healthCheck,omitempty"`

aks-node-controller/parser/templates/localdns.toml.gtpl:50

  • This condition only checks that Duration is non-nil; if Duration is set to an empty string, the template will emit an invalid health_check directive (missing duration) and CoreDNS will fail to parse the forward block. Consider also checking that the duration value is non-empty, and similarly avoid emitting an empty domain value.

This issue also appears on line 119 of the same file.

        {{- if and $override.HealthCheck $override.HealthCheck.Duration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from 88ccb0b to 4666fe8 Compare August 11, 2026 20:49
Copilot AI review requested due to automatic review settings August 11, 2026 20:50
@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from 4666fe8 to 2780ba7 Compare August 11, 2026 20:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 7 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (2)

aks-node-controller/parser/templates/localdns.toml.gtpl:121

  • Same issue as the node listener block above: duration: "" (or domain: "") will still satisfy the current pointer-based condition and render an invalid health_check line (or domain with an empty value). Guard against empty strings to avoid generating an invalid Corefile.
        {{- if and $override.HealthCheck $override.HealthCheck.Duration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

aks-node-controller/parser/templates/localdns.toml.gtpl:50

  • The template guards on $override.HealthCheck.Duration being non-nil, but does not guard against the duration being an empty string. If a caller sets duration: "" (non-nil pointer), this will render health_check with an empty first argument, which CoreDNS will treat as an invalid Corefile. Similarly, an explicitly-empty domain would emit domain .

Prefer checking the computed strings are non-empty before rendering the directive/optional domain segment.

This issue also appears on line 119 of the same file.

        {{- if and $override.HealthCheck $override.HealthCheck.Duration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from 2780ba7 to f7055da Compare August 11, 2026 21:06
Copilot AI review requested due to automatic review settings August 11, 2026 21:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 7 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (4)

aks-node-controller/parser/templates/localdns.toml.gtpl:121

  • Same issue as above in the KubeDNS overrides block: checking $override.HealthCheck.Domain will render domain even when the value is an empty string (non-nil pointer), producing invalid CoreDNS syntax. Check .GetDomain instead.
        {{- if and $override.HealthCheck $override.HealthCheck.GetDuration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

aks-node-controller/parser/templates/localdns.toml.gtpl:50

  • domain rendering guard checks pointer presence ($override.HealthCheck.Domain) rather than a non-empty value. If domain is explicitly set to an empty string, this will emit domain and produce an invalid health_check directive. Prefer checking .GetDomain (non-empty) to match the duration handling.

This issue also appears on line 119 of the same file.

        {{- if and $override.HealthCheck $override.HealthCheck.GetDuration}}
        health_check {{$override.HealthCheck.GetDuration}}{{if $override.HealthCheck.GetNoRec}} no_rec{{end}}{{if $override.HealthCheck.Domain}} domain {{$override.HealthCheck.GetDomain}}{{end}}
        {{- end}}

pkg/agent/datamodel/types.go:2624

  • These new LocalDNS override fields are added to the datamodel, but there are no references to them elsewhere in pkg/agent (including the in-repo LocalDNS Corefile template in pkg/agent/baker.go). As-is, AgentBaker's rendered LocalDNS Corefile will ignore healthCheck and failfastAllUnhealthyUpstreams, which appears to contradict the PR description about Corefile rendering updates. Either wire these fields into the AgentBaker Corefile template/rendering path, or clarify that only the aks-node-controller renderer is expected to honor them.
	// Determines the handling of requests when all upstream servers are unhealthy.
	FailfastAllUnhealthyUpstreams *bool `json:"failfastAllUnhealthyUpstreams,omitempty"`
	// Configures CoreDNS forward plugin health checking behavior for upstream servers.
	HealthCheck *LocalDNSHealthCheck `json:"healthCheck,omitempty"`

aks-node-controller/parser/helper_test.go:2055

  • The matrix covers nil/missing duration cases, but it doesn't cover the edge case where domain is explicitly set to an empty string (non-nil pointer). Given the template currently guards domain by pointer presence in some places, adding a test for Domain: to.Ptr("") (with duration set) helps prevent emitting domain in the generated Corefile.
		{
			name: "duration and domain",
			healthCheck: &aksnodeconfigv1.LocalDnsHealthCheck{
				Duration: to.Ptr("1s"),
				Domain:   to.Ptr("health.local."),
			},
			wantContains:    "health_check 1s domain health.local.",
			wantNotContains: []string{"no_rec"},
		},

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from f7055da to d0703f1 Compare August 11, 2026 21:47
Copilot AI review requested due to automatic review settings August 11, 2026 21:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 9 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (4)

pkg/agent/baker.go:2387

  • Same issue as the earlier forward block: FailfastAllUnhealthyUpstreams is a *bool, so pointer presence makes this condition truthy even when the underlying value is false. Switch the template to use a bool getter (e.g., GetFailfastAllUnhealthyUpstreams()) so explicit false omits the directive.
        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/baker.go:2316

  • This if $override.FailfastAllUnhealthyUpstreams check will be true for any non-nil *bool, including when the value is explicitly set to false, so failfast_all_unhealthy_upstreams may render incorrectly. Use a bool-returning accessor (e.g., GetFailfastAllUnhealthyUpstreams()) or otherwise compare the underlying bool value rather than pointer presence.

This issue also appears on line 2385 of the same file.

        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/datamodel/types.go:2643

  • FailfastAllUnhealthyUpstreams is a *bool, but templates can’t dereference it. Without a GetFailfastAllUnhealthyUpstreams() accessor, call sites (like the legacy Corefile template) will treat any non-nil pointer as truthy and may render failfast_all_unhealthy_upstreams even when explicitly set to false. Add a getter that returns false when the pointer is nil and the pointed value otherwise, mirroring the generated proto API.
	ServeStale                  string `json:"serveStale,omitempty"`
	// Determines the handling of requests when all upstream servers are unhealthy.
	FailfastAllUnhealthyUpstreams *bool `json:"failfastAllUnhealthyUpstreams,omitempty"`
	// Configures CoreDNS forward plugin health checking behavior for upstream servers.
	HealthCheck *LocalDNSHealthCheck `json:"healthCheck,omitempty"`

pkg/agent/baker_test.go:813

  • The legacy Corefile renderer test only exercises FailfastAllUnhealthyUpstreams=true. Add a sub-case with FailfastAllUnhealthyUpstreams=false and assert failfast_all_unhealthy_upstreams is omitted, to prevent regressions (and to catch pointer-truthiness rendering bugs in templates).
			It("renders localdns forward health knobs", func() {
				config.AgentPoolProfile.LocalDNSProfile = &datamodel.LocalDNSProfile{
					EnableLocalDNS: true,
					VnetDNSOverrides: map[string]*datamodel.LocalDNSOverrides{".": {
						QueryLogging: "Log", Protocol: "PreferUDP", ForwardDestination: "VnetDNS", ForwardPolicy: "Sequential",
						MaxConcurrent: to.Int32Ptr(1000), CacheDurationInSeconds: to.Int32Ptr(3600), ServeStaleDurationInSeconds: to.Int32Ptr(3600), ServeStale: "Immediate",
						FailfastAllUnhealthyUpstreams: to.BoolPtr(true), HealthCheck: &datamodel.LocalDNSHealthCheck{Duration: to.StringPtr("1s"), NoRec: to.BoolPtr(true), Domain: to.StringPtr("health.local.")},
					}},
					KubeDNSOverrides: map[string]*datamodel.LocalDNSOverrides{".": {
						QueryLogging: "Error", Protocol: "PreferUDP", ForwardDestination: "ClusterCoreDNS", ForwardPolicy: "Sequential",
						MaxConcurrent: to.Int32Ptr(1000), CacheDurationInSeconds: to.Int32Ptr(3600), ServeStaleDurationInSeconds: to.Int32Ptr(3600), ServeStale: "Immediate",
						HealthCheck: &datamodel.LocalDNSHealthCheck{Duration: to.StringPtr("2s"), Domain: to.StringPtr("")},
					}},
				}
				localDNSCoreFile, err := GenerateLocalDNSCoreFile(config, config.AgentPoolProfile, false)
				Expect(err).To(BeNil())
				Expect(localDNSCoreFile).To(ContainSubstring("health_check 1s no_rec domain health.local."))
				Expect(localDNSCoreFile).To(ContainSubstring("failfast_all_unhealthy_upstreams"))
				Expect(localDNSCoreFile).To(ContainSubstring("health_check 2s"))
				Expect(localDNSCoreFile).ToNot(ContainSubstring("domain \n"))
			})

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from d0703f1 to a8cf296 Compare August 11, 2026 21:57
Copilot AI review requested due to automatic review settings August 11, 2026 21:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 9 changed files in this pull request and generated 1 comment.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (3)

pkg/agent/baker.go:2387

  • Same issue as above: FailfastAllUnhealthyUpstreams is a *bool, so this if will be true whenever the pointer is non-nil (including when it points to false). Use a bool-returning getter and call it here to avoid rendering failfast_all_unhealthy_upstreams when explicitly disabled.
        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/baker.go:2316

  • FailfastAllUnhealthyUpstreams is a *bool, but in Go templates any non-nil pointer is truthy regardless of its underlying value. This if will therefore render failfast_all_unhealthy_upstreams even when the field is explicitly set to false.

Fix by dereferencing via a bool-returning getter (nil/false => omit, true => render) and use that getter in the template (as the aks-node-controller template does).

This issue also appears on line 2385 of the same file.

        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/datamodel/types.go:2644

  • Add a GetFailfastAllUnhealthyUpstreams() helper on LocalDNSOverrides (matching the protobuf getter style) so templates can safely treat this tri-state *bool as a boolean. Without a getter, template code will tend to test the pointer itself, which is truthy for both true and false when non-nil.
	// Determines the handling of requests when all upstream servers are unhealthy.
	FailfastAllUnhealthyUpstreams *bool `json:"failfastAllUnhealthyUpstreams,omitempty"`
	// Configures CoreDNS forward plugin health checking behavior for upstream servers.
	HealthCheck *LocalDNSHealthCheck `json:"healthCheck,omitempty"`
}

Comment thread pkg/agent/baker_test.go
Comment on lines +793 to +813
It("renders localdns forward health knobs", func() {
config.AgentPoolProfile.LocalDNSProfile = &datamodel.LocalDNSProfile{
EnableLocalDNS: true,
VnetDNSOverrides: map[string]*datamodel.LocalDNSOverrides{".": {
QueryLogging: "Log", Protocol: "PreferUDP", ForwardDestination: "VnetDNS", ForwardPolicy: "Sequential",
MaxConcurrent: to.Int32Ptr(1000), CacheDurationInSeconds: to.Int32Ptr(3600), ServeStaleDurationInSeconds: to.Int32Ptr(3600), ServeStale: "Immediate",
FailfastAllUnhealthyUpstreams: to.BoolPtr(true), HealthCheck: &datamodel.LocalDNSHealthCheck{Duration: to.StringPtr("1s"), NoRec: to.BoolPtr(true), Domain: to.StringPtr("health.local.")},
}},
KubeDNSOverrides: map[string]*datamodel.LocalDNSOverrides{".": {
QueryLogging: "Error", Protocol: "PreferUDP", ForwardDestination: "ClusterCoreDNS", ForwardPolicy: "Sequential",
MaxConcurrent: to.Int32Ptr(1000), CacheDurationInSeconds: to.Int32Ptr(3600), ServeStaleDurationInSeconds: to.Int32Ptr(3600), ServeStale: "Immediate",
HealthCheck: &datamodel.LocalDNSHealthCheck{Duration: to.StringPtr("2s"), Domain: to.StringPtr("")},
}},
}
localDNSCoreFile, err := GenerateLocalDNSCoreFile(config, config.AgentPoolProfile, false)
Expect(err).To(BeNil())
Expect(localDNSCoreFile).To(ContainSubstring("health_check 1s no_rec domain health.local."))
Expect(localDNSCoreFile).To(ContainSubstring("failfast_all_unhealthy_upstreams"))
Expect(localDNSCoreFile).To(ContainSubstring("health_check 2s"))
Expect(localDNSCoreFile).ToNot(ContainSubstring("domain \n"))
})

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. The legacy pkg/agent template now uses GetFailfastAllUnhealthyUpstreams instead of pointer truthiness, and I added coverage for explicit false to ensure failfast_all_unhealthy_upstreams is omitted. Package tests compile and pass.

@aks-node-assistant

Copy link
Copy Markdown
Contributor

Clawpilot AgentBaker Linux gate detective

  • Run: 176131639
  • Failed job/stage/task: Run AgentBaker E2E / Test_AzureLinuxV3_MANA
  • Summary: AzureLinux V3 MANA E2E reached VM validation, then failed the eBPF host-routing iptables allowlist check because a DHCP INPUT rule did not match expected patterns.
  • Likely cause / signature: $(System.Collections.Hashtable.sig) — Network validation signature: unexpected DHCP INPUT iptables rule while eBPF host routing is enabled.
  • Confidence: Medium
  • Recommended owner/action: Node Lifecycle/ACN review whether PR 9174 LocalDNS/kubelet config changes can influence eBPF host-routing rules; if not, treat as transient network-validation flake. Later retry for this PR succeeded.
  • Strongest alternative: Infrastructure/network-rule race is the strongest alternative and is plausible because a later same-PR run succeeded; PR-change remains possible because this PR touches LocalDNS and agent config surfaces.
  • Evidence links: run, timeline, flakiness wiki
  • Wiki signature: e2e-ebpf-host-routing-dhcp-input-rule-unexpected

Copilot AI review requested due to automatic review settings August 12, 2026 18:30
@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from a8cf296 to c9ff53f Compare August 12, 2026 18:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 9 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file
Suppressed comments (4)

pkg/agent/baker.go:2393

  • Same issue as above in the KubeDNS override block: checking the *bool pointer directly will render failfast_all_unhealthy_upstreams even when the value is explicitly set to false. Use a boolean getter instead.
        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/baker_test.go:817

  • This test covers the enabled case for failfast_all_unhealthy_upstreams, but it doesn't cover the important edge-case where the field is explicitly set to false (non-nil pointer). Given the template uses Go template truthiness, adding an explicit-false assertion will prevent regressions.
				Expect(localDNSCoreFile).To(ContainSubstring("health_check 1s no_rec domain health.local."))
				Expect(localDNSCoreFile).To(ContainSubstring("failfast_all_unhealthy_upstreams"))
				Expect(localDNSCoreFile).To(ContainSubstring("health_check 2s"))
				Expect(localDNSCoreFile).ToNot(ContainSubstring("domain \n"))

pkg/agent/baker.go:2319

  • FailfastAllUnhealthyUpstreams is a *bool, but the template currently checks the pointer itself. In Go templates any non-nil pointer is truthy, so setting this to false (non-nil) would still render failfast_all_unhealthy_upstreams. Switch the condition to a boolean getter that dereferences the pointer safely.

This issue also appears on line 2391 of the same file.

        {{- if $override.FailfastAllUnhealthyUpstreams}}
        failfast_all_unhealthy_upstreams
        {{- end}}

pkg/agent/datamodel/types.go:2644

  • FailfastAllUnhealthyUpstreams is optional (*bool) but LocalDNSOverrides doesn't currently expose a getter like the generated proto does. Adding GetFailfastAllUnhealthyUpstreams() avoids Go-template pointer-truthiness bugs and keeps the legacy renderer consistent with the aks-node-controller template usage.
	// Determines the handling of requests when all upstream servers are unhealthy.
	FailfastAllUnhealthyUpstreams *bool `json:"failfastAllUnhealthyUpstreams,omitempty"`
	// Configures CoreDNS forward plugin health checking behavior for upstream servers.
	HealthCheck *LocalDNSHealthCheck `json:"healthCheck,omitempty"`
}

@saewoni
Saewon Kwak (saewoni) force-pushed the sakwa/localdns-forward-knobs branch from c9ff53f to 02661e6 Compare August 12, 2026 18:39
Copilot AI review requested due to automatic review settings August 12, 2026 18:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 9 changed files in this pull request and generated no new comments.

Files not reviewed (2)
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/kubelet_config.pb.go: Generated file
  • aks-node-controller/pkg/gen/aksnodeconfig/v1/localdns_config.pb.go: Generated file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants