feat(Kind): add AddManifest resource for kubectl apply after cluster ready - #1481
Draft
tamirdresher wants to merge 1 commit into
Draft
feat(Kind): add AddManifest resource for kubectl apply after cluster ready#1481tamirdresher wants to merge 1 commit into
tamirdresher wants to merge 1 commit into
Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/CommunityToolkit/Aspire/main/eng/scripts/dogfood-pr.sh | bash -s -- 1481Or
iex "& { $(irm https://raw.githubusercontent.com/CommunityToolkit/Aspire/main/eng/scripts/dogfood-pr.ps1) } 1481" |
tamirdresher
force-pushed
the
feat/kind-with-manifest
branch
4 times, most recently
from
July 27, 2026 06:06
5180420 to
d708bd3
Compare
…ready
Adds Kubernetes manifest resources for Kind clusters, aligned with the existing K3s manifest pattern while keeping Kind's host-side kubectl execution model.
Public API surface:
cluster.AddManifest(name, manifestPath)
.WithNamespace(ns)
.WithRecursive()
.WithServerSideApply(forceConflicts: false)
.WithFieldManager("aspire-apphost")
.WithApplyTimeout(TimeSpan.FromMinutes(5))
.WithCrdWaitTimeout(TimeSpan.FromMinutes(5))
.WithCrdWaitBehavior(CrdWaitBehavior.Fail)
cluster.AddManifestFromContent(name, content)
.WithNamespace(ns)
.WithServerSideApply(forceConflicts: false)
.WithFieldManager("aspire-apphost")
Key behavior:
- K8sManifestResource extends KindDeployedResource(name, parent) and is exported for Aspire tooling.
- AddManifest resolves relative paths against AppHostDirectory; Kustomize detection happens at apply time and supports case-insensitive marker variants.
- AddManifestFromContent applies inline YAML via kubectl apply -f - using stdin, avoiding temporary files.
- KubectlManager probes kubectl cluster-info before apply, bounds apply with ApplyTimeout, and fails CRD waits by default with an opt-in best-effort mode.
- WithRecursive is honored for file/directory applies and ignored with a warning for Kustomize or stdin applies.
- Manifest resources rely on Running/FailedToStart state rather than a default empty-selector workload health check.
Downstream resources can WaitFor the manifest resource so they only start once kubectl apply succeeds:
var crds = cluster.AddManifest("crds", "./manifests/crds.yaml");
builder.AddContainer("my-operator", "my-org/operator")
.WithReference(cluster)
.WaitFor(crds);
Manifests persist with the cluster - deleted with session clusters, retained with persistent clusters.
Tests cover resource registration, parent wiring, null guards, AppHost-relative path resolution, absolute-path preservation, Kustomize detection, stdin apply argument shape and input piping, recursive ignore behavior, cluster-info retry, apply timeout cancellation, CRD wait argument shape, and default/best-effort CRD wait behavior.
Closes: none (net new)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7c8c8109-2ae6-407b-8740-8c3018abe8f1
tamirdresher
force-pushed
the
feat/kind-with-manifest
branch
from
July 27, 2026 07:05
d708bd3 to
5c80dc4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
For dev-loop scenarios that run against a Kind cluster during F5 (operators, controllers, admission webhooks, or in-source contributions to an OSS Kubernetes project) the AppHost often needs to apply a set of
CustomResourceDefinition,Namespace,ServiceAccount,ClusterRole,ClusterRoleBinding, orConfigMapobjects to the cluster after the cluster reaches Running but before dependent resources start.Today
CommunityToolkit.Aspire.Hosting.Kindhandles the Helm side of this viacluster.AddHelmChart(...), but if you just have static YAML you have to hand-roll a lifecycle hook in the AppHost. This PR adds the natural equivalent ofAddHelmChartfor thekubectl applycase.This adds runtime manifest apply (at F5 /
aspire start). It is distinct fromAspire.Hosting.Kubernetes'sPublishAsKubernetesService(r => r.AdditionalResources.Add(...)), which is build-time (baked into a generated Helm chart ataspire publish). Both have legitimate use cases; this PR fills the runtime gap for local Kind clusters.What this adds
A new resource + extension API surface, all public APIs exported for Aspire tooling:
Design
Mirrors the existing Kind deployed-resource pattern and K3s manifest semantics while keeping Kind's host-side kubectl execution model:
KindHelmChartResourceK8sManifestResource : KindDeployedResourceHelmManagerKubectlManager(wrapsIProcessRunner, internal static argument builders for testability)KindHelmChartResourceBuilderExtensionsKindManifestResourceBuilderExtensionsLifecycle callback matches
AddHelmChartstep-for-step:WaitForResourceAsync(parent.Name, Running)PublishAsync(BeforeResourceStartedEvent)PublishUpdateAsync(Starting)KubectlManager.ApplyAsync->PublishUpdateAsync(Running)}PublishUpdateAsync(FailedToStart)-> rethrow }KubectlManager.ApplyAsyncnow probeskubectl cluster-infobefore apply so the Kind resource state and Kubernetes API reachability are both satisfied before manifests are applied.New public API
AddManifest(name, manifestPath)kubectl apply -f <path> --kubeconfig=<cluster kubeconfig>orkubectl apply -k <path>for Kustomize directoriesAddManifestFromContent(name, content)kubectl apply -f - --kubeconfig=<cluster kubeconfig>with content on stdinWithNamespace(ns)(inherited fromKindDeployedResource)--namespace <ns>(does not create the namespace for manifests)WithRecursive()--recursivefor normal file/directory applies; ignored with a warning for Kustomize/stdinWithServerSideApply(forceConflicts: false)--server-side(+--force-conflicts)WithFieldManager(name)--field-manager <name>WithApplyTimeout(timeout)kubectl applyexecution time (default 5 minutes)WithCrdWaitTimeout(timeout)Establishedwait timeout (default 5 minutes)WithCrdWaitBehavior(behavior)Failby default;BestEffortlogs a warning and continuesRound 2 changes
After review against
CommunityToolkit.Aspire.Hosting.K3s'sAddK8sManifestpattern, this branch also:KindManifestResourcetoK8sManifestResourceand updated the dashboard resource type label toK8s Manifest/K8s Kustomize.AddManifestpaths againstbuilder.ApplicationBuilder.AppHostDirectorybefore passing them to kubectl.kubectl apply -k;WithRecursive()is ignored with a warning for Kustomize.kubectl applyoutput for CRDs and waited for applied CRDs to reachEstablished.AddManifestFromContent(name, content), implemented withkubectl apply -f -and stdin support in the internalIProcessRunnerabstraction.Round 4 — squad review fixes
Addressed the 5-agent review findings:
[AspireExport]toK8sManifestResourceso the public API export claim is accurate..WithCrdWaitTimeout(...)and.WithCrdWaitBehavior(CrdWaitBehavior.BestEffort)for opt-in best-effort behavior.kubectl cluster-infoAPI-reachability retry loop beforekubectl apply..WithApplyTimeout(...)and cancellation for hungkubectl applycalls.Kustomization/Kustomization.yml.--kubeconfig=<path>from debug command logs.operatorkeyword example by renaming it tooperatorContainer.WaitFor(manifestResource).Round 5 — verification-pass fixes
Addressed the final verification-pass findings:
CrdWaitBehaviorfor Aspire tooling.--kubeconfig=<path>and--kubeconfig <path>debug-log forms.1s, and capped values above 1 hour.Tests
xUnit v3 coverage in
KindManifestTests.cs/AddKindClusterTests.csnow includes:WithRecursive,WithServerSideApply(forceConflicts),WithFieldManager,WithApplyTimeout,WithCrdWaitTimeout, andWithCrdWaitBehaviorKubectlManager.CreateApplyArguments,CreateWaitArguments, and cluster-info retry behaviorFull
CommunityToolkit.Aspire.Hosting.Kind.Testsproject passes locally: 174 / 174.Real-world usage
Upstreamed from https://github.com/tamirdresher/aspire-argocd-dev-loop, which spins up an argo-cd F5 inner loop against a Kind cluster and uses
AddManifestto install the ArgoCD manifests before theargocd-serverandrepo-serverprojects start. Prior to this API the sample carried a hand-rolledOnBeforeStarthook with its ownProcess.Start("kubectl", ...).Open questions for reviewers
No blockers remaining from prior review rounds.
Deferred: naming consistency between this integration's
AddManifestandCommunityToolkit.Aspire.Hosting.K3s'sAddK8sManifest. Both are[AspireExport]-ed public API in separate packages; unifying would be a breaking change for K3s consumers. Happy to align on either name if maintainers have a preference.Follow-up
I have a separate small PR queued:
WithPortMapping(hostPort, containerPort)as a fluent shorthand over the existingWithKindConfig(Action<KindConfigModel>)for exposing NodePort services on127.0.0.1. Shipping it separately to keep this PR focused on the new resource type.Docs
README section updated in
src/CommunityToolkit.Aspire.Hosting.Kind/README.md: usage examples for single file, recursive directory, Kustomize auto-detect, inline content, server-side apply, CRD wait behavior, namespace behavior, security notes, and theAPI referencetable entries.Checklist
KindHelmChartResourceand K3sK8sManifestResourcesemantics)[AspireExport]Follow-on docs
learn.microsoft.com/dotnet/aspire/community-toolkit/hosting-kindlives inMicrosoftDocs/aspire-docs, outside this fork's tree, and will need a matching update forAddManifest. That can be handled as a separate PR after this merges.