feat(exposure): CloudFront mode for domain-less deployments#680
Draft
allamand wants to merge 3 commits into
Draft
feat(exposure): CloudFront mode for domain-less deployments#680allamand wants to merge 3 commits into
allamand wants to merge 3 commits into
Conversation
Contributor
Author
|
Companion issue for the external agent-platform repo charts (agent-gateway, langfuse ingresses): aws-samples/sample-agent-platform-on-eks#11 |
446ae43 to
d3d7ed9
Compare
d3d7ed9 to
7d0d5ce
Compare
Implements conditional ingress rendering: - exposure.mode: 'domain' (default) — HTTPS:443, host-based routing, TLS - exposure.mode: 'cloudfront' — HTTP:80, no host, CloudFront terminates TLS Changes: - Add exposure.mode to config schema and template - Update ingress templates (keycloak, argo-workflows, grafana, jupyterhub, kubeflow) - Pass exposure_mode annotation through registry valuesObject - Add hub:cloudfront Taskfile task (creates ALB + CloudFront distribution) - Update fleet-secret chart to propagate exposure_mode annotation Closes #677
7d0d5ce to
e9bd3b3
Compare
Collaborator
Review:
|
| # | Issue | Detail | Fix |
|---|---|---|---|
| 1 | No OriginReadTimeout set |
CloudFront config JSON doesn't specify OriginReadTimeout — defaults to 30s. The Terraform reference (platform/infra/terraform/common/cloudfront.tf) uses 60s. Agent Gateway SSE requires the origin to send data within this window or CloudFront drops the connection. |
Add "OriginReadTimeout": 60 to CustomOriginConfig in the distribution JSON |
| 2 | No OriginKeepaliveTimeout set |
Defaults to 5s. Terraform reference uses 30s. Short keepalive means CloudFront opens new TCP connections frequently, adding latency. | Add "OriginKeepaliveTimeout": 30 to CustomOriginConfig |
| 3 | Missing X-Forwarded-Proto / X-Forwarded-Port custom headers |
Terraform adds X-Forwarded-Proto: https and X-Forwarded-Port: 443 as custom origin headers. Without these, Keycloak generates redirect URIs with http:// instead of https:// (it sees the ALB connection as HTTP). |
Add CustomHeaders to the origin config with these two headers |
| 4 | No separate cache behavior for /keycloak/* |
Terraform has an ordered_cache_behavior for Keycloak with TTL=0 and all headers/cookies forwarded. The branch uses a single default behavior. Keycloak requires all cookies and headers for session management — the AllViewer origin request policy should cover this, but explicit TTL=0 prevents stale auth responses. |
Add an ordered_cache_behavior for /keycloak/* with MinTTL=0, DefaultTTL=0, MaxTTL=0 |
| 5 | No destroy cleanup for CloudFront/ALB | The destroy task doesn't delete the CloudFront distribution, the pre-created ALB, or the dedicated security group. These will be orphaned on teardown. |
Add CloudFront disable+delete, ALB delete, and SG delete to the destroy task (CloudFront requires disabling first, then waiting, then deleting) |
| 6 | CloudFront deployment propagation delay | CloudFront distributions take 5-15 minutes to deploy. wait_for_deployment = false in Terraform skips this, but the Taskfile should either wait or warn that the domain won't be reachable immediately. |
Add a wait loop or print a warning after hub:cloudfront |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements conditional ingress rendering based on
exposure.modeconfig:domain(default): HTTPS:443, host-based routing, ACM cert requiredcloudfront: HTTP:80, no host, CloudFront terminates TLS — no custom domain neededProblem
In Workshop Studio environments there's no custom domain or Route53 hosted zone. The ALB controller fails with
no certificate found for hostbecause ingresses specify HTTPS with a host that has no ACM cert.Changes
exposure.modetoconfig.yamlschema and templateexposure_modeannotation through registryvaluesObjectto addon chartshub:cloudfrontTaskfile task (creates ALB + CloudFront distribution, updates Secrets Manager)fleet-secretchart to propagateexposure_modeannotationTesting
Verified on live hub cluster:
/keycloakvia ALBplatformgroup without cert errorsCloses #677