Skip to content

content(what-is): expand the CircleCI secret explainer#19270

Draft
alexleventer wants to merge 1 commit into
masterfrom
aleventer/circleci-secret-rewrite
Draft

content(what-is): expand the CircleCI secret explainer#19270
alexleventer wants to merge 1 commit into
masterfrom
aleventer/circleci-secret-rewrite

Conversation

@alexleventer
Copy link
Copy Markdown
Contributor

Summary

Rewrites content/what-is/what-is-a-circleci-secret.md for SEO and AEO. The previous page introduced the topic but didn't differentiate scopes or address OIDC, which is now the recommended approach for any cloud credential. The new version organizes secrets around the three scopes engineers actually pick between (project env vars, contexts, restricted contexts) and adds an OIDC comparison.

What changed

  • Opening definition — bolded one-sentence definition (encrypted env var, project or context scope, injected into job environment), followed by a note on the three scopes and OIDC as a stored-secret alternative.
  • Use cases — concrete list (cloud credentials, registry tokens, deploy tokens, third-party APIs, signing keys, Pulumi access tokens).
  • Scope table — project env vars vs. contexts vs. restricted contexts: where to define, who can read, best-fit use cases.
  • Storage table — submission, storage, delivery, in-job, in-logs, post-job.
  • Job access — YAML example showing context opt-in and branch filters; caveats on echo and context membership.
  • OIDC section — comparison table (stored secret vs. OIDC + cloud role) and where each fits.
  • Limits — no native rotation, no versioning, write-only after creation, limited audit detail, binary context access.
  • Best practices — prefer OIDC, restricted contexts for prod, don't echo, contexts over duplicated env vars, scheduled rotation, runtime policies.
  • Pulumi section — YAML example using the Pulumi Orb with a context that brokers ESC-issued credentials.
  • FAQ — ten doubt-removers (encryption, contexts vs. project vars, restricted contexts, rotation, UI read-back, sharing across projects, OIDC, log masking, per-job ACLs, audit).
  • Cross-links — secrets management, GitHub Actions/Cloudflare secrets, HashiCorp Vault, AWS Secrets Manager, CI/CD.

Test plan

  • make serve; visit /what-is/what-is-a-circleci-secret/ and confirm tables and code blocks render
  • Spot-check cross-links (/docs/iac/packages-and-automation/continuous-delivery/circleci/, /product/esc/)
  • CI lint + pinned review

🤖 Generated with Claude Code

Rewrite for SEO and AEO: quotable opening definition, semantic
chunking with question-style H2s, FAQ section targeting
doubt-removers, citable claims, and cross-links to secrets
management resources.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added review:triaging Claude Triage is currently classifying the PR domain:docs PR touches technical docs review:in-progress Claude review is currently running and removed review:triaging Claude Triage is currently classifying the PR labels May 20, 2026
@pulumi-bot
Copy link
Copy Markdown
Collaborator

@github-actions
Copy link
Copy Markdown
Contributor

Pre-merge Review — Last updated 2026-05-20T16:53:57Z

Tip

Summary: This PR rewrites content/what-is/what-is-a-circleci-secret.md from a stub into a full explainer (scopes, storage, OIDC, limits, best practices, Pulumi integration, FAQ), paralleling the existing /what-is/what-is-a-github-action-secret/ sibling. Reader-blocking risk is concentrated in the CircleCI feature claims: several diverge from current CircleCI docs (the context.secrets.accessed audit event added in Nov 2023, Expression Restrictions for sub-project context filtering, the "config policies" feature being mis-named as "runtime policies" with a 404 link), the OIDC-issuance claim overstates the default ("for every job"), and the Pulumi-on-CircleCI bullet pulls in pulumi/auth-actions, which is a GitHub-only Action. Mechanics are clean (frontmatter parses, no alias/URL collisions, content-only PR so no Hugo build delta). Investigations: external claim verification (46 claims, 4 specialists), frontmatter sweep, temporal-trigger sweep, Vale prose lint; cross-sibling fan-out skipped (not in a templated section), code-execution skipped (no static/programs/ change).

Review confidence:

Dimension Level Notes
mechanics HIGH
facts LOW 9 contradicted CircleCI-feature claims and 3 uncited Pulumi-integration claims need author resolution before this lands.
Investigation log
  • Cross-sibling reads: not run (not in a templated section)
  • External claim verification: 25 of 46 claims verified (8 unverifiable, 9 contradicted) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 30 Pass 1, 0 Pass 2, 16 Pass 3 (verified 6, contradicted 8, unverifiable 2).
  • Cited-claim spot-checks: not run (no cited claims)
  • Frontmatter sweep: ran on body + meta_desc
  • Temporal-trigger sweep: ran (recency words present in diff; spot-check in-review)
  • Code execution: not run (no static/programs/ change)
  • Code-examples checks: not run (no fenced code blocks in content files)
  • Editorial-balance pass: not run (not under content/blog/)
🚨 Outstanding ⚠️ Low-confidence 💡 Pre-existing ✅ Resolved
9 11 0 0

🔍 Verification trail

46 claims extracted · 25 verified · 8 unverifiable · 9 contradicted
  • L3 in content/what-is/what-is-a-circleci-secret.md "meta_desc: 'CircleCI secrets are encrypted environment variables for CI/CD jobs. Learn project env vars, contexts, restricted contexts, OIDC, and rotation best…" → ➖ not-a-claim (evidence: This is a meta description tag in the PR author's own documentation file — it is a descriptive summary written by the author for SEO purposes, not a falsifiable factual assertion attributed to a third party.; source: content/what-is/what-is-a-circleci-secret.md L3)
  • L10 in content/what-is/what-is-a-circleci-secret.md "A CircleCI secret is an encrypted environment variable that CircleCI injects into a job's execution environment as a regular env var, scoped either to a single…" → ➖ not-a-claim (evidence: The text at L10 is the PR author's own opening definition sentence in the document being authored — it is a faithful description of CircleCI's behavior written by the PR author as the article's thesis statement, not a third-party-attribute…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L10 in content/what-is/what-is-a-circleci-secret.md "CircleCI secrets cover credentials a pipeline needs at runtime without putting them in .circleci/config.yml." → ✅ verified (evidence: The file at line 10 states: "Secrets cover the credentials a pipeline needs at runtime — cloud access keys, deploy tokens, container registry credentials, and the like — without putting them in .circleci/config.yml." This directly matche…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L10 in content/what-is/what-is-a-circleci-secret.md "CircleCI secrets can be scoped either to a single project or to an organization-level context that selected projects can opt into." → ✅ verified (evidence: The file's own opening sentence states: "A CircleCI secret is an encrypted environment variable that CircleCI injects into a job's execution environment as a regular env var, scoped either to a single project or to an organization-level co…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L12 in content/what-is/what-is-a-circleci-secret.md "Pulumi ESC integrates with CircleCI's OIDC issuer so Pulumi ESC environments can broker AWS, GCP, and other cloud credentials into pipelines without storing th…" → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L12 in content/what-is/what-is-a-circleci-secret.md "Pulumi ESC integrates with CircleCI's OIDC issuer so Pulumi ESC environments can broker AWS, GCP, and other cloud credentials into pipelines without storing th…" → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L12 in content/what-is/what-is-a-circleci-secret.md "CircleCI provides three primary secret scopes: project environment variables, organization contexts, and restricted contexts." → ❌ contradicted (framing: shifted — claim asserts three distinct scopes including "restricted contexts" as a peer scope, but sources show restricted contexts are a sub-feature of organi…; evidence: CircleCI's official docs and authoritative third-party sources describe two secret scopes: project-level environment variables and organization contexts. "Restricted contexts" are a security restriction applied to contexts, not a separate…; source: https://infisical.com/blog/github-actions-vs-circleci-for-secrets-management; https://circleci.com/docs/guides/security/contexts/; https://page.circleci.com/docs/env-vars/)
  • L49 in content/what-is/what-is-a-circleci-secret.md "CircleCI restricted contexts can be locked down by branch, tag, or organization security group." → ❌ contradicted (framing: shifted — the claim presents branch, tag, and security group as equivalent parallel lock-down mechanisms for restricted contexts, but the source treats securit…; evidence: The official CircleCI contexts docs describe restricting contexts primarily via "security groups" (VCS teams), with branch/tag restrictions achievable only indirectly through expression restrictions or config policies — not as direct, equi…; source: https://circleci.com/docs/contexts/?section=pipelines)
  • L53 in content/what-is/what-is-a-circleci-secret.md "Once a CircleCI secret is written, the value can no longer be read from the UI or API — only updated or removed." → ✅ verified (framing: strengthened — the source_hint /what-is/what-is-hashicorp-vault/ is a nearby hyperlink in the same paragraph, not the origin of this claim; the claim is about…; evidence: The claim appears verbatim in the article at the referenced location: "Once a secret is written, the value can no longer be read from the UI or API — only updated or removed." This is also corroborated within the same article's limits sect…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L61-62 in content/what-is/what-is-a-circleci-secret.md "CircleCI secret values are discarded with the ephemeral executor after the job ends." → ✅ verified (evidence: The file's protection table explicitly states: "After job ends | Discarded with the ephemeral executor", which directly matches the claim that CircleCI secret values are discarded with the ephemeral executor after the job ends.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L68 in content/what-is/what-is-a-circleci-secret.md "Inside a .circleci/config.yml, project env vars are available automatically to jobs; context-scoped secrets require the workflow to opt into the context." → ✅ verified (evidence: (escalated from pass1) CircleCI's official support documentation states: "Contexts must be explicitly attached to workflows for their environment variables to be available," and "If the context is missing from the workflow, environment var…; source: https://support.circleci.com/hc/en-us/articles/360057057571-Why-Are-Environment-Variables-from-My-Context-Not-Found)
  • L68 in content/what-is/what-is-a-circleci-secret.md "Project env vars in CircleCI are available automatically to jobs; context-scoped secrets require the workflow to opt into the context." → ✅ verified (evidence: The file itself at the "How do jobs access secrets?" section states verbatim: "Project env vars are available automatically; context-scoped secrets require the workflow to opt into the context." The table earlier in the same file corrobora…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L76 in content/what-is/what-is-a-circleci-secret.md "The code example uses cimg/base:stable as the Docker image for the deploy job." → ✅ verified (evidence: The code example in the file at the relevant section shows - image: cimg/base:stable as the Docker image for the deploy job, exactly matching the claim.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L84 in content/what-is/what-is-a-circleci-secret.md "The code example sets AWS_DEFAULT_REGION to us-west-2 in the deploy job environment." → ✅ verified (evidence: The code example in the file shows the deploy job with an environment block that sets AWS_DEFAULT_REGION: us-west-2 directly under the run step's environment key: environment:\n AWS_DEFAULT_REGION: us-west-2.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L99 in content/what-is/what-is-a-circleci-secret.md "CircleCI masks known secret values from job output, but transformations such as echo $TOKEN | base64 or set -x can leak values past the masker." → ✅ verified (evidence: (escalated from pass1) CircleCI's official docs and blog confirm masking only covers known secret values in job output. Multiple authoritative sources confirm transformations bypass the masker: TruffleHog's research shows "CI/CD jobs that…; source: https://trufflesecurity.com/blog/secrets-leak-in-ci-cd (transformation bypass); https://circleci.canny.io/cloud-feature-requests/p/ability-to-specify-which-environment-variables-are-masked-by-secret-masking (set -x); https://circleci.com/blog/keep-environment-variables-private-with-secret-masking/ (masking scope))
  • L100 in content/what-is/what-is-a-circleci-secret.md "A project not listed in a CircleCI context cannot see its env vars, even if jobs in that project reference the names." → ✅ verified (evidence: The file at line ~100 states: "A project not listed in a context can't see its env vars, even if jobs in that project reference the names." This is an exact match to the claim.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L104 in content/what-is/what-is-a-circleci-secret.md "CircleCI issues an OpenID Connect token for every job, signed by CircleCI and verifiable by any cloud provider that accepts OIDC." → ❌ contradicted (framing: shifted — claim says "for every job" but source requires jobs to use at least one context or be explicitly configured; the token is not issued automatically fo…; evidence: The claim says CircleCI issues an OIDC token "for every job," but the official CircleCI docs state: "A job can be configured to use these tokens to access compatible cloud services" and multiple sources confirm the token is only available…; source: https://circleci.com/docs/openid-connect-tokens/)
  • L108 in content/what-is/what-is-a-circleci-secret.md "Stored CircleCI secrets have a credential lifetime that lasts until manually rotated, often months." → ✅ verified (evidence: The file's comparison table at the relevant line explicitly states for stored secrets: "Credential lifetime: Until you rotate it (often months)" and "Rotation: Manual" — directly matching the claim.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L109 in content/what-is/what-is-a-circleci-secret.md "OIDC + cloud role credentials have a lifetime of minutes, controlled by the provider, and rotate automatically per run." → ✅ verified (evidence: The file's comparison table at the relevant section explicitly states: "OIDC + cloud role | Minutes (provider-controlled) | Automatic per run | Only the specific job's identity" — matching the claim exactly.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L111 in content/what-is/what-is-a-circleci-secret.md "Stored secrets remain useful for SaaS APIs that don't accept OIDC." → ➖ not-a-claim (evidence: The text "Stored secrets remain useful for SaaS APIs that don't accept OIDC." appears verbatim in the file at the end of the OIDC comparison section. This is a general editorial statement authored by the PR author themselves describing a w…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L111 in content/what-is/what-is-a-circleci-secret.md "For AWS, Azure, GCP, HashiCorp Vault, and Pulumi ESC, OIDC is the recommended path for CircleCI credential management." → ✅ verified (evidence: The file at L111 states verbatim: "For AWS, Azure, GCP, HashiCorp Vault, and Pulumi ESC, OIDC is the recommended path." This exactly matches the claim, including all five named provider…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L115 in content/what-is/what-is-a-circleci-secret.md "CircleCI does not natively expire or refresh stored secret values; users must update them out of band or use OIDC." → ✅ verified (evidence: The file at L115 (in the limits section) states: "No native rotation. CircleCI doesn't expire or refresh values for you; you have to update them out of band (or use OIDC to avoid storing them at all)." This is an exact match to the cla…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L116 in content/what-is/what-is-a-circleci-secret.md "Updating a CircleCI secret overwrites the previous value with no first-class versioning; the only history is in the team's records." → ✅ verified (evidence: The file at the "limits" section states verbatim: "No first-class versioning. Updating a secret overwrites the previous value; the only history is in your team's records." This exactly matches the claim.; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L117 in content/what-is/what-is-a-circleci-secret.md "A stored CircleCI secret value can be updated or removed but not read back through the UI or API after creation." → ✅ verified (evidence: The file at L117 explicitly states: "A stored value can be updated or removed but not read back through the UI or API. That's a feature for security but can be frustrating during debugging." This is also corroborated earlier in the same fi…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L118 in content/what-is/what-is-a-circleci-secret.md "CircleCI's audit log captures create/update/delete events on secrets and contexts, but not per-job reads." → ❌ contradicted (framing: shifted — claim states "not per-job reads" but CircleCI has logged context.secrets.accessed (a per-job context read event) since November 2023; the claim is ac…; evidence: (escalated from pass1) CircleCI does log a context.secrets.accessed event when a context's secrets are used during job execution, as confirmed by the CircleCI changelog and support docs: "Review the Action field of context.secrets.access…; source: https://circleci.com/changelog/audit-log-includes-context-accessed/ and https://support.circleci.com/hc/en-us/articles/31307485547419)
  • L119 in content/what-is/what-is-a-circleci-secret.md "CircleCI context-level access is binary: a project either opts into a context or it doesn't; there's no sub-context filtering inside a project beyond restricte…" → ❌ contradicted (framing: narrowed — claim asserts context-level access is binary (project opts in or not, with only branch/tag rules as the sole exception), but CircleCI's Expression R…; evidence: (escalated from pass1) CircleCI's official docs and changelog show that context access is NOT binary at the project level. Beyond project restrictions, CircleCI supports "Expression Restrictions" that allow fine-grained, sub-project filter…; source: https://circleci.com/changelog/expression-based-context-restrictions/ and https://circleci.com/docs/contexts/?section=pipelines)
  • L124 in content/what-is/what-is-a-circleci-secret.md "* Prefer OIDC to stored secrets. For any cloud that supports it (AWS, Azure, GCP, HashiCorp Vault, [Pulumi ESC](/produ…" → ✅ verified (evidence: The file content/what-is/what-is-hashicorp-vault.md exists in the repository with the matching page title "What is HashiCorp Vault?", confirming the internal link /what-is/what-is-hashicorp-vault/ is valid and the anchor text is accura…; source: repo:content/what-is/what-is-hashicorp-vault.md)
  • L128 in content/what-is/what-is-a-circleci-secret.md "* Rotate stored secrets on a schedule. Document a cadence and use the CircleCI API or a small script to push…" → ✅ verified (evidence: The URL https://circleci.com/docs/api/v2/index.html resolves successfully and is the official CircleCI API v2 reference documentation, as confirmed by the page content: "This describes the resources that make up the CircleCI API v2."; source: https://circleci.com/docs/api/v2/index.html)
  • L129 in content/what-is/what-is-a-circleci-secret.md "* Pair secrets with policy. CircleCI runtime policies and config validation can block jobs that try to consu…" → ❌ contradicted (framing: shifted — claim links to "CircleCI runtime policies" at /docs/runtime-policies/, but the actual CircleCI feature is "config policies" at /docs/guides/config-po…; evidence: CircleCI's feature for blocking jobs that consume unauthorized contexts is called "config policies," documented at circleci.com/docs/guides/config-policies/ and circleci.com/docs/manage-contexts-with-config-policies/. No CircleCI documenta…; source: WebSearch ran query "circleci.com/docs/runtime-policies"; top results show the feature is "config policies" at circleci.com/docs/guides/config-policies/ and circleci.com/docs/manage-contexts-with-config-policies/)
  • L133 in content/what-is/what-is-a-circleci-secret.md "The recommended source for PULUMI_ACCESS_TOKEN in CircleCI is OIDC into Pulumi ESC, with ESC then brokering AWS, Azure, GCP, or other credentials into the sa…" → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L139 in content/what-is/what-is-a-circleci-secret.md "The Pulumi CircleCI Orb is versioned at pulumi/pulumi@2 (major version 2)." → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L144 in content/what-is/what-is-a-circleci-secret.md "The code example uses cimg/node:lts as the Docker image for the preview job." → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L161 in content/what-is/what-is-a-circleci-secret.md "With the Pulumi ESC + CircleCI OIDC pattern, cloud credentials never live in CircleCI as long-lived secrets — ESC issues per-run AWS/Azure/GCP credentials base…" → ➖ not-a-claim (evidence: The claim is a faithful description of the PR author's own design for the Pulumi ESC + CircleCI OIDC integration pattern, as described in the same document: "Pulumi ESC integrates with CircleCI's OIDC issuer so Pulumi ESC environments can…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L162 in content/what-is/what-is-a-circleci-secret.md "PULUMI_ACCESS_TOKEN itself can be issued from the CircleCI OIDC token via pulumi/auth-actions-equivalent flows, eliminating the only static secret the pipe…" → ❌ contradicted (framing: shifted — pulumi/auth-actions exchanges GitHub OIDC tokens, not CircleCI OIDC tokens; the claim attributes a CircleCI OIDC→PULUMI_ACCESS_TOKEN flow to this G…; evidence: pulumi/auth-actions is a GitHub Actions action (topics: ["github-actions"], description: "automatically generates and exchanges GitHub's OpenID Connect tokens by Pulumi Access Tokens") that works exclusively with GitHub's OIDC issuer —…; source: gh api repos/pulumi/auth-actions)
  • L165 in content/what-is/what-is-a-circleci-secret.md "Pulumi supports IaC pipelines in TypeScript, Python, Go, C#, Java, or YAML on CircleCI." → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L171 in content/what-is/what-is-a-circleci-secret.md "CircleCI secret values are stored encrypted at rest, transmitted over TLS, and injected into a job's execution environment only when the job runs." → ✅ verified (evidence: (escalated from pass1) CircleCI's official blog states "Our efforts to protect your secrets include encryption at rest and in transit," and the security docs confirm all networking uses HTTPS/TLS. CircleCI support docs confirm "Project env…; source: https://circleci.com/blog/securing-ci-cd-pipelines-with-circleci-contexts-rest-api/ ; https://circleci.com/docs/security/ ; https://support.circleci.com/hc/en-us/articles/31307485547419)
  • L171 in content/what-is/what-is-a-circleci-secret.md "Once a CircleCI secret is written, the value can no longer be read through the UI or API; only updated or removed." → ✅ verified (framing: strengthened — the claim at L171 is a narrower paraphrase of the same statement appearing verbatim in the document's storage section; both the claim and the so…; evidence: The file at content/what-is/what-is-a-circleci-secret.md states in the storage section: "Once a secret is written, the value can no longer be read from the UI or API — only updated or removed." This is also corroborated in the limits secti…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L179 in content/what-is/what-is-a-circleci-secret.md "A restricted context layers branch, tag, or security-group rules on top of a regular context, and only jobs that match the rules can read the context's secrets." → ✅ verified (framing: strengthened — claim simplifies "expression restrictions" (which evaluate branch/tag pipeline metadata) and "security groups" into a unified "branch, tag, or s…; evidence: CircleCI docs confirm restricted contexts support security groups, project restrictions, and expression restrictions (branch/tag conditions): "To invoke a job that uses a restricted context, a user must be a member of one of the security g…; source: https://circleci.com/docs/contexts/?section=pipelines)
  • L183 in content/what-is/what-is-a-circleci-secret.md "When a CircleCI secret is rotated, old jobs in flight finish with the previous value; new jobs pick up the new value." → 🤷 unverifiable (evidence: (escalated from pass1) Sources confirm that CircleCI injects secrets at job start time and "each job starts fresh, reads from the current context," so new jobs pick up the new value after rotation. However, no authoritative CircleCI source…; source: WebSearch ran query "CircleCI secret rotation in-flight jobs behavior"; top results (infisical.com, circleci.com/blog) confirm new-job behavior but don't address in-flight jobs explicitly.; intuition: The "old jobs in flight finish with the previous value" assertion is a plausible inference from CircleCI's inject-at-st…)
  • L195 in content/what-is/what-is-a-circleci-secret.md "OIDC issues short-lived credentials per job run, eliminates long-lived stored values, and dramatically reduces rotation overhead for cloud providers that suppo…" → ✅ verified (framing: strengthened — claim adds "eliminates long-lived stored values, and dramatically reduces rotation overhead" as elaboration; source's table and text confirm the…; evidence: The file at the OIDC section states: "For AWS, Azure, GCP, HashiCorp Vault, and Pulumi ESC, OIDC is the recommended path. The job presents the OIDC token, the provider validates the iss…; source: repo:content/what-is/what-is-a-circleci-secret.md)
  • L199 in content/what-is/what-is-a-circleci-secret.md "CircleCI masks known secret values from job output by default, replacing matches with asterisks, but the masker does not work for transformations such as `echo…" → ✅ verified (framing: strengthened — claim says "asterisks" but CircleCI uses XXXXX or **** as the masking character; this is a colloquial description, not a material factual error.…; evidence: (escalated from pass1) CircleCI's official blog confirms secret masking scans logs for matches and substitutes them (with XXXXX/****), and explicitly notes it won't stop transformations. Nathan Davison's research confirms "you can't just r…; source: https://circleci.com/blog/keep-environment-variables-private-with-secret-masking/ ; https://nathandavison.com/blog/shaking-secrets-out-of-circleci-builds ; https://trufflesecurity.com/blog/secrets-leak-in-ci-cd)
  • L203 in content/what-is/what-is-a-circleci-secret.md "There is no per-job ACL within a regular CircleCI context; if a project opts into a context, every job in that project's workflow can declare it." → ❌ contradicted (framing: shifted — the claim says "no per-job ACL" but CircleCI context access is inherently per-job (each job must declare the context), and per-job restrictions (expr…; evidence: (escalated from pass1) CircleCI docs state "you can use the context key in the workflows section of a project's configuration file to give any job(s) access" — access is granted per-job by explicitly declaring the context on each job, not…; source: https://circleci.com/docs/guides/security/contexts/ and https://support.circleci.com/hc/en-us/articles/32887072348955-Restricting-access-to-contexts)
  • L207 in content/what-is/what-is-a-circleci-secret.md "CircleCI's audit log records secret-management events (create, update, delete, context membership changes) but does not record per-job reads of a stored value." → ❌ contradicted (framing: shifted — claim says CircleCI "does not record per-job reads of a stored value," but context.secrets.accessed is logged per job execution when context secret…; evidence: (escalated from pass1) CircleCI's audit log does include a context.secrets.accessed event that fires when a context's secrets are used during workflow/job execution (added Nov 2023). The official circleci-docs source lists this event alo…; source: https://github.com/circleci/circleci-docs/blob/master/jekyll/_cci2/audit-logs.adoc; https://circleci.com/changelog/audit-log-includes-context-accessed/)
  • L211 in content/what-is/what-is-a-circleci-secret.md "Pulumi pairs with CircleCI through the Pulumi Orb and Pulumi ESC: cloud credentials are minted per run via OIDC, ESC brokers access, and the orb runs `pulumi u…" → 🤷 unverifiable (evidence: verification did not converge within 8 turns)
  • L215 in content/what-is/what-is-a-circleci-secret.md "The page cross-references /what-is/what-is-a-github-action-secret/ as an existing page." → ✅ verified (evidence: The file content/what-is/what-is-a-github-action-secret.md exists in the repository with the title "What is a GitHub Actions Secret?", confirming the cross-referenced page /what-is/what-is-a-github-action-secret/ is a real, existing pa…; source: repo:content/what-is/what-is-a-github-action-secret.md)
  • L215-220 in content/what-is/what-is-a-circleci-secret.md "The following pages exist on the Pulumi site: /what-is/what-is-secrets-management/, /what-is/what-is-a-github-action-secret/, /what-is/what-is-a-cloudflare-sec…" → ✅ verified (evidence: All six content files exist in the repo: content/what-is/what-is-secrets-management.md, what-is-a-github-action-secret.md, what-is-a-cloudflare-secret.md, what-is-hashicorp-vault.md, what-is-aws-secrets-manager.md, and `what-is-c…; source: repo:content/what-is/what-is-secrets-management.md, repo:content/what-is/what-is-a-github-action-secret.md, repo:content/what-is/what-is-a-cloudflare-secret.md, repo:content/what-is/what-is-hashicorp-vault.md, repo:content/what-is/what-is-aws-secrets-manager.md, repo:content/what-is/what-is-ci-cd.md)

🚨 Outstanding in this PR

These must be resolved or refuted before merging.

  • [L12] content/what-is/what-is-a-circleci-secret.md — current: "CircleCI provides three primary scopes (project environment variables, organization contexts, and restricted contexts)…" CircleCI's own docs frame the model as two scopes — project env vars and contexts — with restricted contexts being a security-restriction layer (security groups + expression restrictions) applied to contexts, not a third peer scope. The page itself is internally inconsistent: L49 already says restricted contexts "can be further locked down by branch, tag, or organization security group" (i.e. on top of contexts), and the scope-table at L43–47 lists restricted-context as a row even though the prose treats it as a sub-feature. Suggest aligning the framing throughout:

    CircleCI provides two primary scopes — project environment variables and organization contexts — and adds restricted-context rules (security groups, plus branch/tag expression restrictions) on top of contexts for higher-sensitivity values. It encrypts values at rest and masks them in job output by default.
    

    The "What are the CircleCI secret scopes…" H2 (L39) and its table (L43–47) should be reworked the same way — two scopes + a restricted-context modifier — so the article doesn't pull in opposite directions.

  • [L49] content/what-is/what-is-a-circleci-secret.md — current: "…restricted contexts can be further locked down by branch, tag, or organization security group." CircleCI restricts contexts primarily via security groups (VCS team membership); branch/tag conditions are a separate mechanism called expression restrictions (and historically were achievable only indirectly through config policies). Listing all three as equivalent peer controls misrepresents how the two features compose. Suggest:

    [Contexts](https://circleci.com/docs/contexts/) are the more powerful tier: secrets live at the organization level, projects opt in by listing the context in their workflow, and **restricted contexts** add security-group rules — and, via [expression restrictions](https://circleci.com/docs/contexts/?section=expression-restrictions), branch- or tag-based gating — on top of a regular context. The standard pattern: project env vars for repo-specific tokens, contexts for shared services, restricted contexts for production-only credentials.
    
  • [L104] content/what-is/what-is-a-circleci-secret.md — current: "CircleCI issues an OpenID Connect token for every job, signed by CircleCI and verifiable by any cloud provider that accepts OIDC." CircleCI's OIDC docs scope this more narrowly — the OIDC token (CIRCLE_OIDC_TOKEN, and now CIRCLE_OIDC_TOKEN_V2) is exposed to jobs that use at least one context. "For every job" overstates the default. Suggest:

    CircleCI jobs that use at least one context receive a CircleCI-signed [OpenID Connect token](https://circleci.com/docs/openid-connect-tokens/) (`CIRCLE_OIDC_TOKEN_V2`), which any cloud provider that accepts OIDC can verify. The job can exchange that token for short-lived cloud credentials without ever holding a long-lived access key.
    
  • [L118] content/what-is/what-is-a-circleci-secret.md — current: "Limited audit detail. The audit log captures create/update/delete events on secrets and contexts, but not per-job reads." CircleCI added a context.secrets.accessed event in November 2023 that fires when a context's secrets are read during job execution. The article's blanket "not per-job reads" claim is wrong for context secrets (it's still correct for project env vars). Suggest:

    * **Limited audit detail.** The audit log captures create/update/delete events on secrets and contexts, and a [`context.secrets.accessed` event](https://circleci.com/changelog/audit-log-includes-context-accessed/) when a job reads a context's secrets; per-job reads of project env vars aren't logged.
    

    The same fix needs to be mirrored in the FAQ at L207 — see that bullet below.

  • [L119] content/what-is/what-is-a-circleci-secret.md — current: "Context-level access is binary. A project either opts into a context or it doesn't; there's no sub-context filtering inside a project beyond restricted-context branch/tag rules." CircleCI's Expression Restrictions (rolled out in 2023) allow finer-grained gating on regular contexts — branch, tag, pipeline-trigger source, etc. — without requiring a restricted context. Suggest rewording so "binary" doesn't read as a hard limit:

    * **Project-level opt-in.** A project opts into a context as a whole; finer-grained gating relies on [context expression restrictions](https://circleci.com/docs/contexts/?section=expression-restrictions) or restricted-context security groups.
    

    The FAQ at L203 makes the same claim and needs the same fix — see that bullet below.

  • [L129] content/what-is/what-is-a-circleci-secret.md — current: " Pair secrets with policy. CircleCI runtime policies and config validation can block jobs that try to consume contexts they shouldn't."* Two problems in one bullet: (a) the CircleCI feature is called config policies, not "runtime policies", and (b) the link https://circleci.com/docs/runtime-policies/ returns 404 — confirmed via direct curl during this review. The closest authoritative page is https://circleci.com/docs/guides/config-policies/manage-contexts-with-config-policies/ (200). Suggest:

    * **Pair secrets with policy.** [CircleCI config policies](https://circleci.com/docs/guides/config-policies/manage-contexts-with-config-policies/) and config validation can block jobs that try to consume contexts they shouldn't.
    
  • [L162] content/what-is/what-is-a-circleci-secret.md — current: "PULUMI_ACCESS_TOKEN itself can be issued from the OIDC token (via pulumi/auth-actions-equivalent flows), eliminating the only static secret the pipeline used to need." pulumi/auth-actions is a GitHub Actions action — its description on pulumi/auth-actions is "automatically generates and exchanges GitHub's OpenID Connect tokens by Pulumi Access Tokens" and it works against GitHub's OIDC issuer, not CircleCI's. Citing it (even as "-equivalent") on a CircleCI page implies there's an analogous Pulumi-shipped CircleCI integration, but there isn't one today — the equivalent on CircleCI is a manual exchange against Pulumi Cloud's STS endpoint, or going through Pulumi ESC. Suggest replacing the parenthetical with what the team would actually do on CircleCI today:

    * `PULUMI_ACCESS_TOKEN` itself can be issued from the CircleCI OIDC token by exchanging it against Pulumi Cloud's STS endpoint (or by letting Pulumi ESC broker it), eliminating the only static secret the pipeline used to need.
    

    If there's a recommended snippet for the CircleCI exchange flow, link to it; if not, this is the right place to flag that as a follow-up.

  • [L203] content/what-is/what-is-a-circleci-secret.md — current: "Indirectly, via restricted contexts with branch, tag, or security-group rules. There isn't a per-job ACL within a regular context; if a project opts into a context, every job in that project's workflow can declare it." (FAQ "Can I limit which jobs in a project can read a context?") This repeats the L119 issue in the FAQ — Expression Restrictions on a regular context can gate access by branch/tag/pipeline-trigger without requiring a restricted context. Suggest:

    Yes. Use [context expression restrictions](https://circleci.com/docs/contexts/?section=expression-restrictions) for branch-, tag-, or pipeline-trigger-based gating on a regular context, or escalate to a restricted context for security-group rules. Inside a project, jobs still have to explicitly declare each context they want to consume.
    

    Keep this fix synchronized with the L119 limits bullet.

  • [L207] content/what-is/what-is-a-circleci-secret.md — current: "CircleCI's audit log records secret-management events (create, update, delete, context membership changes). It does not record per-job reads of a stored value." (FAQ "How do I audit who used a CircleCI secret?") Same factual error as L118 — context.secrets.accessed does log per-job context reads. Suggest aligning with the L118 fix:

    CircleCI's audit log records secret-management events (create, update, delete, context membership changes) and emits a [`context.secrets.accessed` event](https://circleci.com/changelog/audit-log-includes-context-accessed/) when a job reads a context's secrets; per-job reads of project env vars aren't logged. For tighter audit, use OIDC and rely on the cloud provider's audit log (CloudTrail, etc.) to see what the issued credential did.
    

⚠️ Low-confidence

Review each and resolve as appropriate — these don't block the PR.

  • [L12] content/what-is/what-is-a-circleci-secret.md — current: "Pulumi ESC integrates with CircleCI's OIDC issuer so Pulumi ESC environments can broker AWS, GCP, and other cloud credentials into pipelines without storing them on either side." The verifier couldn't converge on a citation. Pulumi ESC does support OIDC providers in general, but this article doesn't link to a Pulumi page that documents the CircleCI → ESC OIDC flow specifically (the /product/esc/ link points at the product landing page, not setup docs). Author Q: can you cite the Pulumi docs page that walks through configuring Pulumi ESC with CircleCI's OIDC issuer? Same question applies to the closely related claim at L133 and the closing paragraph at L211.

  • [L133] content/what-is/what-is-a-circleci-secret.md — current: "…the recommended source is OIDC into Pulumi ESC, with ESC then brokering AWS, Azure, GCP, or other credentials into the same run." This is the article's central recommendation for the Pulumi-on-CircleCI pattern, but it's uncited. Author Q: is there a Pulumi-side doc (registry orb page, ESC + CircleCI guide, etc.) that endorses this as the recommended path? A link here would let a reader actually wire it up; today the next paragraph jumps straight to a YAML snippet without pointing to setup docs. The same recommendation appears at L161 ("Cloud credentials never live in CircleCI as long-lived secrets…") and L211 ("Pulumi pairs with CircleCI through the Pulumi Orb…"); whatever link you add should resolve all three.

  • [L183] content/what-is/what-is-a-circleci-secret.md — current: "Old jobs in flight finish with the previous value; new jobs pick up the new value." (FAQ "How do I rotate a CircleCI secret?") The verifier found the new-job behavior in CircleCI's blog/support docs but no authoritative source for the in-flight-job half. The intuition is plausible (secrets are injected at job-start, so a running job holds the value it started with) — but stated as fact without a citation, it's a small reader-trap if CircleCI rotates secrets through some other mechanism. Author Q: can you confirm the in-flight behavior from CircleCI docs, or soften to "Jobs already running keep using the value they started with" with a hedge? Not a merge blocker, but worth firming up.

Style findings

Found by pattern-based linting; Findings may be false positives.

  • line 37: [style] weasel word — 'usually' is a weasel word!
  • line 181: [style] first person — Avoid first-person pronouns such as ' I '.
  • line 185: [style] first person — Avoid first-person pronouns such as ' I '.
  • line 189: [style] first person — Avoid first-person pronouns such as ' I '.
  • line 191: [style] wordiness — 'it is' is too wordy.
  • line 193: [style] first person — Avoid first-person pronouns such as ' I '.
  • line 201: [style] first person — Avoid first-person pronouns such as ' I '.
  • line 205: [style] first person — Avoid first-person pronouns such as ' I '.

📋 Triaged verifier findings

I double-checked these and realized they weren't real findings — click to expand
  • [L12] content/what-is/what-is-a-circleci-secret.md"Pulumi ESC integrates with CircleCI's OIDC issuer so Pulumi ESC environments can broker AWS, GCP, and other cloud credentials into pipelines without storing them on either side."Mis-sourced: duplicate trail entry for the same sentence the verifier emitted twice; the live ⚠️ author question above already covers it.

  • [L139] content/what-is/what-is-a-circleci-secret.md"The Pulumi CircleCI Orb is versioned at pulumi/pulumi@2 (major version 2)."Mis-sourced: this is a repo-internal description of the YAML snippet's orbs: block (line 139 of the file literally reads pulumi: pulumi/pulumi@2) — the verifier should have read the file rather than the SPA-rendered Orb registry. The major-version pin is faithful to the code example as written. (Author should still spot-check that @2 is the current major when shipping.)

  • [L144] content/what-is/what-is-a-circleci-secret.md"The code example uses cimg/node:lts as the Docker image for the preview job."Mis-sourced: trivially verifiable by reading the file at L144 (- image: cimg/node:lts); the verifier looked outward when it should have looked at the diff itself.

  • [L165] content/what-is/what-is-a-circleci-secret.md"Pulumi supports IaC pipelines in TypeScript, Python, Go, C#, Java, or YAML on CircleCI."Mis-sourced: Pulumi's supported language list is documented on the language-SDKs page (/docs/iac/languages-sdks/) and on the homepage; this is a well-known, repo-confirmable capability rather than an unverifiable external claim.

  • [L211] content/what-is/what-is-a-circleci-secret.md"Pulumi pairs with CircleCI through the Pulumi Orb and Pulumi ESC: cloud credentials are minted per run via OIDC, ESC brokers access, and the orb runs pulumi up or pulumi preview against your stacks."Mis-sourced: the "Learn more" closing paragraph is a summary of claims the article itself makes (and that are individually verified in the trail above for the Orb + pulumi up/pulumi preview pieces). The uncited "ESC brokers access" half is already covered by the L12 / L133 author question.

💡 Pre-existing issues in touched files (optional)

No pre-existing issues in touched files.

✅ Resolved since last review

No items resolved since the last review.

📜 Review history

  • 2026-05-20T16:53:57Z — Initial review: 9 contradicted CircleCI-feature claims (audit-log context.secrets.accessed, Expression Restrictions, "config policies" mis-named as "runtime policies" with a 404 link, OIDC "for every job", pulumi/auth-actions mis-attributed to CircleCI) plus 3 uncited Pulumi-ESC+CircleCI integration claims to firm up. (946848e)

Need a re-review? Want to dispute a finding? Mention @claude and include #update-review.
(For ad-hoc questions or fixes, just @claude — no hashtag.)

@github-actions github-actions Bot added review:outstanding-issues Claude review completed; outstanding has author-actionable findings and removed review:in-progress Claude review is currently running labels May 20, 2026
@alexleventer alexleventer marked this pull request as draft May 20, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain:docs PR touches technical docs review:outstanding-issues Claude review completed; outstanding has author-actionable findings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants