fix: repair post-release dev-preview NUTs (W-23400737) by jhefferman-sfdc · Pull Request #676 · salesforcecli/plugin-lightning-dev

jhefferman-sfdc · 2026-07-22T17:48:35Z

What & Why

The dev-preview e2e NUTs added in #626 (test/commands/lightning/dev/component-preview/*.nut.ts) pass in this repo's own CI but fail in the post-CLI-release integration pipeline (which runs them against a released, bundled CLI). This PR fixes them. Tracked by @W-23400737@.

Three layered failures were diagnosed, each masking the next:

1. API-version drift (the real, deterministic bug)

Scratch orgs are provisioned on the platform's current default API version (e.g. 68.0), but the plugin only bundles LWC dev-server dists for the versions in package.json apiVersionMetadata (66.0/67.0). dependencyLoader.ts does an exact-match check and throws, so the dev server exits before printing a preview URL, so every browser test hits the 30s getPreviewURL timeout.

Fix: added getMaxSupportedApiVersion() (reads package.json apiVersionMetadata) and spawn the dev server with --api-version <max>. This decouples the e2e tests from platform version drift and auto-tracks new versions as they're added to package.json. (Adding real 68.0 support to the shipping plugin is a separate product task.)

2. Sinon cwd cascade (masking bug)

TestSession's constructor stubs process.cwd before creating scratch orgs. If org creation throws, create() rejects without returning, the stub leaks, and every subsequent file dies with a misleading Attempted to wrap cwd which is already wrapped — hiding the true error.

Fix: restore the leaked stub on setup failure before rethrowing, so the real failure surfaces in the file that actually failed. Also added retries (default 3, overridable via TESTKIT_SETUP_RETRIES) for intermittent scratch-org signup (RemoteOrgSignupFailed / C-9999).

3. Secret redaction, frontdoor 401

sf org display user --json now redacts accessToken by default (returns a [REDACTED] ... placeholder), so frontdoor SSO fails with HTTP 401 and the component never renders.

Fix: read the token directly from the org connection via @salesforce/core (Org.create().getConnection().accessToken) instead of shelling out.

Testing

Full NUT suite verified passing locally against a healthy DevHub, and green in CI on both ubuntu-latest and windows-latest:

lightning preview menu               ✔ ✔
lightning preview component error    ✔ ✔ ✔
lightning preview hot module reload  ✔ ✔ ✔
lightning preview component prompts  ✔ ✔ ✔ ✔

12 passing (2m)

Scope: test-helpers only (test/commands/lightning/dev/helpers/) — no changes to shipping plugin code.

🤖 Generated with Claude Code

The dev-preview e2e NUTs added in #626 fail in the post-CLI-release integration pipeline due to three layered issues: 1. API-version drift: scratch orgs are provisioned on the platform's current default API version (e.g. 68.0), but the plugin only bundles dev-server dists for the versions in package.json apiVersionMetadata (66.0/67.0). The dev server rejects the unsupported version and exits before printing a preview URL, so every browser test hits the 30s getPreviewURL timeout. Pin the spawned dev server to the plugin's max supported API version (getMaxSupportedApiVersion) so the tests are decoupled from platform version drift. 2. Sinon cwd cascade: TestSession stubs process.cwd before creating orgs; if org creation throws, the stub leaks and every later file dies with a misleading "Attempted to wrap cwd which is already wrapped", masking the real error. Restore the leaked stub on setup failure, and add retries (TESTKIT_SETUP_RETRIES, default 3) for intermittent scratch-org signup. 3. Secret redaction: `sf org display user --json` now redacts accessToken by default, breaking frontdoor SSO with a 401. Read the token directly from the org connection via @salesforce/core instead. Full NUT suite (12 tests) verified passing locally against a healthy DevHub. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rax-it · 2026-07-22T20:45:51Z

+  const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf8')) as { apiVersionMetadata?: Record<string, unknown> };
+  const versions = Object.keys(pkg.apiVersionMetadata ?? {});
+  if (versions.length === 0) {
+    throw new Error(`No apiVersionMetadata found in ${pkgPath}`);


nit: should we try a default version?

Intentional fail-fast here rather than a default. apiVersionMetadata is a hard requirement for the plugin — the shipping dependencyLoader.ts reads packageJson.apiVersionMetadata directly and would break too if it were ever empty, so an empty map means the plugin is misconfigured, not in a transient state a test should paper over.

There's also no existing "default API version" constant anywhere in the source to fall back to, so a default would have to be a hardcoded magic number (e.g. '67.0'). That would silently rot as versions move, and could pin the tests to a version with no matching @lwc/sfdx-local-dev-dist-<v> dist — producing a more confusing downstream import failure than the clear "No apiVersionMetadata found" we throw at the source.

Happy to revisit if we ever introduce a canonical default version the plugin resolves against.

wjhsf

What happens when the platform gets a new default API version, but this package doesn't get updated?

wjhsf · 2026-07-23T15:30:07Z

 * @returns The session ID string.
 */
-export function getAccessToken(session: TestSession): string {
+export async function getAccessToken(session: TestSession): Promise<string> {


Suggested change

export async function getAccessToken(session: TestSession): Promise<string> {

async function getAccessToken(session: TestSession): Promise<string> {

not used outside this file, doesn't need to be exported

Good catch — addressed in follow-up #677. getAccessToken is now non-exported (it's only used by getPreview in this file).

wjhsf · 2026-07-23T15:33:51Z

+// Number of times TestSession will retry scratch org creation before failing. Scratch org
+// signup is intermittently flaky (RemoteOrgSignupFailed / C-9999); retrying avoids spurious
+// failures in the post-release pipeline. Overridable via TESTKIT_SETUP_RETRIES.
+const SETUP_RETRIES = Number.parseInt(process.env.TESTKIT_SETUP_RETRIES ?? '', 10) || 3;


Curious use of Number.parseInt instead of global parseInt.

Switched to the global parseInt in #677 for consistency — the rest of the repo (src/configMeta.ts, src/shared/previewUtils.ts, and devServerUtils.ts right next door) uses the global form, so Number.parseInt was the odd one out here. No behavioral difference either way.

jhefferman-sfdc · 2026-07-23T15:47:20Z

@wjhsf: What happens when the platform gets a new default API version, but this package doesn't get updated?

That's precisely the drift this change is built to absorb, so the answer is "the e2e tests keep passing." Before this fix they broke on exactly that scenario.

The mechanism: startLightningDevServer no longer lets the org's (newer) default API version reach the dev server. It pins the preview to getMaxSupportedApiVersion() — the highest version in the plugin's own apiVersionMetadata, i.e. the newest version this package actually ships a @lwc/sfdx-local-dev-dist-<v> for. So:

Platform bumps its default (e.g. 68.0), plugin not yet updated → tests still run against the plugin's max supported version (e.g. 67.0), which has a matching dist, and stay green. No spurious post-release failures.
A human adds 68.0 to the plugin (new dist dep + apiVersionMetadata entry) → getMaxSupportedApiVersion() automatically returns 68.0 and the tests track forward with zero test changes.

The one thing this deliberately does not do is give the plugin real support for a newer API version than it ships — that's a separate product task (adding the dist + metadata + types), not something a test should silently paper over. The tests validate what the plugin actually supports today, decoupled from platform drift. Full root-cause writeup is in the #676 description.

jhefferman-sfdc requested review from a team as code owners July 22, 2026 17:48

rax-it reviewed Jul 22, 2026

View reviewed changes

rax-it approved these changes Jul 22, 2026

View reviewed changes

jhefferman-sfdc merged commit a7e69e6 into main Jul 23, 2026
24 of 27 checks passed

jhefferman-sfdc deleted the jhefferman/W-23400737-fix-post-release-nuts branch July 23, 2026 14:39

jhefferman-sfdc mentioned this pull request Jul 23, 2026

chore: added schemas folder (W-23365673) #675

Open

wjhsf reviewed Jul 23, 2026

View reviewed changes

jhefferman-sfdc mentioned this pull request Jul 23, 2026

chore: address post-merge NUT review comments (W-23400737) #677

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: repair post-release dev-preview NUTs (W-23400737)#676

fix: repair post-release dev-preview NUTs (W-23400737)#676
jhefferman-sfdc merged 1 commit into
mainfrom
jhefferman/W-23400737-fix-post-release-nuts

jhefferman-sfdc commented Jul 22, 2026 •

edited

Loading

Uh oh!

rax-it Jul 22, 2026 •

edited

Loading

Uh oh!

jhefferman-sfdc Jul 22, 2026

Uh oh!

Uh oh!

wjhsf left a comment

Uh oh!

wjhsf Jul 23, 2026

Uh oh!

jhefferman-sfdc Jul 23, 2026

Uh oh!

wjhsf Jul 23, 2026

Uh oh!

jhefferman-sfdc Jul 23, 2026

Uh oh!

jhefferman-sfdc commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	export async function getAccessToken(session: TestSession): Promise<string> {
	async function getAccessToken(session: TestSession): Promise<string> {

Uh oh!

Conversation

jhefferman-sfdc commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & Why

1. API-version drift (the real, deterministic bug)

2. Sinon cwd cascade (masking bug)

3. Secret redaction, frontdoor 401

Testing

Uh oh!

rax-it Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jhefferman-sfdc Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wjhsf left a comment

Choose a reason for hiding this comment

Uh oh!

wjhsf Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

jhefferman-sfdc Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

wjhsf Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

jhefferman-sfdc Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

jhefferman-sfdc commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jhefferman-sfdc commented Jul 22, 2026 •

edited

Loading

rax-it Jul 22, 2026 •

edited

Loading