Skip to content

Type-check test files: every app tsconfig excludes test/ #1299

Description

@vivek7405

Problem

Test files are outside every app's tsconfig.json include, so webjs typecheck never reads them and a type error in a test is invisible until a human notices it.

website/tsconfig.json:16 includes app/**/*, components/**/*, lib/**/*, modules/**/* and nothing else, while 25 .ts files live under website/test/. npm run typecheck in website/ therefore reports success on a tree where a test file could be arbitrarily broken.

This is not hypothetical. During review of #1259 (PR #1295) a helper was written const quoted = (key) => ... in website/test/ssr/docs-links.test.ts, an implicitly-any parameter in a .ts file. Root AGENTS.md says plainly that "any has no carve-out at all", strict: true is set in that same tsconfig, and every sibling arrow in the file is annotated. Nothing caught it. It survived until a reviewer read the line by eye, and it would have merged otherwise.

The gap is systemic, not specific to website/. All four in-repo apps and the scaffold share the shape:

Config include
website/tsconfig.json:16 app, components, lib, modules
examples/blog/tsconfig.json app, components, modules, lib, middleware.*
docs/tsconfig.json app, middleware.ts
packages/ui/packages/website/tsconfig.json app, middleware.ts
packages/cli/lib/create.js:537-543 app, components, modules, lib, middleware.*

The last row is the one that matters most: every app webjs create generates ships this gap, so an app author who follows the scaffold's own testing guidance gets no type checking on the tests they write.

Design / approach

Add the test directory to the include of each app tsconfig, and to the array the scaffold generator emits.

The obvious objection is that test files import node:test and node:assert, and may use shapes the app config is strict about. So this is likely to surface a batch of pre-existing errors on first run. That is the point of filing it rather than doing it as a drive-by: the fix is one line per config, and the work is whatever tsc then reports.

Two decisions the implementer should make deliberately rather than by default:

  1. One config or two. Extending each app's include is the simplest thing and keeps one webjs typecheck command honest. A separate tsconfig.test.json extending the base is the alternative, and is worth it only if the tests genuinely need looser settings than the app. Prefer the single config until something forces the split, since a second config that nobody runs reproduces this bug in a new place.
  2. Whether checkJs matters. The app configs set allowJs: true, checkJs: false, so .js tests would be parsed but not checked even after inclusion. Browser tests under website/test/components/browser/ are .js. Decide whether those are in scope or explicitly left out, and say which in the config comment.

Implementation notes (for the implementing agent)

Where to edit:

  • website/tsconfig.json L16, the include array.
  • examples/blog/tsconfig.json, docs/tsconfig.json, packages/ui/packages/website/tsconfig.json, same field.
  • packages/cli/lib/create.js L537-543, the include array the scaffold writes. This is a generator emitting a literal, so the change is to the array, and it must be verified by generating an app rather than by reading the diff.

Landmines:

  • Expect a wall of errors on the first run, and read them before "fixing" them. These files have never been type-checked, so some errors will be real bugs (like the implicit any above) and some will be the config being wrong for test code. Do not blanket-add any or @ts-expect-error to make it green; that converts an invisible gap into a visible lie.
  • website/package.json runs node scripts/copy-registry.mjs before typecheck, because modules/ui/components/ is a gitignored mirror. Run npm run typecheck, not a bare tsc, or you will get phantom missing-module errors.
  • erasableSyntaxOnly: true is set in these configs (AGENTS.md invariant 10). Any fix must stay erasable: no enum, no value namespace, no constructor parameter properties.
  • The scaffold's include is also read by editor tooling and @webjsdev/intellisense, so widening it changes what the editor checks in a generated app. That is the desired outcome, but it means the scaffold change needs the generate-and-boot verification, not just a unit assertion.
  • website/test/fixtures/ may hold deliberately-malformed files. Check before including it wholesale; it may need an exclude entry.

Invariants to respect:

  • Root AGENTS.md: "any has no carve-out at all", and the derive-the-type rule. The whole point of this issue is to make that enforceable rather than aspirational.
  • AGENTS.md invariant 10 (erasable TypeScript only).

Tests + docs surfaces:

  • test/scaffolds/** covers generated-app shape; assert the emitted include contains the test dir.
  • Verify by generating an app and running webjs typecheck in it, per the scaffold rule in AGENTS.md (generators emit strings, so an escaping or shape bug only shows in a freshly generated app).
  • website/AGENTS.md and the per-app AGENTS.md files if they describe what typecheck covers.
  • references/testing.md and references/typescript.md in the skill, if either states what is type-checked.
  • The docs site page for testing or TypeScript, if it makes the same claim.

Acceptance criteria

  • website/tsconfig.json includes the test directory, and npm run typecheck in website/ type-checks the files under website/test/
  • The same holds for examples/blog, docs, and packages/ui/packages/website
  • packages/cli/lib/create.js emits an include covering the test directory, verified by generating an app and running webjs typecheck in it
  • Every error the change surfaces is either fixed properly or explicitly justified in the PR body; none is silenced with a blanket any or @ts-expect-error
  • A counterfactual proves it fires: an implicitly-any parameter added to a test file makes webjs typecheck fail, and it passes once annotated
  • A test/scaffolds/** assertion covers the generated include
  • Docs updated wherever the type-check surface is described

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions