test(design-system): setup Storybook docs tests [AR-74627]#587
Conversation
✅ Deploy Preview for drivenets-design-system ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
github-actions
Bot
added
ci
tooling
design-system
cursor
iromanchuk-dn
changed the title
StyleShit
left a comment
StyleShit
left a comment
There was a problem hiding this comment.
I reviewed mostly the "infra" part, but I don't really understand the tests themselves
| - name: Install Playwright Chromium | ||
| run: pnpm --filter @drivenets/design-system install:playwright |
There was a problem hiding this comment.
This is already a dependency of the test:coverage task, so there is no need to run it manually. Chromium should be installed at this point
| - name: Build Storybook for docs snippet tests | ||
| run: pnpm build:storybook --affected |
There was a problem hiding this comment.
If we're going to build storybook for every PR now, let's use turbo cache. See the build-storybook job for reference
| - name: Run Storybook docs snippet tests | ||
| run: pnpm --filter @drivenets/design-system exec vitest --project=storybook-docs --run |
There was a problem hiding this comment.
This is kinda "implementation details" -- let's create a new npm script for this (i.e., test:docs or something)
Maybe it should also have its own CI job?
| "test:unit": "vitest --project=unit", | ||
| "test:browser": "vitest --project=browser", | ||
| "test:requires-build": "vitest --project=requires-build", | ||
| "test:storybook-docs": "pnpm build:storybook && vitest --project=storybook-docs", |
There was a problem hiding this comment.
I think build:storybook should be a dependency of test:storybook-docs, so turbo can cache it better
Also, add test:storybook-docs as a turbo cached task
| it('Show code panel matches staged authoring rules', async () => { | ||
| const snippet = await readShowCodeSnippet(page, { | ||
| docsStoryId: DOCS_STORY_ID, | ||
| storyName: 'Default', | ||
| }); | ||
|
|
||
| expect(snippet).toContain("type: 'code'"); | ||
| expect(snippet).toContain('withTanStackRouter'); | ||
| expect(snippet).toMatchSnapshot(); | ||
| }); |
There was a problem hiding this comment.
Sorry, but I'm not really following - what's the point of these tests? Isn't it checking the implementation details of the stories' examples? I don't really understand. Let's talk?
There was a problem hiding this comment.
Let me explain the purpose of these tests by starting from the problem we've faced.
In Design System we have 2 places where we show code samples of our components usage:
- DS MCP
- "Show Code" button in Storybook's story
Each one uses its own pipeline of how stories.tsx file get's converted into code sample and it's quite tricky to understand what exact code will be produces without actual testing.
Recently, we have introduced React Compiler into Storybook build, it has broken 80% of code samples shown via "Show Code" button and noone noticed it until now.
We use AI Agent quite often especially for stories. AI doesn't verify code quality in DS MCP and "Show Code".
Eventually, we get poor quality of code snippets.
In order to solve this I've decided to add snapshot tests for every component for both types of code samples. Also I've updated skills to force AI to add those tests. This way AI will review the quality of code snippets and adjust stories to make everything perfect. Essentially, these tests give AI visibility it wouldn't have otherwise (the only way to read "Show Code" snippet is via storybook UI).
…e-code-quality-in-storybook-files
00ad22c
4 participants
Summary
Storybook Autodocs Show code snippets and MCP manifest snippets are what agents and developers copy from docs, but nothing in CI caught when those outputs drifted after story edits. Dev Storybook also differs from the production build (e.g. React Compiler), so manual checks against
pnpm startwere unreliable.This PR adds automated docs snippet tests (
*.docs.test.ts) as a dedicated Viteststorybook-docsproject. Global setup builds and servesstorybook-staticon an ephemeral port (same artifact as GitHub Pages), then tests verify both outputs per story:components.jsonand snapshots the storysnippetShared helpers live under
tests/storybook/.test:storybook-docsruns the project explicitly; defaultpnpm testexcludes it (likerequires-build). CI builds Storybook and runs docs tests on PRs. Initial coverage:ds-breadcrumbandds-button-v3. Agent skills and workflows are updated to point authors at the new pattern.