test(e2e): add ExTester end-to-end test for the Deepnote notebook flow#430
Draft
tkislan wants to merge 2 commits into
Draft
test(e2e): add ExTester end-to-end test for the Deepnote notebook flow#430tkislan wants to merge 2 commits into
tkislan wants to merge 2 commits into
Conversation
Adds a black-box ExTester (vscode-extension-tester) E2E suite that drives the full Deepnote happy path through the real VS Code UI: open a workspace folder and a one-notebook `.deepnote` file, create a Deepnote environment, select it for the notebook (kernel connects, venv + deepnote-toolkit provisioned), run the cell, and assert the rendered output contains "hello world". Beyond the test itself, this wires up reproducible setup and the fixes required to make it pass in a headless/sandboxed VS Code instance: - enable-proposed-api.js allow-lists the extension's proposed APIs in the test VS Code's product.json (the extension does not activate otherwise) - open the containing folder as a workspace (the serializer otherwise blocks on a "no workspace folder" snapshot warning) and open the notebook via Quick Open, since ExTester's `code -r` reuse-window silently no-ops in the sandbox - accept the simple folder dialog via its OK button (Enter navigates into dirs) - run via the toolbar "Run All" button and re-issue until output renders (deepnote.runallcells is gated behind context keys unset under automation) - idempotent environment creation with a stable name (reuses the venv) - exclude e2e artifacts from the VSIX in .vscodeignore Verified passing locally on headless Linux (Xvfb, VS Code 1.111.0), including a clean-state run that provisions the venv from scratch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQk7n13UfmeDo2ujy8X4LX
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #430 +/- ##
===========================
===========================
🚀 New features to boost your workflow:
|
Verified the extension does not need a proposed-API grant: although it declares Jupyter's `enabledApiProposals`, the core flow — activation, the notebook serializer, kernel execution, and output rendering — runs on stable VS Code APIs (output goes through `controller.createNotebookCellExecution` with a `replaceCells` fallback; the one genuinely-proposed call is guarded). On a plain stable VS Code the proposals are simply ungranted (a non-fatal log) and nothing breaks — which is how the published Marketplace / Open VSX extension runs for end users. So the E2E suite no longer patches the test VS Code's `product.json`: - remove `test/e2e/enable-proposed-api.js` - remove the `setup:e2e:proposed-api` script and drop it from `setup:e2e` - update the plan doc accordingly (§0, §6.x, §8, §9) Verified: `npm run test:e2e` passes with no Deepnote entry in the test VS Code's `product.json` allow-list, including a clean-state run that builds the venv from scratch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LQk7n13UfmeDo2ujy8X4LX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai ignore
Summary
Adds a black-box ExTester (
vscode-extension-tester) E2E test that drives the full Deepnote happy path through the real VS Code UI: open a workspace folder + a one-notebook.deepnotefile → create a Deepnote environment → select it for the notebook (kernel connects, venv +deepnote-toolkitprovisioned) → Run All → assert the rendered output containshello world.What's included
test/e2e/— the suite (helloWorld.e2e.test.ts), config (tsconfig.json,.mocharc.js,settings.json), and the hello-world fixturepackage.json—vscode-extension-testerdevDep +compile-e2e/setup:e2e*/test:e2escripts.gitignore/.vscodeignore— ignore ExTester artifacts and keep them out of the packaged VSIXspecs/e2e-extester-testing-plan.md— self-contained plan/reference, with a "§0 Implementation reality" section documenting the fixes belowKey fixes needed to pass headlessly
code -rreuse-window silently no-ops in the sandboxdeepnote.runallcellsis gated behind context keys that are unset under automation)No proposed-API grant needed
The extension declares Jupyter's
enabledApiProposals, but the core flow (activation, the notebook serializer, kernel execution, output rendering) runs on stable VS Code APIs, so the suite runs on a plain stable VS Code with noproduct.jsonallow-listing — the way the published Marketplace / Open VSX extension runs for end users. (An earlier revision patchedproduct.json; that was verified unnecessary and removed.)How to run
Verification
Verified passing locally on headless Linux (Xvfb, VS Code 1.111.0), with no proposed-API grant in
product.json, including a clean-state run that provisions the venv from scratch:🤖 Generated with Claude Code
https://claude.ai/code/session_01LQk7n13UfmeDo2ujy8X4LX