You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/ai.md
+3-51Lines changed: 3 additions & 51 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,15 @@
1
1
---
2
2
permalink: /ai
3
-
title: Testing with AI 🪄
3
+
title: Testing with AI
4
4
---
5
5
6
-
# 🪄 Testing with AI
6
+
# Testing with AI
7
7
8
8
**CodeceptJS is the first open-source test automation framework with AI** features to improve the testing experience. CodeceptJS uses AI provider like OpenAI or Anthropic to auto-heal failing tests, assist in writing tests, and more...
9
9
10
10
Think of it as your testing co-pilot built into the testing framework
11
11
12
-
> 🪄 **AI features for testing are experimental**. AI works only for web based testing with Playwright, WebDriver, etc. Those features will be improved based on user's experience.
12
+
> This is guide on using AI features inside CodeceptJS. To control CodeceptJS via AI Agents, see [Agentic Testing Guide](/agents/).
13
13
14
14
## How AI Improves Automated Testing
15
15
@@ -164,50 +164,6 @@ The AI SDK supports 20+ providers including:
164
164
165
165
See [AI SDK Providers](https://ai-sdk.dev/docs/foundations/providers-and-models) for complete list and configuration details.
166
166
167
-
## Writing Tests with AI Copilot
168
-
169
-
If AI features are enabled when using [interactive pause](/basics/#debug) with `pause()` command inside tests:
170
-
171
-
For instance, let's create a test to try ai features via `gt` command:
172
-
173
-
```
174
-
npx codeceptjs gt
175
-
```
176
-
177
-
Name a test and write the code. We will use `Scenario.only` instead of Scenario to execute only this exact test.
When pause mode started you can ask GPT to fill in the fields on this page. Use natural language to describe your request, and provide enough details that AI could operate with it. It is important to include at least a space char in your input, otherwise, CodeceptJS will consider the input to be JavaScript code.
195
-
196
-
```
197
-
I.fill checkout form with valid values without submitting it
198
-
```
199
-
200
-

201
-
202
-
GPT will generate code and data and CodeceptJS will try to execute its code. If it succeeds, the code will be saved to history and you will be able to copy it to your test.
203
-
204
-

205
-
206
-
This AI copilot works best with long static forms. In the case of complex and dynamic single-page applications, it may not perform as well, as the form may not be present on HTML page yet. For instance, interacting with calendars or inputs with real-time validations (like credit cards) can not yet be performed by AI.
207
-
208
-
Please keep in mind that GPT can't react to page changes and operates with static text only. This is why it is not ready yet to write the test completely. However, if you are new to CodeceptJS and automated testing AI copilot may help you write tests more efficiently.
209
-
210
-
> 👶 Enable AI copilot for junior test automation engineers. It may help them to get started with CodeceptJS and to write good semantic locators.
211
167
212
168
## Self-Healing Tests
213
169
@@ -409,9 +365,7 @@ ai: {
409
365
410
366
CodeceptJS uses three main prompts for AI features:
411
367
412
-
- `writeStep` - generates test code in interactive pause mode
413
368
- `healStep` - suggests fixes for failing tests
414
-
- `generatePageObject` - creates page objects from HTML
415
369
416
370
To customize a prompt, generate it using:
417
371
@@ -452,9 +406,7 @@ You can also override prompts programmatically in config:
452
406
```js
453
407
ai: {
454
408
prompts: {
455
-
writeStep: (html, input) => [{ role:'user', content:'As a test engineer...' }]
456
409
healStep: (html, { step, error, prevSteps }) => [{ role:'user', content:'As a test engineer...' }]
457
-
generatePageObject: (html, extraPrompt='', rootLocator=null) => [{ role:'user', content:'As a test engineer...' }]
Copy file name to clipboardExpand all lines: docs/helpers/Playwright.md
+13-16Lines changed: 13 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,6 +5,7 @@ sidebar: auto
5
5
title: Playwright
6
6
---
7
7
8
+
8
9
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
9
10
10
11
## Playwright
@@ -46,14 +47,14 @@ Type: [object][6]
46
47
*`url`**[string][9]?** base url of website to be tested
47
48
*`browser`**(`"chromium"` | `"firefox"` | `"webkit"` | `"electron"`)?** a browser to test on, either: `chromium`, `firefox`, `webkit`, `electron`. Default: chromium.
48
49
*`show`**[boolean][27]?** show browser window.
49
-
*`restart`**([string][9] | [boolean][27])?** restart strategy between tests. Possible values:* 'context' or **false** - restarts [browser context][45] but keeps running browser. Recommended by Playwright team to keep tests isolated.
50
+
*`restart`**([string][9] | [boolean][27])?** restart strategy between tests. Possible values:* 'context' or **false** - restarts [browser context][44] but keeps running browser. Recommended by Playwright team to keep tests isolated.
50
51
* 'session' or 'keep' - keeps browser context and session, but cleans up cookies and localStorage between tests. The fastest option when running tests in windowed mode. Works with `keepCookies` and `keepBrowserState` options. This behavior was default before CodeceptJS 3.1
51
-
*`timeout`**[number][18]?** * [timeout][46] in ms of all Playwright actions .
52
+
*`timeout`**[number][18]?** * [timeout][45] in ms of all Playwright actions .
52
53
*`disableScreenshots`**[boolean][27]?** don't save screenshot on failure.
53
54
*`emulate`**any?** browser in device emulation mode.
54
55
*`video`**[boolean][27]?** enables video recording for failed tests; videos are saved into `output/videos` folder
55
56
*`keepVideoForPassedTests`**[boolean][27]?** save videos for passed tests; videos are saved into `output/videos` folder
56
-
*`trace`**[boolean][27]?** record [tracing information][47] with screenshots and snapshots.
57
+
*`trace`**[boolean][27]?** record [tracing information][46] with screenshots and snapshots.
57
58
*`keepTraceForPassedTests`**[boolean][27]?** save trace for passed tests.
58
59
*`fullPageScreenshots`**[boolean][27]?** make full page screenshots on failure.
59
60
*`uniqueScreenshotNames`**[boolean][27]?** option to prevent screenshot override if you have scenarios with the same name in different suites.
*`electron`**[object][6]?** (pass additional electron options
76
-
*`channel`**any?** (While Playwright can operate against the stock Google Chrome and Microsoft Edge browsers available on the machine. In particular, current Playwright version will support Stable and Beta channels of these browsers. See [Google Chrome & Microsoft Edge][48].
77
-
*`ignoreLog`**[Array][10]<[string][9]>?** An array with console message types that are not logged to debug log. Default value is `['warning', 'log']`. E.g. you can set `[]` to log all messages. See all possible [values][49].
77
+
*`channel`**any?** (While Playwright can operate against the stock Google Chrome and Microsoft Edge browsers available on the machine. In particular, current Playwright version will support Stable and Beta channels of these browsers. See [Google Chrome & Microsoft Edge][47].
78
+
*`ignoreLog`**[Array][10]<[string][9]>?** An array with console message types that are not logged to debug log. Default value is `['warning', 'log']`. E.g. you can set `[]` to log all messages. See all possible [values][48].
78
79
*`ignoreHTTPSErrors`**[boolean][27]?** Allows access to untrustworthy pages, e.g. to a page with an expired certificate. Default value is `false`
79
80
*`bypassCSP`**[boolean][27]?** bypass Content Security Policy or CSP
80
81
*`highlightElement`**[boolean][27]?** highlight the interacting elements. Default: false. Note: only activate under verbose mode (--verbose).
81
82
*`recordHar`**[object][6]?** record HAR and will be saved to `output/har`. See more of [HAR options][3].
82
-
*`testIdAttribute`**[string][9]?** locate elements based on the testIdAttribute. See more of [locate by test id][50].
83
+
*`testIdAttribute`**[string][9]?** locate elements based on the testIdAttribute. See more of [locate by test id][49].
83
84
*`storageState`**([string][9] | [object][6])?** Playwright storage state (path to JSON file or object)
84
85
passed directly to `browser.newContext`.
85
86
If a Scenario is declared with a `cookies` option (e.g. `Scenario('name', { cookies: [...] }, fn)`),
0 commit comments