Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion constructs/playwright-check.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ new PlaywrightCheck("critical-e2e-monitor", {
playwrightConfigPath: "./playwright.config.ts",
engine: Engine.node("24"), // Optional: override the auto-detected engine.
installCommand: "npm ci",
testCommand: "npx playwright test --retries=2",
testCommand: "npx playwright test --max-failures=5",
pwProjects: ["chromium"],
pwTags: ["e2e"],
})
Expand Down Expand Up @@ -145,6 +145,10 @@ new PlaywrightCheck("critical-e2e-monitor", {

<Info>If you configure the `testCommand` property and `pwTags` or `pwProjects` the options will be merged and all configurations applied to `npx playwright test`.</Info>

<Info>
Checkly applies these Playwright defaults when you do not set them yourself: `workers: 4`, `retries: 2`, `use.video: "on-first-retry"`, and `use.screenshot: "on-first-retry"`. Checkly also records traces on every run, uses the Checkly reporter, and appends a Checkly user-agent suffix. Checkly supports Playwright's `globalTimeout`; the timer starts when `npx playwright test` begins.
</Info>

</ResponseField>

<ResponseField name="pwProjects" type="string">
Expand Down
56 changes: 43 additions & 13 deletions detect/synthetic-monitoring/playwright-checks/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sidebarTitle: 'Configuration'
Use the `checkly.config.ts/js` file to define your Playwright Check Suite. Each Playwright Check Suite can be connected to references in your `playwright.config.ts/js` file.

<Info>
**A Playwright Check Suite can run up to 30 minutes**. [Please contact us in the Checkly Web App](https://app.checklyhq.com/?support=true) or get in touch with your account executive if you're interested in longer runs.
**The Playwright test execution stage can run for up to 30 minutes**. Use Playwright's `globalTimeout` to set a custom suite-level limit. [Contact us in the Checkly Web App](https://app.checklyhq.com/?support=true) or get in touch with your account executive if you need longer runs.
</Info>

## Playwright Check Suite definition
Expand Down Expand Up @@ -103,14 +103,38 @@ A Playwright Check Suite inherits multiple properties from the abstract `Check`
Check out [the reference documentation](/constructs/project#param-checks-playwright-checks) for more information on these settings.

<Note>
**Checks' Retry strategy is not applicable for Playwright checks.** Playwright includes its own retry features that can be set up directly in your `playwright.config.ts/js` file with the [`retries`](https://playwright.dev/docs/test-retries) option. This allows for more detailed management of test retries within Playwright, when your check runs.
**Checkly retry strategies do not apply to Playwright Check Suites.** Use Playwright's [`retries`](https://playwright.dev/docs/test-retries) option in your `playwright.config.ts/js` file or `testCommand` to control test retries.
</Note>

Additionally, Playwright Check Suites provide specific configuration for your Playwright monitoring.
### Checkly-applied Playwright settings

When Checkly runs a Playwright Check Suite, it applies Playwright settings that produce useful monitoring results and artifacts.

Checkly applies these defaults only when you do not set the option yourself:

| Playwright option | Checkly default |
|-------------------|-----------------|
| `workers` | `4` |
| `retries` | `2` |
| `use.video` | `on-first-retry` |
| `use.screenshot` | `on-first-retry` |

Your Playwright config or `testCommand` value takes precedence for these options.

Checkly enforces these options for every run. User configuration does not override them:

| Playwright setting | Checkly behavior |
|--------------------|------------------|
| `use.trace` | `on` for every run |
| User agent | Checkly appends a [user-agent suffix](/platform/allowlisting-traffic#default-checkly-user-agent) |

Checkly supports Playwright's [`globalTimeout`](https://playwright.dev/docs/api/class-testconfig#test-config-global-timeout). Use it to control the maximum duration of the Playwright suite after `npx playwright test` starts.

Playwright Check Suites also provide specific configuration for your Playwright monitoring.

* `installCommand:` Override the command to install dependencies. `npm install --dev` is used by default.

* `testCommand:` Override the command to test. `npx playwright test` is used by default with the tags, projects, and config file options your check specifies.
* `testCommand:` Override the command that runs tests. `npx playwright test` is used by default with the tags, projects, and config file options your check specifies.

* `group:` The group construct this check belongs to.

Expand All @@ -132,22 +156,28 @@ installCommand: 'yarn install --frozen-lockfile'
installCommand: 'npm install --prefix ./e2e'
```

**Enable Playwright tracing to debug failures**
**Set a lower test timeout:**

```typescript
testCommand: 'npx playwright test --trace=on'
testCommand: 'npx playwright test --timeout=60000'
```

**Set a specific test timeout:**
**Set a global suite timeout:**

```typescript
testCommand: 'npx playwright test --timeout=60000'
testCommand: 'npx playwright test --global-timeout=120000'
```

**Stop early after failures:**

```typescript
testCommand: 'npx playwright test --max-failures=5'
```

**Combine multiple test flags:**

```typescript
testCommand: 'npx playwright test --trace=retain-on-failure --max-failures=5'
testCommand: 'npx playwright test --timeout=60000 --max-failures=5'
```

### Example Checkly config
Expand All @@ -169,7 +199,7 @@ export default defineConfig({
pwProjects: ['chromium', 'firefox', 'webkit'],
pwTags: '@e2e',
installCommand: 'npm install --dev',
testCommand: 'npx playwright test --trace=on',
testCommand: 'npx playwright test --max-failures=5',
activated: true,
muted: false,
group: productionGroup,
Expand Down Expand Up @@ -318,17 +348,17 @@ If tests fail due to authentication issues:
- Verify environment variables are set correctly in Checkly. You can verify your environment variables using `npx checkly env ls`, or looking at your Global or Check's environment variables in the Checkly Webapp to ensure any `process.env.VARIABLE_NAME` call from your test is defined.
- For persistent authentication, use Playwright's [`storageState`](https://playwright.dev/docs/auth#reuse-authentication-state) feature.

**webServer configuration requires additional files**
#### webServer configuration requires additional files

When using `webServer` in your Playwright config, you need to include the server files in your check configuration:

- Add server files to the `include` parameter in your [Playwright check configuration](/constructs/playwright-check/#param-include)
- Include all files needed to start the server (e.g., server scripts, config files)
- Alternatively, skip `webServer` on Checkly and use your deployed URL: `...(process.env.CHECKLY ? {} : { webServer: { ... } })`

**No tests found**
#### No tests found

If Playwright couldn't find any tests to run, this usually means a mismatch between your config and your test files:
If Playwright does not find any tests to run, this usually means a mismatch between your config and your test files:

- Verify your `--grep` pattern matches actual test tags or projects
- Verify the `testDir` path in `playwright.config.ts` matches the actual directory containing your test files
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,82 +6,92 @@ tags: ["synthetic-monitoring", "playwright-checks", "environment-variables"]
---

<Note>
For information on creating and managing environment variables and secrets (including the difference between variables and secrets, inheritance, and where to set them), see the [Environment Variables documentation](/platform/variables).
To create and manage environment variables and secrets, including inheritance and where to set them, see the [Environment Variables documentation](/platform/variables).
</Note>

## Built-in variables available in Playwright Check Suites
## Built-in variables in Playwright Check Suites

Checkly sets the following environment variables on every Playwright Check Suite run:

| Variable | Description |
|----------|-------------|
| `CHECKLY` | Set to `1` for all check runs executed by Checkly.<br/>Useful to distinguish Checkly runs from local runs. |
| `CHECKLY_RUN_SOURCE` | Indicates how the check was triggered.<br/>Useful to customize the nature of your test depending on how it is run.<br/>See [CHECKLY_RUN_SOURCE values](#checkly-run-source-values) below. |
| `CI` | Set to `1` for CLI runs (`npx checkly test` or `npx checkly trigger`) and [deployment](/integrations/ci-cd/github/deployments)-triggered checks. |
| `CHECKLY` | Checkly sets this to `1` for every Playwright Check Suite run. Use it to distinguish Checkly runs from local runs. |
| `CHECKLY_RUN_SOURCE` | Identifies what triggered the check run. Use it to adjust test behavior for schedules, manual triggers, deployments, or CLI runs. See [`CHECKLY_RUN_SOURCE` values](#checkly-run-source-values). |
| `CI` | Checkly sets this to `1` for CLI runs (`npx checkly test` or `npx checkly trigger`) and checks started by [deployments](/integrations/ci-cd/github/deployments). |
| `ACCOUNT_ID` | The UUID of the Checkly account. |
| `CHECK_NAME` | The name of the check. |
| `CHECKLY_CHECK_ID` | The UUID of the check. |
| `CHECKLY_REGION` | The region where the check was executed. |
| `CHECKLY_REGION` | The region where Checkly ran the check. |

### `CHECKLY_RUN_SOURCE` values

| Value | Description |
|-------|-------------|
| `CLI_DEPLOY` | Used for the first run of Checks deployed using `npx checkly deploy`. |
| `DEPLOYMENT` | Used for Check runs triggered as part of a [CI/CD deployment](/integrations/ci-cd/github/deployments). |
| `GROUP_RUN_ALL` | Used for Check runs triggered from a Group's edit screen, by clicking the "Run all checks" button. |
| `SCHEDULE_NOW` | Used for Check runs triggered manually by clicking "Schedule now" in the webapp. |
| `SCHEDULER` | Used for Check runs triggered as part of their regular schedule. |
| `TEST_NO_RECORD` | Used for Check runs triggered by the `npx checkly test` CLI command. |
| `TEST_RECORD` | Used for Check runs triggered by the `npx checkly test --record` or `npx checkly pw-test` CLI commands. |
| `TRIGGER_API` | Used for Check runs triggered by the API. |
| `TRIGGER_NO_RECORD` | Used for Check runs triggered by the `npx checkly trigger` CLI command. |
| `TRIGGER_RECORD` | Used for Check runs triggered by the `npx checkly trigger --record` CLI command. |
| `CLI_DEPLOY` | The first run after `npx checkly deploy`. |
| `DEPLOYMENT` | A [CI/CD deployment](/integrations/ci-cd/github/deployments) started the run. |
| `GROUP_RUN_ALL` | The "Run all checks" action on the group's edit screen started the run. |
| `SCHEDULE_NOW` | The "Schedule now" action in the web app started the run. |
| `SCHEDULER` | The regular check schedule started the run. |
| `TEST_NO_RECORD` | `npx checkly test` started the run. |
| `TEST_RECORD` | `npx checkly test --record` or `npx checkly pw-test` started the run. |
| `TRIGGER_API` | The API started the run. |
| `TRIGGER_NO_RECORD` | `npx checkly trigger` started the run. |
| `TRIGGER_RECORD` | `npx checkly trigger --record` started the run. |

## Use built-in variables

Common ways to use Checkly's built-in environment variables:

Use Checkly's built-in environment variables to tune test behavior for different run contexts.

### Adjust behavior based on trigger type

Set different behavior for scheduled runs versus manual triggers:
Use different timeouts for scheduled runs and manual triggers:

```typescript highlight={4}
import { test } from '@playwright/test';

test('API health check', async ({ request }) => {
// Scheduled monitors should fail fast. Manual runs can wait longer for debugging.
const isScheduledRun = process.env.CHECKLY_RUN_SOURCE === 'SCHEDULER';

// Use shorter timeout for scheduled runs
const timeout = isScheduledRun ? 5000 : 30000;

const response = await request.get('https://api.example.com/health', {
timeout
timeout,
});

// ... assertions
});
```

### Configure Playwright based on environment

Adjust your Playwright configuration to always record traces on Checkly.
### Configure Playwright by environment

In your `playwright.config.ts`, use the CHECKLY variable to identify it's from Checkly.
Use the built-in [`CHECKLY` environment variable](#built-in-variables-in-playwright-check-suites) in your `playwright.config.ts` to use different settings for Checkly runs and local development.

```typescript playwright.config.ts highlight={7}
```typescript playwright.config.ts highlight={5-9}
import { defineConfig } from '@playwright/test';

export default defineConfig({

// Run one worker in Checkly when tests share state or accounts.
workers: process.env.CHECKLY === '1' ? 1 : undefined,
use: {
// Always trace in Checkly to compare successful and failed runs, only on retry elsewhere
trace: process.env.CHECKLY === '1' ? 'on' : 'on-first-retry',
// Checkly runs against a public URL. Local runs use the dev server.
baseURL: process.env.CHECKLY === '1'
? 'https://www.example.com'
: 'http://localhost:3000',
},
});
```

<Note>
Playwright Check Suites record traces for successful and failed runs. Checkly uses these traces to show dedicated Network and Browser console sections, so you can compare requests, logs, and errors when you troubleshoot.
</Note>

You can also adjust these settings for Checkly runs:

- `maxFailures`: stop early during monitoring runs, for example after the first failure.
- `timeout`: give tests more or less time in Checkly than during local development.
- `expect.timeout`: tune assertion timeouts for production or staging environments.
- `retries`: override Checkly's default retry count if your suite needs a different policy.

### Access region information

Use region information for debugging or region-specific behavior:
Expand All @@ -92,13 +102,12 @@ import { test } from '@playwright/test';
test('geo-specific content test', async ({ page }) => {
await page.goto('https://example.com');

// Log region for debugging
// Include the region in logs to compare results across locations.
console.log(`Running in region: ${process.env.CHECKLY_REGION}`);

// Region-specific assertions
// Add assertions only for regions where the content should appear.
if (process.env.CHECKLY_REGION?.startsWith('eu-')) {
// Expect EU-specific content, like a cookie banner
// Example: check for an EU-specific cookie banner.
}
});
```

Loading
Loading