checkly pw-test

Run Playwright tests with Checkly monitoring features

The checkly pw-test command runs Playwright tests with Checkly monitoring features and reuses the Playwright CLI arguments. This command automatically records test sessions without requiring the --record flag.

Prerequisites

Before using checkly pw-test, ensure you have:

• An initialized Checkly CLI project
• Valid Checkly account authentication (run npx checkly login if needed)
• Playwright tests in your project
• A playwright.config.ts or playwright.config.js file.

Basic Usage

Use -- to separate Checkly flags from Playwright test options.

Terminal

npx checkly pw-test [checkly options] -- [playwright options]

Checkly Command Options

Define checkly pw-test specific options before the -- separator:

Option	Description
--config	The Checkly CLI configuration file. If not passed, uses the checkly.config.ts|js file in the current directory.
--create-check	Create a Checkly check from the Playwright test.
--env, -e	Env vars to be passed to the test run. Default: []
--env-file	dotenv file path to be passed. For example --env-file="./.env"
--location, -l	The location to run the checks at.
--private-location	The private location to run checks at.
--[no-]record	Record test results in Checkly as a test session with full logs, traces and videos.
--reporter	A list of custom reporters for the test output.
--stream-logs	Stream logs from the test run to the console.
--test-session-name	A name to use when storing results in Checkly
--timeout	A timeout (in seconds) to wait for checks to complete.
--verbose	Always show the full logs of the checks.

Playwright Test Options

Define Playwright test runner flags after the -- separator:

Option	Description
--project	Select Playwright projects
--grep	Filter tests by pattern
--grep-invert	Exclude tests by pattern

The --reporter, --list, and --headed options are not supported. Find more Playwright options in the Playwright test runner docs .

Checkly Command Options

--config · string

Specify a configuration file to use instead of the checkly.config.ts or checkly.config.js in the current directory.

Usage:

Terminal

npx checkly pw-test --config="./checkly.staging.config.ts"

--create-check · boolean

Add a new Playwright Check Suite with your Playwright configuration to your checkly.config.

Usage:

Terminal

npx checkly pw-test --create-check

Examples:

Running this command:

Terminal

 npx checkly pw-test --create-check -- --project="Mobile Chrome" --grep="@critical"

Adds a new Playwright Check Suite to your checkly.config.ts:

checkly.config.ts

const config = defineConfig({
  projectName: "Playwright Project",
  logicalId: "playwright-project",

  checks: {
    playwrightConfigPath: "./playwright.config.ts",
    playwrightChecks: [
      {
        logicalId: "playwright-check-project-mobile-chrome-grep-critical",
        name: 'Playwright Test: "--project=Mobile Chrome" --grep=@critical',
        testCommand:
          'npx playwright test "--project=Mobile Chrome" --grep=@critical',
        locations: ["eu-central-1"],
        frequency: 10,
      },
    ],
  },
})

If there’s no existing checkly.config.ts file, the --create-check option will create one.

--env, -e

Pass environment variables to the test run. Can be specified multiple times to set multiple variables.

Usage:

Terminal

npx checkly pw-test --env API_KEY=123 --env BASE_URL=https://example.com
npx checkly pw-test -e NODE_ENV=production -e DEBUG=true

--env-file · string

Specify a dotenv file path to load environment variables from. This is useful for managing multiple environment variables in a single file.

Usage:

Terminal

npx checkly pw-test --env-file="./.env"

--location, -l · string

Specify the geographic location where the checks should run. This determines which Checkly data center executes your tests.

Usage:

Terminal

npx checkly pw-test --location="us-east-1"
npx checkly pw-test -l="eu-west-1"

--private-location · string

Run checks at a specific private location. Private locations allow you to test internal applications or services behind a firewall.

Usage:

Terminal

npx checkly pw-test --private-location="office-network"
npx checkly pw-test --private-location="staging-vpc"

--[no-]record · boolean

The pw-test automatically records your test results as a test session with full logs, traces, and videos. Use --no-record to disable recording.

Usage:

Terminal

npx checkly pw-test --record
npx checkly pw-test --no-record

--reporter · string

Specify a custom reporter for the test output. Available options: list, dot, ci, github, json

Usage:

Terminal

npx checkly pw-test --reporter="json"
npx checkly pw-test --reporter="github"

--stream-logs · boolean

Stream logs from the test run to the console in real-time. This provides immediate feedback during test execution.

Usage:

Terminal

npx checkly pw-test --stream-logs

--test-session-name · string

Provide a custom name for the test session when storing results in Checkly. This helps identify and organize test runs.

Usage:

Terminal

npx checkly pw-test --test-session-name="Release v1.2.3 tests"
npx checkly pw-test --test-session-name="Daily regression suite"

--timeout · number · default: "600"

Set a timeout (in seconds) to wait for checks to complete.

Usage:

Terminal

npx checkly pw-test --timeout="300"
npx checkly pw-test --timeout="1200"

The current maximum timeout is 1200 seconds (20 minutes).

--verbose · boolean

Enable verbose output to always show the full logs of the checks. This provides detailed information about test execution.

Usage:

Terminal

npx checkly pw-test --verbose

Common Playwright Test Options

The pw-test command will reuse your playwright.config settings. To overwrite these values from the command line, Playwright Check Suites supports most Playwright test runner options.

Here are some commonly used command line options for running your Playwright tests in the Checkly infrastructure.

--project · string

Select specific Playwright projects to run. This allows you to run only a subset of your configured projects from your Playwright configuration.

Usage:

Terminal

npx checkly pw-test -- --project="chromium"
npx checkly pw-test -- --project="firefox"

You can only run projects specified in your playwright.config file.

--grep · string

Filter tests to run based on a pattern match against test titles. Only tests matching the pattern will be executed.

Usage:

Terminal

npx checkly pw-test -- --grep="@smoke"
npx checkly pw-test -- --grep="login|authentication"

--grep-invert · string

Exclude tests from running based on a pattern match against test titles. Tests matching the pattern will be skipped.

Usage:

Terminal

npx checkly pw-test -- --grep-invert="@slow"
npx checkly pw-test -- --grep-invert="flaky|unstable"

Playwright Configuration vs. Applied Command Line Options

Playwright Check Suites and the pw-test command read and parse your existing playwright.config to apply your configuration.

To differentiate between your local testing setup and the Checkly monitoring environment, you can rely on the set CI environment variable or override specific settings using command line options.

playwright.config.ts

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  use: {
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    // ...
  ]
});

If you run npx checkly pw-test without additional arguments, the test suite will:

• run with 2 retries
• use 1 worker
• run all defined projects including chromium

All these options can be further customized by passing command line arguments to pw-test.

Terminal

# Run with 4 workers, only in chromium, and 3 retries
npx checkly pw-test -- --workers=4 --project="chromium" --retries=3

Examples

Run tests in multiple browsers

Terminal

# Using different Playwright projects
npx checkly pw-test -- --project=chromium --project=firefox

Run tests by pattern

Terminal

# Filter tests by grep pattern
npx checkly pw-test -- --grep="login"

# Exclude tests by pattern
npx checkly pw-test -- --grep-invert="skip|todo"

Run tests by file

Terminal

# Run a specific test file
npx checkly pw-test -- test.spec.ts

Key Features

• pw-test automatically records test sessions (no --record flag needed)
• Your Playwright configuration applies automatically (traces, videos, screenshots)
• View all artifacts in Checkly’s UI

Related Commands

• checkly test - Test your setup before deployment
• checkly deploy - Deploy your Checkly configuration