#Integrate with Playwright Test

With PlaywrightAiFixture, you can use Midscene's visual interaction and assertion capabilities directly in an existing Playwright Test project. Playwright Test organizes and runs the tests, while Midscene interacts with the page and generates execution reports.

#Before you begin

• Runtime: Install Node.js ^20.19.0 || ^22.12.0 || >=24.0.0 and pnpm. Have an existing Playwright Test project that can run tests with Chromium.
• Model configuration and verification: Complete model configuration and model verification, then save the configuration in the project's .env file or shell environment.
• Platform environment: Install Chromium and make sure the browser can access the target page.
• (Optional) Try Playground first: Run an action and an assertion against your target interface in the Chrome extension Playground to verify screenshots, interactions, and model behavior.

#1. Install dependencies and configure the project

npm install @midscene/web dotenv --save-dev

yarn add @midscene/web dotenv --save-dev

pnpm add @midscene/web dotenv --save-dev

bun add @midscene/web dotenv --save-dev

deno add npm:@midscene/web npm:dotenv --save-dev

Add the following configuration to your existing playwright.config.ts, while keeping your project's other settings and reporters:

import 'dotenv/config';
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  timeout: 90_000,
  use: { browserName: 'chromium' },
  reporter: [
    ['list'],
    ['@midscene/web/playwright-reporter'],
  ],
});

dotenv/config loads model settings from the project's .env file (settings in the Chrome extension Playground are not automatically shared with test runs). Because model calls take longer than standard DOM operations, increase timeout as needed for your tests.

#2. Create a fixture

Create e2e/fixture.ts to extend Playwright's test with PlaywrightAiFixture:

import { test as base } from '@playwright/test';
import {
  PlaywrightAiFixture,
  type PlayWrightAiFixtureType,
} from '@midscene/web/playwright';

export const test = base.extend<PlayWrightAiFixtureType>(
  PlaywrightAiFixture(),
);

In tests that use Midscene, import test from ./fixture while continuing to import expect from @playwright/test.

#3. Write a test

import { expect } from '@playwright/test';
import { test } from './fixture';

test('add a todo', async ({ page, aiAct, aiAssert }) => {
  await page.goto('https://todomvc.com/examples/react/dist/');
  await expect(page).toHaveTitle(/TodoMVC/);

  await aiAct('Add a todo named Try Midscene');
  await aiAssert('The todo list contains Try Midscene');
});

You can combine Playwright's page and expect with Midscene's AI methods in the same test. In this example, Playwright opens the page and checks its title, while Midscene adds a todo item and asserts the result on the page. Beyond aiAct and aiAssert, the fixture also provides aiTap, aiInput, aiQuery, aiWaitFor, and more; see the Agent API reference.

#4. Run the test and view the report

(Optional) Verify your model configuration before running the test:

pnpm dlx @midscene/test@latest model verify

Run the test from the project directory:

pnpm exec playwright test e2e/todo.spec.ts

After the test finishes, the Midscene reporter prints the report path in the terminal (saved under midscene_run/report/ by default). Open the generated HTML file in a browser to inspect step-by-step screenshots, AI execution details, and assertion results.

#Common methods

In practice, you typically perform actions with aiAct or direct interaction methods and then verify the outcome with aiAssert—completing an action does not guarantee that the expected UI state was reached. For full parameter and return value details, see the Agent API reference.

#Further configuration

#Agent options

Pass Agent options when creating the fixture:

import { test as base } from '@playwright/test';
import {
  PlaywrightAiFixture,
  type PlayWrightAiFixtureType,
} from '@midscene/web/playwright';

export const test = base.extend<PlayWrightAiFixtureType>(
  PlaywrightAiFixture({
    waitForNetworkIdleTimeout: 2000,
    replanningCycleLimit: 30,
  }),
);

You can also pass other Agent options such as waitAfterAction and modelConfig. Test and report metadata (testId, reportFileName, groupName, and groupDescription) are automatically derived from the current test info and do not need to be set manually. See Playwright SDK configuration for platform options and shared Agent options for common settings.

#Work with a specific page

The AI methods injected by the fixture operate on Playwright's default page fixture. To interact with another Page instance in the same test (such as one created via context.newPage()), use agentForPage to get the corresponding Agent:

import { test } from './fixture';

test('inspect another page', async ({ context, agentForPage }) => {
  const page = await context.newPage();
  await page.goto('https://todomvc.com/examples/react/dist/');

  const agent = await agentForPage(page);
  await agent.aiAssert('The page contains a todo input');
});

For details on new tabs and navigation behavior, see Web / Playwright SDK.

#Lifecycle

Within a test, the fixture creates one Agent per Page and reuses it across AI calls. When a page closes or the test finishes, the fixture automatically calls agent.destroy() and collects the report, so you do not need to destroy fixture-managed Agents manually. Browser contexts and pages remain managed by Playwright.

If you bypass the fixture and instantiate an Agent directly with new PlaywrightAgent(page), you must call agent.destroy() yourself; see Lifecycle and resource ownership.

#Report options

Pass options to the reporter in playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['@midscene/web/playwright-reporter', {
      type: 'merged',
      outputFormat: 'single-html',
    }],
  ],
});

• type: 'merged' (default) combines all tests into a single report; 'separate' generates a separate report for each test.
• outputFormat: 'single-html' (default) embeds screenshots into a single HTML file; 'html-and-external-assets' saves screenshots as external files to reduce the HTML file size.

When using 'html-and-external-assets', keep the HTML file and its asset directory in their relative paths, and preview the report through a local HTTP server (opening it directly via file:// is not supported):

pnpm dlx serve midscene_run/report

Then open the URL printed in the terminal. When sharing or archiving the report, include both the HTML file and its asset directory.

For a complete working project, see the Playwright Test example.