#See interfaces like a human
E2E testing in natural language

Midscene.js provides a vision-based GUI Agent and a natural-language E2E testing framework for web, mobile, and desktop apps.

🔥 Latest news Starting with v1.16, Midscene supports using Jev 1.4’s visual capabilities to drive E2E testing. Try it out

#Describe steps and expectations in natural language

Describe what a user wants to do and what they should see. Express the same intent in YAML or in your existing JavaScript tests.

YAML · @midscene/test

Vitest + JavaScript

beforeEach:
  - gotoUrl: https://your-shop.example

cases:
  - name: "Find headphones under $100"
    steps:
      - aiAct: "Search for headphones and filter the results to prices under $100"
      - aiWaitFor: "The filtered search results are visible"
      - aiAssert: "There are products in the search results, and each costs less than $100"

import { test } from 'vitest';
import { PlaywrightAgent } from '@midscene/web/playwright';

// Model configuration, page setup, and cleanup omitted
test("Find headphones under $100", async () => {
  const agent = new PlaywrightAgent(page);
  await agent.aiAct("Search for headphones and filter the results to prices under $100");
  await agent.aiWaitFor("The filtered search results are visible");
  await agent.aiAssert("There are products in the search results, and each costs less than $100");
}, 180_000);

#A GUI Agent built for testing

Midscene’s GUI Agent brings visual actions and assertions to your existing Playwright, Puppeteer, or JavaScript tests.

#See, act, and verify like a human

#One API, across platforms

Visual localization does not depend on a particular UI rendering stack. Platform adapters provide screenshots and actions, bringing the same approach to web, mobile, and desktop interfaces, including HTML canvas.

#Model options and cost

#Benchmarks

• AndroidWorld: Pass@1 of 93.1% and Pass@3 of 97.4%. Read the benchmark report
• MobileWorld: Pass@1 of 78.6%, completing 92 of 117 tasks. Read the benchmark report
• AppControlBench: Pass@1 of 96.7%, passing 58 of 60 tasks. Read the benchmark report

#A new E2E testing framework designed for natural language

Midscene Test (@midscene/test) is designed from the ground up for natural-language E2E testing. Write actions and assertions in YAML, extend test steps with TypeScript Nodes, and manage setup, cleanup, retries, and execution reports within one framework. Get started with Midscene Test

#AI steps and Nodes in one YAML file

Organize tests in YAML so people and AI agents can read, write, and review natural-language intent alongside code steps.

1beforeEach:2  - order.prepare: { status: paid }Node3  - gotoUrl: ${shopUrl}/orders/latestNode45cases:6  - name: A paid order supports a full refund7    steps:8      - aiAct: Apply for a full refund and submitAI9      - aiAssert: The page shows that the refund was submittedAI10      - order.checkRefund: { status: refunding }Node1112afterEach:13  - clearCookies: {}Node

#Customize your test project

Extend Midscene Test with TypeScript Nodes for your business operations. Register them to make them callable from YAML, then generate a Markdown Node Spec describing the available operations and their inputs. People and AI agents can use the Spec to reference these Nodes in test cases. Develop custom Nodes

#Who is using Midscene

ByteDanceAlibabaCtripXiaomiRome OSVolcengineDouyinTikTokLarkSoda MusicDoubao

#Next steps

• Try visual interaction: Follow the quick start to try natural-language actions and assertions
• Create a test project: Write and run YAML test cases with Midscene Test
• Let AI coding agents operate your UI: Connect them through Midscene Skills