#Midscene Test overview

Midscene Test is a new framework for AI-driven end-to-end (E2E) testing.

Provided by @midscene/test, it separates declarative test cases from programmable extensions: use YAML and natural language to describe test flows, and TypeScript to extend business operations and test infrastructure.

#Testing Engineering Challenges in the AI Era

Even though GUI Agents can already drive E2E testing processes, there are still many practical demands before actual production deployment. Typical challenges include:

1. Need to Integrate Deterministic Scripts: In actual testing scenarios, there is a clear need to combine deterministic scripts (such as API calls, data preparation, and test case lifecycle control) to handle background or setup/teardown tasks, ensuring test engineering remains stable, fast, and cost-effective.
2. Preference for a Declarative Core Flow: A strong desire for the core testing flow to be in a YAML-like declarative language, rather than embedding a large amount of natural language inside .ts / .js code, which burdens the codebase and hurts the long-term maintenance experience.
3. Requirement for Extensibility to Co-maintain Scripts: The need for sufficient extensibility to let Agents and humans co-maintain test scripts, keeping clear boundaries and collaboration models between deterministic engineering and natural language intent to avoid tangled code and natural language.

To address these practical deployment challenges, Midscene has designed a new testing framework, @midscene/test.

#Design Philosophy: Balancing Declarative Semantics and Deterministic Engineering

To embrace the evolution of test engineering, @midscene/test returns to the first principles of end-to-end (E2E) testing and advocates a design philosophy of balancing "Declarative Semantics" and "Deterministic Engineering".

To address these challenges, Midscene's test framework focuses on providing the following capabilities:

#Official Atomic Capabilities with High Extensibility

Midscene provides common AI operations such as aiAct and aiAssert, plus platform operations for browsers and devices. The scaffold generated by midscene-test create explicitly registers these official Nodes for the selected platform; handwritten configuration must register them explicitly.

To handle complex business scenarios, Midscene supports custom atomic nodes (custom Nodes) defined in TypeScript. You can wrap APIs, data preparation, or custom toolchains into cohesive blocks and call them directly in upper-level YAML test cases. This keeps the authoring process simple while allowing the test engineering to scale for custom requirements.

#Writing Daily Test Cases via Declarative YAML Files

Focusing on "What to test", expressing business test intent. At this layer, test authors can describe UI actions (such as aiAct) and assertions (such as aiAssert) in natural language, while calling and composing custom business capabilities provided by "Deterministic Engineering" (such as order.prepare). Test authors can focus on business flows without needing to worry about underlying details, which enhances authoring efficiency and long-term maintainability.

#Auto-Generating an Agent-Friendly Node Spec

midscene-test nodes exports the official and custom Nodes effectively registered for the current Project, together with their validation rules (Zod Schema), as a Markdown Node Spec. This removes the cost of manually maintaining a project API manual and gives human authors and AI Agents an accurate view of the current project's capabilities.

#Engineering Demands of E2E Projects

In actual enterprise-level adoption, E2E testing projects are by no means one-time, AI-driven instant verifications. Instead, they must run continuously as long-term assets, facing real and rigorous engineering demands such as stability, speed, and maintainability. To address this, Midscene provides supporting engineering capabilities to ensure the stability and efficiency of E2E projects in production:

• Unified Observability and Logging: Both AI visual steps and your custom Node executions are centrally recorded in the same execution lifecycle. The test report and runtime logs visualize the inputs, outputs, duration, execution status, and screenshots of every step. This simplifies troubleshooting and replay for human engineers and provides structured context for AI Agents to autonomously diagnose issues and optimize test cases.
• Standardized Lifecycle and Concurrency Control: Out-of-the-box lifecycle hooks (Before/After), concurrent environment isolation, and sandboxing ensure that your custom framework and test cases run stably and predictably in CI (Continuous Integration) pipelines.

#Example Scenario: Verify an Order Refund Flow

Suppose an e-commerce team needs to verify the refund flow for paid orders:

1. Deterministic Engineering (TypeScript Nodes): call an order API or script before each case to create test data, and clean up the test order when the case finishes.
2. Declarative Semantics (AI Natural Language): use YAML instructions to have the Agent submit a refund request in the UI and verify the result.

#Project File Structure

The recommended file structure for this example project is as follows:

ecommerce-tests/
├── cases/
│   └── refund.yaml            # Declarative cases: describe test cases and steps via YAML files
└── midscene.config.ts         # Deterministic engineering: register custom Nodes (e.g. order.prepare), define execution environments

• midscene.config.ts: Maintained by automation/test platform developers to build the underlying "Deterministic Engineering" (writing custom Zod Schemas, execution logic for custom Nodes, configuring browsers, etc.).
• cases/refund.yaml: Maintained by QA engineers (or AI Agents) to compose and express high-level business test intents via "Declarative Semantics" (combining natural language and custom Nodes).

#1. Auto-Generated Node Spec Example

In this scenario, the custom Node order.prepare defined by engineering builders appears in the Node Spec generated by the nodes tool:

## order.prepare

- **Title**: Prepare order data
- **Description**: Pre-create an order with a specified status in the database.
- **Parameter schema (JSON Schema)**:

{
  "type": "object",
  "properties": {
    "status": {
      "type": "string",
      "description": "Order status, such as paid or refunded."
    }
  },
  "required": ["status"]
}

#2. Writing and Running Test Cases (YAML)

After reading the reference, test authors (or AI Agents) can directly call and compose the Node in the declarative YAML file:

# 1. Deterministic Engineering: prepare data and the environment
beforeEach:
  - order.prepare:
      status: paid
  - browser.openRefundPage: {}

# 2. Declarative Semantics: express business test intent
cases:
  - name: A paid order supports a full refund
    steps:
      - aiAct: Click Apply for refund, select Full refund, and submit
      - aiAssert:
          prompt: The page shows that the refund request was submitted and the refund amount equals the full order amount
          message: Failed to submit the full refund request

  - name: A paid order supports a partial refund
    steps:
      - aiAct: Click Apply for refund, enter a refund amount of 10, and submit
      - aiAssert:
          prompt: The page shows that the refund request was submitted and the refund amount is 10
          message: Failed to submit the partial refund request

# 3. Deterministic Engineering: clean up resources
afterEach:
  - order.cleanup: {}

Framework maintainers register custom Nodes such as order.prepare, browser.openRefundPage, and order.cleanup to manage data and pages. Midscene Nodes handle UI actions and assertions. Test intent remains separate from technical implementation, so the two can evolve independently.

#Next steps

• Read Create and use test projects to create a project, write YAML cases, and inspect results.
• Read Develop custom Nodes to implement business operations and share data between Nodes.
• Read Configure test projects to manage runtime resources and configure multiple execution projects.
• Read Provide application knowledge to a GUI Agent to learn how to supply relevant application knowledge and help a GUI Agent execute tasks more reliably.
• Existing tasks/flow YAML projects can keep the midscene command and gain the shared kernel and new report without rewriting files. Convert complete files and configuration before switching them to midscene-test; see Migrate to Midscene Test.