• English
  • Migrate to Midscene Test

    Legacy YAML and native Midscene Test share the same underlying execution kernel and report page, but they have separate user-facing entries. The midscene command owns legacy tasks/flow files, YAML batch configuration, and legacy CLI options. The midscene-test command owns native cases/steps files, TypeScript or JavaScript project configuration, lifecycle hooks, and Nodes.

    What “legacy YAML” means

    In this guide, “legacy YAML” specifically means the YAML automation protocol from @midscene/cli, not every YAML file. It typically has these characteristics: the workflow has top-level tasks and sequences actions through tasks[].flow; platform, Agent, and output configuration is written directly in workflow YAML; batch execution uses YAML configuration with fields such as files, setup, concurrent, and retry; and the entry command is midscene. A file with top-level cases/steps and lifecycle hooks that runs through midscene-test is native Test YAML, not legacy YAML.

    The legacy path receives compatibility maintenance and necessary fixes only. It will not receive new syntax, capabilities, configuration fields, or CLI options. Use native Midscene Test for new projects, new cases, and future capability development; existing legacy projects can continue to run and migrate gradually.

    There are two migration paths:

    • Upgrade in place: keep legacy cases, batch configuration, directory layout, and the midscene command unchanged. This adopts the shared execution kernel and new report without changing the input contract.
    • Native migration: rewrite complete files as Test Documents, Cases, and Steps; replace batch YAML with project configuration; then switch those files to midscene-test.

    The complete legacy field reference remains available in YAML script runner and Workflow in YAML format for maintaining and migrating existing projects.

    Path 1: upgrade legacy execution in place

    If the project already uses @midscene/cli, upgrade it and keep the existing command. midscene preserves the legacy input contract while executing through the shared kernel and generating the new report page:

    pnpm exec midscene ./flow.yaml
    pnpm exec midscene --config ./batch.yaml

    Platform configuration, agent, environment interpolation, task continuation, whole-file retries, output files, and summary retain their legacy semantics. Reports use the new Test page; existing HTML files need no conversion, so rerun to regenerate them. Legacy agent.generateReport: false remains respected.

    Legacy flags such as --files, --setup, --concurrent, --retry, --continue-on-error, --summary, --headed, --keep-window, --share-browser-context, --dotenv-override, --dotenv-debug, --<target>.<field>, and --no-<target>.<field> remain accepted by midscene. They are not midscene-test options and do not appear in its help.

    Command and file boundaries
    • midscene accepts only legacy tasks/flow workflow files and YAML batch configuration.
    • midscene-test accepts only native cases/steps workflow files and TypeScript or JavaScript project configuration.
    • A YAML file cannot mix legacy tasks with native cases or lifecycle hooks.
    • One command invocation cannot select both formats. During gradual migration, keep them in separate directories or file selections and run two commands.
    • --project, --result-dir, and midscene-test nodes apply only to native Test projects. Legacy scheduling and platform flags remain on midscene.

    Both commands validate this boundary before creating browser, device, or Agent resources. They report the wrong-format files and point to the matching command instead of silently rerouting them.

    Path 2: migrate to a native Test project

    Use the following order so each step can be run and compared independently:

    1. Run the existing project with midscene and save its exit status, summary, and new report as a baseline.
    2. Use pnpm dlx @midscene/test create to create a Test project for the target platform. Start with the generated midscene.config.ts and setup.
    3. Map the legacy batch configuration to an Execution Project. Move platform, Agent, and resource creation into setup.
    4. Convert one complete tasks/flow file to cases/steps, move it into the native file selection, and run it with midscene-test. Keep unconverted files on midscene.
    5. Run pnpm exec midscene-test nodes --project <name> to generate the current Project's Node Spec, then validate action names, inputs, and string shorthand against it.
    6. After all files are converted, remove the legacy batch configuration, legacy command, and legacy-only command-line options.

    Case structure mapping

    Legacy YAML:

    web:
      url: https://example.com
    
    tasks:
      - name: Search for a product
        continueOnError: true
        flow:
          - ai: Search for a mug
          - aiAssert: The results contain a mug
            errorMessage: The expected product was not found

    Native Test:

    beforeEach:
      - gotoUrl: https://example.com
    
    cases:
      - name: Search for a product
        steps:
          - aiAct: Search for a mug
          - aiAssert:
              prompt: The results contain a mug
              message: The expected product was not found
    Legacy fieldNative Test field or mechanismNotes
    taskscasesOne legacy task becomes one Case.
    tasks[].namecases[].namePreserve the name.
    tasks[].flowcases[].stepsOne flow item becomes one Step.
    tasks[].continueOnError: trueNative Case continuation defaultmidscene preserves legacy task continuation. Native Test continues to the next Case by default.
    Omitted continueOnErrorLegacy failure policymidscene stops the legacy file. There is no native onFailure option: put shared prerequisites in beforeAll or dependent steps in one Case.
    Initial page or app in a top-level platform fieldsetup plus lifecycle NodesCreate resources in setup. Use gotoUrl, launch, or another Node in beforeEach when every Case must return to an initial state.
    No legacy equivalentbeforeAll, beforeEach, afterEach, afterAllNative file and Case lifecycle hooks.
    No legacy equivalentcases[].tagsSelect Cases per Execution Project.

    Keeping old files unchanged preserves their failure policy. Rewriting them as native Cases adopts native semantics; it is not an exact scheduling equivalence. Step $: { continue-on-error: true } continues later Steps in that phase, not later legacy tasks, and does not make a failed Case pass.

    Flow action mapping

    The legacy runtime keeps accepting the fields in the left column and adapts them internally to the shared kernel. That internal adaptation is not a native Test input contract. When migrating a file, write the native form in the right column and use the Nodes registered in the project's midscene-node-reference.md as the source of truth.

    Legacy flow fieldNative Test Node / inputMigration notes
    ai, aiAction, aiAct, instructionaiAct.promptUse aiAct; move remaining settings into options.
    aiAssertaiAssert.promptRename errorMessage to message; move extraction settings into options.
    aiQuery, aiNumber, aiString, aiBoolean, aiAsk, aiLocateSame-name Node with promptPlace images and extraction settings under prompt or options as described by the Node Spec.
    aiWaitForaiWaitFor.promptLegacy timeout maps to options.timeoutMs in the Node input.
    aiTapaiTap.promptNormalize legacy locate, images, and locate settings into prompt and options.
    aiScrollaiScrollUse the locate description as prompt and scrolling settings as options.
    aiInputaiInput.value and aiInput.promptMove the entered value to value and the locate description to prompt.
    aiKeyboardPressaiKeyboardPress.keyName and optional promptUse the inputs described by the project Node Spec.
    sleepsleep.msFor example, change sleep: 1000 to sleep: { ms: 1000 }. New cases can also use the common wait Node.
    javascriptjavascript.scriptIts return value is recorded in the Step result.
    recordToReport, logScreenshotrecordToReportMap the title to title and content to options.content.
    runGherkinScenariorunGherkinScenario.scenarioLegacy execution disables caching for this action.
    runAdbShellrunAdbShellMap the command to command; preserve legacy timeout as Node input.
    FinalizeLegacy-only planning markerAccepted by the legacy runtime; omit it when writing native Cases. It is not a public Node.
    Other platform and custom actionsA registered same-name Node, or the generic action NodeRun midscene-test nodes first and rewrite inputs according to the target platform's Node Spec.
    Flow item nameStructured Node resultmidscene still writes named legacy outputs. Native Test has no named cross-Step result expression; share data through custom Nodes and the project context.

    Concrete example: migrate one order-submission Case

    The following legacy case fills in a recipient, waits for the submit button, submits the order, and records the result. It uses task failure policy, an action alias, locate input, action timeout, an assertion message, a fixed wait, and report recording:

    web:
      url: https://shop.example.com/checkout
    
    tasks:
      - name: Submit an order
        continueOnError: true
        flow:
          - aiInput: Alice
            locate: Recipient name input
          - aiWaitFor: The submit order button is enabled
            timeout: 10000
          - ai: Click the submit order button
          - aiAssert: The page shows that the order was submitted
            errorMessage: No success message appeared after submission
          - sleep: 500
          - recordToReport: Order submission result
            content: The checkout flow completed

    The same Case in native Test YAML is:

    beforeEach:
      - gotoUrl: https://shop.example.com/checkout
    
    cases:
      - name: Submit an order
        steps:
          - aiInput:
              prompt: Recipient name input
              value: Alice
          - aiWaitFor:
              prompt: The submit order button is enabled
              options:
                timeoutMs: 10000
          - aiAct: Click the submit order button
          - aiAssert:
              prompt: The page shows that the order was submitted
              message: No success message appeared after submission
          - wait:
              duration: 500
              unit: ms
          - recordToReport:
              title: Order submission result
              options:
                content: The checkout flow completed

    The conversion has several parts:

    1. web.url no longer sits beside case content. Project setup creates the browser. Because every Case must start on the checkout page, gotoUrl in beforeEach owns the URL.
    2. The task becomes a Case. Native Test already continues after a failed Case, so this example needs no continuation option.
    3. The aiInput value and locate description move from legacy sibling fields to the Node's value and prompt.
    4. aiWaitFor.timeout limits that wait operation, so it becomes options.timeoutMs. To limit the complete Step instead, use $: { timeout: 10000 }.
    5. ai becomes aiAct, and aiAssert.errorMessage becomes aiAssert.message.
    6. The fixed delay uses the native common wait Node. The report title and body move to recordToReport.title and options.content.

    This example changes the input structure without changing business execution order. During a real migration, run pnpm exec midscene-test nodes --project <name> to generate the current Project's midscene-node-reference.md. Use this Node Spec—not examples from another platform—to confirm whether gotoUrl, aiInput, aiWaitFor, and the other referenced Nodes are available and how to provide their inputs.

    Script and Agent configuration mapping

    midscene continues to read top-level legacy YAML configuration. In a native migration, move these fields out of Case YAML and into midscene.config.ts, setup, or Nodes:

    Legacy script fieldNative Test configuration or mechanismNotes
    targetWeb Project setupThis field is deprecated. Do not carry it into new configuration; configure the actual Playwright, Puppeteer, or bridge runtime.
    page, browser, webWeb Project setup plus Web NodesPut browser launch, CDP, viewport, Cookie, headers, and connection settings in setup. Put repeatable navigation and Cookie operations in lifecycle Nodes.
    androidAndroid Project setup plus Android NodesUse device ID, screenshot, and input settings to create the Device and Agent. Put launch and other case actions in beforeEach or steps.
    iosiOS Project setup plus iOS NodesPut WDA connection and Device/Agent creation in setup. Use registered Nodes for application launch and interaction.
    harmonyHarmonyOS Project setup plus HarmonyOS NodesPut HDC connection, device settings, and application mappings in setup. Use registered Nodes for application and shell actions.
    computerComputer Project setup plus Computer NodesPut display and device settings in setup. Use registered Nodes for UI actions.
    interfaceCustom Project setup plus createMidsceneNodes()Import and create the custom Interface, Device, and Agent in setup, then expose the Agent to Nodes.
    agent.generateReportLegacy report settingOld YAML can disable its report. Native Test always generates a combined report.
    agent.reportFileName, agent.testIdLegacy Agent artifact settingsOpen the report using the path printed by the CLI or the report link in the summary. testId remains deprecated.
    agent.groupName, agent.groupDescriptionProject and Case names plus report structureThere is no same-name native field. Express grouping through Project names, Case names, or tags.
    agent.replanningCycleLimit, agent.aiContexts, agent.aiActContext, agent.aiActionContext, agent.cache, agent.screenshotShrinkFactorAgent options in setupContinue passing the relevant options when creating the Agent instead of placing them in Case YAML.
    agent.outputFormat, agent.persistExecutionDump, agent.autoPrintReportMsgTest reporting or Agent options in setupPrefer the combined Test report. Preserve an Agent option in setup only when its Agent-specific behavior is still required.
    config.output, or output under a platform fieldTest structured resultsmidscene still writes legacy output files. Native Test uses --result-dir, summary.json, and the report. Return business data from custom Nodes or write it to external storage.
    config.unstableLogContentTest report and structured resultsThere is no same-name native field. Migrate consumers away from the legacy experimental log-content file.
    tasksNative Workflow DocumentMap it to cases and lifecycle hooks as described above.

    See Configure the runtime environment for complete setup and platform Node registration examples.

    Batch configuration mapping

    Legacy batch.yaml:

    files:
      - flows/cases/**/*.yaml
    setup: flows/setup.yaml
    concurrent: 2
    retry: 1
    continueOnError: false
    summary: result.json
    headed: true

    Native Test moves execution settings into midscene.config.ts:

    import { defineTestProject } from '@midscene/test/config';
    import { nodes, setup } from './test-runtime';
    
    export default defineTestProject({
      nodes,
      projects: [
        {
          name: 'default',
          setup,
          files: {
            include: ['flows/cases/**/*.yaml'],
          },
          retry: 1,
        },
      ],
      test: {
        bail: 1,
      },
      output: {
        reportDir: './midscene_run/report',
      },
    });
    Legacy batch settingNative Test configuration or mechanismNotes
    filesprojects[].files.includemidscene preserves old order and repeated invocations. Native selection is sorted and deduplicated, with no files.order switch.
    setupLegacy prerequisite file; native setup / beforeAllThe legacy host runs the old setup file first. In native Test, create resources in setup and express file prerequisites as lifecycle Steps; there is no setupFile option.
    concurrentLegacy file scheduler; native test.maxConcurrencymidscene preserves file concurrency. Native concurrency is between independent Projects, not files inside one Project.
    retryLegacy whole-file retry; native projects[].retryNative retries only a failed Case. Keep legacy files on midscene when whole-file replay is required; there is no retryScope switch.
    continueOnError: falsetest.bail: 1Stop scheduling new files or Cases when the failure threshold is reached. Concurrent work that has already started still completes cleanup.
    continueOnError: truetest.bail: 0Disable global failure-threshold stopping. midscene separately preserves old task continuation; native Cases continue by default.
    summaryTest summary.json, report, and programmatic APIsmidscene still writes the legacy summary. Native Test has no same-name setting; migrate consumers and use --result-dir to select the structured result directory.
    shareBrowserContextLegacy resource host; native Project setupThe legacy host creates independent Pages/Agents over the shared browser context. Native Test shares resources within a Project through setup and uses independent resources between concurrent Projects.
    headedBrowser or device setup launch settingsFor example, Playwright uses chromium.launch({ headless: false }).
    keepWindowsetup cleanup policyUse only for local debugging; the project must decide whether to delay or skip browser teardown.
    dotenvOverride, dotenvDebugProject environment-loading policyNative Test reads process.env. If you rely on an .env file, load it explicitly in the config or launch script and configure precedence and debugging there.
    web, android, ios, harmony, computer, interfaceProject setup, variables, and registered platform NodesMove device connection and Agent creation into setup; keep serializable environment differences in variables.

    This native example selects files and adopts Case retries; it does not preserve the old setup-file or concurrent-file scheduling. For parallel native execution, split files across independent Projects and use test.maxConcurrency. Legacy file concurrency and document setup stay behind the midscene entry and are not public Test configuration.

    The midscene entry preserves legacy output naming: explicit flow name values, numeric keys for unnamed extracted results, and the existing overwrite behavior for duplicate names.

    Validate the migration

    Run the legacy version with midscene and the converted version with midscene-test, then compare at least:

    • Process exit status and continuation after a failure.
    • Whether retries apply to one Case or the complete file.
    • Environment variables, platform connection settings, and initial page state.
    • AI action inputs, timeouts, and assertion failure messages.
    • Business output consumers and Step results in the new Test report.
    • Page, Agent, device, and test-data isolation during concurrent execution.
    • Lifecycle hooks and resource cleanup after failures, timeouts, and Ctrl+C interruption.

    After migration, write new cases with cases/steps. Keep tasks/flow only in files that have not yet been migrated so the compatibility format does not expand into new code.