# Midscene - Vision-Driven UI Automation > AI-powered, vision-driven UI automation for every platform. ## Other - [BDD-style scripts with Gherkin](/advanced/bdd-style-scripts-with-gherkin.md): This feature is available starting in Midscene 1.10.BDD-related capabilities are still in Beta. The API may change in future releases. Gherkin is a plain-text syntax for describing behavior examples. It is commonly used in BDD (Behavior-Driven Development) workflows. A BDD scenario usually contains three kinds of steps: Given describes the precondition, When describes the user action, and Then describes the expected result. For example, when you need to test adding a todo item, you can write the case as the following Gherkin script. Longer GUI cases can also be written in one Scenario. You can organize the steps by business phase, and each phase can use a group of Given, When, and Then. This style is useful when one business scenario naturally contains several UI phases, such as cart review, shipping selection, payment, and final confirmation. In AI-driven GUI automation, Gherkin is a natural fit for describing operation cases. It keeps the readability of natural language while providing a stable step structure. This structure also helps models generate more consistent cases. See the official Cucumber Gherkin reference for the full Gherkin syntax. - [Midscene AndroidWorld Benchmark Report](/android-world-benchmark-report.md): This is Midscene's test report for the AndroidWorld benchmark. In this run, Midscene achieved Pass@1 93.10%, Pass@2 95.69%, and Pass@3 97.41%. - [Automate with scripts in YAML](/automate-with-scripts-in-yaml.md): In most cases, developers write automation scripts just to perform some smoke tests, like checking for the appearance of certain content or verifying that a key user path is accessible. In such situations, maintaining a large test project is unnecessary. Midscene offers a way to perform automation using .yaml files, which helps you focus on the script itself rather than the testing framework. This allows any team member to write automation scripts without needing to learn any API. Here is an example. By reading its content, you should be able to understand how it works. - [Awesome Midscene](/awesome-midscene.md): A curated list of community projects that extend Midscene.js capabilities across different platforms and programming languages. - [Introducing Instant Actions and Deep Think](/blog-introducing-instant-actions-and-deep-think.md): From Midscene v0.14.0, we have introduced two new features: Instant Actions and Deep Think. - [Bridge mode by Chrome extension](/bridge-mode.md): The bridge mode in the Midscene Chrome extension is a tool that allows you to use local scripts to control the desktop version of Chrome. Your scripts can connect to either a new tab or the currently active tab. Using the desktop version of Chrome allows you to reuse all cookies, plugins, page status, and everything else you want. You can work with automation scripts to complete your tasks. This mode is commonly referred to as 'man-in-the-loop' in the context of automation. - [Caching AI Planning and DOM Localization](/caching.md): Midscene supports caching two kinds of data: AI Planning steps and matched element localization information. The former can be used across automation workflows to reduce AI model calls and improve execution efficiency; for web automation, the DOM localization part (XPath) in the latter can significantly reduce repeated locating overhead, but it is currently web-only and has certain limitations. Effect With caching hit, time cost is significantly reduced. For example, in the following case, execution time was reduced from 51 seconds to 28 seconds. before after - [Changelog](/changelog.md) - [](/common/get-cdp-url.md) - [](/common/prepare-android.md) - [](/common/prepare-ios.md) - [](/common/quick-model-config.md) - [](/common/setup-env.md) - [](/common/start-experience.md) - [Consume Report Files](/consume-report-file.md): Midscene HTML report files capture the full execution history of a single Agent, making them useful for replay and debugging. Starting in v1.7.0, you can extract raw screenshots and JSON data from a report file, or convert the report into Markdown so other tools can consume it. - [Data privacy](/data-privacy.md): ⁠Midscene.js is an open-source project (GitHub: Midscene) under the MIT license. You can see all the codes in the public repository. When using Midscene.js, your page data (including the screenshot) is sent directly to the AI model provider you choose. No third-party platform will have access to this data. All you need to be concerned about is the data privacy policy of the model provider. If you prefer building Midscene.js and its Chrome Extension in your own environment instead of using the published versions, you can refer to the Contributing Guide to find building instructions. - [FAQ](/faq.md) - [Integrate with any interface](/integrate-with-any-interface.md): You can use Midscene Agent to control any interface—such as IoT devices, in-house apps, and in-vehicle displays—by implementing a UI operation class that conforms to AbstractInterface. After implementing the UI operation class, you get the full capabilities of Midscene Agent: the TypeScript GUI Automation Agent SDK, supporting integration with any interfacethe playground for debuggingcontrolling the interface with YAML scriptsSkills support through CLI commands - [Integrate with Playwright](/integrate-with-playwright.md): Playwright.js is an open-source automation library developed by Microsoft, mainly used for end-to-end testing and web scraping of web applications. There are two ways to integrate with Playwright: Directly integrate and call the Midscene Agent via script, suitable for quick prototyping, data scraping, and automation scripts.Integrate Midscene into Playwright test cases, suitable for UI testing scenarios. - [Integrate with Puppeteer](/integrate-with-puppeteer.md): Puppeteer is a Node.js library which provides a high-level API to control Chrome or Firefox over the DevTools Protocol or WebDriver BiDi. Puppeteer runs in the headless (no visible UI) by default but can be configured to run in a visible ("headful") browser. - [Midscene.js - Vision-Driven UI Testing & Automation](/introduction.md): Midscene is an open-source SDK for vision-driven UI testing and automation. You describe each step in natural language, and Midscene drives a multimodal model to plan and operate the interface for you — across web, mobile, desktop, and even surfaces. - [LLMs.txt documentation](/llm-txt.md): How to get tools like Cursor, Windstatic, GitHub Copilot, ChatGPT, and Claude to understand Midscene.js. We support LLMs.txt files for making the Midscene.js documentation available to large language models. - [MCP integration has been retired](/mcp.md): Midscene no longer ships MCP servers. Use Skills to let AI coding agents drive Midscene through the platform CLIs. If you still need the MCP server packages, pin Midscene to 1.9.8. This is the final version that includes MCP support. Remove any agent configuration that references the retired MCP packages: @midscene/web-bridge-mcp@midscene/android-mcp@midscene/ios-mcp@midscene/harmony-mcp@midscene/computer-mcp@midscene/mcp If your previous MCP configuration set MIDSCENE_MCP_CHROME_PATH, move that value to MIDSCENE_CHROME_PATH for Skills and CLI usage. The old variable is still accepted as a temporary migration alias. For code-level automation, use the JavaScript SDK, YAML runner, or the platform CLIs listed in Skills. - [Midscene MobileWorld Benchmark Report](/mobile-world-benchmark-report.md): This report presents Midscene results on the MobileWorld benchmark. The evaluation covers 117 tasks, with a Pass@1 score of 78.63% (92/117). - [Configure Your Model](/model-common-config.md) - [All configuration options](/model-config.md): Midscene reads all model configuration from operating-system environment variables. Midscene integrates the OpenAI SDK by default for AI calls. The SDK defines the parameter shape, and most model providers (or deployment tools) offer compatible endpoints. This doc focuses on Midscene model configuration. For how we choose models, see Model strategy. For quick recipes for popular models, see Configure your model. - [Model Strategy](/model-strategy.md): This guide focuses on how we choose models in Midscene. If you need configuration instructions, head to Configure your model. - [Android](/platforms/android.md): Midscene connects to Android devices through adb to automate apps and system interfaces. This guide covers device connection, model configuration, Playground, and JavaScript SDK integration with @midscene/android. - [Desktop](/platforms/desktop.md): Midscene uses native keyboard and mouse controls to automate desktop applications on Windows, macOS, and Linux. This guide covers platform setup, model configuration, Playground, and JavaScript SDK integration with @midscene/computer. - [HarmonyOS](/platforms/harmonyos.md): Midscene connects to HarmonyOS NEXT devices through HarmonyOS Device Connector (HDC) to automate apps and system interfaces. This guide covers device connection, model configuration, Playground, and JavaScript SDK integration with @midscene/harmony. - [More platforms](/platforms/index.md): Midscene supports mobile devices, HarmonyOS devices, and desktop applications in addition to Web browsers. Choose a platform based on the target interface and its connection method. Midscene uses a multimodal visual model to understand screenshots, so automation works with the rendered interface instead of depending on the underlying UI structure. The same approach applies to native apps and cross-platform technology stacks. - [iOS](/platforms/ios.md): Midscene connects to iOS devices through WebDriverAgent to automate apps and system interfaces. This guide covers WebDriverAgent setup, model configuration, Playground, and JavaScript SDK integration with @midscene/ios. - [Quick experience by Chrome extension](/quick-experience.md): Midscene.js provides a Chrome extension. By using it, you can quickly experience the main features of Midscene on any webpage, without needing to set up a code project. ⁠The extension shares the same code as the npm @midscene/web packages, so you can think of it as a playground or a way to debug with Midscene. Prompt : Sign up for Github, you need to pass the form validation, but don't actually click. View the full report of this task: report.html - [Quick Start](/quick-start.md): Get a feel for Midscene in a few minutes. There are two ways to start, and you can do either one first: No code — install the Chrome extension and try Midscene on any web page, without setting up a project.Write a script — use the JavaScript SDK to build a repeatable automation. Both need a multimodal model configured, so let's set that up first. - [API reference](/reference/index.md): Use this page to look up shared Agent APIs and platform-specific constructors, options, actions, and helper methods. This page documents API contracts. For installation, end-to-end workflows, and troubleshooting, follow the linked guides. Platform Agents inherit the Shared Agent APIs unless a platform section documents a difference. This page retains a small number of complete examples to show how related APIs work together. For end-to-end integration and best practices, follow the guide links in each platform section. - [](/showcases-android.md) - [](/showcases-computer.md) - [](/showcases-harmony.md) - [](/showcases-ios.md) - [](/showcases-web.md) - [Showcases](/showcases.md) - [Control any platform with Skills](/skills.md): Agent Skills are a format for extending AI coding agents with specialized capabilities. Midscene provides Agent Skills that let AI coding tools (like Claude Code, Cline, etc.) drive UI automation through CLI commands. Skills work by running CLI commands directly in the terminal. The AI agent acts as the brain: it takes screenshots, analyzes the UI, and decides which actions to perform next. - [Use JavaScript to optimize the AI automation code](/use-javascript-to-optimize-ai-automation-code.md): Many developers love using aiAct or ai to accomplish automation tasks, even packing long, complex logic into a single natural-language instruction. It feels “smart,” but in practice you may run into unstable reproducibility and slower performance. This article shares an approach for writing automation scripts with JavaScript and structured APIs. - [YAML script runner](/yaml-script-runner.md): Midscene defines a YAML-based scripting format so you can quickly author automation scripts, then run them from the command line without extra setup. For more details on YAML scripts, see Automate with scripts in YAML. For example, you can write a YAML script like this: Run it with one command: The CLI prints execution progress and generates a visual report when it finishes, while keeping setup simple.