#Get started: Desktop · Midscene Test · Model API Key

These are the setup steps for Midscene on Desktop with Midscene Test and a Model API Key. To choose another combination, change the answers below or see all setup paths.

Step 1

#Install Node.js

Install Node.js ^20.19.0 || ^22.12.0 || >=24.0.0, then check that node -v works in your terminal.

The commands below use pnpm. If it is not installed yet, run npm install -g pnpm.

Step 2

#Grant desktop permissions

• macOS: Grant two separate system permissions to the terminal or IDE that launches Playground or runs your scripts (such as Terminal, iTerm2, VS Code, Cursor, or WebStorm):
  1. Accessibility (for mouse and keyboard control): Enable your terminal or IDE in System Settings > Privacy & Security > Accessibility (see nut.js macOS setup).
  2. Screen Recording (for desktop screenshots): Enable your terminal or IDE in System Settings > Privacy & Security > Screen Recording. After granting access, completely quit and relaunch the terminal or IDE before starting Playground.
• Windows: No extra setup is required for standard applications. However, Windows User Interface Privilege Isolation (UIPI) blocks non-elevated processes from injecting mouse clicks or keystrokes into windows running as Administrator (the cursor still moves to the target coordinates, but clicks and key presses are silently dropped). Prefer running the target app without Administrator privileges; if it must run elevated, launch the terminal or Node.js process running Midscene as Administrator as well (see Windows: clicks have no effect on some apps).
• Linux (desktop GUI): Install ImageMagick for screen capture support.
• Linux (headless server / CI): To run desktop automation on a headless Linux host (such as GitHub Actions), install Xvfb and ImageMagick, then enable headless mode to start a virtual display automatically:

# Install dependencies
sudo apt-get install -y xvfb x11-xserver-utils imagemagick

When using Xvfb, make sure the target application is started inside the same virtual display environment (not needed on machines with a physical display).

To control a remote Windows desktop over RDP, see Desktop environment setup.

Step 3

#Create a test project

pnpm dlx @midscene/test create my-tests \
  --platform computer \
  --package-manager pnpm
cd my-tests

The command generates the project files, installs dependencies, and generates the Node Spec. It does not run tests. For the generated files, see Project structure.

Step 4

#Configure a model

In the my-tests directory, copy .env.example to .env:

cp .env.example .env

Choose a model below. Its configuration has four values. Open Get API key to create a key with the provider, then replace ...... in MIDSCENE_MODEL_API_KEY with it.

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_MODEL_FAMILY="doubao-seed"

For other versions and usage notes, see Doubao Seed series.

To compare models, see the Model selection guide. For compatible models such as UI-TARS, see Use a model API key.

Fill in the four MIDSCENE_MODEL_* values in .env with this configuration.

Step 5

#Verify the model and finish setup

Verify model connectivity and Midscene compatibility. The command reads .env from the current directory and does not need a browser or device:

pnpm exec midscene-test model verify

If the check fails, see Verify model connectivity.

To control a display other than the primary one, set COMPUTER_DISPLAY_ID in .env. On headless Linux, also set MIDSCENE_COMPUTER_HEADLESS_LINUX=true.

Step 6

#Run the example and inspect the report

The example opens React TodoMVC in a browser on the target display and first checks that the list is empty. Do not move the mouse or type while it runs. If todos remain from an earlier run, select All and delete every todo, including completed ones.

pnpm test

The terminal prints the results and a Report: path. Open the HTML report, then select a case and a step to inspect screenshots, inputs, outputs, and errors. Reports are written to midscene_run/report/ by default.

When the case passes, your model, environment, and project are all working. For execution options, see Run tests.

Step 7

#Next steps

• Explore more examples: Showcases includes demo videos and full execution reports across platforms.

• Test your own app: Open the target application on the controlled display, then describe the steps with aiAct and aiAssert. To start or reset the app from a case, develop a custom Node.

• Write and organize cases: Write YAML cases covers inputs, variables, tags, and hooks.
• Run in CI: Run in GitHub Actions.

Want to try another platform or way to use Midscene? Back to top to change your selections.