#Get started: iOS · Playground · Model API Key

These are the setup steps for Midscene on iOS with Playground and a Model API Key. To choose another combination, change the answers below or see all setup paths.

Step 1

#Connect the iOS device through WDA

Midscene communicates with iOS physical devices and simulators through a running WebDriverAgent (WDA) service. If WDA is already running and reachable from your computer, skip directly to Verify the WDA service status.

#Build and start WebDriverAgent

Before you begin, prepare the following iOS development environment:

• macOS (required for iOS development and device debugging)
• Xcode and Xcode Command Line Tools
• An iOS Simulator or physical iOS device

Follow the official Appium documentation to build and start WebDriverAgent (WDA):

• Simulator setup: Run Prebuilt WDA
• Physical device setup: Real Device Configuration

#Configure a physical device and forward the port

iOS Simulators expose WDA directly on localhost and do not require port forwarding—if you are using a simulator or a remote WDA service, skip to Verify the WDA service status.

When using a physical iOS device:

1. Connect the device to your Mac via USB and tap Trust This Computer when prompted on the device.
2. Enable Developer Mode (Settings > Privacy & Security > Developer Mode) and UI Automation (Settings > Developer > Enable UI Automation) on the device.
3. After signing and starting WDA on the device, use iproxy (installable via brew install libimobiledevice) to forward the WDA port (8100 by default) from the device to your Mac:

iproxy 8100 8100 YOUR_DEVICE_ID

Replace YOUR_DEVICE_ID with your device's UDID, and keep both WDA and iproxy running. When connecting multiple devices or using custom ports, forward each device to a distinct local port and use that port in Playground or your project configuration.

#Verify the WDA service status

Open the WDA /status endpoint in your browser or terminal from the computer running Midscene:

• Default local URL: http://localhost:8100/status (if using a custom port, remote host, or gateway, use the actual service address)
• Expected JSON response:

{
  "value": {
    "build": {
      "version": "10.1.1",
      "time": "Sep 24 2025 18:56:41",
      "productBundleIdentifier": "com.facebook.WebDriverAgentRunner"
    },
    "os": {
      "testmanagerdVersion": 65535,
      "name": "iOS",
      "sdkVersion": "26.0",
      "version": "26.0"
    },
    "device": "iphone",
    "ios": {
      "ip": "10.91.115.63"
    },
    "message": "WebDriverAgent is ready to accept commands",
    "state": "success",
    "ready": true
  },
  "sessionId": "BCAD9603-F714-447C-A9E6-07D58267966B"
}

A JSON response containing "ready": true confirms that WDA is ready to accept commands. Note the WDA host and port (or gateway base URL) so you can use the same connection settings in Playground and your project.

Step 2

#Configure a model

Choose a model below. Its configuration has four values. Open Get API key to create a key with the provider, then replace ...... in MIDSCENE_MODEL_API_KEY with it.

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_MODEL_FAMILY="doubao-seed"

For other versions and usage notes, see Doubao Seed series.

To compare models, see the Model selection guide. For compatible models such as UI-TARS, see Use a model API key.

Keep the configuration at hand. You will paste it into Playground in a later step.

Step 3

#Launch Playground

Make sure Node.js (^20.19.0 || ^22.12.0 || >=24.0.0) is installed and node -v works in your terminal. The command below uses pnpm. If it is not installed yet, run npm install -g pnpm.

1. Start the iOS Playground CLI:

pnpm dlx @midscene/ios-playground

2. In the Playground window that opens, click the gear icon in the top-right corner, paste your model configuration, and save.
3. If your browser does not open automatically, visit the local Playground URL printed in the terminal, and keep the terminal process running.
4. If the connection panel appears, enter your WDA host and port (localhost and 8100 by default) and create the Agent (if WDA is already running on the default local port, Playground may connect automatically; when connecting through a gateway, enter the full WebDriverAgent base URL instead of host / port).
5. Confirm that the preview area displays your device or simulator screen. If no screen appears, check your WDA service and port forwarding first.

Step 4

#Run an action and verify the result

On your iOS device or simulator, open Safari and complete any first-run prompts. Enter the following TodoMVC URL in the address bar:

https://todomvc.com/examples/react/dist/#/

Keep the device unlocked with Safari in the foreground.

1. In Playground, select aiAct, enter the following instruction, and run it:

Add a todo named "Try Midscene", then mark it as completed.

2. Once the completed todo item appears on the page, switch to aiAssert, enter the following assertion, and run it:

The list contains a todo named "Try Midscene", and it is marked as completed.

3. Confirm that the assertion passes and inspect the step-by-step screenshots in the results panel. At this point, WDA connectivity, screen capture, touch input, and model calls have all been verified together.

4. (Optional) Switch to aiQuery, enter the following query, and run it:

Return the visible todo items as {name: string, completed: boolean}[].

The result contains each todo item's name and completion status. aiQuery extracts data from the current UI without changing the page.

If execution fails, check the failed step and error message: for missing screenshots or unresponsive touch input, see the FAQ; for model request errors, see Verify model connectivity.

After verification, delete the "Try Midscene" item in TodoMVC. If you plan to run the Midscene Test iOS example next, switch to the All filter and make sure the list is completely empty (including completed items), as the example template checks for an empty list before running and does not overwrite existing items.

Step 5

#Next steps

• Explore more examples: Showcases includes demo videos and full execution reports across platforms.

• Write repeatable cases: Create a Midscene Test project with the same platform and model. Cases are YAML files that you can run locally or in CI.

• Add to Vitest: Call Midscene from your Vitest tests with the same platform and model.

• Learn how Midscene works: The Basics explains the vision-based approach and when to use aiAct, instant actions, and insight APIs.

Want to try another platform or way to use Midscene? Back to top to change your selections.