#Get started: Android · Playground · Model API Key

These are the setup steps for Midscene on Android with Playground and a Model API Key. To choose another combination, change the answers below or see all setup paths.

Step 1

#Connect the Android device

Midscene uses adb on your computer to communicate with the target Android device over either USB or wireless adb. If adb is already installed and your device is connected, skip directly to Verify the device connection.

#Install adb

• Install adb via Android Studio or the standalone Android command-line tools.
• Run the following command to verify the installation:

adb --version

Output similar to the following confirms a working installation:

Android Debug Bridge version 1.0.41
Version 34.0.4-10411341
Installed as /usr/local/bin/adb
Running on Darwin 24.3.0 (arm64)

• Configure the ANDROID_HOME environment variable as described in Android environment variables, then check that it takes effect:

echo $ANDROID_HOME

The output should point to your local Android SDK directory, for example:

/Users/your_username/Library/Android/sdk

#Enable debugging and connect the device

The steps below use a USB connection as an example. If your device is already connected over wireless adb, you can keep using that connection.

1. In your phone's Developer options, enable USB debugging (and USB debugging (Security settings) if available, so Midscene can inject touch events).
2. Connect the phone to your computer using a USB cable that supports data transfer.
3. Keep the phone unlocked and accept the Allow USB debugging prompt on the screen.

#Verify the device connection

Run the following command to check the connection status:

adb devices -l

When the device status shows device, the connection is ready:

List of devices attached
s4ey59	device usb:34603008X product:cezanne model:M2006J device:cezan transport_id:3

If multiple devices are connected, note the device ID in the first column (such as s4ey59 above) so you can select the target device in Playground or your project configuration.

If the target device does not appear with the device status, check the following:

• Empty list: For USB, check the data cable (ensure it is not charge-only), USB connection mode, and USB debugging toggle; for wireless adb, check the network and adb connect status.
• unauthorized: Unlock the phone screen and accept the Allow USB debugging prompt on the device.
• offline: Reconnect the USB cable or wireless adb session, and restart adb if needed. Proceed only after the status shows device.

Step 2

#Configure a model

Choose a model below. Its configuration has four values. Open Get API key to create a key with the provider, then replace ...... in MIDSCENE_MODEL_API_KEY with it.

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_MODEL_FAMILY="doubao-seed"

For other versions and usage notes, see Doubao Seed series.

To compare models, see the Model selection guide. For compatible models such as UI-TARS, see Use a model API key.

Keep the configuration at hand. You will paste it into Playground in a later step.

Step 3

#Launch Playground

Make sure Node.js (^20.19.0 || ^22.12.0 || >=24.0.0) is installed and node -v works in your terminal. The command below uses pnpm. If it is not installed yet, run npm install -g pnpm.

1. Start the Android Playground CLI:

pnpm dlx @midscene/android-playground

2. In the Playground window that opens, click the gear icon in the top-right corner, paste your model configuration, and save.

3. If your browser does not open automatically, visit the local Playground URL printed in the terminal, and keep the terminal process running.
4. In the connection panel on the left, select your target ADB device and create the Agent (if multiple devices are connected, match the device ID against adb devices -l).
5. Confirm that the preview area displays your phone's current screen. If no screen appears, check the adb connection and screenshot permissions first.

Step 4

#Run an action and verify the result

On your Android device, open the browser app and complete any first-run prompts. Enter the following TodoMVC URL in the address bar:

https://todomvc.com/examples/react/dist/#/

Keep the device unlocked with the browser in the foreground.

1. In Playground, select aiAct, enter the following instruction, and run it:

Add a todo named "Try Midscene", then mark it as completed.

2. Once the completed todo item appears on the page, switch to aiAssert, enter the following assertion, and run it:

The list contains a todo named "Try Midscene", and it is marked as completed.

3. Confirm that the assertion passes and inspect the step-by-step screenshots in the results panel. At this point, device connectivity, screen capture, touch input, and model calls have all been verified together.

4. (Optional) Switch to aiQuery, enter the following query, and run it:

Return the visible todo items as {name: string, completed: boolean}[].

The result contains each todo item's name and completion status. aiQuery extracts data from the current UI without changing the page.

If execution fails, check the failed step and error message: for missing screenshots or unresponsive touch input, see the FAQ; for model request errors, see Verify model connectivity.

After verification, delete the "Try Midscene" item in TodoMVC. If you plan to run the Midscene Test Android example next, switch to the All filter and make sure the list is completely empty (including completed items), as the example template checks for an empty list before running and does not overwrite existing items.

Step 5

#Next steps

• Explore more examples: Showcases includes demo videos and full execution reports across platforms.

• Write repeatable cases: Create a Midscene Test project with the same platform and model. Cases are YAML files that you can run locally or in CI.

• Add to Vitest: Call Midscene from your Vitest tests with the same platform and model.

• Learn how Midscene works: The Basics explains the vision-based approach and when to use aiAct, instant actions, and insight APIs.

Want to try another platform or way to use Midscene? Back to top to change your selections.