> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt.

# Android environment setup

Midscene connects to and controls Android devices over `adb`, supporting UI automation across native apps, hybrid apps, and system interfaces.

This guide prepares the Android environment: install adb and connect the device. Playground, Midscene Test, and the JavaScript SDK all use this connection.

## Connect the device through adb {#connect-device}

Midscene uses `adb` on your computer to communicate with the target Android device over either USB or wireless `adb`. If `adb` is already installed and your device is connected, skip directly to [Verify the device connection](#verify-device-connection).

### Install adb

- Install `adb` via [Android Studio](https://developer.android.com/studio) or the standalone [Android command-line tools](https://developer.android.com/studio#command-line-tools-only).
- Run the following command to verify the installation:

```bash
adb --version
```

Output similar to the following confirms a working installation:

```log
Android Debug Bridge version 1.0.41
Version 34.0.4-10411341
Installed as /usr/local/bin/adb
Running on Darwin 24.3.0 (arm64)
```

- Configure the `ANDROID_HOME` environment variable as described in [Android environment variables](https://developer.android.com/tools/variables), then check that it takes effect:

```bash
echo $ANDROID_HOME
```

The output should point to your local Android SDK directory, for example:

```log
/Users/your_username/Library/Android/sdk
```

### Enable debugging and connect the device

The steps below use a USB connection as an example. If your device is already connected over wireless `adb`, you can keep using that connection.

1. In your phone's **Developer options**, enable **USB debugging** (and **USB debugging (Security settings)** if available, so Midscene can inject touch events).
2. Connect the phone to your computer using a USB cable that supports data transfer.
3. Keep the phone unlocked and accept the **Allow USB debugging** prompt on the screen.

![android usb debug](/images/android-usb-debug-en.png)

### Verify the device connection {#verify-device-connection}

Run the following command to check the connection status:

```bash
adb devices -l
```

When the device status shows `device`, the connection is ready:

```log
List of devices attached
s4ey59	device usb:34603008X product:cezanne model:M2006J device:cezan transport_id:3
```

If multiple devices are connected, note the device ID in the first column (such as `s4ey59` above) so you can select the target device in Playground or your project configuration.

If the target device does not appear with the `device` status, check the following:

- **Empty list**: For USB, check the data cable (ensure it is not charge-only), USB connection mode, and **USB debugging** toggle; for wireless `adb`, check the network and `adb connect` status.
- **`unauthorized`**: Unlock the phone screen and accept the **Allow USB debugging** prompt on the device.
- **`offline`**: Reconnect the USB cable or wireless `adb` session, and restart `adb` if needed. Proceed only after the status shows `device`.

## Verify {#verify}

When `adb devices -l` shows the target device with the `device` status, verify the environment in Playground. Open [Get started](/get-started.md?platform=android\&scene=playground), where Android and Playground are already selected, choose how to connect a model, and follow the steps to run one `aiAct` action and one `aiAssert` assertion. A passing assertion confirms that the adb connection, screen capture, touch input, and model calls work together.

## FAQ {#faq}

### The device is missing, unauthorized, or offline

Run `adb devices -l` and follow the troubleshooting steps in [Verify the device connection](#verify-device-connection) to check the USB cable, debugging authorization, and connection status. Make sure the target device status is `device` before investigating screenshot or input issues.

### Why can't I control the device even though it is connected (`INJECT_EVENTS` error)?

If tapping or swiping fails with a permission error like the following:

```text
Error:
Exception occurred while executing 'tap':
java.lang.SecurityException: Injecting input events requires the caller (or the source of the instrumentation, if any) to have the INJECT_EVENTS permission.
```

Some Android OEM ROMs (such as Xiaomi / MIUI / HyperOS, vivo, and OPPO) require an additional toggle in **Developer options** to allow simulated input. Make sure both of the following options are enabled:

1. **USB debugging**
2. **USB debugging (Security settings)** or **Disable permission monitoring** (if present)

![android usb debug](/images/android-usb-debug-en.png)

### Text input is cleared or disappears after typing

If text is entered properly but disappears when the soft keyboard closes, the input field may be clearing its value in response to the `ESC` key event sent to dismiss the keyboard. You can customize the keyboard dismissal strategy; see the [Android JavaScript SDK guide](/sdk/android.md#text-input-is-cleared-or-disappears-after-typing) for options and examples.

### English text is altered by the Android input method

Compare the input value recorded in the execution report with the text actually entered into the field. If the value in the report is correct but characters in the input field are reordered, missing, or intercepted by Pinyin composition, check the active input method and autocorrect settings on the device. See the [Android JavaScript SDK guide](/sdk/android.md#english-text-is-altered-by-the-android-input-method) for details on input channels.

### Screenshots of secure pages (such as password inputs) are black

First switch to a regular page and check whether screenshots work normally. If only password screens or sensitive pages appear black, the app likely enables system screenshot protection (`FLAG_SECURE`) rather than `adb` being disconnected. See the [Android JavaScript SDK guide](/sdk/android.md#screenshots-of-secure-pages-such-as-password-inputs-are-black) for screenshot mechanisms and limitations.

### Why does scrcpy fall back to ADB screenshots?

To prevent the AI model from acting on stale frames, Midscene strictly validates scrcpy frame timestamps: any frame that cannot be proven to have been captured after the most recent completed action—or that exceeds the maximum frame age limit—is rejected. If the active scrcpy stream fails to deliver a fresh frame in time, Midscene automatically restarts scrcpy once, and falls back to `adb` screenshots only if the restarted stream also fails.

This commonly occurs when an Android hardware encoder pauses frame output on a completely static screen, or during stream startup and transient USB transport hiccups. Note that a frame-freshness warning alone does not indicate insufficient bandwidth or an excessive video bitrate—do not adjust the video bitrate solely in response to this warning.

### How do I configure the scrcpy video bitrate?

See the [Android JavaScript SDK guide](/sdk/android.md#how-do-i-configure-the-scrcpy-video-bitrate).

### How do I access Android Playground remotely from another machine on the LAN?

By default, both the Playground server and its built-in scrcpy streaming server bind only to `127.0.0.1`. To access Android Playground from another computer on your network, explicitly bind both servers to a reachable interface via environment variables (for example, on macOS or Linux):

```bash
MIDSCENE_PLAYGROUND_HOST=0.0.0.0 MIDSCENE_SCRCPY_HOST=0.0.0.0 pnpm dlx @midscene/android-playground
```

Then open Playground in a browser using the host machine's LAN IP address (or hostname) and the Playground port printed at startup (note that `0.0.0.0` is a wildcard bind address, not a URL you can open in a browser), and ensure your firewall allows incoming connections on both the Playground and scrcpy ports.

`MIDSCENE_SCRCPY_HOST` applies to the standalone Android Playground CLI and can also be set to a specific local network IP (such as `192.168.1.100`, which the auto-opened local preview page will also use). If you instantiate `ScrcpyServer` directly in code, pass the `host` option to its constructor instead.

:::warning Security Notice
The scrcpy streaming server does not currently authenticate viewer sessions. Binding it to a network interface exposes device information and live screen streams to any client that can reach the port. Enable remote access only on trusted private networks and restrict access with firewall rules.
:::

### How do I use a custom adb path or remote adb server?

See the [Android JavaScript SDK guide](/sdk/android.md#how-do-i-use-a-custom-adb-path-or-remote-adb-server).