Quick Experience with Android

By using Midscene.js playground, you can quickly experience the main features of Midscene on Android devices, without needing to write any code.

Preparation

Install Node.js

Install Node.js 18 or above globally.

Prepare an API Key

Prepare an API key from a visual-language (VL) model. You will use it later.

You can check the supported models in Choose a model

Install adb

adb is a command-line tool that allows you to communicate with an Android device. There are two ways to install adb:

Verify adb is installed successfully:

adb --version

When you see the following output, adb is installed successfully:

Android Debug Bridge version 1.0.41
Version 34.0.4-10411341
Installed as /usr/local/bin//adb
Running on Darwin 24.3.0 (arm64)

Connect Android device with adb

In the developer options of the system settings, enable the 'USB debugging' of the Android device, if the 'USB debugging (secure settings)' exists, also enable it, then connect the Android device with a USB cable

android usb debug

Verify the connection:

adb devices -l

When you see the following output, the connection is successful:

List of devices attached
s4ey59	device usb:34603008X product:cezanne model:M2006J device:cezan transport_id:3

Run Playground

npx --yes @midscene/android-playground

Config API Key

Click the gear button to enter the configuration page and paste your API key config.

Refer to Config Model and Provider document, config the API Key.

Start experiencing

After the configuration, you can immediately experience Midscene. There are three main tabs in the extension:

  • Action: interact with the web page. This is also known as "Auto Planning". For example:
type Midscene in the search box click the login button
  • Query: extract JSON data from the web page
extract the user id from the page, return in \{ id: string \}
  • Assert: validate the page
the page title is "Midscene"
  • Tap: perform a single tap on the element where you want to click. This is also known as "Instant Action".
the login button

Enjoy !

For the different between "Auto Planning" and "Instant Action", please refer to the API document.

Want to write some code ?

After experiencing, you may want to write some code to integrate Midscene. There are multiple ways to do that. Please refer to the documents below: