Quick experience by Chrome extension

Midscene.js provides a Chrome extension. By using it, you can quickly experience the main features of Midscene on any webpage, without needing to set up a code project.

⁠The extension shares the same code as the npm @midscene/web packages, so you can think of it as a playground or a way to debug with Midscene.

Preparation

Prepare the config for the AI model you want to use. You can check the supported models in Choose a model

Install and config

Install Midscene extension from chrome web store: Midscene

Or download the installation package and install it manually (not recommended, as automatic updates cannot be obtained)：Releases

Start the extension (may be folded by Chrome extension icon), setup the config by pasting the config in the K=V format:

OPENAI_API_KEY="sk-replace-by-your-own"
# ...all other configs here (if any)

Start experiencing

After the configuration, you can immediately experience Midscene. It provides multiple key operation tabs, including but not limited to:

  • Action: interact with the web page. This is also known as "Auto Planning". For example:
type Midscene in the search box
click the login button
  • Query: extract JSON data from the web page
extract the user id from the page, return in \{ id: string \}
  • Assert: validate the page
the page title is "Midscene"
  • Tap: perform a single tap on the element where you want to click. This is also known as "Instant Action".
the login button

All Agent APIs can be directly debugged and run in the Playground! Interactive, extraction, and verification methods are fully covered, with visual operations and verification that boost your automation development efficiency!

Enjoy !

For the different between "Auto Planning" and "Instant Action", please refer to the API document.

Want to write some code?

After experiencing, you may want to write some code to integrate Midscene. There are multiple ways to do that. Please refer to the documents below:

FAQ

Extension fails to run and shows 'Cannot access a chrome-extension:// URL of different extension'

It's mainly due to conflicts with other extensions injecting <iframe /> or <script /> into the page. Try disabling the suspicious plugins and refresh.

To find the suspicious plugins:

  1. Open the Devtools of the page, find the <script> or <iframe> with a url like chrome-extension://{ID-of-the-suspicious-plugin}/....
  2. Copy the ID from the url, open chrome://extensions/ , use cmd+f to find the plugin with the same ID, disable it.
  3. Refresh the page, try again.
On this page