Quick Experience by Chrome Extension

Midscene.js provides a Chrome extension. By using it, you can quickly experience the main features of Midscene on any webpage, without needing to set up a code project.

⁠The extension shares the same code as the npm @midscene/ packages, so you can think of it as a playground or a way to debug with Midscene.

Preparation

Prepare an API key from one of these models: OpenAI GPT-4o, Qwen-2.5-VL, UI-TARS, or any other supported providers. We will be using it soon.

You can check the supported models in Choose a model

Install and config

Install Midscene extension from chrome web store: Midscene

Start the extension (may be folded by Chrome extension icon), setup the config by pasting the config in the K=V format:

OPENAI_API_KEY="sk-replace-by-your-own"
# ...all other configs here (if any)

Start experiencing

After the configuration, you can immediately experience Midscene. There are three main tabs in the extension:

  • Action: use action to interact with the web page, like "type Midscene in the search box" or "click the login button".
  • Query: use query to extract JSON data from the web page, like "extract the user id from the page, return in { id: string }".
  • Assert: use assert to validate the web page, like "the page title is "Midscene"".

Enjoy !

Want to write some code ?

After experiencing, you may want to write some code to integrate Midscene. There are multiple ways to do that. Please refer to the documents below:

FAQ

  • Extension fails to run and shows 'Cannot access a chrome-extension:// URL of different extension'

It's mainly due to conflicts with other extensions injecting <iframe /> or <script /> into the page. Try disabling the suspicious plugins and refresh.

To find the suspicious plugins:

  1. Open the Devtools of the page, find the <script> or <iframe> with a url like chrome-extension://{ID-of-the-suspicious-plugin}/....
  2. Copy the ID from the url, open chrome://extensions/ , use cmd+f to find the plugin with the same ID, disable it.
  3. Refresh the page, try again.