• English

      • Quick experience by Chrome extension

      Midscene.js provides a Chrome extension. By using it, you can quickly experience the main features of Midscene on any webpage, without needing to set up a code project.

      ⁠The extension shares the same code as the npm @midscene/web packages, so you can think of it as a playground or a way to debug with Midscene.

      Prompt : Sign up for Github, you need to pass the form validation, but don't actually click.

      View the full report of this task: report.html

      Install Chrome extension

      Install Midscene extension from chrome web store: Midscene

      Start the extension (may be folded by Chrome extension icon), you should see the sidebar named "Midscene" in the right side of the browser.

      Set up API keys for model

      Set your model configs into the environment variables. You may refer to Model strategy for more details.

      export MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
export MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
export MIDSCENE_MODEL_NAME="replace-with-your-model-name"
export MIDSCENE_MODEL_FAMILY="replace-with-your-model-family"

      For more configuration details, please refer to Model strategy and Model configuration.

      See more showcases: showcases

      Start experiencing

      After configuration, you can start using Midscene right away. It provides several key operation tabs:

      • Act: interact with the page. This is Auto Planning, corresponding to aiAct. For example:
      Type “Midscene” in the search box, run the search, and open the first result
      Fill out the registration form and make sure every field passes validation
      • Query: extract JSON data from the interface, corresponding to aiQuery.

      Similar methods include aiBoolean(), aiNumber(), and aiString() for directly extracting booleans, numbers, and strings.

      Extract the user ID from the page and return JSON data in the { id: string } structure
      • Assert: understand the page and assert; if the condition is not met, throw an error, corresponding to aiAssert.
      There is a login button on the page, with a user agreement link below it
      • Tap: click on an element. This is Instant Action, corresponding to aiTap.
      Click the login button

      For the difference between Auto Planning and Instant Action, see the API document.

      FAQ

      Can I install the extension manually?

      If you cannot access the Chrome Web Store, you can download the installation package from the Releases page on GitHub and install it manually. However, this is not recommended as automatic updates will not be available.

      Extension fails to run and shows 'Cannot access a chrome-extension:// URL of different extension'

      It's mainly due to conflicts with other extensions injecting <iframe /> or <script /> into the page. Try disabling the suspicious plugins and refresh.

      To find the suspicious plugins:

      1. Open the Devtools of the page, find the <script> or <iframe> with a url like chrome-extension://{ID-of-the-suspicious-plugin}/....
      2. Copy the ID from the url, open chrome://extensions/ , use cmd+f to find the plugin with the same ID, disable it.
      3. Refresh the page, try again.