• English

      • Configure Your Model

      A quick default

      Midscene requires a multimodal model to operate the UI. If you just want to get running, use the environment variables in the example below.

      Midscene drives the UI with a multimodal model. Set these environment variables to get started. The example below uses Qwen3.x via OpenRouter — it is easy to obtain and a solid default:

      export MIDSCENE_MODEL_BASE_URL="https://openrouter.ai/api/v1"
export MIDSCENE_MODEL_API_KEY="your-openrouter-api-key"
export MIDSCENE_MODEL_NAME="qwen/qwen3.7-plus"
export MIDSCENE_MODEL_FAMILY="qwen3"

      For all supported models and the different ways to set environment variables, read on.

      Supported models

      The following models are officially supported by Midscene. When configuring a model, in addition to the Base URL, API key, and model name typically required by an LLM, declare the correct MIDSCENE_MODEL_FAMILY to identify the model family and get the best compatibility.

      Model selection tips
      • For models from the same provider, prioritize newer versions. They generally offer a better overall balance of quality, speed, and pricing; avoid spending too much tuning effort on older versions.
      • Models from different providers can vary significantly in quality, speed, and pricing. Run a small comparison using representative tasks, then choose the best fit for your use case.
      • Midscene supports multi-model collaboration. One default model is usually enough for UI localization and interactions, but configuring separate Planning and Insight models lets you use different models for their strengths in performance, cost, and more, for a better overall result. For more information, see Advanced: Multi-model Collaboration.

      Doubao Seed Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      2.x seriesDoubao-Seed-2.1-turbo, Doubao-Seed-2.0-Litedoubao-seedDoubao-Seed-2.1-turbo has the fastest localization speed and strong localization quality in our current private evaluation set. Recommended.
      1.x seriesDoubao-Seed-1.6-Vision, Doubao-Seed-1.8doubao-seedThe 1.x series is an older generation of Doubao models and is no longer competitive overall. We recommend using the 2.x series instead.

      Environment variable configuration example, using doubao-seed-2.1-turbo:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_MODEL_API_KEY="...."
MIDSCENE_MODEL_NAME="doubao-seed-2.1-turbo"
MIDSCENE_MODEL_FAMILY="doubao-seed"

      If your Volcano Engine account has fast tier quota enabled, add the extra request body below to use it. This usually improves model response speed by about 30%-50%.

      MIDSCENE_MODEL_EXTRA_BODY_JSON={"service_tier":"fast"}
      Compatibility

      For compatibility with existing configurations, MIDSCENE_MODEL_FAMILY="doubao-vision" remains supported. New configurations should use doubao-seed.

      Qwen Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      Qwen3.x seriesqwen3.7-plus, qwen3.5-plus, qwen3.6-plusqwen3Based on localization evaluation results, the recommended order is Qwen3.7 > Qwen3.5 > Qwen3.6. The previous qwen3.5 and qwen3.6 families remain compatible.
      Qwen3-VL seriesqwen3-vl-plusqwen3-vlAs an older model generation, it is not recommended. Use the Qwen3.x series instead.
      Qwen2.5-VL seriesqwen-vl-max-latestqwen2.5-vlAs an older model generation, it is not recommended. Use the Qwen3.x series instead.

      Environment variable configuration example, using qwen3.7-plus:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="qwen3.7-plus"
MIDSCENE_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

      Google Gemini Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      Gemini 3.x seriesgemini-3.5-flash, gemini-3-flash-previewgeminigemini-3.5-flash currently performs best for localization in our private evaluation set.

      Environment variable configuration example, using gemini-3.5-flash:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_MODEL_FAMILY="gemini"

      OpenAI GPT Series

      • Common model provider: OpenAI
      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      GPT-5 seriesgpt-5.4, gpt-5.5, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-lunagpt-5Models before GPT-5.4 do not support visual localization and can only be used as Planning or Insight models. In practical localization tests, GPT-5.5 and GPT-5.6 perform noticeably better than GPT-5.4; we recommend using GPT-5.5 or GPT-5.6 first.

      Environment variable configuration example, using gpt-5.5:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_MODEL_API_KEY="sk-..."
MIDSCENE_MODEL_NAME="gpt-5.5"
MIDSCENE_MODEL_FAMILY="gpt-5"

      You can also use GPT through Use Codex App Server (OAuth, no API Key).

      GPT-5 notes
      • For UI localization with GPT, Midscene currently supports gpt-5.4 and later models. To get the best localization quality, image requests need "detail": "original". According to OpenAI, this option is available on gpt-5.4 and future models, while smaller GPT-5 variants such as gpt-5.4-mini and gpt-5.4-nano, as well as older models, do not support original and will fail if you send it. See the Images and Vision guide and the Computer use guide.
      • According to OpenAI, GPT-5 may still struggle with non-Latin text and with text that is too small in the image. See the Images and Vision guide.
      • In the computer use guide, OpenAI says they observe good performance around 1440x900 and 1600x900. We recommend adjusting screenshot sizes accordingly. In Midscene, you can control screenshot compression with screenshotShrinkFactor in the agent options. For browser automation, you can also control the page size and scale through the browser viewport.
      • With Azure OpenAI, Azure may not handle "detail": "original" correctly, causing click-coordinate offsets. See Clicks are offset when using Azure OpenAI.
      • If you use an older GPT-5 model, we recommend using it only as the planning model and pairing it with another multimodal model for localization. See the multi-model combination example.
      Model-native thinking

      Midscene disables model-native thinking by default for the best execution speed and stability. To turn it on for any of the models above, set MIDSCENE_MODEL_REASONING_ENABLED="true". Some families accept extra controls such as MIDSCENE_MODEL_REASONING_BUDGET and MIDSCENE_MODEL_REASONING_EFFORT. See Model-Native Thinking Mode.

      Moonshot Kimi Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      K3 serieskimi-k3kimi3According to the Kimi documentation, K3 always has reasoning enabled and cannot be disabled. Its reasoning effort defaults to max.
      K2.x serieskimi-k2.5, kimi-k2.6kimi—

      Environment variable configuration example, using kimi-k3:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="kimi-k3"
MIDSCENE_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

      Xiaomi MiMo Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      V2.x seriesmimo-v2.5xiaomi-mimoOnly the Omni series supports multimodal input; the Pro series is text-only and cannot be used for Midscene visual tasks.

      Environment variable configuration example, using mimo-v2.5:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="mimo-v2.5"
MIDSCENE_MODEL_FAMILY="xiaomi-mimo"

      Zhipu GLM-V Series

      Model versionCommonly used model namesCorresponding MODEL_FAMILYNotes
      GLM-5V seriesglm-5v-turboglm-v—
      GLM-4.6 seriesglm-4.6vglm-vglm-4.6v is open-source.

      Environment variable configuration example, using glm-5v-turbo:

      🎯 Use as default model
      🧠 Use as Planning model
      🔎 Use as Insight model
      MIDSCENE_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="glm-5v-turbo"
MIDSCENE_MODEL_FAMILY="glm-v"

      Learn more about the open-source GLM-4.6V model

      Ways to set environment variables

      Midscene reads all model configuration from environment variables. Below are common approaches, but feel free to adopt any method used in your project.

      Method 1: Set variables in the system

      The Midscene Chrome extension also accepts this export KEY="value" format.

      # Replace with your own API key
export MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
export MIDSCENE_MODEL_API_KEY="sk-abcde..."
export MIDSCENE_MODEL_NAME="qwen3.7-plus"
export MIDSCENE_MODEL_FAMILY="qwen3"

      Method 2: Create a .env file (for CLI tools)

      Create a .env file in the directory where you run the project. Midscene CLI tools load this file automatically.

      MIDSCENE_MODEL_BASE_URL="https://.../compatible-mode/v1"
MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"
MIDSCENE_MODEL_NAME="qwen3.7-plus"
MIDSCENE_MODEL_FAMILY="qwen3"

      Keep in mind:

      1. You do not need to prefix each line with export.
      2. Only the Midscene CLI automatically reads this file. For the JavaScript SDK, load it manually as shown below.

      Method 3: Load variables via dotenv

      Dotenv is a zero-dependency npm package that loads variables from .env into Node.js process.env.

      Our demo project uses this method.

      # install dotenv
npm install dotenv --save

      Create a .env file in the project root and add (no export prefix):

      MIDSCENE_MODEL_API_KEY="sk-abcdefghijklmnopqrstuvwxyz"

      Import dotenv in your script; it will read .env automatically:

      import 'dotenv/config';

      Other Compatible Models

      The following are smaller-parameter models compatible with Midscene and designed for automation scenarios. Their smaller size lowers hardware requirements for deployment, but they may be limited when handling complex tasks or large page screenshots. Evaluate them against your tasks and deployment constraints before choosing a model.

      Zhipu AutoGLM Series

      Zhipu AutoGLM is an open-source mobile UI automation model (9B parameters) from Zhipu AI.

      After obtaining an API key from Z.AI (Global) or BigModel (CN), configure:

      MIDSCENE_MODEL_BASE_URL="https://api.z.ai/api/paas/v4" # Or https://open.bigmodel.cn/api/paas/v4
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="autoglm-phone"
MIDSCENE_MODEL_FAMILY="auto-glm" # Or "auto-glm-multilingual"

      About MIDSCENE_MODEL_FAMILY Configuration

      AutoGLM provides two model versions, distinguished by MIDSCENE_MODEL_FAMILY:

      • auto-glm - Corresponds to AutoGLM-Phone-9B, optimized for Chinese mobile applications
      • auto-glm-multilingual - Corresponds to AutoGLM-Phone-9B-Multilingual, supports English and other languages

      Choose the appropriate version based on your application language.

      Note

      AutoGLM is best suited for mobile interactions and operations. If you need to use aiAssert, aiQuery, or other APIs requiring page understanding/assertions, you should configure separate MIDSCENE_INSIGHT_MODEL_... environment variables to let an independent Insight model handle page comprehension. See Model Strategy for more information about multi-model configuration.

      Learn more about Zhipu AutoGLM

      UI-TARS Series

      Use the deployed doubao-1.5-ui-tars on Volcano Engine:

      MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="...."
MIDSCENE_MODEL_NAME="ep-2025..." # Inference endpoint ID or model name from Volcano Engine
MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"

      About MIDSCENE_MODEL_FAMILY

      This variable selects the UI-TARS version. Supported values:

      • vlm-ui-tars – for the 1.0 release
      • vlm-ui-tars-doubao – for the 1.5 release deployed on Volcano Engine (equivalent to vlm-ui-tars-doubao-1.5)
      • vlm-ui-tars-doubao-1.5 – for the 1.5 release deployed on Volcano Engine
      Info

      The legacy configurations MIDSCENE_USE_VLM_UI_TARS=DOUBAO or MIDSCENE_USE_VLM_UI_TARS=1.5 are still supported but deprecated. Please migrate to MIDSCENE_MODEL_FAMILY.

      Migration mapping:

      • MIDSCENE_USE_VLM_UI_TARS=1.0 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars"
      • MIDSCENE_USE_VLM_UI_TARS=1.5 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"
      • MIDSCENE_USE_VLM_UI_TARS=DOUBAO → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao"

      Multi-model combination example

      For more information on combining multiple models, see Advanced: Multi-model Collaboration.

      Below is an example using GPT-5.4 for Planning/Insight and Qwen 3.5 for vision. Use GPT-5.4 for Planning and/or Insight to handle heavy reasoning, while Qwen 3.5 focuses on visual grounding. You can enable either role or both—toggle them based on your workload.

      # Default multimodal model: Qwen 3.5
export MIDSCENE_MODEL_BASE_URL="https://..."       # Qwen 3.5 endpoint
export MIDSCENE_MODEL_API_KEY="..."                # Your Qwen 3.5 API key
export MIDSCENE_MODEL_NAME="qwen3.5-plus"
export MIDSCENE_MODEL_FAMILY="qwen3.5"

# Planning model: GPT-5.4
export MIDSCENE_PLANNING_MODEL_API_KEY="sk-..."    # Your GPT-5.4 API key
export MIDSCENE_PLANNING_MODEL_BASE_URL="https://..." 
export MIDSCENE_PLANNING_MODEL_NAME="gpt-5.4"
export MIDSCENE_PLANNING_MODEL_FAMILY="gpt-5"

# Insight model: GPT-5.4
export MIDSCENE_INSIGHT_MODEL_API_KEY="sk-..."     # Your GPT-5.4 API key
export MIDSCENE_INSIGHT_MODEL_BASE_URL="https://..."
export MIDSCENE_INSIGHT_MODEL_NAME="gpt-5.4"
export MIDSCENE_INSIGHT_MODEL_FAMILY="gpt-5"

      More

      For additional, advanced model settings, see All configuration options.

      Troubleshooting model service connectivity issues

      Midscene includes a built-in model verify command for troubleshooting model service connectivity issues and basic compatibility issues.

      Put your model configuration in a .env file, then run the following model verify command to verify whether the current model configuration can support Midscene:

      # If the current project has @midscene/cli installed, use the local midscene command
npx midscene model verify

# If the current project does not have @midscene/cli installed, or you want to use the latest version
npx @midscene/cli@latest model verify

      The command reads the .env file in the current working directory. Dotenv debug logging is enabled by default, and values from .env override existing shell environment variables.

      To isolate basic model service connectivity issues, you can also run the following minimal curl request.

      MIDSCENE_MODEL_BASE_URL='replace with your baseUrl'
MIDSCENE_MODEL_API_KEY='replace with your API key'
MIDSCENE_MODEL_NAME='replace with your model name'

curl -X POST "${MIDSCENE_MODEL_BASE_URL%/}/chat/completions" \
  -H "Authorization: Bearer ${MIDSCENE_MODEL_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "'"${MIDSCENE_MODEL_NAME}"'",
  "messages": [
    {
      "role": "user",
      "content": "What is 1+1?"
    }
  ]
}'