Use a model API key
Connect Midscene using an API key from a model provider. This guide covers choosing a model, configuring it, and verifying the connection. If you already have a ChatGPT subscription, you can also follow Use a ChatGPT subscription.
The model configuration shown on this page consists of environment variables. Provide it according to how you use Midscene:
- In Playground, paste the configuration text from this page directly into the settings page.
- With the SDK or CLI, load the configuration as described in Set environment variables.
For complete parameter definitions, see Model configuration reference.
Supported models
Midscene supports these multimodal models for operating user interfaces: Doubao Seed, Qwen, DeepSeek, Google Gemini, OpenAI GPT, Moonshot Kimi, Xiaomi MiMo, and Zhipu GLM-V, as well as the other compatible models AutoGLM and UI-TARS described below.
Each configuration requires a Base URL, API key, model name, and MIDSCENE_MODEL_FAMILY. The model family (MIDSCENE_MODEL_FAMILY) determines how Midscene adapts to the selected model.
Doubao Seed Series
- Common model provider: Volcano Engine
Environment variable configuration example, using doubao-seed-2-1-turbo-260628:
If your Volcano Engine account has Fast Tier quota enabled, add the extra request body below to use it. This usually improves model response speed by about 30%-50%.
Qwen Series
- Common model provider: Alibaba Cloud
Environment variable configuration example, using qwen3.7-plus:
DeepSeek Series
Midscene supports DeepSeek starting from v1.12.0.
- Common model provider: DeepSeek - Vision
Environment variable configuration example, using deepseek-flash:
Google Gemini Series
- Common model provider: Google Gemini
Environment variable configuration example, using gemini-3.5-flash:
OpenAI GPT Series
To connect using your ChatGPT subscription, see Use a ChatGPT subscription.
- Common model provider: OpenAI
Use an API Key
Environment variable configuration example, using gpt-6-sol:
Enable Fast mode (service_tier="fast")
Both GPT-5.6 Sol and GPT-6 Astra support Fast mode. Add the service_tier request body parameter to enable it. For example, with GPT-5.6 Sol:
According to OpenAI, Fast mode can deliver up to 2–2.5× the speed at twice the Standard rate. Refer to the official documentation for current speed, pricing, and usage restrictions.
To reuse a Codex CLI session, see Use Codex App Server.
When using GPT, note the following:
- GPT-5.4 or later is required for UI localization. Midscene does not support earlier versions.
- According to OpenAI, GPT-5 may still struggle with non-Latin text and with text that is too small in the image. See the Images and Vision guide.
- In the computer use guide, OpenAI says they observe good performance around
1440x900and1600x900. We recommend adjusting screenshot sizes accordingly. In Midscene, you can control screenshot compression withscreenshotShrinkFactorin the agent options. For browser automation, you can also control the page size and scale through the browserviewport. - Azure OpenAI's Chat Completions API may not handle
"detail": "original"correctly, causing click-coordinate offsets. If you encounter this issue, you can switch to the Responses API. See Clicks are offset when using Azure OpenAI.
Moonshot Kimi Series
- Common model provider: Moonshot AI platform
Environment variable configuration example, using kimi-k3:
Xiaomi MiMo Series
- Common model provider: Xiaomi MiMo API Open Platform
Environment variable configuration example, using mimo-v2.6-pro:
Zhipu GLM-V Series
- Common model providers: Z.AI (Global), BigModel (CN)
Environment variable configuration example, using glm-5v-turbo:
Learn more about the open-source GLM-4.6V model
- Github: https://github.com/zai-org/GLM-V
- Hugging Face: https://huggingface.co/zai-org/GLM-4.6V
Set environment variables
Midscene reads model configuration from environment variables. Choose the method that matches how you use Midscene, or keep using your project's existing environment-management approach.
Set environment variables in the current shell
The following example works in Bash and Zsh. These variables are available only to the current shell session and its child processes.
Load .env with the CLI
Create a .env file in the directory where you run the midscene command. @midscene/cli loads this file automatically.
Do not add export at the beginning of each line. When running YAML tasks, existing variables in the current shell take precedence over values with the same name in .env. To let .env override them, use --dotenv-override.
midscene model verify is an exception. This command uses values from .env to override variables with the same name in the current shell.
Load .env with dotenv for the JavaScript SDK
The Midscene JavaScript SDK reads model configuration directly from the Node.js process environment (process.env). If the shell, container, or deployment platform already provides these variables, you do not need dotenv. Use dotenv only when the configuration is stored in a .env file and needs to be loaded into process.env.
Create a .env file in the directory where you run the script:
Import dotenv before creating a Midscene Agent:
If a variable with the same name already exists in the current shell, dotenv keeps the existing value by default. The Midscene demo project uses this loading method as well.
Verify the setup
After setting the environment variables, run one of the following commands:
For failed checks and connectivity problems, see Model debugging and observability.
Optional: configure multiple models
Multi-model setup is optional. For most use cases, a Default model is enough for UI localization and actions. Add a Planning or Insight model only when complex planning or page understanding requires a separate model. You can configure either one or both.
For guidance on when to combine models, see Model strategy.
The following example uses Qwen 3.5 as the Default model for visual grounding. GPT-5.4 serves as the Planning and Insight models for complex reasoning.
Other compatible models
The following smaller models are also compatible with Midscene and designed for automation. They require less deployment hardware but may struggle with complex tasks or large screenshots. Evaluate them against your tasks and deployment constraints before choosing one.
Zhipu AutoGLM Series
Zhipu AutoGLM is an open-source mobile UI automation model (9B parameters) from Zhipu AI.
After obtaining an API key from Z.AI (Global) or BigModel (CN), configure:
About MIDSCENE_MODEL_FAMILY Configuration
AutoGLM provides two model versions, distinguished by MIDSCENE_MODEL_FAMILY:
auto-glm- Corresponds to AutoGLM-Phone-9B, optimized for Chinese mobile applicationsauto-glm-multilingual- Corresponds to AutoGLM-Phone-9B-Multilingual, supports English and other languages
Choose the appropriate version based on your application language.
AutoGLM is best suited for mobile interaction. APIs such as aiAssert and aiQuery require page understanding. When using these APIs, configure a separate Insight model with the MIDSCENE_INSIGHT_MODEL_... environment variables. See Model strategy for details.
Learn more about Zhipu AutoGLM
- Github: https://github.com/zai-org/Open-AutoGLM
- Hugging Face: https://huggingface.co/zai-org/AutoGLM-Phone-9B
UI-TARS Series
Use the deployed doubao-1.5-ui-tars on Volcano Engine:
About MIDSCENE_MODEL_FAMILY
This variable selects the UI-TARS version. Supported values:
vlm-ui-tars– for the 1.0 releasevlm-ui-tars-doubao– for the 1.5 release deployed on Volcano Engine (equivalent tovlm-ui-tars-doubao-1.5)vlm-ui-tars-doubao-1.5– for the 1.5 release deployed on Volcano Engine
The legacy configurations MIDSCENE_USE_VLM_UI_TARS=DOUBAO or MIDSCENE_USE_VLM_UI_TARS=1.5 are still supported but deprecated. Please migrate to MIDSCENE_MODEL_FAMILY.
Migration mapping:
MIDSCENE_USE_VLM_UI_TARS=1.0→MIDSCENE_MODEL_FAMILY="vlm-ui-tars"MIDSCENE_USE_VLM_UI_TARS=1.5→MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"MIDSCENE_USE_VLM_UI_TARS=DOUBAO→MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao"
Next steps
- Learn when to use Default, Planning, and Insight models in Model strategy.
- Look up every environment variable in Model configuration reference.
- Diagnose connectivity and compatibility issues in Model debugging and observability.

