feat: add Chinese UI, Responses models, and mirrored Windows setup
Localize the web console and persist custom model and capability settings. Use OpenAI Responses exclusively and move runtime limits into environment configuration. Bootstrap uv and the virtual environment on Windows, support optional China mirrors, and package source distributions without environments, build output, or credentials. Update regression tests and repository quality checks for the new behavior.
This commit is contained in:
chenmingxuan committed
1 parent
371aa6df56
commit
2ddec53954
183 files changed
+3427
-11857
No files matched your search
@@ -1,3 +1,11 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
|
||||
|
||||
每次调用设备操作工具前，用一小段简体中文说明当前观察、即将执行的动作和目的；操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文，精确引用的界面文字仍保留原文。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are an autonomous and highly efficient Android Device Execution Agent. Your goal is to accomplish the user's objective on the device.
|
||||
|
||||
|
||||
@@ -1,11 +1,17 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are the Step Summarizer for an Android UI automation agent. Your task is to synthesize the Before/After screenshots and physical action into a concise, high-information-density, strictly objective first-person historical memory.
|
||||
|
||||
---
|
||||
|
||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句，约 100–200 个汉字；保留关键原文证据，不为凑字数添加内容).
|
||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks in your output. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||
|
||||
---
|
||||
@@ -13,7 +19,7 @@ You are the Step Summarizer for an Android UI automation agent. Your task is to
|
||||
# CORE OBSERVATION RULES
|
||||
1. **Zero Subjective Validation (STRICT)**:
|
||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
2. **Action & Visual Delta Structure**:
|
||||
- **Target & Action**: Identify what control/button was targeted (referencing the red visual indicator on the BEFORE screen) and the physical action performed.
|
||||
- **Objective Transition**: Describe the physical screen change (e.g., page transitioned, modal dialog opened, list scrolled revealing new items, checkbox/toggle toggled, keyboard appeared).
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are the Step Summarizer for an Android UI automation agent. Exactly ONE screenshot is available for this step — the decision frame captured when the action was issued. There is NO after-action screenshot. Your task is to describe, strictly objectively and in first person, what THIS screen showed and where the physical action landed.
|
||||
|
||||
@@ -11,8 +17,8 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
||||
---
|
||||
|
||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句，约 100–200 个汉字；保留关键原文证据，不为凑字数添加内容).
|
||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||
|
||||
---
|
||||
@@ -20,7 +26,7 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
||||
# CORE OBSERVATION RULES
|
||||
1. **Zero Subjective Validation (STRICT)**:
|
||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
2. **Preserve Critical Data**: Faithfully transcribe visible text, toast alerts, error banners, verification codes, prices, account names, or tracking numbers that appear on this screen.
|
||||
|
||||
---
|
||||
|
||||
@@ -14,8 +14,7 @@
|
||||
|
||||
"""Universal Multi-Model FlashRunner for Artemis.
|
||||
|
||||
Executes autonomous reactive mobile workflows across Google Gemini,
|
||||
OpenAI GPT-4o/o3, Anthropic Claude 3.5/3.7, and OpenRouter endpoints.
|
||||
Executes autonomous reactive mobile workflows through OpenAI Responses.
|
||||
|
||||
Conversation shape (shared with the Pro operator, history redesign §3.2):
|
||||
the prompt is built every turn from a session :class:`TranscriptLedger` —
|
||||
@@ -38,6 +37,7 @@ active window until its turn is chunked, whose user lines then carry it on.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from artemis.config.settings import settings
|
||||
import base64
|
||||
from dataclasses import dataclass, field
|
||||
import json
|
||||
@@ -83,7 +83,7 @@ from artemis.memory.transcript import PRO_UI_LIST_MARKER, TranscriptLedger, mark
|
||||
from artemis.services.llm import (
|
||||
RobustChatModelWrapper,
|
||||
acomplete,
|
||||
get_google_llm,
|
||||
get_openai_llm,
|
||||
get_llm,
|
||||
invoke_llm_with_timeout_message,
|
||||
)
|
||||
@@ -284,7 +284,9 @@ class FlashRunner:
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get operator LLM from config, using default: {e}")
|
||||
|
||||
return RobustChatModelWrapper(get_google_llm(model_name="gemini-2.5-flash"), self.ctx)
|
||||
return RobustChatModelWrapper(
|
||||
get_openai_llm(model_name=settings.OPENAI_MODEL), self.ctx
|
||||
)
|
||||
|
||||
def _render_system_prompt(self, tools_declaration: list) -> str:
|
||||
"""Renders the system prompt from the flash_runner.md template.
|
||||
|
||||
@@ -26,6 +26,7 @@ frame, the ``flash_summarizer.md`` neutral-wording contract, and versioned
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from artemis.config.settings import settings
|
||||
import base64
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
@@ -36,7 +37,7 @@ from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.memory.step_memory import JobKey, StepMemoryService
|
||||
from artemis.services.llm import RobustChatModelWrapper, get_google_llm, get_llm
|
||||
from artemis.services.llm import RobustChatModelWrapper, get_openai_llm, get_llm
|
||||
from artemis.services.token_meter import record_llm_usage
|
||||
from artemis.utils.logger import get_logger
|
||||
from artemis.utils.task_tree import format_actions_clean
|
||||
@@ -162,15 +163,15 @@ class VisualStepSummarizer(StepMemoryService):
|
||||
)
|
||||
|
||||
# Initialize lightweight VLM: prioritize explicit model_name
|
||||
target_model = model_name or "gemini-2.5-flash-lite"
|
||||
target_model = model_name or settings.OPENAI_MODEL
|
||||
self._model_name = target_model
|
||||
try:
|
||||
if model_name:
|
||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
||||
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||
else:
|
||||
self._llm = get_llm(ctx, name="summarizer", is_utils=True)
|
||||
except Exception:
|
||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
||||
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||
try:
|
||||
configured = getattr(self._llm, "model", None) or getattr(self._llm, "model_name", None)
|
||||
if isinstance(configured, str) and configured:
|
||||
@@ -341,7 +342,7 @@ class VisualStepSummarizer(StepMemoryService):
|
||||
"""Meter one raw-model lens call as an ``llm_usage`` trace, best-effort.
|
||||
|
||||
Gateway-wrapped models already meter at the wrapper exit; only the raw
|
||||
``get_google_llm`` bypass needs explicit metering here. Lens prompts
|
||||
``get_openai_llm`` bypass needs explicit metering here. Lens prompts
|
||||
are tiny and must not overwrite the session's ``last_prompt_tokens``
|
||||
(the compaction thresholds' live context base), hence
|
||||
``update_last_prompt=False``.
|
||||
|
||||
Reference in new issue
Block a user