docs: add agent guides and simplify Chinese README
This commit is contained in:
chenmingxuan committed
1 parent
8e757d4626
commit
e1185a5104
6 files changed
+174
-870
No files matched your search
@@ -1,179 +0,0 @@
|
|||||||
name: CI
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
pull_request:
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
concurrency:
|
|
||||||
group: ci-${{ github.workflow }}-${{ github.ref }}
|
|
||||||
cancel-in-progress: true
|
|
||||||
|
|
||||||
env:
|
|
||||||
PYTHONUTF8: "1"
|
|
||||||
UV_NO_PROGRESS: "1"
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
python-quality:
|
|
||||||
name: Python 3.12 / ${{ matrix.os }}
|
|
||||||
runs-on: ${{ matrix.os }}
|
|
||||||
timeout-minutes: 30
|
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
matrix:
|
|
||||||
os:
|
|
||||||
- ubuntu-latest
|
|
||||||
- windows-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Check out repository
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python 3.12
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: "3.12"
|
|
||||||
|
|
||||||
- name: Check public dependency sources before installation
|
|
||||||
run: python scripts/check_dependency_sources.py
|
|
||||||
|
|
||||||
- name: Set up uv
|
|
||||||
uses: astral-sh/setup-uv@v6
|
|
||||||
with:
|
|
||||||
enable-cache: true
|
|
||||||
cache-dependency-glob: uv.lock
|
|
||||||
|
|
||||||
- name: Install locked development environment
|
|
||||||
run: uv sync --dev --locked
|
|
||||||
|
|
||||||
- name: Check Python formatting
|
|
||||||
run: uv run ruff format --check .
|
|
||||||
|
|
||||||
- name: Lint Python
|
|
||||||
run: uv run ruff check .
|
|
||||||
|
|
||||||
- name: Enforce quality ratchets
|
|
||||||
run: uv run python scripts/quality_ratchet.py
|
|
||||||
|
|
||||||
- name: Type-check protected core modules
|
|
||||||
run: uv run pyright --project pyright-core.json
|
|
||||||
|
|
||||||
# Device, cloud, manual, and end-to-end suites require dedicated
|
|
||||||
# infrastructure and intentionally do not run on GitHub-hosted runners.
|
|
||||||
- name: Run deterministic Python tests
|
|
||||||
if: github.event_name == 'workflow_dispatch'
|
|
||||||
run: >-
|
|
||||||
uv run pytest
|
|
||||||
tests/unit
|
|
||||||
tests/tools
|
|
||||||
packages/artemis-client/tests
|
|
||||||
-q
|
|
||||||
--cov=artemis
|
|
||||||
--cov=mcp_server
|
|
||||||
--cov=apps/admin_console
|
|
||||||
--cov-fail-under=60
|
|
||||||
|
|
||||||
frontend:
|
|
||||||
name: Frontend tests and production build
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
timeout-minutes: 20
|
|
||||||
defaults:
|
|
||||||
run:
|
|
||||||
working-directory: apps/showcase_ui
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Check out repository
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Node.js
|
|
||||||
uses: actions/setup-node@v4
|
|
||||||
with:
|
|
||||||
node-version: "22.22.3"
|
|
||||||
cache: npm
|
|
||||||
cache-dependency-path: apps/showcase_ui/package-lock.json
|
|
||||||
|
|
||||||
- name: Install locked frontend dependencies
|
|
||||||
run: npm ci
|
|
||||||
|
|
||||||
- name: Run frontend tests
|
|
||||||
run: npm test -- --watch=false --browsers=ChromeHeadless
|
|
||||||
|
|
||||||
- name: Build frontend for production
|
|
||||||
run: npm run build
|
|
||||||
|
|
||||||
package:
|
|
||||||
name: Build and smoke-test Python packages
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
timeout-minutes: 30
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Check out repository
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python 3.12
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: "3.12"
|
|
||||||
|
|
||||||
- name: Set up uv
|
|
||||||
uses: astral-sh/setup-uv@v6
|
|
||||||
with:
|
|
||||||
enable-cache: true
|
|
||||||
cache-dependency-glob: uv.lock
|
|
||||||
|
|
||||||
- name: Set up Node.js
|
|
||||||
uses: actions/setup-node@v4
|
|
||||||
with:
|
|
||||||
node-version: "22.22.3"
|
|
||||||
cache: npm
|
|
||||||
cache-dependency-path: apps/showcase_ui/package-lock.json
|
|
||||||
|
|
||||||
- name: Build frontend assets for the wheel
|
|
||||||
run: |
|
|
||||||
npm ci --prefix apps/showcase_ui
|
|
||||||
npm run build --prefix apps/showcase_ui
|
|
||||||
|
|
||||||
- name: Build client and Artemis distributions
|
|
||||||
run: |
|
|
||||||
uv build --package artemis-client --wheel --out-dir dist
|
|
||||||
uv build --package artemis --wheel --sdist --out-dir dist
|
|
||||||
|
|
||||||
- name: Install wheels in a clean environment
|
|
||||||
run: |
|
|
||||||
uv venv .smoke-venv --python 3.12
|
|
||||||
uv pip install --python .smoke-venv/bin/python dist/artemis_client-*.whl dist/artemis-*.whl
|
|
||||||
|
|
||||||
- name: Smoke-test installed package and CLI
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
smoke_dir="$RUNNER_TEMP/artemis-wheel-smoke"
|
|
||||||
mkdir -p "$smoke_dir"
|
|
||||||
cd "$smoke_dir"
|
|
||||||
source "$GITHUB_WORKSPACE/.smoke-venv/bin/activate"
|
|
||||||
python - <<'PY'
|
|
||||||
from importlib.metadata import files, version
|
|
||||||
|
|
||||||
import artemis
|
|
||||||
from artemis.resources import get_bundled_config_path, get_bundled_showcase_dist
|
|
||||||
|
|
||||||
bundled_config = get_bundled_config_path("artemis.jsonc")
|
|
||||||
bundled_ui = get_bundled_showcase_dist()
|
|
||||||
assert artemis.__version__ == version("artemis")
|
|
||||||
assert bundled_config is not None and bundled_config.is_file()
|
|
||||||
assert bundled_ui is not None and (bundled_ui / "index.html").is_file()
|
|
||||||
assert not any("node_modules" in str(path) for path in (files("artemis") or ()))
|
|
||||||
print(f"Installed Artemis {artemis.__version__} with bundled runtime resources")
|
|
||||||
PY
|
|
||||||
artemis --version
|
|
||||||
artemis --help
|
|
||||||
|
|
||||||
- name: Upload distributions
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: python-distributions
|
|
||||||
path: dist/*
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 7
|
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
# ARTEMIS 仓库协作指南
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
本仓库是基于 ARTEMIS 扩展的 Android AI 自动化测试平台，提供简体中文网页、OpenAI Responses 模型调用、批量任务、历史检索及 Word/Excel 报告。后端使用 Python 3.12+、FastAPI、LangGraph，前端使用 Angular，AI 客户端通过 MCP 接入。
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
- 本文件适用于整个仓库；子目录的 AGENTS.md 在对应目录内优先。
|
||||||
|
- 当前任务的用户指令优先于仓库指南。保留用户已有修改，只处理本次任务范围。
|
||||||
|
- 本文件指导开发；使用软件执行 Android 测试时，阅读 [自动化 Skill](skills/artemis-automation/SKILL.md)。
|
||||||
|
- 以下是项目原生命令；宿主要求 RTK 等包装器时按宿主规则调用，不把本机路径或工具写成项目安装前提。
|
||||||
|
|
||||||
|
## Quick Commands
|
||||||
|
|
||||||
|
在仓库根目录执行，前端命令见局部指南：
|
||||||
|
|
||||||
|
| 用途 | 命令 |
|
||||||
|
| --- | --- |
|
||||||
|
| 安装锁定的开发依赖 | `uv sync --dev --locked` |
|
||||||
|
| 启动已准备好的网页服务 | `uv run artemis ui --no-open` |
|
||||||
|
| 环境诊断 | `uv run artemis doctor` |
|
||||||
|
| 默认确定性测试 | `uv run pytest` |
|
||||||
|
| 格式检查 | `uv run ruff format --check .` |
|
||||||
|
| 静态检查 | `uv run ruff check .` |
|
||||||
|
| 质量基线检查 | `uv run python scripts/quality_ratchet.py` |
|
||||||
|
| 核心类型检查 | `uv run pyright --project pyright-core.json` |
|
||||||
|
| 源码打包 | `uv run --no-sync python scripts/package_source.py` |
|
||||||
|
|
||||||
|
Windows 首次安装使用 `start.bat`，国内镜像使用 `start-cn.bat`。开发中需要保留当前源码时传入 `-SkipUpdate`，避免启动脚本拉取远端更新。
|
||||||
|
|
||||||
|
## Key Paths
|
||||||
|
|
||||||
|
- `artemis/agents/`、`artemis/runtime/`：Flash/Pro 执行、设备与任务运行时。
|
||||||
|
- `artemis/llm/`、`artemis/services/llm.py`：模型请求、超时、重试与恢复。
|
||||||
|
- `artemis/config/`、`.env.example`：运行配置、环境变量及默认值。
|
||||||
|
- `config/artemis.jsonc`、`artemis/resources/config/artemis.jsonc`：开发与打包使用的配置，默认值变更须同步。
|
||||||
|
- `apps/admin_console/`：HTTP API、统一队列、批次、历史及报告服务。
|
||||||
|
- `apps/showcase_ui/`：Angular 网页；遵循 [前端指南](apps/showcase_ui/AGENTS.md)。
|
||||||
|
- `mcp_server/`：MCP 工具与客户端使用规则。
|
||||||
|
- `packages/artemis-client/`：远程 Python SDK；`skills/artemis-automation/`：随源码分发的 AI 使用技能。
|
||||||
|
- `scripts/`：安装、镜像、升级、检查及打包脚本。
|
||||||
|
- `tests/unit/`、`tests/tools/`、`packages/artemis-client/tests/`：默认测试范围。
|
||||||
|
|
||||||
|
## Architecture Boundaries
|
||||||
|
|
||||||
|
- 模型传输只使用 OpenAI Responses，不恢复 Gemini 专用调用或 Chat Completions 分支。API 模型名与 Flash/Pro 执行模式分开处理。
|
||||||
|
- 网页和工作区 MCP 工具复用后台配置、队列、历史和报告服务；不要另建调度器或第二套状态存储。
|
||||||
|
- 批次创建与执行分开；保持 request_id 幂等性。同设备串行，失败项不自动重跑；模型重试不能变成业务用例重提。
|
||||||
|
- 区分“执行完成”与“验证通过”。报告结论必须有实际证据，保留未验证、失败、跳过和中断状态。
|
||||||
|
- 设备动作回放已移除；只读历史步骤、截图和上下文回忆仍保留，不因 replay 命名误删这些能力。
|
||||||
|
- 密钥仅通过网页或环境配置写入，读取配置不得回显；MCP 不接受密钥。全局配置修改影响后续任务。
|
||||||
|
- 默认 high、请求超时 300 秒、失败后重试 5 次、间隔 10 秒，视频分析与录屏关闭。接口不支持思考强度时明确报错，不静默降级。
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- Python 使用绝对导入、4 空格缩进和双引号，由 Ruff 检查；不要放宽质量基线来掩盖新增问题。
|
||||||
|
- 新增可调参数集中进入配置层，并更新环境示例、网页/MCP 入口和必要文档，避免散落硬编码。
|
||||||
|
- MCP 工具增删时同步工具注册、CLI 安装白名单、测试、Skill 和能力文档。
|
||||||
|
- Windows PowerShell 启动脚本保持兼容 Windows PowerShell 5.1，避免编码导致解析失败。升级只做 Git 快进，保护已有配置和本地修改。
|
||||||
|
- 前端遵循现有简体中文文案与视觉样式；不直接编辑构建产物。
|
||||||
|
|
||||||
|
## Definition of Done
|
||||||
|
|
||||||
|
- Python 行为改动运行默认测试、Ruff 两项检查、质量基线和核心类型检查；前端改动遵循局部指南的测试与构建要求。
|
||||||
|
- 文档改动核对命令、相对链接和功能说明；仅文档变更无需启动设备或调用付费模型。
|
||||||
|
- 默认测试不依赖设备、密钥或私有服务；外部测试按现有 android、integration、e2e、cloud、manual 标记分层，不默认执行全量外部测试。
|
||||||
|
- 设备验证沿用用户指定设备；多设备且未指定时先确认。编写应用交互测试前，先通过 ARTEMIS 探索实际页面和操作路径。
|
||||||
|
- 提交前检查 `git diff --check` 和暂存范围；报告实际执行的验证及未完成项，不将未运行的检查写成通过。
|
||||||
|
|
||||||
|
## Boundaries
|
||||||
|
|
||||||
|
- 始终保护用户任务、历史库、环境配置和凭据。不要把个人设备序列号、绝对目录或 API Key 写入源码。
|
||||||
|
- 删除历史、清空设备数据、停止用户正在运行的任务或覆盖已有配置，需要明确授权；已有授权不重复询问。
|
||||||
|
- 不提交 `.env`、虚拟环境、依赖目录、执行记录和编译产物；发布源码使用仓库打包脚本。
|
||||||
|
- 不通过强推、禁用 TLS、绕过钩子或降低测试要求来完成交付。推送成功须核验远端提交。
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [用户入口](README.md)：安装、配置与日常使用。
|
||||||
|
- [AI 自动化](docs/ai-automation.md)：MCP、Skill 及升级行为。
|
||||||
|
- [工作区说明](docs/automation-workspace.md)：报告、批量导入和历史检索。
|
||||||
+2
-2
@@ -20,8 +20,8 @@ Every contribution—big or small—helps make ARTEMIS better. Thank you for bui
|
|||||||
## Test layers
|
## Test layers
|
||||||
|
|
||||||
`make test` is the required, deterministic suite. It does not need an Android
|
`make test` is the required, deterministic suite. It does not need an Android
|
||||||
device, model credentials, or the optional private cloud service. This is the
|
device, model credentials, or the optional private cloud service. Run these
|
||||||
same suite used by pull-request CI.
|
checks locally; this repository does not include a GitHub Actions workflow.
|
||||||
|
|
||||||
- `make test-integration` runs cross-component tests that may need configured
|
- `make test-integration` runs cross-component tests that may need configured
|
||||||
model credentials.
|
model credentials.
|
||||||
|
|||||||
@@ -1,326 +1,78 @@
|
|||||||
<p align="center">
|
# ARTEMIS-Eapil
|
||||||
<img src="./docs/assets/artemis-banner.png?v=7" alt="ARTEMIS Banner" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
基于 [ARTEMIS](https://github.com/google/artemis) 扩展的 Android AI 自动化测试平台。用自然语言描述任务，即可通过简体中文网页或 MCP 驱动真机、模拟器执行操作并收集结果。模型统一使用 **OpenAI Responses API**。
|
||||||
<strong>Let AI assistants and test suites use real phones like a human.</strong>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
## 主要功能
|
||||||
<a href="./README.md"><b>English</b></a> •
|
|
||||||
<a href="./README_CN.md">中文文档</a> •
|
|
||||||
<a href="#workflow-showcase">Workflow Showcase</a> •
|
|
||||||
<a href="#quick-start">Quick Start</a> •
|
|
||||||
<a href="#mcp-setup">MCP for IDEs</a> •
|
|
||||||
<a href="#benchmarks">Benchmarks</a> •
|
|
||||||
<a href="https://discord.gg/wF2FN4WHGY">Discord Community</a>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
- **自然语言任务**：Flash 适合明确的操作流程，Pro 适合需要计划、分支探索和验证的任务。
|
||||||
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
- **可保存的模型配置**：支持自定义接口、模型、思考强度和超时重试，以及视频分析与录屏开关。
|
||||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
- **批量测试**：下载 Excel 模板、校验导入、顺序执行，支持暂停、继续和复制失败项。
|
||||||
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
- **报告与历史**：导出 Word、Excel 和批次 ZIP；历史按日期分组，支持搜索和筛选。
|
||||||
<a href="https://developers.openai.com/api/docs/guides/responses"><img src="https://img.shields.io/badge/API-OpenAI%20Responses-4285F4.svg" alt="OpenAI Responses"></a>
|
- **AI 接入**：随源码提供 MCP 与 Skill，让 AI 完成设备诊断、任务执行、历史分析和报告交付。
|
||||||
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<!-- Demo Showcase -->
|
## 安装与启动
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/demo.gif" alt="Artemis in Action" width="100%" />
|
|
||||||
<br>
|
|
||||||
<em>Live Demo: Setup driving routes and calculate total durations in Google Maps, then open YouTube to play a Coldplay song.</em>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## Key Highlights
|
准备一台开启 USB 调试并完成授权的 Android 手机，或已启动的 Android 模拟器。首次安装需要联网，建议使用 Git 克隆以支持后续更新。
|
||||||
|
|
||||||
* **Cross-App Automation**: Executes testing workflows and everyday tasks on Android from natural language instructions.
|
### Windows
|
||||||
* **Multimodal Targeting**: Uses element indices when available, with coordinate and visual locating fallbacks for custom interfaces.
|
|
||||||
* **IDE Diagnostics**: **Model Context Protocol (MCP)** integration lets **Antigravity, Claude Code, and Windsurf** drive test devices and collect **Logcat** output and screenshots.
|
|
||||||
* **Flash Execution**: A reactive observe-and-act loop with asynchronous history summaries, typically **3–5s per step**.
|
|
||||||
* **Pro Exploration**: Checks targets before individual actions and returns blocked actions to the Operator for recovery. Supports long-running exploratory and stability tests.
|
|
||||||
* **AndroidWorld Results**: **99%+ task completion** on Google Research's **AndroidWorld** benchmark (100+ multi-step tasks).
|
|
||||||
|
|
||||||
<a id="workflow-showcase"></a>
|
在 PowerShell 中执行：
|
||||||
## Antigravity × ARTEMIS: Autonomous Testing Workflow
|
|
||||||
|
|
||||||
**Antigravity** uses **ARTEMIS** through MCP to turn a test request into a plan, device execution, and a diagnostic report:
|
|
||||||
|
|
||||||
<table width="100%">
|
|
||||||
<tr>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>1. Prompt Input (Task Dispatch)</b><br>
|
|
||||||
<sub>Describe your test scenario and target metrics in Antigravity</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-1-prompt.png" width="100%" alt="Step 1: Prompt Input in Antigravity" />
|
|
||||||
</td>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>2. Test Plan Generation</b><br>
|
|
||||||
<sub>Formulates a step-by-step test plan & architecture for review</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-2-plan.png" width="100%" alt="Step 2: Test Plan Generation" />
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
<tr>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>3. Autonomous Test Execution</b><br>
|
|
||||||
<sub>Drives real device, navigates UI, and profiles performance</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-3-exec.png" width="100%" alt="Step 3: Autonomous Test Execution" />
|
|
||||||
</td>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>4. Final Report</b><br>
|
|
||||||
<sub>Delivers structured audit findings, metric tables, and raw datasets</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-4-report.png" width="100%" alt="Step 4: Final Report" />
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
</table>
|
|
||||||
|
|
||||||
<a id="quick-start"></a>
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
Ensure an Android device (with **USB Debugging** enabled) or emulator is connected. The one-click startup script will automatically:
|
|
||||||
- **Install System Toolchains**: Detect and auto-install ADB, scrcpy, FFmpeg, and Python (`uv`) dependencies.
|
|
||||||
- **Mount Global MCP Server & AI Agent Rules**: Prompt to automatically install global MCP configurations and the **Artemis Mobile Testing Mindset (`rules.md`)** into your AI IDEs (**Antigravity**, **Cursor**, **Claude Code**, **Codex**, **Windsurf**, **VS Code**, **Cline/Roo**, **OpenClaw**).
|
|
||||||
|
|
||||||
### macOS and Linux
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Clone repo & navigate to directory
|
|
||||||
git clone https://github.com/google/artemis.git && cd artemis
|
|
||||||
|
|
||||||
# 2. One-click launch
|
|
||||||
./start.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Windows PowerShell
|
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
# 1. Clone repo & navigate to directory
|
git clone https://git.playones.com/chenmingxuan/Artemis-Eapil.git
|
||||||
git clone https://github.com/google/artemis.git
|
cd Artemis-Eapil
|
||||||
cd artemis
|
|
||||||
|
|
||||||
# 2. One-click launch
|
|
||||||
.\start.bat
|
.\start.bat
|
||||||
```
|
```
|
||||||
|
|
||||||
> PowerShell does not search the current directory for executable scripts by default, so use `.\start.bat` without a trailing `\`. In Command Prompt (CMD), use `start.bat` instead.
|
国内网络使用 `.\start-cn.bat`。启动脚本会检测并安装缺少的 uv、Python 和所需工具，创建或复用 `.venv`，同步依赖并构建网页。启动后访问 **http://127.0.0.1:8000**。
|
||||||
|
|
||||||
> **Tip**: Opens `http://localhost:8000` in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: `uv run artemis run "Open Settings, find Battery and tell me current level" --profile flash`.
|
常用选项：
|
||||||
|
|
||||||
### Model and feature settings
|
|
||||||
|
|
||||||
The web console now uses Simplified Chinese. Open **系统设置与运行准备 → 模型配置** to edit and save the Responses base URL, model, API key, request timeout, and retry count. **测试连接** tests the current form without saving; **保存配置** persists it to `.env`. Leave the key blank to retain the existing credential. Existing keys are never returned to the browser.
|
|
||||||
|
|
||||||
All models use **OpenAI Responses**, including custom gateways. Set `OPENAI_API_KEY`, `OPENAI_BASE_URL` (the API root, usually ending in `/v1`), and `OPENAI_MODEL` (for example `mimo-v2.6-flash`). Gemini, Anthropic and Chat Completions transports are no longer supported. Video analysis sends sampled image frames and requires image input support. Structured output uses Responses function calling with local schema validation.
|
|
||||||
|
|
||||||
The **功能开关** section persists video analysis/recording, final reports, and Flash step summaries. Changes apply to new tasks and survive restart. Environment variables take precedence over `.env`; explicit per-node or task overrides take precedence over global defaults. See [`.env.example`](.env.example) for timeout settings and the [Chinese setup guide](README_CN.md#网页模型配置与功能开关) for details.
|
|
||||||
|
|
||||||
<a id="mcp-setup"></a>
|
|
||||||
<a id="mcp"></a>
|
|
||||||
<details>
|
|
||||||
<summary><b>MCP Setup for Codex / Antigravity / Claude Code / Windsurf (Click to expand)</b></summary>
|
|
||||||
|
|
||||||
<br>
|
|
||||||
|
|
||||||
ARTEMIS includes a native **Model Context Protocol (MCP)** server. Connect your real phone directly into AI IDEs:
|
|
||||||
|
|
||||||
### 1. One-Click Auto Install (Recommended)
|
|
||||||
|
|
||||||
Running `./start.sh` (macOS/Linux) or `.\start.bat` (Windows PowerShell) will prompt you to configure global MCP and testing rules for detected IDEs (or you can install/update anytime later manually using the commands below):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Auto-install MCP server & global rules for Antigravity / Jetski:
|
|
||||||
uv run artemis mcp --install antigravity
|
|
||||||
|
|
||||||
# Or install for all supported AI IDEs (including Codex):
|
|
||||||
uv run artemis mcp --install all
|
|
||||||
```
|
|
||||||
|
|
||||||
> **Tip**: You can also configure MCP interactively during first-time setup via `uv run artemis init`.
|
|
||||||
> **Pro Tip**: If you want to use the `artemis` command globally without `uv run` in any directory, run `uv tool install -e .` once in the project root.
|
|
||||||
|
|
||||||
### 2. Manual Configuration (Optional)
|
|
||||||
|
|
||||||
If you prefer to configure manually, run `uv run artemis mcp --generate-config <client>` (for example, `codex` or `antigravity`) to output the appropriate TOML or JSON snippet. Replace `/path/to/artemis` with your actual repo path and point `command` to your `.venv` Python executable:
|
|
||||||
|
|
||||||
* **Codex** (`~/.codex/config.toml`):
|
|
||||||
```toml
|
|
||||||
[mcp_servers.artemis]
|
|
||||||
command = "/path/to/artemis/.venv/bin/python"
|
|
||||||
args = ["-m", "mcp_server"]
|
|
||||||
cwd = "/path/to/artemis"
|
|
||||||
|
|
||||||
[mcp_servers.artemis.env]
|
|
||||||
PYTHONUNBUFFERED = "1"
|
|
||||||
PYTHONPATH = "/path/to/artemis"
|
|
||||||
```
|
|
||||||
|
|
||||||
* **Antigravity** (`~/.gemini/jetski/mcp_config.json`):
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"artemis": {
|
|
||||||
"command": "/path/to/artemis/.venv/bin/python",
|
|
||||||
"args": ["-m", "mcp_server"],
|
|
||||||
"cwd": "/path/to/artemis",
|
|
||||||
"env": {
|
|
||||||
"PYTHONUNBUFFERED": "1"
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"mobile_run_task": { "eager": true },
|
|
||||||
"mobile_manage_task": { "eager": true },
|
|
||||||
"mobile_get_device_state": { "eager": true },
|
|
||||||
"mobile_inspect_trace": { "eager": true },
|
|
||||||
"mobile_diagnose": { "eager": true }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
* **Claude Desktop** (`claude_desktop_config.json`):
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"artemis": {
|
|
||||||
"command": "/path/to/artemis/.venv/bin/python",
|
|
||||||
"args": ["-m", "mcp_server"],
|
|
||||||
"cwd": "/path/to/artemis"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Mount Behavioral Rules for AI Agents (Highly Recommended)
|
|
||||||
|
|
||||||
To ensure your AI coding assistant acts with the rigor of a senior mobile test engineer and never hallucinates UI interactions, we provide a dedicated testing mindset rules file at [`mcp_server/rules.md`](./mcp_server/rules.md) (covering **Active Exploration before coding**, **Flash vs. Pro routing strategy**, **Latency & Timing compensation**, and the **"Dynamic-First, Coordinate-Fallback" locator pattern**).
|
|
||||||
|
|
||||||
You can mount or copy [`mcp_server/rules.md`](./mcp_server/rules.md) into your AI IDE's rule configuration:
|
|
||||||
* **Antigravity**: Add the contents of `rules.md` to your Workspace Rules, Global Rules settings, or agent instructions.
|
|
||||||
* **Claude Code**: Run `artemis mcp --install claude` to install the rules to `~/.claude/rules/artemis.md` (install to exactly one location — Claude Code loads both `~/.claude/CLAUDE.md` and `~/.claude/rules/*.md`, so duplicating the rules wastes context).
|
|
||||||
* **Cursor**: Copy the contents into `.cursorrules` or create a rule file at `.cursor/rules/artemis.mdc`.
|
|
||||||
* **Codex**: Add the contents to `~/.codex/AGENTS.md` (or the active `AGENTS.override.md`).
|
|
||||||
* **Windsurf / OpenClaw**: Add the rules to your workspace rules or global system prompts.
|
|
||||||
|
|
||||||
> For more details on the testing mindset and MCP architecture, see the [MCP Server README](./mcp_server/README.md).
|
|
||||||
|
|
||||||
### 4. Prompt Your Phone in the IDE Chat
|
|
||||||
In Codex, Antigravity, or Claude Code, simply prompt:
|
|
||||||
> *"Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, verify if there are any unexpected popups after login, and return screenshots of the final page."*
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<a id="python-sdk"></a>
|
|
||||||
<details>
|
|
||||||
<summary><b>Python SDK Integration (Click to expand)</b></summary>
|
|
||||||
|
|
||||||
<br>
|
|
||||||
|
|
||||||
Install the zero-runtime-dependency client on the development machine. ADB,
|
|
||||||
agents, models, and image processing remain on the device host:
|
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
uv add "artemis-client @ git+https://github.com/google/artemis.git#subdirectory=packages/artemis-client"
|
.\start-cn.bat -SetupOnly # 仅准备 Python 环境并检查已有 MCP 配置
|
||||||
|
.\start.bat -Port 8001 # 指定端口
|
||||||
|
.\start.bat -SkipUpdate # 跳过 Git 源码更新检查
|
||||||
```
|
```
|
||||||
|
|
||||||
```python
|
镜像与下载超时在 [config/downloads.json](config/downloads.json) 中设置。scrcpy 默认从 GitHub 下载，无法访问时可配置可信的 scrcpy_url 或预先安装。
|
||||||
import asyncio
|
|
||||||
from artemis_client import ArtemisClient
|
|
||||||
|
|
||||||
|
### macOS / Linux
|
||||||
|
|
||||||
async def main():
|
克隆仓库后在项目目录执行 `./start.sh`。国内镜像入口和启动时自动更新目前由 Windows 脚本提供。
|
||||||
client = ArtemisClient(
|
|
||||||
"http://artemis-host:8000",
|
|
||||||
device_serial="emulator-5554", # optional: target specific device serial
|
|
||||||
default_profile="flash", # "flash" (fast reactive) or "pro" (deep reasoning)
|
|
||||||
)
|
|
||||||
|
|
||||||
result = await client.run(
|
## 配置与使用
|
||||||
"Open System Settings, go to 'Battery', verify battery percentage is displayed, and check for any crash dialogs.",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert result.succeeded, f"Test failed: {result.error or result.status}"
|
1. 在首页“系统设置与运行准备”中选择设备，打开“模型配置”。
|
||||||
print(f"✅ Test Passed! Device: {result.device_serial} | Trace ID: {result.trace_id}")
|
2. 填写支持 Responses 的接口地址、API Key 和模型名称，测试连接后保存。接口地址通常是以 `/v1` 结尾的 API 根地址。
|
||||||
|
3. 输入任务目标，选择 Flash 或 Pro 后执行；在工作区查看步骤、截图和结果。
|
||||||
|
4. 通过“报告与批量”导出报告或导入 Excel 用例；任务队列支持历史搜索。
|
||||||
|
|
||||||
|
预设模型为 `gpt-5.6-luna`、`mimo-v2.6-flash`，也可填写自定义名称。默认模型是 `mimo-v2.6-flash`，思考强度 `high`，单次请求超时 **300 秒**，失败后最多重试 **5 次**，间隔 **10 秒**；视频分析与录屏默认关闭。模型及思考参数是否可用取决于所配置网关；不支持时会明确报错。
|
||||||
|
|
||||||
if __name__ == "__main__":
|
也可在项目 `.env` 中设置 `OPENAI_API_KEY`、`OPENAI_BASE_URL` 和 `OPENAI_MODEL`，其他参数见 [.env.example](.env.example)。直接编辑环境文件后需重启相应服务。不要提交自己的密钥。
|
||||||
asyncio.run(main())
|
|
||||||
|
报告会区分“执行完成”和“验证通过”；没有验证证据的结果标为“未验证”。任务步骤与截图可查看，设备动作回放不提供。
|
||||||
|
|
||||||
|
## 让 AI 使用
|
||||||
|
|
||||||
|
环境与模型配置完成后，例如为 Codex 接入 MCP：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uv run artemis mcp --install codex
|
||||||
```
|
```
|
||||||
|
|
||||||
</details>
|
在客户端重新加载 MCP，并按 [AI 接入指南](docs/ai-automation.md) 安装仓库中的 [artemis-automation Skill](skills/artemis-automation/SKILL.md)。Skill 不会随启动脚本自动安装到用户全局目录。其他客户端可通过 `uv run artemis mcp --help` 查看接入方式。
|
||||||
|
|
||||||
## Usage Modes
|
示例指令：
|
||||||
|
|
||||||
<p align="center">
|
> 使用 artemis-automation，在我指定的模拟器上验证系统设置首页可以打开，不修改设置，完成后输出 Word 和 Excel 报告。
|
||||||
<img src="./docs/assets/artemis-ui-showcase-en.png" alt="Artemis Web Console" width="100%" />
|
|
||||||
<br />
|
|
||||||
<sub><b>Console Overview</b>: <b>① View Switcher</b> (Home / Workspace) · <b>② Model & Replay</b> (Flash/Pro status & video replay) · <b>③ Live Agent Stream</b> (Action perception, target coordinates & structured results) · <b>④ Prompt Dock</b> (Natural language dispatch) · <b>⑤ Task Queue & Dashboard</b> (Lifecycle & history)</sub>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
* **Web Visual Test Console (`uv run artemis ui`)**: Real-time screen projection and interactive panel, supporting natural language test dispatch, live reasoning telemetry, action trajectories, and execution replay; manage server lifecycle anytime from any terminal using `uv run artemis restart`, `uv run artemis stop`, and `uv run artemis status`;
|
## 更新与排错
|
||||||
* **MCP Server**: Connects **Antigravity, Claude Code, Windsurf**, and other MCP clients to real devices for bug reproduction and test execution;
|
|
||||||
* **Developer CLI (`uv run artemis run`)**: Direct terminal execution for automated test cases, exploratory stability inspection, or AndroidWorld benchmarks with high-fidelity structured terminal output;
|
|
||||||
* **Python SDK**: Integrates as a standard Python library into existing automated testing frameworks (e.g., pytest) or CI/CD pipelines with strongly typed Pydantic structured outputs and assertion support.
|
|
||||||
|
|
||||||
<a id="on-device-helper"></a>
|
Windows 用户再次运行一键脚本时，会检查 Git 上游、项目依赖、前端构建和已有 MCP 配置，仅对源码做快进更新。本地修改、分支分叉或网络认证失败时保留本地版本；有任务执行或排队时停止更新启动。更新后需在 AI 客户端重新加载 MCP。
|
||||||
## What ARTEMIS Installs on Your Phone
|
|
||||||
|
|
||||||
The first task on a device installs the **Artemis Accessibility Helper**, a small
|
ZIP 安装可继续运行；需要自动拉取源码时，请改用 Git 克隆安装。遇到启动或设备问题，在项目目录运行 `uv run artemis doctor`。
|
||||||
accessibility service that reads the screen layout without taking the
|
|
||||||
UiAutomation connection. Tools using UiAutomation can suppress the helper unless
|
|
||||||
they enable `FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES`. You will see
|
|
||||||
a collapsed "Artemis test helper is running" notification and a new entry under
|
|
||||||
Settings > Accessibility; both are that helper. It listens only on the phone
|
|
||||||
itself and sends nothing elsewhere.
|
|
||||||
|
|
||||||
* Pre-install it (avoids the ~3 s delay on the first task): `uv run artemis helper install`
|
更多说明：[批量任务与报告](docs/automation-workspace.md) · [AI 接入](docs/ai-automation.md) · [开发协作指南](AGENTS.md)
|
||||||
* Inspect it: `uv run artemis helper status` / `uv run artemis doctor`
|
|
||||||
* Remove it any time: `uv run artemis helper uninstall`
|
|
||||||
* Use UIAutomator2 instead: `ARTEMIS_HIERARCHY_BACKEND=uiautomator` in `.env`
|
|
||||||
* Prevent automatic installation: `ARTEMIS_HELPER_AUTO_INSTALL=false` in `.env`
|
|
||||||
|
|
||||||
If the helper ever fails mid-task, ARTEMIS falls back to UIAutomator2 and says
|
基于上游 ARTEMIS 开发，遵循 [Apache-2.0 许可证](LICENSE)。
|
||||||
so in the task timeline, in `mobile_manage_task` status, and in the final report.
|
|
||||||
|
|
||||||
<a id="benchmarks"></a>
|
|
||||||
## Benchmarks: AndroidWorld (SOTA 99%+)
|
|
||||||
|
|
||||||
Artemis achieved a **99%+ completion rate** on [AndroidWorld](https://github.com/google-research/android_world), Google Research's benchmark spanning 20+ apps and 100+ multi-step tasks.
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/androidworld_leaderboard.png?v=2" alt="AndroidWorld Benchmark Comparison" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## How ARTEMIS is Architected
|
|
||||||
|
|
||||||
* **Pre-Execution Checks and Action Bursts**: Pro checks the target against the live UI tree and pixels before dispatching an individual action. Action bursts handle transient controls without waiting for another model turn.
|
|
||||||
* **Element Locating**: Combines accessibility hierarchies and OCR with visual models for custom Canvas, Compose, and Flutter interfaces.
|
|
||||||
* **Shared History Compression**: Flash and Pro replace older screenshots with visual summaries and compress completed steps into searchable history chunks. Context thresholds control when raw turns are replaced.
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/artemis_architecture_diagram.png" alt="ARTEMIS System Architecture Diagram" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## Execution Profiles: Flash vs. Pro
|
|
||||||
|
|
||||||
ARTEMIS supports two execution profiles tailored for different automation requirements:
|
|
||||||
|
|
||||||
* **Flash Profile (`--profile flash`)**: Fast and token-efficient reactive loop (~3–5s per step): one model observes the live screen, thinks, and acts, with no graph orchestration. Ideal for routine, deterministic UI tasks. The loop is unbounded by default (`agent.flash.max_turns`, 0 = unlimited) because history is compressed rather than capped: Flash shares the Pro session transcript ledger (session-relative `T+mm:ss` clock, screenshots folded into visual summaries, older steps chunked into eras and recallable on demand via `search_history` / `replay_steps`) and can query the session recording through `video_analyzer`. Transient UI (auto-fading control bars, toasts) is handled by chaining taps into one `click_sequence`. *Limitations*: No task plan or notes, no pre-execution safety net, no checkpoint verification or final report, and no ADB shell.
|
|
||||||
* **Pro Profile (`--profile pro`)**: A planning and verification workflow (~15–40s per step), built as a multi-agent graph. A **Planner** maintains a living Markdown task plan with milestones and `verify` / `assert` check items; the **Operator** executes it with the full toolset (Explorer grounding whose `flash` / `pro` / `ultra` tier is a user setting per profile — `pro.explorer.mode` / `flash.explorer_mode` in `config/artemis.jsonc` or `--explorer-pro-mode` — never chosen by the agent; notes, history recall, video analysis, ADB diagnostics). Every single action passes a pre-execution **Safety Net** (XML-first, pixel fallback), while multi-action **fast-action bursts** fire back to back to beat turn latency on transient UI. A blocked or failed action opens an **execution incident** that stays in the Operator's context until a later action succeeds, so recovery is handled by the Operator itself with no separate repair agent. A read-only **Checker** verifies plan checkpoints and runs an exit final review against the original goal (`--verification-level`: `off` / `final` (default) / `checkpoints` / `strict`), and plan milestone edits get an advisory review. Handles 100+ step long-horizon workflows, `[Loop:continuous]` monitoring, and an optional written report.
|
|
||||||
|
|
||||||
## Roadmap
|
|
||||||
|
|
||||||
- [ ] **Android Studio Integration**: Native IDE plugin and workflow integration to enable in-editor debugging, test recording, and automated device control directly within Android Studio.
|
|
||||||
- [ ] **iOS Platform Expansion**: Extending multimodal perception and mobile automation to iOS devices and simulators.
|
|
||||||
- [ ] **On-Device Lightweight VLMs**: Local execution with lightweight edge vision models for low-latency, privacy-first automation.
|
|
||||||
- [ ] **Real-time Duplex Voice Interaction**: Voice-driven task dispatch with real-time conversational control and interruption handling.
|
|
||||||
|
|
||||||
## Community & Contributing
|
|
||||||
|
|
||||||
Contributions are warmly welcomed!
|
|
||||||
* **Star the repo** to follow updates and releases
|
|
||||||
* Join the [Discord Community](https://discord.gg/wF2FN4WHGY) for technical discussions
|
|
||||||
* Open an [Issue](https://github.com/google/artemis/issues) or submit a [Pull Request](https://github.com/google/artemis/pulls)
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is licensed under the [Apache License 2.0](LICENSE).
|
|
||||||
|
|
||||||
This project includes source code developed by [Minitap, Inc.](https://github.com/minitap-ai/mobile-use).
|
|
||||||
+2
-395
@@ -1,396 +1,3 @@
|
|||||||
> AI 接入：[全流程 MCP 与可分发 Skill](docs/ai-automation.md)。
|
# ARTEMIS-Eapil 中文说明
|
||||||
|
|
||||||
> 新增工作区功能：[Word/Excel 报告、Excel 批量任务与历史检索](docs/automation-workspace.md)。
|
项目文档已统一为简体中文，请阅读 [README](README.md)，了解项目功能、安装启动、模型配置和 AI 接入方法。
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/artemis-banner-cn.png?v=7" alt="ARTEMIS Banner" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<strong>让 AI 助手与测试套件像人一样直接操作真机。</strong>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<a href="./README.md">English</a> •
|
|
||||||
<a href="./README_CN.md"><b>中文文档</b></a> •
|
|
||||||
<a href="#workflow-showcase">全流程演示</a> •
|
|
||||||
<a href="#quick-start">快速上手</a> •
|
|
||||||
<a href="#mcp-setup">MCP 接入 IDE</a> •
|
|
||||||
<a href="#benchmarks">基准评测</a> •
|
|
||||||
<a href="https://discord.gg/wF2FN4WHGY">Discord 社区</a>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
|
||||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
|
||||||
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
|
||||||
<a href="https://developers.openai.com/api/docs/guides/responses"><img src="https://img.shields.io/badge/API-OpenAI%20Responses-4285F4.svg" alt="OpenAI Responses"></a>
|
|
||||||
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<!-- 演示效果图 -->
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/demo.gif" alt="Artemis 演示效果" width="100%" />
|
|
||||||
<br>
|
|
||||||
<em>实机演示：在 Google Maps 中设置驾车路线并计算总耗时，随后打开 YouTube 播放 Coldplay 的歌曲。</em>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## 核心亮点
|
|
||||||
|
|
||||||
* **跨 App 自动化**：根据自然语言指令，在 Android 上执行测试流程和日常任务。
|
|
||||||
* **多模态定位**：优先使用元素索引，对自绘界面提供坐标和视觉定位兜底。
|
|
||||||
* **IDE 内诊断**：通过 **MCP 协议**，让 **Antigravity、Claude Code、Windsurf** 操作测试设备，收集 **Logcat** 输出和截图。
|
|
||||||
* **Flash 执行**：使用观察、执行循环和异步历史摘要，单步通常约 **3–5 秒**。
|
|
||||||
* **Pro 探索**：单个动作执行前校验目标，被拦截的动作交回 Operator 处理。支持长时间探索和稳定性测试。
|
|
||||||
* **AndroidWorld 结果**：在 Google Research **AndroidWorld** 基准评测（100+ 多步任务）中取得 **99%+ 任务完成率**。
|
|
||||||
|
|
||||||
<a id="workflow-showcase"></a>
|
|
||||||
<a id="全流程演示"></a>
|
|
||||||
## Antigravity × ARTEMIS：全流程自主测试演示
|
|
||||||
|
|
||||||
**Antigravity** 通过 MCP 调用 **ARTEMIS**，将测试需求转为计划、真机操作和诊断报告：
|
|
||||||
|
|
||||||
<table width="100%">
|
|
||||||
<tr>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>1. 输入测试提示词 (Task Dispatch)</b><br>
|
|
||||||
<sub>在 Antigravity 中用自然语言描述测试需求与目标指标</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-1-prompt.png" width="100%" alt="步骤一：输入测试提示词" />
|
|
||||||
</td>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>2. 生成测试方案 (Test Plan Generation)</b><br>
|
|
||||||
<sub>自动拆解任务，生成详细测试步骤与架构图供确认</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-2-plan.png" width="100%" alt="步骤二：生成测试方案" />
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
<tr>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>3. 自主执行测试 (Autonomous Test Execution)</b><br>
|
|
||||||
<sub>驱动真机操作、界面导航并实时分析性能指标</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-3-exec.png" width="100%" alt="步骤三：自主执行测试" />
|
|
||||||
</td>
|
|
||||||
<td width="50%" align="center">
|
|
||||||
<b>4. 交付最终报告 (Comprehensive Final Report)</b><br>
|
|
||||||
<sub>生成结构化测试报告，交付性能图表、结论与原始数据</sub><br><br>
|
|
||||||
<img src="./docs/assets/workflow-4-report.png" width="100%" alt="步骤四：交付最终报告" />
|
|
||||||
</td>
|
|
||||||
</tr>
|
|
||||||
</table>
|
|
||||||
|
|
||||||
<a id="quick-start"></a>
|
|
||||||
<a id="快速上手"></a>
|
|
||||||
## 快速上手
|
|
||||||
|
|
||||||
确保电脑已连接 Android 实体机（已开启 **USB 调试**）或 Android 模拟器。一键启动脚本将会自动完成以下配置：
|
|
||||||
- **安装系统环境依赖**：自动检测并安装 ADB、scrcpy、FFmpeg 与 Python（`uv`）运行时及项目依赖。
|
|
||||||
- **全局挂载 MCP 服务与测试准则 (Rules)**：主动引导并自动将全局 MCP 服务与 **Artemis 移动端测试思维准则 (`rules.md`)** 挂载至你使用的 AI IDE（支持 **Antigravity**、**Cursor**、**Claude Code**、**Codex**、**Windsurf**、**VS Code**、**Cline/Roo**、**OpenClaw**）。
|
|
||||||
|
|
||||||
### macOS 与 Linux
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. 克隆代码仓库并进入目录
|
|
||||||
git clone https://github.com/google/artemis.git && cd artemis
|
|
||||||
|
|
||||||
# 2. 一键启动
|
|
||||||
./start.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Windows PowerShell
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
# 1. 克隆代码仓库并进入目录
|
|
||||||
git clone https://github.com/google/artemis.git
|
|
||||||
cd artemis
|
|
||||||
|
|
||||||
# 2. 一键启动
|
|
||||||
.\start.bat
|
|
||||||
```
|
|
||||||
|
|
||||||
> PowerShell 默认不会从当前目录查找可执行脚本，因此必须使用 `.\start.bat`，且命令末尾不要添加 `\`。如果使用传统命令提示符（CMD），则运行 `start.bat`。
|
|
||||||
|
|
||||||
首次启动会检测 uv：已有则复用，缺少则从 Astral 官方安装。随后检查项目 `.venv`，缺少时自动安装所需的 Python 3.12 并创建环境；已有且满足 Python 3.12+ 要求的环境直接复用。依赖与 `uv.lock` 一致时跳过安装，否则只同步缺失或变化的依赖，保留已有开发工具。安装失败会停止启动并显示原因，不会删除已有环境。
|
|
||||||
|
|
||||||
国内网络可直接双击 **`start-cn.bat`**，或在 PowerShell 中运行：
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\start-cn.bat
|
|
||||||
# 仅安装 Python 环境，也支持镜像模式
|
|
||||||
.\start-cn.bat -SetupOnly
|
|
||||||
# 与上面的入口等价
|
|
||||||
.\start.bat -Mirror China
|
|
||||||
# 本次使用官方源
|
|
||||||
.\start.bat -Mirror Official
|
|
||||||
```
|
|
||||||
|
|
||||||
下载配置位于 `config/downloads.json`。将 `mode` 改成 `China` 后，普通 `start.bat` 也默认使用国内源；命令行 `-Mirror` 优先。`timeout_seconds` 控制下载超时（默认 180 秒，范围 10～3600 秒）。国内模式使用：
|
|
||||||
|
|
||||||
| 组件 | 下载来源 |
|
|
||||||
| --- | --- |
|
|
||||||
| uv 与 Python 依赖 | 清华 PyPI 镜像；uv 直接从 wheel 提取并校验 SHA256，无需预装 Python |
|
|
||||||
| Python 运行时 | 南京大学 python-build-standalone 镜像 |
|
|
||||||
| Node.js 与 npm 包 | npmmirror |
|
|
||||||
| ADB | 腾讯 Android SDK 镜像 |
|
|
||||||
| FFmpeg | `imageio-ffmpeg` Python 包附带的程序 |
|
|
||||||
| scrcpy | 默认仍使用 GitHub 官方发布，可用下述配置替换 |
|
|
||||||
|
|
||||||
镜像模式从 `uv.lock` 导出固定版本和哈希，再通过所选索引安装，避免锁文件中的官方直链绕过镜像；不会改写锁文件或升级已锁定版本。npm 使用本次启动进程的 registry 配置。所有设置仅影响当前启动进程，不修改用户全局 pip、uv、npm 或 nvm 配置。已有 `UV_DEFAULT_INDEX`（或 `UV_INDEX_URL`）、`UV_PYTHON_INSTALL_MIRROR`、`npm_config_registry` 环境变量优先。
|
|
||||||
|
|
||||||
需要公司内网源或替换不可用镜像时，在 `config/downloads.json` 的 `overrides` 中填写对应 HTTPS 地址；空字符串表示使用所选模式的默认值。`python_index` 是以 `/simple` 结尾的索引，`python_download_mirror` 是 Python 发布目录，`node_mirror` 是包含版本子目录的 Node.js 下载根路径，`adb_url` 和 `scrcpy_url` 是可直接下载的 Windows ZIP 地址。scrcpy 暂无内置国内镜像，GitHub 不可达时请配置可信的 `scrcpy_url` 或预先安装 scrcpy。`uv_installer_url` 只用于没有设置 Python 镜像时的 PowerShell 安装器路径。镜像缺包或下载失败会显示错误；修正地址后重试即可，不会静默切换到官方源。
|
|
||||||
|
|
||||||
镜像说明参考：[清华 PyPI](https://mirrors.tuna.tsinghua.edu.cn/help/pypi/)、[uv 下载镜像配置](https://docs.astral.sh/uv/reference/environment/#uv_python_install_mirror)。
|
|
||||||
|
|
||||||
也可先单独准备 Python 环境，不启动页面、不安装设备工具、不配置 MCP：
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\start.bat -SetupOnly
|
|
||||||
# 环境准备好后正常启动
|
|
||||||
.\start.bat
|
|
||||||
# 可选：指定端口、不自动打开浏览器
|
|
||||||
.\start.bat -Port 8001 -NoOpen
|
|
||||||
```
|
|
||||||
|
|
||||||
源码压缩包不携带 `.venv`、`node_modules`、前端编译结果或 APK。解压到可写目录后运行 `start.bat`，首次完整启动会安装所需 Node.js/npm 并构建网页，需要联网；已有构建时跳过。`.env` 不存在时才从 `.env.example` 创建，已有配置不会覆盖。Android 辅助 APK 的源码位于 `packages/artemis-accessibility-helper`，需要该功能时按该目录的构建脚本生成。
|
|
||||||
|
|
||||||
重新打源码包可运行 `uv run --no-sync python scripts/package_source.py`，输出到 `artifacts/`；脚本会排除本地密钥、运行数据、缓存、依赖目录与编译产物，并检查 ZIP 完整性。
|
|
||||||
|
|
||||||
> **提示**：启动后将自动在默认浏览器中打开 Web 控制台（`http://localhost:8000`），提供设备连接向导、实时投屏、任务演练与状态回放面板。你也可以通过命令行直接运行：`uv run artemis run "打开系统设置，找到电池选项并告诉我当前电量" --profile flash`。
|
|
||||||
|
|
||||||
### 已安装用户自动更新
|
|
||||||
|
|
||||||
再次运行 `start.bat`（或国内镜像 `start-cn.bat`）会检查 Git 上游，安全快进到最新源码，再检查 Python 依赖、前端源码/包锁文件和已有 MCP 配置。更新后需要在 AI 客户端重新加载 MCP。源码有未提交修改、分支分叉或网络不可用时保留本地版本；正在执行任务时先停止更新启动，不打断原任务。
|
|
||||||
|
|
||||||
ZIP 解压目录没有 `.git`，会提示改用 Git 克隆以获得自动源码更新，并继续使用当前源码；依赖和已有 MCP 配置仍会检查。`start.bat -SkipUpdate` 仅跳过 Git 检查。完整行为见 [AI 接入与更新说明](docs/ai-automation.md)。
|
|
||||||
|
|
||||||
### 网页模型配置与功能开关
|
|
||||||
|
|
||||||
Web 控制台使用简体中文。在首页打开「系统设置与运行准备」，可以修改接口地址、模型名称、API Key、思考强度、请求超时、重试次数和间隔。点击「测试连接」会使用当前表单发送一次真实的 Responses 请求；点击「保存配置」才会写入工作目录的 `.env`。密钥留空会保留现有值，网页不会读取或显示已有密钥。
|
|
||||||
|
|
||||||
所有模型调用统一使用 **OpenAI Responses API**，不再提供 Gemini、Anthropic 或 Chat Completions 调用路径。自定义网关必须实现 `/responses`，接口地址填写根地址（通常以 `/v1` 结尾）。例如：
|
|
||||||
|
|
||||||
```dotenv
|
|
||||||
OPENAI_API_KEY=你的密钥
|
|
||||||
OPENAI_BASE_URL=https://你的网关/v1
|
|
||||||
OPENAI_MODEL=mimo-v2.6-flash
|
|
||||||
OPENAI_TIMEOUT_SECONDS=300
|
|
||||||
OPENAI_MAX_RETRIES=5
|
|
||||||
OPENAI_RETRY_INTERVAL_SECONDS=10
|
|
||||||
OPENAI_REASONING_EFFORT=high
|
|
||||||
ARTEMIS_VIDEO_ENABLED=false
|
|
||||||
```
|
|
||||||
|
|
||||||
模型可选择 `gpt-5.6-luna`、`mimo-v2.6-flash` 或输入自定义名称。思考强度默认 high，接口不支持时明确报错，可手动选择“由模型默认”。
|
|
||||||
|
|
||||||
「功能开关」提供默认关闭的视频分析与录屏、最终报告生成、Flash 步骤摘要压缩，可分别保存。设置对**新任务**生效，重启后保留。环境变量优先于 `.env`；在网页保存时会同时更新当前服务进程的配置，服务的启动环境若显式定义了同名变量，后续重启仍以启动环境为准。手动配置的节点模型或任务级选项优先于全局默认值。
|
|
||||||
|
|
||||||
视频分析通过抽取画面帧调用模型，因此需要支持图片输入的模型。`mimo-v2.6-flash` 已验证文本、图片、工具调用、流式输出及结构化函数调用；对于不支持 `json_schema` 的网关，结构化结果使用 Responses 函数调用并在本地校验。音频输入是否可用取决于端点能力，拒绝音频时视频分析会尝试仅分析画面。
|
|
||||||
|
|
||||||
视频分段时长、分析超时、检查点超时等配置及单位见 [`.env.example`](.env.example)。可选 Cloud Vision OCR 独立于模型调用，未配置 OCR 密钥也可运行。
|
|
||||||
|
|
||||||
<a id="mcp-setup"></a>
|
|
||||||
<a id="mcp"></a>
|
|
||||||
<details>
|
|
||||||
<summary><b>Codex / Antigravity / Claude Code / Windsurf MCP 配置（点击展开）</b></summary>
|
|
||||||
|
|
||||||
<br>
|
|
||||||
|
|
||||||
ARTEMIS 内置原生 **Model Context Protocol (MCP)** 服务。只需将以下配置加入你的 IDE 配置文件中，即可在编写代码时直接驱动真机：
|
|
||||||
|
|
||||||
### 1. 一键自动安装到 IDE（推荐）
|
|
||||||
|
|
||||||
运行 `./start.sh`（macOS/Linux）或 `.\start.bat`（Windows PowerShell）启动脚本时，会主动询问是否自动挂载全局 MCP 与测试行为准则（支持跳过并在之后随时手动执行以下命令挂载）：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 一键安装全局 MCP 服务与 Rules 到 Antigravity / Jetski：
|
|
||||||
uv run artemis mcp --install antigravity
|
|
||||||
|
|
||||||
# 或一键安装到所有支持的 AI IDE（包括 Codex）：
|
|
||||||
uv run artemis mcp --install all
|
|
||||||
```
|
|
||||||
|
|
||||||
> **提示**：你也可以在首次运行 `uv run artemis init` 配置向导时，交互式完成 IDE 的 MCP 自动挂载。
|
|
||||||
> **进阶提示**：如果希望在任意目录下都不需要加 `uv run` 就能全局直接使用 `artemis` 命令，可在项目根目录下执行一次 `uv tool install -e .`。
|
|
||||||
|
|
||||||
### 2. 手动配置（可选）
|
|
||||||
|
|
||||||
如果你习惯手动复制配置，可运行 `uv run artemis mcp --generate-config <client>`（例如 `codex` 或 `antigravity`）获取对应的 TOML 或 JSON 配置。请将 `command` 填写为项目下 `.venv` 虚拟环境中的 Python 绝对路径，并将 `/path/to/artemis` 替换为项目实际路径：
|
|
||||||
|
|
||||||
* **Codex** (`~/.codex/config.toml`)：
|
|
||||||
```toml
|
|
||||||
[mcp_servers.artemis]
|
|
||||||
command = "/path/to/artemis/.venv/bin/python"
|
|
||||||
args = ["-m", "mcp_server"]
|
|
||||||
cwd = "/path/to/artemis"
|
|
||||||
|
|
||||||
[mcp_servers.artemis.env]
|
|
||||||
PYTHONUNBUFFERED = "1"
|
|
||||||
PYTHONPATH = "/path/to/artemis"
|
|
||||||
```
|
|
||||||
|
|
||||||
* **Antigravity** (`~/.gemini/jetski/mcp_config.json`)：
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"artemis": {
|
|
||||||
"command": "/path/to/artemis/.venv/bin/python",
|
|
||||||
"args": ["-m", "mcp_server"],
|
|
||||||
"cwd": "/path/to/artemis",
|
|
||||||
"env": {
|
|
||||||
"PYTHONUNBUFFERED": "1"
|
|
||||||
},
|
|
||||||
"tools": {
|
|
||||||
"mobile_run_task": { "eager": true },
|
|
||||||
"mobile_manage_task": { "eager": true },
|
|
||||||
"mobile_get_device_state": { "eager": true },
|
|
||||||
"mobile_inspect_trace": { "eager": true },
|
|
||||||
"mobile_diagnose": { "eager": true }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
* **Claude Desktop** (`claude_desktop_config.json`)：
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"artemis": {
|
|
||||||
"command": "/path/to/artemis/.venv/bin/python",
|
|
||||||
"args": ["-m", "mcp_server"],
|
|
||||||
"cwd": "/path/to/artemis"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. 挂载智能体行为规范 Rules（强烈推荐）
|
|
||||||
|
|
||||||
为使 AI 编程助手具备资深移动端测试工程师的严谨思维，避免凭空臆测 UI 交互，我们提供了专属的测试思维行为规范文件 [`mcp_server/rules.md`](./mcp_server/rules.md)（涵盖**可运行代码原则与真机探索**、**Flash 与 Pro 任务路由策略**、**延迟与时间补偿机制**以及**“动态优先、坐标兜底”定位模式**）。
|
|
||||||
|
|
||||||
你可以将 [`mcp_server/rules.md`](./mcp_server/rules.md) 挂载或复制到你的 AI IDE 规则配置中：
|
|
||||||
* **Antigravity**：将 `rules.md` 内容添加至工作区规则（Workspace Rules）或全局规则设置中。
|
|
||||||
* **Claude Code**：运行 `artemis mcp --install claude` 自动安装规则至 `~/.claude/rules/artemis.md`（只安装到单一位置——Claude Code 会同时加载 `~/.claude/CLAUDE.md` 与 `~/.claude/rules/*.md`，重复安装会浪费上下文）。
|
|
||||||
* **Cursor**：将内容复制到 `.cursorrules` 文件或在 `.cursor/rules/artemis.mdc` 中创建新规则。
|
|
||||||
* **Codex**：将内容添加至 `~/.codex/AGENTS.md`（或当前生效的 `AGENTS.override.md`）。
|
|
||||||
* **Windsurf / OpenClaw**：将内容添加到工作区规则或全局 System Prompt 中。
|
|
||||||
|
|
||||||
> 更多规范设计细节与 MCP 架构说明，请参阅 [MCP Server 文档](./mcp_server/README.md)。
|
|
||||||
|
|
||||||
### 4. 在 IDE 中体验真机协同
|
|
||||||
在 Codex / Antigravity / Claude Code 对话框中直接输入：
|
|
||||||
> *"请帮我把刚刚修改的代码编译成 APK 并安装到手机上，打开登录页面输入测试账号，验证登录后是否有异常弹窗，并把最终页面截图回传。"*
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
<a id="python-sdk"></a>
|
|
||||||
<details>
|
|
||||||
<summary><b>Python SDK 集成（点击展开）</b></summary>
|
|
||||||
|
|
||||||
<br>
|
|
||||||
|
|
||||||
开发电脑只需安装零运行时依赖的薄客户端；ADB、Agent、模型与图像处理全部留在设备主机：
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
uv add "artemis-client @ git+https://github.com/google/artemis.git#subdirectory=packages/artemis-client"
|
|
||||||
```
|
|
||||||
|
|
||||||
```python
|
|
||||||
import asyncio
|
|
||||||
from artemis_client import ArtemisClient
|
|
||||||
|
|
||||||
|
|
||||||
async def main():
|
|
||||||
client = ArtemisClient(
|
|
||||||
"http://artemis-host:8000",
|
|
||||||
device_serial="emulator-5554", # 可选：指定目标设备序列号（不传则自动选择空闲设备）
|
|
||||||
default_profile="flash", # "flash" 极速校验 或 "pro" 深度推理自愈
|
|
||||||
)
|
|
||||||
|
|
||||||
result = await client.run(
|
|
||||||
"打开系统设置，进入『电池』页面，验证是否正常显示电量百分比，确认页面无异常报错弹窗。",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert result.succeeded, f"测试执行失败: {result.error or result.status}"
|
|
||||||
print(f"✅ 测试通过！设备: {result.device_serial} | Trace ID: {result.trace_id}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
asyncio.run(main())
|
|
||||||
```
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
## 使用方式
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/artemis-ui-showcase.png" alt="Artemis 可视化控制台" width="100%" />
|
|
||||||
<br />
|
|
||||||
<sub><b>控制台功能概览</b>：<b>① 视图切换</b>（主页与工作区） · <b>② 运行模式与回放</b>（Flash/Pro 状态与视频回放） · <b>③ 实时感知推理流</b>（动作感知、目标坐标与结构化总结） · <b>④ 提示词输入坞 (Prompt Dock)</b>（自然语言下发） · <b>⑤ 任务队列看板</b>（生命周期与历史回溯）</sub>
|
|
||||||
</p>
|
|
||||||
|
|
||||||
* **Web 可视化测试控制台 (`uv run artemis ui`)**：集成设备实时投屏与交互面板，支持通过自然语言下发测试用例，实时观测推理步骤、操作轨迹、截图留存与异常状态回放；支持在任意终端使用 `uv run artemis restart`、`uv run artemis stop`、`uv run artemis status` 一键重启、关停或查看服务状态；
|
|
||||||
* **原生 MCP 协议 (IDE 协同)**：作为标准 MCP 服务器接入 **Antigravity、Claude Code、Windsurf** 等开发环境，在 IDE 中直接驱动真机完成自动化测试与 Bug 复现验证；
|
|
||||||
* **开发者命令行 CLI (`uv run artemis run`)**：支持通过终端直接执行自动化测试用例、探索性稳定性巡检或 AndroidWorld 基准评测，提供高保真结构化终端输出；
|
|
||||||
* **Python SDK**：作为标准 Python 库集成至现有自动化测试框架（如 pytest）或 CI/CD 流水线，提供基于 Pydantic 的强类型结构化结果与断言支持。
|
|
||||||
|
|
||||||
<a id="on-device-helper"></a>
|
|
||||||
## ARTEMIS 会在手机上安装什么
|
|
||||||
|
|
||||||
首次在某台设备上执行任务时，ARTEMIS 会安装 **Artemis Accessibility Helper**：一个用于读取屏幕布局的小型无障碍服务，
|
|
||||||
不占用 UiAutomation 连接。使用 UiAutomation 的工具需要启用 `FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES`，否则可能使该服务暂停运行。
|
|
||||||
手机上会出现一条折叠的"Artemis test helper is running"通知，以及"设置 > 无障碍"里的一个新条目，都是它。
|
|
||||||
它只监听手机本机回环地址，不向外发送任何数据。
|
|
||||||
|
|
||||||
* 预先安装（避免首个任务多等约 3 秒）：`uv run artemis helper install`
|
|
||||||
* 查看状态：`uv run artemis helper status` / `uv run artemis doctor`
|
|
||||||
* 随时移除：`uv run artemis helper uninstall`
|
|
||||||
* 改用 UIAutomator2：在 `.env` 中设置 `ARTEMIS_HIERARCHY_BACKEND=uiautomator`
|
|
||||||
* 禁止自动安装：在 `.env` 中设置 `ARTEMIS_HELPER_AUTO_INSTALL=false`
|
|
||||||
|
|
||||||
任务中途 helper 失效时，ARTEMIS 会回退到 UIAutomator2，并在任务时间线、`mobile_manage_task` 状态和最终报告里明确说明。
|
|
||||||
|
|
||||||
<a id="benchmarks"></a>
|
|
||||||
<a id="基准评测"></a>
|
|
||||||
## 基准评测：AndroidWorld (SOTA 99%+)
|
|
||||||
|
|
||||||
Artemis 在 Google Research 发布的 [AndroidWorld](https://github.com/google-research/android_world) 基准评测中取得 **99%+ 任务完成率**。该评测涵盖 20+ 款应用与 100+ 项多步任务。
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/androidworld_leaderboard.png?v=2" alt="AndroidWorld 评测基准对比" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## ARTEMIS 是如何建构的
|
|
||||||
|
|
||||||
* **执行前校验与动作序列**：Pro 在下发单个动作前，通过实时 UI 树和像素校验目标。快速动作序列用于处理瞬态控件，无需等待下一轮模型响应。
|
|
||||||
* **元素定位**：结合无障碍树、OCR 和视觉模型，支持 Canvas、Compose、Flutter 等自绘界面。
|
|
||||||
* **共享历史压缩**：Flash 和 Pro 将较早的截图替换为视觉摘要，并把已完成步骤压缩为可检索的历史分块。上下文阈值决定何时替换原始记录。
|
|
||||||
|
|
||||||
<p align="center">
|
|
||||||
<img src="./docs/assets/artemis_architecture_diagram.png" alt="ARTEMIS 架构系统示意图" width="100%" />
|
|
||||||
</p>
|
|
||||||
|
|
||||||
## 运行模式：Flash vs. Pro
|
|
||||||
|
|
||||||
ARTEMIS 提供两种运行模式以适应不同的自动化需求：
|
|
||||||
|
|
||||||
* **Flash 模式 (`--profile flash`)**：使用历史压缩的响应式循环（单步约 3–5 秒）：单个模型观察实时屏幕、思考、执行，不经过图编排，适合常规确定性 UI 操作。默认不限步数（`agent.flash.max_turns`，0 表示不限），因为历史是被压缩而不是被截断：Flash 与 Pro 共用同一套会话记录账本（相对测试时间 `T+mm:ss`、截图折叠为视觉摘要、更早的步骤分块归档并可通过 `search_history` / `replay_steps` 按需召回），并可调用 `video_analyzer` 分析整段会话录屏；对自动消失的控制栏、toast 等瞬态控件，用 `click_sequence` 把多次点击串成一个原子序列。*局限性*：没有任务计划与笔记、没有执行前安全网、没有检查点校验与最终报告、不能执行 ADB 命令。
|
|
||||||
* **Pro 模式 (`--profile pro`)**：包含计划和校验的执行流程（单步约 15–40 秒），由多智能体图编排：**Planner** 维护一份带里程碑和 `verify` / `assert` 检查项的 Markdown 任务计划；**Operator** 按计划执行，拥有全套工具（Explorer 元素定位，其 `flash` / `pro` / `ultra` 三档感知深度是按运行模式设置的用户配置——`config/artemis.jsonc` 里的 `pro.explorer.mode` / `flash.explorer_mode` 或 `--explorer-pro-mode`——智能体自身不会选择档位；笔记、历史召回、视频分析、ADB 诊断）。单个动作执行前都经过 **Safety Net** 校验（XML 优先、像素兜底），而一轮里的多个动作则作为**快速动作序列 (fast-action burst)** 连发，赶在瞬态控件消失前完成操作。动作被拦截或失败时会打开一条**执行事故 (execution incident)**，持续留在 Operator 的上下文里直到后续动作执行成功，恢复由 Operator 自己完成，不再有独立的修复智能体。只读的 **Checker** 校验计划中的检查点，并在退出前对照原始目标做终审（`--verification-level`：`off` / `final`（默认）/ `checkpoints` / `strict`），计划里程碑的改动会得到建议式复核。支持 100+ 步的长程复杂业务流、`[Loop:continuous]` 持续监控以及可选的书面报告。
|
|
||||||
|
|
||||||
## 路线图
|
|
||||||
|
|
||||||
- [ ] **Android Studio 深度集成**：推出官方 IDE 插件与协同工作流，支持在 Android Studio 内直接进行自动化测试、设备交互与断点调试。
|
|
||||||
- [ ] **iOS 跨平台支持**：将视觉感知与自动化执行引擎拓展至 iOS 真机与模拟器。
|
|
||||||
- [ ] **端侧轻量化模型**：支持离线运行的轻量级 Edge VLM，实现低延迟与隐私安全的本地自动化。
|
|
||||||
- [ ] **实时语音双工交互**：支持自然语音下发任务与实时打断（Barge-in）控制。
|
|
||||||
|
|
||||||
## 社区与贡献
|
|
||||||
|
|
||||||
欢迎通过以下方式参与项目建设：
|
|
||||||
* **Star 本项目**以关注最新进展与更新
|
|
||||||
* 加入 [Discord 社区](https://discord.gg/wF2FN4WHGY) 参与技术探讨与功能建议
|
|
||||||
* 提交 [Issue](https://github.com/google/artemis/issues) 反馈 Bug，欢迎发起 [Pull Request](https://github.com/google/artemis/pulls) 贡献代码
|
|
||||||
|
|
||||||
## 开源许可证
|
|
||||||
|
|
||||||
本项目基于 [Apache License 2.0](LICENSE) 协议开源。
|
|
||||||
|
|
||||||
本项目包含由 [Minitap, Inc.](https://github.com/minitap-ai/mobile-use) 开发的源代码。
|
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# 前端协作指南
|
||||||
|
|
||||||
|
## Local Purpose
|
||||||
|
|
||||||
|
本目录维护简体中文 Angular 网页，包括设备与任务操作、模型配置、报告、批量导入和历史检索。
|
||||||
|
|
||||||
|
## Inherits
|
||||||
|
|
||||||
|
继承仓库根目录 AGENTS.md；本文件仅补充前端规则，当前用户指令优先。
|
||||||
|
|
||||||
|
## Local Commands
|
||||||
|
|
||||||
|
在本目录执行，包管理器使用 npm 和现有 package-lock.json：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm ci
|
||||||
|
npm start
|
||||||
|
```
|
||||||
|
|
||||||
|
完成前端改动后执行以下检查；无头测试需要已安装 Chrome：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
npm test -- --watch=false --browsers=ChromeHeadless
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
开发服务器通过 proxy.conf.json 连接本地后端；需要联调时同时启动根目录的网页服务。
|
||||||
|
|
||||||
|
## Local Conventions
|
||||||
|
|
||||||
|
- 页面组件在 src/app/components，服务在 src/app/services；先复用已有组件、样式及 API 服务。
|
||||||
|
- 保持现有简体中文文案、颜色、间距和控件样式，菜单、弹窗和表单必须清晰可读。
|
||||||
|
- 修改任务或配置交互时覆盖加载、保存、失败和空状态；不要将执行中误标成录屏中或把完成误标成验证通过。
|
||||||
|
- 使用相邻的 .spec.ts 测试交互和请求数据；不要只测试实现细节。
|
||||||
|
- 后端字段调整需同时核对类型、请求和响应；设备序列号、报告格式、批次编号等不能在前端猜测。
|
||||||
|
- UI 有明显变化时在浏览器验证实际布局和关键操作，构建通过不能代替可读性检查。
|
||||||
|
|
||||||
|
## Local Risks
|
||||||
|
|
||||||
|
- 不读取或展示已保存密钥；模型配置的空密钥表示保留原值，测试连接与保存配置保持分离。
|
||||||
|
- 不为查看历史而启动真实任务，不自动重试业务用例，不恢复已移除的设备动作回放。
|
||||||
|
- 不编辑 dist 或 node_modules；提交源码及必要锁文件。源码包不携带编译后的网页。
|
||||||
Reference in new issue
Block a user