From 5ada543a094af964d1a594621f21aac034a4dcb8 Mon Sep 17 00:00:00 2001 From: somew1nd Date: Tue, 25 Aug 2026 17:09:10 -0700 Subject: [PATCH] docs: standardize image embedding widths to 100% and streamline emoji usage --- README.md | 104 ++++++++++++++++++++++++++------------------------- README_CN.md | 68 +++++++++++++++++---------------- 2 files changed, 88 insertions(+), 84 deletions(-) diff --git a/README.md b/README.md index 8c1f01f..b0dd199 100644 --- a/README.md +++ b/README.md @@ -26,44 +26,46 @@

- Artemis in Action + Artemis in Action +
+ Live Demo: Setup driving routes and calculate total durations in Google Maps, then open YouTube to play a Coldplay song.

-## ✨ Key Highlights +## Key Highlights -* 🤖 **Cross-App Automation & Autonomous AI Assistant**: Operates not just as a robust testing framework, but as an autonomous agent capable of handling complex cross-app workflows and daily tasks via natural language; -* 🧪 **Zero-Maintenance Test Automation**: Built upon a "Dynamic-First, Coordinate-Fallback" multimodal locating engine, eliminating fragile XPath/ID selector maintenance and remaining resilient to UI redesigns, system updates, and resolution drift; -* 🐞 **One-Click Bug Repro & Logcat Diagnostics in IDE**: Native **Model Context Protocol (MCP)** integration allows **Antigravity, Claude Code, and Windsurf** to drive physical test devices via natural language, automatically capturing crash stacks from **Logcat** and keyframe screenshots; -* ⚡ **Ultra-Fast Execution (3–5s per Step)**: Pioneered an **Optimistic Asynchronous Pipeline** that completely decouples UI interaction from heavy LLM reasoning, achieving rapid regression throughput in Flash mode; -* 🛡️ **Popup Self-Healing & 10+ Hour Exploration**: Proprietary **Safety Net** double-checks targets before action execution to intercept and clear interfering system popups; Pro mode supports **10+ hours** of continuous exploratory & monkey-plus stability testing; -* 🏆 **Industry-Leading SOTA**: Achieved **99%+ task completion** on Google Research's **AndroidWorld** benchmark (100+ complex multi-step tasks). +* **Cross-App Automation & Autonomous AI Assistant**: Operates not just as a robust testing framework, but as an autonomous agent capable of handling complex cross-app workflows and daily tasks via natural language; +* **Zero-Maintenance Test Automation**: Built upon a "Dynamic-First, Coordinate-Fallback" multimodal locating engine, eliminating fragile XPath/ID selector maintenance and remaining resilient to UI redesigns, system updates, and resolution drift; +* **One-Click Bug Repro & Logcat Diagnostics in IDE**: Native **Model Context Protocol (MCP)** integration allows **Antigravity, Claude Code, and Windsurf** to drive physical test devices via natural language, automatically capturing crash stacks from **Logcat** and keyframe screenshots; +* **Ultra-Fast Execution (3–5s per Step)**: Pioneered an **Optimistic Asynchronous Pipeline** that completely decouples UI interaction from heavy LLM reasoning, achieving rapid regression throughput in Flash mode; +* **Popup Self-Healing & 10+ Hour Exploration**: Proprietary **Safety Net** double-checks targets before action execution to intercept and clear interfering system popups; Pro mode supports **10+ hours** of continuous exploratory & monkey-plus stability testing; +* **Industry-Leading SOTA**: Achieved **99%+ task completion** on Google Research's **AndroidWorld** benchmark (100+ complex multi-step tasks). -## 🤝 Antigravity × ARTEMIS: Autonomous Testing Workflow +## Antigravity × ARTEMIS: Autonomous Testing Workflow Experience seamless collaboration between **Antigravity** and **ARTEMIS** via native MCP integration — taking you from a natural language requirement to a production-grade diagnostic report in four automated steps: @@ -71,13 +73,13 @@ Experience seamless collaboration between **Antigravity** and **ARTEMIS** via na
- 1️⃣ Prompt Input (Task Dispatch)
+ 1. Prompt Input (Task Dispatch)
Describe your test scenario and target metrics in Antigravity

Step 1: Prompt Input in Antigravity 		- 2️⃣ Test Plan Generation
+ 2. Test Plan Generation
Formulates a step-by-step test plan & architecture for review

Step 2: Test Plan Generation
- 3️⃣ Autonomous Test Execution
+ 3. Autonomous Test Execution
Drives real device, navigates UI, and profiles performance

Step 3: Autonomous Test Execution 		- 4️⃣ Comprehensive Final Report
+ 4. Comprehensive Final Report
Delivers structured audit findings, metric tables, and raw datasets

Step 4: Final Report
-## ⚡ Quick Start +## Quick Start Ensure an Android device (with **USB Debugging** enabled) or emulator is connected. The one-click startup script will automatically: -- 🛠️ **Install System Toolchains**: Detect and auto-install ADB, scrcpy, FFmpeg, and Python (`uv`) dependencies. -- 🔌 **Mount Global MCP Server & AI Agent Rules**: Prompt to automatically install global MCP configurations and the **Artemis Mobile Testing Mindset (`rules.md`)** into your AI IDEs (**Antigravity**, **Cursor**, **Claude Code**, **Codex**, **Windsurf**, **VS Code**, **Cline/Roo**, **OpenClaw**). +- **Install System Toolchains**: Detect and auto-install ADB, scrcpy, FFmpeg, and Python (`uv`) dependencies. +- **Mount Global MCP Server & AI Agent Rules**: Prompt to automatically install global MCP configurations and the **Artemis Mobile Testing Mindset (`rules.md`)** into your AI IDEs (**Antigravity**, **Cursor**, **Claude Code**, **Codex**, **Windsurf**, **VS Code**, **Cline/Roo**, **OpenClaw**). -### 🍎 macOS and 🐧 Linux +### macOS and Linux ```bash # 1. Clone repo & navigate to directory @@ -87,7 +89,7 @@ git clone https://github.com/google/artemis.git && cd artemis ./start.sh ``` -### 🪟 Windows PowerShell +### Windows PowerShell ```powershell # 1. Clone repo & navigate to directory @@ -100,12 +102,12 @@ cd artemis > PowerShell does not search the current directory for executable scripts by default, so use `.\start.bat` without a trailing `\`. In Command Prompt (CMD), use `start.bat` instead. -> 💡 **Tip**: Opens `http://localhost:8000` in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: `uv run artemis run "Open Settings, find Battery and tell me current level" --profile flash`. +> **Tip**: Opens `http://localhost:8000` in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: `uv run artemis run "Open Settings, find Battery and tell me current level" --profile flash`.
-🔌 MCP Setup for Codex / Antigravity / Claude Code / Windsurf (Click to expand) +MCP Setup for Codex / Antigravity / Claude Code / Windsurf (Click to expand)
@@ -123,7 +125,7 @@ uv run artemis mcp --install antigravity uv run artemis mcp --install all ``` -> 💡 **Tip**: You can also configure MCP interactively during first-time setup via `uv run artemis init`. +> **Tip**: You can also configure MCP interactively during first-time setup via `uv run artemis init`. > **Pro Tip**: If you want to use the `artemis` command globally without `uv run` in any directory, run `uv tool install -e .` once in the project root. ### 2. Manual Configuration (Optional) @@ -188,17 +190,17 @@ You can mount or copy [`mcp_server/rules.md`](./mcp_server/rules.md) into your A * **Codex**: Add the contents to `~/.codex/AGENTS.md` (or the active `AGENTS.override.md`). * **Windsurf / OpenClaw**: Add the rules to your workspace rules or global system prompts. -> 💡 For more details on the testing mindset and MCP architecture, see the [MCP Server README](./mcp_server/README.md). +> For more details on the testing mindset and MCP architecture, see the [MCP Server README](./mcp_server/README.md). ### 4. Prompt Your Phone in the IDE Chat In Codex, Antigravity, or Claude Code, simply prompt: -> 💬 *"Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, verify if there are any unexpected popups after login, and return screenshots of the final page."* +> *"Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, verify if there are any unexpected popups after login, and return screenshots of the final page."*
-🐍 Python SDK Integration (Click to expand) +Python SDK Integration (Click to expand)
@@ -229,31 +231,31 @@ if __name__ == "__main__":
-## 🕹️ Usage Modes +## Usage Modes

- Artemis Web Console + Artemis Web Console
- 💡 Console Overview: ① View Switcher (Home / Workspace) · ② Model & Replay (Flash/Pro status & video replay) · ③ Live Agent Stream (Action perception, target coordinates & structured results) · ④ Prompt Dock (Natural language dispatch) · ⑤ Task Queue & Dashboard (Lifecycle & history) + Console Overview: ① View Switcher (Home / Workspace) · ② Model & Replay (Flash/Pro status & video replay) · ③ Live Agent Stream (Action perception, target coordinates & structured results) · ④ Prompt Dock (Natural language dispatch) · ⑤ Task Queue & Dashboard (Lifecycle & history)

-* 🖥️ **Web Visual Test Console (`uv run artemis ui`)**: Real-time screen projection and interactive panel, supporting natural language test dispatch, live reasoning telemetry, action trajectories, and execution replay; -* 🔌 **Native MCP Protocol (IDE Collaboration)**: Operates as a standard MCP server seamlessly integrating with **Antigravity, Claude Code, Windsurf**, etc., directly driving real devices inside the IDE to verify bugs and run test cases; -* 💻 **Developer CLI (`uv run artemis run`)**: Direct terminal execution for automated test cases, exploratory stability inspection, or AndroidWorld benchmarks with high-fidelity structured terminal output; -* 🐍 **Python SDK**: Integrates as a standard Python library into existing automated testing frameworks (e.g., pytest) or CI/CD pipelines with strongly typed Pydantic structured outputs and assertion support. +* **Web Visual Test Console (`uv run artemis ui`)**: Real-time screen projection and interactive panel, supporting natural language test dispatch, live reasoning telemetry, action trajectories, and execution replay; +* **Native MCP Protocol (IDE Collaboration)**: Operates as a standard MCP server seamlessly integrating with **Antigravity, Claude Code, Windsurf**, etc., directly driving real devices inside the IDE to verify bugs and run test cases; +* **Developer CLI (`uv run artemis run`)**: Direct terminal execution for automated test cases, exploratory stability inspection, or AndroidWorld benchmarks with high-fidelity structured terminal output; +* **Python SDK**: Integrates as a standard Python library into existing automated testing frameworks (e.g., pytest) or CI/CD pipelines with strongly typed Pydantic structured outputs and assertion support. -## 📊 Head-to-Head Comparison +## Head-to-Head Comparison -| Evaluation Dimension | Traditional Test Automation (Appium / Maestro) | Generic Mobile VLM Agents | **ARTEMIS ☕ (Next-Gen AI Testing)** | +| Evaluation Dimension | Traditional Test Automation (Appium / Maestro) | Generic Mobile VLM Agents | **ARTEMIS (Next-Gen AI Testing)** | | :--- | :--- | :--- | :--- | -| **Test Case Maintenance** | ❌ Fragile XPath/ID dependencies; UI changes cause test failures | ⚠️ Unreliable execution; cannot be reused as regression tests | 🧪 **Zero Maintenance**: Natural language test cases resilient to UI drift & redesigns | -| **Execution Latency & Throughput** | ⚡ Fast script execution, but extreme setup and locator debugging costs | ❌ Sluggish 20–30s per step; too slow for regression testing | ⚡ **High Throughput**: Optimistic Async Pipeline runs at 3–5s per step | -| **Popup Resilience & Self-Healing** | ❌ System popups or permissions immediately crash the script | ❌ Easily gets stuck or loops endlessly on unexpected dialogs | 🛡️ **Pre-Execution Safety Net**: Automatically intercepts and clears interfering popups | -| **Diagnostics & Multimedia** | ❌ Blind static waits (sleep); cannot assert dynamic video/animations | ❌ Static screenshots only; no system logs or underlying state | 🐞 **Deep Diagnostics**: Live video stream analysis & **Logcat crash stack capture** | -| **Dev Environment Integration** | ❌ Standalone runner; requires manual log collection upon failure | ❌ Isolated web demos; difficult to embed into dev pipelines | 🔌 **Native MCP & SDK**: Drive physical test devices and debug directly inside Antigravity / Claude Code | +| **Test Case Maintenance** | ❌ Fragile XPath/ID dependencies; UI changes cause test failures | ⚠️ Unreliable execution; cannot be reused as regression tests | **Zero Maintenance**: Natural language test cases resilient to UI drift & redesigns | +| **Execution Latency & Throughput** | ⚡ Fast script execution, but extreme setup and locator debugging costs | ❌ Sluggish 20–30s per step; too slow for regression testing | **High Throughput**: Optimistic Async Pipeline runs at 3–5s per step | +| **Popup Resilience & Self-Healing** | ❌ System popups or permissions immediately crash the script | ❌ Easily gets stuck or loops endlessly on unexpected dialogs | **Pre-Execution Safety Net**: Automatically intercepts and clears interfering popups | +| **Diagnostics & Multimedia** | ❌ Blind static waits (sleep); cannot assert dynamic video/animations | ❌ Static screenshots only; no system logs or underlying state | **Deep Diagnostics**: Live video stream analysis & **Logcat crash stack capture** | +| **Dev Environment Integration** | ❌ Standalone runner; requires manual log collection upon failure | ❌ Isolated web demos; difficult to embed into dev pipelines | **Native MCP & SDK**: Drive physical test devices and debug directly inside Antigravity / Claude Code | -## 🏆 Benchmarks: AndroidWorld (SOTA 99%+) +## Benchmarks: AndroidWorld (SOTA 99%+) Evaluated on [AndroidWorld](https://github.com/google-research/android_world) — Google Research's gold-standard benchmark spanning 20+ real apps and 100+ complex multi-step tasks: **Artemis demonstrated exceptional robustness across the entire benchmark suite, achieving a 99%+ completion rate.** @@ -261,19 +263,19 @@ Evaluated on [AndroidWorld](https://github.com/google-research/android_world) AndroidWorld Benchmark Comparison

-## 🚀 How ARTEMIS is Architected +## How ARTEMIS is Architected -* 🛡️ **Pre-Touch Pixel Gate & Speculative Chaining**: Eliminates "silent misclicks" from inference latency race conditions. Milliseconds before dispatch, a local UI guard intercepts unexpected dialogs (0 tokens, 0 cloud wait), a Micro-ROI gate verifies target stability, and speculative chained taps hit transient UI (e.g. auto-fading video controls) before they expire; -* 🎯 **Three-Layer Progressive Grounding Engine**: Fuses local OCR with accessibility hierarchies (~150ms, 0 tokens) to drive 85%+ of standard actions via drift-free numeric indices, gracefully falling back to spatial vision models for custom Canvas/Compose/Flutter UI and sandboxed CV probing for subtle pixel states; -* 🧠 **Elastic Dual Engine with In-Flight Context Compactor**: Seamlessly toggles between high-throughput reactive CI loops (Flash Mode, 3–5s/step) and multi-step cognitive state graphs (Pro Mode), using background visual deltas and DOM pruning to slash token consumption by >70% for 10+ hours of continuous, unattended soak testing. +* **Pre-Touch Pixel Gate & Speculative Chaining**: Eliminates "silent misclicks" from inference latency race conditions. Milliseconds before dispatch, a local UI guard intercepts unexpected dialogs (0 tokens, 0 cloud wait), a Micro-ROI gate verifies target stability, and speculative chained taps hit transient UI (e.g. auto-fading video controls) before they expire; +* **Three-Layer Progressive Grounding Engine**: Fuses local OCR with accessibility hierarchies (~150ms, 0 tokens) to drive 85%+ of standard actions via drift-free numeric indices, gracefully falling back to spatial vision models for custom Canvas/Compose/Flutter UI and sandboxed CV probing for subtle pixel states; +* **Elastic Dual Engine with In-Flight Context Compactor**: Seamlessly toggles between high-throughput reactive CI loops (Flash Mode, 3–5s/step) and multi-step cognitive state graphs (Pro Mode), using background visual deltas and DOM pruning to slash token consumption by >70% for 10+ hours of continuous, unattended soak testing.

ARTEMIS System Architecture Diagram

-## ⚡ Execution Profiles: Flash vs. Pro +## Execution Profiles: Flash vs. Pro -| Feature / Dimension | ⚡ **ARTEMIS Flash** (`--profile flash`) | 🧠 **ARTEMIS Pro** (`--profile pro`) | +| Feature / Dimension | **ARTEMIS Flash** (`--profile flash`) | **ARTEMIS Pro** (`--profile pro`) | | :--- | :--- | :--- | | **Design Purpose** | **Lightweight & Fast**: Direct deterministic UI actions | **Deep Reasoning**: Multi-step planning & complex self-healing | | **Step Latency** | **3–5 seconds** / step | **15–30 seconds** / turn (includes planning & verification) | @@ -282,7 +284,7 @@ Evaluated on [AndroidWorld](https://github.com/google-research/android_world) | **Self-Healing** | Local step retries | **Safety Net Gate** + dialog suppression + crash recovery + snapshot rollback | | **Media Analysis** | **Basic visual perception** + High-Speed OCR | Full `scrcpy`/`ffmpeg` video stream analysis + Logcat logs | -## 🗺️ Roadmap +## Roadmap - [x] **Optimistic Asynchronous Pipeline**: Ultra-lean main loop + background context compression & milestone checks. - [x] **Pre-Execution Safety Net**: Millisecond pre-check gate & speculative chained actions. @@ -294,13 +296,13 @@ Evaluated on [AndroidWorld](https://github.com/google-research/android_world) - [ ] **On-Device Lightweight VLMs**: Zero-cloud local execution with lightweight edge vision models. - [ ] **Real-time Duplex Voice Mode**: Natural voice input with real-time interruption (barge-in) control. -## 🤝 Community & Contributing +## Community & Contributing Contributions are warmly welcomed! -* ⭐ **Star the repo** to follow updates and releases -* 💬 Join the [Discord Community](https://discord.gg/wF2FN4WHGY) for technical discussions -* 🐛 Open an [Issue](https://github.com/google/artemis/issues) or submit a [Pull Request](https://github.com/google/artemis/pulls) +* **Star the repo** to follow updates and releases +* Join the [Discord Community](https://discord.gg/wF2FN4WHGY) for technical discussions +* Open an [Issue](https://github.com/google/artemis/issues) or submit a [Pull Request](https://github.com/google/artemis/pulls) -## 📄 License +## License This project is licensed under the [Apache License 2.0](LICENSE). diff --git a/README_CN.md b/README_CN.md index 2adb81d..11443df 100644 --- a/README_CN.md +++ b/README_CN.md @@ -26,7 +26,9 @@

- Artemis 演示效果 + Artemis 演示效果 +
+ 实机演示：在 Google Maps 中设置驾车路线并计算总耗时，随后打开 YouTube 播放 Coldplay 的歌曲。

## ✨ 核心亮点 @@ -36,7 +38,7 @@ * 🐞 **IDE 内一键 Bug 复现与 Logcat 诊断**：原生支持 **MCP 协议**，可在 **Antigravity、Claude Code、Windsurf** 中直接用自然语言驱动真机复现缺陷，自动抓取关键帧截图与 **Logcat 崩溃堆栈**，完成研发测试闭环； * ⚡ **极速执行吞吐（单步 3–5 秒）**：首创**全链路乐观异步流水线 (Optimistic Async Pipeline)**，解耦重度推理与页面操作，在 Flash 模式下实现丝滑的高频交互与快速回归验证； * 🛡️ **干扰弹窗自愈与超长程巡检**：独创 **Safety Net** 执行前校验机制，自动识别并清除系统权限弹窗、通知遮挡等异常；Pro 模式支持连续 **10+ 小时** 无人值守稳定性巡检与探索性测试 (Exploratory / Monkey-plus Testing)； -* 🏆 **业界顶尖 SOTA**：在 Google Research **AndroidWorld** 基准评测（100+ 复杂长程系统与应用交互任务）中取得 **99%+ 任务完成率**。 +* **业界顶尖 SOTA**：在 Google Research **AndroidWorld** 基准评测（100+ 复杂长程系统与应用交互任务）中取得 **99%+ 任务完成率**。 @@ -47,24 +49,24 @@ @@ -73,13 +75,13 @@ -## ⚡ 快速上手 +## 快速上手 确保电脑已连接 Android 实体机（已开启 **USB 调试**）或 Android 模拟器。一键启动脚本将会自动完成以下配置： - 🛠️ **安装系统环境依赖**：自动检测并安装 ADB、scrcpy、FFmpeg 与 Python（`uv`）运行时及项目依赖。 - 🔌 **全局挂载 MCP 服务与测试准则 (Rules)**：主动引导并自动将全局 MCP 服务与 **Artemis 移动端测试思维准则 (`rules.md`)** 挂载至你使用的 AI IDE（支持 **Antigravity**、**Cursor**、**Claude Code**、**Codex**、**Windsurf**、**VS Code**、**Cline/Roo**、**OpenClaw**）。 -### 🍎 macOS 与 🐧 Linux +### macOS 与 Linux ```bash # 1. 克隆代码仓库并进入目录 @@ -89,7 +91,7 @@ git clone https://github.com/google/artemis.git && cd artemis ./start.sh ``` -### 🪟 Windows PowerShell +### Windows PowerShell ```powershell # 1. 克隆代码仓库并进入目录 @@ -102,7 +104,7 @@ cd artemis > PowerShell 默认不会从当前目录查找可执行脚本，因此必须使用 `.\start.bat`，且命令末尾不要添加 `\`。如果使用传统命令提示符（CMD），则运行 `start.bat`。 -> 💡 **提示**：启动后将自动在默认浏览器中打开 Web 控制台（`http://localhost:8000`），提供设备连接向导、实时投屏、任务演练与状态回放面板。你也可以通过命令行直接运行：`uv run artemis run "打开系统设置，找到电池选项并告诉我当前电量" --profile flash`。 +> **提示**：启动后将自动在默认浏览器中打开 Web 控制台（`http://localhost:8000`），提供设备连接向导、实时投屏、任务演练与状态回放面板。你也可以通过命令行直接运行：`uv run artemis run "打开系统设置，找到电池选项并告诉我当前电量" --profile flash`。 @@ -125,7 +127,7 @@ uv run artemis mcp --install antigravity uv run artemis mcp --install all ``` -> 💡 **提示**：你也可以在首次运行 `uv run artemis init` 配置向导时，交互式完成 IDE 的 MCP 自动挂载。 +> **提示**：你也可以在首次运行 `uv run artemis init` 配置向导时，交互式完成 IDE 的 MCP 自动挂载。 > **进阶提示**：如果希望在任意目录下都不需要加 `uv run` 就能全局直接使用 `artemis` 命令，可在项目根目录下执行一次 `uv tool install -e .`。 ### 2. 手动配置（可选） @@ -231,22 +233,22 @@ if __name__ == "__main__": -## 🕹️ 使用方式 +## 使用方式

- Artemis 可视化控制台 + Artemis 可视化控制台
- 💡 控制台功能布局：① 顶栏视图切换（主页与工作区） · ② 运行模式与录屏回放（Flash/Pro 状态与视频回放） · ③ 实时感知推理流（动作分解、点击坐标与结构化总结） · ④ 自然语言下发胶囊（自然语言驱动真机） · ⑤ 任务队列看板（状态流转与历史回溯） + 控制台功能布局：① 顶栏视图切换（主页与工作区） · ② 运行模式与录屏回放（Flash/Pro 状态与视频回放） · ③ 实时感知推理流（动作分解、点击坐标与结构化总结） · ④ 自然语言下发胶囊（自然语言驱动真机） · ⑤ 任务队列看板（状态流转与历史回溯）

-* 🖥️ **Web 可视化测试控制台 (`uv run artemis ui`)**：集成设备实时投屏与交互面板，支持通过自然语言下发测试用例，实时观测推理步骤、操作轨迹、截图留存与异常状态回放； -* 🔌 **MCP 协议集成 (AI IDE 协同)**：作为标准 MCP 服务器无缝接入 **Antigravity、Claude Code、Windsurf** 等开发环境，在 IDE 中直接驱动真机完成自动化测试与 Bug 复现验证； -* 💻 **命令行工具 (`uv run artemis run`)**：支持通过 CLI 直接执行自动化用例、稳定性巡检或 AndroidWorld 基准评测，提供高保真结构化终端输出； -* 🐍 **Python SDK**：作为标准 Python 库无缝集成至现有自动化测试框架（如 pytest）或 CI/CD 流水线，提供基于 Pydantic 的强类型结构化结果与断言支持。 +* **Web 可视化测试控制台 (`uv run artemis ui`)**：集成设备实时投屏与交互面板，支持通过自然语言下发测试用例，实时观测推理步骤、操作轨迹、截图留存与异常状态回放； +* **MCP 协议集成 (AI IDE 协同)**：作为标准 MCP 服务器无缝接入 **Antigravity、Claude Code、Windsurf** 等开发环境，在 IDE 中直接驱动真机完成自动化测试与 Bug 复现验证； +* **命令行工具 (`uv run artemis run`)**：支持通过 CLI 直接执行自动化用例、稳定性巡检或 AndroidWorld 基准评测，提供高保真结构化终端输出； +* **Python SDK**：作为标准 Python 库无缝集成至现有自动化测试框架（如 pytest）或 CI/CD 流水线，提供基于 Pydantic 的强类型结构化结果与断言支持。 -## 📊 方案横向对比 +## 方案横向对比 -| 评估维度 | 传统自动化测试框架 (Appium / Maestro) | 常见移动端 VLM Agent | **ARTEMIS ☕ (下一代 AI 测试平台)** | +| 评估维度 | 传统自动化测试框架 (Appium / Maestro) | 常见移动端 VLM Agent | **ARTEMIS (下一代 AI 测试平台)** | | :--- | :--- | :--- | :--- | | **用例编写与维护** | ❌ 强依赖 XPath/ID，UI 微调即导致大面积报错 | ⚠️ 缺乏工程化封装，执行不可靠，无法作为用例复用 | 🧪 **零脚本维护**：自然语言直接定义用例，无惧 UI 漂移与改版 | | **执行延迟与吞吐** | ⚡ 脚本执行快，但编写与定位调试耗时极长 | ❌ 单步推理动辄 20-30 秒，无法满足回归测试要求 | ⚡ **高吞吐低延迟**：首创乐观异步流水线，单步仅需 3-5 秒 | @@ -256,7 +258,7 @@ if __name__ == "__main__": -## 🏆 基准评测：AndroidWorld (SOTA 99%+) +## 基准评测：AndroidWorld (SOTA 99%+) 在 Google Research 发布的业界基准评测 [AndroidWorld](https://github.com/google-research/android_world)（涵盖 20+ 款常用应用与 100+ 项复杂多步长程任务）中：**Artemis 在全套长程任务评测中展现了高鲁棒性，取得了超过 99% 的任务完成率。** @@ -264,19 +266,19 @@ if __name__ == "__main__": AndroidWorld 评测基准对比

-## 🚀 ARTEMIS 是如何建构的 +## ARTEMIS 是如何建构的 -* 🛡️ **Pre-Touch 触控前像素守门与预测性连点**：彻底杜绝大模型推理延迟造成的「静默误触」。动作下发前毫秒级拦截意外弹窗（0 Token、0 云端等待），Micro-ROI 局部校验目标稳定性；针对视频全屏等瞬态控件，结合历史先验毫秒级连击唤醒，在淡出窗口期内瞬时闭环； -* 🎯 **三层递进式目标定位引擎 (Progressive Grounding)**：首层通过本地 OCR 与无障碍树几何融合（~150ms、0 Token），以无偏移数字索引承载 85%+ 常见交互；二层对 Flutter/Compose/Canvas 等自绘组件自动回退至空间视觉模型，三层辅以沙箱 CV 探针核验细微像素状态； -* 🧠 **弹性双执行引擎与运行时上下文动态压缩**：兼顾 3–5 秒秒级响应的 CI 高吞吐回归（Flash 模式）与深度长程认知状态图探索（Pro 模式），后台异步生成视觉增量并剪除冗余 DOM，长程任务 Token 消耗锐降 70%+，支持 10+ 小时无人值守稳定性压测。 +* **Pre-Touch 触控前像素守门与预测性连点**：彻底杜绝大模型推理延迟造成的「静默误触」。动作下发前毫秒级拦截意外弹窗（0 Token、0 云端等待），Micro-ROI 局部校验目标稳定性；针对视频全屏等瞬态控件，结合历史先验毫秒级连击唤醒，在淡出窗口期内瞬时闭环； +* **三层递进式目标定位引擎 (Progressive Grounding)**：首层通过本地 OCR 与无障碍树几何融合（~150ms、0 Token），以无偏移数字索引承载 85%+ 常见交互；二层对 Flutter/Compose/Canvas 等自绘组件自动回退至空间视觉模型，三层辅以沙箱 CV 探针核验细微像素状态； +* **弹性双执行引擎与运行时上下文动态压缩**：兼顾 3–5 秒秒级响应的 CI 高吞吐回归（Flash 模式）与深度长程认知状态图探索（Pro 模式），后台异步生成视觉增量并剪除冗余 DOM，长程任务 Token 消耗锐降 70%+，支持 10+ 小时无人值守稳定性压测。

ARTEMIS 架构系统示意图

-## ⚡ 运行模式对比：Flash vs. Pro +## 运行模式对比：Flash vs. Pro -| 特性对比 | ⚡ **ARTEMIS Flash** (`--profile flash`) | 🧠 **ARTEMIS Pro** (`--profile pro`) | +| 特性对比 | **ARTEMIS Flash** (`--profile flash`) | **ARTEMIS Pro** (`--profile pro`) | | :--- | :--- | :--- | | **设计定位** | **轻量极速**：面向常规确定性操作 | **深度规划**：面向长流程与复杂自愈 | | **单步延迟** | **3–5 秒** / 步 | **15–30 秒** / 轮 (包含深度推理与全局校验) | @@ -285,7 +287,7 @@ if __name__ == "__main__": | **自愈机制** | 基础局部重试 | **Safety Net 校验** + 弹窗处理 + 崩溃恢复 + 快照回滚 | | **多媒体分析** | 基础视觉感知 + 高速 OCR | 完整 `scrcpy`/`ffmpeg` 视频流分析 + Logcat 日志采集 | -## 🗺️ 路线图 +## 路线图 - [x] **全链路乐观异步流水线**：主流程轻量化，后台并发执行记忆压缩与断言校验。 - [x] **Safety Net 执行前校验**：动作下发前双层核验与预测性链式执行。 @@ -297,13 +299,13 @@ if __name__ == "__main__": - [ ] **端侧轻量化模型**：支持在设备本地离线运行的轻量级 Edge VLM 模型。 - [ ] **实时语音双工交互**：支持自然语音输入与实时打断控制。 -## 🤝 社区与贡献 +## 社区与贡献 欢迎通过以下方式参与项目建设： -* ⭐ **Star 本项目**以关注最新进展与更新 -* 💬 加入 [Discord 社区](https://discord.gg/wF2FN4WHGY) 参与技术探讨与功能建议 -* 🐛 提交 [Issue](https://github.com/google/artemis/issues) 反馈 Bug，欢迎发起 [Pull Request](https://github.com/google/artemis/pulls) 贡献代码 +* **Star 本项目**以关注最新进展与更新 +* 加入 [Discord 社区](https://discord.gg/wF2FN4WHGY) 参与技术探讨与功能建议 +* 提交 [Issue](https://github.com/google/artemis/issues) 反馈 Bug，欢迎发起 [Pull Request](https://github.com/google/artemis/pulls) 贡献代码 -## 📄 开源许可证 +## 开源许可证 本项目基于 [Apache License 2.0](LICENSE) 协议开源。
- 1️⃣ 输入测试提示词 (Task Dispatch)
+ 1. 输入测试提示词 (Task Dispatch)
在 Antigravity 中用自然语言描述测试需求与目标指标

步骤一：输入测试提示词 		- 2️⃣ 生成测试方案 (Test Plan Generation)
+ 2. 生成测试方案 (Test Plan Generation)
自动拆解任务，生成详细测试步骤与架构图供确认

步骤二：生成测试方案
- 3️⃣ 自主执行测试 (Autonomous Test Execution)
+ 3. 自主执行测试 (Autonomous Test Execution)
接管真机自动化操作，跳过广告并实时采集性能指标

步骤三：自主执行测试 		- 4️⃣ 交付最终报告 (Comprehensive Final Report)
+ 4. 交付最终报告 (Comprehensive Final Report)
生成结构化测试报告，交付性能图表、结论与原始数据

步骤四：交付最终报告