ARTEMIS: Next-Gen AI Mobile Test Automation & Autonomous Assistant Platform
_{Autonomous Real-time Testing, Exploration & Mobile Interaction System}

⚡ Test Real Devices from Antigravity & Claude Code • Cross-App Automation • Zero-Maintenance Testing • Bug Repro & Logcat Diagnostics

English • 中文文档 • Workflow Showcase • Quick Start • MCP for IDEs • Benchmarks • Discord Community

Live Demo: Setup driving routes and calculate total durations in Google Maps, then open YouTube to play a Coldplay song.

✨ Key Highlights

• 🤖 Cross-App Automation & Autonomous AI Assistant: Operates not just as a robust testing framework, but as an autonomous agent capable of handling complex cross-app workflows and daily tasks via natural language;
• 🧪 Zero-Maintenance Test Automation: Built upon a "Dynamic-First, Coordinate-Fallback" multimodal locating engine, eliminating fragile XPath/ID selector maintenance and remaining resilient to UI redesigns, system updates, and resolution drift;
• 🐞 One-Click Bug Repro & Logcat Diagnostics in IDE: Native Model Context Protocol (MCP) integration allows Antigravity, Claude Code, and Windsurf to drive physical test devices via natural language, automatically capturing crash stacks from Logcat and keyframe screenshots;
• ⚡ Ultra-Fast Execution (3–5s per Step): Pioneered an Optimistic Asynchronous Pipeline that completely decouples UI interaction from heavy LLM reasoning, achieving rapid regression throughput in Flash mode;
• 🛡️ Popup Self-Healing & 10+ Hour Exploration: Proprietary Safety Net double-checks targets before action execution to intercept and clear interfering system popups; Pro mode supports 10+ hours of continuous exploratory & monkey-plus stability testing;
• 🏆 Industry-Leading SOTA: Achieved 99%+ task completion on Google Research's AndroidWorld benchmark (100+ complex multi-step tasks).

🤝 Antigravity × ARTEMIS: Autonomous Testing Workflow

Experience seamless collaboration between Antigravity and ARTEMIS via native MCP integration — taking you from a natural language requirement to a production-grade diagnostic report in four automated steps:

1️⃣ Prompt Input (Task Dispatch) Describe your test scenario and target metrics in Antigravity	2️⃣ Test Plan Generation Formulates a step-by-step test plan & architecture for review
3️⃣ Autonomous Test Execution Drives real device, navigates UI, and profiles performance	4️⃣ Comprehensive Final Report Delivers structured audit findings, metric tables, and raw datasets

⚡ Quick Start

Ensure an Android device (with USB Debugging enabled) or emulator is connected.

# 1. Clone repo & navigate to directory
git clone https://github.com/google/artemis.git && cd artemis

# 2. One-click launch (automatically installs ADB, scrcpy, FFmpeg, uv runtime, and opens console)
# 🍎 macOS & 🐧 Linux
./start.sh

# 🪟 Windows (CMD / PowerShell)
start.bat

💡 Tip: Opens http://localhost:8000 in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: artemis run "Open Settings, find Battery and tell me current level" --profile flash.

🔌 MCP Setup for Antigravity / Claude Code / Windsurf (Click to expand)

ARTEMIS includes a native Model Context Protocol (MCP) server. Connect your real phone directly into AI IDEs:

1. One-Click Auto Install (Recommended)

Run the built-in installer to automatically configure or update your AI IDE (automatically resolves your virtual environment Python and repository path):

# Auto-install for Antigravity / Jetski:
artemis mcp --install antigravity

# Or install for all detected AI IDEs (Antigravity, Cursor, Claude Desktop/Code, OpenClaw):
artemis mcp --install all

💡 Tip: You can also configure MCP interactively during first-time setup via artemis init.

2. Manual Configuration (Optional)

If you prefer to configure manually, run artemis mcp --generate-config antigravity (or all) to output the exact JSON snippet, or paste the following into your IDE configuration (replace /path/to/artemis with your actual repo path and point command to your .venv Python executable):

• Antigravity (~/.gemini/jetski/mcp_config.json):

{
  "mcpServers": {
    "artemis": {
      "command": "/path/to/artemis/.venv/bin/python",
      "args": ["-m", "mcp_server"],
      "cwd": "/path/to/artemis",
      "env": {
        "PYTHONUNBUFFERED": "1"
      },
      "tools": {
        "mobile_run_task": { "eager": true },
        "mobile_manage_task": { "eager": true },
        "mobile_get_device_state": { "eager": true },
        "mobile_inspect_trace": { "eager": true }
      }
    }
  }
}

• Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "artemis": {
      "command": "/path/to/artemis/.venv/bin/python",
      "args": ["-m", "mcp_server"],
      "cwd": "/path/to/artemis"
    }
  }
}

3. Mount Behavioral Rules for AI Agents (Highly Recommended)

To ensure your AI coding assistant acts with the rigor of a senior mobile test engineer and never hallucinates UI interactions, we provide a dedicated testing mindset rules file at mcp_server/rules.md (covering Active Exploration before coding, Flash vs. Pro routing strategy, Latency & Timing compensation, and the "Dynamic-First, Coordinate-Fallback" locator pattern).

You can mount or copy mcp_server/rules.md into your AI IDE's rule configuration:

• Antigravity: Add the contents of rules.md to your Workspace Rules, Global Rules settings, or agent instructions.
• Claude Code: Copy or include the contents of rules.md in your project's CLAUDE.md file.
• Cursor: Copy the contents into .cursorrules or create a rule file at .cursor/rules/artemis.mdc.
• Windsurf / OpenClaw: Add the rules to your workspace rules or global system prompts.

💡 For more details on the testing mindset and MCP architecture, see the MCP Server README.

4. Prompt Your Phone in the IDE Chat

In Antigravity or Claude Code, simply prompt:

💬 "Build the latest changes into an APK, install it on the connected device, open the login screen with a test account, verify if there are any unexpected popups after login, and return screenshots of the final page."

🐍 Python SDK Integration (Click to expand)

Embed the mobile automation engine into your Python workflows in just a few lines:

import asyncio
from artemis.interfaces.sdk import ArtemisClient


async def main():
    # Initialize client (choose "flash" for fast UI checks or "pro" for deep reasoning & self-healing)
    client = ArtemisClient(default_profile="flash")

    # Execute natural language end-to-end test case
    result = await client.run(
        "Open System Settings, go to 'Battery', verify battery percentage is displayed, and check for any crash dialogs."
    )

    # Structured assertions & execution tracing
    assert result.status == "SUCCESS", f"Test failed: {result.failure_reason}"
    print(f"✅ Test Passed! Turns: {result.turns} | Trace ID: {result.trace_id}")


if __name__ == "__main__":
    asyncio.run(main())

🕹️ Usage Modes

_{💡 Console Overview: ① View Switcher (Home / Workspace) · ② Model & Replay (Flash/Pro status & video replay) · ③ Live Agent Stream (Action perception, target coordinates & structured results) · ④ Prompt Dock (Natural language dispatch) · ⑤ Task Queue & Dashboard (Lifecycle & history)}

• 🖥️ Web Visual Test Console (artemis ui): Real-time screen projection and interactive panel, supporting natural language test dispatch, live reasoning telemetry, action trajectories, and execution replay;
• 🔌 Native MCP Protocol (IDE Collaboration): Operates as a standard MCP server seamlessly integrating with Antigravity, Claude Code, Windsurf, etc., directly driving real devices inside the IDE to verify bugs and run test cases;
• 💻 Developer CLI (artemis run): Direct terminal execution for automated test cases, exploratory stability inspection, or AndroidWorld benchmarks with high-fidelity structured terminal output;
• 🐍 Python SDK: Integrates as a standard Python library into existing automated testing frameworks (e.g., pytest) or CI/CD pipelines with strongly typed Pydantic structured outputs and assertion support.

📊 Head-to-Head Comparison

Evaluation Dimension	Traditional Test Automation (Appium / Maestro)	Generic Mobile VLM Agents	ARTEMIS ☕ (Next-Gen AI Testing)
Test Case Maintenance	❌ Fragile XPath/ID dependencies; UI changes cause test failures	⚠️ Unreliable execution; cannot be reused as regression tests	🧪 Zero Maintenance: Natural language test cases resilient to UI drift & redesigns
Execution Latency & Throughput	⚡ Fast script execution, but extreme setup and locator debugging costs	❌ Sluggish 20–30s per step; too slow for regression testing	⚡ High Throughput: Optimistic Async Pipeline runs at 3–5s per step
Popup Resilience & Self-Healing	❌ System popups or permissions immediately crash the script	❌ Easily gets stuck or loops endlessly on unexpected dialogs	🛡️ Pre-Execution Safety Net: Automatically intercepts and clears interfering popups
Diagnostics & Multimedia	❌ Blind static waits (sleep); cannot assert dynamic video/animations	❌ Static screenshots only; no system logs or underlying state	🐞 Deep Diagnostics: Live video stream analysis & Logcat crash stack capture
Dev Environment Integration	❌ Standalone runner; requires manual log collection upon failure	❌ Isolated web demos; difficult to embed into dev pipelines	🔌 Native MCP & SDK: Drive physical test devices and debug directly inside Antigravity / Claude Code

🏆 Benchmarks: AndroidWorld (SOTA 99%+)

Evaluated on AndroidWorld — Google Research's gold-standard benchmark spanning 20+ real apps and 100+ complex multi-step tasks: Artemis demonstrated exceptional robustness across the entire benchmark suite, achieving a 99%+ completion rate.

🚀 How ARTEMIS is Architected

• ⚡ Optimistic Asynchronous Pipeline: The front-facing loop responds in milliseconds, while memory pruning and assertion verification run concurrently in the background without blocking execution;
• 🛡️ Safety Net Pre-Execution Gate: Dual-layer pre-check validates target availability milliseconds before action dispatch, instantly intercepting unexpected popups to eliminate blind clicks;
• ⏱️ Time-Sensitive Speculative Chaining: Overcomes LLM inference latency for transient UI elements (e.g. video fullscreen) by predicting target coordinates and executing rapid chained taps.

🔍 Click to expand: Architecture Deep Dive & Pipeline Diagram

1. ⚡ Optimistic Asynchronous Pipeline

• Status Quo & Pain Points: Conventional mobile agents rely on a fully synchronous blocking model — every single action must wait sequentially for the LLM to prune historical context, check milestone assertions, and audit long-term plans. This inflates per-step latency to 20–40 seconds, creating a sluggish user experience.
• Artemis's Architectural Solution: Inspired by Optimistic Concurrency Control (OCC) and Snapshot Isolation in database systems, Artemis completely decouples the main execution loop from heavy auxiliary computation:
  • High-Throughput Main Loop: The front-facing execution path is strictly narrowed to a high-speed "Perception → Decision → Safety Gate → Execution" pipeline;
  • Background Concurrent Tasks: Context token pruning, milestone checkers (Checker), and planner validations (Planner) run dynamically in parallel without halting device interaction;
  • Snapshot Isolation & Rollback: The agent optimistically charges forward. If background verification detects a deviation, Artemis instantly rolls back via state snapshots (Rollback) and injects self-healing feedback.

2. 🛡️ Pre-Execution Safety Net & Time-Sensitive Speculative Chaining

• Status Quo & The Transient UI Dilemma:
  • Mobile applications feature numerous time-sensitive transient UI controls (e.g. video fullscreen: tapping the screen wakes up the floating overlay, followed immediately by tapping the fullscreen icon).
  • Traditional agents tap the screen to reveal controls, take a new screenshot, and wait 3–15 seconds for LLM reasoning. By the time the click is dispatched, the player controls have already auto-faded away — causing the click to strike the underlying video, triggering an endless loop of accidental pausing and waking.
• Artemis's Architectural Solution:
  • Speculative Chained Actions: Upon recognizing time-sensitive dependencies, the agent dispatches compound chained actions (Wakeup → Millisecond Chained Tap) to hit the target within its transient visibility window;
  • Two Pillars Ensuring Reliable Chaining:
    1. Historical UI Prior Prediction: Predicts the target control's wake-up coordinates based on prior interaction history and app layout heuristics;
    2. Safety Net Pre-Execution Gate: Milliseconds before the chained action lands, the Safety Net instantly verifies that the target control was successfully revealed at the expected coordinates. If the wakeup failed or an unexpected popup intercepted it, execution is immediately blocked to prevent blind clicks.

⚡ Execution Profiles: Flash vs. Pro

Feature / Dimension	⚡ ARTEMIS Flash (--profile flash)	🧠 ARTEMIS Pro (--profile pro)
Design Purpose	Lightweight & Fast: Direct deterministic UI actions	Deep Reasoning: Multi-step planning & complex self-healing
Step Latency	3–5 seconds / step	15–30 seconds / turn (includes planning & verification)
Task Duration	Minute-level short tasks (typically ≤35 steps)	Runs stably for 10+ hours; monitoring tasks support 24/7 execution
Best Suited For	Well-defined standard UI tasks	Complex cross-app workflows, failure self-healing, continuous monitoring
Self-Healing	Local step retries	Safety Net Gate + dialog suppression + crash recovery + snapshot rollback
Media Analysis	Basic visual perception + High-Speed OCR	Full scrcpy/ffmpeg video stream analysis + Logcat logs

🗺️ Roadmap

• Optimistic Asynchronous Pipeline: Ultra-lean main loop + background context compression & milestone checks.
• Pre-Execution Safety Net: Millisecond pre-check gate & speculative chained actions.
• Time-Sensitive Media Tasks: Fullscreen video and audio stream analysis with scrcpy & ffmpeg.
• Native MCP Server: Seamless integration with tools like Antigravity and Claude Desktop.
• Web Visual Console: Live screen projection, interactive playground, and trajectory review.
• AndroidWorld SOTA: Achieved 99%+ task completion rate.
• Cross-Platform Extensions: Exploring iOS and desktop Web perception and execution.
• On-Device Lightweight VLMs: Zero-cloud local execution with lightweight edge vision models.
• Real-time Duplex Voice Mode: Natural voice input with real-time interruption (barge-in) control.

🤝 Community & Contributing

Contributions are warmly welcomed!

• ⭐ Star the repo to follow updates and releases
• 💬 Join the Discord Community for technical discussions
• 🐛 Open an Issue or submit a Pull Request

📄 License

This project is licensed under the Apache License 2.0.