feat: add official Codex skill and harden task execution
Install and update artemis-use with task diagnostics and plugin guidance. Preserve cross-device file receipts, review canonical reports, guard stage transitions and reduce small history summaries. Improve exploration lifecycle, request recovery and browser live connections.
This commit is contained in:
chenmingxuan committed
1 parent
2328a4df0c
commit
e1ea099592
128 files changed
+4944
-700
No files matched your search
+8
-18
@@ -1,6 +1,6 @@
|
||||
# 🔌 Universal MCP Server for ARTEMIS
|
||||
|
||||
This directory contains the **Model Context Protocol (MCP)** server for **ARTEMIS**. It exposes mobile device automation to clients such as **Antigravity**, **Cursor**, **Claude Code**, **OpenClaw**, and **Windsurf**.
|
||||
This directory contains ARTEMIS's connection-only MCP bridge and the tools hosted by the user-started web service. Official automatic configuration and Skill installation currently support Codex.
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
@@ -12,7 +12,7 @@ mcp_server/
|
||||
├── bridge.py # Connection-only, lightweight stdio -> local HTTP
|
||||
├── shared.py # Tools hosted in the user-started web process
|
||||
├── catalog.json # Offline schemas; regenerate with scripts/generate_mcp_catalog.py
|
||||
├── rules.md # AI Agent Testing Mindset & system prompt rules for IDEs
|
||||
├── rules.md # Compatibility link to the official artemis-use Skill
|
||||
├── background/ # Background process execution
|
||||
│ └── task_runner.py # Subprocess runner for Artemis Agent
|
||||
├── notifiers/ # Multi-environment notification adapters
|
||||
@@ -50,17 +50,9 @@ When you start an asynchronous mobile test via `mobile_run_task`, the user-start
|
||||
| **OpenClaw / Slack / Discord / CI/CD** | **HTTP Webhook (`WebhookNotifier`)** | Sends structured JSON POST payloads to custom endpoints configured via `OPENCLAW_WEBHOOK_URL`, `MCP_NOTIFICATION_WEBHOOK`, or `ARTEMIS_WEBHOOK_URL`. |
|
||||
| **Universal Custom IDE / CLI Hooks** | **Custom Script Hook (`ScriptNotifier`)** | Set `ARTEMIS_NOTIFY_CMD="my-script --title '{title}' --message '{message}' --trace-id '{trace_id}'"` in your environment to execute any custom command or script upon event completion. |
|
||||
|
||||
## 🧠 AI Agent Behavioral Rules (`rules.md`)
|
||||
## Official Codex Skill
|
||||
|
||||
When connecting ARTEMIS to AI coding assistants like **Antigravity**, **Claude Code**, **Cursor**, or **Windsurf**, providing the assistant with domain-specific testing discipline is critical for generating reliable test code.
|
||||
|
||||
The included [`rules.md`](./rules.md) file contains the **Mobile Testing Mindset (ARTEMIS Integration)** guideline. It teaches the AI agent how to properly collaborate with ARTEMIS:
|
||||
|
||||
1. **The Runnable Code Principle & Active Exploration**: Instructs the AI to never guess or hallucinate UI transitions. Instead, it must first explore the live app via ARTEMIS MCP tools (`mobile_run_task`, etc.) to discover and verify real-world interaction paths before writing test scripts.
|
||||
2. **Flash vs. Pro Routing Strategy**: Guides the AI to choose **Flash** for rapid, straightforward UI actions (no step cap by default) and **Pro** for tasks that need a persistent plan, verified checkpoints, polling loops, or ADB / video / log diagnosis.
|
||||
3. **Latency & Timing Compensation**: Clarifies the difference between AI exploratory latency (e.g., model decision intervals) and the deterministic timing requirements of final test code.
|
||||
4. **"Dynamic-First, Coordinate-Fallback" Locator Pattern**: Teaches the AI to prioritize dynamic UI locators (Resource IDs, OCR text, semantics) for layout resilience, while implementing absolute coordinate fallbacks for maximum execution reliability.
|
||||
5. **Environment Self-Diagnosis**: Tells the AI to call `mobile_diagnose` first whenever a tool errors or the user says ARTEMIS is not working, and how to act on the returned fix list (run local commands itself, relay device prompts to the user, never move API keys through the chat, reload the MCP server after config changes).
|
||||
[`artemis-use`](../skills/artemis-use/SKILL.md) explains Flash, Pro and guided exploration, task diagnosis, evidence review, batching and plugin development. Detailed references load only when relevant. Its whole official directory is maintained by ARTEMIS; no global rule block is injected into AGENTS.md.
|
||||
|
||||
### How Global Rules & MCP are Installed
|
||||
Only Codex configuration is installed: `uv run artemis mcp --install codex`. It preserves other clients and services. The user starts the web service with `start.bat`, `start-cn.bat`, or `uv run artemis ui`. The stdio bridge never starts a backend or standalone worker; all tool execution shares the web process. Offline initialization and tools/list still work, while tool calls explain how to start the service. Codex configuration is optional (`required = false`) so failures cannot block unrelated sessions. Reload both the web process and old MCP connections after upgrading.
|
||||
@@ -97,16 +89,14 @@ uv run artemis mcp
|
||||
### IDE Configuration
|
||||
|
||||
#### One-Click Auto Install (Recommended)
|
||||
Run the automated installer to detect and configure your IDE (Antigravity, Cursor, Claude Code/Desktop, OpenClaw):
|
||||
Start ARTEMIS normally to synchronize Codex, or repair the integration explicitly:
|
||||
```bash
|
||||
uv run artemis mcp --install all
|
||||
# Or install specifically for Antigravity:
|
||||
uv run artemis mcp --install antigravity
|
||||
uv run artemis mcp --install codex
|
||||
```
|
||||
|
||||
#### Manual JSON Config Generation
|
||||
Run `uv run artemis mcp --generate-config antigravity` (or `all`) to output ready-to-use configuration JSON with resolved `.venv` Python and project paths.
|
||||
Run `uv run artemis mcp --generate-config codex` to output a configuration template with resolved Python and project paths.
|
||||
|
||||
## 完整 AI 自动化
|
||||
|
||||
新增 `mobile_workspace`、`mobile_config`、`mobile_task_file`、`mobile_batch`、`mobile_search_history`、`mobile_export_report`，与网页共用后台服务。参见 [接入说明](../docs/ai-automation.md) 和 [Skill](../skills/artemis-automation/SKILL.md)。更新已有客户端后重新安装/加载 MCP，以刷新工具白名单。
|
||||
工作区工具与网页共用后台服务，覆盖普通与探索任务、批次、历史、诊断、人工复验及报告导出。参见 [接入说明](../docs/ai-automation.md) 和 [Skill](../skills/artemis-use/SKILL.md)。网页启动时同步官方 Skill 与 MCP 接入；已有审批策略及工具白名单保留。更新后重新加载 MCP 才能使用新版工具定义。
|
||||
+2
-2
@@ -33,7 +33,7 @@ except Exception as exc:
|
||||
mcp = FastMCP(
|
||||
"artemis",
|
||||
instructions=(
|
||||
"ARTEMIS is an autonomous mobile AI agent and Android UI automation engine. "
|
||||
"ARTEMIS executes Android, computer and mixed tests with Flash, Pro or interactive exploration. The user starts the service; MCP only connects. Use the official artemis-use Skill for workflows and plugin development. "
|
||||
"For the full workflow use mobile_workspace (devices/queue), mobile_config (non-secret saved defaults), "
|
||||
"mobile_task_file (XLSX template/preview), mobile_batch (JSON cases + execution), "
|
||||
"mobile_search_history and mobile_export_report (Word/Excel/ZIP). "
|
||||
@@ -43,7 +43,7 @@ mcp = FastMCP(
|
||||
"Use mobile_run_task to launch autonomous UI workflows on connected Android devices or emulators, "
|
||||
"mobile_manage_task to check status or steer execution, "
|
||||
"mobile_get_device_state to inspect real-time device screen/hierarchy, "
|
||||
"mobile_inspect_trace to inspect detailed execution steps and visual action overlays, "
|
||||
"mobile_inspect_trace for bounded report/usage/request/log/evidence inspection and step screenshots; use stable request_id for task submission and lookup uncertain replies, "
|
||||
"and mobile_diagnose whenever a tool errors, no device is found, or the user says "
|
||||
"ARTEMIS is not working: it checks the environment and returns ordered fix steps."
|
||||
),
|
||||
|
||||
+103
-8
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"instructions": "ARTEMIS is an autonomous mobile AI agent and Android UI automation engine. For the full workflow use mobile_workspace (devices/queue), mobile_config (non-secret saved defaults), mobile_task_file (XLSX template/preview), mobile_batch (JSON cases + execution), mobile_search_history and mobile_export_report (Word/Excel/ZIP). For interactive test design use mobile_exploration: one conversation, user guidance, recorded demonstrations, resumable active-time limits and versioned cases for Flash/Pro. Task action replay is unavailable; trace inspection is read-only. Use mobile_run_task to launch autonomous UI workflows on connected Android devices or emulators, mobile_manage_task to check status or steer execution, mobile_get_device_state to inspect real-time device screen/hierarchy, mobile_inspect_trace to inspect detailed execution steps and visual action overlays, and mobile_diagnose whenever a tool errors, no device is found, or the user says ARTEMIS is not working: it checks the environment and returns ordered fix steps.",
|
||||
"instructions": "ARTEMIS executes Android, computer and mixed tests with Flash, Pro or interactive exploration. The user starts the service; MCP only connects. Use the official artemis-use Skill for workflows and plugin development. For the full workflow use mobile_workspace (devices/queue), mobile_config (non-secret saved defaults), mobile_task_file (XLSX template/preview), mobile_batch (JSON cases + execution), mobile_search_history and mobile_export_report (Word/Excel/ZIP). For interactive test design use mobile_exploration: one conversation, user guidance, recorded demonstrations, resumable active-time limits and versioned cases for Flash/Pro. Task action replay is unavailable; trace inspection is read-only. Use mobile_run_task to launch autonomous UI workflows on connected Android devices or emulators, mobile_manage_task to check status or steer execution, mobile_get_device_state to inspect real-time device screen/hierarchy, mobile_inspect_trace for bounded report/usage/request/log/evidence inspection and step screenshots; use stable request_id for task submission and lookup uncertain replies, and mobile_diagnose whenever a tool errors, no device is found, or the user says ARTEMIS is not working: it checks the environment and returns ordered fix steps.",
|
||||
"tools": [
|
||||
{
|
||||
"name": "mobile_get_device_state",
|
||||
@@ -99,7 +99,7 @@
|
||||
},
|
||||
{
|
||||
"name": "mobile_inspect_trace",
|
||||
"description": "Retrieves execution details of a mobile automation task (running or finished).\n\nUse it to monitor progress, verify answers, and diagnose agent errors for\nboth Flash and Pro tasks. Every result includes the assigned `device_serial`.\n\n### Actions\n- **'view_summary'**: High-level execution summary across all steps.\n Pro: hierarchical task plan with per-step status; Flash: full execution\n chain with each step's reasoning and action.\n- **'search'**: Deterministic keyword / step-range lookup over the full\n stored history (screen descriptions, exact actions and results,\n reasoning, tool calls, notes, on-screen OCR/UI-tree text incl.\n package/activity names, compressed-history ledgers). Every hit carries\n its step number; a `step_range` also returns that range's per-step\n action ledger. Requires `query` and/or `step_range`.\n- **'view_step_screenshots'**: Local file paths of one step's screenshots:\n `before_screenshot` (what the agent saw), `after_screenshot` (only set\n when the action failed/was intercepted, else null), and\n `action_overlay_screenshot` (the action visually marked — e.g. red circle\n for taps — key for verifying the agent tapped the right element).\n- **'view_step_details'**: Stored details of one step exactly as the agent's\n own context showed it: what the screen showed, its reasoning, every tool\n call it made (name, arguments, result — e.g. what the explorer or OCR\n reported), the planned action, any safety-net interception and the\n execution result. Coordinates are normalized [x, y]. Works for Flash and\n Pro tasks.\n\nArgs:\n action: `\"view_summary\"`, `\"search\"`, `\"view_step_screenshots\"`, or\n `\"view_step_details\"`.\n trace_id: The task's session identifier from `mobile_run_task`.\n step_number: 1-indexed step to query; required for the two per-step\n actions, omit otherwise.\n query: Keywords for `\"search\"` (case-insensitive; every\n whitespace-separated term is matched independently).\n step_range: Optional `[start, end]` step range (inclusive) for\n `\"search\"`.\n max_results: Maximum `\"search\"` hits to return (server-side cap applies).\n",
|
||||
"description": "Retrieves execution details of a mobile automation task (running or finished).\n\nUse it to monitor progress, verify answers, and diagnose agent errors for\nboth Flash and Pro tasks. Every result includes the assigned `device_serial`.\n\n### Actions\n- view_report / view_usage / view_requests / view_logs / view_evidence:\n bounded read-only report, measured token/cache totals, request phases and IDs,\n redacted log tails, or source-specific evidence index. offset/limit paginate;\n start with view_report, then inspect only relevant evidence. No export is generated.\n- **'view_summary'**: High-level execution summary across all steps.\n Pro: hierarchical task plan with per-step status; Flash: paginated execution\n chain with each step's reasoning and action.\n- **'search'**: Deterministic keyword / step-range lookup over the full\n stored history (screen descriptions, exact actions and results,\n reasoning, tool calls, notes, on-screen OCR/UI-tree text incl.\n package/activity names, compressed-history ledgers). Every hit carries\n its step number; a `step_range` also returns that range's per-step\n action ledger. Requires `query` and/or `step_range`.\n- **'view_step_screenshots'**: Local file paths of one step's screenshots:\n `before_screenshot` (what the agent saw), `after_screenshot` (when actually captured, otherwise null), and\n `action_overlay_screenshot` (the action visually marked — e.g. red circle\n for taps — key for verifying the agent tapped the right element).\n- **'view_step_details'**: Stored details of one step exactly as the agent's\n own context showed it: what the screen showed, its reasoning, every tool\n call it made (name, arguments, result — e.g. what the explorer or OCR\n reported), the planned action, any safety-net interception and the\n execution result. Coordinate units depend on the tool: Cua uses screenshot\n pixels; consult the recorded source geometry, never reuse another window. Works for Flash and\n Pro tasks.\n\nArgs:\n action: `\"view_summary\"`, `\"search\"`, `\"view_step_screenshots\"`, or\n `\"view_step_details\"`.\n trace_id: The task's session identifier from `mobile_run_task`.\n step_number: 1-indexed step to query; required for the two per-step\n actions, omit otherwise.\n query: Keywords for `\"search\"` (case-insensitive; every\n whitespace-separated term is matched independently).\n step_range: Optional `[start, end]` step range (inclusive) for\n `\"search\"`.\n max_results: Maximum `\"search\"` hits to return (server-side cap applies).\n",
|
||||
"inputSchema": {
|
||||
"properties": {
|
||||
"action": {
|
||||
@@ -153,6 +153,16 @@
|
||||
"default": 5,
|
||||
"title": "Max Results",
|
||||
"type": "integer"
|
||||
},
|
||||
"offset": {
|
||||
"default": 0,
|
||||
"title": "Offset",
|
||||
"type": "integer"
|
||||
},
|
||||
"limit": {
|
||||
"default": 20,
|
||||
"title": "Limit",
|
||||
"type": "integer"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -165,7 +175,7 @@
|
||||
},
|
||||
{
|
||||
"name": "mobile_manage_task",
|
||||
"description": "Manages the lifecycle and retrieves the status of a background mobile automation task.\n\nThis is your primary diagnostic and control tool for tasks started by\n`mobile_run_task` or the web workspace. You MUST use it to poll the task status when your\n1-minute fallback timer triggers (as required by `mobile_run_task`). You\ncan also steer a subagent that is stuck or off-track, or abort a task.\n\n### Actions\n- **'status'**: Returns `trace_id`, `status` ('running'/'pausing'/'task_paused'/\n 'resuming'/'completed'/'failed'/'cancelled'/'interrupted'), `device_serial` (which phone owns this task in multi-device\n setups), `task_desc`, `model`, `elapsed_seconds`, a `test_summary` (when\n the run declared verification check items: machine-readable\n passed/failed/inconclusive/unchecked counts plus failed-item details —\n no need to parse report prose), `test_result` (Flash and Pro: status\n passed/failed/inconclusive/blocked, reason and evidence; completed only\n means execution ended), and `progress` (Flash:\n current turn, latest thought/action — intervene if turns climb without\n progress or the thought indicates it is stuck; Pro: the active task plan\n — check it still aligns with your goal).\n- **'pause'**: Requests a pause, returning `pausing` until the runner releases\n the device. Pro can interrupt a pending model response or read-only diagnostic;\n a device action already in progress must finish safely first. Check status\n for `task_paused` before assuming the device is free.\n- **'resume'**: Queues a paused task for its device again; optional `instruction`\n provides guidance before the next action.\n- **'inject_instruction'**: Injects real-time guidance mid-flight when the\n subagent errs, stalls, or loops. An in-flight model response or read-only\n helper is cancelled immediately, then the runner observes the device again\n and consumes the new guidance. An already-started device action finishes\n safely before the runner changes direction. Requires\n `instruction` (unless `release_loop=True`). To end a continuous\n monitoring task ([Loop:continuous] milestone) gracefully, you MUST pass\n `release_loop=True` — this is the only signal that unlocks the\n milestone's completion; natural-language \"please stop\" phrasing in\n `instruction` is NOT interpreted as a stop signal.\n- **'guidance_history'**: Lists submitted instructions with `pending` or\n `consumed` status; a submitted instruction is not proof it was applied.\n- **'stop'**: Forcefully terminates the subagent, immediately halting\n device interactions and releasing the device. Use when the task is done,\n irreparably broken, or running out of control. Prefer\n `inject_instruction` with `release_loop=True` when a monitoring task\n should wind down cleanly instead of being killed.\n\nArgs:\n action: `\"status\"`, `\"pause\"`, `\"resume\"`, `\"inject_instruction\"`,\n `\"guidance_history\"`, or `\"stop\"`.\n trace_id: The task's session identifier from `mobile_run_task` or the web UI.\n instruction: Guidance string; required for `inject_instruction`\n (optional when `release_loop=True`), omit otherwise.\n release_loop: With `inject_instruction`: explicit user stop signal\n that authorizes the subagent to complete continuous monitoring\n loops and finish the task gracefully.\n",
|
||||
"description": "Manages the lifecycle and retrieves the status of a background mobile automation task.\n\nThis is your primary diagnostic and control tool for tasks started by\n`mobile_run_task` or the web workspace. Poll with reasonable intervals while\nfollowing a running task; use request diagnostics for unexplained waits.\nYou can also steer a runner that is stuck or off-track, or stop a task.\n\n### Actions\n- lookup: Resolve a submission by request_id without starting or resubmitting it.\n- deletion_preview / delete: Show affected session IDs, then delete only when\n the user requested it (user_confirmed=true). Active tasks cannot be deleted.\n An exploration deletes its conversation runs, preserving independent reruns.\n- review: Record ONLY the user's explicitly stated passed/failed decision with\n review_reason and user_confirmed=true. Never label the agent's verdict as human.\n- **'status'**: Returns `trace_id`, `status` ('running'/'pausing'/'task_paused'/\n 'resuming'/'completed'/'failed'/'cancelled'/'interrupted'), `device_serial` (which phone owns this task in multi-device\n setups), `task_desc`, `model`, `elapsed_seconds`, a `test_summary` (when\n the run declared verification check items: machine-readable\n passed/failed/inconclusive/unchecked counts plus failed-item details —\n no need to parse report prose), `test_result` (Flash and Pro: status\n passed/failed/inconclusive/blocked, reason and evidence; completed only\n means execution ended), and `progress` (Flash:\n current turn, latest thought/action — intervene if turns climb without\n progress or the thought indicates it is stuck; Pro: the active task plan\n — check it still aligns with your goal).\n- **'pause'**: Requests a pause, returning `pausing` until the runner releases\n the device. Pro can interrupt a pending model response or read-only diagnostic;\n a device action already in progress must finish safely first. Check status\n for `task_paused` before assuming the device is free.\n- **'resume'**: Queues a paused task for its device again; optional `instruction`\n provides guidance before the next action.\n- **'inject_instruction'**: Injects real-time guidance mid-flight when the\n subagent errs, stalls, or loops. An in-flight model response or read-only\n helper is cancelled immediately, then the runner observes the device again\n and consumes the new guidance. An already-started device action finishes\n safely before the runner changes direction. Requires\n `instruction` (unless `release_loop=True`). To end a continuous\n monitoring task ([Loop:continuous] milestone) gracefully, you MUST pass\n `release_loop=True` — this is the only signal that unlocks the\n milestone's completion; natural-language \"please stop\" phrasing in\n `instruction` is NOT interpreted as a stop signal.\n- **'guidance_history'**: Lists submitted instructions with `pending` or\n `consumed` status; a submitted instruction is not proof it was applied.\n- **'stop'**: Forcefully terminates the subagent, immediately halting\n device interactions and releasing the device. Use when the task is done,\n irreparably broken, or running out of control. Prefer\n `inject_instruction` with `release_loop=True` when a monitoring task\n should wind down cleanly instead of being killed.\n\nArgs:\n action: `\"status\"`, `\"pause\"`, `\"resume\"`, `\"inject_instruction\"`,\n `\"guidance_history\"`, or `\"stop\"`.\n trace_id: The task's session identifier from `mobile_run_task` or the web UI.\n instruction: Guidance string; required for `inject_instruction`\n (optional when `release_loop=True`), omit otherwise.\n release_loop: With `inject_instruction`: explicit user stop signal\n that authorizes the subagent to complete continuous monitoring\n loops and finish the task gracefully.\n",
|
||||
"inputSchema": {
|
||||
"properties": {
|
||||
"action": {
|
||||
@@ -173,6 +183,7 @@
|
||||
"type": "string"
|
||||
},
|
||||
"trace_id": {
|
||||
"default": "",
|
||||
"title": "Trace Id",
|
||||
"type": "string"
|
||||
},
|
||||
@@ -192,11 +203,44 @@
|
||||
"default": false,
|
||||
"title": "Release Loop",
|
||||
"type": "boolean"
|
||||
},
|
||||
"request_id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Request Id"
|
||||
},
|
||||
"review_status": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Review Status"
|
||||
},
|
||||
"review_reason": {
|
||||
"default": "",
|
||||
"title": "Review Reason",
|
||||
"type": "string"
|
||||
},
|
||||
"user_confirmed": {
|
||||
"default": false,
|
||||
"title": "User Confirmed",
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"action",
|
||||
"trace_id"
|
||||
"action"
|
||||
],
|
||||
"title": "mobile_manage_taskArguments",
|
||||
"type": "object"
|
||||
@@ -209,7 +253,7 @@
|
||||
},
|
||||
{
|
||||
"name": "mobile_run_task",
|
||||
"description": "Starts an autonomous mobile UI automation subagent on a connected Android device.\n\nDelegates a mobile workflow to a background agent. Non-blocking: returns\nimmediately with `trace_id` (for `mobile_manage_task` / `mobile_inspect_trace`),\n`device_serial`, `stdout_log`/`stderr_log` paths, and `notes_dir` (Pro only).\nOn completion (success or failure) a Reactive Wakeup notifies you — but only\nwhen `conversation_id` was provided; without it, rely on your fallback timer.\n\n### Model selection (routing)\n- **Flash** (default, preferred): simple, deterministic tasks with a clear\n UI path. No step cap by default (history is compressed, and the agent\n can recall earlier steps and query the session recording), but no ADB\n shell, no pre-execution safety net, no persistent plan or notes, and\n no midway checkpoints or persistent written notes. An exit-only read-only\n Checker judges the prompt's expectations and returns test_result;\n execution completion is separate from test pass. Unsuitable for exploration,\n multi-branch recovery, structured monitoring/polling, or tasks that\n must report back large amounts of detail.\n- **Pro**: complex, exploratory, or multi-branch tasks; continuous\n monitoring/polling; tasks needing ADB shell, system logs, verified\n checkpoints, multi-step planning, or detailed written output (via notes\n and `expected_output_desc`).\n\nTiming is not precise: the agent's own inference adds ~5 s per step (Flash)\nor ~30 s per turn (Pro) on top of any requested waits — account for this\nwhen the task involves waiting. For recurring workflows, run once to\ndiscover the path, then author a deterministic script instead of\nre-delegating.\n\n### Follow-up protocol (CRITICAL)\n1. **Fallback timer**: you MUST set a background timer and check the task\n status via `mobile_manage_task` at least once every 1 minute — mobile\n tasks can stall silently, so never rely on the completion wakeup alone.\n2. **Post-task inspection**: after completion, inspect the execution logs\n to verify success (`stderr_log` carries critical failures and Python\n tracebacks — essential for debugging).\n3. **Pro notes**: for Pro runs, read `notes_dir` to extract the results and\n plans the subagent recorded.\n\nArgs:\n task_desc: Clear, self-contained task description with all context,\n exact UI actions, and termination conditions (execution is one-shot).\n For Pro, may also specify what the subagent must record in its notes.\n conversation_id: Optional conversation ID used to route the completion\n wakeup notification back to the caller. Omit if unknown.\n model: `\"Flash\"` or `\"Pro\"` — see model selection above.\n locked_app_package: Optional package name to lock execution to; the\n agent auto-launches it and restricts actions to that app.\n app_path: Optional local APK path to install before running.\n expected_output_desc: Optional, Pro only. If set, a summarization agent\n writes a report to `output.md` in `notes_dir`. Ignored for Flash.\n device_serial: Optional device serial (e.g. \"emulator-5554\") to bind\n execution to a specific device; distinct devices run concurrently.\n If omitted, an available device is selected automatically. When\n several devices are attached, confirm the target with the user first\n (`adb devices -l` lists serials and authorization states).\n verification_level: Optional, Pro only. Coarse Checker preset: `\"off\"`\n (no audit; the Operator self-reports), `\"final\"` (one exit review\n against the goal, the default), `\"checkpoints\"` (every plan\n checkpoint is verified + exit review), `\"strict\"` (checkpoints with\n a larger repair budget; a failed assert halts the run). Ignored for\n Flash.\n explorer_mode: Optional, Pro only. Explorer perception version used\n by the Operator: `\"flash\"` (1-shot detection, the default),\n `\"pro\"` (3-turn ReAct), `\"ultra\"` (deep pixel reasoning; slowest).\n Ignored for Flash.\n runtime_overrides: Optional TaskOptions JSON for this task only. Supports\n execution_target ('android', 'computer', 'mixed') and plugins\n ([{id, profile}] references to saved authorized configurations; never secrets).\n Omitted plugins selects matching enabled defaults; [] disables plugins.\n Default Android tasks can use mixed extensions on demand without changing targets.\n Computer tasks require an observation plugin, not an Android device.\n Query mobile_workspace first; installation/authorization is in the plugin page.\n timeout_seconds (total execution, default 1800; excludes queue/pause),\n observation_timeout_seconds (60), observation_max_attempts (3),\n device_command_timeout_seconds (10), action_failure_attempts (3),\n worker_cleanup_timeout_seconds (30),\n model (API model name), reasoning_effort, request timeout/retries,\n reLine truncated
|
||||
"description": "Submit a Flash/Pro Android, computer or mixed task to the user-started ARTEMIS service.\n\nDelegates a mobile workflow to a background agent. Non-blocking: returns\nimmediately with `trace_id` (for `mobile_manage_task` / `mobile_inspect_trace`),\n`device_serial`, `stdout_log`/`stderr_log` paths, and `notes_dir` (Pro only).\nCompletion notifications depend on the caller's configured notification backend;\nconversation_id alone does not guarantee the agent client receives a wakeup.\n\n### Model selection (routing)\n- **Flash** (default, preferred): simple, deterministic tasks with a clear\n UI path. No step cap by default (history is compressed, and the agent\n can recall earlier steps and query the session recording), but no ADB\n shell, no pre-execution safety net, no persistent plan or notes, and\n no midway checkpoints or persistent written notes. An exit-only read-only\n Checker judges the prompt's expectations and returns test_result;\n execution completion is separate from test pass. Unsuitable for exploration,\n multi-branch recovery, structured monitoring/polling, or tasks that\n must report back large amounts of detail.\n- **Pro**: complex, exploratory, or multi-branch tasks; continuous\n monitoring/polling; tasks needing ADB shell, system logs, verified\n checkpoints, multi-step planning, or detailed written output (via notes\n and `expected_output_desc`).\n\nFor interactive case design use mobile_exploration: user guidance, demonstration\nanalysis and reusable case generation. It has independent model settings.\nInference time varies by model, network and device; do not subtract a fixed\nper-step estimate from required waits. Deterministic test code needs explicit\ntiming and state-based waits verified against the real application.\n\n### Follow-up\n1. Poll mobile_manage_task at reasonable intervals (usually 30–60 seconds)\n while following a task; do not assume an unconfigured background notification.\n2. Read test_result and view_report after completion. Inspect relevant request,\n log or step evidence for failures and unexplained waits.\n3. Read Pro notes when its plan or detailed output is relevant to the user.\n\nReuse request_id for retries of the same submission; lookup with\nmobile_manage_task(action=\"lookup\", request_id=...). Never repeat a task\nwith a fresh ID merely because its admission reply was lost.\n\nArgs:\n request_id: Stable caller-generated ID (letters, numbers, _ or -, up to 100).\n task_desc: Clear, self-contained task description with all context,\n exact UI actions, and termination conditions (execution is one-shot).\n For Pro, may also specify what the subagent must record in its notes.\n conversation_id: Optional conversation ID used to route the completion\n wakeup notification back to the caller. Omit if unknown.\n model: `\"Flash\"` or `\"Pro\"` — see model selection above.\n locked_app_package: Optional package name to lock execution to; the\n agent auto-launches it and restricts actions to that app.\n app_path: Optional local APK path to install before running.\n expected_output_desc: Optional, Pro only. If set, a summarization agent\n writes a report to `output.md` in `notes_dir`. Ignored for Flash.\n device_serial: Optional device serial (e.g. \"emulator-5554\") to bind\n execution to a specific device; distinct devices run concurrently.\n If omitted, an available device is selected automatically. When\n several devices are attached, confirm the target with the user first\n (`adb devices -l` lists serials and authorization states).\n verification_level: Optional, Pro only. Coarse Checker preset: `\"off\"`\n (no audit; the Operator self-reports), `\"final\"` (one exit review\n against the goal, the default), `\"checkpoints\"` (every plan\n checkpoint is verified + exit review), `\"strict\"` (checkpoints with\n a larger repair budget; a failed assert halts the run). Ignored for\n Flash.\n explorer_mode: Optional, Pro only. Explorer perception version used\n by the Operator: `\"flash\"` (1-shot detection, the default),\n `\"pro\"` (3-turn ReAct), `\"ultra\"` (deep pixel reasoning; slowest).\n Ignored for Flash.\n runtime_overrides: Optional TaskOptions JSON for this task only. Supports\n execution_target ('android', 'computer', 'mixed') and plugins\n ([{id, profile}] references to saved authorized configurations; never secrets).\n Omitted plugins selects matching enabled defaults; [] disables plugins.\n Default Android tasks can use mixed extensions on demand without changing targets.\n Computer tasks require an observation plugin, not an Android device.\n Query mobile_workspace first; installation/authorization is in the plugin page.\n timeout_seconds (total execution, default 1800; excludes queue/pause),\n observation_timeout_seconds (60), observation_max_attempts (3Line truncated
|
||||
"inputSchema": {
|
||||
"properties": {
|
||||
"task_desc": {
|
||||
@@ -317,6 +361,18 @@
|
||||
],
|
||||
"default": null,
|
||||
"title": "Runtime Overrides"
|
||||
},
|
||||
"request_id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Request Id"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -333,13 +389,40 @@
|
||||
},
|
||||
{
|
||||
"name": "mobile_workspace",
|
||||
"description": "List connected Android devices and the unified running/queued task state.\n\nSelect the user's device; if multiple devices exist and none is chosen, ask first.\nThis does not submit tasks. On connection/device errors use mobile_diagnose.\nFor computer-only work use include_devices=false to skip ADB discovery entirely.\n",
|
||||
"description": "List connected Android devices and the unified running/queued task state.\n\nSelect the user's device; if multiple devices exist and none is chosen, ask first.\nplugin_id filters the installed plugin list. probe_plugin=true performs a bounded\nconnection/schema probe of that installed, authorized plugin (starts its runtime,\nexecutes no task action). It never installs or grants permissions.\nThis does not submit tasks. On connection/device errors use mobile_diagnose.\nFor computer-only work use include_devices=false to skip ADB discovery entirely.\n",
|
||||
"inputSchema": {
|
||||
"properties": {
|
||||
"include_devices": {
|
||||
"default": true,
|
||||
"title": "Include Devices",
|
||||
"type": "boolean"
|
||||
},
|
||||
"plugin_id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
],
|
||||
"default": null,
|
||||
"title": "Plugin Id"
|
||||
},
|
||||
"probe_plugin": {
|
||||
"default": false,
|
||||
"title": "Probe Plugin",
|
||||
"type": "boolean"
|
||||
},
|
||||
"profile": {
|
||||
"default": "default",
|
||||
"title": "Profile",
|
||||
"type": "string"
|
||||
},
|
||||
"include_development": {
|
||||
"default": false,
|
||||
"title": "Include Development",
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"title": "mobile_workspaceArguments",
|
||||
@@ -531,6 +614,11 @@
|
||||
"default": 30,
|
||||
"title": "Size",
|
||||
"type": "integer"
|
||||
},
|
||||
"batch_id": {
|
||||
"default": "",
|
||||
"title": "Batch Id",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"title": "mobile_search_historyArguments",
|
||||
@@ -595,12 +683,14 @@
|
||||
},
|
||||
{
|
||||
"name": "mobile_exploration",
|
||||
"description": "Explore ONE complex test with user guidance and produce a reusable case.\n\nRead-only list/get/case never execute tasks. create stores {goal,config?,task_config?};\nstart/continue explicitly enqueue or resume Pro exploration. Independent defaults:\ngpt-6.1-sol/high, 1800 active seconds; timeout pauses, never fails by itself.\nmessage payload={text}; answer pending questions or guide without discarding history.\ndemonstrate payload={source:'android'|'computer'|'both'} pauses at a safe point and\nrecords while retaining the device/plugin lease. Wait for recording before the user\noperates. demo_finish analyzes the SAVED clip; demo_extend payload={seconds:900};\ndemo_cancel preserves evidence. Default 900s, warning 60s before expiry, auto-pause.\nsave_case payload={expected_version,case:{name,goal,preconditions?,evidence?}};\ngenerate payload={source_run_id,expected_version} starts evidence-based synthesis,\npoll get.generations for completion. validate payload={version,profile,task_config?}\nexplicitly runs the saved version in Flash/Pro; batch additionally accepts name and\ncreates a DRAFT batch, never starts it. Exploration pass and rerun pass are separate.\nUse stable request_id for every mutation except pause/stop, reusing it on transport\nretries. No API keys; configure secrets in the Web UI. mobile_config(section='exploration')\ncontrols non-secret defaults. After errors call mobile_diagnose before resubmitting.\n",
|
||||
"description": "Explore ONE complex test with user guidance and produce a reusable case.\n\ndeletion_preview lists affected conversation runs; delete requires user_confirmed=true\nafter the user requests deletion. Independent validation runs are preserved.\nRead-only list/get/case never execute tasks. create stores {goal,config?,task_config?};\nstart/continue explicitly enqueue or resume Pro exploration. Independent defaults:\ngpt-6.1-sol/high, 1800 active seconds; timeout pauses, never fails by itself.\nmessage payload={text}; answer pending questions or guide without discarding history.\ndemonstrate payload={source:'android'|'computer'|'both'} pauses at a safe point and\nrecords while retaining the device/plugin lease. Wait for recording before the user\noperates. demo_finish analyzes the SAVED clip; demo_extend payload={seconds:900};\ndemo_cancel preserves evidence. Default 900s, warning 60s before expiry, auto-pause.\nsave_case payload={expected_version,case:{name,goal,preconditions?,evidence?}};\ngenerate payload={source_run_id,expected_version} starts evidence-based synthesis,\npoll get.generations for completion. validate payload={version,profile,task_config?}\nexplicitly runs the saved version in Flash/Pro; batch additionally accepts name and\ncreates a DRAFT batch, never starts it. Exploration pass and rerun pass are separate.\nUse stable request_id for every mutation except pause/stop/delete, reusing it on transport\nretries. No API keys; configure secrets in the Web UI. mobile_config(section='exploration')\ncontrols non-secret defaults. After errors call mobile_diagnose before resubmitting.\n",
|
||||
"inputSchema": {
|
||||
"properties": {
|
||||
"action": {
|
||||
"enum": [
|
||||
"list",
|
||||
"deletion_preview",
|
||||
"delete",
|
||||
"get",
|
||||
"create",
|
||||
"start",
|
||||
@@ -686,6 +776,11 @@
|
||||
],
|
||||
"default": null,
|
||||
"title": "Version"
|
||||
},
|
||||
"user_confirmed": {
|
||||
"default": false,
|
||||
"title": "User Confirmed",
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
|
||||
+3
-74
@@ -1,76 +1,5 @@
|
||||
# Mobile Testing Mindset (ARTEMIS Integration)
|
||||
# ARTEMIS 客户端规则
|
||||
|
||||
The user must start the ARTEMIS web service before tool use. MCP only connects to it; never auto-start a daemon or run a standalone agent as a fallback. An offline service does not prevent MCP initialization. When tools report it offline, ask the user to start ARTEMIS rather than retrying diagnostics or submitting tasks.
|
||||
官方客户端说明已迁移到 [artemis-use](../skills/artemis-use/SKILL.md)，包含 Flash、Pro、探索模式、任务诊断与插件开发规范。按任务读取 Skill 中相应参考，不再把整份规则注入全局 AGENTS.md。
|
||||
|
||||
For guided design of one complex case use `mobile_exploration`, sharing the Web conversation and queue. Independent exploration defaults are `gpt-6.1-sol/high`; active execution timeout pauses with saved progress, not failure. `get/list/case` are read-only; mutations use stable request IDs. Wait for `recording` before manual demonstration, then finish to analyze the saved clip. A demonstration is not an independent successful rerun. Cases are versioned; Flash/Pro reruns have separate actual verdicts. See [exploration workflow](../docs/exploration-mode.md).
|
||||
|
||||
Write executable tests based on device behavior verified with **ARTEMIS**. Before authoring a test, explore the target application and confirm its screens, transitions, and interactions. Use ARTEMIS device actions or ADB commands to investigate software and hardware behavior instead of assuming how an interaction works.
|
||||
|
||||
### 1. The Runnable Code Principle & ARTEMIS Exploration
|
||||
- **When tasked with authoring tests**, deliver runnable test code with verified interactions and explicit wait conditions.
|
||||
- Before writing any test code, you must use the ARTEMIS MCP tools to interactively run and explore the target application. This allows you to discover the exact sequence of UI states, transitions, and required interactions.
|
||||
- Analyze the user's target testing framework to exploit its native capabilities and maximize test stability.
|
||||
- **Timing & Latency Management (Exploration vs. Execution)**:
|
||||
- **Precise Timing in Final Code**: While ARTEMIS's AI exploration inherently involves model latency and is not strictly time-precise, you must bridge this gap in your final deliverables. Use ARTEMIS to discover and verify the interaction path, then implement exact, deterministic timing and wait conditions (`sleep`, explicit/implicit waits) in your authored test scripts, as local test execution runs without LLM overhead.
|
||||
- **Compensating for Model Latency During Exploration**: When delegating exploratory tasks to ARTEMIS that involve waiting periods (e.g., waiting 30 seconds for a page load or timer), adjust the requested wait duration in your task description based on the selected model's natural step interval:
|
||||
- **Flash Model**: The average processing interval between steps is roughly **5 seconds**. For a required 30-second delay, instruct the agent to wait for approximately **25 seconds**.
|
||||
- **Pro Model**: The processing interval between turns is typically **~30 seconds** (due to multi-agent planning and verification). The natural pipeline delay often covers the required waiting time without adding long explicit delay commands.
|
||||
- **Pragmatic Timing Judgment**: When a user request mentions performing an action for a specific duration (e.g., "stay on this screen for 2 minutes"), evaluate whether exact timing is functionally critical. Often, these durations are rough guidelines rather than strict test constraints—exercise flexibility and pragmatic engineering judgment to achieve the verification goal efficiently.
|
||||
|
||||
|
||||
### 2. ARTEMIS Closed-Loop Architecture Mastery
|
||||
Deeply understand and select between ARTEMIS's dual execution models (**ARTEMIS Flash** and **ARTEMIS Pro**) based on the task scenario, and master their corresponding workflows:
|
||||
|
||||
- **ARTEMIS Flash (Fast / Reactive Model)**:
|
||||
- **Applicable Scenarios**: Designed for simple, highly deterministic, lightweight UI operations or direct automation workflows. There is no step cap by default (history is compressed rather than truncated), so task length alone is not a reason to choose Pro; choose Pro when the task needs a persistent plan, verified checkpoints, notes or a written report, ADB / log diagnostics, or multi-branch exploration.
|
||||
- **Execution Mechanism (Reactive Loop)**: Does not enter the LangGraph multi-node orchestration: there is no Planner, no pre-execution safety net, and no Checker. `FlashRunner` receives the user's goal, the latest UI element list, and the live screenshot, performs "Observe-Think-Act" with a single LLM, and calls action tools until it reports the task status. It chains taps into one `click_sequence` to catch transient UI (auto-fading control bars, toasts) before they expire, and it can call `ask_explorer` for element grounding, `search_history` / `replay_steps` / `get_step_screenshot` to look up compressed earlier steps, and `video_analyzer` over the session recording. History lives in the same session transcript ledger as Pro (session-relative `T+mm:ss` clock, screenshots folded into visual summaries, older steps chunked into eras).
|
||||
|
||||
- **ARTEMIS Pro (Deep / Multi-Agent Graph-Driven Closed-Loop Model)**:
|
||||
- **Applicable Scenarios**: Designed for long-range, highly complex, dynamic multi-branch tasks; continuous monitoring / polling loops; and tasks requiring deep system diagnostics (ADB, logs, video), verified checkpoints, or a detailed written report.
|
||||
- **Execution Mechanism (Plan-Execute-Verify-Summarize Closed-Loop)**:
|
||||
- **Planner**: Deconstructs complex, high-level testing goals into a living Markdown task plan with milestones and `verify` / `assert` check items. Later milestone edits get an advisory review (a hint plus reason back to the Operator; the plan is never rolled back).
|
||||
- **Operator**: Consumes the plan, analyzes the current screen state (screenshots plus the UI element list, with the Explorer for grounding at the `explorer_mode` perception depth: `flash` 1-shot, `pro` 3-turn, `ultra` deep zoom), keeps notes, and executes precise device interactions; it can also recall compressed history, analyze the session recording, and run ADB commands as a supplement to UI actions.
|
||||
- **Safety Net & Execution Incidents**: A single turn-ending action is vetted by a pre-execution safety net (XML-first, pixel fallback). A multi-action turn is a **fast-action burst** that fires back to back without the safety net, which is how the Operator defeats turn latency on transient UI. When an action is intercepted or fails, the system opens an **execution incident** that persists in the Operator's context (with the consecutive-failure count shown as a fact) until a later action executes successfully; recovery is the Operator's own decision and there is no separate repair agent.
|
||||
- **Checker (Verification)**: A read-only verifier with the same observation tools as the Operator audits plan-declared checkpoints and performs an exit final review against the original goal; `verification_level` selects the depth (`off`, `final` = exit review only and the default, `checkpoints` = every checkpoint plus exit review, `strict` = checkpoints with a larger repair budget where a failed assert halts the run).
|
||||
- **Outputter (Optional)**: Synthesizes the entire execution trace into a human-readable report detailing every action step and visual result.
|
||||
|
||||
### 3. Device & Environment Constraints & Multi-Device Management
|
||||
- **Device Selection & Multi-Device Execution**: ARTEMIS supports multi-device execution and per-device concurrency. You can control device targeting via two modes:
|
||||
- **Direct Device Specification**: Explicitly provide the target phone's serial number via `device_serial` to `mobile_run_task` or `mobile_get_device_state`. Tasks targeting distinct devices run concurrently without blocking each other.
|
||||
- **Automatic Device Selection**: When `device_serial` is omitted or set to `None`, ARTEMIS automatically selects an available connected device or allocates an idle device from the device pool.
|
||||
- **Prioritize User Choice for Device Selection**:
|
||||
- When multiple connected devices or emulators are detected, or whenever device selection is ambiguous, **YOU MUST PRIORITIZE ASKING THE USER** to select or confirm their preferred device serial before launching a task.
|
||||
- **Device Diagnosis with `adb devices`**:
|
||||
- Use `adb devices` (or `adb devices -l`) via bash command execution to diagnose attached hardware, inspect connection status (`device`, `unauthorized`, `offline`), and retrieve device serials and models whenever preparing tasks or troubleshooting device issues.
|
||||
- **ADB & File Transfer**: While ARTEMIS excels at device automation, it operates within the device boundary. To extract diagnostic files, logs, or test artifacts from the mobile device to the host PC for further analysis, you must manually execute appropriate `adb` commands (e.g., `adb -s <serial> pull ...`).
|
||||
- **Hardware Prerequisites**: Running ARTEMIS requires at least one physically connected, fully authorized Android device (e.g., a Pixel phone) or an active emulator.
|
||||
- **Per-Device Mutual Exclusion**: ARTEMIS manages per-device execution mutexes (`DeviceExecutionLock`). A device can only execute a single task at a time (FIFO queue), while different devices can execute tasks in parallel.
|
||||
|
||||
### 4. Robust Test Code Design (The "Dynamic-First, Coordinate-Fallback" Philosophy)
|
||||
*If your task involves authoring test code, you must adhere to the following design principles for maximum reliability:*
|
||||
- **Adapt to Framework Capabilities**: You must first assess what locating mechanisms the user's test framework supports (e.g., resource IDs, XPath, text matching, OCR, image template matching, or absolute/relative coordinates).
|
||||
- **The Core Principle: Dynamic-First, Coordinate-Fallback**:
|
||||
- Wherever supported by the framework, **always prioritize dynamic element locating** (using IDs, text, OCR, etc.) to ensure the test code can withstand UI layout drifts and resolution changes.
|
||||
- Use **absolute coordinates as a reliable fallback** to guarantee execution success when dynamic locators fail or are unavailable.
|
||||
- **Implementing Resilient Locating Patterns**:
|
||||
- **Dual-Capable Frameworks**: If the framework supports both dynamic and coordinate-based locating, implement the **Try-Catch Fallback Pattern**:
|
||||
1. **Try**: Attempt to interact with the element using dynamic locators (IDs, OCR, text) for maximum resilience against UI changes.
|
||||
2. **Catch**: If the dynamic attempt fails, fall back to the precise absolute coordinates verified during your ARTEMIS exploration.
|
||||
- **Coordinate-Only Frameworks**: If the framework only supports coordinates, ensure the coordinates are well-documented, and where possible, parameterized or made relative to screen boundaries to mitigate resolution differences.
|
||||
- **Dynamic-Only Frameworks**: If the framework does not support coordinate-based clicks, focus entirely on generating highly robust dynamic locators, leveraging ARTEMIS's element descriptions and XML tree analysis.
|
||||
|
||||
### 5. Environment Self-Diagnosis (`mobile_diagnose`)
|
||||
- **Diagnose before guessing**: Whenever an ARTEMIS tool returns an error, a task fails to start or is rejected, no device is found, or the user reports that ARTEMIS "doesn't work" in the IDE, call `mobile_diagnose` first. Do not troubleshoot by re-running `mobile_run_task` on trial and error, and do not ask the user to run `adb` commands by hand before you have the report.
|
||||
- **If `mobile_diagnose` is not in your tool list at all**, the MCP server itself failed to start: check the IDE's MCP server log and run `uv run artemis doctor` in the project directory (the CLI shares the same checks).
|
||||
- **Follow `next_steps` in order**: The report lists fixes in dependency order (Python runtime → config → MCP host → LLM credentials → device → optional video toolchain). `Run:` lines are single, local, non-destructive shell commands (one command per line, no `&&`) you may execute yourself (ask before installing software with winget / brew / apt). `Guidance:` lines describe something only the user can do (unlock the phone, tap "Allow" on the USB-debugging prompt, plug in a cable, free a port, tell you which IDE they use) — relay them verbatim. `Docs:` lines are references.
|
||||
- **Let ARTEMIS self-heal first**: Call `mobile_diagnose(attempt_fix=true)` before asking for manual ADB intervention; it regenerates corrupted ADB RSA keys, restarts the ADB server when that is safe (no device ready, no task holding a device), and removes stale device locks / queue tickets left behind by crashed runners.
|
||||
- **Deeper checks on demand**: `verify_credentials=true` validates the configured API keys live (~12s; use when a key exists but tasks fail with auth / quota errors; result in `credentials`). `probe_device=true` runs a true end-to-end check — screenshot plus UI hierarchy through UIAutomator (~20s; use when the device shows connected but observation or tasks fail; result in `device_probe`).
|
||||
- **Emulators**: `launch_avd="<name>"` starts an installed Android Virtual Device in the background via ARTEMIS's emulator manager and returns immediately; boot takes 1-3 minutes, so re-run `mobile_diagnose` after ~60s. Never run `emulator -avd` in the shell yourself — it is a foreground process and hangs the shell.
|
||||
- **Busy devices**: `tasks: {active, queued}` lists tasks currently holding or waiting for devices. A busy device means wait, or stop the task with `mobile_manage_task(action="stop", trace_id=...)`.
|
||||
- **Credentials never go through the chat**: If the LLM key is missing, ask the user to add it to the `.env` file at the path the report shows (`host.env_file`) or to the MCP server's `env` block, then to restart the MCP server. `artemis init` is interactive and cannot be driven from a tool call.
|
||||
- **Restart after configuration changes**: Settings are read when the server starts. After editing `.env`, `config/artemis.jsonc`, or the MCP client configuration, reload the MCP server in the IDE before calling `mobile_diagnose` again; otherwise it reports the old state.
|
||||
- **Failed tasks**: `logs.last_failed_task` gives the trace id, its `stderr_log`, and `recent_errors` (the error lines from that stderr, so you need not open the file); read them before re-running the task. Repeat `mobile_diagnose` until `verdict` is `ready` or `degraded` before delegating tasks again (`degraded` means tasks can run but something optional is off: video tools, the OCR key, a secondary credential, or a host warning such as an interpreter mismatch or a squatted daemon port; relay its `[OPTIONAL]` steps to the user, do not loop on them).
|
||||
|
||||
## Full Workflow Tools
|
||||
|
||||
For AI-operated batches, configuration, history search and Word/Excel reports, use the distributed `skills/artemis-automation/SKILL.md`. `mobile_workspace` lists devices/queue; `mobile_config` reads defaults and only saves non-secret changes on an explicit user request. `mobile_task_file` previews XLSX; `mobile_batch` accepts JSON cases directly (create a draft with a stable request_id, then start when execution is authorized). `mobile_search_history` searches all terminal tasks; `mobile_export_report` returns real file paths. Flash/Pro are execution profiles, not API model names. Video recording/analysis default off. Action replay is removed; trace inspection remains read-only.
|
||||
启动 ARTEMIS 会自动同步该 Skill；显式修复可执行 `uv run artemis mcp --install codex`。安装成功后，安装器只移除旧全局文件内带 ARTEMIS 标记的规则块，保留用户其他说明。
|
||||
+13
-2
@@ -21,7 +21,8 @@ class ToolExecutor:
|
||||
"""One shared tool loop; synchronous legacy HTTP calls cannot block the UI.
|
||||
|
||||
Device clients keep their loop affinity across MCP connections. Cancellation
|
||||
propagates to the tool coroutine instead of abandoning a thread per request.
|
||||
propagates to async tools; dispatched synchronous writes may finish in their
|
||||
worker thread, so callers must resolve the original request rather than retry it.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
@@ -45,7 +46,17 @@ class ToolExecutor:
|
||||
async def invoke(self, fn, *args, **kwargs):
|
||||
if self.loop is None or not self.loop.is_running():
|
||||
raise RuntimeError("ARTEMIS 主程序正在启动或关闭，请稍后重试。")
|
||||
future = asyncio.run_coroutine_threadsafe(fn(*args, **kwargs), self.loop)
|
||||
|
||||
async def dispatch():
|
||||
if inspect.iscoroutinefunction(fn):
|
||||
return await fn(*args, **kwargs)
|
||||
# Legacy tools use synchronous HTTP to call this same web service.
|
||||
# Execute them off both event loops, and never call them speculatively
|
||||
# to discover whether their return value is awaitable.
|
||||
result = await asyncio.to_thread(fn, *args, **kwargs)
|
||||
return await result if inspect.isawaitable(result) else result
|
||||
|
||||
future = asyncio.run_coroutine_threadsafe(dispatch(), self.loop)
|
||||
return await asyncio.wrap_future(future)
|
||||
|
||||
async def stop(self):
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Durable admission receipts in the existing automation store; never replay a write."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
from apps.admin_console.services.automation_store import identifier, store
|
||||
|
||||
|
||||
def lookup(request_id: str) -> dict:
|
||||
record = store.get("mcp_task_request", identifier(request_id))
|
||||
if record is None:
|
||||
return {"request_id": request_id, "status": "not_found"}
|
||||
job = store.job(record["trace_id"])
|
||||
return {
|
||||
**record.get("response", {}),
|
||||
**({"status": job["status"]} if job else {}),
|
||||
"status_scope": "task_queue" if job else "submission_receipt",
|
||||
"request_id": request_id,
|
||||
"trace_id": record["trace_id"],
|
||||
"submission_state": record["state"],
|
||||
"message": "请求已登记；请按 trace_id 查询详细状态，不要用新 request_id 自动重提。",
|
||||
}
|
||||
|
||||
|
||||
def run_once(request_id: str, parameters: dict, execute) -> dict:
|
||||
identifier(request_id)
|
||||
fingerprint = hashlib.sha256(
|
||||
json.dumps(parameters, sort_keys=True, ensure_ascii=False).encode("utf-8")
|
||||
).hexdigest()
|
||||
with store.connection() as connection:
|
||||
connection.execute("BEGIN IMMEDIATE")
|
||||
row = connection.execute(
|
||||
"SELECT payload FROM automation_records WHERE kind='mcp_task_request' AND id=?",
|
||||
(request_id,),
|
||||
).fetchone()
|
||||
if row:
|
||||
if json.loads(row[0])["fingerprint"] != fingerprint:
|
||||
raise ValueError("request_id 已用于不同任务内容，请使用新的编号")
|
||||
connection.commit()
|
||||
return {**lookup(request_id), "deduplicated": True}
|
||||
record = {
|
||||
"trace_id": str(uuid4()),
|
||||
"fingerprint": fingerprint,
|
||||
"state": "submitting",
|
||||
}
|
||||
connection.execute(
|
||||
"INSERT INTO automation_records VALUES ('mcp_task_request',?,?,?)",
|
||||
(request_id, json.dumps(record), time.time()),
|
||||
)
|
||||
connection.commit()
|
||||
try:
|
||||
response = execute(record["trace_id"])
|
||||
except Exception:
|
||||
# A lost reply says nothing about whether the queue accepted the task.
|
||||
store.put("mcp_task_request", request_id, {**record, "state": "unknown"})
|
||||
raise
|
||||
store.put("mcp_task_request", request_id, {**record, "state": "returned", "response": response})
|
||||
return {**response, "request_id": request_id}
|
||||
@@ -121,10 +121,19 @@ async def _download(
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def mobile_workspace(include_devices: bool = True) -> dict:
|
||||
async def mobile_workspace(
|
||||
include_devices: bool = True,
|
||||
plugin_id: str | None = None,
|
||||
probe_plugin: bool = False,
|
||||
profile: str = "default",
|
||||
include_development: bool = False,
|
||||
) -> dict:
|
||||
"""List connected Android devices and the unified running/queued task state.
|
||||
|
||||
Select the user's device; if multiple devices exist and none is chosen, ask first.
|
||||
plugin_id filters the installed plugin list. probe_plugin=true performs a bounded
|
||||
connection/schema probe of that installed, authorized plugin (starts its runtime,
|
||||
executes no task action). It never installs or grants permissions.
|
||||
This does not submit tasks. On connection/device errors use mobile_diagnose.
|
||||
For computer-only work use include_devices=false to skip ADB discovery entirely.
|
||||
"""
|
||||
@@ -132,10 +141,36 @@ async def mobile_workspace(include_devices: bool = True) -> dict:
|
||||
_request("GET", "/api/status"), _request("GET", "/api/plugins")
|
||||
)
|
||||
devices = await _request("GET", "/api/devices") if include_devices else {}
|
||||
installed = plugins.get("plugins", [])
|
||||
if plugin_id:
|
||||
_id(plugin_id)
|
||||
installed = [
|
||||
item for item in installed if (item.get("manifest") or item).get("id") == plugin_id
|
||||
]
|
||||
if not installed:
|
||||
raise ValueError("插件未安装，请在网页插件页安装并授权")
|
||||
development = None
|
||||
if include_development:
|
||||
from artemis import __version__
|
||||
import sys
|
||||
|
||||
development = {
|
||||
"python_executable": sys.executable,
|
||||
"artemis_version": __version__,
|
||||
"plugin_schema_version": 2,
|
||||
"installation_root": str(ROOT_DIR),
|
||||
}
|
||||
probe = None
|
||||
if probe_plugin:
|
||||
probe = await _request(
|
||||
"POST", f"/api/plugins/{_id(plugin_id)}/diagnose", json={"profile": profile}
|
||||
)
|
||||
return {
|
||||
"devices": devices.get("devices", []),
|
||||
"queue": status,
|
||||
"plugins": plugins.get("plugins", []),
|
||||
"plugins": installed,
|
||||
"plugin_probe": probe,
|
||||
"development": development,
|
||||
"blocked_resources": plugins.get("blocked_resources", {}),
|
||||
"execution_targets": ["android", "computer", "mixed"],
|
||||
}
|
||||
@@ -298,6 +333,7 @@ async def mobile_search_history(
|
||||
end: float | None = None,
|
||||
page: int = 1,
|
||||
size: int = 30,
|
||||
batch_id: str = "",
|
||||
) -> dict:
|
||||
"""Search ALL persisted terminal tasks by name, prompt, ID or case metadata, with pagination.
|
||||
|
||||
@@ -306,6 +342,8 @@ async def mobile_search_history(
|
||||
Return items,total,page,size; inspect evidence/report before claiming a test passed.
|
||||
"""
|
||||
params = {"q": query, "status": status, "page": page, "size": size}
|
||||
if batch_id:
|
||||
params["batch_id"] = _id(batch_id)
|
||||
if start is not None:
|
||||
params["start"] = start
|
||||
if end is not None:
|
||||
|
||||
@@ -10,6 +10,8 @@ from mcp_server.tools.automation import _id, _request
|
||||
async def mobile_exploration(
|
||||
action: Literal[
|
||||
"list",
|
||||
"deletion_preview",
|
||||
"delete",
|
||||
"get",
|
||||
"create",
|
||||
"start",
|
||||
@@ -33,9 +35,12 @@ async def mobile_exploration(
|
||||
demonstration_id: str | None = None,
|
||||
after: int = 0,
|
||||
version: int | None = None,
|
||||
user_confirmed: bool = False,
|
||||
) -> Any:
|
||||
"""Explore ONE complex test with user guidance and produce a reusable case.
|
||||
|
||||
deletion_preview lists affected conversation runs; delete requires user_confirmed=true
|
||||
after the user requests deletion. Independent validation runs are preserved.
|
||||
Read-only list/get/case never execute tasks. create stores {goal,config?,task_config?};
|
||||
start/continue explicitly enqueue or resume Pro exploration. Independent defaults:
|
||||
gpt-6.1-sol/high, 1800 active seconds; timeout pauses, never fails by itself.
|
||||
@@ -49,11 +54,19 @@ async def mobile_exploration(
|
||||
poll get.generations for completion. validate payload={version,profile,task_config?}
|
||||
explicitly runs the saved version in Flash/Pro; batch additionally accepts name and
|
||||
creates a DRAFT batch, never starts it. Exploration pass and rerun pass are separate.
|
||||
Use stable request_id for every mutation except pause/stop, reusing it on transport
|
||||
Use stable request_id for every mutation except pause/stop/delete, reusing it on transport
|
||||
retries. No API keys; configure secrets in the Web UI. mobile_config(section='exploration')
|
||||
controls non-secret defaults. After errors call mobile_diagnose before resubmitting.
|
||||
"""
|
||||
base = "/api/explorations"
|
||||
if action in {"deletion_preview", "delete"}:
|
||||
path = f"{base}/{_id(exploration_id)}"
|
||||
if action == "deletion_preview":
|
||||
return await _request("GET", path + "/deletion-preview")
|
||||
if not user_confirmed:
|
||||
raise ValueError("删除探索需要用户明确要求；先用 deletion_preview 核对影响范围")
|
||||
return await _request("DELETE", path)
|
||||
|
||||
if action in {"list", "get", "case"}:
|
||||
if payload or request_id:
|
||||
raise ValueError("查询不接受写入内容")
|
||||
|
||||
@@ -57,6 +57,8 @@ async def mobile_inspect_trace(
|
||||
query: str | None = None,
|
||||
step_range: list[int] | None = None,
|
||||
max_results: int = 5,
|
||||
offset: int = 0,
|
||||
limit: int = 20,
|
||||
) -> Any:
|
||||
"""Retrieves execution details of a mobile automation task (running or finished).
|
||||
|
||||
@@ -64,8 +66,12 @@ async def mobile_inspect_trace(
|
||||
both Flash and Pro tasks. Every result includes the assigned `device_serial`.
|
||||
|
||||
### Actions
|
||||
- view_report / view_usage / view_requests / view_logs / view_evidence:
|
||||
bounded read-only report, measured token/cache totals, request phases and IDs,
|
||||
redacted log tails, or source-specific evidence index. offset/limit paginate;
|
||||
start with view_report, then inspect only relevant evidence. No export is generated.
|
||||
- **'view_summary'**: High-level execution summary across all steps.
|
||||
Pro: hierarchical task plan with per-step status; Flash: full execution
|
||||
Pro: hierarchical task plan with per-step status; Flash: paginated execution
|
||||
chain with each step's reasoning and action.
|
||||
- **'search'**: Deterministic keyword / step-range lookup over the full
|
||||
stored history (screen descriptions, exact actions and results,
|
||||
@@ -74,15 +80,15 @@ async def mobile_inspect_trace(
|
||||
its step number; a `step_range` also returns that range's per-step
|
||||
action ledger. Requires `query` and/or `step_range`.
|
||||
- **'view_step_screenshots'**: Local file paths of one step's screenshots:
|
||||
`before_screenshot` (what the agent saw), `after_screenshot` (only set
|
||||
when the action failed/was intercepted, else null), and
|
||||
`before_screenshot` (what the agent saw), `after_screenshot` (when actually captured, otherwise null), and
|
||||
`action_overlay_screenshot` (the action visually marked — e.g. red circle
|
||||
for taps — key for verifying the agent tapped the right element).
|
||||
- **'view_step_details'**: Stored details of one step exactly as the agent's
|
||||
own context showed it: what the screen showed, its reasoning, every tool
|
||||
call it made (name, arguments, result — e.g. what the explorer or OCR
|
||||
reported), the planned action, any safety-net interception and the
|
||||
execution result. Coordinates are normalized [x, y]. Works for Flash and
|
||||
execution result. Coordinate units depend on the tool: Cua uses screenshot
|
||||
pixels; consult the recorded source geometry, never reuse another window. Works for Flash and
|
||||
Pro tasks.
|
||||
|
||||
Args:
|
||||
@@ -97,6 +103,17 @@ async def mobile_inspect_trace(
|
||||
`"search"`.
|
||||
max_results: Maximum `"search"` hits to return (server-side cap applies).
|
||||
"""
|
||||
from mcp_server.tools.automation import _id, _request
|
||||
|
||||
_id(trace_id)
|
||||
if offset < 0 or not 1 <= limit <= 100:
|
||||
raise ValueError("offset 必须非负，limit 必须在 1–100 之间")
|
||||
if action in {"view_report", "view_usage", "view_requests", "view_logs", "view_evidence"}:
|
||||
return await _request(
|
||||
"GET",
|
||||
f"/api/sessions/{trace_id}/inspection",
|
||||
params={"view": action.removeprefix("view_"), "offset": offset, "limit": limit},
|
||||
)
|
||||
project_root = env_utils.get_project_root()
|
||||
db_path = os.path.join(trace_store.TRACES_DIR, "data_engine.db")
|
||||
if not os.path.exists(db_path):
|
||||
@@ -135,8 +152,8 @@ async def mobile_inspect_trace(
|
||||
session_start_time = session_row["start_time"] if session_row else None
|
||||
|
||||
cursor.execute(
|
||||
"SELECT * FROM steps WHERE session_id = ? ORDER BY step_number ASC",
|
||||
(trace_id,),
|
||||
"SELECT * FROM steps WHERE session_id = ? ORDER BY step_number ASC LIMIT ? OFFSET ?",
|
||||
(trace_id, limit, offset),
|
||||
)
|
||||
rows = cursor.fetchall()
|
||||
|
||||
@@ -252,7 +269,10 @@ async def mobile_inspect_trace(
|
||||
summary_lines.append(f"### Step {s_num} (at {rel_t}, duration {dur_str})")
|
||||
summary_lines.append(f"- **Thinking / Motivation:** {thought}")
|
||||
summary_lines.append(f"- **Action Taken:** `{act_str}` -> `{res}`\n")
|
||||
return "\n".join(summary_lines)
|
||||
return (
|
||||
"\n".join(summary_lines)[:40000]
|
||||
+ f"\n\nPage offset={offset}, limit={limit}; use offset={offset + limit} for more steps."
|
||||
)
|
||||
|
||||
try:
|
||||
summary_markdown = build_plan_and_history(
|
||||
@@ -268,7 +288,10 @@ async def mobile_inspect_trace(
|
||||
f"**Session ID:** `{trace_id}` | **Model:** Pro{device_info_str}\n\n"
|
||||
+ summary_markdown
|
||||
)
|
||||
return summary_markdown
|
||||
return (
|
||||
summary_markdown[:40000]
|
||||
+ f"\n\nPage offset={offset}, limit={limit}; use offset={offset + limit} for more steps."
|
||||
)
|
||||
except Exception as e:
|
||||
return {
|
||||
"error": "Formatting error",
|
||||
|
||||
@@ -254,18 +254,28 @@ def _mark_liveness_failure(trace_id: str, status_data: dict[str, Any]) -> None:
|
||||
@mcp.tool()
|
||||
def mobile_manage_task(
|
||||
action: str,
|
||||
trace_id: str,
|
||||
trace_id: str = "",
|
||||
instruction: str | None = None,
|
||||
release_loop: bool = False,
|
||||
request_id: str | None = None,
|
||||
review_status: str | None = None,
|
||||
review_reason: str = "",
|
||||
user_confirmed: bool = False,
|
||||
) -> dict[str, Any]:
|
||||
"""Manages the lifecycle and retrieves the status of a background mobile automation task.
|
||||
|
||||
This is your primary diagnostic and control tool for tasks started by
|
||||
`mobile_run_task` or the web workspace. You MUST use it to poll the task status when your
|
||||
1-minute fallback timer triggers (as required by `mobile_run_task`). You
|
||||
can also steer a subagent that is stuck or off-track, or abort a task.
|
||||
`mobile_run_task` or the web workspace. Poll with reasonable intervals while
|
||||
following a running task; use request diagnostics for unexplained waits.
|
||||
You can also steer a runner that is stuck or off-track, or stop a task.
|
||||
|
||||
### Actions
|
||||
- lookup: Resolve a submission by request_id without starting or resubmitting it.
|
||||
- deletion_preview / delete: Show affected session IDs, then delete only when
|
||||
the user requested it (user_confirmed=true). Active tasks cannot be deleted.
|
||||
An exploration deletes its conversation runs, preserving independent reruns.
|
||||
- review: Record ONLY the user's explicitly stated passed/failed decision with
|
||||
review_reason and user_confirmed=true. Never label the agent's verdict as human.
|
||||
- **'status'**: Returns `trace_id`, `status` ('running'/'pausing'/'task_paused'/
|
||||
'resuming'/'completed'/'failed'/'cancelled'/'interrupted'), `device_serial` (which phone owns this task in multi-device
|
||||
setups), `task_desc`, `model`, `elapsed_seconds`, a `test_summary` (when
|
||||
@@ -311,6 +321,31 @@ def mobile_manage_task(
|
||||
that authorizes the subagent to complete continuous monitoring
|
||||
loops and finish the task gracefully.
|
||||
"""
|
||||
from mcp_server.tools.automation import _id, _request
|
||||
import asyncio
|
||||
|
||||
if action == "lookup":
|
||||
from mcp_server.task_requests import lookup
|
||||
|
||||
return lookup(_id(request_id))
|
||||
_id(trace_id)
|
||||
if action in {"deletion_preview", "delete", "review"}:
|
||||
if action != "deletion_preview" and not user_confirmed:
|
||||
raise ValueError("此动作需要用户明确要求；AI 自行判断不能记作人工复验")
|
||||
if action == "deletion_preview":
|
||||
return asyncio.run(_request("GET", f"/api/sessions/{trace_id}/deletion-preview"))
|
||||
if action == "delete":
|
||||
return asyncio.run(_request("POST", f"/api/sessions/{trace_id}/delete"))
|
||||
if review_status not in {"passed", "failed"}:
|
||||
raise ValueError("review_status 必须为 passed 或 failed")
|
||||
result = asyncio.run(
|
||||
_request(
|
||||
"PUT",
|
||||
f"/api/automation/reports/{trace_id}/review",
|
||||
json={"status": review_status, "reason": review_reason},
|
||||
)
|
||||
)
|
||||
return {"trace_id": trace_id, "test_result": result.get("test_result")}
|
||||
status_data = trace_store.read_status(trace_id) or _web_session_status(trace_id)
|
||||
if not status_data:
|
||||
return {
|
||||
@@ -438,6 +473,15 @@ def mobile_manage_task(
|
||||
test_result = resolve_case_result(None, current_status, response.get("error") or "")
|
||||
response["test_result"] = test_result.model_dump() if test_result else None
|
||||
|
||||
from apps.admin_console.services.case_review import reviewed_result
|
||||
from apps.admin_console.services.automation_store import AutomationStore
|
||||
from pathlib import Path
|
||||
|
||||
review_store = AutomationStore(Path(db_path))
|
||||
response["test_result"] = reviewed_result(
|
||||
trace_id, response.get("test_result"), review_store
|
||||
)
|
||||
|
||||
if current_status in ("running", "pending", "task_paused", "resuming") and is_alive:
|
||||
progress: dict[str, Any] = {}
|
||||
|
||||
@@ -472,12 +516,13 @@ def mobile_manage_task(
|
||||
conn2.row_factory = sqlite3.Row
|
||||
cur2 = conn2.cursor()
|
||||
cur2.execute(
|
||||
"SELECT count(*) as cnt FROM traces WHERE session_id = ? AND type = 'llm_call'",
|
||||
"SELECT coalesce(max(step_number),0) as cnt FROM steps WHERE session_id = ?",
|
||||
(session_id,),
|
||||
)
|
||||
row_cnt = cur2.fetchone()
|
||||
if row_cnt:
|
||||
progress["current_turn"] = row_cnt["cnt"]
|
||||
progress["current_step"] = row_cnt["cnt"]
|
||||
|
||||
cur2.execute(
|
||||
"SELECT payload FROM traces WHERE session_id = ? AND type in ('raw_thinking', 'llm_call') "
|
||||
|
||||
@@ -267,14 +267,15 @@ def mobile_run_task(
|
||||
verification_level: str | None = None,
|
||||
explorer_mode: str | None = None,
|
||||
runtime_overrides: dict[str, Any] | None = None,
|
||||
request_id: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Starts an autonomous mobile UI automation subagent on a connected Android device.
|
||||
"""Submit a Flash/Pro Android, computer or mixed task to the user-started ARTEMIS service.
|
||||
|
||||
Delegates a mobile workflow to a background agent. Non-blocking: returns
|
||||
immediately with `trace_id` (for `mobile_manage_task` / `mobile_inspect_trace`),
|
||||
`device_serial`, `stdout_log`/`stderr_log` paths, and `notes_dir` (Pro only).
|
||||
On completion (success or failure) a Reactive Wakeup notifies you — but only
|
||||
when `conversation_id` was provided; without it, rely on your fallback timer.
|
||||
Completion notifications depend on the caller's configured notification backend;
|
||||
conversation_id alone does not guarantee the agent client receives a wakeup.
|
||||
|
||||
### Model selection (routing)
|
||||
- **Flash** (default, preferred): simple, deterministic tasks with a clear
|
||||
@@ -291,23 +292,25 @@ def mobile_run_task(
|
||||
checkpoints, multi-step planning, or detailed written output (via notes
|
||||
and `expected_output_desc`).
|
||||
|
||||
Timing is not precise: the agent's own inference adds ~5 s per step (Flash)
|
||||
or ~30 s per turn (Pro) on top of any requested waits — account for this
|
||||
when the task involves waiting. For recurring workflows, run once to
|
||||
discover the path, then author a deterministic script instead of
|
||||
re-delegating.
|
||||
For interactive case design use mobile_exploration: user guidance, demonstration
|
||||
analysis and reusable case generation. It has independent model settings.
|
||||
Inference time varies by model, network and device; do not subtract a fixed
|
||||
per-step estimate from required waits. Deterministic test code needs explicit
|
||||
timing and state-based waits verified against the real application.
|
||||
|
||||
### Follow-up protocol (CRITICAL)
|
||||
1. **Fallback timer**: you MUST set a background timer and check the task
|
||||
status via `mobile_manage_task` at least once every 1 minute — mobile
|
||||
tasks can stall silently, so never rely on the completion wakeup alone.
|
||||
2. **Post-task inspection**: after completion, inspect the execution logs
|
||||
to verify success (`stderr_log` carries critical failures and Python
|
||||
tracebacks — essential for debugging).
|
||||
3. **Pro notes**: for Pro runs, read `notes_dir` to extract the results and
|
||||
plans the subagent recorded.
|
||||
### Follow-up
|
||||
1. Poll mobile_manage_task at reasonable intervals (usually 30–60 seconds)
|
||||
while following a task; do not assume an unconfigured background notification.
|
||||
2. Read test_result and view_report after completion. Inspect relevant request,
|
||||
log or step evidence for failures and unexplained waits.
|
||||
3. Read Pro notes when its plan or detailed output is relevant to the user.
|
||||
|
||||
Reuse request_id for retries of the same submission; lookup with
|
||||
mobile_manage_task(action="lookup", request_id=...). Never repeat a task
|
||||
with a fresh ID merely because its admission reply was lost.
|
||||
|
||||
Args:
|
||||
request_id: Stable caller-generated ID (letters, numbers, _ or -, up to 100).
|
||||
task_desc: Clear, self-contained task description with all context,
|
||||
exact UI actions, and termination conditions (execution is one-shot).
|
||||
For Pro, may also specify what the subagent must record in its notes.
|
||||
@@ -351,6 +354,39 @@ def mobile_run_task(
|
||||
fps, h264_kbps, h265_kbps; it does not enable recording on its own.
|
||||
Unknown fields and credentials are rejected.
|
||||
"""
|
||||
parameters = dict(
|
||||
task_desc=task_desc,
|
||||
conversation_id=conversation_id,
|
||||
model=model,
|
||||
locked_app_package=locked_app_package,
|
||||
app_path=app_path,
|
||||
expected_output_desc=expected_output_desc,
|
||||
device_serial=device_serial,
|
||||
verification_level=verification_level,
|
||||
explorer_mode=explorer_mode,
|
||||
runtime_overrides=runtime_overrides,
|
||||
)
|
||||
if request_id is not None:
|
||||
from mcp_server.task_requests import run_once
|
||||
|
||||
return run_once(request_id, parameters, lambda sid: _run_task(trace_id=sid, **parameters))
|
||||
return _run_task(trace_id=str(uuid.uuid4()), **parameters)
|
||||
|
||||
|
||||
def _run_task(
|
||||
*,
|
||||
trace_id: str,
|
||||
task_desc: str,
|
||||
conversation_id: str | None,
|
||||
model: str,
|
||||
locked_app_package: str | None,
|
||||
app_path: str | None,
|
||||
expected_output_desc: str | None,
|
||||
device_serial: str | None,
|
||||
verification_level: str | None,
|
||||
explorer_mode: str | None,
|
||||
runtime_overrides: dict[str, Any] | None,
|
||||
) -> dict[str, Any]:
|
||||
if runtime_overrides:
|
||||
from apps.admin_console.services.task_import import TaskOptions
|
||||
|
||||
@@ -370,9 +406,6 @@ def mobile_run_task(
|
||||
# plain tool error rather than a failed trace on disk.
|
||||
verification_level, explorer_mode = _normalize_pro_tuning(verification_level, explorer_mode)
|
||||
|
||||
# 1. Generate a unique trace_id
|
||||
trace_id = str(uuid.uuid4())
|
||||
|
||||
# 2. Initialize the trace store directory and status.json
|
||||
trace_store.init_trace(
|
||||
trace_id=trace_id,
|
||||
|
||||
Reference in new issue
Block a user