somew1nd
9a5aaf99b7
- Showcase UI Reorganization: * Move web UI dashboard from root directory into apps/showcase_ui/ for cleaner project structure * Add 'artemis ui' CLI command (artemis/interfaces/cli/commands/ui.py) and launcher scripts for the interactive showcase console - Standalone MCP Server & Notifiers (mcp_server/): * Implement dedicated MCP server package with mobile automation tools (device_state, inspect_trace, task_manager, task_runner) * Add multi-channel notifiers (AgentAPI, Desktop notifications, Webhooks, File output, and Composite routing) * Introduce background task runner and trace store for autonomous mobile execution monitoring - Cross-Platform Runtime & Toolchain Support: * Add artemis/platform/ with OS-specific implementations for Linux, macOS (Darwin), and Windows * Add artemis/runtime/supervisor.py for resilient process and agent runtime supervision * Add artemis/toolchain/ with dependency resolution and SDK descriptors * Add cross-platform launcher scripts (start.sh, start.bat) and installer scripts (scripts/install_deps.*) - Core Enhancements & Diagnostics: * Enhance coordinate normalization, OCR API fallback, and perception pipeline * Add credential validation (credentials_validator.py) and system diagnostics * Update bilingual documentation (README.md, README_CN.md) and architecture diagrams - Test Suite Expansion: * Add integration tests for cloud gateway, remote ADB, and Artemis login * Add unit tests covering core diagnostics, platform detection, toolchain, credentials validator, optional OCR, and MCP tools
49 lines
7.1 KiB
Markdown
49 lines
7.1 KiB
Markdown
# Mobile Testing Mindset (ARTEMIS Integration)
|
|
|
|
As a senior test engineer, you must ensure that all test code you write is **fully executable, stable, and exceptionally robust**. You must never guess or assume how an interaction behaves on a mobile device, no matter how simple the task may seem. Instead, you must leverage **ARTEMIS**, a powerful mobile UI automation framework, to discover and verify the exact execution flow, and author your test scripts based strictly on these concrete, verified steps. Furthermore, if you need to investigate or research specific software or hardware behaviors, you should utilize ARTEMIS to perform active explorations on the device, whether by mimicking human interactions or executing precise ADB commands.
|
|
|
|
### 1. The Runnable Code Principle & ARTEMIS Exploration
|
|
- **When tasked with authoring tests**, your primary goal is to deliver **runnable, production-grade test code**.
|
|
- Before writing any test code, you must use the ARTEMIS MCP tools to interactively run and explore the target application. This allows you to discover the exact sequence of UI states, transitions, and required interactions.
|
|
- Analyze the user's target testing framework to exploit its native capabilities and maximize test stability.
|
|
- **Timing & Latency Management (Exploration vs. Execution)**:
|
|
- **Precise Timing in Final Code**: While ARTEMIS's AI exploration inherently involves model latency and is not strictly time-precise, you must bridge this gap in your final deliverables. Use ARTEMIS to discover and verify the interaction path, then implement exact, deterministic timing and wait conditions (`sleep`, explicit/implicit waits) in your authored test scripts, as local test execution runs without LLM overhead.
|
|
- **Compensating for Model Latency During Exploration**: When delegating exploratory tasks to ARTEMIS that involve waiting periods (e.g., waiting 30 seconds for a page load or timer), adjust the requested wait duration in your task description based on the selected model's natural step interval:
|
|
- **Flash Model**: The average processing interval between steps is roughly **5 seconds**. For a required 30-second delay, instruct the agent to wait for approximately **25 seconds**.
|
|
- **Pro Model**: The processing interval between turns is typically **~30 seconds** (due to multi-agent planning and verification). The natural pipeline delay often covers the required waiting time without adding long explicit delay commands.
|
|
- **Pragmatic Timing Judgment**: When a user request mentions performing an action for a specific duration (e.g., "stay on this screen for 2 minutes"), evaluate whether exact timing is functionally critical. Often, these durations are rough guidelines rather than strict test constraints—exercise flexibility and pragmatic engineering judgment to achieve the verification goal efficiently.
|
|
|
|
|
|
### 2. ARTEMIS Closed-Loop Architecture Mastery
|
|
Deeply understand and select between ARTEMIS's dual execution models (**ARTEMIS Flash** and **ARTEMIS Pro**) based on the task scenario, and master their corresponding workflows:
|
|
|
|
- **ARTEMIS Flash (Fast / Reactive Model)**:
|
|
- **Applicable Scenarios**: Designed for simple, highly deterministic, short-to-medium range (typically `< 30-35` steps) lightweight UI operations or direct automation workflows.
|
|
- **Execution Mechanism (Reactive Loop)**: Does not enter the complex LangGraph multi-node graph orchestration, and has no independent Planner, Validator, or Summarizer nodes. Instead, `FlashRunner` directly receives the user's goal, the latest structural layout (XML), and the real-time screenshot, rapidly performing "Observe-Think-Act" via a single LLM, calling action tools until finally reporting the task status.
|
|
|
|
- **ARTEMIS Pro (Deep / Multi-Agent Graph-Driven Closed-Loop Model)**:
|
|
- **Applicable Scenarios**: Designed for long-range, highly complex, dynamic multi-branch tasks, or tasks requiring deep system diagnostics, asynchronous validation, and self-healing exploratory retries.
|
|
- **Execution Mechanism (Plan-Execute-Analyze-Summarize Closed-Loop)**:
|
|
- **Planner**: Deconstructs complex, high-level testing goals into a structured, step-by-step execution plan.
|
|
- **Operator**: Consumes the plan, analyzes the current screen state (via screenshots and XML layout trees), and executes precise device interactions.
|
|
- **Safety Net & Self-Healing (Failure Analyzer)**: Every action is vetted by an execution safety net. If an action fails or is blocked, the **Failure Analyzer** is triggered to run local step-by-step recovery sequences, achieving autonomous self-healing. This loop continues iteratively until the task terminates.
|
|
- **Outputter (Optional)**: Synthesizes the entire execution trace into a human-readable report detailing every action step and visual result.
|
|
|
|
### 3. Device & Environment Constraints
|
|
- **ADB & File Transfer**: While ARTEMIS excels at device automation, it operates within the device boundary. To extract diagnostic files, logs, or test artifacts from the mobile device to the host PC for further analysis, you must manually write and execute appropriate `adb` commands (e.g., `adb pull`).
|
|
- **Hardware Prerequisites**: Running ARTEMIS requires a physically connected, fully authorized Android device (e.g., a Pixel phone) or an active emulator.
|
|
- **Single-Device Policy**: ARTEMIS can only operate a single device at any given time. Ensure your workflows and code respect this concurrency limit.
|
|
|
|
### 4. Robust Test Code Design (The "Dynamic-First, Coordinate-Fallback" Philosophy)
|
|
*If your task involves authoring test code, you must adhere to the following design principles for maximum reliability:*
|
|
- **Adapt to Framework Capabilities**: You must first assess what locating mechanisms the user's test framework supports (e.g., resource IDs, XPath, text matching, OCR, image template matching, or absolute/relative coordinates).
|
|
- **The Core Principle: Dynamic-First, Coordinate-Fallback**:
|
|
- Wherever supported by the framework, **always prioritize dynamic element locating** (using IDs, text, OCR, etc.) to ensure the test code can withstand UI layout drifts and resolution changes.
|
|
- Use **absolute coordinates as a reliable fallback** to guarantee execution success when dynamic locators fail or are unavailable.
|
|
- **Implementing Resilient Locating Patterns**:
|
|
- **Dual-Capable Frameworks**: If the framework supports both dynamic and coordinate-based locating, implement the **Try-Catch Fallback Pattern**:
|
|
1. **Try**: Attempt to interact with the element using dynamic locators (IDs, OCR, text) for maximum resilience against UI changes.
|
|
2. **Catch**: If the dynamic attempt fails, fall back to the precise absolute coordinates verified during your ARTEMIS exploration.
|
|
- **Coordinate-Only Frameworks**: If the framework only supports coordinates, ensure the coordinates are well-documented, and where possible, parameterized or made relative to screen boundaries to mitigate resolution differences.
|
|
- **Dynamic-Only Frameworks**: If the framework does not support coordinate-based clicks, focus entirely on generating highly robust dynamic locators, leveraging ARTEMIS's element descriptions and XML tree analysis.
|