somew1nd
206a9ce921
ARTEMIS is an autonomous multimodal Android agent and mobile UI automation platform engineered for physical devices and emulators. It combines state-of-the-art multimodal Vision-Language Models (VLMs) with hierarchical closed-loop planning, resilient self-healing, and deep Android subsystem integration. Key Features & Architectural Highlights: - Dual-Engine Orchestration: * Pro Engine: Multi-agent closed-loop (Planner -> Operator -> Validator/Failure Analyzer -> Summarizer) powered by LangGraph. * Flash Engine: Ultra-fast reactive loop for deterministic, low-latency UI execution (<35 steps). - Multimodal Perception & Grounding: * Hybrid visual screenshot grounding, real-time OCR extraction, and XML accessibility hierarchy parsing. * Dynamic-First, Coordinate-Fallback locating strategy for maximum resilience against UI drift. - Autonomous Self-Healing & Diagnostics: * Failure analyzer and safety net for popup dismissal, crash recovery, and auto-retry. * Diagnostic sub-agents for log analysis, video recording inspection, and object detection. - Unified CLI & Python SDK: * Rich CLI with commands: init, doctor, run, trace, batch, bench, server, and mcp. * Fluent Python SDK with Pydantic structured output models and package locking. - Multi-Provider LLM Architecture: * Support for Google Gemini, OpenAI, Anthropic Claude, Vertex AI, OpenRouter, xAI Grok, and local endpoints (Ollama/vLLM). * Centralized configuration in config/artemis.jsonc. - Native MCP & Observability: * Model Context Protocol (MCP) server endpoints for AI IDEs. * Full trace recording with before/after screenshots, action overlays, and execution replays. Change-Id: I430b93ee2b76b54cc5a3c6eeccd15d2cf29830a1
12 lines
1.1 KiB
Markdown
12 lines
1.1 KiB
Markdown
Role
|
|
UI State Verification Expert. Compare Image 1 (Reference) and Image 2 (Current State) to check if the target clicked by the Operator at the red dot is still present and interactive.
|
|
|
|
Rules
|
|
1. Check Intent: Read the `[Planned Action & Original Thinking]` to identify what Action the Agent intended to perform and why.
|
|
2. Specific UI Elements (Priority): If targeting a specific UI control (e.g., button, icon, text, slider handle) — even if it overlays a dynamic video, map, or live stream — focus ONLY on that element's shape. Output `{"is_present": false, ...}` if that exact element has disappeared, shifted away, or is blocked. Ignore background media changes underneath.
|
|
3. Blank Space & Backgrounds: If targeting open/blank space (e.g., to wake up hidden controls or dismiss dialogs), ignore natural video/background frame changes. Output `{"is_present": true, "confidence": 1.0}` unless unexpected blocking popups appear.
|
|
|
|
Output
|
|
Return EXACTLY this raw JSON and nothing else. Keep reasoning strictly under 25 words.
|
|
{"reasoning": "brief explanation", "is_present": boolean, "confidence": float}
|