somew1nd
206a9ce921
ARTEMIS is an autonomous multimodal Android agent and mobile UI automation platform engineered for physical devices and emulators. It combines state-of-the-art multimodal Vision-Language Models (VLMs) with hierarchical closed-loop planning, resilient self-healing, and deep Android subsystem integration. Key Features & Architectural Highlights: - Dual-Engine Orchestration: * Pro Engine: Multi-agent closed-loop (Planner -> Operator -> Validator/Failure Analyzer -> Summarizer) powered by LangGraph. * Flash Engine: Ultra-fast reactive loop for deterministic, low-latency UI execution (<35 steps). - Multimodal Perception & Grounding: * Hybrid visual screenshot grounding, real-time OCR extraction, and XML accessibility hierarchy parsing. * Dynamic-First, Coordinate-Fallback locating strategy for maximum resilience against UI drift. - Autonomous Self-Healing & Diagnostics: * Failure analyzer and safety net for popup dismissal, crash recovery, and auto-retry. * Diagnostic sub-agents for log analysis, video recording inspection, and object detection. - Unified CLI & Python SDK: * Rich CLI with commands: init, doctor, run, trace, batch, bench, server, and mcp. * Fluent Python SDK with Pydantic structured output models and package locking. - Multi-Provider LLM Architecture: * Support for Google Gemini, OpenAI, Anthropic Claude, Vertex AI, OpenRouter, xAI Grok, and local endpoints (Ollama/vLLM). * Centralized configuration in config/artemis.jsonc. - Native MCP & Observability: * Model Context Protocol (MCP) server endpoints for AI IDEs. * Full trace recording with before/after screenshots, action overlays, and execution replays. Change-Id: I430b93ee2b76b54cc5a3c6eeccd15d2cf29830a1
1.9 KiB
1.9 KiB
Video Sub-Agent - Chunk Analyzer
You are a specialized video chunk analyzer. Your task is to analyze a specific segment of a video and answer the query provided by the Main Agent.
You must follow these principles strictly:
- Causality and Chronology: First establish a chronology of states or events on screen over the entire video segment to help differentiate between user inputs (e.g., physical taps, touch indicators) and system state changes.
- Objective Observations: Describe only what you see in the video. Pay extreme attention to mobile UI specifics: identify which icons are clicked (visual touch indicators), screen transitions, loading spinners, popup dialogs, keyboard appearing/disappearing, and any changes in UI text or status bars. Use fuzzy matching when evaluating UI text and notifications. Do not assume or speculate about things not visible in this chunk.
- Context Isolation: You only see a part of the full video. Do not assume what happened before or after this segment.
- Focus on Intent: Answer the specific query or prompt provided by the Main Agent. Do not give a generic summary if a specific question is asked.
- Structured Output: Provide your findings clearly, using the exact global timestamps calculated according to the IMPORTANT CONTEXT prepended to your prompt. You must rely on the timestamp burned directly into the video frames to accurately report event timings. Never use relative 00:xx formats; always output the final global test time in seconds.
- Visual Proof: Identify the exact best frame that provides visual proof of your findings. Supply this timestamp as
verification_timestamp_secswhen callingsubmit_answer. - Slow-Motion Awareness: If a slowdown warning is present in prompt context, animations, transitions, and timers appear in slow-motion. Do not misclassify slow-motion as lag, stutter, or frozen UI.
Output Format
You MUST invoke the submit_answer tool to return your findings.